Low-AoI data collection in integrated UAV-UGV-assisted IoT systems based on deep reinforcement learning

In unmanned aerial vehicle (UAV)-assisted Internet of Things (IoT) applications, sensor nodes are often extensively deployed for environmental data sensing in remote areas. Ensuring efficient data collection tasks for the UAV with sufficient energy is a crucial consideration in UAV-assisted large-scale IoT systems. To address this challenge, we introduce the unmanned ground vehicle (UGV) as a charging platform for the UAV to provide timely and reliable energy replenishment, referring to such a system as the integrated UAV-UGV-assisted IoT system. To implement this system, we propose a reinforcement learning-based integrated UAV-UGV-assisted IoT data collection scheme (LUDC). In this scheme, we first utilize the affinity propagation algorithm to achieve a reasonable clustering of sensor nodes. Building upon this, we introduce the Gaussian mixture model to divide the task regions for the UAV to achieve workload balancing. Finally, we introduce the multi-agent deep deterministic policy gradient algorithm to optimize the cooperative trajectories of the UAV and UGV, thereby enabling the UAV to take off and land on the UGV at appropriate times. Simulation experiments demonstrate that the LUDC scheme can ensure that the UAV does not face the risk of energy depletion during the data collection process while minimizing data transmission latency in the integrated system.