Aerial Inspection Behaviors via RL-based Quadrotor Control for Under-canopy Forest Environments

Technical Summary: Aerial Inspection Behaviors via RL-based Quadrotor Control for Under-canopy Forest Environments

Problem Statement
This paper addresses the challenge of deploying autonomous quadrotors for inspection missions within under-canopy forest environments. While quadrotors are well-suited for confined and cluttered spaces, conventional autonomy pipelines typically rely on a structured control stack (e.g., PID or geometric tracking) combined with motion planners. The authors investigate whether a learned, end-to-end motor-level controller can serve as a practical alternative execution layer within a planner-guided autonomy stack. The specific problem involves achieving simultaneous position and yaw reference tracking (view-pose tracking) to support various inspection behaviors, such as point-to-point navigation, circular scanning, and helical trajectories, while operating under state measurement noise and external disturbances.

Methodology
The proposed solution utilizes a hierarchical autonomy architecture comprising three distinct layers:

1. High-Level Navigation (TSP Planner): Given a known map of the forest and a set of user-defined inspection regions, a Traveling Salesman Problem (TSP) planner computes the optimal sequence for visiting these targets. The cost function is based on Euclidean distance, ignoring altitude and yaw for the initial sequence generation.
2. Mid-Level Path Planning (RRT Planner):* An informed 3D Rapidly-exploring Random Tree Star (RRT*) planner generates collision-free paths between the sequential targets defined by the TSP. This planner respects the specific tracking limitations and safety boundaries of the lower-level controller (e.g., ensuring waypoints are within a 3-meter radius to account for training termination conditions). It outputs a sequence of collision-free waypoints and heading profiles.
3. Low-Level Execution (RL Controller): The core contribution is a deep Reinforcement Learning (RL) policy that maps state observations directly to motor RPM commands.
   • Formulation: The problem is modeled as a Partial Observable Markov Decision Process (POMDP).
   • Observation Space: Includes noisy position error, quaternion error, linear velocity, angular velocity, and the previous action (to maintain Markov properties). Gaussian noise is injected during training to enhance robustness.
   • Action Space: The policy outputs normalized motor RPMs, which are converted to raw RPM values in the simulation.
   • Reward Shaping: A multi-component reward function encourages survival (avoiding crashes), precise position tracking (using dual-bandwidth exponential rewards for horizontal and vertical errors), velocity minimization, accurate geodesic angle tracking (yaw), and action smoothness (penalizing rapid changes in motor commands).
   • Architecture: The policy is a Multi-Layer Perceptron (MLP) with 17 inputs and 4 outputs, trained using the Stable-Baselines3 library within a Gym-PyBullet-Drones environment.

Key Contributions
The authors present three primary contributions:

1. End-to-End Control Policy: An RL-based low-level controller that maps state observations directly to motor RPMs for simultaneous position and yaw tracking, eliminating the need for explicit decomposition into outer-loop position and inner-loop attitude control.
2. System Integration: A demonstration that this learned controller can be reliably deployed as the execution module within a higher-level navigation framework (TSP + RRT*) for long-range under-canopy missions.
3. Validation: The complete system is validated across five distinct inspection scenarios, proving the viability of the RL controller in complex forest-like environments.

Results and Validation
The system was evaluated in a simulated forest environment containing 200 cylindrical tree trunks. The validation covered five specific scenarios:

1. Under-canopy Navigation: The quadrotor successfully navigated between eight inspection targets, with the TSP determining the order and RRT* generating the path. The RL controller tracked the resulting collision-free waypoints.
2. View-Pose Tracking: The drone inspected five trees at varying altitudes, maintaining a 1-meter safety distance and pointing its body x-axis toward each tree.
3. Stationary Scene Scanning: The quadrotor maintained a fixed position while rotating 360 degrees to scan a specific area.
4. Circular Trajectory Tracking: The drone tracked a circular path around a tree trunk at a constant altitude and 50 cm horizontal distance.
5. Helical Trajectory Tracking: The drone executed a helix trajectory around a tree, varying its yaw to maintain a view of the trunk center throughout the ascent/descent.

In all scenarios, the controller demonstrated smooth and accurate motion with small position and yaw tracking errors. The results indicate that the RL-based controller effectively handles the execution of complex inspection behaviors generated by the high-level planners.

Significance and Claims
The paper claims that a learned motor-level stabilizing controller, when supported by a navigation guidance layer, is a viable and effective low-level execution module for under-canopy forest inspection. The work suggests that end-to-end policies can reduce the need for hand-designed controller decomposition while preserving the safety guarantees provided by higher-level planners. The authors remain modest regarding immediate deployment, noting that the current work is a simulation-based validation. They state that future work will focus on sim-to-real transfer and real-hardware validation to assess the controller's applicability in practical, real-world aerial inspection missions.