Home / Navigator
Dossier 11 / Unmanned crew
An FPV racer needs a pilot in the goggles for every roll. An autonomous craft needs a briefing, a map, and a lost-link plan. The sticks may stay on the bench. The aircraft still has to fly, see, and decide what "safe" means when the radio goes quiet.
Autonomy is not a switch. It is a crew that takes jobs away from the human, one at a time. First the mixer holds attitude. Then the autopilot holds a point or a line. Then a companion computer looks at a camera or a lidar and says “tree, go around.” Only the last step looks like a film: the aircraft leaves, does the work, and comes home without a hand on the gimbal.
The wartime parallel is a bomber with a navigator and a bomb-aimer, not a fighter on the stick. The flight controller is still the instrument bay. The extra brain – often a Raspberry Pi, an NVIDIA Jetson, or a phone-class board – is the navigator’s table. The firmware plate names the stacks that sit on the FC. This plate is about the mission on top.
If the aircraft cannot fail safe without you, it is not autonomous. It is remote control with extra modes. Write the lost-link behaviour before you write the waypoint list.
Delivery, docks that swap packs, and search patterns are the same idea with more logistics. The airframe still has to know where it is and what it is allowed to hit.
Ladder
| Rung | What the aircraft does alone | Who still flies |
|---|---|---|
| Stabilize | Holds attitude. That is every modern FC. | The pilot. FPV and camera ships live here. |
| Hold and return | GPS position, altitude, a button that says “home.” | The pilot chooses when. INAV and camera apps sit here. |
| Waypoint mission | A list of coordinates, camera actions, and a landing. | A planner on the ground. ArduPilot and PX4 missions. |
| Sense and avoid | Stereo, lidar, or a depth net sees a wall and steers around it. | The route is still yours. The gap-filling is the machine’s. |
| Perceive and act | A detector finds a target, a tracker keeps it, a planner writes setpoints. | A supervisor. This is where YOLO, SLAM, and ROS 2 enter. |
Example projects
The everyday autonomous drone is still a waypoint file. Mission Planner or QGroundControl draws the route. The FC flies AUTO, LOITER, RTL. Geofence and failsafe are first-class. No neural net is required. Most mapping wings and inspection copters in the field still live on this rung, and they should: it is boring in the useful way.
An open ROS 2 framework (MIT, 2025-26) that talks to both PX4 and ArduPilot, simulates faster than real time, and deploys the same code onto NVIDIA Jetson with JetPack and DeepStream. Perception in the box: YOLO on the camera, KISS-ICP on 3D lidar. Multi-vehicle worlds, Docker, and a Gym-style step so you can train or test without rebuilding the hangar each time.
If you want one public tree that means “autonomy” the way Betaflight means “racer,” start the briefing here.
A long-range quad or a foam wing that must come home does not need ROS. INAV’s GPS hold, cruise, and return-to-home are autonomy at the hold-and-return rung: enough for a scout, not enough for a building you have never mapped. Many students meet “the aircraft flew itself” here first.
Indoor research fleets: a nano-quad, a positioning deck, and a Python API. Crazyswarm and the Bitcraze stack are how universities fly ten aircraft in a room and publish the log. Autonomy here is often a motion-capture hall plus a script, which is honest: the room is the navigator until you put SLAM on the craft.
You do not debug a planner over a car park. PX4 and ArduPilot software-in-the-loop sit in Gazebo (now Harmonic in current recipes). Microsoft AirSim, still used as a research gym, feeds cameras and physics to learning agents. Aerial autonomy stack and several PX4-ROS2-SLAM repos ship Docker so the world, the FC, and the detector start with one command.
Closed products do the same rungs with less source. Skydio made its name on onboard obstacle maps. Camera drones fly waypoint grids from a phone. Dock systems (the Dock 4 story on the home board) take off, swap a pack, and go again. Study them as doctrine, not as something you can fork.
AI hangar
A mission file is geometry. AI enters when the aircraft must name what it sees, or build a map when GPS will not. The flight controller still flies the setpoints. The model writes the advice.
YOLO (the Ultralytics line: v8, v11, and the nano weights people actually put on a Jetson) is the default detector on research quads. Aerial autonomy stack exports it to ONNX and runs it on the GPU. Bounding boxes become ROS 2 messages; a simple visual-servo loop turns “person in frame” into velocity. You can train on your own pictures or start from a general weight and fine-tune.
Sister data: the VisDrone set is the usual public pile of aerial boxes if you need a drone-looking dataset rather than street photos.
Visual-inertial odometry and SLAM replace the satellite. Names you will meet: ORB-SLAM3, RTAB-Map, OpenVINS, VINS-Fusion, and lidar odometry such as KISS-ICP. Ready recipes wire RTAB-Map to PX4 under ROS 2 and Gazebo so a simulated X500 can build a 3D scan. GPS-denied return-home is this stack plus a memory of the way in, not a magic model.
NVIDIA Isaac ROS Visual SLAM is the vendor-flavoured cousin when the companion is a Jetson and you want a supported package rather than a paper repo.
Stereo cameras and small lidars make a cost map. Academic planners (EGO-Planner, Fast-Planner, and the UZH RPG agile-autonomy line) turn that map into a trajectory the FC can track in offboard or guided mode. Monocular depth networks can fill a hole when you have only one lens; they lie more often than a lidar, so treat them as a hint.
PX4’s own avoidance and path-planning modules are the in-tree version of “do not hit the wall” without a custom paper stack.
Reinforcement learning shows up as hover and altitude toys (DQN on PX4 SITL) and as research into agile flight in AirSim or custom gyms. It is not how you inspect a bridge tomorrow. Large vision-language models can write a waypoint list from a sentence (“photograph the north face”); they do not yet replace the geofence or the failsafe. Keep them above the autopilot, never in the motor loop.
If a repo promises “fully autonomous, no GPS, detects targets, returns home” in one breath, read it as a teaching stack. Confirm sensors, logs, and a real lost-link test before you believe the poster.
Companion computer sees. It publishes a pose, a box, or a velocity. ROS 2 (or MAVLink, or XRCE-DDS) carries that to the FC. The FC still owns the mixer, the ESCs, and the failsafe. That split is the same as a wartime crew: the navigator does not hold the yoke when the engine coughs.
On the bench you will meet MAVSDK, MAVROS, and PX4’s uXRCE-DDS bridge. They are the wireless telegram between the two brains. Learn one of them after you can fly a waypoint mission by hand in the planner.
The browser trainer will not teach this. It is a stick school. Autonomy starts when you close the goggles and open a map.
