UAVs are becoming more and more present in our everyday life and there are lots of different projects that are being currently developed in order to control their flight, handle their stability, make it possible to edit automatic missions that the drones will execute and anything that the developers can think of. On October 2014, the Linux Foundation announced the creation of the DroneCode Project which is to become “a common, shared open source platform for Unmanned Aerial Vehicles (UAVs)”. Parrot started to sell Linux based drones in 2010 and obviously needed to take part in that adventure. This Lightning Talk will try to give a quick overview of the projects that are developed by the Dronecode community and explain why and how I started a few months ago to port an open source autopilot name Ardupilot to Parrot’s drones. This Lightning Talk will also present the current status of this project, and the many possibilities that can come from it. Julien BERAUD

1. The DroneCode Project - A Step in Open
Source Drones
Julien BERAUD
October 5, 2015
The DroneCode Project - A Step in Open Source Drones 1 / 21

2. The DroneCode Project
”An open source, collaborative project that
brings together existing and future open
source drone projects under a nonprofit
structure governed by The Linux Foundation.
The result will be a common, shared open
source platform for Unmanned Aerial
Vehicles (UAVs).”
The DroneCode Project - A Step in Open Source Drones 2 / 21

3. DroneCode Software
The DroneCode Project - A Step in Open Source Drones 3 / 21

4. Parrot Drones 1/3
▶ Parrot AR Drone
▶ Parrot 6 SoC - ARM926EJS
▶ Linux Kernel
▶ Busybox
The DroneCode Project - A Step in Open Source Drones 4 / 21

5. Parrot Drones 2/3
▶ Parrot AR Drone 2.0
▶ TI OMAP3630 - ARM Cortex-A8
▶ Linux Kernel
▶ Busybox
The DroneCode Project - A Step in Open Source Drones 5 / 21

6. Parrot Drones 3/3
▶ Parrot Bebop
▶ Parrot 7 SoC - Dual Cortex-A9
▶ Linux Kernel
▶ Busybox
The DroneCode Project - A Step in Open Source Drones 6 / 21

7. Drone Software
▶ Drone Software
▶ Drone OnBoard Software
▶ Computer/Smartphone/Tablet Software
▶ Cloud Software
▶ Protocols to bind them together
▶ Parrot Drones OnBoard Software
▶ Custom Proprietary Flight Stack
▶ Running entirely on Linux
▶ No PREEMPT_RT Patch
▶ Quite different from other existing drones
The DroneCode Project - A Step in Open Source Drones 7 / 21

8. Drone Software
▶ Drone Software
▶ Drone OnBoard Software
▶ Computer/Smartphone/Tablet Software
▶ Cloud Software
▶ Protocols to bind them together
▶ Parrot Drones OnBoard Software
▶ Custom Proprietary Flight Stack
▶ Running entirely on Linux
▶ No PREEMPT_RT Patch
▶ Quite different from other existing drones
The DroneCode Project - A Step in Open Source Drones 7 / 21

9. DroneCode OnBoard Software
▶ OnBoard Software
▶ The Flight Stack
▶ Specific to Drones
▶ 2 Flight Stacks are part of the DroneCode Project
The DroneCode Project - A Step in Open Source Drones 8 / 21

10. DroneCode OnBoard Software
▶ OnBoard Software
▶ The Flight Stack
▶ Specific to Drones
▶ 2 Flight Stacks are part of the DroneCode Project
The DroneCode Project - A Step in Open Source Drones 8 / 21

11. Dronecode Flight Stacks
▶ APM or Ardupilot - http://dev.ardupilot.com/
▶ GPLv3 License
▶ First developed to run on an Arduino
▶ PX4 - https://pixhawk.org/dev/start
▶ BSD License
▶ First developed to run on an STM32 microcontroller
▶ Originally developed to run on top of NuttX RTOS
The DroneCode Project - A Step in Open Source Drones 9 / 21

12. Dronecode Flight Stacks
▶ APM or Ardupilot - http://dev.ardupilot.com/
▶ GPLv3 License
▶ First developed to run on an Arduino
▶ PX4 - https://pixhawk.org/dev/start
▶ BSD License
▶ First developed to run on an STM32 microcontroller
▶ Originally developed to run on top of NuttX RTOS
The DroneCode Project - A Step in Open Source Drones 9 / 21

13. OpenSource Linux drones appearing
▶ January 2015 - Andrew Tridgell at LCA
▶ Article on LWN - https://lwn.net/Articles/630018/
▶ Drone running Linux with the APM flight stack
▶ BeagleBone Black with Pixhawk Fire Cape
▶ Several Linux boards supported natively in APM
▶ BeagleBone with Pixhawk Fire Cape
▶ Erle Brain
▶ NavIO
▶ Qualcomm starting to work on PX4 port on Linux
The DroneCode Project - A Step in Open Source Drones 10 / 21

14. Drone Architecture
▶ QuadCopter
▶ Impossible to fly entirely manually - Autopilot
▶ Sensors, user commands
▶ Motor speeds
▶ Ardupilot
▶ Reads from sensors
▶ Enhanced Kalmann Filter(EKF)
▶ Estimate the state of the Drone
▶ Sensors
▶ Inertial Measurement Unit (IMU)
▶ Barometer
▶ Compass
▶ GPS, Optical Flow Sensor, Sonar, ...
▶ External microcontroller for motor control
The DroneCode Project - A Step in Open Source Drones 11 / 21

15. Drone Architecture
▶ QuadCopter
▶ Impossible to fly entirely manually - Autopilot
▶ Sensors, user commands
▶ Motor speeds
▶ Ardupilot
▶ Reads from sensors
▶ Enhanced Kalmann Filter(EKF)
▶ Estimate the state of the Drone
▶ Sensors
▶ Inertial Measurement Unit (IMU)
▶ Barometer
▶ Compass
▶ GPS, Optical Flow Sensor, Sonar, ...
▶ External microcontroller for motor control
The DroneCode Project - A Step in Open Source Drones 11 / 21

16. Drone Architecture
▶ QuadCopter
▶ Impossible to fly entirely manually - Autopilot
▶ Sensors, user commands
▶ Motor speeds
▶ Ardupilot
▶ Reads from sensors
▶ Enhanced Kalmann Filter(EKF)
▶ Estimate the state of the Drone
▶ Sensors
▶ Inertial Measurement Unit (IMU)
▶ Barometer
▶ Compass
▶ GPS, Optical Flow Sensor, Sonar, ...
▶ External microcontroller for motor control
The DroneCode Project - A Step in Open Source Drones 11 / 21

17. Drone Architecture
▶ QuadCopter
▶ Impossible to fly entirely manually - Autopilot
▶ Sensors, user commands
▶ Motor speeds
▶ Ardupilot
▶ Reads from sensors
▶ Enhanced Kalmann Filter(EKF)
▶ Estimate the state of the Drone
▶ Sensors
▶ Inertial Measurement Unit (IMU)
▶ Barometer
▶ Compass
▶ GPS, Optical Flow Sensor, Sonar, ...
▶ External microcontroller for motor control
The DroneCode Project - A Step in Open Source Drones 11 / 21

18. Parrot Bebop Architecture
The DroneCode Project - A Step in Open Source Drones 12 / 21

19. APM - https://github.com/diydrones/ardupilot
AP_HAL class
class AP_HAL::HAL {
...
AP_HAL::UARTDriver* uartA;
AP_HAL::UARTDriver* uartB;
AP_HAL::UARTDriver* uartC;
AP_HAL::UARTDriver* uartD;
AP_HAL::UARTDriver* uartE;
AP_HAL::I2CDriver* i2c;
AP_HAL::I2CDriver* i2c1;
AP_HAL::I2CDriver* i2c2;
AP_HAL::SPIDeviceManager* spi;
...
};
Userland drivers
hal.i2c1->write(BEBOP_BLDC_I2C_ADDR , 1, &data);
The DroneCode Project - A Step in Open Source Drones 13 / 21

20. Drivers modifications
▶ Baro - MS5607
▶ Existing driver for MS5611
▶ Scaling factor
▶ Compass - AK8963
▶ Existing driver for AL8963
▶ Only as a slave device for an IMU
▶ Implement the use of a direct connection on i2c
▶ IMU
▶ MPU6050 in i2c instead of spi
▶ No PREEMPT_RT patch
▶ Jitter issues
▶ Implement the use of the MPU6050 FIFO
▶ Motor Controller
▶ New driver to write
▶ Added in AP_HAL_Linux
▶ Implements the RCOutput class
▶ GPS almost supported
▶ Add frames / Fixes
The DroneCode Project - A Step in Open Source Drones 14 / 21

21. Drivers modifications
▶ Baro - MS5607
▶ Existing driver for MS5611
▶ Scaling factor
▶ Compass - AK8963
▶ Existing driver for AL8963
▶ Only as a slave device for an IMU
▶ Implement the use of a direct connection on i2c
▶ IMU
▶ MPU6050 in i2c instead of spi
▶ No PREEMPT_RT patch
▶ Jitter issues
▶ Implement the use of the MPU6050 FIFO
▶ Motor Controller
▶ New driver to write
▶ Added in AP_HAL_Linux
▶ Implements the RCOutput class
▶ GPS almost supported
▶ Add frames / Fixes
The DroneCode Project - A Step in Open Source Drones 14 / 21

22. Drivers modifications
▶ Baro - MS5607
▶ Existing driver for MS5611
▶ Scaling factor
▶ Compass - AK8963
▶ Existing driver for AL8963
▶ Only as a slave device for an IMU
▶ Implement the use of a direct connection on i2c
▶ IMU
▶ MPU6050 in i2c instead of spi
▶ No PREEMPT_RT patch
▶ Jitter issues
▶ Implement the use of the MPU6050 FIFO
▶ Motor Controller
▶ New driver to write
▶ Added in AP_HAL_Linux
▶ Implements the RCOutput class
▶ GPS almost supported
▶ Add frames / Fixes
The DroneCode Project - A Step in Open Source Drones 14 / 21

23. Drivers modifications
▶ Baro - MS5607
▶ Existing driver for MS5611
▶ Scaling factor
▶ Compass - AK8963
▶ Existing driver for AL8963
▶ Only as a slave device for an IMU
▶ Implement the use of a direct connection on i2c
▶ IMU
▶ MPU6050 in i2c instead of spi
▶ No PREEMPT_RT patch
▶ Jitter issues
▶ Implement the use of the MPU6050 FIFO
▶ Motor Controller
▶ New driver to write
▶ Added in AP_HAL_Linux
▶ Implements the RCOutput class
▶ GPS almost supported
▶ Add frames / Fixes
The DroneCode Project - A Step in Open Source Drones 14 / 21

24. Drivers modifications
▶ Baro - MS5607
▶ Existing driver for MS5611
▶ Scaling factor
▶ Compass - AK8963
▶ Existing driver for AL8963
▶ Only as a slave device for an IMU
▶ Implement the use of a direct connection on i2c
▶ IMU
▶ MPU6050 in i2c instead of spi
▶ No PREEMPT_RT patch
▶ Jitter issues
▶ Implement the use of the MPU6050 FIFO
▶ Motor Controller
▶ New driver to write
▶ Added in AP_HAL_Linux
▶ Implements the RCOutput class
▶ GPS almost supported
▶ Add frames / Fixes
The DroneCode Project - A Step in Open Source Drones 14 / 21

25. Motor Controller
RCOutput_Bebop
class Linux::LinuxRCOutput_Bebop : public AP_HAL::RCOutput {
public:
LinuxRCOutput_Bebop();
void init(void* dummy);
void set_freq(uint32_t chmask, uint16_t freq_hz);
uint16_t get_freq(uint8_t ch);
void enable_ch(uint8_t ch);
void disable_ch(uint8_t ch);
void write(uint8_t ch, uint16_t period_us);
void write(uint8_t ch, uint16_t* period_us, uint8_t len);
uint16_t read(uint8_t ch);
void read(uint16_t* period_us, uint8_t len);
void set_esc_scaling(uint16_t min_pwm, uint16_t max_pwm);
int read_obs_data(BebopBLDC_ObsData &data);
...
The DroneCode Project - A Step in Open Source Drones 15 / 21

26. Motor Controller 2/2
set_ref_speed
void LinuxRCOutput_Bebop::_set_ref_speed(
uint16_t rpm[BEBOP_BLDC_MOTORS_NUM])
{
struct bldc_ref_speed_data data;
int i;
data.cmd = BEBOP_BLDC_SETREFSPEED;
for (i=0; i<BEBOP_BLDC_MOTORS_NUM; i++)
data.rpm[i] = htobe16(rpm[i]);
data.enable_security = 0;
data.checksum = _checksum((uint8_t *) &data, sizeof(data) - 1);
if (!_i2c_sem->take(0))
return;
hal.i2c1->write(BEBOP_BLDC_I2C_ADDR ,
sizeof(data),
(uint8_t *)&data);
_i2c_sem->give();
}
The DroneCode Project - A Step in Open Source Drones 16 / 21

27. RCInput over Wi-Fi
Protocol
#define RCINPUT_UDP_NUM_CHANNELS 8
#define RCINPUT_UDP_VERSION 2
struct __attribute__((packed)) rc_udp_packet {
uint32_t version;
uint64_t timestamp_us;
uint16_t sequence;
uint16_t pwms[RCINPUT_UDP_NUM_CHANNELS];
};
▶ Basic linux utility written
▶ joystick_remote
https://github.com/jberaud/joystick_remote
▶ Using linux joystick input driver
▶ Already ported on Parrot SkyController
The DroneCode Project - A Step in Open Source Drones 17 / 21

28. RCInput over Wi-Fi
Protocol
#define RCINPUT_UDP_NUM_CHANNELS 8
#define RCINPUT_UDP_VERSION 2
struct __attribute__((packed)) rc_udp_packet {
uint32_t version;
uint64_t timestamp_us;
uint16_t sequence;
uint16_t pwms[RCINPUT_UDP_NUM_CHANNELS];
};
▶ Basic linux utility written
▶ joystick_remote
https://github.com/jberaud/joystick_remote
▶ Using linux joystick input driver
▶ Already ported on Parrot SkyController
The DroneCode Project - A Step in Open Source Drones 17 / 21

29. Trying to fly
▶ First flight = crash
▶ Logging system used to understand the reason of the crash.
▶ Log files sent to one of the maintainers
▶ Log Analysis
▶ All sensor datas
▶ All interesting values
▶ Possibility to find the cause of a problem
▶ Solution found
▶ The order of the motors wasn’t right
The DroneCode Project - A Step in Open Source Drones 18 / 21

30. Trying to fly
▶ First flight = crash
▶ Logging system used to understand the reason of the crash.
▶ Log files sent to one of the maintainers
▶ Log Analysis
▶ All sensor datas
▶ All interesting values
▶ Possibility to find the cause of a problem
▶ Solution found
▶ The order of the motors wasn’t right
The DroneCode Project - A Step in Open Source Drones 18 / 21

31. Trying to fly
▶ First flight = crash
▶ Logging system used to understand the reason of the crash.
▶ Log files sent to one of the maintainers
▶ Log Analysis
▶ All sensor datas
▶ All interesting values
▶ Possibility to find the cause of a problem
▶ Solution found
▶ The order of the motors wasn’t right
The DroneCode Project - A Step in Open Source Drones 18 / 21

32. Trying to fly
▶ First flight = crash
▶ Logging system used to understand the reason of the crash.
▶ Log files sent to one of the maintainers
▶ Log Analysis
▶ All sensor datas
▶ All interesting values
▶ Possibility to find the cause of a problem
▶ Solution found
▶ The order of the motors wasn’t right
The DroneCode Project - A Step in Open Source Drones 18 / 21

33. Log Analysis
▶ Motors order
▶ Cause of the crash
The DroneCode Project - A Step in Open Source Drones 19 / 21

34. Submitting patches to master
▶ Discussions with Tridge
▶ A bit of reworking
▶ Patches got accepted
▶ 1 month of work
▶ 17 patches submitted for the basic support
▶ Coming up next
▶ Wiki page for users to hack their bebop
▶ Currently missing sensor support
▶ Integration as an alternative to our flight stack
▶ Benefits
▶ Performance comparison
▶ Access to the dronecode ecosystem
▶ New piloting modes
The DroneCode Project - A Step in Open Source Drones 20 / 21

35. Submitting patches to master
▶ Discussions with Tridge
▶ A bit of reworking
▶ Patches got accepted
▶ 1 month of work
▶ 17 patches submitted for the basic support
▶ Coming up next
▶ Wiki page for users to hack their bebop
▶ Currently missing sensor support
▶ Integration as an alternative to our flight stack
▶ Benefits
▶ Performance comparison
▶ Access to the dronecode ecosystem
▶ New piloting modes
The DroneCode Project - A Step in Open Source Drones 20 / 21

36. Submitting patches to master
▶ Discussions with Tridge
▶ A bit of reworking
▶ Patches got accepted
▶ 1 month of work
▶ 17 patches submitted for the basic support
▶ Coming up next
▶ Wiki page for users to hack their bebop
▶ Currently missing sensor support
▶ Integration as an alternative to our flight stack
▶ Benefits
▶ Performance comparison
▶ Access to the dronecode ecosystem
▶ New piloting modes
The DroneCode Project - A Step in Open Source Drones 20 / 21

37. Submitting patches to master
▶ Discussions with Tridge
▶ A bit of reworking
▶ Patches got accepted
▶ 1 month of work
▶ 17 patches submitted for the basic support
▶ Coming up next
▶ Wiki page for users to hack their bebop
▶ Currently missing sensor support
▶ Integration as an alternative to our flight stack
▶ Benefits
▶ Performance comparison
▶ Access to the dronecode ecosystem
▶ New piloting modes
The DroneCode Project - A Step in Open Source Drones 20 / 21

38. Conclusion
▶ Useful links
▶ http://dev.ardupilot.com/
▶ https://github.com/diydrones
▶ https://github.com/jberaud
▶ https://pixhawk.org
▶ Parrot is recruiting
▶ Linux userland and kernel developers
▶ Android Java developers
▶ iOS developers
▶ Questions ?
The DroneCode Project - A Step in Open Source Drones 21 / 21

39. Conclusion
▶ Useful links
▶ http://dev.ardupilot.com/
▶ https://github.com/diydrones
▶ https://github.com/jberaud
▶ https://pixhawk.org
▶ Parrot is recruiting
▶ Linux userland and kernel developers
▶ Android Java developers
▶ iOS developers
▶ Questions ?
The DroneCode Project - A Step in Open Source Drones 21 / 21

40. Conclusion
▶ Useful links
▶ http://dev.ardupilot.com/
▶ https://github.com/diydrones
▶ https://github.com/jberaud
▶ https://pixhawk.org
▶ Parrot is recruiting
▶ Linux userland and kernel developers
▶ Android Java developers
▶ iOS developers
▶ Questions ?
The DroneCode Project - A Step in Open Source Drones 21 / 21