Autonomous Drone Hunter

onboard detection and tracking for GPS-denied environments

Project Venom was a research project at Columbia’s Creative Machines Lab from fall 2016 to summer of 2018. Its mission is summarized below. In the first year, I joined the team as an undergraduate research assistant, and then was promoted to lead the project from fall 2017 until its conclusion. The final project team consisted of eight undergraduate and Masters level research assistants from computer science, mechanical engineering, and electrical engineering. By May 2018, we developed a flying hunting drone prototype that tracked target drones using Tiny-YOLO on a Jetson TX2, computed the relative target location from the resulted image coordinates, and sent the desired next pose via MAVROS to a PX4 based flight controller. Furthermore, we completed a proof of concept of an end-to-end AI-Pilot trained and tested in our custom simulator that sends stick-inputs to the flight controller to hunt and take down another drone. Since this work is unpublished, I cannot publicly share a more detailed report at this time.

As small, unmanned aerial vehicles (UAVs), also known as drones, move from the hobbyist market into main-stream, reports of misuse, reckless flying, and drone use for malicious purposes become more frequent. Additionally, the fast paced progress in AI raises fears of a dangerous combination of AI and UAVs. In order to effectively regulate drones and protect people against UAV threats, a system is needed that can stop or even apprehend a rogue drone. Such a system would have to be robust enough to capture both a drone piloted by a human or powered by an AI-pilot. Project Venom was developing a UAV platform that detects, hunts, and takes down other small UAVs in GPS-deprived environments, relying solely on on-board computation. The drone platform is comprised of a PX4 based flight controller, a RGBD stereo camera and an onboard computing module that uses pre-trained machine learning models to classify other drones within its sensor range as well as navigate and take down the drone once authority has been granted by a human operator. In parallel we were developing an AI-Pilot that could autonomously hunt another drone; we believe that an AI-Piloted drone—once it surpasses the skill level of a human pilot—can only be taken down by another AI-Pilot.

Publication: Wyder PM, Chen YS, Lasrado AJ, Pelles RJ, Kwiatkowski R, et al. (2019) Autonomous drone hunter operating by deep learning and all-onboard computations in GPS-denied environments. PLOS ONE 14(11): e0225092. https://doi.org/10.1371/journal.pone.0225092