2016-02-09: NASA Administrator Charles Bolden tours the Autonomy Incubator

After delivering his State Of NASA address at LaRC, NASA Administrator Charlie Bolden visited the Autonomy Incubator for a tour and conversation about the state of autonomous systems and NASA missions.While with us in Building 1222, the Administrator heard about sensing, perception, localization, human-machine teaming, motion planning, and collaborative autonomy.

Anna Trujillo spoke with Administrator Bolden about Human Systems Interaction (HSI) protocols used to communicate with drones via semantics and visual cues. During the tour, she demonstrated how a scientist could define a complex flight path using simple gestures and natural language.

NASA Administrator Charlie Bolden (L) speaks with Anna Trujillo (R)

Dr. Loc Tran and Administrator Bolden discussed object detection, classification, and tracking. Dr. Tran's work currently focuses on distinguishing different kinds of objects for, among many reasons, to determine if an object is a hazard to be avoided.

NASA Administrator Charlie Bolden (L) meets with Dr. Loc Tran (R)
while Jim Neilan demonstrates visual odometry on the Mars Flyer

Standing with Dr. Danette Allen, Administrator Bolden observed multiple Unmanned Aerial Vehicles (UAV) in flight as they performed a cooperative mission under development in support of NASA's earth science objectives.

NASA Administrator Charlie Bolden (L) talks with Dr. Danette Allen (R)
while LaRC Center Director Dave Bowles looks on

2016-01-28: Congressional Visit

The Autonomy Incubator was a stop on the center tour for US Representatives Scott Rigell (R-Va) and Rob Wittman (R-Va) where they saw GL-10 and were treated to an autonomous mission demonstration featuring object classification, localization, detect-and-avoid, and multi-agent collaborative trajectories.

US Rep. Rigell (left) speaking with Jim Neilan in the Autonomy Incubator

US Rep. Rigell (second from right) speaking with Bill Fredericks
about GL-10 in the Autonomy Incubator

To read more about their day at LaRC, check out the article on the LaRC website.

2016-01-20: NASA HQ Associate Administrators Visit the Autonomy Incubator.

NASA Associate Administrator Robert Lightfoot and Deputy Associate Administrator Lesa Roe toured the Autonomy Incubator (AI) on January 20, 2016.

Dr. Danette Allen (Left) with Deputy Associate Administrator Lesa Roe (Center)
and Associate Administrator Robert Lightfoot (Right) in LaRC's Autonomy Incubator

Their visit to the AI included demonstrations of cooperative autonomy and a showcase of capabilities such as Localization, Object Detection and Classification, and Object Avoidance on the Mars Reusable Electric Flyer.

Jim Neilan points out sensors on our Mars Reusable Electric Flyer
while Dr. Loc Tran and Zak Johns look on.

A notional Mars Sample Return mission scenario was presented in which three Unmanned Aerial Vehicles (UAVs) survey and identify and Area of Interest (AOI) for possible sample collection. A rover is summoned to the AOI and delivers a fourth UAV as well as its own paylaod. The UAVs fly collaboratively (in formation) while the rover completes its own related mission.

Rover "Herbie" exhibits "Follow Me" behavior behind a UAV
to deliver the fourth intelligent agent to complete the mission

Rover "Herbie" has delivered the fourth UAV and exits the scene

   

2016-01-19: NASA LaRC Autonomy Incubator 2016 Spring Interns Arrive

The Autonomy Incubator welcomed our spring interns on January 19, 2016. Pictured below from left to right are:

• Erica L. Meszaros of Eastern Michigan University
• Leonardo Herrera of Cicese Research Center
• Kevin French of the University of Florida
• Michael Janov of Northeastern University, and 
• Spencer Watza of Western Michigan University.

 

Stay tuned for future posts and to learn more about the great work to come from the Intern Team.

2015-12-11: Goodbye to our fall interns Deegan Atha and Rebecca Goodpasture

Today marks the last day for Deegan Atha and Rebecca Goodpasture as interns at the Autonomy Incubator. In this post, we ask them to reflect on their experience this fall.

DEEGAN

What did you do this semester?

     For my internship I was a computer vision intern in the Autonomy Incubator. My main project was to develop an object detection and classification solution to be utilized onboard a UAV. This entailed researching the state of the art in the field, developing an application based on this research, and then implementing it onboard a vehicle to test. The biggest accomplishment over the course of the internship was being able to run a full flight test of the detection and classification with a UAV. Over the course of the internship I learned many things. The biggest thing I learned was about the state of the art in convolutional neural networks as well as object segmentation approaches with point clouds. Secondly, I learned about the Agile methodology and how to work in a fast pace environment. Third I learned about the challenges still facing UAV’s and what research needs to be completed in the future to create autonomous vehicles. One of the best aspects about this internship was the freedom to develop my own solution for the project and own what I was working on. This allowed me to fully engage in the project and learn a great deal. Furthermore, it has inspired me to pursue graduate school in the field upon completion of my undergraduate degree. As a result of my project, I have been able to present the work twice at the Autonomy Incubator showcase to the Langley community. In addition, my mentor will present my work at the Aviation 2016 conference and we hope to submit a paper to a computer vision or robotics conference. In addition to a great project I have been able to learn about the NASA community as a whole and the many endeavors here at Langley as well as participate in outreach events to further enhance my experience. 

As the internship ends, what are your next steps?

     I will be returning to Purdue for the Spring semester where I will be in my Junior year. I will be rejoining the CAM2 (Continuous Analysis of Many CAMeras) research team that I am a member as well as rejoining an EPICS (Engineering Projects in Community Service) team that works on accessible classroom technologies for the Indiana School for the Blind and Visually Impaired. 

REBECCA

What did you do this semester?

     Coming into the internship, my mentor, Dr. Danette Allen, instructed me to update the blog, tweet, and manage video and photo equipment. From this framework, I developed the following objectives: produce summaries of the Autonomy Incubator’s actives that were interesting and accessible for the average American, increase exposure for the Autonomy Incubator by establishing a robust social media presence across platforms, and to maximize audience interaction with human interest, visual content and trending topics. 

     To accomplish these objectives, I created and worked on a variety of projects. I began a photo series called #MeetOurTeam, consisting of human interest photographs overlaid with brief text, which introduced social media audiences to the unique individuals behind the Autonomy Incubator team. I created an Instagram for the team, developed a Facebook page which is pending review, and worked with an intern in the CIO’s office to develop a NASA Langley webpage for the Autonomy Incubator team. For public relations outreach, I worked with the graphics department to develop a slide publicizing our monthly showcase, I created Autonomy Incubator lapel pins for the team, mini fliers with our social media information for guests and interested parties, and I transitioned the social media content to conform with NASA Style Guide standards. 

     Other milestones of my internship include the lobby video I developed that plays for guests of our building and is also shown at expos and during demonstrations at NASA. The content I created for social media posts received record setting impressions and engagements with our audience, and dramatically increased our followers in the tech field (overall audience increase of over 35%). The largest project I worked on was the Mars Flyer Video, which involved working with a variety of groups on site.

     I learned a great deal during my internship, most notably within the realms of project management, social media marketing, and photo and video editing software. It was a privilege to work with the amazing people at NASA, and the opportunities to participate in outreach events and to organize tours for my peers were invaluable insights into life at Langley Research Center.

As the internship ends, what are your next steps?

     As of tomorrow, I will be traveling around Europe for a month collecting videos, photos and interviews for my travel website lueurtravel.com. When I return home, I will continue to take online and open source courses in engineering, coding and science, as I am considering pursuing a second degree in aerospace engineering. 

2015-11-04: Autonomy Incubator Celebrates One Year in B1222

Wednesday, November 4th, marked the one year anniversary of our team’s move to B1222, also known as the Old Reid Center. 

 

Our first week in B1222

Since our move, the Autonomy Incubator has hosted over 35 student interns, given more than 50 tours (to center directors, the NASA Deputy Administrator, OMB, OSTP - just to name a few!), and brought in close to 20 outside speakers to share their work with the Center. The facility itself measure over 70,000 cubic feet, and the forest, mars landscape and residential zone all help our researchers accurately develop autonomous capabilities. 

A young visitor navigates a UAV using Meghan Chandarana's
developments with gesture-based controls.

The large auditorium space enables us to test and demonstrate our algorithms. A fan favorite, Dr. Loc Tran's tree-dodging demo highlights our work with machine learning and with obstacle detection and avoidance.

In the past year, the Autonomy Incubator has worked on a variety of projects. We host student interns who lead their own research, such as Bilal Mehdi and Javier Puig-Navarro's work on coordinated trajectories for small and micro UAS, and we also continuously adapt to reflect research improvements and industry needs. You can read more about our most recent projects here.

A UAV autonomously navigates through our forest.

James Rosenthal and Jim Neilan set up our rover and "Green Machine" UAV for a demo.

An overhead view of our auditorium a year after our move.
From right to left: our residential area, forest, and the Mars landscape.

We are also pleased to announce that the Autonomy Incubator has received a project extension and expansion in name. As the need for autonomous capabilities becomes more apparent, NASA is transitioning our team name from “Autonomy Incubator” to “Langley Autonomy and Robotics Center” although we will never stop incubating new and innovative missions!

An increasing number of projects at NASA Langley Research Center require the development of autonomous capabilities to ensure mission success, and we're looking forward to working with a variety of different teams on site. As we work to maximize our interaction with other groups at Langley, our past month’s showcase became an experiment in internal communications. We received a record number of NASA visitors due to heightened publicity for the event, and we left the showcase with a stronger connection to our audience and with their work.

As we reflect on all that has been accomplished in the past year, we also look towards developing autonomous capabilities to meet the future challenges that NASA will encounter in missions in space, science, and aeronautics. To stay up-to-date with the projects we're working on, follow us on social media.

Twitter: @AutonomyIncub8r

Instagram: @AutonomyIncubator

2015-11-03: Autonomy Incubator Showcase

Thank you to everyone who attended our October Showcase at NASA Langley Research Center!

The Autonomy Incubator hosts monthly showcases for our NASA colleagues, during which we report out on how we are rising to the autonomy and robotics challenges in space, science, and aeronautics. These events are snapshots of selected technologies and capabilities that we’re developing as we advance towards our goal of precise and reliable mobility and manipulation in dynamic, unstructured, and data-deprived environments. Check out the time-lapse video of our showcase, below.

We began this month’s showcase by highlighting the work of our fall computer vision intern, Deegan Atha. Deegan, a rising junior at Purdue University, discussed the on-board object classification capabilities he is developing with the help of his mentor, AI engineer Jim Neilan. As Deegan showed our audience the camera he’s been using and how it can be easily added onto different vehicles, he discussed the processing time per image (about 3 seconds) and our work using Euclidean segmentation for detection of objects.

In the coming month, Deegan will be working on in-flight training data, building 3D on a vehicle to optimize accuracy, and ultimately demonstrating the object classification working on a flying vehicle. While we are not ready for him to leave he AI, we are looking forward to his exit presentation!

Deegan Atha, our computer vision intern, discusses his work in object classification.

In the above image, we can see the object classification working, as the
computer recognizes the chair and the NASA employee.

Jim Neilan highlights the different UAV designs in our lab.

Following Deegan, Jim Neilan briefly discussed our vehicles and the importance of our technology's platform portability. Although we might test on specific vehicles, like Herbie, our beloved rover, the autonomous behaviors we are developing are not designed to be vehicle-specific. 

The microphone was then handed to Anna Trujillo, our human factors engineer. At the Autonomy Incubator, we're working on creating autonomous systems that limit the need for constant piloting of a vehicle. However, while we're creating behaviors that can function independently of a pilot we still need a human to define, monitor, and adjust mission specifications. Anna discussed her work developing controls that can be easily understood by a person without piloting experience. She began with the example of a science mission, in which NASA scientists would be using a UAV to test ozone and CO2 levels. Using a display program Anna developed, scientists can use a tablet to get raw data values in real-time, without waiting for post-processing. Information such as vehicle position, time and sensor data can help the scientists ensure the test is running smoothly, and that the data they're getting makes sense. 

Display system created by Anna, designed to enable scientists to gather data in real time.

A package delivery display, in which the human operator can designate the size
of the package and locations of takeoff, drop off and landing.

Anna has also created a user display for package delivery. Using this system, a delivery person can easily input information about the size of the package and then choose separate locations of where the vehicle will takeoff from, drop off a package at, and land.  After the user designates this information, the underlying algorithms determine mission specifics, such as trajectory, without being directed by a pilot. To emphasize the general accessibility of the application, Anna asked a young boy in the audience to input values for the package delivery. He began with inputting  a package code, the first of which was pre-programmed. for a routine delivery. For example, the NASA Langley mail office might deliver the same type of package to our center director at the same time everyday. To show what would happen in the event of a non-routine package delivery, our volunteer created a new code, and then quickly and easily filled in the different data sets by hand. 

Ben Kelley with Herbie, our beloved rover. 

Continuing our discussion of human/vehicle interaction, Ben Kelley, our software architect, showcased his "Follow Me" behavior. In the "Follow Me" demo, our "Green Machine" UAV follows a rover. The behavior occurs when the UAV receives messages of the rover's position and alters its current waypoint (destination) to reflect the location one meter above the rover.

The "Green Machine" in flight, exhibiting the "Follow Me" behavior.

The UAV performs the "tree-dodging" developed by Dr. Loc Tran.

Dr. Loc Tran works on a variety of machine-learning projects at the Autonomy Incubator, and he presented his work with obstacle detection and avoidance. This "tree-dodging" behavior is an important area of research to highlight, as maneuverability and obstacle avoidance is integral to safe UAV flight. "Tree-dodging" begins when Loc gives the UAV the command to take off, with the directive that it is to navigate itself through the trees. Without any intervention from Loc or a pilot, the UAV uses a front-facing camera to detect and avoid certain features, such as branches from one of our artificial trees. If the UAV makes a mistake with trajectory planning, Loc will later correct the vehicle and designate a new rule of response. Through repetition of this process, the vehicle becomes skilled enough to navigate the course perfectly, and ideally, will be able to do the same in a course it has never encountered. One day, "tree-dodging" may help deliver supplies to areas that are difficult to navigate through, such as rainforests, and might be ported to underwater or planetary exploration applications.

Our pilot, Zach Johns, with the Mars Flyer. 

The showcase ended with a demonstration of the Mars Flyer and the autonomous capabilities our team is designing for NASA missions to the red planet. Jim Neilan, our project lead, pointed to a small computer with visual odometry algorithms onboard the Mars Flyer and explained it's ability to reliably navigate the Mars Flyer vehicle in a GPS-denied environment such as Mars. Visual odometry begins with data from the downward-facing camera. Processed onboard and in real-time, calculations of the movement of different ground features enable the computer to accurately create a 3-D map of the terrain.

Zach flies the Mars Flyer, while Dave North (on the right), explains the vehicle design.

At the Autonomy Incubator, we are continuously adapting to reflect research improvements and new NASA projects that require the the development of different autonomous capabilities. To stay up-to-date with these projects, follow us on social media.

Twitter: @AutonomyIncub8r

Instagram: @AutonomyIncubator