2018-08-24: An Introduction to the HINGE Project

Bryan Barrows and Erica Meszaros- some of the stars of HINGE.

With the recent introduction to the ATTRACTOR project, there are also pieces that follow behind it that have been patiently waiting for their time to shine on the Ai blog.  HINGE is a project working to support the goals of ATTRACTOR, but before diving too deep, the waterfall of acronyms that it provides must be introduced first.

HINGE stands for Human Informed Natural-language GANs Evaluation.  As you can see, within that acronym is the acronym GAN, meaning Generative Adversarial Network, a machine learning algorithm that lets a computer generate data from an example data set, such as producing an image from a description.

So, what does all this really mean, and what exactly is the project?  Well, with ATTRACTOR and its aim to increase trust in and trustworthiness of autonomous systems, it is critical to examine how human operators interact with these systems of the human/machine interaction (HMI).  HINGE was kind of the first step under the HMI team, examining ways in which autonomous systems use search and rescue (SAR) missions to get useful information from human colleagues.  GANs are able to produce images of the missing person, which then can help the machine identify them; however, more descriptive data is needed to train the GAN and, more importantly, we need to ensure that we can get critical information in an emergency situation.  In summary, HINGE is a project designed to gather data of image descriptions in order to help us understand what descriptive information we need and how we can best provide it.  Of course, we also get to test out some fun machine learning algorithms.

Now, it's time to meet the HMI team!  Lisa Le Vie is the Principal Investigator and lead for the HINGE effort.  Essentially, she sees everything from a global point of view, and where its headed in the future.

Lisa giving a presentation of her work at the 2018 Aviation conference.

Bryan Barrows, a graduate from Virginia Tech, focuses on natural language processing (NLP) and data analysis.  He worked with Lisa and Mary Carolyn earlier this year to collect a data set from employees on center at the Langley cafeteria, which consists of over five hundred descriptions used to train the GAN.  Over the course of the summer, he has been closely working with Lisa and Erica Meszaros on understanding the collected descriptions.  Bryan has developed and employed several NLP methods to better deduce and understand the description data.  His analysis of the collected HINGE data set has led to the derivation of several key semantic features that are helpful for training machine learning models, as well as understanding the desired image representations produced from the GAN.

Bryan was once a NIFS intern, but he is now a civil servant at NASA.

Erica L. Meszaros, a returning Ai intern joined the HMI team at the beginning of the summer.  Because of her background in linguistics and modalities of interaction for human/autonomous system interfaces, she has been focusing primarily on applying linguistic analysis to the HINGE data.  "We’re approaching this analysis from a lot of really interesting angles informed by machine learning requirements, situational and contextual interaction, and modalities of communication, which makes it a really neat kind of interdisciplinary puzzle," she stated.

In addition to all of her HINGE work, she has also assisted me greatly with the extensive analyses we have written for the blog.  This one is included!

 Erica has spent a lot of time on the BirdGAN.

Mary Carolyn Last and Miranda Smith are no longer at the Ai,  but they are very important to mention.  Both of these young women were a big part in the project, and their work does not go unappreciated.

At this point in time, HINGE is actually wrapping up! HMI is still an important area of focus for understanding, informing, and improving trust in autonomous systems, though, and the team is "hoping to use the work from the HINGE project to move toward different research directions," as Erica stated.

A lot of the HINGE work also supports Jim Ecker and his research on environmental understanding using generative models, like GANs, which we will expand on later!  The team is in the process of doing a second data collection, so be sure to stay updated to hear what's next!

Special thanks to Erica Meszaros for writing assistance.