2017-06-22: Autonomy Incubator Intern Kastan Day Takes Up 3DEEGAN Mantle

Kastan Day had his first internship with the Ai last year, when he made up the video production half of the social media team. Now, he's back from his freshman year at Swarthmore and making a glorious return— not as a videographer, but as a computer scientist.

"Yeah, this is pretty different," he said, pecking at the command line on his computer. With only a year of college under his belt, he's hitting the ground running as an intern. "Progress in the real world is way different than progress at school," he added.

Kastan's work this summer follows the work that Ai member Loc Tran and intern Deegan Atha did last summer, a computer vision and deep learning effort playfully named 3DEEGAN. Here's a helpful video Kastan made on the subject last summer:

While the Ai's computer vision work so far has focused on letting UAVs identify objects in real time— recognizing a tree in the field of view and changing course to avoid it, for example— Kastan is taking a more targeted approach. He's developing a system of unique markers that the computer vision algorithm can recognize instantly, sort of like a barcode at the grocery store. The cash register doesn't have to visually recognize your Twix bar based on its size, shape, and features; it just scans the barcode and matches the pattern up with the one that represents "Twix bar" in its library. By applying unique markers of a known size to objects in the UAV's field of vision, things like identification, calculating distance, and determining pose become much, much easier.

Kastan holds the webcam up to a screen full of markers to test if the computer
vision algorithm recognizes them.

What the machine sees is in the window on the left. See the green outlines
around all the markers? The algorithm works!

"[The markers] are easier to recognize if the search space is smaller, so we only made sixty-four of them as opposed to two hundred or a thousand," Kastan said.

Now, to make sure his system is as efficient and as accurate as possible, he has to determine what kind of grid to use when he generates the markers.

"The three-by-threes give a lot of false positives, but the eight-by-eights are hard to identify quickly," he explained. "We're looking for a solution in between."

To do the time-consuming work of printing out and testing each kind of grid, from three-by-three to eight-by-eight, Kastan has enlisted the help of the Ai's three high school volunteers. This summer's class of volunteers includes Ian Fenn from York High, Dylan Miller from Smithfield High, and Xuan Nguyen from Kecoughtan High. Their job entails printing out a test sheet of markers and then moving it around in front of the camera to see how it behaves.

"We're seeing how many times they're identified and how many times we get false things," Ian said. "We're also seeing if the larger patterns are more easily identifiable than the small ones."

Ian, left, and Dylan, right, move a sheet of five-by-five grid markers farther away
from the camera rig to see at what distance the algorithm stops identifying them.

Xuan also has has an additional relationship to the Ai beyond her scientific contributions.

"I'm more into engineering, but I also like computer science," she said. "My uncle Loc said every engineer needs to learn how to code."

Xuan takes notes about each grid's performance at different distances.

While the high school interns work, Kastan's next steps include "setting up marker detection in a simulated Gazebo environment inside a ROS framework." Basically, he's simulating how the algorithm will behave in the real world.