Columbia High students create light-to-sound device for colorblind fourth grader

Since October, participants Anders Daly, Heron Urban, and Miles Walsh have been working on a device to assist local fourth grader Kepler Gilchrist, who is colorblind. Using a sensor to read light from an object, the device converts information into a range of tones — mapping three primary colors to different instruments — so he can perceive color through sound.

While Gilchrist can see color, he often has trouble differentiating them. For example, when he looks at a rainbow, it appears different every time; sometimes it’s blue and yellow, sometimes it’s just blue or just yellow, and sometimes it’s blue, yellow, and green. He also has trouble distinguishing between pink and gray, blue and purple, and red and green.

While other products exist that can express color, whether verbally or textually, they’re only useful to a point, Daly explained. “We wanted to develop something that can give him a more accurate perception of not just color, but how you understand color.”

The device will allow Gilchrist to experience a richer, more nuanced sense of color — almost like artificial synesthesia. “We’re trying to make it so he can experience color on a deeper level than just hearing a word…a broader sense of color,” Heron said.

For Walsh, writing code has been the most challenging piece of the puzzle. Integration, assembling ideas into a single, cohesive piece, has posed an additional hurdle.

But when those pieces have aligned, it’s been all the more rewarding. Since they had never worked with a Bluetooth-capable device before, getting it connected was a big milestone, particularly because it lacks a user interface to choose what to pair with. “You have to hardwire it into the actual device instead, which is janky, but it was nice to make it work,” Walsh said.

“It’s the little things,” Daly added. “Every step is a small victory.”

The final product will resemble a stylus: Gilchrist can point it at something, press a button, and the corresponding instrument will play. Though still to be determined, he hopes to hear a fiddle for green, a piano for blue, and a trumpet for red.

Later this month, the trio will attend “Demo Day” in the Bay Area, where they’ll present the product via a slide deck to a panel of venture capitalists, who will provide feedback and announce whether they’ll invest in it “Shark Tank-style.”

While the trio doesn’t expect the product to be fully complete by competition time, they hope the pitch will earn them some money to take the project further. 

Gilchrist, meanwhile, will patiently await the finished product and is excited to read — or hear — colors for the first time. “I don’t want to have to ask someone every time I go to pick up a crayon, colored pencil, or marker,” he said.