Visitors to Oswego’s historic Fort Ontario will get to see past and present merge, thanks to an augmented reality project by a SUNY Oswego graduate student.

Theodore Johnson, a student in the master’s in human-computer interaction program, completed the project thanks to a grant from the college’s new Impact Scholars program, robust archives and expert teamwork.

Working with Paul Lear, site manager for Fort Ontario State Historic Site, as well as fort staff and faculty mentor Damian Schofield, Johnson created an augmented reality tour on an app called Artivive. Visitors can pick up a flipbook when they come to the fort, and take it along on a self-guided tour. When they come to a building or location pictured, they point their smartphone at the matching photo and the app will play a slideshow of what the attraction looked like through the years. 

Through buttons on the app, visitors can learn more about the history of the viewed location and access the fort’s website for more information. “It’s so simple and effective. That’s why it’s so awesome,” Johnson said of the app.

“The whole point of this is to bring history alive,” explained Schofield, director of Oswego’s HCI program. “It’s on site so it’s real and virtual coming together. It’s like a video extension of the audio tour.”

The project integrated photos from the 1860s all the way to a recent wedding on the fort grounds, Lear said. The site has had countless uses dating back to the location’s first fortification by the British in 1755, and has been continuously rebuilt following destruction during the French and Indian War, American Revolution and War of 1812.

Translating that rich history to the viewer experience has involved Schofield and Lear exploring various augmented reality applications and options, but when Johnson came aboard with the grant, the graduate student suggested making the Artivive app the solution. 

For his “Hypermedia” HCI class, Johnson said, “I discovered Artivive by doing my first project at the Safe Haven Museum using augmented reality videos via the diorama display. We chose locations on the diorama display for Artivive to recognize and then made historical videos about the refugees with me doing a voice over during the videos.” 

The Safe Haven museum commemorates the only U.S. shelter of its kind for European refugees of the Holocaust during World War II, as nearly 1,000 people, most of them Jewish, were hosted on the fort grounds. 

“We have hundreds of photos in the archive, and with the way Theo produced it, visitors can carry the books all over the 75 to 80 acres we have,” Lear said. “We keep looking for new ways to bring history to life, and working with the college has been fruitful.”

Building success

Johnson also has his bachelor’s degree from Oswego in psychology with a minor in cognitive science. Because the college’s HCI degree combines those and other fields in teaching how to make technology function to improve humans’ lives, he found the program appealing.

“In my classes, I’ve learned how to go over the data process, create an application, and how to design, test a hypothesis and do user testing,” Johnson said. “It’s a close-knit program where we can bounce ideas off each other. The HCI program really sets you up for success. You get all the experience you need.”

He will work with the fort to have visitors use the app and book, which will allow for user experience testing as to how the solution performs, how easy users can access it and whether any additional tweaks can improve it. 

In addition, Johnson will pursue publishing his research and the results of the project. The process is widely applicable because many museums and exhibitions can benefit from augmented-reality tours, Johnson added.

The college’s new Impact Scholars program supports graduate students’ commitment to and capacity for sustained, informed and deliberate community engagement. Scholars submit a proposal that highlights their work in collaboration with community partners to engage and support a community organization, business, institution or not-for-profit. Students are mentored by faculty and utilize their skills and intellect to help solve a community problem or enhance community practices.

“With Theo's innovative project, you can see how funding graduate students to extend and apply their course projects in our communities can have long-term impact,” said Kristen Eichhorn, SUNY Oswego’s dean of graduate studies. “This is why we are so excited about the implementation of our Impact Scholars program.”

Schofield and Lear agreed that they would love to see HCI students and the fort work together on additional projects, including for the Safe Haven refugee shelter museum.