W&M’s Highland launches nation’s first wearable augmented reality tour
Historic first: Highland, the William & Mary-owned historic home of James Monroe, will be the first site in the country to introduce augmented reality tours using smart glasses. Photo by Jason Woodle
A team effort: The tour is the result of a collaboration between Highland, AR storytelling software developer ARtGlass and the Japanese technology firm Epson. Photo by Adrienne Berard
Bringing characters to life: The new technology will provide a platform to resurrect those who once lived at Highland, principally, the enslaved population, who made up the bulk of the farm’s residents. Photo courtesy James Monroe's Highland
The science of storytelling: ARtGlass Cofounder and CEO Greg Werkheiser '96 (left) discusses updates to Highland's AR tour with Chief Technology Officer Luigi Percuoco. Photo by Adrienne Berard
Months of research: Sara Bon-Harper, Highland's executive director, has spent the past six months working with a team of researchers to create the scenes depicted on the AR tour. Photo courtesy Epson
The guest house: On the tour, the first view of the presidential guest house is through the eyes of an enslaved cook named Hannah. She strikes up a conversation with an enslaved blacksmith named Nelson, talking about the two men who built the house, Peter and George. Photo courtesy James Monroe's Highland
Update: James Monroe’s Highland announced the augmented reality experience will be fully implemented into their tours beginning July 25, 2018. Complete information about these tours and their fees is available on the Highland website. - Ed.
Just off Route 795 in Albemarle County is a 535-acre farm that appears much like it did in 1817, when it served as the primary residence for William & Mary alumnus President James Monroe. Cattle graze on rolling hills. A succession of ash trees leads up to the property, where stone chimneys top clapboard houses that encircle an old yard.
While it may appear rustic, the site is in the midst of a technological revolution. Highland, the William & Mary-owned historic homestead of James Monroe, will be the first site in the country to introduce augmented reality tours using smart glasses. Beginning Feb. 5, visitors will be invited to participate in beta testing of the AR experience, which Highland intends to make a permanent offering.
“We're all about trying new things,” said Sara Bon-Harper, the site’s executive director. “This augmented reality experience is a way to invite people to investigate history in a new way.”
Topography serves as a form of biography in the tour, as the context for the time in which Monroe lived is layered over the land he once owned. The idea to introduce AR tours came from a collaboration between Highland, AR storytelling software developer ARtGlass, and the Japanese technology firm Epson. ARtGlass and Epson have already partnered to bring wearable AR experiences to 25 cultural sites throughout Europe, but Highland is their first site in North America.
{{youtube:medium|Nwe1uaV9tqs, Highland is the first U.S. historic site to launch a wearable augmented reality tour}} When ARtGlass announced they were looking for a U.S. pilot site, Highland made its case. Their reasoning was twofold, Bon-Harper explained. First, the site is small, which makes for a tight-knit team with a shorter chain of command. Highland just added its seventh full-time staff member.
“In general, I like to see opportunity in resource constraints,” Bon-Harper said. “We can afford to be nimble in the same way that a startup can. All that we have between us and being the first is the ability to move quickly and expand our bandwidth sufficiently to work really hard for several months and make this happen.”
The second argument for choosing Highland, Bon-Harper said, was its compelling story.
“The power of historic sites is tied to the power of the storytelling experience,” said Greg Werkheiser, ARtGlass cofounder and CEO, and a 1996 graduate of William & Mary. “It's not just about showing up and looking at objects and places. It's about being intellectually and emotionally moved by the experience.”
A new chapter
Adopting AR will fundamentally change interpretation at the site, Bon-Harper said. For example, it will enable guests to see the archaeological remnants of Monroe’s original house — despite the fact they are currently buried beneath a quiet boxwood garden.
The new technology also resurrects the stories of those who once lived at Highland, including the enslaved population who made up the bulk of the farm’s residents. Bon-Harper and her team spent six months researching the characters, using historical archives and contemporary accounts from the plantation. They were able to pull dialogue, dress and mannerisms from the historical record, which translates into narration taken directly from the pages of history.
In rendering the characters for AR, Bon-Harper made the decision to depict them in hand-drawn animation versus lifelike actors. She wanted to integrate research into the presentation, while still leaving room for interpretation. The rendering needed to be accurate, but Bon-Harper feared making it too literal would hinder imagination.
The solution was to contract a local artist named Tim Jones. He and a team of Highland researchers designed each character, including their dress and props, to track with what is in historical record.
“In the same way that literature invites us to imagine the way people look, using drawings brings you into this literary and historical imagination,” said Bon-Harper. “We want you to feel invited into that world.”
While hand-drawn sketches may seem old-fashioned, animating the renderings relied on an advanced technological infrastructure. ARtGlass’s proprietary software allows their programmers to layer any combination of three-dimensional holograms, still photographs, video, architectural models and characters within the binocular transparent screens of the glasses.
Whereas some historic sites have begun offering AR experiences using smart phones and tablets that display 2D images, ARtGlass’ software and Epson’s smart glasses combine to immerse the visitor in what Werkheiser calls a “heads-up and hands free” experience. ARtGlass’ software also solves another obstacle to widespread adoption of AR tours at cultural sites.
“Our tours don’t rely on WiFi, beacons, or QR codes that a visitor needs to scan, all of which frequently fail,” Werkheiser explained. Instead, the ARtGlass software instructs the camera built into the Epson smart glasses to “recognize” a digital map of places and things at a historic site. ARtGlass creates digital fingerprints for objects that, when processed by a complex algorithm, reliably trigger the start of each stop of the tour.
“We essentially modified facial recognition technology to apply to historical objects,” said Werkheiser.
Based on their experiences deploying at sites in Europe, Werkheiser believes the Highland AR experience will draw in more visitors overall, as well younger and more diverse guests that historic sites rely on to sustain their mission. While they may initially come for the new tech, he hopes they stay for the story.
“We see the technology as an amazing means to get to the cultural storytelling ends,” Werkheiser said. “Ultimately, the goal is to transform the way that human beings interact with, appreciate, learn from and leverage their history.”
Landscape helps tell the story
The tour begins at a field just beyond the welcome center. Once donning the glasses, dormant farmland transforms into a stage upon which Monroe’s life and legacy is projected. Superimposed on the landscape are maps, original documents, and portraits revealing Monroe’s role in the 1820 Missouri Compromise.
The legislation kept the balance of slave and free states in the U.S. and was credited with staving off civil war — while keeping millions of Americans in bondage. Next Monroe drafts the presidential doctrine that would later bear his name, the right of New World nations to be free from European colonial rule, a bold stance coming from a young country. 
The story of Highland is one that has evolved over time, as more is discovered about the site and the man who owned it. The fifth president of the United States, Monroe attended William & Mary before joining the Continental Army in 1776. He contributed a 50-year career in public service, serving as Secretary of State and Secretary of War, four terms as Virginia’s governor and two terms as president from 1817 to 1825.
While Monroe was a dedicated public servant, he was also a slaveholder, which is a necessary and complicating fact of the Founding Father’s legacy, Bon-Harper says. He owned as many as 250 enslaved persons in his lifetime — and never freed any of them. At the same time, Monroe strongly believed in and called for the abolition of slavery, calling it “one of the evils still remaining, incident to our Colonial system.”
It’s that contradiction that is vital to understanding Monroe and the plantation he called home, Bon-Harper says. It’s a contradiction she made sure was part of the AR experience.
“One of my really hard lines on interpretation at our site is that we interpret slavery in the same set of narratives that we interpret James Monroe,” she said. “And we interpret them in the same discussion. We can’t have two separate conversations, one about Monroe and one about slavery. In discussing Monroe, we talk about slavery. We talk about enslaved individuals and we talk about the larger context of slavery.” 
Monroe lived at Highland with his family, including wife Elizabeth Kortright Monroe, from 1799 to 1823. He sold the property at a time when the South’s economy was shifting away from tobacco and grain to cotton farming in the 1820s. While the family occupied the property, Monroe was one of the county’s largest slaveholders. A Virginia census showed 49 enslaved people living at Highland in 1810.
“Enslaved people were the bulk of the occupants at Highland,” said Bon-Harper. “The Monroes were only here intermittently, even when it was their main residence — and there were more enslaved people living here than there were Monroes during that period. Their story is pretty crucial to the particulars of the property.”
A house built by slaves
A letter written Sept. 6, 1818, from Monroe to his son-in-law, George Hay, describes the construction of the Monroes’ guest house that year. Two enslaved craftsmen, a man named George and a carpenter by the name “Peter Malorry,” were credited with building the house. It was that letter, along with tree ring dating, which led researchers to determine that the house, Highland’s main standing structure, was actually the presidential guest house and not a remnant of the main house as was previously thought.
On the tour, the first view of the guesthouse is through the eyes of an enslaved cook named Hannah. She strikes up a conversation with an enslaved blacksmith named Nelson, talking about the two men who built the house.
“When Mr. Monroe brought them down from Loudoun County last summer to build this house, all they talked about was missing their wives,” Nelson says. “It was so hard for them to be apart and never know when and for how long they can stay in the same place.”
It’s through Nelson that the viewer learns of Monroe’s two properties, Oak Hill and Highland, and the pain caused to enslaved families that were torn between the two.
“I bet it never occurs to him,” Hannah says of Monroe. “When he separates our husbands and wives or, God forbid, our little ones from their parents.”
For Bon-Harper, it was crucial that the guest house be viewed through the prism of slavery.
“One really important piece I felt was to have the first interaction with the presidential guest house be through the enslaved men who actually did the construction,” said Bon-Harper. “In the letter, Monroe talks about building a new house and he refers to them, one by name and one by occupation, so we know who these men are. We felt it was important to bring out their part of the story.”
While historical characters play a central role in telling the story of Highland, the AR narrative also focuses on larger historical trends that shaped the lives of its inhabitants. The rise of cotton production in the South decreased the economic viability of mixed grain and tobacco farming in regions like the Virginia Piedmont, Bon-Harper said. The drop in Highland’s profitability in the late 1810s and early 1820s led to a growing sense of uncertainty among those who lived there.
In 1826, facing potential financial ruin, Monroe sold the house at Highland and later its farmland. With the sale of the land, many of Highland’s enslaved men, women and children were dispersed southward to a cotton plantation called Casa Bianca in Jefferson County, Florida. Monroe had stipulated that enslaved families be kept together, but historians have not confirmed that promise was kept.
“In the AR tour, we bring to life individuals, because people connect with the human side,” Bon-Harper said. “But we also talk about those larger narratives of history like politics, economics and ecology. We contextualize each character in the same way we contextualize Monroe and his historical significance.”
Public beta testing of the AR tour begins Feb. 5 and will continue into early spring. During this period, guests who purchase regular admission may experience the new AR tour at no cost in exchange for completing a survey. After the testing period, the complete Highland experience will cost $19 and include both a guided tour and the AR tour.
