History Re-Experienced : Implementing Mixed Reality Systems into Historic House Museums

As immersive technologies have become ubiquitous today, traditional museums are finding success augmenting existing exhibits to increase visitors’ satisfaction. However, due to the immutable nature of house museums, and their tendency to place visitors in direct contact with historical artifacts, museum managers are seeking original approaches to cultural preservation. Implementing mixed reality systems into historic house museums is one such approach. The goal of this study is to develop and test a conceptual matrix that guides how designers use the affordances of mixed reality systems to create experiences that align with the range of historical narratives found in house museums. Experiences that can contribute to improving visitors’ satisfaction, self-interpretation, and understanding of the homeowner’s life and the community within which they lived. Building on human-centered design methods, the researcher developed and tested a prototype of an augmented reality (AR) mobile application centered on the Pope House Museum in Raleigh, North Carolina. The outcome of the research suggests house museum visitors should have agency in deciding the lens through which they experience the variety of historical narratives present in the home.


I. INTRODUCTION
At the intersection of history, home, and cultural preservation stands historic house museums. Viewed as instruments of communication [1], house museums are invaluable social and cultural apparatuses [2] through which visitors can experience history in the living environment of a home. Giovanni Pinna, former chairman of the International Committee for Historic House Museums, attributes the significance of these cultural sites to their ability "to evoke history and put the visitor into direct contact with it" [1]. To continue a legacy as a source of living history, house museums bear the responsibility of remaining immutable in a changing society. The unchanging ethos of preserved historic homes and the impossibility of manipulating their meaning [1] make it difficult for house museums to adapt to the needs and desires of today's visitors [2]. As a result, museums managers are finding difficulties responding to visitors' interests and generating adequate revenue to keep the doors of house museums open. Linda Young, historian, and author Manuscript received October 10, 2019; revised September 11, 2020. Shadrick Addy is with The Ohio State University, Columbus, OH 43210 USA (e-mail: addy.29@osu.edu). of the book, Historic House Museums in the United States and the United Kingdom: A History, asserts that "it is a major challenge of institutional survival for many house museums to entice sufficient paying visitors to maintain their operations" [2].
As traditional museums are turning towards mixed reality systems and other contemporary apparatuses to increase the entertainment value of exhibitions, the in-situ nature of house museums raises new challenges during the implementation of modern technologies into historic homes. Because house museums reflect the atmosphere of a historic era [3], implementing technology foreign to the period represented within the home can take away from the authenticity of exhibited artifacts. Therefore, museum managers primarily rely on tour guides and original artifacts in the home to educate and serve as sources of entertainment for visitors. However, in today's technology-driven society, a limited approach to enriching the house museum visitation experience cannot compete with traditional museums that are implementing mixed reality systems to increase visitors' satisfaction [4]. If a goal of house museums is to bridge personal connections between visitors and the history present in the home [1], historic houses must become an interpretive space [5] that encourages imagination, discovery, and reflection through innovative approaches.
This research aims to contribute to the preservation of house museums by developing a practical approach to implementing mixed reality systems into historic homes. This paper presents an analysis of a conceptual matrix used to inform the design and implementation of an augmented reality (AR) mobile application prototype at the Pope House Museum in Raleigh, North Carolina.

A. Research Context: Pope House Museum
Dr. Manassa Thomas Pope was one of North Carolina's most prominent African American homeowners during the nineteenth century [6]. A physician and political figure, Dr. Pope built his family's home in 1901 in a segregated district in Raleigh, NC (Fig. 1)

II. RESEARCH METHODS
This section provides an overview of user experience research methods used to ground the investigation. Martin and Hanington recognize the following approaches as legitimate design research activities [7].

A. Observational Study
The primary focus of the observational study was to understand how docents guide visitors during a tour at the Pope House Museum. A secondary focus of the study was to understand how visitors moved through the museum and interact with artifacts present in the home. Using a semi-structured observational approach, the researcher occasionally participated in the tour alongside visitors.

B. Personas & User Journey Maps
Based on the information gathered from the observational study at the Pope House Museum, the researcher developed personas of visitors to situate design explorations and ground the investigation in a human-centered research approach. The journey map articulates how docents guide visitors during a visitation tour before the implementation of the mixed reality system. During the development of the prototype, the researcher revised the journey map to express how the implementation of the mixed reality system would change the museum visitation experience.

C. Case Studies
To understand current and emerging uses of mixed reality systems, the researcher reviewed extended reality (XR) experiences and the means through which curators and educators implemented them within and outside of the museum environment. Experiences reviewed by the researcher included Derek Ham's "I Am A Man" VR Experience [8], the Terracotta Warriors of the First Emperor AR-enabled exhibition at The Franklin Institute [9], Anne Frank House VR [10], and the England Originals AR mobile application [11]. The researcher analyzed the means through which visitors and participants accessed and experienced each application. From the analysis, the researcher drew key functionalities of the experiences most applicable to the implementation of mixed reality systems into house museums.

D. Prototype & User Testing
The researcher developed a functional prototype of an AR mobile application for the Pope House Museum in Raleigh, NC. User testing the prototype was essential to understanding which elements of the AR mobile application were the most successful or challenging to implement based on various conditions within the museum.

III. CONCEPTUAL FRAMEWORK
This section provides an overview of existing frameworks used to construct the conceptual matrix that informed the design and implementation of the AR mobile application.

A. Virtuality Continuum
The Virtuality Continuum is a framework developed by Paul Milgram and Fumio Kishino to categorize different mixed reality displays [12]. The authors identified six classes of mixed reality display environments used as metrics for distinguishing the various subsets of mixed reality experiences.
Initial studies (not presented in this paper) explored various forms of experiences along the virtuality continuum. Later explorations (the focus of this paper) consider only the use of augmented reality and hybrid reality displays (Fig. 3). The researcher defines augmented reality as cases in which two information types, visual and verbal cues, [4] augment a real environment using see-through hand-held displays [12]. Although hybrid reality (HR) is not explicitly located on the virtuality continuum, the researcher refers to HR as cases in which direct interaction between virtual and physical artifacts takes place. Therefore, the researcher situates HR in the gray area between augmented reality and augmented virtuality on the Virtuality Continuum.

B. Experiential Value Taxonomy
The experiential value taxonomy is a system for categorizing key attributes that influence consumption behavior [13]. The investigation excludes the extrinsic value dimension from the taxonomy and considers only intrinsic, active, and reactive values (Fig. 4). The exclusion of extrinsic values is due to the research's primary focus on the types of mixed reality experiences used to enhance the museum visitation experience. The investigation places less consideration on the actual devices (mobile phones, head-mounted displays, etc.) used to view mixed reality experiences. Therefore, playfulness and aesthetics response are the only two attributes adapted from Mathwick and colleagues' experiential value taxonomy and Holbrook's Consumer value framework. The researcher refers to playfulness and aesthetic values as self-oriented experiences   [14]. The differentiating factor between playfulness and aesthetics is that playfulness involves an active engagement within an activity [13]; whereas aesthetics value refers to an appreciation of the design or beauty of the experience [15].

C. Mobile Media Alignment Framework
Brett Oppegaard, an Associate Professor in the School of Communications at the University of Hawaii, developed the Mobile Media Alignment framework to serve as a conceptual guide for designing human-centered augmented reality experiences [16]. Oppegaard suggests that designers put the framework into a practical use and create experiences that effectively align digital information over physical environments based on a three-tier Mobile Media Alignment strategy that consists of location, spatial, and contextual alignments [16]. According to Oppegaard, the three-tier guidance system, "is an idea that could lead people, especially designers and users, toward a new perspective on augmented reality and augmented places it can create" [16].

IV. CONCEPTUAL MATRIX EXPLAINED
Combining the Virtuality Continuum, Experiential Value taxonomy, and the Mobile Media Alignment framework, the researcher developed a conceptual matrix to ground the investigation into existing frameworks (Table I). Features of the mobile application and methods of implementation within the museum fall into individual cells within the matrix (Table  II & Table III). This section provides an overview of how the conceptual matrix informed the development of the mobile application prototype.

A. Augmented Reality 1) Location alignment
Oppegaard refers to location as the conditions under which interaction occurs and the context from which we incorporate information into use [16]. It is through location alignment that both spatial and contextual alignment becomes a possibility [16]. For the augmented reality experience, the researcher regarded location as the places within the museum where augmentation would take place. In the house museum environment, location alignment refers to the means through which the mobile application would initiate mixed reality experiences based on the visitor's position within the home.
During observational studies at the Pope House Museum, the researcher noted that the piano in the former parlor of the museum was an important artifact and location where visitors frequently gathered during tours. The AR mobile application utilizes photos placed on the piano as image markers to trigger augmented experiences that present information about the family to visitors using visual and verbal cues (Fig. 6).

2) Spatial alignment
Oppegaard describes Spatial Alignment as an awareness of everything physically nearby the user yet outside of their direct perceptions [16]. The researcher used a window in the formal parlor, one that visitors often stare through as docents describe historic life in the once segregated neighborhood, to explore spatial alignment features of the AR mobile application. To provide visitors with spatial information, the researcher used an AR marker placed on the window frame to trigger visual cues. As visitors point their smartphone's camera at the marker, a historic view of the exterior of the home renders over the window frame.

3) Contextual alignment
Oppegaard defines Contextual Alignment as the precise matching of experiences to the user's needs in the specific context of the user and in customized configurations [16]. Specifically, this exploration refers to contextual alignment as an act of providing agency to visitors to decide their visitation preferences. Data collected from the observation study informed the development of features within the AR mobile application that allows visitors to choose between two visitation modes (Fig. 7). In Story Mode, a virtual docent guides visitors through the museum in a scripted sequential progression. Tour Mode allows visitors to freely explore the museum and receive information cues by interacting with artifacts based on their interests.

B. Hybrid Reality 1) Location alignment
According to Steven Neale and colleagues, handling artifacts provides a spatial and physical understanding of the artifact [17]. Docents advised visitors to not touch exhibited artifacts during observed visitation tours at the Pope House. Therefore, when considering location alignment for the hybrid reality experience, the researcher used 3D printed objects as triggers to render virtual models of museum artifacts that visitors could physically hold and interact with (Fig. 8). Fig. 8. A participant uses a 3D printed handle to interact with a museum artifact enclosed in a display case.

2) Spatial alignment
Artifacts in the display case at Pope House were taken from various locations within the home and exhibited in the dining area. Spatial Alignment in the HR exploration involves providing visitors with information about the origins of encased artifacts and how the family used them within the home.

3) Contextual alignment
For the HR contextual alignment exploration, the researcher considered how the application could enable virtual avatars of family members to present historical narratives of the home to visitors. A feature incorporated within the prototype allows visitors to point their device's camera at a marker to trigger a virtual avatar of a family member playing the piano. Visitors also have the agency to choose between virtual avatars of specific family members to interact with during the tour.

4) Playfulness and aesthetics
Both the augmented reality and hybrid reality experiences incorporate elements of playfulness and aesthetic values. Mathewick and colleagues refer to playfulness as an active engagement within an activity [13]. Holbrook describes aesthetic value as an appreciation of some consumption experience [14]. As both values are self-oriented [14], the researcher designed the prototype to give visitors agency to tour the home through a docent-guided (story mode) or self-guided (tour mode) approach. Through play, the interactive artifacts and dynamic information cues provide opportunities for visitors to actively engage with historical narratives present in the home.

V. TESTING THE PROTOTYPE
Testing the prototype at the Pope House was an essential part of the research investigation (Fig. 9). Data collected from the visitors and docents during the user test provided insights on the usability of trackers in different lighting conditions, and the user experience of the mixed reality system. Participants began the user test at the start of the tour and received instructions on how to use the application only upon request. With the participants' consent, the researcher captured photos and videos as they interacted with the mobile application for documentation and further analysis.

A. Takeaways
The museum managers and docents were excited about the potential of the prototype and how mixed reality can contribute to the preservation of the Pope House's tangible and intangible cultural artifacts. The docents were able to use the application with little instructional guidance. Feedback from docents included suggestions on placement of AR markers within the home and changing the names of the visitation modes (Story Mode and Tour Mode) to be more concise descriptors of the two experiences. Most visitors stated that they preferred Tour Mode over Story Mode, as the former provided more information about the Pope family and was less scripted than the latter. Visitors acknowledged that holding the 3D printed handle and using it to interact with virtual artifacts made the visitation tour much more engaging.
During the user test, the researcher observed that photos used as markers were much easier to track and provided little interruption. In contrast, the designed AR markers (VuMarks) were more difficult to track due to their small size and low lighting within the museum. Markers placed on the floor (Fig.   10) provided more interruptions than those within arm's reach of participants.

VI. CONCLUSION
This investigation explored how house museums can implement mixed reality systems to increase visitors' satisfaction. The goal of this research was to emphasize the cultural significance of house museums and contribute to its preservation through mixed reality experiences that complement exhibited artifacts within the home. It was important that the implementation of the system did not seem out of context, but instead provided an additional lens through which visitors can experience the historical narratives present in the home.
Situating the investigation in a real-world context at the Pope House Museum was an essential part of the design process. Gathering factual historical data about the Pope family to develop the prototype allowed the researcher to test the mobile application in-situ and collect feedback from the museum visitors to improve the system's usability. Working with the museum staff and visitors to realize the objectives of the project further emphasizes the collaborative and interdisciplinary nature of the investigation. Designers, curators, and educators alike must continue to work together to find innovative ways to contribute to the preservation of historic house museums.
Future investigations should consider how mixed reality experiences developed to complement house museum visitation can facilitate interaction between museum visitors. Designers must regard the house museum as a collective memory shaped by the family that once lived in the home, and community members of today that continue to walk in the footsteps of the homeowner's historic legacy.

CONFLICT OF INTEREST
The author declares no conflict of interest.

AUTHOR CONTRIBUTIONS
The author, Shadrick Addy, conducted the research, developed the AR mobile application prototype, facilitated the user test, analyzed the data, and wrote the paper.