Case Study
At the Human-Centered Computing (HCC) Research Group, we investigate the interconnectedness of data, algorithms and context in order to enable meaningful human-machine collaboration. Within the HCC Data Lab, we establish formats and activities that open up alternative perspectives on data, contribute to knowledge transfer, and support interdisciplinary collaboration.
The case study »Augmenting Viktoria« is situated at the intersection of HCI research and digital museum mediation. Digital technologies are increasingly being used in museum mediation, however, their effective use is still under research. We contribute to this line of research by exploring the potential of mixed reality for museum mediation. Our case study focuses on the sculpture »Viktoria« at the Deutsches Historisches Museum.
The case study was guided by
the following questions.
1
How can museum visitors without prior knowledge of art history be enabled to discover the iconological meaning of an exhibit?
2
How can different needs in terms of guidance be reflected in the selected modes of activity?
3
How can modes of activity be translated into a strictly gaze-based navigation and selection technique?
»Viktoria« at Deutsches Historisches Museum
Our case study focuses on one object: »Viktoria«, a 3.90-m tall allegorical representation of the goddess of victory, was completed in white marble by the German sculptor Fritz Schaper in 1885. The sculpture opens up especially two interconnected lines of interpretation which can be linked to the visual and material properties of the object: first, the iconographic meaning of a number of attributes and second, the object’s biography and its interconnectedness with German history.
Regions of Interest
Once we had chosen »Viktoria« as the object that we want to focus on with our case study, we carried out a two-stage visitor study and a gaze-fixation study with the HoloLens at the museum. The results helped us to better understand how visitors interact naturally with the sculpture, while no additional interpretation is offered. We also learned more about the visitors‘ information needs and what elements of the sculpture potentially spark their interest.
Based on the self-reported interest derived from the visitor study as well as the gaze-fixations recorded with the HoloLens, we were able to define a total of seven »Regions of Interest« (ROIs) (see example image on the left). These ROIs were used as reference points for the content development process. Our selection also included ROIs that had been missed by visitors during the gaze-fixation study, especially if these missed ROIs could be linked to the interpretation about recurring self-reported areas of interest derived from our visitor study.
Object-centered mediation
Our HoloLens-based prototype aims to encourage visitors to closely examine the sculpture while supporting them in deciphering its meaning. The design process was informed by two fundamental concepts within museum mediation: object-centered mediation and modes of guidance. We address two concerns with our prototype that coincide with the growing interest in incorporating digital media into museum education:
1
digital media has the potential to divert visitors‘ attention from the museum exhibit
2
common technological and narrative approaches often reduce the visitors‘ initiative
»Presence«
We designed the HoloLens prototype in such a way that the visitors’ gaze is directed to the museum exhibit’s material presence and quality. We achieve this by applying an object-centred interpretation strategy that minimises the need to avert the eyes away from the object. The HoloLens technology allows us to directly connect interpretative content to the visual properties or object areas at which a visitor is currently looking. Our object-centered interpretation and design approach revolves around the concept of »presence«. The first facet of this concept relates to the material presence of the museum object and its effects on meaning making. The second facet relates to a sense of co-presence with the sculpture in a mixed reality-experience. Presence, in the second sense, is commonly understood as a result of immersion.
Gaze-based Interaction
We decided to solely rely on gaze-based interaction and use an ephemeral style of highlighting that would not interfere with the aesthetic quality of the sculpture. This also means that the visitors‘ attention is not drawn to (virtual) buttons or the use of gesture-navigation. Only when focussing on a »Region of Interest«, a dwell-timer appears. If the visitors‘ gaze remains on the ROI, the dwell-timer is completed and the content is displayed.
Modes of Guidance
We explore three modes of guidance with our prototype. The exemplary modes of guidance correspond to typical analog approaches in museum mediation: self-guided, guided, and bi-directional. Depending on the mode of guidance, visitors experience different realisations of how they can navigate (user- or system-initiated) and select a Region of Interest and access the associated interpretative content.
Depending on the mode of guidance, the visitor has different options to navigate and confirm their selection by competing the dwell-timer.
Self-guided mode during which the visitor decides at all times what aspect of the sculpture they want to explore
In the self-guided mode, visitors can explore the museum object and initiate the access to interpretation themselves. As soon as their gaze enters the collider that surrounds a ROI, the gaze starts to emit particles and they can decide to interact with it. By remaining within the collider area of the ROI with their gaze, the dwell-timer is completed and the associated content is triggered. The visitors only select as many ROI as they wish (and in their own pace and order).
Guided mode during which the system has the initiative and guides the visitor through a predefined narrative
In the guided format, the system highlights each of the RoI in a fixed pre-defined order. Visitors are guided completely by the system. The ROI are consecutively highlighted with particles. As soon as the visitors enter the collider area surrounding the highlighted ROI with their gaze and complete the dwell-timer, the system triggers the associated content. The system guides the visitor to all ROIs until all content has been accessed and a conclusion is provided.
Bi-directional, dialogic mode in which the visitor and the system alternatively share the initiative.
The bi-directional format realises a »dialogic« mode of guidance in which both the visitor and the system can take initiative. First, the initiative is given to the user to explore the sculpture actively (user-initiative). After the first ROI has been completed, the system has the initiative to cue the next ROI (system-initiative). This alternation goes on until the ROIs that were marked as central to understanding the meaning of the sculpture have been accessed.
Augmentation & Content
We defined a content unit for each »Region of Interest« that we tailored to the information needs gathered from the visitor study. We also added an introduction that would mark the start of the MR experience and a conclusion. We developed the interpretative content in close dialogue with the museum and made sure to include a variety of media, including three dimensional objects and augmentations, still images, animations, visual effects, and audio text, each positioned in relation to the sculpture.
Reconstruction
When visitors fixate on the ROI surrounding the hexagonal mould at the right shoulder of »Viktoria«, a schematic 3D reconstruction of the missing arm of the sculpture is augmented. Viktoria’s right arm was originally stretched upwards holding a laurel wreath. The arm was damaged and lost during the Second World War. The laurel wreath is one of the main attributes of »Viktoria« and supports the identification of the sculpture as the goddess of victory. The laurel wreath symbolised fame and victory already in ancient Rome. An audio text offers background information while the augmentation of the arm and the laurel wreath is displayed.
Spatial Timeline
Another ROI, located at the base of the sculpture, triggers a spatial timeline visualisation which places »Viktoria« in context with other artworks. This display aims to illustrate how the emulation of artworks led to the incorporation of visual characteristics and symbolic meaning known from Greek and Roman antiquity in sculptures created during Prussian Classicism (19th century). Images of the related sculptures include the famous »Viktoria« by Friedrich Drake that sits on top of the Berlin Victory Column (Siegessäule) or the Viktoria of the »Quadriga« that was created by Johann Gottfried Schadow for the Brandenburg Gate.
Inscription
The timeline is linked to the ROI surrounding the inscription on the lower right side of the statue’s base. Through our gaze-fixation study we learned that this physical element of the sculpture is commonly overlooked by visitors. However, it provides information that many mentioned during the visitor study. The inscription shows 1885 as the year of the sculpture’s completion and the artist’s name »Schaper«. The first mark of the spatial timeline visualisation that is associated with this ROI is thus created by the year 1885 as a visible aspect of the physical object.
Materiality
When visitors focus on the head of »Viktoria«, the content associated with this ROI is triggered: the sculpture’s height, total weight, and the weight of each wing are displayed as augmentations. Additionally, photos that were taken during the installation at the museum are displayed next to the sculpture. Their placement is based on the position from which the photo was taken. Thus, the visitors can view the photo and the sculpture at the same time from the same angle. An audio text offers additional background information.
Evaluation
We had planned to conduct an in situ a user study in March 2020 to evaluate our prototype. Unfortunately, due to the closure of museums caused by COVID-19, the user study had to be cancelled. Instead, we set up a low-scale testing with six participants (four employees of the museum and two third-party contractors working for the museum) in June 2021.
Of the six participants, two had prior experience with the HoloLens. All other participants wore the HoloLens for the first time. One of the participants wore varifocal glasses, which influenced their ability to focus on either close or distant elements. Overall, participants reported only little to no problems associated with the handling of the HoloLens, some reported slight discomfort while wearing the head-mounted display.
Of the six participants, two tested the guided mode, two the self-guided mode and two the bi-directional mode. In order to not influence their expectation and behaviour, they were not informed that there are three modes of guidance and which one they were assigned. However, the small number of participants and their prior familiarity with the sculpture and its meaning limited our insights regarding the influence of the mode of guidance on the learning experience and overall user experience. Nonetheless, we were able to gain a better understanding on the perceived presence of the sculpture.
All participants completed a questionnaire that gave us an impression about whether our goals were reached. Considering our goal to design the mixed reality application in such a way that does not overpower the museum object, the participants confirmed that the sculpture was in their focus at all times during the experience. The participants further reported that the visual interpretative content was not more present to them than the sculpture and that it did not distract them from viewing the sculpture. These answers indicate that one of our design goals – to not divert the visitors‘ attention from the museum exhibit – was indeed reached.
The participants were also asked to judge the overall experience on a likert scale from 1 (very bad) to 7 (very good). Four participants chose »rather good«, one participant »good« and another participant »very good«, which is a positive result that can be interpreted as a motivation to further explore the options of using the HoloLens for object-centered museum mediation.
If proceeded further, we would recommend to conduct a thorough user testing in order to better understand how to improve navigation and selection and give visitors an overview of what they can expect in order to help them to make an informed decision. For example, explain the three modes of guidance at the beginning and let visitors chose their preferred mode. We would also advise to prioritise three-dimensional content (if possible), such as augmentations and extensions of the object, in oder to truly merge the physical object with additional holographic elements. In our case, this particularly applied to the spatial timeline where we only used two-dimensional photos of other sculptures.
Publication
We published a detailed description of the concept, design, and development of our case study in this paper:
Glinka, K., Fischer, P. T., Müller-Birn, C., Krohn, S. (2020). „Investigating Modes of Activity and Guidance for Mediating Museum Exhibits in Mixed Reality“.
In: J.H. Israel / C. Kassung / J. Sieck (Eds.). Kultur und Informatik: Extended Reality. Berlin: vwh
Source Code
Our prototype is available as documented source code.
This application is licensed under MIT License.
Contact
Freie Universität Berlin
FB Mathematik und Informatik
Institut für Informatik
Human-Centered Computing (HCC)
Königin-Luise-Str. 24/26
D – 14195 Berlin
Email: hcc@zedat.fu-berlin.de
Team
The HCC Research Group’s contribution to the case study integrated perspectives from collaborative computing, human-computer interaction, and cultural studies.
Prof. Dr. Claudia-Müller Birn
Katrin Glinka
Dr. Patrick Tobias Fischer
Acknowledgments
The case study was developed in collaboration between the Human-Centered Computing Research Group at the Freie Universität Berlin, Stiftung Preußischer Kulturbesitz, and Deutsches Historisches Museum and was funded by Cluster of Excellence »Bild Wissen Gestaltung« and the Minister of State for Culture and the Media (BKM) as part of museum4punkt0.