At the Human-Centered Computing (HCC) Research Group, we investigate the interconnectedness of data, algorithms and context in order to enable meaningful human-machine collaboration. Within the HCC Data Lab, we establish formats and activities that open up alternative perspectives on data, contribute to knowledge transfer, and support interdisciplinary collaboration.
The case study »Augmenting Viktoria« is situated at the intersection of HCI research and digital museum mediation. Digital technologies are increasingly being used in museum mediation, however, their effective use is still under research. We contribute to this line of research by exploring the potential of mixed reality for museum mediation. Our case study focuses on the sculpture »Viktoria« at the Deutsches Historisches Museum.
The case study was guided by
the following questions.
How can museum visitors without prior knowledge of art history be enabled to discover the iconological meaning of an exhibit?
How can different needs in terms of guidance be reflected in the selected modes of activity?
How can modes of activity be translated into a strictly gaze-based navigation and selection technique?
»Viktoria« at Deutsches Historisches Museum
Our case study focuses on one object: »Viktoria«, a 3.90-m tall allegorical representation of the goddess of victory, was completed in white marble by the German sculptor Fritz Schaper in 1885. The sculpture opens up especially two interconnected lines of interpretation which can be linked to the visual and material properties of the object: first, the iconographic meaning of a number of attributes and second, the object’s biography and its interconnectedness with German history.
Regions of Interest
Once we had chosen »Viktoria« as the object that we want to focus on with our case study, we carried out a two-stage visitor study and a gaze-fixation study with the HoloLens at the museum. The results helped us to better understand how visitors interact naturally with the sculpture, while no additional interpretation is offered. We also learned more about the visitors‘ information needs and what elements of the sculpture potentially spark their interest.
Our HoloLens-based prototype aims to encourage visitors to closely examine the sculpture while supporting them in deciphering its meaning. The design process was informed by two fundamental concepts within museum mediation: object-centered mediation and modes of guidance. We address two concerns with our prototype that coincide with the growing interest in incorporating digital media into museum education:
digital media has the potential to divert visitors‘ attention from the museum exhibit
common technological and narrative approaches often reduce the visitors‘ initiative
We designed the HoloLens prototype in such a way that the visitors’ gaze is directed to the museum exhibit’s material presence and quality. We achieve this by applying an object-centred interpretation strategy that minimises the need to avert the eyes away from the object. The HoloLens technology allows us to directly connect interpretative content to the visual properties or object areas at which a visitor is currently looking. Our object-centered interpretation and design approach revolves around the concept of »presence«. The first facet of this concept relates to the material presence of the museum object and its effects on meaning making. The second facet relates to a sense of co-presence with the sculpture in a mixed reality-experience. Presence, in the second sense, is commonly understood as a result of immersion.
We decided to solely rely on gaze-based interaction and use an ephemeral style of highlighting that would not interfere with the aesthetic quality of the sculpture. This also means that the visitors‘ attention is not drawn to (virtual) buttons or the use of gesture-navigation. Only when focussing on a »Region of Interest«, a dwell-timer appears. If the visitors‘ gaze remains on the ROI, the dwell-timer is completed and the content is displayed.
Modes of Guidance
We explore three modes of guidance with our prototype. The exemplary modes of guidance correspond to typical analog approaches in museum mediation: self-guided, guided, and bi-directional. Depending on the mode of guidance, visitors experience different realisations of how they can navigate (user- or system-initiated) and select a Region of Interest and access the associated interpretative content.
Depending on the mode of guidance, the visitor has different options to navigate and confirm their selection by competing the dwell-timer.
Self-guided mode during which the visitor decides at all times what aspect of the sculpture they want to explore
In the self-guided mode, visitors can explore the museum object and initiate the access to interpretation themselves. As soon as their gaze enters the collider that surrounds a ROI, the gaze starts to emit particles and they can decide to interact with it. By remaining within the collider area of the ROI with their gaze, the dwell-timer is completed and the associated content is triggered. The visitors only select as many ROI as they wish (and in their own pace and order).
Guided mode during which the system has the initiative and guides the visitor through a predefined narrative
In the guided format, the system highlights each of the RoI in a fixed pre-defined order. Visitors are guided completely by the system. The ROI are consecutively highlighted with particles. As soon as the visitors enter the collider area surrounding the highlighted ROI with their gaze and complete the dwell-timer, the system triggers the associated content. The system guides the visitor to all ROIs until all content has been accessed and a conclusion is provided.
Bi-directional, dialogic mode in which the visitor and the system alternatively share the initiative.
The bi-directional format realises a »dialogic« mode of guidance in which both the visitor and the system can take initiative. First, the initiative is given to the user to explore the sculpture actively (user-initiative). After the first ROI has been completed, the system has the initiative to cue the next ROI (system-initiative). This alternation goes on until the ROIs that were marked as central to understanding the meaning of the sculpture have been accessed.
Augmentation & Content
We defined a content unit for each »Region of Interest« that we tailored to the information needs gathered from the visitor study. We also added an introduction that would mark the start of the MR experience and a conclusion. We developed the interpretative content in close dialogue with the museum and made sure to include a variety of media, including three dimensional objects and augmentations, still images, animations, visual effects, and audio text, each positioned in relation to the sculpture.
The timeline is linked to the ROI surrounding the inscription on the lower right side of the statue’s base. Through our gaze-fixation study we learned that this physical element of the sculpture is commonly overlooked by visitors. However, it provides information that many mentioned during the visitor study. The inscription shows 1885 as the year of the sculpture’s completion and the artist’s name »Schaper«. The first mark of the spatial timeline visualisation that is associated with this ROI is thus created by the year 1885 as a visible aspect of the physical object.
We had planned to conduct an in situ a user study in March 2020 to evaluate our prototype. Unfortunately, due to the closure of museums caused by COVID-19, the user study had to be cancelled. Instead, we set up a low-scale testing with six participants (four employees of the museum and two third-party contractors working for the museum) in June 2021.
Of the six participants, two had prior experience with the HoloLens. All other participants wore the HoloLens for the first time. One of the participants wore varifocal glasses, which influenced their ability to focus on either close or distant elements. Overall, participants reported only little to no problems associated with the handling of the HoloLens, some reported slight discomfort while wearing the head-mounted display.
Of the six participants, two tested the guided mode, two the self-guided mode and two the bi-directional mode. In order to not influence their expectation and behaviour, they were not informed that there are three modes of guidance and which one they were assigned. However, the small number of participants and their prior familiarity with the sculpture and its meaning limited our insights regarding the influence of the mode of guidance on the learning experience and overall user experience. Nonetheless, we were able to gain a better understanding on the perceived presence of the sculpture.
All participants completed a questionnaire that gave us an impression about whether our goals were reached. Considering our goal to design the mixed reality application in such a way that does not overpower the museum object, the participants confirmed that the sculpture was in their focus at all times during the experience. The participants further reported that the visual interpretative content was not more present to them than the sculpture and that it did not distract them from viewing the sculpture. These answers indicate that one of our design goals – to not divert the visitors‘ attention from the museum exhibit – was indeed reached.
The participants were also asked to judge the overall experience on a likert scale from 1 (very bad) to 7 (very good). Four participants chose »rather good«, one participant »good« and another participant »very good«, which is a positive result that can be interpreted as a motivation to further explore the options of using the HoloLens for object-centered museum mediation.
If proceeded further, we would recommend to conduct a thorough user testing in order to better understand how to improve navigation and selection and give visitors an overview of what they can expect in order to help them to make an informed decision. For example, explain the three modes of guidance at the beginning and let visitors chose their preferred mode. We would also advise to prioritise three-dimensional content (if possible), such as augmentations and extensions of the object, in oder to truly merge the physical object with additional holographic elements. In our case, this particularly applied to the spatial timeline where we only used two-dimensional photos of other sculptures.
We published a detailed description of the concept, design, and development of our case study in this paper:
Glinka, K., Fischer, P. T., Müller-Birn, C., Krohn, S. (2020). „Investigating Modes of Activity and Guidance for Mediating Museum Exhibits in Mixed Reality“.
In: J.H. Israel / C. Kassung / J. Sieck (Eds.). Kultur und Informatik: Extended Reality. Berlin: vwh
Our prototype is available as documented source code.
This application is licensed under MIT License.
Freie Universität Berlin
FB Mathematik und Informatik
Institut für Informatik
Human-Centered Computing (HCC)
D – 14195 Berlin
The HCC Research Group’s contribution to the case study integrated perspectives from collaborative computing, human-computer interaction, and cultural studies.
Prof. Dr. Claudia-Müller Birn
Dr. Patrick Tobias Fischer
The case study was developed in collaboration between the Human-Centered Computing Research Group at the Freie Universität Berlin, Stiftung Preußischer Kulturbesitz, and Deutsches Historisches Museum and was funded by Cluster of Excellence »Bild Wissen Gestaltung« and the Minister of State for Culture and the Media (BKM) as part of museum4punkt0.