{"id":9944,"date":"2024-06-04T08:18:30","date_gmt":"2024-06-04T08:18:30","guid":{"rendered":"https:\/\/www.ub.edu\/artsoundscapes\/?p=9944"},"modified":"2025-05-20T10:56:44","modified_gmt":"2025-05-20T10:56:44","slug":"multisensory-map","status":"publish","type":"post","link":"https:\/\/www.ub.edu\/artsoundscapes\/multisensory-map\/","title":{"rendered":"BLOG 26 | Welcome to our multisensory map \u2013 a portal to the captivating world of Artsoundscapes!"},"content":{"rendered":"

[et_pb_section fb_built=\u00bb1″ _builder_version=\u00bb4.9.2″ _module_preset=\u00bbdefault\u00bb custom_padding=\u00bb|||0px||\u00bb][et_pb_row _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb custom_padding=\u00bb0px||0px||false|false\u00bb][et_pb_column type=\u00bb4_4″ _builder_version=\u00bb4.9.2″ _module_preset=\u00bbdefault\u00bb][et_pb_text _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb]<\/p>\n

By Daniel Ben\u00edtez and Lidia Alvarez-Morales<\/span><\/p>\n

[\/et_pb_text][et_pb_text _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb hover_enabled=\u00bb0″ sticky_enabled=\u00bb0″]<\/p>\n

The dissemination strategy of the Artsoundscapes project has moved a step forward with the development of a multisensory interactive map<\/a>, released on our website in May 2024. We envision this map as a direct glimpse into the scope of our research, serving as a window into a world where archaeology, acoustics, and technology converge.<\/p>\n

[\/et_pb_text][et_pb_text _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb hover_enabled=\u00bb0″ sticky_enabled=\u00bb0″]<\/p>\n

On a first instance, the multisensory map<\/a> was conceived as a comprehensive visual narrative of the geographical expanse covered by the Artsoundscapes extensive fieldwork campaigns in Siberia<\/a>, Catalonia<\/a>, Valencia<\/a>, C\u00e1diz <\/a>and South Africa<\/a>. In other to showcase the group’s work around the world, other campaigns in which the project has collaborated, or that belong to previous research projects, have also been included. A second goal was for the map to serve as a multisensory repository to experience the acoustics of the great variety of rock art sites we have studied, so that we transcend the complexities of data representation by offering a more intuitive and accessible representation of the auditory landscapes we have documented.<\/span><\/p>\n

[\/et_pb_text][et_pb_text _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb hover_enabled=\u00bb0″ sticky_enabled=\u00bb0″]<\/p>\n

In this entry of our blog, we explain the development process of the multisensory map<\/a>, emphasizing the integration of spatial audio rendered by means of auralizations with visual material gathered on-site. <\/span><\/p>\n

[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_column type=\u00bb4_4″ _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_text _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb custom_margin=\u00bb||-21px||false|false\u00bb custom_padding=\u00bb42px||||false|false\u00bb]<\/p>\n

Where We’ve Been: Protecting the location of the rock art sites<\/o:p><\/span><\/h4>\n

[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_column type=\u00bb4_4″ _builder_version=\u00bb4.9.2″ _module_preset=\u00bbdefault\u00bb][et_pb_text _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb custom_margin=\u00bb-11px||||false|false\u00bb hover_enabled=\u00bb0″ sticky_enabled=\u00bb0″]<\/p>\n

The first task in the development of this multisensory map<\/a> involved setting the location of all the sites studied during each fieldwork campaign, so that the sonic journey was accurately represented. However, finding a compromise between accuracy and rock art preservation was challenging. Having developed the first version of the map with the sites\u2019 exact locations, the archaeologists in the project raised the need of not revealing such information since many of these open rock art sites are particularly susceptible to the impacts of human interference, lacking conventional fences or barriers. Therefore, this cautious approach was adopted to mitigate the potential risk of these fragile and often unprotected cultural manifestation falling victim to vandalism. <\/span><\/p>\n

[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb custom_margin=\u00bb6px||6px||true|false\u00bb custom_padding=\u00bb6px||6px||true|false\u00bb][et_pb_column type=\u00bb4_4″ _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_image src=\u00bbhttps:\/\/www.ub.edu\/artsoundscapes\/wp-content\/uploads\/2024\/06\/1.png\u00bb title_text=\u00bb1″ align=\u00bbcenter\u00bb _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb width=\u00bb77%\u00bb max_width=\u00bb77%\u00bb custom_margin=\u00bb2px||2px||false|false\u00bb custom_padding=\u00bb2px||2px||true|false\u00bb][\/et_pb_image][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb custom_margin=\u00bb1px||8px||false|false\u00bb custom_padding=\u00bb1px||8px||false|false\u00bb][et_pb_column type=\u00bb4_4″ _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_text _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb text_font_size=\u00bb2px\u00bb header_6_text_color=\u00bb#000000″ custom_margin=\u00bb||0px||false|false\u00bb custom_padding=\u00bb0px||0px||false|false\u00bb]<\/p>\n

Fig 1<\/strong>. Example of two rock art motifs extracted with an angle grinder, Abric de Benirrama (municipality of La Vall de Gallinera, Province of Alicante – Spain). Photograph by Margarita D\u00edaz-Andreu.<\/em><\/p>\n

[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb custom_margin=\u00bb1px||8px||false|false\u00bb custom_padding=\u00bb1px||8px||false|false\u00bb][et_pb_column type=\u00bb4_4″ _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_text _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb text_font_size=\u00bb2px\u00bb header_6_text_color=\u00bb#000000″ custom_margin=\u00bb||0px||false|false\u00bb custom_padding=\u00bb0px||0px||false|false\u00bb]<\/p>\n

In view of the above, instead of using the exact coordinates for each rock art site on the interactive map, we decided to set them at an approximate location. This is clarified in each card on the map. <\/span><\/p>\n

[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb custom_margin=\u00bb6px||6px||true|false\u00bb custom_padding=\u00bb6px||6px||true|false\u00bb][et_pb_column type=\u00bb4_4″ _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_image src=\u00bbhttps:\/\/www.ub.edu\/artsoundscapes\/wp-content\/uploads\/2024\/06\/2.jpg\u00bb title_text=\u00bb2″ align=\u00bbcenter\u00bb _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb width=\u00bb77%\u00bb max_width=\u00bb77%\u00bb custom_padding=\u00bb0px||0px||false|false\u00bb][\/et_pb_image][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb custom_margin=\u00bb1px||8px||false|false\u00bb custom_padding=\u00bb1px||8px||false|false\u00bb][et_pb_column type=\u00bb4_4″ _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_text _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb text_font_size=\u00bb13px\u00bb custom_margin=\u00bb1px||1px||true|false\u00bb custom_padding=\u00bb1px||1px||true|false\u00bb]<\/p>\n

Fig 2.<\/strong> Screenshot of the map showing the approximate location of the studied sites. <\/em><\/p>\n

[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_column type=\u00bb4_4″ _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_text _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb custom_margin=\u00bb||-21px||false|false\u00bb custom_padding=\u00bb42px||||false|false\u00bb]<\/p>\n

The Wonders We’ve Seen<\/span>: Selecting the Visual material<\/o:p><\/span><\/h4>\n

[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_column type=\u00bb4_4″ _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_text _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb custom_margin=\u00bb-11px||||false|false\u00bb]<\/p>\n

For each rock art site, we gathered a comprehensive set of photographs documenting various aspects of the fieldwork, including some general views of the rock art sites, details of the art, and the acoustic tests. Some of these photographs are taken with the 360-degree camera (Ricoh Theta) to capture not only panoramic views of the surroundings but, more importantly, a 360-degree view from the position of the loudspeaker (representing the perspective of the emitter) and from the microphone position (listener perspective).<\/span><\/p>\n

[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_column type=\u00bb4_4″ _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_image src=\u00bbhttps:\/\/www.ub.edu\/artsoundscapes\/wp-content\/uploads\/2024\/06\/3.png\u00bb title_text=\u00bb3″ _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][\/et_pb_image][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb custom_margin=\u00bb1px||8px||false|false\u00bb custom_padding=\u00bb1px||8px||false|false\u00bb][et_pb_column type=\u00bb4_4″ _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_text _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb text_font_size=\u00bb13px\u00bb custom_margin=\u00bb1px||1px||true|false\u00bb custom_padding=\u00bb1px||1px||true|false\u00bb]<\/p>\n

Fig 3. <\/strong>Top \u2013Partial view of one of the rock art panels at Christmas shelter, Kamberg, South Africa. Bottom \u2013 Picture taken during the acoustic measurements at the same site from one of the listener positions. Photographs by Neemias Santos da Rosa @Artsoundscapes project.<\/em><\/p>\n

[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb custom_margin=\u00bb1px||8px||false|false\u00bb custom_padding=\u00bb1px||8px||false|false\u00bb][et_pb_column type=\u00bb4_4″ _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_text _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb text_font_size=\u00bb13px\u00bb custom_margin=\u00bb1px||1px||true|false\u00bb custom_padding=\u00bb1px||1px||true|false\u00bb]<\/p>\n

In particular, the 360-degree photographs taken from the perspective of the listener are used in the map to create the immersive videos – commonly referred to as 360-degree videos or spherical videos – latter posted on YouTube. In these videos the orientation of the viewpoint can be modified by using a mouse or any other compatible devices. By adding Ambisonics audio to the 360 videos, it is also possible to dynamically change the aural perspective, by synchronizing the visual movement with an appropriate sound representation while the video is reproducing, making the videos interactive.<\/span><\/p>\n

Hosting these immersive videos on Youtube serves as an excellent dissemination method in terms of interaction and accessibility to the research findings.<\/span><\/p>\n

[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_column type=\u00bb4_4″ _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_text _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb custom_margin=\u00bb||-21px||false|false\u00bb custom_padding=\u00bb42px||0px||false|false\u00bb]<\/p>\n

Restoring Sound to Rock Art Sites: First-Order Ambisonics impulse responses & Auralizations<\/o:p><\/span><\/h4>\n

[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_column type=\u00bb4_4″ _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_text _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb custom_margin=\u00bb-11px||||false|false\u00bb hover_enabled=\u00bb0″ sticky_enabled=\u00bb0″]<\/p>\n

The sonic material included in our multisensory map<\/a> comprises a selection of sound extracts related with prehistoric instruments contextualized in each site by means of auralization<\/strong> techniques. An auralization represents the process that allows a pre-recorded sound to obtain the acoustic properties of a specific place [1], i.e. enabling the reproduction of any audio as if it were emanating from a specific location in such place. Basically, to generate an auralization two elements are required: an anechoic sound \u2013 i.e. a sound recorded in a reflections-free environment, where background noises, echoes and\/or reverberation are not present \u2013 and an Impulse Response (IR) which represents the acoustic signature of a site for a specific pair of source and listener configuration.<\/span><\/p>\n

The anechoic recordings<\/strong> selected for this map were made by the Artsoundscapes project back in 2021 (find out more about the process in this link<\/a>). They feature a variety of prehistoric instruments such as flutes, shakers, and bass drums, serving as a homogenous representation of how diverse sonic elements would manifest in the distinct rock art sites explored. While these sounds cannot be considered a plausible representation or a recreation of the actual rhythms and melodies used by those who created the art, they allow to sonically explore the acoustic characteristics of each site. Each particular sound (an afro Venezuelan flute, an Australian \u201cbig seeds\u201d percussion shaker, a cow horn and a bass drum) lasts 15 seconds, conforming a 1 minute video in each card.<\/span><\/p>\n

[\/et_pb_text][et_pb_text _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb]<\/p>\n

Obtaining the spatial IRs<\/strong> that represent the acoustics of each site involved a methodology described in detail in other entries of our blog (see this one<\/a> or this one<\/a> for further information).\u00a0In summary, throughout each measurement campaign, a comprehensive set of HOA (High Order Ambisonics) recordings is gathered on site with a Zylia microphone by following international standards [2]. The recordings of the test signal used are later processed to obtained the spatial IRs under the MATLAB environment with the \u201cMIMO script<\/a>\u201d, developed by Adriano Farina (details here<\/a>),<\/u> or its updated version, SIMO script<\/a>, depending on the equipment (loudspeaker) used in field. Then, the spatial IRs in HOA format (16 channels) are decoded to first-order Ambisonics (4 channels) offering a spatial representation which aligns with YouTube’s specifications. This process ensures a rigorous and standardized treatment of the acquired audio data as we detailed in several publications<\/a>.<\/span><\/p>\n

This initial phase of processing the audio data feeds into the second processing phase: creating the auralized sound files<\/strong>. This involves convolving the anechoic recordings with the first-order Ambisonics IRs. The convolution process, along with other tasks such as audio joining and normalization, are also performed in MATLAB.<\/span><\/p>\n

Although in each studied site, many source-receiver combinations have been considered, only a representative one from each site has been selected for the videos due to the need to reduce the amount of data presented. Also, in order to sonically compare sites easily, the same audio track has been auralized with the selected S-R combination in all the sites.<\/span><\/p>\n

[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_column type=\u00bb4_4″ _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_text _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb custom_margin=\u00bb||-21px||false|false\u00bb custom_padding=\u00bb42px||||false|false\u00bb hover_enabled=\u00bb0″ sticky_enabled=\u00bb0″]<\/p>\n

Tips for Using the multisensory Web Map<\/a> <\/o:p><\/span><\/h4>\n

[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_column type=\u00bb4_4″ _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_text _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb custom_margin=\u00bb-11px||||false|false\u00bb]<\/p>\n

Using our map is really easy. You just need to have a look at all the locations marked in it. As you may see, each colour represents a different fieldwork campaign. By clicking on any of the pins, a site card will pop up, including some information, the 360\u00ba video and a picture of the selected S-R combination taken during the acoustic measurements.<\/p>\n

The 360\u00ba videos are hosted on YouTube, but they are linked to the map. As explained above, YouTube works with first-order Ambisonics audio, and renders it into binaural audio in real time. Thus, using headphones is mandatory for having the right spatial audio representation! YouTube also adapts the acoustic experience depending on the angle, so whenever you turn around, the sound will follow your \u201chead\u201d movements. Try to look backwards to feel how sound comes from behind.<\/p>\n

Here’s a brief video-demo to give you a sneak peek<\/p>\n

[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_column type=\u00bb4_4″ _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_video src=\u00bbhttps:\/\/youtu.be\/99wopf2qCF4″ _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][\/et_pb_video][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_column type=\u00bb4_4″ _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_text _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb hover_enabled=\u00bb0″ sticky_enabled=\u00bb0″]<\/p>\n

Note:\u00a0<\/strong>This multisensory map<\/a> has been designed and developed by Daniel Ben\u00edtez and Lidia Alvarez Morales. @Artsoundscapes project.<\/p>\n

[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_column type=\u00bb4_4″ _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_text _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb]<\/p>\n

Without further ado, we happily invite you to immerse yourself in this map, letting you not just visually but sonically explore this selection of rock art sites. We encourage you to explore several rock art sites on the map to compare how the sound experience varies (or stays the same) across them. Just click at the link below and enjoy!<\/p>\n

[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_column type=\u00bb4_4″ _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb][et_pb_code _builder_version=\u00bb4.9.3″ _module_preset=\u00bbdefault\u00bb text_orientation=\u00bbcenter\u00bb module_alignment=\u00bbcenter\u00bb]