Reintegration of interpretative multimedia works back into museum digital repository

Alexey Tikhonov

Multimedia in museum is not just about enriching experience but often the only way to adequately represent object in museum collection. Some objects, even quite common, like books, do not lend themselves easily for presentation as a functional object. Valuable book is usually displayed in a glass case and it simply couldn’t be read that way. About the only way to represent the content of the manuscript adequately would be to use some kind of multimedia representation of its pages, for example IIIF presentation fed to appropriate reader,[1] included in installation or published online.

In other cases, multimedia components provide invaluable context without which the object in question is unreadable or add accessibility features that open objects to people who couldn’t have experienced them otherwise. And while the museums are traditionally object based, at the core it’s the human experience that is preserved and without the means to present that experience museum doesn’t really fulfill its function. All of this makes the task of preserving some of derivative multimedia works at least as important as preserving digital versions of museum objects.

Primary strategy for preserving derivative multimedia works

Multimedia installation is one of the hardest types of objects for long term digital preservation, mostly because it is very technological, and technology is quite volatile. The difference between preserving multimedia object acquired by museum and the one produced by or for museum is that the museum being at the very the initiator of the project can plan ahead before the creation of multimedia and control at least some aspects of it.

The most important strategy is keeping the function and technology separated: technology becomes obsolete, the function isn’t. For example: we want our visitors to be able to read the book in our museum in perpetuity but the means to achieve it might change from photocopies to digital reader to some kind of direct neuro interface or something we can’t even think of right now. That actually provides us with the most important (and sometimes the only) part of multimedia installation that we want to preserve: documentation and functional description. Having this in sufficient detail we can recreate the experience described with whatever technological tools we have at hand at some point in the future. However, that leaves out content produced for multimedia.  

Creative (interpretative) and algorithmical derivatives

Keeping in mind separation of technology and function it’s much easier to prepare content for long term preservation than to preserve the end product in its entirety. Another argument for that is that the data and new content produced for specific project most likely will add some insight even outside the context of that project, so it makes a lot of sense to keep new content and its connections at least linked to museum objects in CMS, or even in the same digital repository if it allows it.

Additional content (text/images/sounds) for ROSPHOTO web
Figure 1 – Additional content (text/images/sounds) for ROSPHOTO web long read format story is stored in the same CMS/DAM as museum objects and connections between them are kept for future use

While doing that we should avoid data duplication and to do that it helps to categorize two main types of multimedia content derived from museum objects. One is creative or interpretative, for example a video of an actor made look like a portrait drawn by Andy Warhol commenting in sign language over an actual portrait using AR technology.[2] This kind of media object is separated from original object but the connection to original while obvious in this case should be kept. On the other hand, if the multimedia uses low resolution image that is just downscaled version of main object digital representation, we don’t need to store it separately as it’s algorithmically possible to recreate it from master.

Linked data and multimedia produced by museum

Another step to achieve reusability of multimedia content and simplification of long term preservation of interpretative multimedia projects is pushing multimedia projects to consume linked data[3] and conversely publish data used by multimedia as linked data for it to use. Even if data has to be baked in multimedia installation for technological reasons (it’s not connected by design or data access time is critical) relying on standards and APIs such as linked.art[4] or IIIF would mean there is much less for us to keep technologically specific to the current project. If the connection can be kept and multimedia installation can actually use museum CMS as data source via API there is an additional benefit of keeping data always fresh, as any corrections in data would be reflected in the final product automatically.

This suggests another important aspect of strategy for planning multimedia projects – keeping technological control over the project. Of course, it’s not always possible, especially as complex multimedia installations are usually created by specialized companies, but the data structures can be provided as requirements or at least recommendations for multimedia creators.

Source code and functional description

Many multimedia installations, especially interactive ones contain executable code for devices they used. Long term preservation of software is very difficult task, at some point usually involves creating virtualized software environment. Having source code available can greatly simplify preservation strategy[5] allowing to migrate the software to new software environments.

Moreover, well documented code is in itself specialized functional description of the resulting object and can be treated as an extension to primary functional documentation of multimedia. That is another facet of technological control importance for long term preservation. As with almost all digital preservation strategies the early planning is the key here. Museums that acquire completed multimedia artworks don’t have the benefit of the ability to specify requirements for source code availability and API/linked data usage in the project, which is much easier in the early planning stages of project.

Multimedia project using museum CMS
Figure 2 – Multimedia project using museum CMS/Digital repository as data source and providing documentation and source code as additional objects for museum repository
Case study

In 2019 ROSPHOTO museum held St. Petersburg in 19th Century Photography exhibition[6] that included selection of stereo photographs. While stereo photographs themselves are easy to display, they need specialized equipment to achieve stereo effect, namely a stereoscope. While stereoscope is a simple enough tool it either displays one photograph only which needs to be changed by user, or has complex mechanism to switch photographs, and even in this case, the number of photographs is limited.

Stereoscope and stereophotographs replica
Figure 3 – Stereoscope and stereophotographs replica

Hybrid approach was chosen to solve that difficulty – there were modern replica stereoscopes with reprinted stereophotographs that visitors could handle and change themselves, but stereophotographs of better quality are glass based, and glass replicas would be prone to break, stain and complicate experience. To overcome these limitations we created electronic stereoscope to display more stereophotographs and in better quality.  While we had high resolution digital copies in our digital repository (in house product combining CMS and DAM functions) they aren’t readily suitable for use in electronic stereoscope – additional text on prints would distract from stereo effect, grayscale cards were unneeded and sometimes slight adjustments were needed to improve stereo effect.

Therefore, for each object there was produced another digital version specifically for digital stereoscope use. However, this version was not specific to actual hardware we used, and would be useful in any future stereoscope implementation, so while it was not completely algorithmically derivative, it was perfectly reusable and stored as an additional derivative alongside the main digital master. Hardware was based on widely used Raspberry PI platform that allowed usage of Python language for controlling software. Python is quite human readable, so the source code could be considered in a way an additional functional documentation in itself. The system was always connected via API to museum CMS and could download all the descriptive and image data at any point via network command.

Using museum CMS as a source for downloadable content for digital stereoscope
Figure 4 – Using museum CMS as a source for downloadable content for digital stereoscope allowed ROSPHOTO not only to make content corrections while the exhibition was open but even change objects on display

By early planning ahead we chose implementation where hardware can be easily changed and function (ability to see photographs with stereo effect) was kept separate. All the creative derivatives were kept inside the museum digital repository, maintaining connection to original objects. All the data that installation used was acquired via API from museum CMS. And finally, complete technological control was achieved, which was easy in this case because project was realized internally.

This case is quite simple, other complex projects probably wouldn’t let themselves as easily to the implementation of these strategies, but it demonstrates each of the principles discussed here:

1. Plan for digital preservation at the early stage of multimedia creation
2. Document the effect and experience that multimedia is supposed to achieve
3. Keep technology and function separate
4. Keep created and derived content connected to the museum objects
5. Make multimedia installation use data from museum as linked data via API’s
6. Maintain technological control over created multimedia Keep source code of software used wherever possible.


[1] International Image Interoperability Framework. [Available at: https://iiif.io/ (Accessed: 30 April 2021)].

[2] Unmute Art – The True Essence of Warhol. [Available at: https://orpheogroup.com/us/unmute-art/ (Accessed: 30 April 2021)].

[3] Berners-Lee, T. (2006) Linked Data [Available at: https://linked.art (Accessed: 30 April 2021)].

[4] Linked Art [Available at: https://www.w3.org/DesignIssues/LinkedData.html  (Accessed: 30 April 2021)].

[5] Rechert, K., P. Falcão, T. Ensom (2016) Introduction to an emulation-based preservation strategy for software-based artworks. [Available at: https://www.researchgate.net/publication/311614858_Introduction_to_an_emulation-based_preservation_strategy_for_software-based_artworks (Accessed: 30 April 2021)].

[6] ROSPHOTO (2019) St. Petersburg in 19th Century Photography. [Available at: https://rosphoto.org/events/petersburg-in-19th-century-photography/ (Accessed: 30 April 2021)].

Alexey Tikhonov

Alexey Tikhonov

IT projects director at the State Russian Museum and Exhibition Centre ROSPHOTO in St.Petersburg, Russia. He specializes in digital preservation, multimedia and providing access to digital museum objects, is the main developer of CMS/DAM museum information system “Eidotheque”, lead developer of several award-winning multimedia projects. His main areas of interest as a member of The Council for Museum Digital Development under ICOM Russia are development of digital preservation and open access methodologies.