The Museum Data Service (MDS) solves a problem first identified in 1888, when curators from around the country discussed the need for a ‘compendious index’ of their museums’ collections (Roberts, 1992, p. 2). More than a century later, despite the widespread digitisation of collection catalogues (Parry, 2007), it is still not possible to search, let alone work, across the nation’s museum holdings.
Even though it only launched in September 2024, MDS has made strides towards addressing the challenges researchers have long faced when trying to work with museum collections data: bringing together objects records from different museums; mapping diverse data structures to a usable standard; establishing persistent identifiers for all records; and enabling re-use of data through APIs.
Bringing together object records from different museums
Based on survey data from 153 leading museums (Gosling et al., 2022), information from the supplier of the widely used Modes software (pers. comm.), and our assessment of a similar number of very small museums, we estimate there may be 80 million object records in the databases of the UK’s 1,700 Accredited museums. Perhaps fewer than half of these are available online, spread across hundreds of websites.
Our target is to bring half of Accredited museums on board within five years, and all within ten. MDS went live in September 2024 and already has some 4 million object records. We are also unlocking the enormous research potential of the collection summaries that all Accredited museums are required to write (but mostly do not publish), bringing them together as a single, searchable dataset.
Why is this needed?
A 2008 investigation into the needs of researchers wanting to discover physical museum objects concluded that, ‘above all’, they needed ‘online access to the records in museum and collection databases to be provided as quickly as possible, whatever the perceived imperfections or gaps in the records.’ This was ‘an essential first step towards improving discovery services that will benefit researchers’ (Research Information Network, 2008, p. 5).
Similarly, UKRI’s infrastructure roadmap identified that ‘building a new digital repository or ‘aggregator’ would enable connectivity between museum collections data (and metadata about digital assets) already online but currently siloed.’ (UKRI, 2019, p. 83) MDS exceeds this ambition, aiming to connect all object records from all UK museums, not just the ones currently online.
Mapping diverse data structures to a usable standard
MDS has no prescribed metadata schema for incoming data. This is partly to lower a known barrier to entry, but also avoids a key deficiency of ‘traditional’ cultural heritage aggregators that standardise diverse source data, making assumptions that might suit one end purpose, but not others. In MDS, the raw source data remains available to those who need it that way. For those who want to search across datasets, we have taken the pragmatic decision to map the diverse field names of our source records to the 500 or so ‘units of information’ described in Spectrum, the UK museum collections management standard. Our Spectrum mappings are intended to be a stepping stone to other metadata schemas. There is a mapping from Spectrum to the widely-used LIDO schema, while Art UK is already using a similar mapping to bring MDS records onto its platform. As further mappings are completed in collaboration with data users (e.g. to Darwin Core, the Linked Art Data Model, and the Europeana Data Model) we will publish these.
Why is this needed?
The AHRC research programme Towards a National Collection (TaNC) commissioned a user consultation to identify researchers’ needs and requirements for a future UK digital collections infrastructure. The report notes ‘much discussion about connecting collections, interoperability and open standards’. Crucially, it identified this pressing need to bring diverse datasets together more effectively: ‘So 90% of what I do is bringing together data from sources that haven’t been brought together before in formats that have no relation to each other. But they’re completely incompatible. So the idea of bringing everything together in one place in one standard format will basically lower the barrier to entry, because a lot of the very difficult part of the data analysis is analysing things and … shifting the data structures in such a way that they then make sense together.’ (Bailey-Ross et al., 2024, p.19)
Persistent identifiers
Assigning globally unique and persistent identifiers (PIDs) is the very first of the FAIR Principles for findable, accessible, interoperable and reusable data that MDS follows: this ‘is arguably the most important because it will be hard to achieve other aspects of FAIR without [PIDs]’. MDS mints a PID that meets the FAIR data requirement for each object record held. PIDs for those records published via the public website follow the format shown in this example: https://museumdata.uk/objects/926fbbc9-7f7d-393d-8b2a-f947800d5c2e.
Why is this needed?
TaNC’s digital audit asked the 230 leading cultural heritage institutions included in the survey whether they ‘made use of external identifiers to help minimise the problem of broken links. Only 14 respondents (6%) said they did, with a further 33 (14%) saying it was pending’ (Gosling et al., 2022, p. 3). Also identifying low levels of PID awareness and use by cultural heritage organisations, TaNC’s Persistent Identifiers project made several recommendations, but noted that these ‘may still be beyond the reach of smaller organisations with little or no technical capacity, and so shared infrastructure approaches in support of such organisations should be explored’ (Kotarski et al., 2022, p. 17).
Enabling re-use of data through APIs
The MDS website offers researchers and other users two ways to work with collections data. If an object search produces fewer than 1,000 results, key fields from the relevant records can be downloaded as a convenient CSV reference file. Each full record (which might be anything from a handful of fields to several hundred) can be consulted online via its PID. For more records, and all the publicly available fields of those records, an API token can be requested via a simple form and used as documented on Swagger. As with the CSV downloads, the search interface is used to define the dataset returned by the token.
Why is this needed?
Another TaNC Foundation project, Heritage Connector, used various AI and linked open data technologies to generate links between different datasets. As the final report notes: ‘The availability of museum collection catalogues and other data sources as well-documented [APIs] significantly speeded up the project’s technical work. Indeed, the project’s source datasets (V&A collection, SMG collection, Wikidata, etc.) were in part selected because access to data was not an issue with them, though it would likely have been in most other instances.’ (Winters et al., 2022, p. 14) TaNC’s digital audit also asked the 230 participating institutions whether they had ‘an API that allows others to make use of your online collections’. Only 21% did, with a further 16% saying it was pending. Almost half said they did not have an API; the rest did not know. (Gosling et al., 2022, p. 3). Again, confronting this need, MDS provides an API, enabling a connectivity, openness and sharing of data for new forms of research across datasets.
Sources
Bailey-Ross, C., Burgess, E., & Papageorgiou, P. (2024). User Research: UK Gallery, Library, Archive and Museum (GLAM) Digital Collections Infrastructure.
Gosling, K., McKenna, G. and Cooper, A. (2022). Digital Collections Audit.
Kotarski, R., Kirby, J., Madden, F., Mitchell, L., Padfield, J., Page, R., Palmer, R., and Woodburn, M. (2022). Persistent Identifiers as IRO Infrastructure.
Parry, R. (2007). Recoding the Museum: digital heritage and the technologies of change. Abingdon: Routledge.
Research Information Network (2008). Discovering physical objects: Meeting researchers’ needs.
Roberts, D. A., Sharing the Information Resources of Museums. Cambridge: MDA.
UKRI (2019). The UK’s research and innovation infrastructure: opportunities to grow our capability.
Winters, J., Stack, J., Dutia, K., Unwin, J., Lewis, R., Palmer, R., and Wolff, A. (2022). Heritage Connector: A Towards a National Collection Foundation Project Final Report.