This paper aims to present the process of building an evaluation for museum collections management in the context of Portugal and Brazil. This assessment was built during a PhD research in Heritage Studies – Museology, at the Faculty of Arts, University of Porto. The investigation started from a common problem to both countries: the difficulty that museum teams encounter in their daily lives in carrying out all the necessary activities for the management of collections.
The Portuguese and Brazilian museums profiles are mostly small, municipal and with a reduced team. There is a lack of legal and financial autonomy in many museums. National standards are documents that guide with minimum requirements for data collection, with no standards for procedures and use of controlled terminology.
The main objective of the investigation was the creation of an evaluation tool, focused on collections management, based on the Collections Trust’s SPECTRUM standard, aiming to promote continuous improvement and control in collections management through self-regulation.
For the construction of the evaluation, some objectives were defined, as mapping existing procedures, defining their functions, as identifying existence of previous evaluation models, as well developing the evaluation using qualitative and quantitative (as a base using the procedures described in the SPECTRUM), and finally validation of the tool seeking the preservation and promotion of institution´s heritage, dimensioning its human, material and financial resources.
To answer these objectives, the investigation was carried out in three stages. The first stage sought to learn about the practices and strategies of collections management in museums, the second to define the concepts and contexts to be used in the evaluation, through surveys, and the third stage was the construction and validation of the evaluation itself.
First stage – Know the management of collections in museums
In order to know the practices and strategies of Collection Management, a case study was carried out on this theme. A qualitative methodology was chosen, based on interviews, technical visits and documentary collection in museums and guardianship institutions. Eighteen organizations participated in the case study.
Eighteen semi-structured interviews were conducted with professionals from fourteen museological institutions (technicians, area coordinators and managers) and professionals from guardianship institutions (technicians, area coordinator and manager).
To create the data collection guide and to organize all the collected material, an adaptation of the structure of collections management by PAS 197 (British Standards Institute 2009), adopted by Collections Trust in the United Kingdom, through the SPECTRUM standard, was used, as shown in the figure below.
The SPECTRUM standard is international in scope and includes a Portuguese version (Collections Trust 2014; Matos 2012), and has been constantly updated by its users, in discussions at congresses and work groups promoted by the Collections Trust. The Portuguese version (Collections Trust 2014) can be accessed free of charge by any professional, as long as registration is made on the SPECTRUM-PT website.
A 5.0 version, in Portuguese, is being studied to be carried out by a working group of volunteer professionals for translation. In addition to this choice, there are museum professionals in both countries who have undergone training to learn about the standard and some museums have used it to normalize their procedures (Alves, Matos 2018).
From the practices and strategies described above, concepts were defined, helping to construction of the evaluation carried out in second stage.
Second stage – Interpret
For the construction of the evaluation proposal, the concepts of Museology, Information Science and Management were used. The context of museums was studied from local surveys and statistical data from the national statistical institutes of the two countries, in order to observe what the museums do in their daily life to manage their collections and what are the resources they receive from the responsible institutions that guide, assist, qualify and stimulate the development of museums. Also, to observe how are the financial and human resources that museums have available to manage their collections management.
From the defined concepts and developed contexts, an interpretative reading of the collected material was carried out, using as a basis the scheme of figure 1. For coding and sequential analysis of qualitative data, the application MAXQDA 2018v.2 was used.
As a result of the interpretation, it was possible to notice that collection management is mostly done without planning. In both countries, the museums are predominantly small, municipal, with few employees and resources. Most activities are related to access to collections, such as, for example, loans and use of collections (exhibitions). There are no specific models or assessments for collections management to diagnose the status of collections management in any of the studied museums.
Finally, the most used model of evaluation are periodic reports of actions based on goals agreed with the supervising institution, which can present, for example, a goal such as the number of inventoried objects; or performance evaluation of museum professionals (Alves 2020, p. 107).
From these interpretative reading, it was chosen which evaluation model to be used, carried out during the third stage.
Third stage – Construction
The construction of the evaluation started by choosing the format to be used. A reflection was made on which target audience the evaluation would be for, who the main players are and what the applicability is.
From these reflections, it was considered that the evaluation should be from a bottom-up perspective, from the point of view of professionals working in museums. It was discussed what can be solved with a self-assessment carried out by the teams. Should be a tool to assist in the diagnosis of quality, which can be accomplished with a benchmark, pointing out the parameters of excellence. Should assist in the standardizing of procedures. Should be user-friendly, as well accessible in terms of infrastructure and software, with a didactic supporting material for your application.
After a survey on the evaluation models and based on the above criteria, it was decided to build an evaluation instrument in the benchmark format, once allows the use of quantitative and qualitative performance indicators, it is simple to use and apply, does not require a large structure and this model was already applied in the United Kingdom museums (e.g., Collections Trust 2018) and Australia (Arts Tasmania et al. 2016).
From this model, quantitative performance indicators were constructed, which measure practical results, such as the number of objects inventoried in the museum; and qualitative, which measure whether the results reached a standard, which is not quantifiable, such as having published a collection management policy.
The created evaluation was published on a website for validation with peers in the area, museum professionals. To identify who would be the professionals interested in participating in the testing phase, a survey was conducted to look for peers who could validate the tool: 139 professionals were registered (89 in Portugal and 50 in Brazil). All of these professionals received an invitation letter to participate in the discussion. From the online document, they were invited to send suggestions by email and in the online forum done for this matter or to participate in person at the scientific event Collection Management in Museums: reflections on procedures and discussion on an evaluation proposal that was created for discussion.
In total, 40 changes were suggested and validated, which were incorporated into the evaluation, becoming a new version, 1.2. This version received the name RUMO, which in Portuguese it means the direction of a ship, a metaphor for museum professionals who want to seek the path of improvement.
The benchmark performance indicators were selected in order to balance the quantitative and qualitative perspectives. From the 52 performance indicators, 44.2% (23) are qualitative, 34.6% (18) quantitative and 21.2% (11) are quantitative and qualitative.
These indicators are grouped into seven sections. Section 1, Structure and Ethics, gathers performance indicators on organizational structure, sustainability and governance. Themes such as the museum’s mission, collection management policy, statute, internal regulations and sustainability were created to encourage the museum to define who it is, how its collection should be, what its organizational structure (its organic law) and measure if its resources are adequate for maintenance and improvement.
Section 2, Professional, it was created specifically to measure whether human resources are adequate for all activities carried out and planned at the museum.
Section 3, Infrastructure, it was created to measure whether the museum has adequate infrastructure in its buildings and equipment to preserve the collection, whether it has accessibility, security, digital preservation and collection management system.
Sections 4 to 7 were created from the structure of figure 1, based on the SPECTRUM standard (Collections Trust 2014, 2017). Section 4, Collections development, it was created to measure the procedures related to the collections development, as well as if they are supported by the collecting organization´s mission statement and collections management policy.
Section 5, Collections Information, it was created to measure the procedures related to the information that the institution collects, creates and maintains about its collection and / or objects.
Section 6, Collections access: it was created to measure the procedures related to the right, the opportunity or means of finding, use or interpret the collections of the institution.
Section 7, Collections care and preservation, it was created to measure the procedures related to the safeguarding of the institution’s collections.
At the end of each section, there is the calculated average of the values filled in for each action / activity / procedure, automatically presented in the file in Excel format, which can be done by users who choose to use the Word or PDF version. For example, in the case of section 4, there are seven actions / activities / procedures described, and the arithmetic average of these values from points 4.1 to 4.7 should be the result that allows the museum to perceive the qualification in that area (Alves 2020, pp. 216-220).
The final result of the calculated averages is presented in a spider chart, in the Excel application, in order to help the visualization of the area (s) to be improved.
Reflections on the application of the benchmark
The applicability of this assessment is a challenge that follows after the investigation. Making an estimate for completing an assessment, if each point takes 10 minutes between reading, discussing with peers and locating the reports, the complete completion of 52 points should take between 8 and 9 hours, considering that the information and documentation are organized and affordable. If the documentation is not organized, the time may be longer.
Another factor that can hinder a quick completion of the evaluation occurs in the case of museums without financial and administrative autonomy, as they must calculate the estimated time together with their supervising institution, which, in turn, must fill in questions regarding maintenance, human and resources financial statements or provide the reporting data described in the indicators.
Another issue to be considered for this work to be applicable is the need for the assessment to be reviewed and updated at least every five years. And, most importantly, the use of the evaluation proposal in a practical situation. The evaluation has not yet been tested in a daily situation; therefore, it is not known how the tool will be accepted and used in museums’ practice and routine. However, for these issues, the creation of a working group and discussion on the evaluation proposal, which can be online, was thought of as a solution to encourage and assist the museums’ first contact with the evaluation.
In order to create a solution to this issue, during the investigation, a test was carried out to create a working group. Based on a survey of professionals in the field, it was asked what is the best form of communication in the context of creating a discussion network of professionals who work with collections management and museums in Portugal and Brazil? Based on the response of 133 professionals, the social network Facebook and the Forum were chosen as preferred media.
Two topics were created with themes found recurrently during the investigation: (1) the lack of documentation and (2) ambiguous use of the terms inventory and cataloging. With a difference of 15 days, the texts were published in both media (Facebook and Forum), with the intention of creating a discussion. The responses and the visualization of the pages were monitored and it was noticed that with continuous publishing actions, which contain a content curation, it is possible to reach a considerable number of professionals in the area. And, this group of people can form a community of professionals who work with collections management in museums that discuss the theme and, also, the benchmark proposed in the investigation.
Conclusion
In the current context of museums in Portugal and Brazil, providing museum professionals with only the benchmark assessment proposal is not enough for them to be able to use it in order to contribute to the management of the museum. As important as the evaluation, it is the structure and support that professionals need to use the tool. Museums lack more professionals, which puts a strain on current teams with the lack of time to carry out their daily tasks, without the existence of a regular practice of evaluating the collections management.
In this context, it is not enough for the research to propose an evaluation, it is also necessary to propose a structure for application and other forms of support. With that in mind, a toolkit was created, which guides you on how to make the assessment in seven steps, described in the figure below. This toolkit complements the guidelines at the beginning of the benchmark and, in simple language, guides professionals from the formation of the team to make the assessment to the execution of an improvement plan.
The second proposed action is the use of the Collection Management website forum and social media accounts created for this purpose to connect professionals and create a collaborative network for the exchange of experiences, knowledge, clarification of doubts and, for the improvement of the evaluation and constant update. It would also be an opportunity for professionals to strengthen each other, uniting to overcome common challenges and difficulties.
There is a limitation that still needs to be overcome: the awareness of professionals and managers to the use of the created material. Awareness raising solutions can be developed in future research, or they can also be done through the creation of a working group with professionals in the field who will use this evaluation model. It is important to get the support of managers and guardianship institutions, as the working time to carry out the evaluation and planning of improvements has to be considered in the teams’ schedule and daily life.
Conducting a (self) assessment and reflecting on the practices and procedures in a museum is the responsibility that the museum must provide for “in the service of society and its development […]” (Desvallées et al. 2010, p. 57). For this scenario to materialize, it is necessary to support and empower professionals who work in museums. Museum professionals are overloaded with tasks and, often, are only able to react to the necessary actions of daily life, having to respond to a number of activities many times greater than what can be done during working hours. There is a lack of financial and human resources to plan, standardize and also evaluate. Evaluation and support instruments are effective measures to improve the management of museum collections, so that professionals can improve the core of their activities to bring the museum to life.
References
Alves, J.R. (2020) Avaliação para gestão de coleções em museus: uma proposta de indicadores de desempenho com base na norma SPECTRUM. 3º Ciclo em Estudos do Património Doutoramento, Universidade do Porto. [Available at: https://hdl.handle.net/10216/131189 (Access 10 December 2020)].
Alves, J.R., A. Matos (2018) Processo de normalização de procedimentos: estudo em três museus da cidade de São Paulo. Genius Loci – Lugares e Significados | Places and Meanings. Porto: CITCEM – Centro de Investigação Transdisciplinar Cultura, Espaço e Memória. [Available at: https://ler.letras.up.pt/uploads/ficheiros/17076.pdf (Access 10 March 2019)].
Arts Tasmania et al. (2016) ARTS TASMANIA, ACT MUSEUMS & GALLERIES, HISTORY SA, MUSEUM AND ART GALLERY OF THE NORTHERN TERRITORY, MUSEUMS & GALLERIES QUEENSLAND, MUSEUMS & GALLERIES OF NSW, MUSEUMS AUSTRALIA (VICTORIA) & WESTERN AUSTRALIAN MUSEUM. National Standards for Australian Museums and Galleries version 1.5, Melbourne (Australia), The National Standards Taskforce. [Available at: https://www.amaga.org.au/sites/default/files/uploaded-content/field_f_content_file/nsfamg_v1.5_2016.pdf (Access 12 December 2017)].
British Standards Institute (2009) PAS 197:2009 Code of Practice for Cultural Collections Management. British Standards Institute.
Collections Trust (2014) Spectrum 4.0: Padrão para gestão de coleções de museus do Reino Unido, São Paulo, Secretaria de Estado de Cultura; Associação de Amigos do Museu do Café; Pinacoteca do Estado de São Paulo.
Collections Trust (2017) Spectrum 5.0. [Available at: https://collectionstrust.org.uk/spectrum/spectrum-5/ (Accessed: 10 December 2017)].
Collections Trust (2018) Benchmarks in Collection Care for Museums Archives and Libraries: A Selfassessment Checklist 2.1 Collections Trust. [Available at: https://collectionstrust.org.uk/wp-content/uploads/2016/09/Benchmarks-in-Collections-Care-2.1-1.pdf (Access: 12 April 2019)].
Collections Trust, A. Dawson, S. Hillhouse (2011) SPECTRUM 4.0: The UK Museum Collections Management Standard. Collections Trust. [Available at: https://collectionstrust.org.uk/wp-content/uploads/2016/11/spectrum_4_03.pdf (Access 10 January 2017)].
Desvallées, A., F. Mairesse, M.D. Mariemont (2010) Key concepts of museology, Armand Colin; ICOM.
Matos, A.M.R. (2012) SPECTRUM: uma norma de gestão de coleções para os museus portugueses. Doutoramento em Museologia, Universidade do Porto. [Available at: https://hdl.handle.net/10216/67304 (Access at 10 January 2015)].
