Mediaflex: A Case Study of its Use in a National AudioVisual Archive

Introduction

As the custodian of Australia’s ‘living’ archive, The National Film and Sound Archives of Australia collects, preserves, manages, and shares over 2.3 million audiovisual assets. Broadly, an audiovisual archive is an organisation that collects, preserves, and makes accessible moving image and recorded sound heritage. This can include any or all of film, television, radio, and recorded sound formats as well as related material such as photographs, scripts, lobby cards, costumes and memorabilia. The creation of moving image and recorded sound heritage is just over 100 years old, but the dominance of the audiovisual media in modern life means that audiovisual archives represent significant educational, historical, and cultural resources. As the custodian of Australia’s ‘living’ archive, The National Film and Sound Archives of Australia collects, preserves, manages, and shares over 2.3 million audiovisual assets.

Today, the focus of audiovisual archives is shifting towards a digital future. New collection items are increasingly born digital in nature. Digitisation is being employed as a method by which works currently preserved on fragile analogue technologies such as tape can be more effectively preserved in the digital domain. Clients seek to get access to collections via means of digital delivery. Consequently, audiovisual archives need to either enhance existing collection management systems or install new systems altogether.

In the latter part of the twentieth century, when audiovisual archives used computer systems to manage solely analogue items, audiovisual archives with heritage responsibilities (which could be at a national level, a state level or even a specific area of interest) shared some system needs with broadcasters and their own archives (primarily operated for commercial purposes). But overall, there were too many areas where the respective business needs differed and so, the two types of organisations used systems tailored to their own specific needs.

Recently, the journey that the National Film and Sound Archive (NFSA) of Australia embarked on in moving from its legacy collection management system (with a strong orientation towards analogue material) to Mediaflex, a system capable of managing all of its collection material, both analogue and digital, illustrates that the various types of audiovisual archives nowadays have much more in common when it comes to their system needs.

This paper describes what the NFSA did and how it came to realise that it could use a system typically used by broadcasters to meet the business needs of an audiovisual archive in the twenty first century and thereby be well placed for managing both an expanding digital collection and demand for greater access via digital techniques.

The National Film and Sound Archive

The NFSA has a collection of over 1,600,000 items which includes moving image, recorded sound and associated documents and artefacts of cultural significance. Works range from commercial release documentaries, feature films and sound recordings; websites relevant to the audiovisual industry; newsreels and broadcasts; television and radio productions of all genres including advertisements; independently produced works; home movies on all formats; international productions which have influenced and been experienced by Australians; and unpublished works including oral histories and early field and music performance recordings of particular cultural or historic interest in the audiovisual industry context.

Since 1984, the NFSA had used various iterations of an in-house developed collection management system to keep track of its collection. The last such system integrated all of the NFSA’s collection management requirements for both moving image and recorded sound material into a single system. Business needs met included acquisition management, accessioning, cataloguing, rights management, and loans. The system was designed to manage analogue collection items (assets) and performed this task very efficiently.

NFSA staff were very happy with this system, but in recent times, production and distribution in the audiovisual industries began increasingly to use digital methods. The initial impact on the NFSA was that it started to receive into its collection digital material for which there was no analogue copy. This could still be in a physical format such as digital tape, but it could also be in a file format received electronically. In addition, the NFSA commenced its own digitisation programmes with the goal of preserving content housed on inherently unstable media (eg tape) in a digital format.

The NFSA’s collection management system was modified to cater for digital formats, but, ultimately, the system proved inadequate for the task of managing a rapidly growing digital collection. This caused the NFSA to start considering alternatives to the system. The early part of this century had seen the emergence of a number of Digital Asset Management (DAM) and Media Asset Management (MAM) systems and NFSA staff were very impressed with these systems’ capabilities for managing digital files

However, the NFSA realised two important things about its collection which influenced its thinking in what to do about its collection management system needs. First, the NFSA is still actively collecting analogue items spanning about a century of moving image and recorded sound production.

Second, the NFSA is increasingly using the digitisation of analogue items for preservation outcomes and for improving accessibility. At current digitisation rates, it would take more than 30 years to digitise the entire existing analogue collection. But, the NFSA, in its role of protector of the nation’s audiovisual heritage, cannot consign analogue masters to oblivion. Even though newly created digital masters may constitute a lossless transfer of content from the analogue master, the NFSA is still responsible for looking after the analogue master both as insurance against loss of the digital copy and against the possibility that it may be re-used or re-copied at a later date.

Taking these considerations into account, the NFSA concluded that whilst the rate of adding new analogue items is likely to decline in the near future it is clear that managing analogue items is going to be an important part of the NFSA’ operations for a long time yet. Thus, to meet the NFSA’s core business needs in the future, it was critical to move to a system capable of managing both analogue and digital items.

What did the NFSA do?

In 2005, the NFSA identified three options for going forward:

1. Build a new system from scratch which managed both analogue and digital items in one system.
2. Retain the existing system to handle analogue items, acquire a DAM/MAM to handle digital items and then integrate the two systems.
3. Acquire a DAM/MAM to handle both analogue and digital items.

The NFSA considered that the first option would be too expensive and that it was better to focus on the other two options. The NFSA then conducted a review of DAM & MAM products available in the market place to determine the viability of the remaining options. The conclusion was that there were DAM/MAM products available that could handle both analogue and digital items in one system and that option 3 was probably the most cost effective. But, market testing was the only way to confirm this and so it was decided to leave the question open. Therefore, in the NFSA’s tender documents, the requirements were written in such a way that a vendor could present a proposal based on option 2.

Amongst all the bids received in the NFSA’s public tender process conducted during 2006-2007, no vendor proposed a solution based on option 2. Some of the bids proposed products that only catered to managing digital items, which meant the NFSA alone would have faced the task of integrating a MAM with its aging analogue asset management system. The NFSA judged this to be a task well beyond its resources. Thus, option 2 received no further consideration in the evaluation process and the NFSA focussed on a solution that could address option 3.

After exhaustive deliberations, TransMedia Dynamics (TMD) won the tender for a system to replace the NFSA’s legacy collection management system. In its tender, TMD proposed using its MAM product Mediaflex. The founders of TMD had recognised that many organisations lacked the resources to digitise all of their analogue material any time soon and that for organisations with large collections, management of analogue items would remain important for many years. Hence, they decided to design Mediaflex from the ground up to manage both digital and analogue items. This impressed the NFSA. Many other systems that the NFSA looked at were good at handling digital items, but either partially addressed the needs of managing analogue items or completely ignored them.

The NFSA was also impressed with the data structure that Mediaflex employed for both intellectual and physical description. The multi layer approach that Mediaflex uses was similar in concept to the structure of the NFSA’s own collection management system which had been developed specifically for the NFSA. But, of particular note was the fact that Mediaflex had an extra layer and a more flexible means by which users could describe collection material using the layers. This was especially evident in how Mediaflex handled versions of titles (eg for works that may have some or all of an original release version, a director’s cut, a TV release version etc). The Mediaflex approach was a cleaner and more elegant solution to a problem that had vexed NFSA staff for a long time with their existing system.

Thus, taking into account the above mentioned prime considerations, the NFSA saw in Mediaflex an excellent fit for its business needs.

TMD’s Solution

TMD’s implementation of Mediaflex included some enhancements and additional features necessary to support a Government funded national audiovisual archive together with the migration of data from the NFSA’s legacy system. The NFSA’s role as a national heritage collection body posed many challenges for TMD, the like of which it had not experienced before with broadcast clients.  Some of the key issues addressed included –

• Audio archiving. Mediaflex needed extensive enhancement to meet the data needs of audio archivists at the NFSA. The NFSA was the first TMD client with a major responsibility for archiving of audio material.
• Intellectual and physical description. Mediaflex needed to be modified to include a much wider range of intellectual and physical description than is required by a typical broadcast client. This is largely because of the NFSA’s responsibilities as a national heritage repository. Often the material in the NFSA’s collection is unique to the NFSA and needs to be described to a depth of detail such that clients can identify the content in order to be able to use it.
• Acquisitions. As a national heritage collecting body, it is essential for the NFSA to capture a substantial amount of information for any acquisitions its makes (generally by donation). As the NFSA often acquires material in the form of collections of disparate formats of media, it needs to track the collection from the time of initial negotiation through dispatch from donor, initial receipt and registration by NFSA, initial storage of collection, accessioning of high priority items, re-canning and permanent storage. NFSA staff require at all times a holistic view of an acquisition (which may comprise thousands of reels of film, for example) together with accurate information on where the individual elements are in the NFSA’s processing of the collection. Parts of any one acquisition may be stored in totally different vaults (eg colour film in cold storage, videotape in cool storage), but Mediaflex enables staff to manage the acquisition as a whole without requiring that all items be accessioned. To support this required a significant enhancement of Mediaflex. As valuable input, TMD used the information gleaned from a significant Acquisitions business process re-engineering exercise which the NFSA conducted as part of its preparations for adopting Mediaflex.
• Loans. Unlike just about all other TMD clients, the NFSA loans material to external clients. Hence, Mediaflex needed to be modified to provide the NFSA with the tools to manage the loans process.
• Rights. The way that the NFSA looks at rights and rights management is quite different to TMD’s broadcast clients which had been used as the basis of Mediaflex’s rights management functionality. Now, Mediaflex can be configured to support either view of rights management. Importantly, for the NFSA, they can use Mediaflex’s workflow capabilities in the rights management process.
• Searching by the public. The NFSA provides a Google style public access interface for searching its collection known as Search The Collection (STC) which uses a specialised high performance free text retrieval system to support search requests. The system uses a subset of the data stored in the NFSA’s implementation of Mediaflex and furthermore, the structured data in Mediaflex has been converted into unstructured data. The conversion process requires that the hierarchical structure of the data be re-formatted and “flattened” for import into the search engine. It is this unstructured data which is then indexed to support free text searching by the STC. The data in the STC needs to be regularly refreshed with exports of data from Mediaflex so that the public has timely access to information about the NFSA’s collection.
• Preservation. To better reflect NFSA preservation workflows, Mediaflex’s functionality has been enhanced so that a job can consist of multiple tasks. New NFSA specific tasks have been created including a task for accessioning and a task for digital ingest.
• Rack Numbers. The legacy system used an automatic rack numbering system whilst Mediaflex was designed for manual assignment of rack numbers. A core part of the project was that the NFSA move away from the automatic generation of rack numbers. Thus, newly accessioned items in Mediaflex receive a 7 digit unique number for each carrier in the item. When the item is stored in its permanent vault, the location information is recorded and uploaded into Mediaflex. Items accessioned in the legacy system still retain their rack number as an identifier, but since cutover to Mediaflex, no new rack numbers have been issued.
• Data migration. The legacy system embraced a complex data model in its own right. The extensive enhancement when combined with pre-existing functionality in Mediaflex necessitated a complex data model.  The result was a migration process that involved over 500 data fields.  Other areas of complexity included migrating Acquisition records, Loans data, movement history, and Rights information.
• Authority files. Use of authority files was a major component of the NFSA’s legacy system especially for fields where names were used (eg Director, Cast, Producer, Borrower etc). This is important in the NFSA’s national heritage management responsibilities – for example, there are two Australian film directors called George Miller and it is critical to the NFSA to be able to distinguish between the works of each one. To support such functionality meant not only customising Mediaflex, but it also constituted a data migration challenge.
• Uniform Titles. So that Mediaflex could provide what archivists call “a Uniform Title”, Mediaflex needed modification in its functionality to support what is now called Versioning for Titles.
• Digitisation workflows.  Although Mediaflex already supported the third party hierarchical storage management product DIVArchive, some effort was required so that DIVArchive could be fully integrated into the NFSA’s audio, video and documentation (eg photographs, posters, lobby cards) digitisation workflows (including hardware and software that NFSA preservation staff use in their daily processes). The use of DivArchive entailed a totally different way of working with digital content than had previously been the case at the NFSA. The end result being that Mediaflex supports all stages of the digital preservation process from the movement of the analogue media from its permanent storage location through to the writing of preservation files on LTO tape for storage in the Digital repository.
• Metadata wrapping. Mediaflex needed to be adapted to support the wrapping of metadata into Preservation items in the following formats – audio (BWF), documentation (TIFF) and video (JPEG2000). For video, support was required for duping copies in MPEG2 50I format.
• Support for mass migration methodologies. Mediaflex needed to be adapted to support the mass migration of content from analogue formats such as videotape to digital formats. Given the short lifespan of tape, it is imperative to use systems like the Samma system to perform mass migration projects. Mediaflex was integrated with the NFSA’s Samma devices and the associated workflows, including extracting data from MXF files into Mediaflex item records.
• Digital checksums. To support digital preservation, Mediaflex is being enhanced to support the use of digital checksums in the automated validation of the integrity of digital content. Also, Mediaflex currently supports the tracking of 84 different file types.
• Automatic harvesting of metadata. To meet the NFSA’s digital content management needs, Mediaflex needed enhancement to support the automatic harvesting of metadata from all digital sources. For example, the digital preservation process for an audiotape generates a lot of metadata in the WAV file created. Mediaflex can now capture this information and store it in the individual item record, which, in turn means that the data is searchable and can be used in reports.

Major benefits for the NFSA

The implementation of Mediaflex at the NFSA spanned two years and has proved to be a very complex project. Now that the system is in production, it is possible to articulate some of the major benefits for the NFSA –

• Analogue and digital content is managed within a single system. For any one title, the NFSA can easily identify what items they hold, what format they are and how accessible they are.
• The NFSA has substantially better facilities for managing the preservation, storage and retrieval of digital content, both for born digital items and for items created during the NFSA’s digitisation programs. Using Mediaflex to support digitisation work has led to greater efficiencies in processing and an improvement in productivity.
• The NFSA is now able to utilise the benefits of a workflow based system. Whilst the NFSA did not specifically seek this in its tender process, the fact that this is a fundamental part of the design of Mediaflex ensures that the NFSA will be able to use workflow based processing to manage and control many of its main collection management tasks.
• Curatorial work on the collection is enhanced by being able to access digital content from the desktop.
• The NFSA now has a significantly improved acquisitions process for tracking what was received and what happened to the material received.
• Reporting has been streamlined with the move away from a vast number of “canned” reports that people rarely used to a system where users have complete control over what is reported on and when it is reported.
• Mediaflex can be used as a management information tool. For example, to support specific acquisition programs whereby gaps in the NFSA collection are identified after analysis of reports extracted from Mediaflex. Another example is to use Mediaflex to generate reports based on specific parameters as input to targeted preservation programs. Finally, using Mediaflex as input into the process of valuing the NFSA collection – all Australian Government funded heritage collecting organisations are required to report annually on the value of their collections and whether that value has changed.
• Incorporation of the enhancements into the base Mediaflex product. They are fully maintained by TMD and are available for other Mediaflex customers to deploy.

Conclusion

In this paper we’ve described how a system originally installed in broadcast environments became installed at an audiovisual heritage collecting body.  Whilst broadcasters need Mediaflex’s facilities for major business needs such as pre-production work and playout, things which are of little interest to an organisation like the NFSA, both types of organisations look at digital content in a similar way. Both need to know at any point in time where a digital file is, who is using it, what are they doing with it and how accessible is it to users in general. Although the end purpose may differ, it is still critical to track the file wherever it is within an organisation.

Conversely, there are features in Mediaflex that the NFSA uses which no broadcaster would ever use. But, it is this common ground shared in the managing of digital content that has enabled an audiovisual archive like the NFSA to consider adopting a MAM system like Mediaflex to support its core business. The various facilities for managing and delivering digital content that Mediaflex possesses will be of great value as the NFSA moves increasingly into a digital future.

The fact that many broadcasters are Mediaflex customers provides the NFSA with the confidence that it has a system that will grow as new business needs emerge and that it has avoided the situation of being stuck with a tailor made system, which would prove costly to maintain and enhance.

The Mediaflex implementation project has been a major investment for the NFSA, both financially and in terms of human resources. With Mediaflex, the NFSA now has a system able to cater for all the content it collects and provide a platform for access to that content. The NFSA’s new collection management system leaves it well positioned for a future which will see both collection growth in digital formats and ever greater access to the collection via digital means.

About NFSA

The primary responsibility of the NFSA is to manage the national audiovisual collection and to build, preserve, interpret and make accessible that collection. The NFSA plays a key role in the development of Australia’s audiovisual cultural heritage and places the highest priority on ensuring that all Australians are able to experience and enjoy the accessible treasures held in the collection, from the 19th century beginnings to the present. For additional information on the NFSA please visit www.nfsa.gov.au.