KB Research

Research at the National Library of the Netherlands

Month: November 2014

Positive feedback for Jpylyzer

Last week the Digital Preservation Coalition hosted the Digital Preservation Awards 2014 at the Wellcome Trust in London. Thirteen projects in four categories were shortlisted. One of the finalists for the OPF Award for Research and Innovation was our own Jpylyzer project. Although the award did not go to Johan van der Knijff’s JP2 validation and extraction tool, the voters did leave some positive comments that we would like to share.

This is what our peers said about Jplylyzer

  • Deciding first and second place was a close call for this category. Jpylyzer is narrowly ahead.
  • A much-needed tool — eminently practical and useful.
  • It is a key tool for us in the validation of JP2 files.
  • Good project which allows people to deal with large volumes of data.
  • A practical tool that addresses a practical problem with good uptake.
  • It’s nice to see tools that do jobs really well. This is one of them and helps content managers demystify a complex format.
  • This project offers a straightforward solution to a problem.
  • Demonstrates high level of innovation with focus on value to the community.
  • Important to remember that the file handling challenges continue and JHOVE wasn’t a magic bullet.
  • A thorough research proposal which resulted in a practical use case for the Royal Library of the Netherlands. Encouraging to see wider adoption of Jpylyzer across the digital preservation community.
  • A very useful tool, simple to use and meeting a need. What’s more, it’s free.
  • I know Jpylyzer has been an incredibly useful tool for a number of digital preservation initiatives. While I’m not familiar with it personally a number of users have commented to me on its usefulness.
Johan van der Knijff with William Kilbride

Johan van der Knijff with William Kilbride

 

Succeed technical workshop on the interoperability of digitisation platforms

Succeed Interoperability Workshop
2 October 2014, National library of the Netherlands, The Hague

Speaking the same language is one thing, understanding what the other is saying is another… – Carl Wilson at the Succeed technical workshop on the interoperability of digitisation platforms.

Interoperability is a term widely used to describe the ability of making systems and organisations work together (inter-operate). However, interoperability is not just about the technical requirements for information exchange. In a broader definition, it also includes social, political, and organisational factors that impact system to system performance and is related to questions of (commercial) power and market dominance (See http://en.wikipedia.org/wiki/Interoperability).

On 2 October 2014, the Succeed project organised a technical workshop on the interoperability of digitisation platforms at the National library of the Netherlands in The Hague. 19 researchers, librarians, and computer scientists from several European countries participated in the workshop (see SUCCEED Interoperability Workshop_Participants). In preparation of the workshop, the Succeed project team asked participants to fill out a questionnaire containing several questions on the topic of interoperability. The questionnaire was filled out by 12 participants; the results were presented during the workshop. The programme included a number of presentations and several interactive sessions to come to a shared view on what interoperability is about, what are the main issues and barriers to be dealt with, and how we should approach these.

The main goals of the workshop were:

  1. Establishing a baseline for interoperability based on the questionnaire and presentations of the participants
  2. Formulating a common statement on the value of interoperability
  3. Defining the ideal situation with regard to interoperability
  4. Identifying the most important barriers
  5. Formulating an agenda

1. Baseline

Presentation by Carl Wilson

To establish a baseline (what is interoperability and what is its current status in relation to digitisation platforms), our programme included a number of presentations. We invited Carl Wilson of the Open Preservation Foundation (previously the Open Planets Foundation)  for the opening speech. He set the scene by sharing a number of historical examples (in IT and beyond) of interoperability issues. Carl made clear that interoperability in IT has many dimensions:

  1. Technical dimensions
    Within the technical domain, two types of interoperability can be discerned, i.e.:
    Syntactical interoperability (aligning metadata formats); “speaking the same language”, and Semantical interoperability; “understanding each other
  2. Organizational /Political dimensions
  3. Legal (IPR) dimensions
  4. Financial dimensions

When approaching operability issues, it might help to take into account these basic rules:

  • Simplicity
  • Standards
  • Clarity
  • Test early (automated testing, virtualisation)
  • Test often

Finally, Carl stressed that the importance of interoperability will further increase with the rise of the Internet of Things, as it involves more frequent information exchange between more and more devices.

The Succeed Interoperability platform

After Carl Wilson’s introductory speech, Enrique Molla from the University of Alicante (UA is project leader of the Succeed project) presented the Succeed Interoperability framework, which allows users to test and combine a number of digitisation tools. The tools are made available as web services by a number of different providers, which allows the user to try them out online without having to install any of these tools locally. The Succeed project met with a number of interoperability related issues when developing the platform. For instance, the web services have a number of different suppliers; some of them are not maintaining their services. Moreover, the providers of the web services often have commercial interests, which means that they impose limits such as a maximum number of users of pages tested through the tools.

Presentations by participants

After the demonstration of the Succeed Interoperability platform, the floor was open for the other participants, many of whom had prepared a presentation about their own project and their experience with issues of interoperability.

Bert Lemmens presented the first results of the Preforma Pre-Commercial Procurement project (running January 2014 to December 2017). A survey performed by the project made clear that (technically) open formats are in many cases not the same as libre/ open source formats. Moreover, even when standard formats are used across different projects, they are often implemented in multiple ways. And finally, when a project or institution has found their technically appropriate format, they may often find that limited support is available on how to adopt the format.

Gustavo Candela Romero gave an overview of the services provided by the Biblioteca Virtual Miguel de Cervantes (BVMC) The BVMC developed their service oriented architecture with the purpose of facilitating online access to Hispanic Culture. The BVMC offers their data as OAI-PMH, allowing other institutions or researchers to harvest their content. Moreover, the BVMC is working towards publishing their resources in RDF and making it available through a SPARQL Endpoint.

Alastair Dunning and Pavel Kats explained how Europeana and The European Library are working towards a shared storage system for aggregators with shared tools for the ingestion and mapping process. This will have practical and financial benefits, as shared tools will reduce workflow complexity, are easier to sustain and, finally, cheaper.

Clara Martínez Cantón presented the work of the Digital Humanities Innovation Lab (LINHD), the research centre on Digital Humanities at the National Distance Education University (UNED) in Spain. The LINHD encourages researchers to make use of Linked Data. Clara showed the advantages of using Linked Data in a number of research projects related to metrical repertoires. In these projects, a number of interoperability issues (such as a variety of structures of the data, different systems used, and variation in the levels of access) were by-passed by making use of a Linked Data Model.

Marc Kemps-Snijders made clear how the Meertens Institute strives to make collections and technological advances available to the research community and the general public by providing technical support and developing applications. Moreover, the Meertens Institute is involved in a number of projects related to interoperability, such as Nederlab and CLARIN.

Menzo Windhouwer further elaborated on the projects deployed by CLARIN (Common Language Resources and Technology Infrastructure). CLARIN is a European collaborative effort to create, coordinate and make language resources and technology available and readily useable. CLARIN is involved in setting technical standards and creating recommendations for specific topics. CLARIN has initiated the Component MetaData Infrastructure (CMDI), which is an integrated semantic layer to
achieve semantic interoperability and overcome the differences between different metadata structures.

Presentation of responses to the Succeed questionnaire and overview of issues mentioned

To wrap up the first part of the programme, and to present an overview of the experiences and issues described by the participants, Rafael Carrasco from the University of Alicante presented the results of the Succeed online questionnaire (see also below).

Most institutions which filled out the questionnaire made clear that they are already addressing interoperability issues. They are mainly focusing on technical aspects, such as the normalization of resources or data and the creation of an interoperable architecture and interface. The motives for striving for interoperability were threefold: there is a clear demand by users; interoperability means an improved quality of service; and interoperability through cooperation with partner institutions brings many benefits to the institutions themselves. The most important benefits mentioned were: to create a single point of access (i.e., a better service to users), and to reduce the cost of software maintenance.

Tomasz Parkola and Sieta Neuerburg proceeded by recapturing the issues presented in the presentations. Clearly, all issues mentioned by participants could be placed in one of the dimensions introduced by Carl Wilson, i.e. Technical, Organizational/ Political, Financial, or Legal.

2. What is the value of interoperability?

Having established our baseline of the current status of interoperability, the afternoon programme of the workshop further included a number of interactive sessions, which were led by Irene Haslinger of the National library of the Netherlands. To start off, we asked the participants to write down their notion of the value of interoperability.

IMG_5491

The following topics were brought up:

  • Increased synergy
  • More efficient/ effective allocation of resources
  • Cost reduction
  • Improved usability
  • Improved data accessibility

3. How would you define the ideal situation with regard to interoperability?

After defining the value of interoperability, the participants were asked to describe their ‘ideal situation’.

The participants mainly mentioned their technical ideals, such as:

  • Real time/ reliable access to data providers
  • Incentives for data publishing for researchers
  • Improved (meta)data quality
  • Use of standards
  • Ideal data model and/ or flexibility in data models
  • Only one exchange protocol
  • Automated transformation mechanism
  • Unlimited computing capacity
  • All tools are “plug and play” and as simple as possible
  • Visualization analysis

Furthermore, a number of organizational ideals was brought up:

  • The right skills reside in the right place/ with the right people
  • Brokers (machines & humans) help to achieve interoperability

IMG_5496

 

4. Identifying existing barriers

After describing the ‘ideal world’, we asked the participants to go back to reality and identify the most important barriers which – in their view – stop us from achieving the interoperability ideals described above.

In his presentation of the responses to the questionnaire, Rafael Carrasco had already identified the four issues considered to be the most important barriers for the implementation of interoperability:

  • Insufficient expertise by users
  • Insufficient documentation
  • The need to maintain and/ or adapt third party software or webservices
  • Cost of implementation

The following barriers were added by the participants:

Technical issues (in order of relevance)

  • Pace of technological developments/ evolution
  • Legacy systems
  • Persistence; permanent access to data
  • Stabilizing standards

Organizational/ Political issues (in order of relevance)

  • Communication and knowledge management
  • Lack of 21st century skills
  • No willingness to share knowledge
  • “Not invented here”-syndrome
  • Establishment of trust
  • Bridging the innovation gap; responsibility as well as robustness of tools
  • Conflicts of interest between all stakeholders (e.g. different standards)
  • Decision making/ prioritizing
  • Current (EU) funding system hinders interoperability rather than helping it (funding should support interoperability between rather than within projects)

Financial issues (in order of relevance)

  • Return of investment
  • Resources
  • Time
  • Commercial interests often go against interoperability

Legal issues

  • Issues related to Intellectual Property Rights

5. Formulate an agenda: Who should address these issues?

Having identified the most important issues and barriers, we concluded the workshop by an open discussion centering on the question: who should address these issues?

In the responses to the questionnaire, the participants had identified three main groups:

  • Standardization bodies
  • The research community
  • Software developers

During the discussion, the participants added some more concrete examples;

  • Centres of Competence established by the European Commission should facilitate standardization bodies by both influence the agenda (facilitate resources) and by helping institutions to find the right experts for the interoperability issues (and vice versa)
  • Governmental institutions, including universities and other educational institutions, should strive to improve education in “21st century skills”, to improve users’ understanding of technical issues

At the end of our workshop, we concluded that, to achieve a real impact on the implementation of interoperability, there needs to be a demand from the side of the users, while the institutionIMG_5477s and software developers need to be facilitated both organizationally and financially. Most probably, European centres of competence, such as Impact, have a role to play in this field. This is also most relevant in relation to the Succeed project. One of the project deliverables will be a Roadmap for funding Centres of Competence in work programmes. The role of Centres of Competences in relation to interoperability is one of the topics discussed in this document. As such, the results of the Succeed workshop on interoperability will be used as input for this roadmap.

We would like to thank all participants for their contribution during the workshop and look forward to working with you on interoperability issues in the future!

More pictures on Flickr

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

A standard for preservation imaging

Hans van Dormolen has been selected by IS&T (Society for Imaging Science & Technology) as the recipient of a 2014 Service Award. This gave us cause to interview Hans about the important work he performs for Metamorfoze.

HansvanDormolen

What is this award about?
Every year IS&T presents 5 society related honors and awards and the Service Award is one of them. IS&T presents this award since 1960. From then on, this award has been given to a maximum of 10 people a year who have provided the society with one or more outstanding services. This year I have been selected for such an award for my work in objective capture practices for cultural heritage imaging. I have been giving lectures and teaching classes at the IS&T Archiving congresses since 2008 about objective capture practices as described in the Metamorfoze Preservation Imaging Guidelines. I feel very honored that my work is being rewarded by the IS&T society! I also believe that this reward is good for the national and international movement towards objective capture practices in the cultural heritage community.

Could you tell us more about your work on the Metamorfoze guidelines?
First of all, Metamorfoze is a funding programme which started in 1997, for the conservation of our Dutch Paper Heritage. Among the heritage institutes involved in the programme are libraries, archives and museums. The types of material vary widely and include books, newspapers, manuscripts, letters, photographs, and maps. The programme focuses on papers suffering from intrinsic decay. The objects are captured in high quality images and the original paper is preserved. Metamorfoze has now been running for 17 years. Since 1997, about 100 heritage institutes in the Netherlands have participated in the programme.
Not long after the start of the Metamorfoze programme I became involved in drafting guidelines for preservation imaging. These guidelines are built for the most part on objective capture methods and, having been a professional photographer in my previous career, on my experience in the analogue world.

It all started fifteen years ago in the world of microfilming. Using technical test charts and executing measurements such as contrast or gamma measurements I was able to provide insight into the loss of visual and tonal information of an original on a microfilm by expressing this loss numerically. It is helpful to know how much visual information is lost, because this loss is undesirable and can never be brought back and should therefore be avoided as much as possible. This policy of providing insight and diminishing the loss of visual information was at the core of all the microfilm guidelines that were produced for the conservation program Metamorfoze between 2000-2006. It is this policy that was continued in the Metamorfoze Preservation Imaging Guidelines.

From 2005 to 2012 my colleagues and I worked on drafting the Metamorfoze guidelines. In those 7 years different versions of the guidelines were published. The first concept of the guidelines was published in 2007. After various concepts version 1.0 was published in January 2012. Now, October 2014, this version is being used on a national as well as an international scale by libraries, archives, and museums. Also various camera and scanner manufactures have adapted their equipment in accordance with the Metamorfoze guidelines and are advertising with this fact.

In the Metamorfoze Preservation Imaging guidelines three quality levels are described: Metamorfoze, Metamorfoze Light, and Metamorfoze Extra Light. In quality level Metamorfoze guidelines are provided for the digitization of paper objects that are viewed as art, e.g. the letters of Vincent van Gogh. For this quality level not only the correct capture of all tonal information is of great importance, but also the retention of the exact color of the original in the digital reproduction. In quality level Metamorfoze Light we provide guidelines for the digitization of paper objects for which the retention of the exact color of the original is not as important, e.g. handwritten materials, magazines, books, and newspapers. In quality level Metamorfoze Extra Light guidelines for the digitization of books, newspapers and magazines are provided. This roadmap guarantees a good visual tonal capture for all the different kind of originals on a realistic level.

How important are the Metamorfoze guidelines?
Worldwide there are only two leading sets of imaging guidelines for the digitization of cultural heritage. The Metamorfoze guidelines are one of them. They are being applied in different ways in libraries, archives, and museums not only in Europe and North-America, but also in Asia and South-America. The other guidelines are the FADGI guidelines. These guidelines are published in the USA by the Federal Agencies Digitization Guidelines Initiative. This initiative was formed by people from the cultural heritage community from institutions like the Library of Congress and the National Archives and Records Administration (NARA). The FADGI guidelines differ from the Metamorfoze guidelines in its structure and use. But the aim “good imaging” is the same.

What are your plans for the future?
The Metamorfoze guidelines and the FADGI guidelines are just that: guidelines. Although very precise, they are not a standard like an ISO standard. For two years now I have been a member of an ISO workgroup, the ISO42 JWG26. In this workgroup we are specifying a recommendation for digitization of cultural heritage. Both guidelines, the Metamorfoze and FADGI guidelines, are used as a starting point for this recommendation. Both guidelines receive a lot of international attention and many people want to know which one is the best. With these recommendation we aim to make it clear that it is not about which guidelines are the best, but about how we can digitize our cultural heritage in the most optimal way.

Working according to the Metamorfoze guidelines requires an intense daily use of technical test charts. Using technical test charts is time consuming. With the Universal Test Target (UTT) as a quality management tool, the time spent daily on quality management can be reduced substantially. I therefore plan to publish an article about how to work with UTT according to the Metamorfoze guidelines in the near future.

The use of the Metamorfoze guidelines increases nationally as well as internationally. This use also includes a growing number of requests to teach, present lectures and to support digitization projects in a technical way. This went beyond my daily work in the KB, so I started a spin off company in 2012 to fulfil these requests. In the coming years I intend to expand this business.

IS&T will present Hans van Dormolen with the Service Award on November 6 during the 22nd Color and Imaging Conference in Boston.

Links
Metamorfoze
Metamorfoze Preservation Imaging Guidelines
IS&T
IS&T, Honors & Awards, Current year recipients

Finally ISO 16919 is published, ISO auditing can start

Originally posted on: http://digitalpreservation.nl/seeds/finally-iso-16919-is-published-iso-auditing-can-start/

Rijksmuseum-man-voor-commissie

I blogged before about the enormous amount of time it takes before a draft standard becomes an approved ISO standard and finally this week happened what we were waiting for: ISO officially published the ISO 16919 standard. Fully named: Requirements for bodies providing audit and certification of candidate trustworthy digital repositories. This mouthful of words all comes down to one element: consistency. An audit done with auditors from one part of the world should be comparable with an audit done by auditors in another part of the world. This is a problem that ISO has tackled by having a standard that regulates the accreditation of auditors : ISO 17021 Standard requirements for A&C general management systems. The PTAB group adapted this standard in order to make it complementary to the ISO 16363 -2012 standard Audit and Certification of Trustworthy Digital Repositories. What additions were needed? In short: specific digital preservation knowledge is introduced as a requirement in this standard. This is done by an explicit reference to the OAIS standard ISO 14721-2012, and the ISO 16363-2012. A list of competencies is added, describing the qualifications an auditor should possess to participate in the audit process. This also is focused on digital preservation aspects, adding to general auditors requirements like confidentiality, impartiality, responsibility etc.  Experience in digital preservation is expected and where the knowledge is lacking, training might fill the gaps. The next step will be that qualified auditors will be appointed. Here the national standard bodies play an important role, as they monitor this process. So we are not there yet, but an important milestone is reached. The European framework of audit and certification sees an external audit according to ISO 16363 as the highest level of certification. An organisation can start with certification according to the Data Seal of Approval, followed by the Nestor/DIN 31644 standard. This  will leave some time to train qualified auditors and get experience with the concept of certification in the evolving world of digital preservation.

© 2018 KB Research

Theme by Anders NorenUp ↑