Objectives: Proper testing, assessment and evaluation against the initial proposal and the competing environment is important for EuDML, to support the process of decision about its future sustainability, the process of benchmarking during the developments, and also during the design the requirement for clear interfaces for adding new content repositories in the future. In this sense we will evaluate not only EuDML and its providers, but also eventual competitors.
For this purpose, the following facets will be considered:
Organization – the organizational context and sustainability (risks, threats, weaknesses, potentials, strengths and opportunities for survival and growth)
Services – the technical and functional characteristics of the services (interfaces, reliability, robustness, etc., for both humans and other machines usage)
Contents – the quantity and quality of the contents
- Deliverable 11.1 – Report on evaluation of existing services and content providers The purpose of this document is to give a much more detailed picture of existing services and content providers in EuDML project. To support that, we performed an evaluation and analysis process, which results highlights those services characteristics and system features that are most relevant to the project. These results will facilitate the design decisions and recommendations for EuDML system features and its functionality.
- Deliverable 11.2 – EuDML assessment and evaluation plan This report develops the global plan for self-evaluation of EuDML. An evaluation framework is defined according to findings and recommendations from the DELOS Digital Library Classification and Evaluation Scheme. Assessment and evaluation criteria are structured following the concepts of digital library components (content, system, users) and their relationships (usability, usefulness, and performance), as well as policies governing the system. Further general evaluation criteria are derived from the Digital Library Reference Model Conformance Checklist from project DL.org.
Special evaluation topics and evaluation criteria suitable and applicable to EuDML are derived from the project description, goals, and assertions of the EuDML “Description of Work”. An evaluation roadmap is given which describes how the evaluation framework and criteria will be implemented during two rounds of evaluation – the first with the aim of providing guidance and recommendations for further development during the project, and the second with the aim of guiding further actions beyond the lifetime of the project (issues of sustainability).
- D11.4 – EuDML Assessment and Evaluation – Final Report This evaluation report’s findings are that the project was delivered on schedule, meeting almost all the defined evaluation criteria, while almost all its performance parameters are found to be at or above their expected values. The content-providers and user feedback on the EuDML Release (version 1.4) show that the system is currently stable, functional and useful. Most of the suggested improvements made by users while answering the surveys were translated and tracked as bugs through the Mantis system (the EuDML bug-tracking system). Many of them have been attended to and the result is the current Release (version 2).
