Pan-European infrastructure for ocean & marine data management

Newsletter

4 March 2008

See other editions

Other editions

Content:

Foreword

The development of SeaDataNet is highly important and fits in very well with various recent EU Directives, initiatives and large-scale framework programmes on global and European scales (GEOSS and GMES (including its Marine Core Services), that urge provision of access  to, and exchange of, environmental data and information. The Blue Paper on an integrated EU maritime policy strongly promotes an integrated and intersectorial approach to ocean affairs in Europe which requires reinforced cooperation and effective coordination of all sea-related policies at the different decision-making levels. As one of its measures it proposes the development of a European Marine Observation and Data Network (EMODNet) to facilitate long-term and sustainable access to the interoperable, high-quality data necessary to understand biological, chemical and physical behaviour of seas and oceans.

There is the INSPIRE Directive, which aims for the creation of a European spatial information infrastructure that delivers integrated spatial information services to end-users. Quite recently another initiative was launched by the European Environment Agency for a Shared Environmental Information System (SEIS), that proposes to improve, modernise and streamline the present information systems for providing timely, reliable and relevant information on the state of the environment.     

All these services should allow users to identify and access spatial or geographical information from a wide range of sources, from the local level to the global level, in an inter-operable way for a variety of uses. The implementation is foreseen as ‘system of systems’, indicating that where possible use will be made of existing components, organisations and networks. The emphasis is on improving mutual cohesion, cooperation, exchange of data, harmonisation of quality of data, harmonisation of protocols for data access, not only within sectors and at national scale, but also across sectors and borders.

In the ocean and marine sector, SeaDataNet is the leading initiative in this field, actively operating and further developing a Pan-European infrastructure for managing, indexing and providing access to ocean and marine data sets and data products, acquired via research cruises and other observational activities. Therefore, SeaDataNet is actively engaged in the planning and giving feedback to the discussions around GMES Marine Core Services, INSPIRE, EMODNet, has partnerships with e.g. EuroGOOS, EuroGeoSurveys, ESF Marine Board, and POGO, and also is tuning its developments internationally. 

SeaDataNet (2006 – 2011) is undertaken by the National Oceanographic Data Centres (NODC’s), and marine information services of major research institutes, from nearby all coastal states bordering the European seas. It is funded by the EU as a Research Infrastructures project. The consortium comprises 49 partners from 35 participating countries, including NODC’s, Satellite Data Centres, expert modeling centres and the international organisations IOC, ICES and EU-JRC. The data centers are highly skilled and actively engaged in data management for several decades and have the basic capabilities and facilities for data quality control, perennial safeguarding, retrieval and distribution.

SeaDataNet is successor to a number of earlier EU marine data management projects, such as in particular Sea-Search and MEDAR/MedAtlas, and is funded as an EU Research Infrastructures I3 project within the FP6 framework programme.

It focuses on interconnecting the data centres to provide integrated on-line access to the most comprehensive sets of multi- disciplinary in-situ and remote sensing marine data, meta-data and products. The SeaDataNet architecture has been designed as a multi-disciplinary system from the beginning. It  is able to support a wide variety of data types and to serve several sector communities. SeaDataNet is willing to share its technologies and expertise, to spread and expand its approach, and to build bridges to other well established infrastructures in the  marine domain.

This newsletter is presenting the new V1 architecture of the SeaDataNet infrastructure, illustrating also the services that are provided. Moreover, the newsletter reports on specific project achievements.

Many results will be presented in the International Conference on Marine Data and Information Systems to be held in Athens from March 31 to April 2, 2008, and organised by SeaDataNet. All readers are encouraged to consider participating in this conference. It will give a good opportunity to have a comprehensive overview of ongoing activities since 150 lectures and posters will be presented.

Manzella, Giuseppe - chief editor

Description of the SeaDataNet V1 Architecture

The present V0 version of SeaDataNet provides users at the SeaDataNet portal with common Pan-European metadatabases for finding information about available data sets and data acquisition activities, but users are confronted with different local user interfaces at the 40 connected data centres for requesting or getting access to data sets.

The SeaDataNet V1 infrastructure is harmonising the metadata directories, and will give users a unified and transparent overview and access to the large base of data sets, that are managed by the SeaDataNet data centres. The V1 version will be launched in April 2008 involving a pilot group of data centres. Thereafter it will be rolled out to other data centres in SeaDataNet.

The SeaDataNet V1 infrastructure comprises the following services:

  • Discovery  services  = Metadata directories and User interfaces
  • Vocabulary services  = Common vocabularies and Governance
  • Security services  = Authentication, Authorization & Accounting (AAA)
  • Delivery services  = Requesting and Downloading of data sets
  • Viewing services  = Mapping of metadata
  • Monitoring services  = Statistics on system usage and performance and Registration of data requests and transactions
  • Maintenance services = Entry and updating of metadata by data centres

As a basis for the V1 services common standards have been defined for metadata and data formats, common vocabularies, quality flags, and quality control methods, thereby considering international standards, such as ISO 19115, NetCDF (CF), and best practices from IOC and ICES.

An architecture has been designed to provide a coherent system of the various V1 services and a later extension with the V2 services. For the implementation a range of technical components have been defined, that are partly ready developed and partly in their last stages of development and testing. These make use of recent web technologies and also comprise Java components, to provide multiple platform support and syntactic interoperability. To facilitate sharing of resources and interoperability SeaDataNet has adopted the technology of SOAP Web services for various communication tasks. A Web service is a collection of protocols and standards used for exchanging data between applications or systems. Software applications written in various programming languages and running on various platforms can use Web services to exchange data over the Internet in a manner similar to inter-process communication on a single computer. This interoperability (e.g., between Java and Python, or Windows and Linux applications) is due to the use of open standards. In common usage the term usually refers to those services that use SOAP-formatted XML envelopes for communication and have their interfaces described by WSDL.

The discovery services consist of a number of interrelated metadatabases for describing and querying data sets, research cruises, research projects, monitoring stations, and related organisations. These comprise:

  • European Directory of Marine Environmental Datasets (EDMED)
  • European Directory of Marine Environmental Research Projects (EDMERP)
  • European Directory of Marine Organisations (EDMO)
  • Cruise Summary Reports (CSR)
  • European Directory of the initial Ocean-observing Systems (EDIOS)
  • Common Data Index (CDI)

Data centres can maintain their shares in the central metadatabases via XML ingestion or alternatively via online Content Management Systems. The XML must parse to agreed SeaDataNet XML schemas (XSD) (profiles of ISO 19115). For quality assurance XML validation Web services and Vocabulary Web services have been set up. For generating  their XML output data centres can adopt SeaDataNet software tools, but also a number of data centres might opt to develop their own software applications.

The Vocabulary Web services have been set-up to provide ‘controlled vocabularies’, which are used in the metadata and to label data. This SeaDataNet Vocabulary service provides access to lists of standardised terms that cover a broad spectrum of disciplines of relevance to the oceanographic and wider community. Using standardised sets of terms solves the problem of ambiguities associated with data markup and also enables records to be interpreted by computers. This opens up data sets to a whole world of possibilities for computer aided manipulation, distribution and long term reuse. The Web services are compliant to WS Basic Profile 1.1 of the Web Services Interoperability Organisation (WS-I). This Basic Profile has been adopted as standard for Web services in SeaDataNet. On top of this webservice a Vocabulary Client Interface is operational to provide users the options to search and browse in the various vocabularies and to make and download export files of selected entries in csv format.

Each V1 metadatabase will feature a dedicated User Interface with comparable functionality and look & feel. Next to this for each metadatabase Web services are planned. This will provide important extra functionality, that strengthens the basis for long term continuation and maintenance of the system and enables integration of the metadatabases in other portals and websites with their own layout and look & feel.

SeaDataNet has defined a common SeaDataNet Data policy, which has been adopted by all partners. Its basis is striving for free, but controlled access to all data. Control includes, that users will have to register and agree with a common SeaDataNet licence for data access. Metadata browsing (discovery services) is in the public domain and requires no registration. At the level of each data centre there might be local conditions to data access, which have to be respected. Therefore the SeaDataNet infrastructure includes a user registry and authentication facility between the discovery services and the actual delivery services. The user registry uses the Central Authentication System (CAS), which provides a Single Sign On server and makes use of LDAP  (Lightweight  Directory Access Protocol). Each user is registered via its national NODC and given a role (e.g. academic, commercial, national government, ..). That role can be thought of as a key that a user possesses. By registering each user agrees to the SeaDataNet user licence, which sets out the terms on which users can use the SeaDataNet data sets, once acquired. This licence protects the data originators copyrights. 
  
An important discovery service is the Common Data Index (CDI), that enables users to have a detailed insight of the availability and geographical coverage of marine data, archived at the connected data centers. It gives the description of individual data sets and measurements with key fields like what, where, when, how, who etc. The CDI XML files must be extracted by every data center from their in-house data management systems, which could be file based and/or a relational database.

In the SeaDataNet architecture the CDI is the linking pin from the discovery services towards the delivery services. It is directly related to the data sets, to which the users can request access. SeaDataNet distincts between types of users (roles) and data centres can label differentiating access to data sets related to roles. These data access label(s) are included in each CDI, so that it is known to users under which conditions access can be provided to specific data sets.

Users can freely search and browse in the discovery services and identify interesting data sets (in)directly  via the CDI. Users can include references to these data sets in a Shopping basket, which can contain references to data sets from multiple data centres. To activate their requests from the Shopping basket, users will be prompted by the security services for their user id and password as registered in the CAS user register, administering their identity and their role. If a user is not yet registered, then the user will be asked to register. The combination of the registered role of the user AND the data access label(s) determines the actual access to the requested data sets.

The access to data sets in SeaDataNet V1 is restricted to downloading services. This means that users are able to download selected data sets in common transport formats for those data sets to which access is granted considering the role of the user and the policy label of the data set.

The shopping requests by registered users will have to be processed by the central portal in communication with the data centres, that are managing the requested data sets. Therefore in V1 a download manager and a request status manager are specified. The download manager, a Java program, is implemented at the data management systems of each data centre. The request status manager is integrated in the portal, sorts out the shopping basket requests, validates the combinations of  user roles vs data access labels, and passes agreed requests to the download manager at the data centres for further processing. The download manager will check the requests, retrieves each requested elementary data sets, aggregates  these data sets in zipped comprehensive datasets and makes the comprehensive datasets available on an ftp server or dedicated web address. It is possible that the combination of user role and access label(s) determines that the delivery must be discussed between user and data centre. These cases are identified and also administered by the request status manager and the data centre’s  manager is notified by e-mail. After discussion the data centre can decide to give or deny access, which is administered again at the request status manager. Depending on the decision the data access flow will be triggered or halted.

The request status manager not only regulates the data access procedure, in communication with the download manager and/or the data centre’s manager. It also provides the user with a private web register of his/her individual requests, their processing status and the URLs for downloading the zipped files from each of the relevant data centres, in case of agreed access. In addition it offers administration of all transactions and their specifics for the data providers.

The implementation of the downloading services requires each local system to deliver data sets in the agreed SeaDataNet transport formats (NetCDF, ODV, optional MedAtlas). A JavaTool is available to support data centres in this local conversion.

The viewing services in SeaDataNet V1 are restricted to enabling direct use of SeaDataNet output data sets in the stand-alone Ocean Data View (ODV) data analysis and visualisation package, that has been upgraded for SeaDataNet, and to including geographical mapping services as part of the discovery services for displaying and querying positions of measurements, monitoring sites, ship tracks, etc.

In the coming years the SeaDataNet infrastructure will be extended into Version 2 with a further development of :

  • Viewing services = Quick views and Visualisation of data and data products
  • Product services = Generic and standard products

For the viewing services the OGC standards will be adopted, comprising Web Map Services (WMS), Web Feature Services (WFS) and Web Coverage Services (WCS). These will support the quick viewing and visualization of data sets and data products.

Work is already progressing for the generic production of regional statistical products like mean, seasonal and monthly climatological fields and trends from all available historical and recent data collected for regional seas. In the near future these will be produced and updated by the SeaDataNet infrastructure at regular intervals and provided to users

For the latter a common tool has been adopted and upgraded for SeaDataNet for preparing or qualifying comparable statistical gridded data products at the regional or global scale from merged insitu and satellite data sets. This is the DIVA package, which gives an optimal statistical interpolation algorithm (covariance function). It facilitates precise analysis of data such as climatology, taking in account regional specificities such as bathymetry.

Finally there are monitoring services, that comprise registration of statistics on usage and performance of the SeaDataNet infrastructure, its individual data centres, its portal and its components, next to the data transaction statistics as kept by the request status manager.

Schaap, Dick M.A. and the SeaDataNet Technical Task Team

SeaDataNet and the second IMDIS Conference 2008

SeaDataNet is actively operating and further developing a Pan-European infrastructure for managing, indexing and providing access to ocean and marine environmental data sets and data products (e.g. physical, chemical, and biological properties) and for safeguarding a long term archival and stewardship of these data sets. Data are derived from many different sensors installed on board of research vessels, satellites and in-situ platforms, that are part of the various ocean and marine observing systems.

The implementation of SeaDataNet is carried out by comparison with other experiences at international level. Therefore SeaDataNet has an International Advisory Board, with representatives of NOAA - IOOS, DMAC, Canadian NODC, and EuroGOOS. Furthermore SeaDataNet organises the second International Conference on Marine Data Management Systems (IMDIS 2008), that will take place in Athens - Greece from March 31 to April 2, 2008. The Conference aims at providing an overview and inside knowlewdge of existing and on-going developments for marine environmental data management systems,  showing the progress in development of efficient infrastructures for managing large and diverse data sets, comparing different information systems, exploring the applicability of new information technologies, and establishing long term international cooperation.

A major challenge for the presentations will be demonstrating interoperability and common standards, while developing distributed systems for various marine disciplines. Another challenge will be how to adopt new technologies and to use these in favor of users and data providers.

A total of 150 lectures and posters will be presented and the Conference will be joined by an international community of key players in international ocean and marine data management. SeaDataNet partners will be present to give lectures and posters on their achievements and to discuss developments and possible cooperation.

You are invited and encouraged to participate in the IMDIS Conference. You can find additional information and register at: http://hnodc.hcmr.gr/imdis-2008  

Balopoulos, Efstathios; Maillard, Catherine; Iona, Sissy

SeaDataNet Data Policy

SeaDataNet has defined a common SeaDataNet Data policy, which has been adopted by all partners. The policy aims to strike a balance between the rights of researchers and the need for widespread access through the free and unrestricted sharing and exchange of data, meta-data and data products through SeaDataNet. The policy applies to data managed by SeaDataNet partners for providing access to data managed in the SeaDataNet distributed systems. Therefore the policy has formulated rules for both users and data providers of SeaDataNet.

By maximizing the availability of data to the community at large, SeaDataNet promotes the use of these data, thereby ensuring that their maximum value can be realized and thus contribute to an increased understanding of the marine environment.

The SeaDataNet data policy is consistent with, and in the spirit of, national and international policies and laws. Applicable policies or laws are those related to UN conventions, policies of international bodies often within the UN, policies and laws of the European Union. The SeaDataNet data policy is intended to be fully compatible with the Directive of the European Parliament and of the Council on public access to environmental information, the INSPIRE Directive, IOC, ICES, WMO, GCOS, GEOSS and CLIVAR data principles.

SeaDataNet makes data available freely and without restriction. “Freely” means at no more than the cost of reproduction and delivery, without charge for the data itself. “Without restriction” means without discrimination against, for example, individuals, research groups, or nationality. SeaDataNet makes data available in a timely and easy way to users, but SeaDataNet remains dependent on data contributions. SeaDataNet uses the most adapted and recent advances in communication to ensure widespread access to data collected under auspices of the programme. The SeaDataNet website is the focus point for dissemination.

According to the different types of data assets, the access conditions vary:

  • meta-data are freely accessible without any condition
  • data and products require:
    o    registration
    o    acceptance of additional conditions that may be requested by the data centres. The access rights are granted according to the “role” of the user.
    o    acceptance of an user licence.

The SeaDataNet infrastructure includes a user registry and authentication facility between the Common Data Index (CDI) discovery services and the actual data delivery services. Each user can register via the SeaDataNet portal. Registration requests are processed via the national SeaDataNet partner and after acceptance users are registered in the central SeaDataNet user register and given a role (e.g. academic, commercial, national government, ..). That role can be thought of as a key that a user possesses. By registering each user agrees to the SeaDataNet user licence, which sets out the terms on which users can use the SeaDataNet data sets, once acquired. This licence protects the data originators copyrights. The text of the user licence is given below.

The Common Data Index (CDI) is directly related to the data sets, to which the users can request access. SeaDataNet data centres can label differentiating access conditions to data sets related to roles. These data access label(s) are included in each CDI, so that it is known to users under which conditions access can be provided to specific data sets.

-------------------------
SeaDataNet User Licence:

1. The Licensor grants to the Licensee a non-exclusive and non-transferable licence to retrieve and use data sets and products from the SeaDatanet service in accordance with this licence.

2. Retrieval, by electronic download, and the use of Data Sets is free of charge, unless otherwise stipulated.

3. Regardless of whether the data are quality controlled or not, SeaDataNet and the data source do not accept any liability for the correctness and/or appropriate interpretation of the data. Interpretation should follow scientific rules and is always the user’s responsibility. Correct and appropriate data interpretation is solely the responsibility of data users.

4. Users must acknowledge data sources. It is not ethical to publish data without proper attribution or co-authorship. Any person making substantial use of data must communicate with the data source prior to publication, and should possibly consider the data source(s) for co-authorship of published results.

5. Data Users should not give to third parties any SeaDataNet data or product without prior consent from the source Data Centre.

6. Data Users must respect any and all restrictions on the use or reproduction of data. The use or reproduction of data for commercial purpose might require prior written permission from the data source.

7. Users are requested to inform SeaDataNet of any problems encountered with SeaDataNet-provided data. A timely and easy-to-use feedback procedure is available (sdn-userdesk@seadatanet.org), aimed at correcting data at the data source. This feedback will increase the quality of the data.
---------------------------

The Data policy also formulates rules for SeaDataNet data centres to follow up.

Meta-data and other supplemental information are required to enable the use of data without ambiguity or uncertainty. Meta-data attached to the data sets (Common Data Index (CDI)) and to the other catalogues services EDMED, CSR, EDIOS, EDMERP, and EDMO are developed and managed in accordance with international standards. Each SeaDataNet data centre should compile and make available the related meta-data. 

Uniformity of data format and quality have a high priority. All data and related meta-data and quality indicators should be submitted by using standard codes, formats, and protocols as agreed in SeaDataNet.

SeaDataNet develops and applies quality assurance procedures as appropriate and feasible, and in cooperation with data sources and other organizations. The data source always retains complete responsibility for data quality. To indicate the quality control that has been applied to a specific data set, SeaDataNet utilises quality flags.

The SeaDataNet data centres must assure archival systems, so that the collected data, images, products and meta-data may be safeguarded for future analysis. Inventories of data, images and products and related meta-data should be readily accessible and updated as needed on a routine basis. Long-term survival, integrity, and access to SeaDataNet data will be preserved for future generations by the SeaDataNet data centres.

Maudire, Gilbert and SeaDataNet Steering Committee

Putting Meaning into SeaDataNet

SeaDataNet is built upon a foundation of metadata that tells the software what to do with the data. For this to happen, the metadata has to carry information in a form that machines can understand.

The metadata formats developed and populated during the predecessor project Sea-Search were based on plain language fields, which are virtually useless for this.  Something had to be done.

The solution developed for SeaDataNet is based on technology known as the Semantic Web (Berners-Lee et al., 2001)1. The metadata fields are populated using encoded strings known as a Universal Resource Names (URNs).  Each URN may be resolved through simple string manipulation (eventually to be provided by a SeaDataNet resolution service) into a Universal Resource Locator (URL) that points to a location on the internet holding semantic information.

Each SeaDataNet URL points to the Vocabulary Server developed by BODC as part of the NERC DataGrid project and delivers an XML document containing semantic descriptions of the resource in human-readable form. More importantly, it also contains a list of URLs to other semantically related resources together with a standardised description of the relationship between each pair.  In other words a subject URL, a predicate and an object URL that form the components of a Resources Description Framework (RDF) triple: the fundamental building block of the Semantic Web.

The resulting semantic framework may be used to underpin automated data synthesis, semantic cross-walking or semantic discovery.  For example, if a user enters ‘pigments’ into a search engine it gets matched by a Vocabulary Server search method to a node in the semantic infrastructure. This is realised as an XML document containing the URLs to related broader terms such as ‘chlorophyll pigments’.  These in turn are realised as documents representing even broader terms such as ‘chloropyll-a’ and ‘chlorophyll-b’.  If the broader terms found walking the chain are added to the target list, datasets labelled ‘chlorophyll-a’ may be located by a search for ‘pigments’.

Operationally, the Vocabulary Server used by SeaDataNet contains over 100,000 terms in 100 lists linked by a store of almost 40,000 RDF triples. This provides significant semantic functionality that will be considerably enhanced as work currently underway to double the size of the RDF store by encoding additional mappings is completed.

The SeaDataNet Vocabulary service is based upon the NERC DataGrid (NDG) vocabulary Web service, originally developed in 2006 and operated by BODC.The current version is a recently upgraded version (V1.1) of the Web services, which now is compliant to WS Basic Profile 1.1 of the Web Services Interoperability Organisation (WS-I). This Basic Profile has been adopted as standard for Web services in SeaDataNet. This Web service can be found at: http://www.bodc.ac.uk/products/web_services/vocab/ 

On top of this webservice a vocabulary Client Interface has been developed and is operated by MARIS, to provide users the options to search and browse in the various vocabularies and to make and download export files of selected entries in csv format. These csv files can be used in conjunction with e.g. the JavaTool, that is available for the CDI generation. To harvest the latest versions of the lists from the NDG Web service an automatic UPDATE function is included that runs every time to load the latest updates into a local buffer for feeding the Search and Browse interface.

Lowry, Roy and Williams, Geoff

CDI/THREDDS Interoperability in SeaDataNet

By use of standards for communication and new developments in information technology, the 40 in-situ and satellite marine data platforms of the SeaDataNet partnership are providing metadata, data and products as a unique virtual data centre. The core of the SeaDataNet infrastructure is represented by some discovery services (e.g. EDMED, EDMERP, EDMO, CSR, EDIOS) and the Common Data Index, an XML document allowing the access to data in a distributed system.

The tradition of creating formal data centers to house and distribute ocean data and products has long been shadowed by the parallel existence of direct data downloads from research groups (usually via HTML page-links and/or FTP servers). Historically, at least half the data available to researchers has been available outside the IODE/World Data Center System facilities. During the last decades efforts have been dedicated to create a new system of internet communication (e.g. the Data Access Protocol [DAP]) that would allow users to access distributed datafiles directly, using a family of "software engines" that overlay datafile caches, allowing retrieval of files in desired formats, no matter what the storage format might be. SeaDataNet infrastructure is building some infrastructure blocks allowing to manage some legacy problems related to different information systems. In particular, it is constructing a ‘bridge’ between the SeaDataNet Common Data Index (CDI) directory and the THREDDS middleware services.

Distributed Catalog Services
According to the general definition, geospatial data catalogs are discovery and access systems that use metadata as the target for query on geospatial information. Indexed and searchable metadata provide a disciplined vocabulary against which intelligent geospatial search can be performed within or among communities [D. Nebert, The SDI Cookbook, ver. 2.0, 25/01/2004].
Due to the existence of many disciplinary application profiles, cross-domain a catalog service must be able to address heterogeneity as well as distribution. This implies the implementation of catalog components for query distribution and virtual resource aggregation by applying mediation approaches. These catalogs may be referred to as fully distributed catalogs. In fact, these solutions must implement distributed discovery functionalities in a fully heterogeneous environment, which requires metadata profiles harmonization as well as protocol adaptation and mediation. Catalog clearinghouse component should realize fully distributed catalog services. Indeed, the development of catalog clearinghouse solutions is a near-term challenge for cross-domain spatial data infrastructures, such as: INSPIRE, NSDI and GEOSS.
Catalog clearinghouse component adopts a federated architecture to federate data & metadata services, distributing client queries. Some important benefits are: usability (i.e. one-stop-shop server) and performance -where caching and/or parallelism is enabled. On the other hand, there are some issues to be addressed; important ones are: cyclic queries, data identity, heterogeneous resources mediation.

Standard Specifications
Abstract model for distributed catalog service is formalized by OGC CS-W which complies with the ISO 19119 (Geographic information — Services) requirements. CS-W functionalities includes discovery as mandatory capability (i.e. Metadata Query & Presentation) and a couple of optional capabilities: session (i.e. initialize, close, status) and management (i.e. harvest, transaction).
In our opinion, the distribution aspects should be further specified, becoming another optional capability. Besides, distribution cannot be fully addressed without considering mediation capabilities. In fact, catalog services heterogeneity (e.g. the different CS-W application profiles) is present and it is going to stay. Therefore, we proposed an extended functional model for implementing fully distributed catalog services.

GI-cat
Based on the proposed extended model, we developed a fully distributed catalog solution, called GI-cat. GI-cat implements a framework to federate well-accepted catalog, inventory and access standard services. As for International standards, GI-cat supports the OGC: CS-W, WCS and WMS. As far as Community standards are considered, GI-cat federates the following services: UNIDATA THREDDS catalog service, the CDI (Common Data Index) and the GBIF (Global Biodiversity Information Facility) catalog service.
The solution is especially conceived for Imagery, Gridded and Coverage Data (IGCD). It supports distributed queries over a hierarchical data model, supporting incremental queries (i.e. query over collections, to be subsequently refined. It supports translucent and opaque service chaining. GI-cat is a framework consisting of several modular components which implement the different capabilities of the extended distributed catalog model.

In the framework of the SeaDataNet project, GI-cat is used to mediate between resources made available through federated THREDDS and CDI services, exposing them through a unique and standard CS-W interface.

Nativi Stefano, Mazzetti Paolo, Gambetta Marco, Manzella M.R. Manzella

Tools for education and capacity building developed in the frame of the SeaDataNet European project

In the frame of SeadataNet project, specific software has been developed for the partners of the project.

SeaDataNet developed ISO-19115 descriptions for the catalogues managed in the project: cruise summary reports (CSR), marine environmental data sets (EDMED), marine environmental research projects (EDMERP), common data index (CDI), European directory of the ocean-observing System (EDIOS).

Common vocabularies have been set up to make sure that all partners use Standards for meta-data and communication.

The project has also defined common format for the data exchange which are NetCDF, ODV (Ocean Data View)  both mandatory, and MEDATLAS optional.

Two tools have been designed to help the partners to generate the standard catalogue descriptions and the standard formats of SeaDatNet :

MIKADO : is a XML catalogue description generator used to create XML files for metadata exchange of CSR, EDMED, CDI, EDMERP and EDIOS.

MIKADO can be used into 2 different ways :

  • One manual way, to input manually information for catalogues in order to generate XML files. 
  • One automatic way, to generate these descriptions automatically if information is catalogued in a relational database or in an Excel file.

NEMO is a reformatting software used for data exchange between SeaDataNet partners. Its objective is to reformat any ASCII file of vertical profiles (like CTD, Bottle, XBT) or time-series (like current meters, sea level data) to a SeaDataNet ASCII format  (ODV or MEDATLAS).

These two tools are written in Java Language which means that they are available under multiple environment (Windows 2000, XP, VISTA), Unix (Solaris with the graphical library GTK) and Linux with GTK also.

They both use the SeaDataNet common vocabularies web services to update lists of values which are then uploaded locally. That means that they need network connections in order to have up to date lists of values.

Training and capacity building
MIKADO version 0.2  is operational since 2007 and a number of partners have already used it for their XML generation of Metadata descriptions. It was teached and demonstrated in a training course organised at the IODE centre of Ostende (Belgium).

A new version is under development for the version 1 of the SeaDataNet project which takes into account the evolutions of the XML schemas of the catalogues.

NEMO will be demonstrated at the next training course also in Ostende, during the session organised in June 2008.

M. Fichaut, V. Harscoat, M. Larour, G. Maudire, V. Tosello, M. Tréguer