Pan-European infrastructure for ocean & marine data management

Newsletter

2 August 2012

See other editions

Other editions

Do you want to receive our newsletter? Or do you want to unsubscribe?

Go to the form

Content:

SeaDataNet II: a second phase for the pan European infrastructure

The second phase of the project SeaDataNet started on October 2011 for another 4 years project  with the aim to upgrade the SeaDataNet infrastructure built during previous years. The numbers of the project are quite impressive: 59 institutions from 35 different countries are involved. In particular, 45 data centers are sharing human and financial resources in a common efforts to sustain an operationally robust and state-of-the-art Pan-European infrastructure for providing up-to-date and high quality access to ocean and marine metadata, data and data products.

The main objective of SeaDataNet II is to improve operations and to progress towards an efficient data management infrastructure able to handle the diversity and large volume of data collected via the Pan-European oceanographic fleet and the new observation systems, both in real-time and delayed mode. The infrastructure is based on a semi-distributed system that incorporates and enhance the existing NODCs network.

SeaDataNet aims at serving users from science, environmental management, policy making, and economical sectors. Better integrated data systems are vital for these users to achieve improved scientific research and results, to support marine environmental and integrated coastal zone management, to establish indicators of Good Environmental Status for sea basins, and to support offshore industry developments, shipping, fisheries, and other economic activities. The recent EU communication “MARINE KNOWLEDGE 2020 - marine data and observation for smart and sustainable growth” states that the creation of marine knowledge begins with observation of the seas and oceans.

In addition, directives, policies, science programmes require reporting of the state of the seas and oceans in an integrated pan-European manner: of particular note are INSPIRE, MSFD, WISE-Marine and GMES Marine Core Service. These underpin the importance of a well functioning marine and ocean data management infrastructure. 

SeaDataNet is now one of the major players in informatics in oceanography and collaborative relationships have been created with other EU and non EU projects. In particular SeaDataNet has recognised roles in the continuous serving of common vocabularies, the provision of tools for data management, as well as giving access to metadata, data sets and data products of importance for society.

Use of common vocabularies in all metadatabases and data formats is an important prerequisite towards consistency and interoperability. They consist of lists of standardised terms that cover a broad spectrum of disciplines of relevance to the oceanographic and wider community. Using standardised sets of terms solves the problem of ambiguities associated with data markup and also enables records to be interpreted by computers. This opens up data sets to a whole world of possibilities for computer aided manipulation, distribution and long term reuse. In SeaDataNet II the vocabularies service will be upgraded and further populated.

The implementation of the Ocean Data View (ODV) software for data analysis is strongly supported by SeaDataNet. It is now used by many researchers and is regularly applied for quality control and visualization. In SeaDataNet II additional functionalities are added to the ODV software and it will be adopted also for duplicates management.

The ODV software is also being used in SeaDataNet for producing generic data products for each of the regional seas for various variables. The DIVA software tool ( Data-Interpolating Variational Analysis ) allows to spatially interpolate (or analyse) those observations on a regular grid in an optimal way. As a part of SeaDataNet, the DIVA method has been integrated into ODV, and the integration greatly facilitates the usage of DIVA. Features supported by the ODV/DIVA integration include proper treatment of domain separation due to land masses and undersea ridges or seamounts and the realistic estimation of water mass properties on both sides of the divides. In SeaDataNet II the DIVA software (both offline and online) will be upgraded with new features.

The interfaces developed for each software package and service are facilitating their use. However, the scientists and data managers need to know the concepts behind each software and the correct use of it. 'Best practices' are evolving to become better as improvements are introduced in the softwares. For this reason in SeaDataNet two important activities have been introduced from the beginning: science literacy and training courses. The latter are instrumental to transfer information and give guidance about new developed SeaDataNet standards, tools and services to the SeaDataNet network of connected data centres for wider implementation.

7_article_1_img_1.jpg (76.4 K) 

Activities in SeaDataNet II are:

  • Operational maintenance of the SeaDataNet pan-European catalogues (CDI, EDMED, EDMERP, EDMO, EDIOS, CSR) for providing more and up-to-date information and data
  • Seeking INSPIRE Directive compliance: adopting the ISO-19139 standard for XML description and OGC-CSW exchange of the CDI and CSR catalogues
  • Developing machine to machine interfaces next to existing user interfaces for data and product distribution to regular user communities (operational oceanography, MSFD, ..)
  • Improving data sets duplicate management and introducing Data Preview services before requesting downloading
  • Improving the capability for handling also marine biological data in close cooperation with EurOBIS, MarBEF, and other biological data initiatives
  • Expanding the Product portfolio by aggregated data sets at full basin scale using the data sets made available via the SeaDataNet infrastructure
  • Deploying true synergies with other projects such as Eurofleets and Jerico for streamlining the flow of data from acquisition at Research Vessels and Fixed Monitoring systems to data centres, also exploring Sensor Web Enablement (SWE), and with MyOcean for improved data exchange from real-time systems and data centres to the ocean modelling community
  • Reinforcing the SeaDataNet base of data holders; increasing the number of connected data centres as well as the volumes and types of data sets to be accessed via the SeaDataNet services
  • Maintaining and extending the driver role of the SeaDataNet infrastructure and its community in the further development and implementation of the European Marine Observations and Data Network (EMODNet) as initiated by the EU in the framework of the MSFD Directive
  • Seeking interaction and synergy with international standards developments for IT (OGC, ISO), ocean data management (IODE Ocean Data Standards initiative) and interoperability with European and global portals (INSPIRE, GEOSS, Ocean Data Portal)  
  • Organising feedback from user communities in order to improve services of the infrastructure.

The work plan for SeaDataNet II is organised as a cycle of activities that pass from operation to development to operation. This is indicated in the figure below that also highlights the various activities that are part of the work plan. A very important aspect is that new services, components and standards must be implemented over the whole network and without causing disturbances in the operational functioning of the infrastructure. This will be achieved by versioning of services, parallel installation and testing before moving to production, and careful coordination of upgrades implementation.

7_article_1_img_2.gif (29.5 K) 

This Newsletter is presenting a number of the present services of the SeaDataNet infrastructure and services, and highlighting a number of key achievements in SeaDataNet II so far. The Newsletter also gives information on the progress of associated and related projects. Finally an outlook is given to the targets of the new Ocean Data Interoperability Platform (ODIP) project that recently has been accepted by the EU.

We hope you enjoy the newsletter and will be triggered to visit the SeaDataNet portal (http://www.seadatanet.org) for a try out of its services and to follow its evolution.

SeaDataNet portal and its services

The SeaDataNet infrastructure comprises a network of interconnected data centres and a central SeaDataNet portal. The portal provides users not only background information about SeaDataNet and the various SeaDataNet standards and tools, but also a unified and transparent overview of the metadata and controlled access to the large collections of data sets, managed by the interconnected data centres.

7_article_65_img_0.jpg (46.2 K)

The SeaDataNet architecture comprises the following middleware services:

  • Discovery services = Metadata directories and User interfaces
  • Vocabulary services = Common vocabularies and Governance
  • Security services = Authentication, Authorisation & Accounting
  • Delivery services = Requesting and Downloading of data sets
  • Viewing services = Mapping of metadata
  • Product viewing services = Viewing of generic and standard products in maps
  • Monitoring services = Statistics on system usage and performance and Registration of data requests and transactions
  • Maintenance services = Entry and updating of metadata by data centres

In SeaDataNet II this will be extended with:

  • Viewing services = Quick views and Visualisation of data
  • Machine to Machine interface services = Providing interoperability and automatic exchanges with other systems
  • Delivery services = Requesting and downloading of aggregated data sets
  • Delivery services = Access to real-time oceanography data 

As a basis for the services, common SeaDataNet standards have been defined for metadata and data formats, common vocabularies, quality flags, and quality control methods, based on international standards, such as ISO 19115, NetCDF (CF), ODV, and best practices from IOC and ICES. In SeaDataNet II RTD activities will lead to upgrades and extensions of these standards, which will be taken into operation. SeaDataNet II will continue to operate the gradually upgraded infrastructure to interconnect the data centres and their collections, and to provide access to harmonised metadata, data, aggregated data sets and data products in common formats and common visualisations ready for use by the various user communities. The integration of all different types of data enhances the total availability and richness of data and data services to users.

This involves for all connected data centres and the centres that are managing the central SeaDataNet services:

  • Providing full time and operationally robust services
  • Providing repository capacities for perennial archiving of data sets
  • Providing monitoring services of usage
  • Providing service desk for processing user data requests that need mediation    

SeaDataNet maintains and operates at its portal several discovery services with overviews of marine organisations in Europe and their engagement in marine research projects, managing large datasets, and data acquisition by research vessels and monitoring programmes for the European seas and global oceans:  

  • European Directory of Marine Organisations (EDMO) (at present > 2000 entries) 
  • European Directory of Marine Environmental Data (EDMED) (at present > 4300 entries) 
  • European Directory of Marine Environmental Research Projects (EDMERP) (at present > 2200 entries) 
  • Cruise Summary Reports (CSR) (at present > 43000 entries) 
  • European Directory of  the Ocean Observing Systems (EDIOS) (at present > 10000 entries) 

Note: the look & feel of the portal and all discovery services have recently been upgraded.
 
SeaDataNet maintains and operates Common Vocabulary Web services, covering a broad spectrum of ocean and marine disciplines. The common terms are used to mark up metadata, data and data products in a consistent and coherent way. Governance is regulated by an international board. At present the Vocabulary Services comprise over 120000 terms in over 100 lists.

Moreover SeaDataNet provides documentation and common software tools for metadata and data formating (MIKADO, NEMO, Ends&Bends, Download Manager), Quality Control - Quality Assurance,  statistical analysis (DIVA) and a versatile software package for data analysis and presentation (ODV).  These tools can be downloaded without any restriction from the SeaDataNet portal.

SeaDataNet provides online unified access to distributed datasets via its portal website to the vast resources of marine and ocean datasets, managed by the distributed data centres. This takes place via the Common Data Index (CDI) Data Discovery and Access Service.

Marine Knowledge and EMODNet

Access to marine data is of vital importance for marine research and a key issue for various studies, from climate change prediction to off shore engineering. The EU has recently published a communication 'Marine Knowledge 2020' which underpins the importance of data availability and access. SeaDataNet is contributing to the objectives defined in the communication by giving and harmonising access to marine data from different sources with the aim of:

  • Helping industry, public authorities and researchers find the data and make more effective use of them to develop new products and services.
  • Improving our understanding of how the seas behave.

Information systems have an increasing role in the society and a strong impact on science, technology and business. The preservation of data is therefore also a objective of SeaDataNet:

  • Access to data can generate new science;
  • Retention of unique observational data which is impossible to re-create;
  • Retention of expensively generated data which is cheaper to maintain than to re-generate;
  • Data are necessary to assess compliance with legal requirements;
  • Data are necessary to validate published research results and for use in teaching.

SeaDataNet is federating data providers in the EU and neighbouring coastal States with the objectives to provide access to coherent, quality checked data. The federation is lead by NODCs and thematic data centres that have an important role in the long term preservation and provision of data.

SeaDataNet is providing a major contribution to the development process for the overarching EMODNet (European Marine Observation and Data Network) that is included in the Marine Strategy Framework Directive (MSFD), adopted in December 2007 by the European Parliament and Council. EMODNet is developing into a network of existing and developing European observation systems, linked by a data management structure covering all European coastal waters, shelf seas and surrounding ocean basins. It must facilitate long-term and sustainable access to the high-quality data necessary to understand the biological, chemical and physical behaviour of seas and oceans. Key elements for harmonisation and interoperability are establishing and adopting common standards and protocols for quality control, metadata and data formats, vocabularies and technical protocols. This vision for EMODNet is shared by the EU, ESF Marine Board, EuroGOOS and its regional associations.

EMODnet should respect INSPIRE standards for discovery and access. EMODnet will provide data on scales defined by the regions and subregions of the Marine Strategy Framework Directive. EMODnet, as an open data system, is also considered as a significant observation and monitoring data conduit for the part of the Water Information System for Europe (WISE) that will be developed for dealing with marine information (WISE-Marine) and supporting the data and indicator needs for the initial assessments required by member States in 2012 by the Marine Strategy Framework Directive. WISE and WISE-Marine are thematic branches of the envisaged Shared Environmental Information System (SEIS) based on INSPIRE principles. EMODnet data should be directly available for viewing through WISE-Marine.

The "proof of concept" of EMODnet has been tested through preparatory actions. Portals for a number of maritime basins have been set up for hydrographic, geological, biological, chemical and physical data as well as functional habitat maps. These portals provide access to marine data and data products of a standard format and known quality.

Portals have been initiated for the following marine data themes, covering selected marine basins:

Common approach using SeaDataNet:
The EMODNet approach with thematic portals for specific disciplines and communities and with EMODNet concertation meetings together with MODEG experts is considered as very useful and effective. This way many potential players from a given discipline or theme can be engaged for their own specialism and interest, while the interoperability and cohesion between the thematic portals is achieved by using common standards from OGC for viewing services (WMS, WFS) and SeaDataNet for data discovery and access services and semantic interoperability. In practice most of the portals (chemistry, hydrography, physics, geology (via link with Geo-Seas)) have adopted the SeaDataNet approach of using the CDI data discovery and access service including its flexible data access restrictions for giving overview and access to basic measurements datasets. Biology at present uses only the EurOBIS standards, but in its next phase will include the SeaDataNet standards and approach for giving access to distributed data sets. Thereby it will harvest from the SeaDataNet II project efforts for upgrading the SeaDataNet standards to make them also fit for handling biological data in an interoperability scheme with EurOBIS.

EMODNet encourages more data providers to come forward for data sharing and participating in the process of making complete overviews and homogeneous data products. This will give wider visibility at the policy and management levels both at EU and Member States that should seek integration of EMODnet output and services in management and policy processes and that will decide upon its future sustained funding. In parallel further RTD work as taking place in SeaDataNet II will and must continue on standards and protocols that can be applied as basis for the expanding EMODNet portals. It can be considered that RI projects such as SeaDataNet and its sibbling Geo-Seas are establishing marine domain standards that are adopted and implemented at national data centres. EMODnet stimulates a wider implementation and adoption of these standards, in practice resulting in an expansion of the SeaDataNet infrastructure of connected data centres.

EMODNet next phase:
The EU has arranged funding for the coming 3 years (2012 - 2014) for further developing the EMODnet portals and undertaking supportive studies and activities, which together must result in a sustained and operational EMODNet from 2014 onwards. This should be supported by structured EU funding next to Member States funding. As part of this it is examined how the portals meet the needs of users from industry, public authorities and scientists. It is also an objective to identify data gaps and arguments why these gaps should be filled in future monitoring. Further efforts are being made to extend the geographic range in order to cover all of the waters of EU Member States for one or more sets of parameters.

EMODNet case: Hydrography

A very interesting and illustrative example of what one can achieve with the SeaDataNet approach can be found at the EMODNet Hydrography portal. Its development started in June 2009 and now provides a range of options for freely browsing and downloading new Digital Terrain Models (DTM) for a large part of the European seas. The downloadable tiles are freely available in a number of formats, including support for 3D viewers.

7_article_63_img_1.jpg (116.4 K) 

The EMODnet digital bathymetry with a gridsize of 0,25 * 0,25 minutes has been produced from bathymetric survey data and aggregated bathymetry data sets collated from public and private organizations. These are processed and quality controlled. Each cell of the DTM has statistics such as average, min, max depth, standard deviation, number of survey values used etc. A further refinement will take place in the next phase, by gathering additional survey data sets and improving its DTM resolution, and will result in new releases in time.

The portal includes a discovery service, based upon the SeaDataNet CDI standard, that gives clear information about the background survey data used for the DTMs, their access restrictions, originators and distributors. This way the portal provides originators of hydrographic data sets an attractive shop window for promoting their data sets to potential users, without losing control.

At present the inventory covers more than 8500 survey data sets, managed by 14 data centres from 9 countries and originated from 115 institutes (public authorities, hydrographic services and research institutes).

The portal also provides users a versatile viewing service for the produced EMODNet DTM with many relevant map layers and functions for retrieving depths, creating crossections, downloading DTM tiles, and fully integrated with the CDI service for surveys and a Product Catalogue (SEXTANT) service for used composite DTMs. In practice, each cell in the DTM contains a reference to the most prevailing survey or composite DTM used in its calculation, thus providing a direct link between data product and underlying data sets. This way the portal provides an attractive platform for data providers to advertise their survey data and for users to retrieve freely a high resolution DTM of the European seas with full traceability of data sets used and details of their providers.

EMODNet case: Physics

The EMODNet Physics project started early 2011 in a cooperation between EuroGOOS, MyOcean and SeaDataNet. Many EuroGOOS members are also partners in SeaDataNet; and EuroGOOS and SeaDataNet have an agreement for maintaining and operating the European Directory of Ocean Observing Systems (EDIOS). MyOcean and SeaDataNet have an MoU with the aim to make available a comprehensive dataset of in situ observations from both operational oceanography programmes and scientific surveys to serve both the Operational Oceanography and research communities as well as other users. MyOcean and SeaDataNet also strive for common standards.

EuroGOOS is an association of agencies to further the goals of
GOOS, and in particular the development of Operational Oceanography in the European Sea
areas and adjacent oceans. MyOcean is the implementation project of the GMES Marine Service, aiming at deploying the first concerted and integrated pan-European capacity for Ocean Monitoring and Forecasting.

EuroGOOS, MyOcean and SeaDataNet develop and operate infrastructures for acquiring,
archiving, and giving access to physical data from the global oceans and the regional and
coastal waters of the European maritime regions. These infrastructures are populated by many providers of ocean and marine physical data sets in Europe and thus are very qualified to provide the underlying basis for the EMODnet Physics portal.

7_article_64_img_1.jpg (59.8 K) 

The EMODnet Physical portal must improve for users the overview and access to in-situ physical data sets, both in near real-time and delayed mode. The EMODnet Physics portal will also encourage other physical data providers outside the present communities to come forward, to contribute and to become engaged. A comparable effect can already be seen in the other EMODnet preparatory actions.

7_article_4_img_1.jpg (62.1 K) 

These pillars comprise:

  • The EuroGOOS regional operational observing systems (ROOSes), most of them collaborating with regional Conventions (HELCOM, OSPARCOM, MAP)
  • The MyOcean in situ TAC providing access to physical near real time data acquired by continuous, automatic and permanent observation networks integrated in 6 regional portals and a global one operated by Coriolis
  • SeaDataNet infrastructure, providing access to long term physical time series residing in a distributed system of data centres and giving information about the monitoring stations and their networks / programmes via the European Directory of Ocean Observing Systems (EDIOS).

The EMODnet Physics portal limits itself to the near real-time and archived data sets of fixed stations and ferrybox routes, but an extension to other monitoring systems is foreseen in the next phase.

In the portal the station and track locations contain direct links to view and browse near real-time time series of selected parameters in charts and tables, that are derived from the EuroGOOS ROOSs / MyOcean in-situ TAC system, and to submit requests for downloading near real-time data sets from the in-situ TAC system. The charts and tables are freely available for a sliding time window of 60 days. The stations might also contain direct links to retrieve and browse metadata records from the SeaDataNet CDI service for requesting access to quality controlled and long term time series of selected parameters. This facilitates to download data sets in common formats for agreed requests.

Common Data Index (CDI) Data Discovery and Access Service

The Common Data Index (CDI) service gives users a highly detailed insight in the availability and geographical spreading of a large variety of marine and ocean data sets, that are managed by the 65+ data centres, that are now connected to the SeaDataNet infrastructure. Moreover it provides a unique interface for requesting access, and if granted, for downloading data sets from these distributed data centres across Europe.

There are 2 CDI user interfaces: 

  • CDI Quick Search interface that works with a drill-down method. The user has an immediate overview of the full range of CDI entries both on a map and by specific metadata fields. These fields show their terms and a number of their appearance in the CDI database in brackets. Clicking on any field or term will refresh the selection and other related fields and terms according to that filter. So the user can easily drill down and see the results. Besides the users can combine these with the universal spatial and temporal criteria.
  • CDI Extended Search interface that works with a combination of several criteria that can be set by the user with free text or terms from pull down menus. Once the criteria are set then the user activates the search and the results will be displayed. Specifying no criteria, but directly activating the search will show all CDI entries as results. Thereafter a user can refine its query criteria and do another search.

7_article_66_img_0.jpg (77.6 K) 

The operational CDI service this way gives access to a vast and rapidly increasing resource of marine and ocean datasets, managed by an increasing number of distributed data centres. At present it provides metadata and access to more than 1.000.000 data sets, originating from more than 300 organisations in Europe, covering physical, geological, chemical, biological and geophysical data, and acquired in European waters and global oceans. Already more than 65 data centres from 35 countries are connected. As part of SeaDataNet II and associated projects more data centres are connecting while also the volume and range of types of data sets is expanding.

7_article_66_img_1.gif (104.3 K) 

Figure: Overview of CDI entries per June 2012: >1.000.000 data sets from 300+ originators and 65+ connected data centres 

How does it work?
The process from search to getting access to requested data sets is illustrated and explained below. This allows users to retrieve data sets from distributed data centres in a common way via one unique portal.

7_article_66_img_2.gif (15.3 K) 

The 2 CDI User Interfaces allow to search by a set of criteria. Selected data sets are listed. Geographical locations are indicated on a map. Clicking on the display icon retrieves the full metadata of the data set. This gives information on the what, where, when, how, and who of the data set. It also gives standardised information on the data access restrictions, that apply. The interface features a shopping mechanism, by which selected data sets can be included in a shopping basket.

All users can freely query and browse in the CDI directory; however submitting requests for data access via the shopping basket requires that users are registered in the SeaDataNet central user register, thereby agreeing with the overall SeaDataNet User Licence. This registration also alllows users to submit a request for multiple data sets from multiple data centres in one go and to follow-up its processing via their personal user account.

All data requests are forwarded automatically from the SeaDataNet portal to the relevant data centres. This process is controlled via the Request Status Manager (RSM) service at the portal, that communicates with the data centres. Users receive a confirmation e-mail of their data set requests and a link to the RSM service. By log-in to the RSM service users can check regularly the status of their requests and download data sets from the associated data centres, after access has been granted. On their turn, data centres can follow via the RSM service all transactions for their data sets online and can handle requests which might require their mediation.

New developments in SeaDataNet II
The CDI user interfaces already have many functions for providing useful and attractive support to users and exchanges with other systems. In SeaDataNet II this will be extended with:

  • Machine to Machine interface services = Providing interoperability and automatic exchanges with other systems
  • Delivery services = Requesting and downloading of aggregated data sets
  • Delivery services = Access to real-time oceanography data 

Also efforts are underway for making the CDI format fully INSPIRE compliant.

Making the Common Data Index (CDI) format fully INSPIRE compliant

The SeaDataNet CDI format and service is adopted by several projects and programmes funded by the European Commission such as EMODNet pilots, Geo-Seas, Upgrade Black Sea SCENE,  EuroFleets, Jerico, etc. At present already 65+ NODC’s and marine data centres have successfully implemented the SeaDataNet CDI standard and are maintaining it as part of the CDI Data Discovery and Access service to give overview and access to their data sets. Therefore the CDI format should be considered a de-facto standard for marine metadata in Europe.

CDI concerns many components:

  • CDI metadata format;
  • CDI XML schema;
  • Common Vocabularies;
  • CDI Data Discovery and Access Service
  • MIKADO XML editor for generating CDI XML efficiently from local metadata
  • Validation service.

The CDI format is a marine profile of the ISO 19115 metadata content standard. However the present CDI Schema has been derived from the ISO 19115 DTD that was available at the start of SeaDataNet. Nowadays the ISO 19139 Schema is mature, which has been adopted together with CS-W as part of the INSPIRE Implementing Rules. Therefore one of the targets in SeaDataNet II is to upgrade the CDI Schema to ISO 19139 and to ensure INSPIRE compliance.

For this purpose an analysis has taken place of the present CDI format compliance with ISO 19115:2003 and taking into account specific additional constraints from the European Directive INSPIRE. This has resulted in a new profile documentation, adding also a number of missing elements. Furthermore a related document (SeaDataNet ISO 19115 profile – XML encoding) has been prepared which defines and details the XML schema implementation for this metadata profile, based on the XML schema defined in ISO 19139:2006 TS plus additional definitions and Schematron rules. The Schematron rules are defined to express XML constraints not enforceable by the schema, such as the use of SeaDataNet common vocabularies with controlled terms.

The new draft CDI Format and Schema are published at the SeaDataNet portal in the Standards & Software section. It is planned that it will be accepted at the coming SeaDataNet TTG meeting in September 2012 as new basis. Thereafter it will also be submitted to the Ocean Data Standards (ODS) project of IOC-IODE for sharing the CDI format and Schema as European marine profile with other ocean communities worldwide. 

It will be followed in the coming months by upgrading the related tools and services, such as MIKADO XML editor, CDI software, and validation service for taking the new CDI format and Schema into operation. Because the new CDI format is based upon the same content model it is possible to transform the present CDI metadatabase automatically, securing operations once upgrading. Using the ISO 19139 Schema will also allow for SeaDataNet adopting OGC CS-W services for upgrading the maintenance process of the CDI service. Data centres will be able to generate new CDI entries with a new MIKADO version, equipped with a CS-W component, so that the updating of the central portal can take place by harvesting from the distributed data centres.

SeaDataNet tools - new versions of NEMO and MIKADO

The software tools NEMO and MIKADO have been developed and are maintained to support data providers to generate standard metadata entries and standard data files.

NEMO
NEMO is a reformatting software used for data exchange from SeaDataNet data centres and portal to users. Its objective is to reformat ASCII files of vertical profiles (like CTD, Bottle, XBT) or time-series (like current meters, sea level data) or trajectories (like thermosalinograph data) to a SeaDataNet ASCII format (ODV or MEDATLAS). As the entry file can be all kinds of ASCII format, NEMO must be able to read all these formats to translate them to ODV or MEDATLAS.

Regularly new versions of NEMO are released via the SeaDataNet portal following project requirements and user feedback. For instance, recently NEMO has been updated as part of the Geo-Seas project for preparing ODV files for geological lithological logs using values from classification vocabularies in stead of the usual numeric values. The latest release of NEMO is Version 1.4.5 and it can be downloaded by any user without any constraints.

MIKADO
MIKADO is a ISO-19115 XML catalogue description generator used to create XML files for metadata exchange of the SeaDataNet catalogues: CSR, EDMED, CDI, EDMERP and EDIOS. MIKADO can be used in manual mode and in automatic mode, to generate XML files automatically if information is catalogued in a relational database or in an Excel file. 

Just like NEMO regularly new versions of MIKADO are released via the SeaDataNet portal following project requirements and user feedback, and can be downloaded by any user without any constraints. For instance, recently MIKADO has been updated as part of the Geo-Seas project for preparing extensions to the CDI format for describing use metadata for seismic survey data. For that purpose the OGC Observation & Measurement (O&M) and SensorML Schemas have been adopted that are linked to the CDI Schema. The latest release of MIKADO is Version 2.4.

These two tools are written in Java Language which means that they are portable under multiple environments: Windows (2000, XP, VISTA), Apple, Unix (Solaris) and Linux. They both use the SeaDataNet common vocabularies web services to insure consistency of generated metadata and data whatever the data centre which have processed them.

SeaDataNet II successful Training Workshop

The first SeaDataNet II training course took place at the IOC Project Office for IODE in Ostend, Belgium, from July 2 to 6, 2012. This session was co–organized by IOC-IODE and RBINS–MUMM and welcomed 10 lecturers and 41 trainees. The vast majority of the SeaDataNet II partners were represented.

The full range of SeaDataNet standards, tools and software has been covered. The first two days were targeting the data managers: common vocabularies, the metadata catalogues (including EDIOS and CSR), Nemo and Mikado. The next two days were devoted to Ocean Data View and DIVA, two powerful tools for data quality control and scientific analysis. The last day was devoted to the functioning of the SeaDataNet data discovery and “shopping” system: CDI’s, Download manager, Request Status Manager and the new system monitoring. To conclude, an introduction to the management of biological data gave the participants the flavour of the upcoming developments.

7_article_71_img_1.jpg (48.9 K)

From the preliminary analysis of the survey performed right at the end of the session, it appears that participants express a very high satisfaction level (despite the typical Belgian summer weather!). The organizers also gained some more experience and they intend to make the next SeaDataNet training session as successful as the first one.

The training course presentations are available on the SeaDataNet website.

The training material, including video recordings of the lectures, will be made available on “Ocean Teacher” after editing.

Geo-Seas extending SeaDataNet to marine geology and geophysics has made good progress

The EU FP7 Geo-Seas project, started in May 2009, has made good progress with implementing an e-infrastructure of 26 marine geological and geophysical data centres, located in 17 European maritime countries. In practice Geo-Seas is expanding the SeaDataNet infrastructure to handle marine geological and geophysical data, data products and services, creating a joint infrastructure covering both oceanographic and marine geoscientific data.

Geo-Seas has adopted the SeaDataNet infrastructure approach, and examined the fitness for purpose of the SeaDataNet standards for giving unified access to the geology and geophysics data sets. Thereby Geo-Seas also took into account the experience and developments arising from international geological projects, such as OneGeology and GeoSciML.

This way Geo-Seas has formulated and contributed to adaptations, overall enriching the SeaDataNet standards such as upgrading of the CDI format to handle also tracks and polygons in a detailed way using GML and additional service bindings in the CDI format for linking public (pre)viewing services. Additionally the common vocabularies have been expanded considerably in number of terms but also by adopting GeoSciML vocabularies, e.g. for classification of sediment types. Furthermore a dedicated guideline was prepared how to describe the results from geological analyses on a seabed sample or core in the SeaDataNet ODV format. SeaDataNet has defined ODV ASCII and NetCDF (CF) as standards for harmonised delivery of data sets through the downloading process, while MedATLAS is an optional format. For Geo-Seas an extended list of supported data formats for geological and geophysical was adopted. This has also contributed to upgrading of the SeaDataNet MIKADO tool and portal software.

In the case of seismic data it was necessary to include more metadata in the CDI to accommodate this data type. This was solved by extending the CDI format with Observation & Measurements and SensorML schemas. The O&M schema can contain the additional content fields, so called "user metadata", required for describing geophysical data such as seismic and side scan sonar which are supported by the common vocabularies. 1 CDI links to 1 O&M linking to 1 to many SensorML files. The Seismic CDIs can also give access to thumb nail low resolution images of the seismic data for previewing, before users decide to request access to the high resolution data using the CDI shopping mechanism.

The majority of the Geo-Seas data centres are now fully operational with 24 partners delivering data directly through the Geo-Seas discovery and access service. As a result there are now in excess 70,000 metadata records and corresponding data sets available via the Geo-Seas portal.

7_article_69_img_1.jpg (69.9 K) 

Seismic data has a specific commercial value. Therefore access to the high resolution seismic viewing service needs to be regulated because seismic data are normally only accessable to users after positive negotiation with their data providers. Therefore a high resolution viewing service for seismic data has been developed as an alternative to the standard downloading service. A web based viewer application has been developed that has to be installed at the data centres to give users regulated access for viewing seismic data that are managed at these data centres. Activities are now underway for integrating this high resolution seismic viewing service into the CDI Data Discovery and Access Service and for installing and configuring the service at the seismic data centres. It is foreseen that this alternative service for accessing these type of data sets will be operational from November 2012 onwards.

Geological and geophysical data comprise analytical data and data products which are derived from seabed sediment samples, boreholes, borehole samples, geophysical surveys (seismic, gravity, magnetic) of the seabed and sub-seabed, cone penetration tests, and sides can sonar surveys. As part of Geo-Seas a number of new data products and services are being developed with input from a user consultation and also in synergy with the on-going SeaDataNet II, One-Geology Europe, EMODNet Geology and EMODNet Hydrography projects:

  • Digital Terrain Model and 3D viewing software (available)
  • Digital Terrain Model and 2D viewing service (release October 2012)
  • Viewing services for geological logs (release September 2012)
  • Low resolution seismic viewing service (available)
  • High resolution seismic viewing service (release November 2012)
  • Standardization in seabed habitat mapping (available)

A 3D visualisation tool (3D Viewer) based on the existing open source NASA World Wind JSK application has been developed. This software is freely available after registration and allows the visualisation of Digital Terrain Models (DTM) in the existing CARAIBES NetCDF.

Furthermore to improve the usefulness of habitat maps for end-users, Geo-Seas has developed 2 guidelines for Seabed habitat mapping - sediment characterization resp Seabed habitat mapping - terrain characterization.

Geo-Seas International workshop - at University College Cork, Ireland - 9 & 10 OCTOBER 2012

The Geo-Seas project has implemented an e-infrastructure to allow better discovery and access to a range of harmonised marine geological and geophysical data from 26 federated European data centres. Through the development of an on-line portal and a suite of visualisation tools the Geo-Seas project provides a significant improvement in the discovery and access of marine geological and geophysical data for a range of users.

On the 9 & 10 OCTOBER 2012 the Geo-Seas project will host an international workshop to present the results of this European Commission funded project and provide demonstrations of these tools and services for the discovery, assessment and access to a range of marine geosciences datasets. The workshop will also include a number of keynote presentations by invited speakers from the international marine research community and other related fields.

Go to the Geo-Seas portal for more details and for pre-registering.

International Conference on Marine Data and Information Systems IMDIS 2013 - Lucca (Italy) - 23 - 25 SEPTEMBER 2013 - Call for Papers

The IMDIS series of Conferences is promoting the meeting of different communities working in informatics, data management, research, environmental protection, etc. It is focused on-line access to data, meta-data and products, communication standards and adapted technology to ensure platforms interoperability. IMDIS 2013 aims at providing an overview of the existing information on marine environmental data, and showing the progresses on development of efficient infrastructures for managing large and diverse data sets.

The IMDIS 2013 conference will be held in Lucca (Italy) in the San Romano Auditorium from Monday 23 September to Wednesday 25 September, 2013.

The Conference will be organised in four sessions:

  1. Marine information management
    • Exchange, processing and interactive work with marine data sets from highly heterogeneous sources
    • Federation and integration
    • Network services and technologies
  2. Marine environmental data bases: infrastructures and data access systems
    • Coastal and deep-sea operational oceanography metadata/data systems
    • Physical and bio-chemical databases for climate studies
    • Geophysical and geological metadata/data systems
  3. Data Services in ocean science
    • Standards and quality-assurance issues
    • Services and Visualisation tools
    • User oriented services
  4. Services for Users and Education
    • Historical evolution in data collection and management
    • Tools for dissemination
    • Test bed development for educational purposes
  7_article_73_img_1.jpg (33.4 K)

The IMDIS 2013 website is now operational and will provide you all information about the IMDIS 2013 conference: http://imdis2013.seadatanet.org/

Abstract submission for your oral or poster presentations will be available by the end of September 2012.

International collaboration - new ODIP project

Europe, the USA, Australia and IOC/IODE are making significant progress in facilitating the discovery and access of marine data through the development, implementation, population and operation of national, regional or international distributed ocean and marine observing and data management infrastructures such as SeaDataNet, Geo-Seas, IOOS, the Australian Ocean Portal and the IODE Ocean Data Portal. All of these developments are resulting in the development and implementation of standards for the formats of metadata, data, data products, quality control methods and flags, common vocabularies. They are also providing services for data discovery, viewing and downloading, and software tools for editing, conversions, communication, analysis and presentation, all of which are increasingly being adopted and used by their national and regional marine communities.

There is also a general trend towards the use of the basic ISO and OGC standards, however these allow the use of different profiles and vocabularies. As a result there are differences in the standards used in the different regions which hinder their direct exchange and use at an international and global scale, and as a result act as a barrier to the realisation of global portals such as the IODE Ocean Data Portal and GEOSS. There is therefore a need to undertake activities which lead to the development of common standards, where possible, and otherwise improve interoperability between the national and regional systems and services. However, it should be noted that implementing new standards can have serious implications for the existing infrastructures because these systems comprise many components that may need to be modified. In a number of fields, developments are just starting and there is still a lot of work to be done before practical standards are delivered that can be promoted on a wide scale, for example in the case of using OGC SensorML for real-time data exchange. In cases such as these the synergies within international collaborations can also have direct effects on the development and the later implementation of these standards in the national and regional infrastructures. 

Recently the EU has awarded a submission of an EU partnership, selected from SeaDataNet and Geo-Seas, together with IOC-IODE (in connection with its Open Data Standards (ODS) project) and key USA and Australian partners for the ODIP project. This Ocean Data Interoperability Platform (ODIP) project aims at establishing an EU / USA / Australia/ IOC-IODE coordination platform the objective of which will be achieving the interoperability of ocean and marine data management infrastructures, and to demonstrate this coordination through several joint EU-USA-Australia-IOC/IODE prototypes that would ensure persistent availability and effective sharing of data across scientific domains, organisations and national boundaries.

This way ODIP will contribute to removal of barriers hindering the effective sharing of data across scientific domains and international boundaries. In practice ODIP will organise international workshops to foster the development of common standards and develop prototypes to evaluate and test selected potential standards and interoperability solutions. The ODIP partnership will also provide a forum to harmonise the diverse regional systems, while advancing the European contribution to the global system.

Negotiations are being finalised and most probably ODIP will start end 2012.