More Information on the GCW Data Portal
The GCW Data Portal, hosted by the Norwegian Meteorological Institute ("MetNo") at http://gcw.met.no, is a mechanism to provide access to relevant cryosphere datasets and real-time data streams. It does not host data, although it could at some point in the future. Interoperability is critical to data sharing and to the success of GCW as an operational programme. Interoperability means the ability of systems of software to exchange and make use of information (e.g., interoperability between devices made by different manufacturers, or data stored at different data centres).
The Data Portal is being developed as a Distributed Data Management system, metadata driven, net centric (linkages with other data centres are vital, and implies the brokering of data and metadata), interdisciplinary (dataset agnostic in the open data space). It is expected to be a WMO Information System (WIS) Data Collection and Production Centre (DCPC), and to be compliant to the WMO Integrated Global Observing System (WIGOS) regulatory framework.
The metadata are generated by the data centres hosting the data sets, are harvested at regular intervals by the Data Portal using the Open Archives Initiatives Protocol for Metadata Harvesting (OAI-PHM ) and the Open GeoSpatial Consortium - Catalog Services for the Web (OGC CSW ), currently under testing, and are ingested in the central catalogue for usage by the GCW Portal user community. Currently, the portal does not utilize a distributed search (e.g., OpenSearch), as this is a slower process compared to searching in a central repository. The Data Management services of the Data Portal use OpenDAP (Advanced Software for Remote Data Retrieval) for integration of data.
The Data Portal links with a heterogeneous community of data centres (research and operational), with various degrees of interoperability for metadata, data, standardization, support, programing languages. GCW Data Portal is being used as the data portal backbone for the Year of Polar Prediction (YOPP), the Svalbard Integrated Arctic Earth Observing System (SIOS, https://sios-svalbard.org/), and the Data Portal of the Arctic Polar Regional Climate Centre (Arctic PRCC). These are currently under development.
The challenges being addressed as part of the Data Portal development include:
- Culture: While the operational communities are driven by compliance, and are more hierarchical, the research communities are driven by pragmatism, operating often as informal networks, based on guidelines (more than one way to accomplish results).
- Interoperability:
- For metadata, protocols and structures are available, though the availability and the consistency of semantics and terminology remain a significant gap. For the discovery of metadata and data, controlled vocabularies with definitions well accepted by the scientific community, are needed.
- For data, there are protocols for exchanging data, though formats, semantics/terminology and common data models (CDM) are major gaps. Standards for these need to be developed and adopted at large scale to facilitate the exchange. Legacy systems do not have to be changed, as long as they can be mapped in the common data models, which is a major priority.
As a WMO activity, the GCW Data Portal is committed to WIS, and the application of the ISO19115 WMO Profile (WIGOS metadata and WIS metadata) for the discovery of data. WIGOS metadata is currently under development, in particular regarding cryosphere variables. The WIGOS metadata describes, inter alia, the measured variables, as well as the temporal evolution of instrumentation, procedures of measurement, processing, local conditions, etc., which are required to fully understand the data.
The Data Portal includes data transformation functions which, once the data are available, will allow users to extract data and generate and compare products tailored for their specific needs, e.g. data series, mapping, and visualization.
