Latest revision as of 13:53, 6 July 2016

The GIS Publisher service manages geospatial data and metadata in a gCube empowered e-infrastructure.

Its goal is to deal with heterogeneous GIS platforms by offering a common interface to other web services / applications which need to publish geospatial data. Since each platform has a different approach on how to manage data, the GIS Publisher service is designed as a pluggable service, delegating to each plugin the platform specific interaction with a given platform.

The GIS Publisher service abstracts over all components of a typical Spatial Data Infrastructure (SDI). In the current implementation, it abstracts over (i) GeoServer, reference implementation of the Open Geospatial Consortium (OGC) Web Feature Service (WFS) and Web Coverage Service (WCS) standards, as well as a high performance certified compliant Web Map Service (WMS); (ii) GeoNetwork, the OGC recommended catalog supporting ISO19115/ISO19119/ISO19110 metadata standards represented and transported via the ISO19139 standard; and (iii) Thredds, that uses the Common Data Model (CDM) to read datasets in various formats, and serves them through OPeNDAP, OGC Web Coverage Service (WCS), NetCDF subset, and bulk HTTP file transfer services. The Unidata’s Common Data Model (CDM) merges the OPeNDAP, netCDF, and HDF5 data models to create a common API for many types of data. As currently implemented by the NetCDF Java library, it can read OPeNDAP, netCDF, HDF5, GRIB 1 and 2, BUFR, NEXRAD, and GINI, among others.

Overall, the GIS Publisher service allows to simplify and rely on a functionally-complete SDI allowing the user to obtain subsets of the data, which is crucial for large datasets. It relies on the gCube information system to gather information on which platforms are available in the current scope, balancing the load between the different platform instances. The GIS Publisher realises a distribution policy aiming at balancing the load between all instances of the services composing a scalable and fault-tollerant SDI, and it manages the replicas of the same services.

Overall Design

Applications which need to publish geospatial data can interact with the service via the GIS Publisher CL. A GISRequestConfiguration object is sent to the service, specifying :

• The GISData to publish
• The destination platform type
• Information to generate layers

The service enqueues every received request, assigning it a unique identifier. When a request is handled, the appropriate plugin is selected and invoked. At the end of the process a publish operation object is returned to the client, reporting information on :

• Published data references
• Published metadata
• Error messages (in case some of the operations fail)

CSquare codes Handling

CSquare codes referenced data are commonly used in geospatial applications, but they are not ready to be stored and published on GISServices as geospatial data. In fact they need to be coupled with appropriate geometries. Since this is a common logic in publishing csquarecoded data, the GISPublisher itself implement this, offering facilities to transform these data to the various plugins. The CSquareCodeDB is a postgis database containing various tables, each one representing csquarecodes as geometries in different resolutions.

Modules

The service is distributed as a set of maven artifacts:

• org.gcube.data.publishing.gis-publisher, parent for service and stubs implementation.
• org.gcube.data.publishing.gis-publisher-service, service implementation.
• org.gcube.data.publishing.gis-publisher-stubs, service stubs.
• org.gcube.data.publishing.gis-publisher-plugin-framework, library with common data model and plugin interfaces definition.
• org.gcube.data.publishgin.gis-publisher-client-library, client library to interact with the service.

GIS Publisher CL

Client library is developed on top of gCube CL Framework. It exposes synchronous and asynchronous methods to interact with the service.

Publising data

To publish data, callers need to specify two objects:

• GISRequestConfiguration object : among the other information, this object states
  • destination platform type (i.e GeoServer,THREDDS)
  • to generate styles definition : styles to generate prior layer generation
  • to generate layers definition : information to publish layers based on the passed data

• GISData object : the actual data to publish, it can be one of the following types
  • Streamed geospatial objects (see gCube Streams Library)
  • GISFile (i.e. CSquareCode CSV File, NetCDF etc.)

The following snippet shows how to publish a stream of geospatial objects (in this case cSquarePoints)

//Static import according to CL conventions
 
import static org.gcube.data.publishing.gis.publisher.client.plugins.AbstractPlugin.publisher;
....
 
 
//The actual execution code
 
Publisher publisher=publisher().build();
final ResultWrapper<CSquarePoint> rs=new ResultWrapper<CSquarePoint>(GCUBEScope.getScope(<SCOPE>));
Thread t=new Thread(){
	@Override
	public void run() {
		try {
			for(...){
                                // Read and instantiate points from file or other source
				rs.add(toSendPoint);
			}
		}catch(...){
		}finally{
			rs.close();
		}
	}
};
 
Stream<CSquarePoint> stream=pipe(convert(new URI(rs.getLocator())).of(GenericRecord.class).withDefaults()).through(new ResultGenerator<CSquarePoint>());
t.start();
 
// Generate Style configuration
 
StyleGenerationRequest styleReq=new StyleGenerationRequest();
....
 
// Generate Layer configuration
 
LayerGenerationRequest layerReq= new LayerGenerationRequest();
....
 
// RequestConfiguration Object to be passed along streamed data
 
GISRequestConfiguration config=new GISRequestConfiguration();
config.setRepositoryDestination(GISRepositoryType.GEOSERVER);
...
config.getToGenerateLayers().add(layerReq);
config.getToGenerateStyles().add(styleReq);
 
// Sending syncronous publish request
 
PublishingReport report=publisher.publishData(new StreamedGISData<CSquarePoint>(stream, CSquarePoint.class), config);
 
System.out.println(report);

GIS Publisher Framework Library

The library contains definition of classes shared by the service, the CL and plugins. It includes

• the data model needed to handle requests configuration
• the data model mapping geospatial data
• interface to be implemented by plugins

Implementing a plugin

To develop a plugin, one must declare a dependency on the following maven artifact :

<dependency>
 <groupId>org.gcube.data.publishing</groupId>
 <artifactId>gis-publisher-plugin-framework</artifactId>
 <version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version>
</dependency>

Two interfaces must be implemented by the plugin :

• org.gcube.data.publishing.gis.publisher.plugin.fwk.GISInteractionFactory : A factory of GISInteraction instance. Every request will be handled by a GISInteraction instance obtained calling the factory method public GISInteraction createInteraction(CSquareConverter factory) throws Exception

• org.gcube.data.publishing.gis.publisher.plugin.fwk.GISInteraction : interface defining the methods for actual publishing the data.

The implemented plugin jar must contain a text file named org.gcube.data.publishing.gis.publisher.plugin.fwk.GISInteractionFactory inside the folder /META-INF/services/. This file must contain the fully qualified name of the class implementing the plugin factory.

Available plugins

The following plugins have been developed :

• org.gcube.data.publishing.geoserver-publisher-plugin : interacts with GeoExplorer via the org.gcube.common.geoserverinterface library.