Revision as of 12:47, 19 December 2017

SDI-Service is a REST service aimed to offer a simplified interface to SDI administration over the infrastructure and to most common use cases in dealing with geospatial data. This page illustrates the interface exposed by the service and how to interact with it.

Interaction

Thanks to the SDI-Service application can gather details on SDI facilities offered by the infrastructure. Every response returned by the service is contextualized by the request's gcube-token meaning :

Configuration is related to the context (INFRASTRUCTURE, VO or VRE) of the request
Credentials and accessible data spaces are related to the current ROLES associated with the authenticated calling user

Invocations to SDI-Service may trigger on-the-fly initialization of SDI resources, where needed. Being a SmartGears service, it is automatically published in the infrastructure's Information System with coordinates :

Service Class : SDI
Service Name : sdi-service

Concepts

In this section some basic concepts are explained in order to guide users better exploit SDI-Service facilities.

Data Spaces

Every SDI application implements a feature that can be described as a Data Space, which is a categorization of the application content. Security and access policies are based on the notion of data space (which is implemented in different ways in the various SDI applications), so knowing where to publish data is a crucial factor in data-management.

GeoServer Data Spaces

GeoServer data spaces are implemented as workspaces and for each context the following workspaces are created / provided :

confidentialWorkspace : accessible only by CONTEXT_MANAGER accounts;
contextVisibilityWorkspace : accessible by users belonging to the context (CONTEXT_MANAGER, CONTEXT_USER and CKAN);
sharedWorkspace : accessible from other contexts;
publicWorkspace : publicly accessible;

GeoNetwork Data Spaces

GeoNetwork data spaces are implemented as groups (see here for further details on GeoNetwork security). Following groups are created / provided for each context :

contextGroup : accessible only by context users (CONTEXT_MANAGER, CONTEXT_USER and CKAN);
sharedGroup : accessible from other contexts;
publicGroup : publicly accessible;

Accounts

SDI-Service manages different kind of accounts and exposes them to the user depending on its ROLES in the infrastructure. Every account credential returned by the service has a type, which tells what kind of permissions that account has in its related application. Following is the list of possible Account Types :

CKAN : read only access to resources published in the current context;
CONTEXT_USER : read/write access to context data spaces;
CONTEXT_MANAGER : read/write access to context data spaces and confidential ones;
ADMIN: administration rights;

REST Invocation

SDI-Service is a SmartGears web application offering a REST interface, thus it can be invoked with simple HTTP requests. Every HTTP request to the service must contain a proper gcube-token property (as HTTP header or as query string parameter) in order to be authenticated and authorized. See #REST Interface for interface description.

JAVA Clients

For Java applications, a client library is distributed along with the service. The library facilitates the interaction with the service, automatically dealing with token and providing both an object model and a better fault management (see #Client library for more details). Such library is distributed as a maven artifact with the following coordinates

  <groupId>org.gcube.spatial.data</groupId>
  <artifactId>sdi-library</artifactId>

REST Interface

This section describes the interfaces exposed by SDI-Service and how to interact with it with HTTP requests. It is required that every request is authorized, meaning that it must contain a proper gcube token (see details here).

Exposed Interfaces

This section describes the interfaces exposed by SDI-Service, listing their methods and expected parameters. HTTP Response are returned and the content is represented in JSON.

The base path for every request , referred to as <BASE_PATH> from now on, is : https://<HOSTNAME>/sdi-service/gcube/service

SDI

The SDI Interface allows the management of the SDI resources as a whole, offering ways to :

Gather current configuration details
Retrieve Health Report

Following are the detailed description of said functionalities.

Get Configuration

PATH : <BASE_PATH>/SDI
METHOD : GET

This interface returns the complete configuration of the SDI in the caller's context. The following is a typical response from this interface:

```
{
```
```
  "contextName" : "...",
```
```
  "geonetworkConfiguration" : {
```
```
    "version" : {
```
```
      "major" : 2,
```
```
      "minor" : 6,
```
```
      "build" : 0
```
```
    },
```
```
    "baseEndpoint" : "...",
```
```
    "accessibleCredentials" : [ {
```
```
      "username" : "...",
```
```
      "password" : "...",
```
```
      "accessType" : "CKAN"
```
```
    }, {
```
```
      "username" : "...",
```
```
      "password" : "...",
```
```
      "accessType" : "CONTEXT_USER"
```
```
    }  ],
```
```
    "contextGroup" : "...",
```
```
    "sharedGroup" : "...",
```
```
    "publicGroup" : "..."
```
```
  },
```
```
  "geoserverClusterConfiguration" : {
```
```
    "availableInstances" : [ {
```
```
      "version" : {
```
```
        "major" : 2,
```
```
        "minor" : 1,
```
```
        "build" : 2
```
```
      },
```
```
      "baseEndpoint" : "...",
```
```
      "accessibleCredentials" : [ {
```
```
        "username" : "...",
```
```
        "password" : "...",
```
```
        "accessType" : "ADMIN"
```
```
      } ],
```
```
      "confidentialWorkspace" : ...,
```

      "contextVisibilityWorkspace" : ...,

```
      "sharedWorkspace" : ...,
```
```
      "publicWorkspace" : ...
```
```
    } ]
```
```
  },
```
```
  "threddsConfiguration" : ...
```
```
}
```

Retrieve Health Report

PATH : <BASE_PATH>/SDI/status
METHOD : GET

This interface triggers a set of checks performed over both the IS and the SDI services. Performed checks are :

for each GCore Endpoint of an SDI service, a related configuration Service Endpoint is expected;
for each Service Endpoint configuring an SDI service, a ready GCore Endpoint is expected;

Output format is XML in order to be parsed for programmatic monitoring. In particular, overallStatus and level elements are enumerators which values can be only one between WARNING,ERROR,OK. Following is an example of a Health Report :

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<healthReport>
    <overallStatus>WARN</overallStatus>
    <context>...</context>
    <thredds>
        <overallStatus>ERROR</overallStatus>
        <checkReports>
            <message>Unregistered THREDDS instances found. Check following messages</message>
            <level>ERROR</level>
        </checkReports>
        <checkReports>
            <message>Found unregistered THREDDS hosted on thredds-d-d4s.d4science.org</message>
            <level>WARNING</level>
        </checkReports>
    </thredds>
    <geonetwork>
        <overallStatus>OK</overallStatus>
    </geonetwork>
    <geoserverCluster>
        <overallStatus>ERROR</overallStatus>
        <checkReports>
            <message>Service endpoint [name = GeoServer Migration, host = geoserver1-d-d4s.d4science.org ID = 97bb47ef-db9f-4e89-9af4-c0f857150fad] found but no related GC is present.</message>
            <level>ERROR</level>
        </checkReports>       
    </geoserverCluster>
</healthReport>

GeoServer

This interface is dedicated to handling the GeoServer Cluster of the SDI by offering ways to :

Get Configuration for GeoServer instance
Get Credentials for GeoServer instance
Register GeoServer Instance
Import GeoServer Registration

PATH : <BASE_PATH>/GeoServer

Configuration for GeoServer instance

PATH : <BASE_PATH>/GeoServer/configuration/<HOST_NAME>
METHOD : GET

This interface exposes the same information as per #SDI, limiting it to only the GeoServer instance hosted in <HOST_NAME>. The following is a typical response :

```
{
```
```
      {
```
```
  "version" : {
```
```
    "major" : ..,
```
```
    "minor" : ..,
```
```
    "build" : ..
```
```
  },
```
```
  "baseEndpoint" : ...,
```
```
  "accessibleCredentials" : [ {
```
```
    "username" : "...",
```
```
    "password" : "...",
```
```
    "accessType" : "..."
```
```
  } ],
```
```
  "confidentialWorkspace" : ...,
```
```
  "contextVisibilityWorkspace" : ...,
```
```
  "sharedWorkspace" : ...,
```
```
  "publicWorkspace" : ...
```
```
}
```

Credentials for GeoServer instance

PATH : <BASE_PATH>/GeoServer/credentials/<HOST_NAME>
METHOD : GET

This interface exposes the same information as per #Configuration, limiting it to only accessible credentials. The following is a typical response :

```
{
```
```
        "username" : "...",
```
```
        "password" : "...",
```
```
        "accessType" : "ADMIN"
```
```
    }
```

Register GeoServer instance

PATH : <BASE_PATH>/GeoServer
METHOD : POST

This interface allows the registration of a SmartGears-enabled GeoServer already running in the caller's context. The following is an example of a Registration Request content :

```
{
```
```
  "hostname" : "some.place.org",
```
```
  "majorVersion" : 2,
```
```
  "minorVersion" : 10,
```
```
  "releaseVersion" : 3,
```
```
  "adminPassword" : "*****",
```
```
  "properties" : [ {
```

    "name" : "my own application property",

```
    "value" : "some value"
```
```
  } ],
```
```
  "description" : "Dummy geoserver",
```
```
  "name" : "My GeoServer",
```
```
  "workspaces" : [ {
```
```
    "name" : "myWS",
```
```
    "access" : "PUBLIC"
```
```
  } ]
```
```
}
```

Import GeoServer Registration

PATH : <BASE_PATH>/GeoServer/import/<HOST_NAME>?sourceToken=*****
METHOD : POST

This interface allows the registration of a SmartGears-enabled GeoServer already running in the caller's context by importing its registration from another context (accessed by exploiting the passed sourceToken).

GeoNetwork

This interface is dedicated to handling the GeoNetwork Cluster of the SDI by offering ways to :

Get Configuration for GeoNetwork instance
Get Credentials for GeoNetwork instance
Register GeoNetwork Instance
Import GeoNetwork Registration

PATH : <BASE_PATH>/GeoNetwork

Configuration for GeoNetwork instance

PATH : <BASE_PATH>/GeoNetwork/configuration/<HOST_NAME>
METHOD : GET

This interface exposes the same information as per #SDI, limiting it to only the GeoNetwork instance hosted in <HOST_NAME>. The following is a typical response :

```
{
```
```
      {
```
```
  "version" : {
```
```
    "major" : ..,
```
```
    "minor" : ..,
```
```
    "build" : ..
```
```
  },
```
```
  "baseEndpoint" : ...,
```
```
  "accessibleCredentials" : [ {
```
```
    "username" : "...",
```
```
    "password" : "...",
```
```
    "accessType" : "..."
```
```
  } ],
```
```
  "confidentialWorkspace" : ...,
```
```
  "contextVisibilityWorkspace" : ...,
```
```
  "sharedWorkspace" : ...,
```
```
  "publicWorkspace" : ...
```
```
}
```

Credentials for GeoNetwork instance

PATH : <BASE_PATH>/GeoNetwork/credentials/<HOST_NAME>
METHOD : GET

This interface exposes the same information as per #Configuration, limiting it to only accessible credentials. The following is a typical response :

```
{
```
```
        "username" : "...",
```
```
        "password" : "...",
```
```
        "accessType" : "ADMIN"
```
```
    }
```

Register GeoNetwork instance

PATH : <BASE_PATH>/GeoNetwork
METHOD : POST

This interface allows the registration of a SmartGears-enabled GeoNetwork already running in the caller's context. The following is an example of a Registration Request content :

```
{
```
```
  "hostname" : "some.place.org",
```
```
  "majorVersion" : 2,
```
```
  "minorVersion" : 10,
```
```
  "releaseVersion" : 3,
```
```
  "adminPassword" : "*****",
```
```
  "properties" : [ {
```

    "name" : "my own application property",

```
    "value" : "some value"
```
```
  } ],
```
```
  "description" : "Dummy GeoNetwork",
```
```
  "name" : "My GeoNetwork",
```
```
  "workspaces" : [ {
```
```
    "name" : "myWS",
```
```
    "access" : "PUBLIC"
```
```
  } ]
```
```
}
```

Import GeoNetwork Registration

PATH : <BASE_PATH>/GeoNetwork/import/<HOST_NAME>?sourceToken=*****
METHOD : POST

This interface allows the registration of a SmartGears-enabled GeoNetwork already running in the caller's context by importing its registration from another context (accessed by exploiting the passed sourceToken).

Metadata

The Metadata Interface allows and facilitates the publication of OGC Metadata files in 3 steps :

upload the raw metadata
enrich the uploaded metadata by applying available templates
publish the enriched metadata onto GeoNetwork

Following are explained the methods exposed by this interface.

Upload Metadata

PATH : <BASE_PATH>/Metadata
METHOD : POST

This interface allows the upload of a metadata file into the service, in order to be eventually enriched with templates and published into GeoNetwork. It expects a multipart/form-data content with the metadata file in the field theMeta. The response is a ticket uuid that needs to be used in further publication steps as Apply Template and Publish Metadata. Temporary uploaded files are stored in the temporary directory and their TTL can configured in service configuration file.

Get Available Templates

PATH : <BASE_PATH>/Metadata/list
METHOD : GET

This interface returns a JSON list of descriptor for all available templates. A typical response to this interface is as following :

[
  {
    "templateID": "THREDDS_ONLINE_RESOURCES",
    "name": "Thredds Online Resources",
    "description": "Template for online resources exposed by thredds.",
    "sourceURL": "http://sdi-d4s.d4science.org",
    "parameters": [
      {
        "name": "catalog",
        "value": "The thredds catalog name"
      },
      {
        "name": "filename",
        "value": "The dataset's file name"
      },
      {
        "name": "hostname",
        "value": "Thredds hostname"
      }
    ]
  }
]

Apply Templates

PATH : <BASE_PATH>/Metadata/<UPLOADED_TICKET_UUID>
METHOD : PUT

This interface allows to apply a collection of templates to the previously uploaded metadata file specified by the <UPLOADED_TICKET_UUID>. Expected body of request, contains a collection of TemplateInvocation objects that may vary depending on the related template. Following is a template invocation for THREDDS_ONLINE_RESOURCES template

{
  "invocationSet" : [ {
    "templateID" : "THREDDS_ONLINE_RESOURCES",
    "parameters" : [ {
      "name" : "hostname",
      "value" : "localhost"
    }, {
      "name" : "filename",
      "value" : "myDataset.nc"
    }, {
      "name" : "catalog",
      "value" : "my Catalog"
    } ]
  } ]
}

Publish Metadata

PATH : <BASE_PATH>/Metadata/publish/<UPLOADED_TICKET_UUID>/<GEONETWORK_CATEGORY>
METHOD : GET

This interface allows to publish a previously uploaded metadata file specified by the <UPLOADED_TICKET_UUID> to the current GeoNetwork instance in a <GEONETWORK_CATEGORY>. It is also possible to specify the following options as query parameters :

validate : Boolean value, default is true. Enables GeoNetwork validation on publication.
public : Boolean value, default is false. Adds the published metadata to the public group of GeoNetwork, making it available without authentication.
stylesheet : String value, default is _none_. GeoNetwork stylesheet to be applied on publication.

The response to this interface contains IDs generated for the published Metadata inside GeoNetwork. Following is a typical response :

{
  "publishedUUID" : "theUUID",
  "publishedID" : 12335,
  "appliedTemplates" : [ "THREDDS_ONLINE_RESOURCES" ]
}

Thredds

Thredds interface allows the management of Thredds catalogs in the caller's context.

Create Catalog

PATH : <BASE_PATH>/Thredds
METHOD : PUT

Expected query parameters :

name : String value, the name of the new Catalog
path : The http path at which the catalog will be available on Thredds
folder : The subfolder which will be the root for the new Catalog

This interface allows the creation/update of a catalog in the currently available Thredds instances.

Call to this interface triggers the following behaviour :

Locally instantiation of the thredds catalog xml template to create the new catalog file
Invocation of data-transfer-service on Thredds instance to register catalog file (see displayed text for more information)
Invocation of Thredds Remote Management interface for catalog reinit (see [| thredds documentation]) for more information.
200 Http response is returned along with a Thredds Catalog descriptor (see )

Typical use cases

Get global configuration

Get available GeoServers

Get credentials for a particular GeoServer

Client Library

@@ Line 415: / Line 415: @@
 ===Thredds===
+Thredds interface allows the management of Thredds catalogs in the caller's context.
 ====Create Catalog====
+*PATH : <BASE_PATH>/'''Thredds'''
+*METHOD : '''PUT'''
+Expected query parameters :
+*'''name''' : String value, the name of the new Catalog
+*'''path''' : The http path at which the catalog will be available on Thredds
+*'''folder''' : The subfolder which will be the root for the new Catalog
+This interface allows the creation/update of a catalog in the currently available Thredds instances.
+Call to this interface triggers the following behaviour :
+# Locally instantiation of the thredds catalog xml template to create the new catalog file
+# Invocation of data-transfer-service on Thredds instance to register catalog file (see [[Page name#Section name|displayed text]] for more information)
+# Invocation of Thredds Remote Management interface for catalog reinit (see [[https://www.unidata.ucar.edu/software/thredds/current/tds/reference/RemoteManagement.html | thredds documentation]]) for more information.
+# 200 Http response is returned along with a Thredds Catalog descriptor (see )
 ==Typical use cases==

Difference between revisions of "Interaction with SDI-Service"