Difference between revisions of "URI Resolver"
(→KNIME Resolver) |
(→Analytics Resolver) |
||
Line 401: | Line 401: | ||
* '''AnalyticsExecutor''' GR as ApplicationProfile published at VRE level in the gCube Information System; | * '''AnalyticsExecutor''' GR as ApplicationProfile published at VRE level in the gCube Information System; | ||
− | * '''Analytics-Resolver''' SE published at | + | * '''Analytics-Resolver''' SE published at ROOT level in the gCube Information System. |
===== HTTP POST: Create an Analytics URL ===== | ===== HTTP POST: Create an Analytics URL ===== |
Revision as of 15:43, 28 December 2018
The URI Resolver is a RESTful service which gives access via HTTP(s) to different gCube Resolver services and gCube Applications (like GisViewer) used into the D4Science Infrastructure.
Contents
- 1 Maven
- 2 Runtime Resource
- 3 Query
- 4 Retrieve web service endpoint
- 5 Resolvers
- 6 Uri Resolver Manager
Maven
The artifact is available on Nexus with the following coordinates:
<groupId>org.gcube.data.transfer</groupId> <artifactId>uri-resolver</artifactId> <version>LATEST</version> <type>war</type>
Runtime Resource
The address and the configuration information of the servlet are registered on the gCube Information System as Runtime Resource (RR).
You can see the configuration through Monitor service (https://services.d4science.org/infrastructure-monitor) searching in the root scope the RR with following:
<Type>RuntimeResource</Type> <Category>Service</Category> <Name>HTTP-URI-Resolver</Name>
You must copy such runtime resource in VRE scope if it is need to work at VRE level
Query
The following code snippet can be used to retrieve from the gCube Information System the info to configure the resolver
ScopeProvider.instance.set(<scope>); XQuery query = queryFor(ServiceEndpoint.class); query.addCondition("$resource/Profile/Name/text() eq 'HTTP-URI-Resolver'").setResult("$resource/Profile/AccessPoint"); DiscoveryClient<AccessPoint> client = clientFor(AccessPoint.class); List<AccessPoint> endpoints = client.submit(query); if (endpoints.size() == 0) throw new Exception("No Resolver available"); //Base Address System.out.println(endpoints.get(0).address()); //Query URL parameter System.out.println(endpoints.get(0).propertyMap().get("SMP_URI_parameter").value());
Retrieve web service endpoint
The service endpoint URL [URI_RESOLVER_SERVICE_ENDPOINT] of Resolver services is:
for the development environment
http://data-d.d4science.org
for the production environment
http://data.d4science.org
Resolvers
SMP Resolver
The first version of the component is able to give HTTP resolution to SMP protocol URIs [1].
Once deployed in a SmartGear container as Tomcat, the service can be used to resolve SMP URIs as follows:
URL url = new URL("http://<hostname>:<port>/uri-resolver/smp?smp-uri=smp://.....);
The HTTP URI can be also used to retrieve files using wget or trough a Web Browser:
[URI_RESOLVER_SERVICE_ENDPOINT]/smp?smp-uri=[STORAGE_URI]&fileName=[FILENAME]&contentType=[CONTENT_TYPE]
Apart from the smp-uri mandatory parameter, two optional paramters can be used:
- fileName : to specify the output file name ( default: fromStorageManager)
- contentType : to specify the output file content-type ( default: unknown/unknown)
SMP-ID Resolver
The resolver is able to give HTTP resolution to storage resource by ID [2].
[URI_RESOLVER_SERVICE_ENDPOINT]/id?smp-id=[STORAGE_SMP_ID]&fileName=[FILENAME]&contentType=[CONTENT_TYPE]
For instance, in the development environment you can download the following file with WGET command or through a Web Browser by using the link:
http://data-d.d4science.org/id?smp-id=553f9265e4b0567b75021fce&fileName=dog&contentType=image%2Fjpg
Apart from the smp-id mandatory parameter, SMP-ID Resolver uses two optional parameters (in the query string):
- fileName : to specify the output file name (default: fromStorageManager)
- contentType : to specify the output file content-type (default: unknown/unknown)
The two default cases above will be used if and only if the file name and content-type (mime type) cannot be read from metadata of the file saved in the Storage.
Therefore, in this order:
- Either fileName is read from query string or is read from storage metadata otherwise is default: "fromStorageManager"
- Either contentype is read from query string or is read from storage metadata otherwise is default: "unknown/unknown"
STORAGE-ID Resolver
This works as #SMP-ID Resolver using links having the format:
[URI_RESOLVER_SERVICE_ENDPOINT]/[STORAGE_ID]
HEAD HTTP: Payload validation
STORAGE-ID Resolver implements the HTTP doHead operation. It performs a payload validation (checking if it exists in the storage area) for the resource ID required:
- if the payload exists: it returns status 200 the content-type and content-disposition of the file in the HTTP response;
- if the payload does not exist: it returns status 404 in the HTTP response.
STORAGE-HUB Resolver
The StorageHub Resolver is reachable at the following path:
[URI_RESOLVER_SERVICE_ENDPOINT]/shub
It provides the methods either to download a gCube Workspace file or to get its metadata.
HTTP GET: Download a file
This method is reachable at:
[URI_RESOLVER_SERVICE_ENDPOINT]/shub/[STORAGE_HUB_FILE_ID]
and allows the download of the gCube Workspace file with ID [STORAGE_HUB_FILE_ID] via the StorageHub facility.
HTTP HEAD: Get metadata of a file
This method is reachable at:
[URI_RESOLVER_SERVICE_ENDPOINT]/shub/[STORAGE_HUB_FILE_ID]
and allows to get the metadata of the gCube Workspace file with ID [STORAGE_HUB_FILE_ID] (the metadata are retrieved from the StorageHub component).
The response returns an HTTP Status 204 (NO_CONTENT, so it DOES NOT return a message-body) and adds the headers related to STORAGE_HUB_FILE_ID listed below:
- "Content-Disposition";
- "Content-Location";
- "Content-Type" if not empty;
- "ETag" that returns the entity version.
GIS Resolver
This component is able to resolve GIS Links (as a redirect to Gis Viewer Application) in order to display and navigate on web-map the gis layers stored in iMarine Geonetwork #iMarine_GeoNetwork
HTTP GET: Resolve a Gis Link
The Gis Resolver can be used to resolve Gis Layer using a Web Browser link as follows:
[URI_RESOLVER_SERVICE_ENDPOINT]/gis?gis-UUID=[UUID]&scope=[SCOPE]
The link must include two mandatory parameters:
- gis-UUID : to specify the Metadata Universally Unique Identifier (UUID - used in iMarine Geonetwork). The UUID identifies the gis layer which you want to show in Gis Viewer Application
- scope : to specify the "scope" for discovering the iMarine GeoNetwork service (e.g. scope=/gcube/devsec/devVRE)
Runtime Resource
In order to work, it is necessary to copy from Root scope (/d4science.research-infrastructures.eu) to the VRE where Gis-Resolver must work, the runtime resource with following coordinates:
<Type>RuntimeResource</Type> <Category>Service</Category> <Name>Gis-Resolver</Name>
Generic Resource for Gis Viewer Application
It is necessary to edit the generic resource "Gis Viewer Application" in the Root scope (/d4science.research-infrastructures.eu) adding the VRE scope where the GisViewerApplication must work.
The Generic Resource "Gis Viewer Application" coordiantes are:
<Type>GenericResource</Type> <SecondaryType>ApplicationProfile</SecondaryType> <Name>Gis Viewer Application</Name>
CATALOGUE Resolver
This works as an HTTP Resolver to get/resolve a link to a "Catalogue Entity" stored in one of the instances of gCube Data Catalogue
A Catalogue Entity is either a "group" or an "organization" or a "product" of gCube Data Catalogue.
HTTP POST: Create an Item URL
The resolver provides HTTP POST operation to build a link to a "Catalogue Entity" stored into a specific gCube Data Catalogue.
In order to give a catalogue link for a catalogue entity is needed to pass in the body of the POST request a JSON with the fields:
- gcube_scope as mandatory field. It is the gcube scope where to search the gCube Catalogue Portlet;
- entity_context as mandatory field. It can be: "dataset" or "product", "group", "organization";
- entity_name as mandatory field. It is the entity name, that is its CKAN URL.
Where the JSON format is the following one:
{ "gcube_scope" : "[gcube_scope value]", "entity_context" : "[entity_context value]", "entity_name" : "[entity_name value]" }
How to create the Catalogue Link
You need to perform a POST request to the resolver
[URI_RESOLVER_SERVICE_ENDPOINT]/catalogue
by using:
- the header parameter Content-Type: application/json;
- the body with the JSON format as above specified (the response returns an HTTP link).
An instance of body could be [ex. 1]:
{ "gcube_scope" : "/gcube/devsec/devVRE", "entity_context" : "dataset", "entity_name" : "sarda-sarda" }
The HTTP Response returned:
The response returns in the body a "brief URL" in clear of kind:
[URI_RESOLVER_SERVICE_ENDPOINT]/[ctlg|ctlg-p|ctlg-d|ctlg-o|ctlg-g]/[VRE_NAME]/[entity_name value]
Where:
ctlg | ctlg-p | ctlg-d | ctlg-o | ctlg-g are codes to resolve the couple "catalogue"-"entity_context_value", in detail:
ctlg -> this one is default value to refer a 'product'|'dataset'
ctlg-p|d -> to refer respectively a 'product'|'dataset'
ctlg-o -> to refer an 'organization'
ctlg-g -> to refer a 'group'
For instance, using the request of [ex. 1] in the development environment, the HTTP response returns in the body the URL:
http://data-d.d4science.org/ctlg/devVRE/sarda-sarda
HTTP GET: Resolve an Item URL
The resolver provides HTTP GET operation to give HTTP resolution (as HTTP redirect) for displaying the 'entity_name' through an instance of "gCube Data Catalogue Portlet" (this last one is resolved by the 'gcube_scope' used to generate the catalogue link)
Geonetwork Resolver
The 'Geonetwork Resolver' is a "middle tier" (like a Proxy) to perform queries on gCube Geonetwork services. It uses the 'Geonetwork Manager library' to discovery Geonetwork configurations like: Geonetwork service end-point, Geonetwork accounts and so on. Using the Geonetwork configurations discovered the resolver performs authentication on Geonetwork (if needed) and then forward the requests received to harvesting metadata on that Geonetwork. Authentication is needed to get layers stored as private on Geonetwork.
The resolver is reachable at the following:
[URI_RESOLVER_SERVICE_ENDPOINT]/geonetwork
later referred like "GEONETWORK_RESOLVER_BASE_URL".
The 'Geonetwork Resolver' provides the HTTP GET method that supports the following protocol:
http[s]://[GEONETWORK_RESOLVER_BASE_URL]/[SCOPE]/[MODE]/[VISIBILITY]/[FILTER_KEY]/[FILTER_VALUE]/$$
In detail, the protocol parameters are:
- SCOPE (mandatory): the gCube scope used via 'Geonetwork Manager library' to discovery the Geonetwork service end-point and other configurations (like Geonetwork accounts and so on). It must be separated by character pipe '|'. i.e. gcube|devNext|NextNext;
- MODE (mandatory): must be 'VRE' or 'HARVEST'. To harvest at VRE level or External GEONETWORK level;
- VISIBILITY (mandatory): must be 'PUB' or 'PRV'. When PUB: only public layer/s is/are returned. When PRV: only private layer/s is/are returned;
- FILTER_KEY/FILTER_VALUE (mandatory): couple key/value passed in the path to filter on one of geonetwork:info field returned by ISO19139 schema. Only layer/s complying the filter criteria are returned (i.e. The filter category/interactiveResources returns only the layers belonging to GN category "Interactive resources"). The default value must be null/null if you don't want to apply any filter;
- REQUEST DELIMITER (mandatory). must be '$$'. It is the delimiter to specify the end of the path.
i.e. CKAN harvester configured to retrieve metadata by the URL: https://data.d4science.org/geonetwork/d4science.research-infrastructures.eu%7CgCubeApps%7CRPrototypingLab/VRE/PRV/null/null/$$
Where:
- GEONETWORK_RESOLVER_BASE_URL has value: https://data.d4science.org, so the Geonetwork Resolver service endepoint is: https://data.d4science.org/geonetwork;
- SCOPE has value: 'd4science.research-infrastructures.eu|gCubeApps|RPrototypingLab';
- MODE has value: 'VRE';
- FILTER_KEY has value: null and FILTER_VALUE has value: null;
- VISIBILITY has value: 'PRV';
It retrieves all private layers of the VRE RPrototypingLab.
Parthenos URL Resolver
(DRAFT) This resolver is reachable at:
[URI_RESOLVER_SERVICE_ENDPOINT]/parthenos_registry
It exposes the HTTP methods:
- GET - to resolve an "item" from a Parthenos URL stored in the Parthenos Catalogue at https://ckan-parthenos.d4science.org/
- POST - to get a normalized item name from a Parthenos URL. It applies a normalization function to get from an input Parthenos URL an item name that is catalogue compliant.
HTTP POST: Get normalized Item Name
The resolver provides HTTP POST operation to request a normalized item name for an input Parthenos URL.
It reads the input Parthenos URL (from the 'entity_name' field of a JSON object) and returns its normalized name (as plaintext) to the Parthenos Catalogue URL.
You need to perform a POST request to the resolver parthenos_registry by adding the HTTP header Content-Type: application/json and passing in the body a JSON object with the following format:
{ "entity_name" : "[PARTHENOS entity_name value]" }
An instance of body could be:
{ "entity_name": "/Culturalitalia/unknown/Dataset/oai%3Aculturaitalia.it%3Aoai%3Aculturaitalia.it%3Amuseiditalia-mus_11953" }
and in this case the POST response returns:
- as plaintext in the body: culturalitalia_unknown_dataset_oaiculturaitaliaitoaiculturaitaliaitmuseiditalia_mus_11953
- as Location parameter (in the HTTP header): https://parthenos.d4science.org/group/parthenos_registry/catalogue?path=/dataset/culturalitalia_unknown_dataset_oaiculturaitaliaitoaiculturaitaliaitmuseiditalia_mus_11953
HTTP GET: Resolve a Parthenos URL
The resolver provides HTTP GET operation to return the Parthenos Catalogue URL of an "item" stored as Parthenos URL in the (Parthenos) Catalogue
You simply by resolving via browser the HTTP link of a Parthenos URL (stored in the Parthenos Catalogue), you are performing a GET request to the resolver parthenos_registry to get its Parthenos Catalogue URL.
Analytics Resolver
(under construction)
The resolver provides a method to get the Analytics URLs (that are simply URLs) by which is possible to invoke through the DataMiner Executor the run of an algorithm stored in a certain VRE. By the DataMiner Executor then the user is able to monitoring the algorithm execution.
The Analytics Resolver is reachable at:
[URI_RESOLVER_SERVICE_ENDPOINT]/analytics
It exposes the HTTP methods:
- GET - to resolve an Analytics URL (created by POST method) that invokes the "run" operation for an algorithm stored in a VRE;
- POST - to generate an Analytics URL (then resolved by GET method) for a certain algorithm of a VRE.
Dependencies: in order to work correctly this resolver needs to:
- AnalyticsExecutor GR as ApplicationProfile published at VRE level in the gCube Information System;
- Analytics-Resolver SE published at ROOT level in the gCube Information System.
HTTP POST: Create an Analytics URL
This resolver (protected by AuthZ) is reachable at the following:
[URI_RESOLVER_SERVICE_ENDPOINT]/analytics/create
It provides the HTTP POST method to create Analytics URLs. In order to obtain an Analytics URL, you need to know your token (in the VRE), the algorithm identifier and the parameters to call the algorithm/operator in the VRE.
If you know these things, you can perform a POST request to the resolver /analytics/create by adding the HTTP header gcube-token: [YOUR_VRE_TOKEN] and passing in the body a JSON object with the following format:
# THE DAMANIER INVOCATION MODEL { "dataminer-invocation": { "operator-id": "[THE_OPERATOR_ID]", "action": "RUN", "parameters": { "input": { "param": [ { "key": "[key1]", "value": "[value1]" }, { "key": "key2", "value": "value2" }, ... { "key": "[keyN]", "value": "[valueN]" } ] }, "output": { "param": [ { "key": "[keyA]", "value": "[valuA]" }, { "key": "[keyB]", "value": "[valueB]" }, ... { "key": "[keyZ]", "value": "[valueZ]" } ] } } } }
Where:
- "operator-id" field is Mandatory. It is the DataMiner Operator Identifier;
- "action" field is Mandatory (its value must be "RUN" or "EDIT"). Currently only the "RUN" value working with DataMiner Executor;
- "parameters" field is Optional. It is a list of "input" and "output" parameters used to invoke/call the Operator Identifier operator-id through the DataMiner Executor.
Performing the POST request to the resolver /analytics/create with the JSON object "dataminer-invocation" as presented above, the resolver returns as plaintext in the body (and as value of header parameter Location) an Analytics URL with the following format:
[URI_RESOLVER_SERVICE_ENDPOINT]/analytics/get/[VRE_NAME]?dim=[PUBLIC_URL_TO_DATAMIER_INVOCATION_FILE_GENERATED_BY_POST_REQUEST]
e.g. By using a token of "/gcube/preprod/preVRE" VRE and passing the following "dataminer-invocation" JSON object in the body :
{ "dataminer-invocation": { "operator-id": "org.gcube.dataanalysis.wps.statisticalmanager.synchserver.mappedclasses.transducerers.MPA_INTERSECT", "action": "RUN", "parameters": { "input": { "param": [ { "key": "fileId", "value": "http://publicLinkToFile" } ] } } } }
the POST response returns as plaintext in the body (and as value of header parameter Location) the following Analytics URL [ex.1]:
HTTP GET: Resolve an Analytics URL
This resolver is reachable at
[URI_RESOLVER_SERVICE_ENDPOINT]/analytics/get
It provides the HTTP GET method to resolve an Analytics URL that supports the following protocol:
[URI_RESOLVER_SERVICE_ENDPOINT]/analytics/get/[VRE_NAME]?dim=[PUBLIC_URL_TO_DATAMIER_INVOCATION_FILE_GENERATED_BY_POST_REQUEST]
e.g. By clicking (so simply resolving it via browser) on the previous Analytics URL [see ex.1] it will be resolved as:
https://pre.d4science.org/group/prevre/dataminer-manager?dim=https://ADD_ANALYTICS_URL
KNIME Resolver
(under construction)
The resolver provides a method to get the Knime URLs (that are simply URLs) by which is possible to invoke the KNIME Model Simulation Application.
The KNIME Resolver is reachable at:
[URI_RESOLVER_SERVICE_ENDPOINT]/knime
It exposes the HTTP methods:
- GET (at /get path) - to resolve a Knime URL (created by /create path) that invokes the KNIME Model Simulation Application for the model passed in the query string;
- POST (at /create path) - to generate a Knime URL (then resolved by the /get path) for a KNIME Model Simulation passed by query string.
Dependencies: in order to work correctly this resolver needs to:
- KnimeModelSimulationPortlet GR as ApplicationProfile published at VRE level in the gCube Information System.
HTTP POST: Create a KNIME URL
This resolver (protected by AuthZ) is reachable at the following:
[URI_RESOLVER_SERVICE_ENDPOINT]/knime/create
It provides the HTTP POST method to create Knime URLs. A valid Knime URL is generable if you know your token (in the VRE) and query string to invoke a KNIME Model Simulation.
If you know these things, in order to create a Knime URL you can perform a POST request to the resolver /knime/create at:
[URI_RESOLVER_SERVICE_ENDPOINT]/knime/create?[QUERY_STRING_TO_INVKOE_A_KNIME_MODEL]
by adding the HTTP header gcube-token: [YOUR_VRE_TOKEN] and passing the query string ([QUERY_STRING_TO_INVOKE_A_KNIME_MODEL]) and it creates a Knime URL with the following format:
[URI_RESOLVER_SERVICE_ENDPOINT]/knime/get/[VRE_NAME]?[QUERY_STRING_TO_INVKOE_A_KNIME_MODEL]
that via GET Operation provided by Knime Resolver will be forwarded to the KNIME Model Simulation Application.
HTTP GET: Resolve a KNIME URL
This resolver is reachable at
[URI_RESOLVER_SERVICE_ENDPOINT]/knime/get
It provides the HTTP GET method to resolve an Knime URL that supports the following protocol:
[URI_RESOLVER_SERVICE_ENDPOINT]/knime/get/[VRE_NAME]?[QUERY_STRING_TO_INVKOE_A_KNIME_MODEL]
Uri Resolver Manager
The Uri Resolver Manager is a library to get a public link of a resource reachable from a (Gcube) Http Resolver
At moment, the resource types are GIS or SMP reachable from Uri-Resolver and Gis-Resolver.
The Uri Resolver Manager look up the (Generic Resource) "Uri-Resolver-Map" to read the mapping: Application Type -> (Runtime Resource of its) Resolver, e.g. GIS -> Gis-Resolver, SMP -> HTTP-URI-Resolver
The mandatory parameters to forward correctly the HTTP request to Resolver are dynamically read from Runtime Resource
Maven
The artifact is available on Nexus with the following coordinates:
<groupId>org.gcube.portlets.user</groupId> <artifactId>uri-resolver-manager</artifactId> <version>LATEST</version>
A simple java How-to:
public static void main(String[] args) { try { ScopeProvider.instance.set("/gcube/devsec/devVRE"); UriResolverManager resolver = new UriResolverManager("GIS"); Map<String, String> params = new HashMap<String, String>(); params.put("gis-UUID", "eb1a1b63-f324-47ee-9522-b8f5803e19ec"); params.put("scope", "/gcube/devsec/devVRE"); String shortLink = resolver.getLink(params, true); //true, link is shorted otherwise none System.out.println(shortLink); } catch (UriResolverMapException e) { e.printStackTrace(); } catch (IllegalArgumentException e) { e.printStackTrace(); }catch (Exception e) { e.printStackTrace(); } }
Generic Resource
In order to work, it is necessary to copy from Root scope (/d4science.research-infrastructures.eu) to the VRE where Uri Resolver Manager must work, the generic resource with following coordinates:
<Type>GenericResource</Type> <SecondaryType>UriResolverMap</SecondaryType> <Name>Uri-Resolver-Map</Name>