Data Retrieval APIs
From Gcube Wiki
Data Retrieval APIs provide access to to Information Retrieval facilites over large heterogeneous environments, offering a declarative approach for querying the hosted information and allowing for easy integration of external data sources of information.
This page outlines the design rationale for those APIs.
Contents
Overview
Name | Description | Type | Protocol | Framework |
gCQLParser | The gCube parser for CQL language (extended with gCube elements) | Java | Local | none |
Index Service | Provides indexing functionality for the resources | WS | SOAP | gCore |
OperatorLibrary | The search operators cover the basic functionality that could be encountered in a typical search operation | Java | Local | none |
Search System Service | Web service that offers information retrieval functionality in a gCube infrastructure. | WS | SOAP | gCore |
OpenSearch Library | The OpenSearch Core Library conforms to the latest OpenSearch specification and provides general OpenSearch-related functionality to any component which needs to query OpenS earch providers. | CL | Local | none |
OpenSearchService | TThe OpenSearch Service is a stateful web service responsible for the invocation of the OpenSearch Operator in the context of the provider to be queried. | WS | SOAP | gCore |
gCQLParser API
- Description: Provides functionality that helps presentation layer to use gCube services.
- Type: Local
- Protocol: Java
- Framework: none
- Documentation
- Key Features
- submission of gCQL queries as plain string
- support for gCube extensions in CQL language for presentable information
Index Service API
- Description: Provides indexing functionality for the resources.
- Type: WS
- Protocol: SOAP
- Framework: gCore
- Documentation
- Key Features
- Full Text Index receives CQL queries and transforms them into Lucene queries
- 2 linguistics components are available; the language identifier module, and the lemmatizer module
- database partitioning is supported
- geo-spatial Index Lookup supports one custom CQL relation
- the "geosearch" relation has a number of modifiers that specify parameters such as collection, language, query inclusion type, refiner and ranker
- support of customs refiners and rankers
- the content to be fed into a Geo Index, must be served as a ResultSet containing XML documents conforming to the GeoROWSET schemafield presense and handling by the Geo Index is specified through a GeoIndexType; an XML document conforming to the GeoIndexType schemathe content to be fed into a Geo Index, must be served as a ResultSet containing XML documents conforming to the GeoROWSET schema
OperatorLibrary
- Description: The search operators cover the basic functionality that could be encountered in a typical search operation.
- Type: CL
- Protocol: Java
- Framework: none
- Documentation
- Key Features
- results are transfered via ResultSet
- the comparing parts are either literals (date, string, integer, double literals are supported) or aggregate functions on the results of a search service execution
Search System Service
- Description: Web service that offers information retrieval functionality in a gCube infrastructure.
- Type: WS
- Protocol: SOAP
- Framework: gCore
- Documentation
- Key Features
- queries expressed in Contextual Query Language
- orchestration of search plan production and execution
- transparent detection and utilization of Data Sources
- results streaming exposition through gRS2 endpoint
OpenSearch Library
- Description: The OpenSearch Core Library conforms to the latest OpenSearch specification and provides general OpenSearch-related functionality to any component which needs to query OpenS earch providers.
- Type: CL
- Protocol: Java
- Framework: none
- Documentation
- Key Features
- access to the information hosted by external Providers
- extensions provide functionality not only for standard OpenSearch parameters are supported
- the role of the OpenSearch Operator is to provide support for querying and retrieval of search results via OpenSearch from providers which expose an OpenSearch description document.
- results are returned wrapped in a ResultSet
- support of data transformation by the operator
- both direct and brokered result processing is supported
- support of a set of fixed parameters, which override the user-provided parameters only at the level of the top provider
- MIME type of the results that are to be obtained is abstracted away and treated as low-level information which can be exploited by the way OpenSearch Resources are structured
- the operator accepts a set of query terms and a set of query parameters
- OpenSearch Operator expects to receive all query parameters. Any free-text parameter value should be URL-encoded
OpenSearchService
- Description: The OpenSearch Service is a stateful web service responsible for the invocation of the OpenSearch Operator in the context of the provider to be queried.
- Type: WS
- Protocol: SOAP
- Framework: gCore
- Documentation
- Key Features
- uses a WS-Resource for each OpenSearch provider in order to bind the Service to the Metadata Collection corresponding to the provider.
- maintains a cache per provider WS-Resource for performance and reliability reasons
- he cache can be refreshed either on demand or periodically, based on a configurable time interval