Data Retrieval APIs

From Gcube Wiki
Revision as of 10:20, 27 August 2012 by Rena.tsantouli (Talk | contribs) (OpenSearchService)

Jump to: navigation, search

Data Retrieval APIs provide access to to Information Retrieval facilites over large heterogeneous environments, offering a declarative approach for querying the hosted information and allowing for easy integration of external data sources of information.

This page outlines the design rationale for those APIs.

Overview

Name Description Type Protocol Framework
gCQLParser The gCube parser for CQL language (extended with gCube elements) Java Local none
Index Service Provides indexing functionality for the resources WS SOAP gCore
OperatorLibrary The search operators cover the basic functionality that could be encountered in a typical search operation Java Local none
Search System Service Web service that offers information retrieval functionality in a gCube infrastructure. WS SOAP gCore
OpenSearch Library The OpenSearch Core Library conforms to the latest OpenSearch specification and provides general OpenSearch-related functionality to any component which needs to query OpenS earch providers. CL Local none
OpenSearchService TThe OpenSearch Service is a stateful web service responsible for the invocation of the OpenSearch Operator in the context of the provider to be queried. WS SOAP gCore

gCQLParser API

  • Description: Provides functionality that helps presentation layer to use gCube services.
  • Type: Local
  • Protocol: Java
  • Framework: none
  • Documentation
  • Key Features
  • submission of gCQL queries as plain string
  • support for gCube extensions in CQL language for presentable information


Index Service API

  • Description: Provides indexing functionality for the resources.
  • Type: WS
  • Protocol: SOAP
  • Framework: gCore
  • Documentation
  • Key Features
  • Full Text Index receives CQL queries and transforms them into Lucene queries
  • 2 linguistics components are available; the language identifier module, and the lemmatizer module
  • database partitioning is supported
  • geo-spatial Index Lookup supports one custom CQL relation
  • the "geosearch" relation has a number of modifiers that specify parameters such as collection, language, query inclusion type, refiner and ranker
  • support of customs refiners and rankers
  • the content to be fed into a Geo Index, must be served as a ResultSet containing XML documents conforming to the GeoROWSET schemafield presense and handling by the Geo Index is specified through a GeoIndexType; an XML document conforming to the GeoIndexType schemathe content to be fed into a Geo Index, must be served as a ResultSet containing XML documents conforming to the GeoROWSET schema

OperatorLibrary

  • Description: The search operators cover the basic functionality that could be encountered in a typical search operation.
  • Type: CL
  • Protocol: Java
  • Framework: none
  • Documentation
  • Key Features
  • results are transfered via ResultSet
  • the comparing parts are either literals (date, string, integer, double literals are supported) or aggregate functions on the results of a search service execution

Search System Service

  • Description: Web service that offers information retrieval functionality in a gCube infrastructure.
  • Type: WS
  • Protocol: SOAP
  • Framework: gCore
  • Documentation
  • Key Features
  • queries expressed in Contextual Query Language
  • orchestration of search plan production and execution
  • transparent detection and utilization of Data Sources
  • results streaming exposition through gRS2 endpoint

OpenSearch Library

  • Description: The OpenSearch Core Library conforms to the latest OpenSearch specification and provides general OpenSearch-related functionality to any component which needs to query OpenS earch providers.
  • Type: CL
  • Protocol: Java
  • Framework: none
  • Documentation
  • Key Features
  • access to the information hosted by external Providers
  • extensions provide functionality not only for standard OpenSearch parameters are supported
  • the role of the OpenSearch Operator is to provide support for querying and retrieval of search results via OpenSearch from providers which expose an OpenSearch description document.
  • results are returned wrapped in a ResultSet
  • support of data transformation by the operator
  • both direct and brokered result processing is supported
  • support of a set of fixed parameters, which override the user-provided parameters only at the level of the top provider
  • MIME type of the results that are to be obtained is abstracted away and treated as low-level information which can be exploited by the way OpenSearch Resources are structured
  • the operator accepts a set of query terms and a set of query parameters
  • OpenSearch Operator expects to receive all query parameters. Any free-text parameter value should be URL-encoded

OpenSearchService

  • Description: The OpenSearch Service is a stateful web service responsible for the invocation of the OpenSearch Operator in the context of the provider to be queried.
  • Type: WS
  • Protocol: SOAP
  • Framework: gCore
  • Documentation
  • Key Features
  • uses a WS-Resource for each OpenSearch provider in order to bind the Service to the Metadata Collection corresponding to the provider.
  • maintains a cache per provider WS-Resource for performance and reliability reasons
  • he cache can be refreshed either on demand or periodically, based on a configurable time interval