Difference between revisions of "Data Transfer APIs"
From Gcube Wiki
(→gRS2 Broker) |
|||
(35 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
− | ==Overview== | + | Data Transfer APIs provide access to transfer mechanisms for large sets of multi-type datasets distributed across the system. |
− | == | + | The document outlines the design rationale for those APIs. |
+ | |||
+ | ==Overview== | ||
+ | ===APIs=== | ||
{| {{table}} | {| {{table}} | ||
− | | align="center" style="background:#f0f0f0;"|''' | + | | align="center" style="background:#f0f0f0;"|'''Name''' |
| align="center" style="background:#f0f0f0;"|'''Description''' | | align="center" style="background:#f0f0f0;"|'''Description''' | ||
| align="center" style="background:#f0f0f0;"|'''Type''' | | align="center" style="background:#f0f0f0;"|'''Type''' | ||
Line 8: | Line 11: | ||
| align="center" style="background:#f0f0f0;"|'''Framework''' | | align="center" style="background:#f0f0f0;"|'''Framework''' | ||
|- | |- | ||
− | | [https://gcube.wiki.gcube-system.org/gcube/index.php/ | + | | [https://gcube.wiki.gcube-system.org/gcube/index.php/Data_Transfer_APIs#Data_Transfer_Agent_Library Data Transfer Agent Library]||The CL proxy for the [[#Data_Transfer_Agent_Service |Data Transfer Agent Service]] operations||Java||Local||[https://gcube.wiki.gcube-system.org/gcube/index.php/Integration_and_Interoperability_Facilities_Framework:_Client_Libraries CL] |
|- | |- | ||
− | | [https://gcube.wiki.gcube-system.org/gcube/index.php/ | + | | [https://gcube.wiki.gcube-system.org/gcube/index.php/Data_Transfer_APIs#Data_Transfer_Agent_Service Data Transfer Agent Service]|| Structured/Unstructured transfer operations among data Source/Storages||gCube Web Service||SOAP||[https://gcube.wiki.gcube-system.org/gcube/index.php/Integration_and_Interoperability_Facilities_Framework:_WSRF_API_Framework_Specification gCore] |
|- | |- | ||
− | + | | [https://gcube.wiki.gcube-system.org/gcube/index.php/Data_Transfer_APIs#gRS2_Broker gRS2 Broker]||This component exposes the gCube Resultset as an http endpoint||Java Servlet||REST||[https://gcube.wiki.gcube-system.org/gcube/index.php/Integration_and_Interoperability_Facilities_Framework:_HTTP_API_Framework_Specification HTTP Front End] | |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
|- | |- | ||
|} | |} | ||
+ | |||
+ | ==Data Transfer Agent Library== | ||
+ | * '''Description''': The CL for the Data Transfer Agent operations | ||
+ | * '''Type''': Java | ||
+ | * '''Protocol''': Local | ||
+ | * '''Framework''': [https://gcube.wiki.gcube-system.org/gcube/index.php/Integration_and_Interoperability_Facilities_Framework:_Client_Libraries CL] | ||
+ | * '''Key Features''': | ||
+ | :* ''synchronous/asynchronous transfers over data transfer service operations'' | ||
+ | :* ''transfer monitoring'' | ||
+ | :* ''transfer outcomes to clients'' | ||
+ | * [[Data_Transfer_Agent#Data_Transfer_Agent_Library|'''Documentation''']] | ||
+ | |||
+ | ==Data Transfer Agent Service== | ||
+ | * '''Description''': Structured/Unstructured transfer operations among data Source/Storages | ||
+ | * '''Type''': gCube Web Service | ||
+ | * '''Protocol''': SOAP | ||
+ | * '''Framework''': gCore | ||
+ | * '''Key Features''': | ||
+ | :* ''transfer of local files from an external/internal client to a remote GHN'' | ||
+ | :* ''file transfer from a remote Data Source to a remote GHN using standard protocols ( HTTP, FTP, BitTorrent)'' | ||
+ | :* ''file transfer from a remote Data Source to the gCube Storage Manager ( MongoDB) using standard protocols ( HTTP, FTP, BitTorrent)'' | ||
+ | :* ''tree based data transfer from a remote Data Source to a remote tree-based data storage'' | ||
+ | :* ''transfer statistics publication trough messaging'' | ||
+ | * [[Data_Transfer_Agent#Data_Transfer_Agent_Service|'''Documentation''']] | ||
+ | |||
+ | ==gRS2 Broker== | ||
+ | * '''Description''': This component exposes the gCube Resultset as an http endpoint | ||
+ | * '''Type''': Java Servlet | ||
+ | * '''Protocol''': REST | ||
+ | * '''Framework''': [https://gcube.wiki.gcube-system.org/gcube/index.php/Integration_and_Interoperability_Facilities_Framework:_HTTP_API_Framework_Specification HTTP Front End] | ||
+ | * '''Key Features''': | ||
+ | :* ''results are exposed as an HTTP endpoint'' | ||
+ | :* ''results are in XML format'' | ||
+ | :* ''server response is compressed in order to achieve performance benefits'' | ||
+ | :* ''the records are retrieved iteratively - gradually'' | ||
+ | :* ''number of results can be customized (max parameter)'' | ||
+ | :* ''files are stored locally on the server (garbage collection)'' | ||
+ | * Wiki Doc: |
Latest revision as of 19:15, 2 July 2012
Data Transfer APIs provide access to transfer mechanisms for large sets of multi-type datasets distributed across the system. The document outlines the design rationale for those APIs.
Contents
Overview
APIs
Name | Description | Type | Protocol | Framework |
Data Transfer Agent Library | The CL proxy for the Data Transfer Agent Service operations | Java | Local | CL |
Data Transfer Agent Service | Structured/Unstructured transfer operations among data Source/Storages | gCube Web Service | SOAP | gCore |
gRS2 Broker | This component exposes the gCube Resultset as an http endpoint | Java Servlet | REST | HTTP Front End |
Data Transfer Agent Library
- Description: The CL for the Data Transfer Agent operations
- Type: Java
- Protocol: Local
- Framework: CL
- Key Features:
- synchronous/asynchronous transfers over data transfer service operations
- transfer monitoring
- transfer outcomes to clients
Data Transfer Agent Service
- Description: Structured/Unstructured transfer operations among data Source/Storages
- Type: gCube Web Service
- Protocol: SOAP
- Framework: gCore
- Key Features:
- transfer of local files from an external/internal client to a remote GHN
- file transfer from a remote Data Source to a remote GHN using standard protocols ( HTTP, FTP, BitTorrent)
- file transfer from a remote Data Source to the gCube Storage Manager ( MongoDB) using standard protocols ( HTTP, FTP, BitTorrent)
- tree based data transfer from a remote Data Source to a remote tree-based data storage
- transfer statistics publication trough messaging
gRS2 Broker
- Description: This component exposes the gCube Resultset as an http endpoint
- Type: Java Servlet
- Protocol: REST
- Framework: HTTP Front End
- Key Features:
- results are exposed as an HTTP endpoint
- results are in XML format
- server response is compressed in order to achieve performance benefits
- the records are retrieved iteratively - gradually
- number of results can be customized (max parameter)
- files are stored locally on the server (garbage collection)
- Wiki Doc: