Difference between revisions of "Data Transfer 2"
From Gcube Wiki
m |
|||
| Line 2: | Line 2: | ||
== Overview == | == Overview == | ||
| − | + | The implementation of a reliable data transfer mechanisms between the nodes of a gCube-based Hybrid Data Infrastructure is one of the main objectives when dealing with large set of multi-type datasets distributed across different repositories. | |
| + | |||
| + | To promote an efficient and optimized consumption of these data resources, a number of components have been designed to meet the data transfer requirements. | ||
This document outlines the design rationale, key features, and high-level architecture, the options for their deployment and as well some use cases. | This document outlines the design rationale, key features, and high-level architecture, the options for their deployment and as well some use cases. | ||
| Line 9: | Line 11: | ||
The components belonging to this class are responsible for: | The components belonging to this class are responsible for: | ||
| + | |||
| + | ;Point to Point transfer | ||
| + | :direct transfer invocation to a gCube Node | ||
;reliable data transfer between Infrastructure Data Sources and Data Storages | ;reliable data transfer between Infrastructure Data Sources and Data Storages | ||
:by exploiting the uniform access interfaces provided by gCube and standard transfer protocols | :by exploiting the uniform access interfaces provided by gCube and standard transfer protocols | ||
| − | ; | + | ;automatic transfer optimization |
| − | : | + | :by exploiting best available transfer options between invoker and target nodes |
| − | ; | + | ;advanced and extensible post transfer processing |
| − | : | + | :plugin - oriented implementation to serve advanced use case |
| − | + | == Design == | |
| − | + | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
=== Philosophy=== | === Philosophy=== | ||
Revision as of 13:59, 9 September 2016
Contents
Overview
The implementation of a reliable data transfer mechanisms between the nodes of a gCube-based Hybrid Data Infrastructure is one of the main objectives when dealing with large set of multi-type datasets distributed across different repositories.
To promote an efficient and optimized consumption of these data resources, a number of components have been designed to meet the data transfer requirements.
This document outlines the design rationale, key features, and high-level architecture, the options for their deployment and as well some use cases.
Key features
The components belonging to this class are responsible for:
- Point to Point transfer
- direct transfer invocation to a gCube Node
- reliable data transfer between Infrastructure Data Sources and Data Storages
- by exploiting the uniform access interfaces provided by gCube and standard transfer protocols
- automatic transfer optimization
- by exploiting best available transfer options between invoker and target nodes
- advanced and extensible post transfer processing
- plugin - oriented implementation to serve advanced use case
