Difference between revisions of "How to use Data Transfer 2"
(→Data Transfer Service) |
(→Data Transfer library) |
||
Line 64: | Line 64: | ||
==Data Transfer library== | ==Data Transfer library== | ||
− | === | + | |
+ | The data transfer library is a java library which serves applications as a client to data transfer facilities. | ||
+ | In order to use the library, applications must declare the following dependency in their maven pom files : | ||
+ | <pre> | ||
+ | <dependency> | ||
+ | <groupId>org.gcube.data.transfer</groupId> | ||
+ | <artifactId>data-transfer-library</artifactId> | ||
+ | </dependency> | ||
+ | </pre> | ||
+ | |||
+ | |||
+ | The library is designed in order to offer a simple api to submit transfers to the selected services without dealing with : | ||
+ | |||
+ | * http calls; | ||
+ | * status monitoring; | ||
+ | * transfer channel selection negotiation according to server's capabilities; | ||
+ | |||
+ | ===Submit a transfer=== | ||
+ | In order to submit a transfer to a chosen server, the application needs to get an instance of the class ''org.gcube.data.transfer.library.DataTransferClient''. | ||
+ | Instances of the client are obtained by calling on of the following static methods : | ||
+ | <source> | ||
+ | public static DataTransferClient getInstanceByEndpoint(String endpoint) throws UnreachableNodeException, ServiceNotFoundException; | ||
+ | |||
+ | public static DataTransferClient getInstanceByNodeId(String id) throws HostingNodeNotFoundException, UnreachableNodeException, ServiceNotFoundException; | ||
+ | </source> | ||
+ | |||
+ | To perform a transfer operation, application just need to invoke one of the exposed methods providing : | ||
+ | |||
+ | * a transfer source (i.e. a ''java.io.File'' object or its absolute path); | ||
+ | * a transfer destination a.k.a file destination name for the basic scenario (see [[#Transfer destination]] for more in-depth details); | ||
+ | * ''optional'' set of Plugin invocations (see [[#Plugin invovation]] for more in-depth details). | ||
+ | |||
+ | Please note the library exposes different signature of the same logic in order to mask unwanted functionalities to clients i.e. the following three calls perform the same operation : | ||
+ | |||
+ | <source> | ||
+ | DataTransferClient client=DataTransferClient.getInstanceByEndpoint(...); | ||
+ | String localFile=".."; | ||
+ | String transferredFileName=".."; | ||
+ | client.localFile(localFile,transferredFileName); | ||
+ | </source> | ||
+ | |||
+ | <source> | ||
+ | DataTransferClient client=DataTransferClient.getInstanceByEndpoint(...); | ||
+ | String localFile=".."; | ||
+ | String transferredFileName=".."; | ||
+ | Destination dest=new Destination(transferredFileName); | ||
+ | client.localFile(localFile,dest); | ||
+ | </source> | ||
+ | |||
+ | <source> | ||
+ | DataTransferClient client=DataTransferClient.getInstanceByEndpoint(...); | ||
+ | String localFile=".."; | ||
+ | String transferredFileName=".."; | ||
+ | Destination dest=new Destination(transferredFileName); | ||
+ | client.localFile(localFile,dest,Collections.<PluginInvocation> emptySet()); | ||
+ | </source> | ||
+ | |||
===Transfer destination=== | ===Transfer destination=== | ||
+ | |||
+ | |||
+ | |||
===Plugin invocation=== | ===Plugin invocation=== |
Revision as of 14:56, 12 September 2016
Data Transfer 2 is one of the subsystems forming the gCube Data_Transfer_Facilities. It aims to provide gCube Applications a common layer for efficient and transparent data transfer towards gCube SmartGear nodes. It's designed as a client service architecture exploiting plugin design pattern. A generic overview and its design are described here
Following sections describe how to use and interact with the involved components.
Contents
Data Transfer Service
The Data Transfer Service is a SmartGears-aware web application developped on top of [jersey] framework. Its main functionalities are :
- receive and serve data transfer requests;
- expose capabilities;
At startup it gathers information on :
- current network configuration (i.e. exposed hostname, available ports) in order to negotiate transfer channel with clients;
- available data-transfer plugins
Installation
The data transfer service is released as a war with the following maven coordinates
<groupId>org.gcube.data.transfer</groupId> <artifactId>data-transfer-service</artifactId>
It needs to be hosted in a SmartGears installation in order to run. Please refer to SmartGears for further information.
Interface
In this section we will describe the http interfaces exposed by the service.
Capabilities
The Capabilities interface exposes information regarding :
- Instance details (i.e. hostname, port, nodeId)
- Available plugins
- Available persistence Ids
Capabilities are mapped in a Java Object of class org.gcube.data.transfer.model.TransferCapabilities.
Transfer requests
The Requests interface receives transfer requests from clients, returning the associated ticket ID if the requests has been successfully registered. E request is expected to specify :
- The transfer settings decided by the caller client (including the data source);
- The transfer destination (see #Transfer destination);
- An optional set of plugin invocations (see #Plugin invocation)
Transfer requests are mapped in Java Objects of class org.gcube.data.transfer.model.TransferRequest.
Transfer status
The Status interface provides information on the progress of the transfer identified by its related ticket ID. The transfer status provides information about :
- The related transfer request;
- Transfer statistics (i.e. transferredBytes, elapsed Time);
- Destination file absolute location;
- Overall status;
- Error Message if any;
Data Transfer library
The data transfer library is a java library which serves applications as a client to data transfer facilities. In order to use the library, applications must declare the following dependency in their maven pom files :
<dependency> <groupId>org.gcube.data.transfer</groupId> <artifactId>data-transfer-library</artifactId> </dependency>
The library is designed in order to offer a simple api to submit transfers to the selected services without dealing with :
- http calls;
- status monitoring;
- transfer channel selection negotiation according to server's capabilities;
Submit a transfer
In order to submit a transfer to a chosen server, the application needs to get an instance of the class org.gcube.data.transfer.library.DataTransferClient. Instances of the client are obtained by calling on of the following static methods :
Invalid language.
You need to specify a language like this: <source lang="html4strict">...</source>
Supported languages for syntax highlighting:
4cs, 6502acme, 6502kickass, 6502tasm, 68000devpac, abap, actionscript, actionscript3, ada, aimms, algol68, apache, applescript, arm, asm, asp, asymptote, autoconf, autohotkey, autoit, avisynth, awk, bascomavr, bash, basic4gl, bf, bibtex, blitzbasic, bnf, boo, c, caddcl, cadlisp, cfdg, cfm, chaiscript, chapel, cil, clojure, cmake, cobol, coffeescript, cpp, csharp, css, cuesheet, d, dart, dcl, dcpu16, dcs, delphi, diff, div, dos, dot, e, ecmascript, eiffel, email, epc, erlang, euphoria, ezt, f1, falcon, fo, fortran, freebasic, freeswitch, fsharp, gambas, gdb, genero, genie, gettext, glsl, gml, gnuplot, go, groovy, gwbasic, haskell, haxe, hicest, hq9plus, html4strict, html5, icon, idl, ini, inno, intercal, io, ispfpanel, j, java, java5, javascript, jcl, jquery, kixtart, klonec, klonecpp, latex, lb, ldif, lisp, llvm, locobasic, logtalk, lolcode, lotusformulas, lotusscript, lscript, lsl2, lua, m68k, magiksf, make, mapbasic, matlab, mirc, mmix, modula2, modula3, mpasm, mxml, mysql, nagios, netrexx, newlisp, nginx, nimrod, nsis, oberon2, objc, objeck, ocaml, octave, oobas, oorexx, oracle11, oracle8, oxygene, oz, parasail, parigp, pascal, pcre, per, perl, perl6, pf, php, pic16, pike, pixelbender, pli, plsql, postgresql, postscript, povray, powerbuilder, powershell, proftpd, progress, prolog, properties, providex, purebasic, pycon, pys60, python, q, qbasic, qml, racket, rails, rbs, rebol, reg, rexx, robots, rpmspec, rsplus, ruby, rust, sas, scala, scheme, scilab, scl, sdlbasic, smalltalk, smarty, spark, sparql, sql, standardml, stonescript, systemverilog, tcl, teraterm, text, thinbasic, tsql, typoscript, unicon, upc, urbi, uscript, vala, vb, vbnet, vbscript, vedit, verilog, vhdl, vim, visualfoxpro, visualprolog, whitespace, whois, winbatch, xbasic, xml, xpp, yaml, z80, zxbasic
public static DataTransferClient getInstanceByEndpoint(String endpoint) throws UnreachableNodeException, ServiceNotFoundException; public static DataTransferClient getInstanceByNodeId(String id) throws HostingNodeNotFoundException, UnreachableNodeException, ServiceNotFoundException;
To perform a transfer operation, application just need to invoke one of the exposed methods providing :
- a transfer source (i.e. a java.io.File object or its absolute path);
- a transfer destination a.k.a file destination name for the basic scenario (see #Transfer destination for more in-depth details);
- optional set of Plugin invocations (see #Plugin invovation for more in-depth details).
Please note the library exposes different signature of the same logic in order to mask unwanted functionalities to clients i.e. the following three calls perform the same operation :
Invalid language.
You need to specify a language like this: <source lang="html4strict">...</source>
Supported languages for syntax highlighting:
4cs, 6502acme, 6502kickass, 6502tasm, 68000devpac, abap, actionscript, actionscript3, ada, aimms, algol68, apache, applescript, arm, asm, asp, asymptote, autoconf, autohotkey, autoit, avisynth, awk, bascomavr, bash, basic4gl, bf, bibtex, blitzbasic, bnf, boo, c, caddcl, cadlisp, cfdg, cfm, chaiscript, chapel, cil, clojure, cmake, cobol, coffeescript, cpp, csharp, css, cuesheet, d, dart, dcl, dcpu16, dcs, delphi, diff, div, dos, dot, e, ecmascript, eiffel, email, epc, erlang, euphoria, ezt, f1, falcon, fo, fortran, freebasic, freeswitch, fsharp, gambas, gdb, genero, genie, gettext, glsl, gml, gnuplot, go, groovy, gwbasic, haskell, haxe, hicest, hq9plus, html4strict, html5, icon, idl, ini, inno, intercal, io, ispfpanel, j, java, java5, javascript, jcl, jquery, kixtart, klonec, klonecpp, latex, lb, ldif, lisp, llvm, locobasic, logtalk, lolcode, lotusformulas, lotusscript, lscript, lsl2, lua, m68k, magiksf, make, mapbasic, matlab, mirc, mmix, modula2, modula3, mpasm, mxml, mysql, nagios, netrexx, newlisp, nginx, nimrod, nsis, oberon2, objc, objeck, ocaml, octave, oobas, oorexx, oracle11, oracle8, oxygene, oz, parasail, parigp, pascal, pcre, per, perl, perl6, pf, php, pic16, pike, pixelbender, pli, plsql, postgresql, postscript, povray, powerbuilder, powershell, proftpd, progress, prolog, properties, providex, purebasic, pycon, pys60, python, q, qbasic, qml, racket, rails, rbs, rebol, reg, rexx, robots, rpmspec, rsplus, ruby, rust, sas, scala, scheme, scilab, scl, sdlbasic, smalltalk, smarty, spark, sparql, sql, standardml, stonescript, systemverilog, tcl, teraterm, text, thinbasic, tsql, typoscript, unicon, upc, urbi, uscript, vala, vb, vbnet, vbscript, vedit, verilog, vhdl, vim, visualfoxpro, visualprolog, whitespace, whois, winbatch, xbasic, xml, xpp, yaml, z80, zxbasic
DataTransferClient client=DataTransferClient.getInstanceByEndpoint(...); String localFile=".."; String transferredFileName=".."; client.localFile(localFile,transferredFileName);
Invalid language.
You need to specify a language like this: <source lang="html4strict">...</source>
Supported languages for syntax highlighting:
4cs, 6502acme, 6502kickass, 6502tasm, 68000devpac, abap, actionscript, actionscript3, ada, aimms, algol68, apache, applescript, arm, asm, asp, asymptote, autoconf, autohotkey, autoit, avisynth, awk, bascomavr, bash, basic4gl, bf, bibtex, blitzbasic, bnf, boo, c, caddcl, cadlisp, cfdg, cfm, chaiscript, chapel, cil, clojure, cmake, cobol, coffeescript, cpp, csharp, css, cuesheet, d, dart, dcl, dcpu16, dcs, delphi, diff, div, dos, dot, e, ecmascript, eiffel, email, epc, erlang, euphoria, ezt, f1, falcon, fo, fortran, freebasic, freeswitch, fsharp, gambas, gdb, genero, genie, gettext, glsl, gml, gnuplot, go, groovy, gwbasic, haskell, haxe, hicest, hq9plus, html4strict, html5, icon, idl, ini, inno, intercal, io, ispfpanel, j, java, java5, javascript, jcl, jquery, kixtart, klonec, klonecpp, latex, lb, ldif, lisp, llvm, locobasic, logtalk, lolcode, lotusformulas, lotusscript, lscript, lsl2, lua, m68k, magiksf, make, mapbasic, matlab, mirc, mmix, modula2, modula3, mpasm, mxml, mysql, nagios, netrexx, newlisp, nginx, nimrod, nsis, oberon2, objc, objeck, ocaml, octave, oobas, oorexx, oracle11, oracle8, oxygene, oz, parasail, parigp, pascal, pcre, per, perl, perl6, pf, php, pic16, pike, pixelbender, pli, plsql, postgresql, postscript, povray, powerbuilder, powershell, proftpd, progress, prolog, properties, providex, purebasic, pycon, pys60, python, q, qbasic, qml, racket, rails, rbs, rebol, reg, rexx, robots, rpmspec, rsplus, ruby, rust, sas, scala, scheme, scilab, scl, sdlbasic, smalltalk, smarty, spark, sparql, sql, standardml, stonescript, systemverilog, tcl, teraterm, text, thinbasic, tsql, typoscript, unicon, upc, urbi, uscript, vala, vb, vbnet, vbscript, vedit, verilog, vhdl, vim, visualfoxpro, visualprolog, whitespace, whois, winbatch, xbasic, xml, xpp, yaml, z80, zxbasic
DataTransferClient client=DataTransferClient.getInstanceByEndpoint(...); String localFile=".."; String transferredFileName=".."; Destination dest=new Destination(transferredFileName); client.localFile(localFile,dest);
Invalid language.
You need to specify a language like this: <source lang="html4strict">...</source>
Supported languages for syntax highlighting:
4cs, 6502acme, 6502kickass, 6502tasm, 68000devpac, abap, actionscript, actionscript3, ada, aimms, algol68, apache, applescript, arm, asm, asp, asymptote, autoconf, autohotkey, autoit, avisynth, awk, bascomavr, bash, basic4gl, bf, bibtex, blitzbasic, bnf, boo, c, caddcl, cadlisp, cfdg, cfm, chaiscript, chapel, cil, clojure, cmake, cobol, coffeescript, cpp, csharp, css, cuesheet, d, dart, dcl, dcpu16, dcs, delphi, diff, div, dos, dot, e, ecmascript, eiffel, email, epc, erlang, euphoria, ezt, f1, falcon, fo, fortran, freebasic, freeswitch, fsharp, gambas, gdb, genero, genie, gettext, glsl, gml, gnuplot, go, groovy, gwbasic, haskell, haxe, hicest, hq9plus, html4strict, html5, icon, idl, ini, inno, intercal, io, ispfpanel, j, java, java5, javascript, jcl, jquery, kixtart, klonec, klonecpp, latex, lb, ldif, lisp, llvm, locobasic, logtalk, lolcode, lotusformulas, lotusscript, lscript, lsl2, lua, m68k, magiksf, make, mapbasic, matlab, mirc, mmix, modula2, modula3, mpasm, mxml, mysql, nagios, netrexx, newlisp, nginx, nimrod, nsis, oberon2, objc, objeck, ocaml, octave, oobas, oorexx, oracle11, oracle8, oxygene, oz, parasail, parigp, pascal, pcre, per, perl, perl6, pf, php, pic16, pike, pixelbender, pli, plsql, postgresql, postscript, povray, powerbuilder, powershell, proftpd, progress, prolog, properties, providex, purebasic, pycon, pys60, python, q, qbasic, qml, racket, rails, rbs, rebol, reg, rexx, robots, rpmspec, rsplus, ruby, rust, sas, scala, scheme, scilab, scl, sdlbasic, smalltalk, smarty, spark, sparql, sql, standardml, stonescript, systemverilog, tcl, teraterm, text, thinbasic, tsql, typoscript, unicon, upc, urbi, uscript, vala, vb, vbnet, vbscript, vedit, verilog, vhdl, vim, visualfoxpro, visualprolog, whitespace, whois, winbatch, xbasic, xml, xpp, yaml, z80, zxbasic
DataTransferClient client=DataTransferClient.getInstanceByEndpoint(...); String localFile=".."; String transferredFileName=".."; Destination dest=new Destination(transferredFileName); client.localFile(localFile,dest,Collections.<PluginInvocation> emptySet());