Difference between revisions of "Storage Manager"
(→Example) |
(→storage-manager v 1.0.1) |
||
Line 243: | Line 243: | ||
The following code show the file uploading operation: | The following code show the file uploading operation: | ||
+ | |||
<code> | <code> | ||
GCUBEScope scope = GCUBEScope.getScope("/gcube"); | GCUBEScope scope = GCUBEScope.getScope("/gcube"); | ||
Line 251: | Line 252: | ||
The following code show the file downloading operation: | The following code show the file downloading operation: | ||
+ | |||
<code> | <code> | ||
GCUBEScope scope = GCUBEScope.getScope("/gcube"); | GCUBEScope scope = GCUBEScope.getScope("/gcube"); | ||
Line 260: | Line 262: | ||
The file is downloaded in the local directory: /home/yyy/FilePerTest/ where s2 is running and the file name is test.jpg | The file is downloaded in the local directory: /home/yyy/FilePerTest/ where s2 is running and the file name is test.jpg | ||
− | + | ;Scenario 2 | |
+ | |||
+ | In this example a GCUBE service s1 (identify by: ServiceClass: sc, ServiceName: sn), running in the GCUBEScope /gcube uploads the file: /home/xxx/FilePerTest/test2.jpg in shared mode and in VOLATILE area. | ||
+ | Retrieve the url of the file uploaded by the method getUrl and passes it to another service s2 that is in running on another node. The s2 service retrieves the inputStream of the file that the service s1 had uploaded by the url: | ||
+ | |||
+ | <code> | ||
+ | IClient client=new StorageClient("sc", "sn", "s1", AccessType.SHARED, scope).getClient(); | ||
+ | String id=client.put(true).LFile("/home/xxx/FilePerTest/test.jpg").RFile("/img/test1.jpg"); | ||
+ | String url=client.getUrl().RFile("/img/test1.jpg"); | ||
+ | </code> | ||
+ | |||
+ | The s2 service retrieve the inputStream of the file by the url stored in the "url" variable: | ||
+ | |||
+ | <code> | ||
+ | Handler.activateProtocol(); | ||
+ | URL smsHome = new URL("smp://img/test1.jpg5ezvFfBOLqbElsh5xYVgBTxt8lwt4fl"); | ||
+ | URLConnection uc = null; | ||
+ | uc = ( URLConnection ) smsHome.openConnection(); | ||
+ | InputStream is=uc.getInputStream(); | ||
+ | </code> |
Revision as of 09:52, 17 October 2012
Contents
Role
Library for access and storage of unstructured bytestreams, or files, can be provided through a standards-based, POSIX-like API which supports the organisation and operations normally associated with local file systems whilst offering scalable and fault-tolerant remote storage.
The library acts a facade to the service and allows clients to download, upload, remove, add, and list files. Files have owners and owners may define access rights to files, allowing private, public, or group-based access.
Access Mode
There are 3 ways to access remote files
- Private
- Accessible only to the owner of the file
- Public
- accessible to all users of the community
- Shared
- Accessible to all users of the same group that owner
These types of access will be specified when instantiating the client. If the client is instantiated for example,with the access mode private, all the operations do with this client instance will be in access mode private.
Operations
The StorageManger library implements many operation on remote files, the methods LFile and RFile refer to, respectively, Local File (if needed) and Remote File. In the call's signature, they are preceded by a method that identify a operation, see examples below:
- put(boolean replace)
- put a local file in a remote directory. If replace is true then the remote file, if exist, will be replaced.
- If the remote directory does not exist it will be automatically
- Example: put(true).LFile(“localPath”).RFile(“remotePath”)
- get()
- downloads a file from a remote path to a local directory
- Example: get().LFile(“localPath”).RFile(“remotePath”);
- remove()
- removes a remote file
- Example: remove().RFile(“remotePath”);
- removeDir()
- removes a remote directory and all files present. This operation is recursive.
- Example removeDir().RDir(“remoteDirPath”);
- showDir()
- shows the content of a directory
- Example showDir.RFile(“remoteDirPath”);
- lock()
- locks a remote file to prevent multiple access. Return a id string for unlock the file. Remember that every lock have been a TTL associated (Default: 180000 ms).
- Example lock().RFile(“remotePath”);
- unlock(String lockKey)
- if the file is locked, unlocks the remote file.
- Example unlock(“key”).RFile(“remotePath”);
- getTTL()
- return the TimeToLive associated to a remote file if the file is currently locked
- Example getTTL().RFile("remoteDirPath");
- setTTL(int milliseconds)
- reset the TimeToLive to a remote file that is currently locked. This operation is consented max 5 times for any file.
- Example setTTL(180000).RFile("remoteDirPath");
- getUrl()
- return the url of a file stored in the repository
- Example: getUrl().RFile(“remotePath”);
Sample Usage
The following code explains how to instatiate a client for do remote operation on the storage repository:
GCUBEScope scope = GCUBEScope.getScope("/gcube"); // The Access type can be private | shared | public IClient client=new StorageClient("ServiceClass", "ServiceName", "owner", AccessType.SHARED, scope).getClient();
The following code explains how to upload a new file in the storage repository:
String id=client.put(true).LFile("/home/rcirillo/FilePerTest/pippo.jpg").RFile("/img/pippo.jpg");
The following code explains how to download a file from the storage repository:
client.get().LFile("/home/rcirillo/FilePerTest/pippo.jpg").RFile("/img/pippo.jpg");
or alternatively, download the file by Id
client.get().LFile("/home/rcirillo/FilePerTest/pippo.jpg").RFileById(id);
or alternatively, download the file by stream
InputStream is = client.get().RFileAStream("/img/pippo.jpg");
The following code explains how to show the contents of a remote directory on the storage repository:
List<StorageObject> list=client.showDir().RDir("img");
The following code explains how to lock a file on the storage repository:
String id=client.lock().RFile("/img/pluto.jpg");
The following code explains how to unlock a file on the storage repository:
client.unlock(id).RFile("/img/pluto.jpg");
The following code explains how to remove a file on the storage repository:
client.remove().RFile("/img/pluto.jpg");
or alternatively, remove a file by Id
client.remove()..RFileById(id);
The following code explains how to remove a dir on the storage repository:
client.removeDir().RDir("/img");
Deployment
To release the library at a particular gcube scope you need to create only a generic resource that specifies the address of the server used for storing data. This is an example of generic resource used. The attribute "ip" of tag "service" specifies the ip of the server used for storing data.
<Resource version="0.4.x"> <ID>99afe260-1f49-11e1-92b4-e7131d88e32e</ID> <Type>GenericResource</Type> <Scopes> <Scope>/gcube/devsec/devVRE</Scope> </Scopes> <Profile> <SecondaryType>StorageManager</SecondaryType> <Name>MongoDBServer</Name> <Description>Server MongoDB for StorageManager</Description> <Body> <server_list> <server ip="xxx.xx.xxx.xxx" port="" /> </server_list> </Body> </Profile> </Resource>
Example
- Scenario 1
In this example a GCUBE service s1 (identify by: ServiceClass: sc, ServiceName: sn), running in the GCUBEScope /gcube upload the file: /home/xxx/FilePerTest/test.jpg in shared mode and another instance of the same service, s2 (identify by: ServiceClass: sc2, ServiceName: sn2) running in the same scope but on another node, download it.
The following code show the file uploading operation:
GCUBEScope scope = GCUBEScope.getScope("/gcube"); // The Access type can be private | shared | public IClient client=new StorageClient("sc", "sn", "s1", AccessType.SHARED, scope).getClient(); String id=client.put(true).LFile("/home/xxx/FilePerTest/test.jpg").RFile("/img/test.jpg");
The following code show the file downloading operation:
GCUBEScope scope = GCUBEScope.getScope("/gcube"); // The Access type can be private | shared | public IClient client=new StorageClient("sc2", "sn2", "s2", AccessType.SHARED, scope).getClient(); String id=client.get().LFile("/home/yyy/FilePerTest/test.jpg").RFile("/img/test.jpg");
The file is downloaded in the local directory: /home/yyy/FilePerTest/ where s2 is running and the file name is test.jpg
- Scenario 2
In this example a GCUBE service s1 (identify by: ServiceClass: sc, ServiceName: sn), running in the GCUBEScope /gcube uploads the file: /home/xxx/FilePerTest/test2.jpg in shared mode, retrieve the url of the file uploaded by the method getUrl and passes it to another service s2 that is in running on another node. The s2 service retrieves the inputStream of the file that the service s1 had uploaded by the url:
IClient client=new StorageClient("sc", "sn", "s1", AccessType.SHARED, scope).getClient(); String id=client.put(true).LFile("/home/xxx/FilePerTest/test.jpg").RFile("/img/test1.jpg"); String url=client.getUrl().RFile("/img/test1.jpg");
The s2 service retrieve the inputStream of the file by the url stored in the "url" variable:
Handler.activateProtocol(); URL smsHome = new URL("smp://img/test1.jpg5ezvFfBOLqbElsh5xYVgBTxt8lwt4fl"); URLConnection uc = null; uc = ( URLConnection ) smsHome.openConnection(); InputStream is=uc.getInputStream();
storage-manager v 1.0.1
- VOLATILE area
The new version of storage-manager have a new storage area called VOLATILE. The files inserted on this special area are automatically removed after a period of 7 days from the first upload. The use of this area is intended for all users that need to use storage for temporary data. The use of VOLATILE area is very simple and very similar at the standard usage of storage-manager. The only difference is that for use the VOLATILE area you are add a input parameter on the storage-manager client constructor: The new input parameter is an enum called MemoryType, it can have only 2 values:
- VOLATILE for storing in the VOLATILE area: the files are deleting automatically after 7 days
- PERMANENT: for storing in the standard area: the files are deleting only manually
the use of VOLATILE area is suggested to all users who do not need to maintain data on the storage permanently. Through the use of this area will prevent the user to perform the delete operation data manually. is strongly recommended not to use the permanent storage area for storing data that will not be erased and used for a period longer than seven days
- Scenario 1
In this example a GCUBE service s1 (identify by: ServiceClass: sc, ServiceName: sn), running in the GCUBEScope /gcube upload the file: /home/xxx/FilePerTest/test.jpg in shared mode and in the VOLATILE are. Another instance of the same service, s2 (identify by: ServiceClass: sc2, ServiceName: sn2) running in the same scope but on another node, download it.
The following code show the file uploading operation:
GCUBEScope scope = GCUBEScope.getScope("/gcube");
// The Access type can be private | shared | public
IClient client=new StorageClient("sc", "sn", "s1", AccessType.SHARED, scope, MemoryType.VOLATILE).getClient();
String id=client.put(true).LFile("/home/xxx/FilePerTest/test.jpg").RFile("/img/test.jpg");
The following code show the file downloading operation:
GCUBEScope scope = GCUBEScope.getScope("/gcube");
// The Access type can be private | shared | public
IClient client=new StorageClient("sc2", "sn2", "s2", AccessType.SHARED, scope, MemoryType.VOLATILE).getClient();
String id=client.get().LFile("/home/yyy/FilePerTest/test.jpg").RFile("/img/test.jpg");
The file is downloaded in the local directory: /home/yyy/FilePerTest/ where s2 is running and the file name is test.jpg
- Scenario 2
In this example a GCUBE service s1 (identify by: ServiceClass: sc, ServiceName: sn), running in the GCUBEScope /gcube uploads the file: /home/xxx/FilePerTest/test2.jpg in shared mode and in VOLATILE area. Retrieve the url of the file uploaded by the method getUrl and passes it to another service s2 that is in running on another node. The s2 service retrieves the inputStream of the file that the service s1 had uploaded by the url:
IClient client=new StorageClient("sc", "sn", "s1", AccessType.SHARED, scope).getClient(); String id=client.put(true).LFile("/home/xxx/FilePerTest/test.jpg").RFile("/img/test1.jpg"); String url=client.getUrl().RFile("/img/test1.jpg");
The s2 service retrieve the inputStream of the file by the url stored in the "url" variable:
Handler.activateProtocol(); URL smsHome = new URL("smp://img/test1.jpg5ezvFfBOLqbElsh5xYVgBTxt8lwt4fl"); URLConnection uc = null; uc = ( URLConnection ) smsHome.openConnection(); InputStream is=uc.getInputStream();