Difference between revisions of "StorageHub REST API"

From Gcube Wiki
Jump to: navigation, search
(→‎Folder Listing)
(→‎API)
 
(80 intermediate revisions by 5 users not shown)
Line 1: Line 1:
 +
{| align="right"
 +
||__TOC__
 +
|}
 +
 
= Overview =
 
= Overview =
  
−
The StorageHub APIs provide a simple access (via Java or REST) to SH features.
+
The StorageHub APIs components provide a simple access to StorageHub service.
 +
 
 +
It is conceived to support both Java and REST-based calls.  
 +
 
 +
== Dependencies ==
 +
 
 +
=== Maven coordinates ===
 +
 
 +
<source lang="java">
 +
<dependency>
 +
<groupId>org.gcube.common</groupId>
 +
<artifactId>storagehub-client-library</artifactId>
 +
<version>[1.0.0,2.0.0)</version>
 +
</dependency>
 +
</source>
  
 
= Key features =
 
= Key features =
−
Users must use the personal to access the REST interface. They can access just their own files and the folders shared with them.
+
Users must use the personal token to access the REST interface. They can access just their own files and the folders shared with them.
  
 
'''StorageHub REST interface''' supports the following operations:
 
'''StorageHub REST interface''' supports the following operations:
Line 10: Line 28:
 
* '''Folder Listing''': to list the content of a folder;
 
* '''Folder Listing''': to list the content of a folder;
 
* '''Retrieve VRE Folder''': to retrieve the VREFolder related to the token;
 
* '''Retrieve VRE Folder''': to retrieve the VREFolder related to the token;
 +
* '''Find''': to find a file or a folder by name or pattern;
 
* '''Delete''': to remove a file or a folder (including subfolders);
 
* '''Delete''': to remove a file or a folder (including subfolders);
 
* '''Download''': to download a file or a folder in  ZIP format;
 
* '''Download''': to download a file or a folder in  ZIP format;
Line 16: Line 35:
 
* '''Unzip''': to upload a zip file in a specific folder;
 
* '''Unzip''': to upload a zip file in a specific folder;
 
* '''Upload file''': to upload a file in a folder.
 
* '''Upload file''': to upload a file in a folder.
 +
* '''Versions''': to get a specific version of a file;
  
 
== API ==
 
== API ==
 +
 +
The base-url of this service on D4Science production is the following:
 +
 +
https://api.d4science.org
  
 
== Retrieve Workspace ==
 
== Retrieve Workspace ==
Line 25: Line 49:
 
=== Java ===
 
=== Java ===
 
<source lang="java">
 
<source lang="java">
 +
import org.gcube.common.storagehub.client.dsl.StorageHubClient;
 +
import org.gcube.common.storagehub.client.dsl.FileContainer;
 +
import org.gcube.common.storagehub.model.items.Item;
 +
 +
 
  StorageHubClient shc = new StorageHubClient();
 
  StorageHubClient shc = new StorageHubClient();
−
  FolderContainer rootContianer = shc.getWSRoot()
+
  FolderContainer rootContainer = shc.getWSRoot()
 
</source>
 
</source>
  
Line 37: Line 66:
 
200 The workspace root item (in json format) is returned.
 
200 The workspace root item (in json format) is returned.
  
−
400 The error is specified in the body of the response message
+
500 The error is specified in the body of the response message
 +
 
 +
== Get Item ById ==
 +
 
 +
=== Java ===
 +
<source lang="java">
 +
import org.gcube.common.storagehub.client.dsl.FileContainer;
 +
import org.gcube.common.storagehub.client.dsl.StorageHubClient;
 +
import org.gcube.common.storagehub.model.items.Item;
 +
 
 +
...
 +
 
 +
String theId = {itemIdentifier} // the identifier of the item
 +
StorageHubClient shc = new StorageHubClient();
 +
FileContainer fileContainer = shc.open(theId).asFile();
 +
</source>
 +
 
 +
 
 +
== List Item Versions, Get Item Version ==
 +
 
 +
=== Java ===
 +
<source lang="java">
 +
import org.gcube.common.storagehub.client.dsl.FileContainer;
 +
import org.gcube.common.storagehub.client.dsl.StorageHubClient;
 +
import org.gcube.common.storagehub.model.items.Item;
 +
import org.gcube.common.storagehub.model.service.Version;
 +
 
 +
...
 +
 
 +
String theId = {itemIdentifier} // the identifier of the item
 +
StorageHubClient shc = new StorageHubClient();
 +
FileContainer fileContainer = shc.open(theId).asFile();
 +
List<Version> fileVersions = fileContainer.getVersions();
 +
 
 +
//to download a version
 +
fileContainer.downloadSpecificVersion(versionName)
 +
</source>
  
 
== Folder Listing ==
 
== Folder Listing ==
Line 46: Line 111:
  
 
<source lang="java">
 
<source lang="java">
 +
import org.gcube.common.storagehub.model.items.*;
 +
 +
...
 +
 
StorageHubClient shc = new StorageHubClient();
 
StorageHubClient shc = new StorageHubClient();
 
FolderContainer folderContainer = shc.open("{folderIdentifier}").asFolder();
 
FolderContainer folderContainer = shc.open("{folderIdentifier}").asFolder();
 
List<? extends Item> items = folderContainer.list().getItems();
 
List<? extends Item> items = folderContainer.list().getItems();
 +
 +
for (Item item :  items) {
 +
System.out.println("name:" + item.getName() + " is a File?: " + (item instanceof AbstractFileItem));
 +
System.out.println("name:" + item.getName() + " is a folder?: " + (item instanceof FolderItem));
 +
System.out.println("name:" + item.getName() + " is a shared folder?: " + (item instanceof SharedFolder));
 +
System.out.println("name:" + item.getName() + " is a VRE folder?: " + (item instanceof VreFolder));
 +
}
 +
 
</source>
 
</source>
  
−
=== REST API ===
+
...
 +
//hide or show folder
  
−
GET /workspace/items/{itemIdentifier}/?gcube-token={userToken}
+
<source lang="java">
 +
        StorageHubClient shc = new StorageHubClient();
 +
        FolderContainer folderContainer = shc.open("{folderIdentifier}").asFolder();
 +
folderContainer.setHidden(); //will not appear in the workspace GUI
 +
        folderContainer.setVisible();
 +
</source>
  
−
==== parameters ====
+
=== REST API ===
  
−
{itemIdentifier} the identifier of the item to list the content
+
GET /workspace/items/{folder-identifier}/children?gcube-token={userToken}
  
 
==== responses ====
 
==== responses ====
Line 63: Line 146:
 
200 the list of the Items (in JSON format)
 
200 the list of the Items (in JSON format)
  
−
400 The error is specified in the body of the response message
+
500 The error is specified in the body of the response message
  
 
== Retrieve VRE Folder ==
 
== Retrieve VRE Folder ==
 +
 +
Return the Item representing the root of the VRE folder related to the user token.
 +
 +
=== Java ===
 +
<source lang="java">
 +
StorageHubClient shc = new StorageHubClient();
 +
FolderContainer rootContainer = shc.openVREFolder();
 +
</source>
 +
 +
 +
Check that a folder with name "FolderNameToCheck" exists in the VRE folder related to the user token.
 +
=== Java ===
 +
<source lang="java">
 +
import org.gcube.common.storagehub.client.dsl.*;
 +
import org.gcube.common.storagehub.model.exceptions.StorageHubException;
 +
 +
...
 +
 +
StorageHubClient shc = new StorageHubClient();
 +
FolderContainer vreFolder = shc.openVREFolder();
 +
FolderContainer folderToCheck = null;
 +
try {
 +
      OpenResolver oRes = vreFolder.openByRelativePath("FolderNameToCheck");
 +
      folderToCheck = oRes.asFolder();
 +
} catch (StorageHubException e) {
 +
    System.out.println("FolderNameToCheck does not exists in VRE Folder, creating it");
 +
    folderToCheck = vreFolder.newFolder("FolderNameToCheck description goes here");
 +
}
 +
</source>
 +
 +
=== REST API ===
 +
 +
GET /workspace/vrefolder?gcube-token={user-token}
 +
 +
==== responses ====
 +
 +
200 The VRE folder root item (in json format) is returned.
 +
 +
500 The error is specified in the body of the response message
 +
 +
== Create Folder ==
 +
 +
Creates a new folder under another folder (specified by its id)
 +
 +
=== Java ===
 +
<source lang="java">
 +
  StorageHubClient shc = new StorageHubClient();
 +
  FolderContainer root = shc.getWSRoot();
 +
  //Creating the folder on the root workspace folder
 +
  root.newFolder("name", "description");
 +
</source>
 +
 +
=== REST API ===
 +
 +
POST /workspace/items/{destiantion-folder-id}/create/FOLDER?gcube-token={user-token}
 +
Content-Type: application/x-www-form-urlencoded
 +
{String name, String description, boolean hidden}
 +
 +
 +
==== responses ====
 +
 +
200 The Folder item identifier is returned.
 +
 +
500 The error is specified in the body of the response message
 +
 +
== Share Folder ==
 +
 +
Shares a folder between users setting the access type (WRITE_ALL, ADMINISTRATOR, READ_ONLY, WRITE_OWNER)
 +
 +
=== JAVA ===
 +
 +
<source lang="java">
 +
StorageHubClient shc = new StorageHubClient();
 +
shc.open("{folderID}").asFolder().share(Arrays.asList("user1",.."userN"), AccessType.WRITE_ALL);
 +
</source>
 +
 +
== Move Item ==
 +
 +
Moves a file or a folder (including subfolders) on another folder.
 +
 +
=== Java ===
 +
 +
<source lang="java">
 +
ItemContainer<Item> item = shc.open("itemToMoveId").asItem();
 +
FolderContainer destinationFolder = shc.open("destination").asFolder();
 +
file.move(destinationFolder);
 +
</source>
 +
 +
=== REST API ===
 +
 +
PUT /workspace/items/{item-to-move-id}/move/?gcube-token={user-token}  Content-Type: application/x-www-form-urlencoded {String destinationId}
 +
 +
== Delete Item ==
 +
 +
Deletes (moves to the trash bin or permanently) a file or folder (including subfolders) given its id;
 +
 +
=== Java ===
 +
<source lang="java">
 +
String theId = {itemIdentifier} // the identifier of the item
 +
StorageHubClient shc = new StorageHubClient();
 +
FileContainer fileContainer = shc.open(theId).asFile();
 +
fileContainer.delete();
 +
</source>
 +
 +
=== REST API ===
 +
 +
DELETE /workspace/items/{id}?gcube-token={user-token} (moves to the trash)
 +
 +
DELETE /workspace/items/{id}?force=true&gcube-token={user-token} (delete permanently)
 +
 +
 +
==== responses ====
 +
 +
200 The Folder or item is deleted
 +
 +
500 The error is specified in the body of the response message
 +
 +
== Empty Trash ==
 +
 +
Removes all the items from the trash
 +
 +
=== JAVA ===
 +
 +
<source lang="java">
 +
StorageHubClient shc = new StorageHubClient();
 +
shc.emptyTrash();
 +
</source>
 +
 +
== Download File or Folder  ==
 +
 +
=== Java ===
 +
<source lang="java">
 +
import org.gcube.common.storagehub.client.dsl.FileContainer;
 +
import org.gcube.common.storagehub.client.dsl.StorageHubClient;
 +
import org.gcube.common.storagehub.model.items.Item;
 +
 +
...
 +
 +
String theId = {itemIdentifier} // the identifier of the item
 +
StorageHubClient shc = new StorageHubClient();
 +
StreamDescriptor streamDescr = shc.open(theId).asFile().download();
 +
 +
File output = {output file};
 +
try (BufferedInputStream bi = new BufferedInputStream(streamDescr.getStream()); FileOutputStream fo = new FileOutputStream(output)){
 +
byte[] buf = new byte[2046];
 +
int read = -1;
 +
while ((read=bi.read(buf))!=-1) {
 +
fo.write(buf, 0, read);
 +
}
 +
}
 +
</source>
 +
 +
=== REST API ===
 +
 +
GET /workspace/items/{itemId}/download?gcube-token={user-token}
 +
 +
'''Please note:''' in case the itemId corresponds to a folder you will get a compressed file (zip) of the folder with all its content.
 +
 +
== Upload File ==
 +
 +
Creates a new file under a folder (specified by its id)
 +
 +
=== Java ===
 +
<source lang="java">
 +
  StorageHubClient shc = new StorageHubClient();
 +
  FolderContainer root = shc.getWSRoot();
 +
  //Creating the folder on the root workspace folder
 +
  FileContainer file = null;
 +
  try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){
 +
file = root.uploadFile(is, "name", "description");
 +
        System.out.println("Uploaded " + file.get().getName() + " - Returned Workspace id=" + file.getId());
 +
  } catch (Exception e) {
 +
//print the error
 +
  }
 +
</source>
 +
 +
=== REST API ===
 +
 +
POST /workspace/items/{destination-folder-id}/create/FILE?gcube-token={user-token}
 +
Content-Type: multipart/form-data
 +
{String name, String description, File file}
 +
 +
==== CURL Example ====
 +
 +
curl -F "name=hspentest.csv" -F "description=desc" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/FILE?gcube-token={token}
 +
 +
== Upload Archive ==
 +
 +
Extract the content of the specified archive in a destination Folder (specified by its id)
 +
 +
=== Java ===
 +
<source lang="java">
 +
  StorageHubClient shc = new StorageHubClient();
 +
  FolderContainer root = shc.getWSRoot();
 +
  //Creating the folder on the root workspace folder
 +
  FileContainer file = null;
 +
  try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){
 +
file = root.uploadArchive(is, "parentFolderName");
 +
  } catch (Exception e) {
 +
//print the error
 +
  }
 +
</source>
 +
 +
 +
=== REST API ===
 +
 +
POST /workspace/items/{destination-folder-id}/create/ARCHIVE?gcube-token={user-token}
 +
Content-Type: multipart/form-data
 +
{String parentFolderName, File file}
 +
 +
==== CURL Example ====
 +
 +
curl -F "parentFolderName=hspentest.csv" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/ARCHIVE?gcube-token={token}
 +
 +
== Get Public Url of a File ==
 +
 +
Returns the public URL of a file
 +
 +
=== Java ===
 +
<source lang="java">
 +
  StorageHubClient shc = new StorageHubClient();
 +
  shc.open({item-id}).asFile().getPublicLink();
 +
 
 +
  //returns the public link for a specific version of the file 
 +
  shc.open({item-id}).asFile().getPublicLink({file-version});
 +
 
 +
</source>
 +
 +
=== REST API ===
 +
 +
GET /workspace/items/{item-id}/publiclink?gcube-token={user-token}&version={file-version} //version is optional
 +
 +
== Find By Name ==
 +
 +
Returns all the items with a name that matches a pattern on the first level of a given folder.
 +
The result will be empty in case of none of the item in the folder matches the pattern.
 +
Wildcard (*) can be used at the start or the end of the pattern (eg. starts with  *{name}, ends with = {name}*)
 +
 +
=== Java ===
 +
<source lang="java">
 +
  StorageHubClient shc = new StorageHubClient();
 +
  //getting my root workspace folder
 +
  FolderContainer myRoot = shc.getWSRoot();
 +
  myRoot.findByName("{pattern}");
 +
</source>
 +
 +
=== REST API ===
 +
 +
GET /workspace/items/{parent-folder-id}/items/{pattern}?gcube-token={user-token}
 +
 +
==== responses ====
 +
 +
200 A JSON with the retrieved items or an empty itemList.
 +
 +
500 The error is specified in the body of the response message
 +
 +
 +
== Add User To Vre ==
 +
 +
Adds a storagehub user to a VRE
 +
 +
 +
=== REST API ===
 +
 +
PUT /workspace/groups/{groupName*}/users {String userId}
 +
 +
* full vre name removing the first / and replacing / with - for example: for context '''''/d4science.research-infrastructures.eu/D4Research/open-science-it''''' it will be '''''d4science.research-infrastructures.eu-D4Research-open-science-it'''''
 +
 +
==== Example ====
 +
 +
curl -d  "userId=lucio.lelii" -X PUT -H "gcube-token: ..." -H
 +
"Content-Type: application/x-www-form-urlencoded" https://api.d4science.org/workspace/groups/d4science.research-infrastructures.eu-D4Research-open-science-it/users
 +
 +
 +
== Create User ==
 +
create a new user in storagehub
 +
 +
 +
=== REST API ===
 +
 +
POST /workspace/users {String user, String password}
 +
 +
 +
==== Example ====
 +
 +
curl -d  "user=lucio.lelii&password=pwd" -X POST -H "Authorization: Bearer ..." -H
 +
"Content-Type: application/x-www-form-urlencoded" https://api.d4science.org/workspace/users
 +
 +
= DataMiner and SAI Interactions =
 +
Algorithms created in [[Statistical_Algorithms_Importer|SAI]] and executed by [[DataMiner_Manager|DataMiner]] can interact with StorageHub through the StorageHub REST APIs.
 +
Here are some examples:
 +
* [[Statistical_Algorithms_Importer:_Java_Project_FAQ#StorageHub|StorageHub Facility Java]]
 +
* [[Statistical_Algorithms_Importer:_Python_Project_FAQ#StorageHub|StorageHub Facility Python]]
 +
 +
Furthermore, everything can be facilitated by using the specific input parameters as shown in the SAI wiki section [[Advanced_Input#Workspace_Interaction_by_Item_Id|Workspace Interaction by Item Id]].
 +
 +
= R Client =
 +
An R client implementing the above function is available [https://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-analysis/RConfiguration/RD4SFunctions/workspace_interaction.r here]

Latest revision as of 09:23, 26 January 2022

Overview

The StorageHub APIs components provide a simple access to StorageHub service.

It is conceived to support both Java and REST-based calls.

Dependencies

Maven coordinates

<dependency>
<groupId>org.gcube.common</groupId>
<artifactId>storagehub-client-library</artifactId>
<version>[1.0.0,2.0.0)</version>
</dependency>

Key features

Users must use the personal token to access the REST interface. They can access just their own files and the folders shared with them.

StorageHub REST interface supports the following operations:

  • Retrieve WS: to retrieve the user Workspace;
  • Folder Listing: to list the content of a folder;
  • Retrieve VRE Folder: to retrieve the VREFolder related to the token;
  • Find: to find a file or a folder by name or pattern;
  • Delete: to remove a file or a folder (including subfolders);
  • Download: to download a file or a folder in ZIP format;
  • Get Public Link: to get a public link of a file;
  • Create Folder: to create a folder in the given parent folder;
  • Unzip: to upload a zip file in a specific folder;
  • Upload file: to upload a file in a folder.
  • Versions: to get a specific version of a file;

API

The base-url of this service on D4Science production is the following: 

https://api.d4science.org

Retrieve Workspace

Returns the Item representing the workspace of the user retrieved by the token used for the call.

Java

import org.gcube.common.storagehub.client.dsl.StorageHubClient;
import org.gcube.common.storagehub.client.dsl.FileContainer;
import org.gcube.common.storagehub.model.items.Item;
 
 
 StorageHubClient shc = new StorageHubClient();
 FolderContainer rootContainer = shc.getWSRoot()

REST API

GET /workspace/?gcube-token={user-token}

responses

200 The workspace root item (in json format) is returned.

500 The error is specified in the body of the response message

Get Item ById

Java

import org.gcube.common.storagehub.client.dsl.FileContainer;
import org.gcube.common.storagehub.client.dsl.StorageHubClient;
import org.gcube.common.storagehub.model.items.Item;
 
...
 
String theId = {itemIdentifier} // the identifier of the item 
StorageHubClient shc = new StorageHubClient();
FileContainer fileContainer = shc.open(theId).asFile();


List Item Versions, Get Item Version

Java

import org.gcube.common.storagehub.client.dsl.FileContainer;
import org.gcube.common.storagehub.client.dsl.StorageHubClient;
import org.gcube.common.storagehub.model.items.Item;
import org.gcube.common.storagehub.model.service.Version;
 
...
 
String theId = {itemIdentifier} // the identifier of the item 
StorageHubClient shc = new StorageHubClient();
FileContainer fileContainer = shc.open(theId).asFile();
List<Version> fileVersions = fileContainer.getVersions();
 
//to download a version
fileContainer.downloadSpecificVersion(versionName)

Folder Listing

Returns the content of a Folder

Java

import org.gcube.common.storagehub.model.items.*;
 
...
 
StorageHubClient shc = new StorageHubClient();
FolderContainer folderContainer = shc.open("{folderIdentifier}").asFolder();
List<? extends Item> items = folderContainer.list().getItems();
 
for (Item item :  items) {
	System.out.println("name:" + item.getName() + " is a File?: " + (item instanceof AbstractFileItem));
	System.out.println("name:" + item.getName() + " is a folder?: " + (item instanceof FolderItem));
	System.out.println("name:" + item.getName() + " is a shared folder?: " + (item instanceof SharedFolder));
	System.out.println("name:" + item.getName() + " is a VRE folder?: " + (item instanceof VreFolder));
}

... //hide or show folder 

        StorageHubClient shc = new StorageHubClient();
        FolderContainer folderContainer = shc.open("{folderIdentifier}").asFolder();
	folderContainer.setHidden(); //will not appear in the workspace GUI
        folderContainer.setVisible();

REST API

GET /workspace/items/{folder-identifier}/children?gcube-token={userToken}

responses

200 the list of the Items (in JSON format)

500 The error is specified in the body of the response message

Retrieve VRE Folder

Return the Item representing the root of the VRE folder related to the user token.

Java

 StorageHubClient shc = new StorageHubClient();
 FolderContainer rootContainer = shc.openVREFolder();


Check that a folder with name "FolderNameToCheck" exists in the VRE folder related to the user token.

Java

import org.gcube.common.storagehub.client.dsl.*;
import org.gcube.common.storagehub.model.exceptions.StorageHubException;
 
...
 
StorageHubClient shc = new StorageHubClient();
FolderContainer vreFolder = shc.openVREFolder();
FolderContainer folderToCheck = null;
try { 
      OpenResolver oRes = vreFolder.openByRelativePath("FolderNameToCheck");
      folderToCheck = oRes.asFolder();
} catch (StorageHubException e) {
     System.out.println("FolderNameToCheck does not exists in VRE Folder, creating it");
     folderToCheck = vreFolder.newFolder("FolderNameToCheck description goes here");
}

REST API

GET /workspace/vrefolder?gcube-token={user-token}

responses

200 The VRE folder root item (in json format) is returned.

500 The error is specified in the body of the response message

Create Folder

Creates a new folder under another folder (specified by its id)

Java

   StorageHubClient shc = new StorageHubClient();
   FolderContainer root = shc.getWSRoot();
   //Creating the folder on the root workspace folder
   root.newFolder("name", "description");

REST API

POST /workspace/items/{destiantion-folder-id}/create/FOLDER?gcube-token={user-token} Content-Type: application/x-www-form-urlencoded {String name, String description, boolean hidden}


responses

200 The Folder item identifier is returned.

500 The error is specified in the body of the response message

Share Folder

Shares a folder between users setting the access type (WRITE_ALL, ADMINISTRATOR, READ_ONLY, WRITE_OWNER)

JAVA

StorageHubClient shc = new StorageHubClient();
shc.open("{folderID}").asFolder().share(Arrays.asList("user1",.."userN"), AccessType.WRITE_ALL);

Move Item

Moves a file or a folder (including subfolders) on another folder.

Java

ItemContainer<Item> item = shc.open("itemToMoveId").asItem();
FolderContainer destinationFolder = shc.open("destination").asFolder();
file.move(destinationFolder);

REST API

PUT /workspace/items/{item-to-move-id}/move/?gcube-token={user-token} Content-Type: application/x-www-form-urlencoded {String destinationId}

Delete Item

Deletes (moves to the trash bin or permanently) a file or folder (including subfolders) given its id;

Java

String theId = {itemIdentifier} // the identifier of the item 
StorageHubClient shc = new StorageHubClient();
FileContainer fileContainer = shc.open(theId).asFile();
fileContainer.delete();

REST API

DELETE /workspace/items/{id}?gcube-token={user-token} (moves to the trash)

DELETE /workspace/items/{id}?force=true&gcube-token={user-token} (delete permanently)


responses

200 The Folder or item is deleted

500 The error is specified in the body of the response message

Empty Trash

Removes all the items from the trash

JAVA

 
StorageHubClient shc = new StorageHubClient();
shc.emptyTrash();

Download File or Folder

Java

import org.gcube.common.storagehub.client.dsl.FileContainer;
import org.gcube.common.storagehub.client.dsl.StorageHubClient;
import org.gcube.common.storagehub.model.items.Item;
 
...
 
String theId = {itemIdentifier} // the identifier of the item 
StorageHubClient shc = new StorageHubClient();
StreamDescriptor streamDescr = shc.open(theId).asFile().download();
 
File output = {output file};
try (BufferedInputStream bi = new BufferedInputStream(streamDescr.getStream()); FileOutputStream fo = new FileOutputStream(output)){
	byte[] buf = new byte[2046];			
	int read = -1;
	while ((read=bi.read(buf))!=-1) {
		fo.write(buf, 0, read);
	}
}

REST API

GET /workspace/items/{itemId}/download?gcube-token={user-token}

Please note: in case the itemId corresponds to a folder you will get a compressed file (zip) of the folder with all its content.

Upload File

Creates a new file under a folder (specified by its id)

Java

   StorageHubClient shc = new StorageHubClient();
   FolderContainer root = shc.getWSRoot();
   //Creating the folder on the root workspace folder
   FileContainer file = null;
   try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){
	file = root.uploadFile(is, "name", "description");
        System.out.println("Uploaded " + file.get().getName() + " - Returned Workspace id=" + file.getId());	
   } catch (Exception e) {
	//print the error
   }

REST API

POST /workspace/items/{destination-folder-id}/create/FILE?gcube-token={user-token} Content-Type: multipart/form-data {String name, String description, File file}

CURL Example

curl -F "name=hspentest.csv" -F "description=desc" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/FILE?gcube-token={token}

Upload Archive

Extract the content of the specified archive in a destination Folder (specified by its id)

Java

   StorageHubClient shc = new StorageHubClient();
   FolderContainer root = shc.getWSRoot();
   //Creating the folder on the root workspace folder
   FileContainer file = null;
   try(InputStream is = new FileInputStream(new File("{file-to-upload}"))){
	file = root.uploadArchive(is, "parentFolderName");
   } catch (Exception e) {
	//print the error
   }


REST API

POST /workspace/items/{destination-folder-id}/create/ARCHIVE?gcube-token={user-token} Content-Type: multipart/form-data {String parentFolderName, File file}

CURL Example

curl -F "parentFolderName=hspentest.csv" -F "file=@/home/lucio/Downloads/hspen.csv" protocol://host:port/storagehub/workspace/items/{folder-destination-id}/create/ARCHIVE?gcube-token={token}

Get Public Url of a File

Returns the public URL of a file

Java

   StorageHubClient shc = new StorageHubClient();
   shc.open({item-id}).asFile().getPublicLink();
 
   //returns the public link for a specific version of the file  
   shc.open({item-id}).asFile().getPublicLink({file-version});

REST API

GET /workspace/items/{item-id}/publiclink?gcube-token={user-token}&version={file-version} //version is optional

Find By Name

Returns all the items with a name that matches a pattern on the first level of a given folder. The result will be empty in case of none of the item in the folder matches the pattern. Wildcard (*) can be used at the start or the end of the pattern (eg. starts with *{name}, ends with = {name}*)

Java

   StorageHubClient shc = new StorageHubClient();
   //getting my root workspace folder
   FolderContainer myRoot = shc.getWSRoot();
   myRoot.findByName("{pattern}");

REST API

GET /workspace/items/{parent-folder-id}/items/{pattern}?gcube-token={user-token}

responses

200 A JSON with the retrieved items or an empty itemList.

500 The error is specified in the body of the response message


Add User To Vre

Adds a storagehub user to a VRE


REST API

PUT /workspace/groups/{groupName*}/users {String userId}

  • full vre name removing the first / and replacing / with - for example: for context /d4science.research-infrastructures.eu/D4Research/open-science-it it will be d4science.research-infrastructures.eu-D4Research-open-science-it

Example

curl -d "userId=lucio.lelii" -X PUT -H "gcube-token: ..." -H "Content-Type: application/x-www-form-urlencoded" https://api.d4science.org/workspace/groups/d4science.research-infrastructures.eu-D4Research-open-science-it/users


Create User

create a new user in storagehub


REST API

POST /workspace/users {String user, String password}


Example

curl -d "user=lucio.lelii&password=pwd" -X POST -H "Authorization: Bearer ..." -H "Content-Type: application/x-www-form-urlencoded" https://api.d4science.org/workspace/users

DataMiner and SAI Interactions

Algorithms created in SAI and executed by DataMiner can interact with StorageHub through the StorageHub REST APIs. Here are some examples:

Furthermore, everything can be facilitated by using the specific input parameters as shown in the SAI wiki section Workspace Interaction by Item Id.

R Client

An R client implementing the above function is available here

Retrieved from "https://gcube.wiki.gcube-system.org/index.php?title=StorageHub_REST_API&oldid=34729"