File-Based Access

Library that support facilities for access and manage objects on a remote File Storage Service. Large and small files are organized in directories under a unified interface. The interface is posix like.

Overview

The Library permits manipulation of remote files like a local fileSystem. You can download, upload remove and add a new file object in a remote File Storage Service. Also is possible remove the contents of a remote directory or show the list of object in a remote directory.

Files may be downloaded and shared by several users. Based on the permission given by the owner of the file. the library can interface with different backends like: MongoDB or Terrastore. Providing a common interface to the user who uses, regardless of the backend used.

The library can interface with different backends for storing files, such as MongoDB or Terrastore. The library offers to the user a common interface that is used regardless of the backend. The backend is responsible for storing files, the library rather than to convey the right way storage requirements. Through the use of metadata, the library lets you organize your files in a structured way even if the backend used and flat type.

Key Features

The subsystem comprises the following components:

Structured-files storage:

    supports structured data. You can build folders tree and files. Folders are made through the use of metadata, so even 
    if the File Storage System does not allow the use of structured data, you can also build directory on a flat storage  
    like mongoDB.

Secure storage:

    access to data is so authenticated: userID, group. 
    Each file has an owner and for each file you can specify access rights: private: read and write access allowed only to                              
    the owner of the file, shared: read and write access granted to all group members, public: read access and write to all 
    users allowed

Data balancing:

    the files are organized in chunks and are distribuited on all the servers of backend file storage system based on 
    the actual load of each server

Scalability:

    horizontal scalability and replication are provided which are necessary functions for large deployments. The horizontal 
    scalability is guaranteeed because the files are organized in chunks and are distribuited on all the servers of backend 
    file storage system

Data replication:

    supports asynchronous replication of data between servers for failover and redundancy.

Design

Philosophy

Navigating through folders on a remote storage system, having the ability to download and upload files, masking the backend system. This is the main goal of this library The library is thinked for preserving a unified interface that aligns with their generality and encapsulates them from the variety of File Storage Service Backend. The two layer: core and wrapper library permit the use of the library in standalone mode or in the Gcube framework.

Architecture

The library is divided in two layer: a core library and a wrapper library. The core library is for generic purpose use, external to gCube framework. The wrapper library is thinked for use internal on gCube framework. The interaction between these two levels permits the use of the library within the framework Gcube The wrapper library interacts with IS for discover server resources that will be used from the core library. The core library interacts with a File Storage Service backend. The file Storage Service has the responsability of data storing. At this time there are 2 kind of file storage service supported: Terrastore and MongoDB.

File based access is provided by the following components:

Core library: Implements a high-level facade to the remote APIs of the File Storage Service. The core dialogues directly with a File storage Service that is responsibles for storing data. this level has the responsibility to split files into chunks if the size exceeds a certain threshold, to build the metadata such as: owner, type of object (file or directory), access permissions, etc. .. It also has the task of issuing commands to the File Storage System for the construction of the tree of folders by metadatas if any were needed

Wrapper library: Is a wrapper library for gCube framework. This library has the task of capturing the resources made available in the framework Gcube and pass them to the core library

File Storage Service: Is a service that have the responsability of remote data storage. This is invoked by the core library and can be based on differents technology like MongoDB, Terrastore.

The following diagram illustrates the dependencies between the components of the subsystem: