Tabular Data Flow Manager
The goal of this facility is to realise an integrated environment supporting the definition and management of workflows of tabular data. Each workflow consists of a number of tabular data processing steps where each step is realized by an existing service offered by a gCube based infrastructure.
In the following, the design rationale, key features, high-level architecture, as well as the deployment scenarios are described.
Overview
The goal of this service is to offer a central service for tabular data flow creation, management and monitoring. The data flow can touch different services in order to produce the desired output. Planned flow can be scheduled for deferred execution and the user notified about the flow progress.
Design
Philosophy
The goal of this service is to offer a central service for tabular data flow creation, management and monitoring. The data flow can touch different services in order to produce the desired output. Planned flow can be scheduled for deferred execution and the user notified about the flow progress.
Architecture
The subsystem comprises the following components:
- Tabular Data Flow Service: the central system for the flow creation, management and monitoring;
- Tabular Data Flow UI: the user interface of the service where the user can create, execute and monitor the data flow;
- Tabular Data Agent: an helper component for the service that want to expose tabular data functionality to the data flow service.
A diagram of the relationships between these components is reported in the following figure:
Deployment
The Service should be deployed in a single node, while the agents should be deployed with the service that want to offer his functionality to the flow service. The User Interface can be deployed in the infrastructure portal.
Use Cases
Well suited Use Cases
This component well fit all the cases where is necessary to manage a flow of tabular data between the infrastructure services. An example can be the enhancement of catch statistics offered by the Time Series Service and elaborated using both the Statistical Service and the Occurrence Service.
