Difference between revisions of "GFeed"

Revision as of 16:01, 20 May 2019

Part of the , this service focuses on coordinating data retrieval from multiple sources and transformation/publication towards multiple destination.

This document outlines the service design rationale, key features, and high-level architecture as well as the options for their deployment.

Overview

gFeed is an extendable service which manages :

• Data Retrieval from multiple sources
• Data Transformation and publication towards multiple destinations

Key Features

The subsystem provides for:

Publication towards multiple sources

depending on available implementations, data collected can be published to multiple destination in the same execution

Data Retrieval from multiple sources

depending on available implementations, data can be collected from multiple sources in the same execution

extendable behaviour

Business logic can be extended by deploying multiple plugins both for data retrieval and data publication

re-usability orientation

The subsystem is conceived to promote the reuse of its facilities by separating data retrieval and publication implementations

Design

Philosophy

The system is designed to maximize modularity in order to isolate implementations.

Data Retrieval and transformation is managed by collector implementations, and a common framework is provided in order to maximize code reusal. Each implementation declares which destination can support, delegating to the collector the transformation logic, which heavily depends on the data represented.

Data Publication is managed by controller implementation, and a common framework is provided in order to maximize code reusal. Each implementation deals with a particular destination, dealing with the logic needed to interact with a specific service.

gFeed itself takes care of orchestrating triggered executions, offering monitoring mechanisms through HTTP interfaces.

Architecture

Deployment

Use Cases