Difference between revisions of "Accounting"

From Gcube Wiki
Jump to: navigation, search
m (Architecture)
(Accounting Enabling Layer)
 
(10 intermediate revisions by 2 users not shown)
Line 9: Line 9:
 
== Key Features ==
 
== Key Features ==
  
;xxx
+
;open and extensible accounting model
:description
+
:the underlying accounting model is flexible to adapt to diverse provider needs;
 +
 
 +
;highly modular and extensible architecture
 +
:the entire subsystem comprise a large number of components clearly separating the functional constituents
 +
 
 +
;diverse options for storage
 +
:the subsystem can rely on an array of diverse solutions for actually storing records
  
 
== Design ==
 
== Design ==
Line 36: Line 42:
 
=== Accounting Enabling Layer ===
 
=== Accounting Enabling Layer ===
  
* [[Accounting Lib | Accounting Lib]] is responsible of collecting, harmonizing and storing '''Accounting''' data. [[Accounting Lib | Accounting Lib]] is mainly based and developed on top of facilities provided by [[Document Store Lib | Document Store Lib]]
+
* [[Accounting Library | Accounting Library]] is responsible of collecting, harmonizing and storing '''Accounting''' data. [[Accounting Library | Accounting Library]] is mainly based and developed on top of facilities provided by [[Document Store Lib | Document Store Lib]]
 
* [[Accounting Analytics | Accounting Analytics]] is responsible of exposing a common access point interface to query the collected '''Accounting''' data.
 
* [[Accounting Analytics | Accounting Analytics]] is responsible of exposing a common access point interface to query the collected '''Accounting''' data.
 
* [[Accounting Aggregator | Accounting Aggregator]] is responsible of aggregating the collected '''Accounting''' data.
 
* [[Accounting Aggregator | Accounting Aggregator]] is responsible of aggregating the collected '''Accounting''' data.
Line 42: Line 48:
 
=== Accounting Storage Layer ===
 
=== Accounting Storage Layer ===
  
This layer is not developed by gCube but the selected technology has to be selected to provide [https://en.wikipedia.org/wiki/High_availability HA (High Availability)].
+
This layer is not developed by gCube. Rather it relies on a technology guaranteeing [https://en.wikipedia.org/wiki/High_availability HA (High Availability)].
Actually CouchDB has been selected as storage for accounting data. MongoDB, OrientDB and CouchBase have been investigated. CouchBase has been selected as next storage for accounting Data.
+
In the current settings, it is implemented by relying on [http://couchdb.apache.org/ CouchDB]. MongoDB, OrientDB and CouchBase have been investigated. CouchBase has been selected as next storage for accounting Data.
  
 
=== Accounting Backend Layer ===
 
=== Accounting Backend Layer ===
  
 
* document-store-lib-BACKEND (implementations available: document-store-lib-couchdb, document-store-lib-couchbase, document-store-lib-mongodb)
 
* document-store-lib-BACKEND (implementations available: document-store-lib-couchdb, document-store-lib-couchbase, document-store-lib-mongodb)
* accounting-analytics-persistence-BACKEND (implementations available: accounting-analytics-persistence-couchdb, accounting-analytics-persistence-couchbase is under development see [https://support.d4science.org/issues/1665])
+
* accounting-analytics-persistence-BACKEND (implementations available: accounting-analytics-persistence-couchdb, accounting-analytics-persistence-couchbase)
* accounting-aggregator-persistence-BACKEND (e.g. accounting-aggregator-persistence-couchdb)
+
* accounting-aggregator-persistence-BACKEND (e.g. accounting-aggregator-persistence-couchdb, accounting-aggregator-persistence-couchbase)
  
 
Each component in this layer has been explicitly developed over a certain storage technology.
 
Each component in this layer has been explicitly developed over a certain storage technology.
Line 63: Line 69:
  
 
In this layer we found not only the accounting consumer which retrieve information related to collected data (e.g. [[Accounting Portlet]]) but also all the component which collect data and use accounting-lib to persist in a common pattern.
 
In this layer we found not only the accounting consumer which retrieve information related to collected data (e.g. [[Accounting Portlet]]) but also all the component which collect data and use accounting-lib to persist in a common pattern.
 +
 +
== Client Library ==
 +
There are two libraries to interact with the Accounting subsystem:
 +
* [[Accounting Lib]] that is used by data producers to "store" records in accordance with the [[Accounting Model]];
 +
* [[Accounting Analytics]] that enacts to consume accounting records;
 +
 +
== Installation ==
 +
In the reality the entire subsystem consists of a series of software libraries, thus no installation is actually needed. Such a set of libraries are available by default in any gCube node.
 +
 +
The "external" component actually realising the storage layer (currently CouchDB) is expected to be deployed and registered in the infrastructure. It will be transparently used via the [[Document Store Lib]].

Latest revision as of 15:11, 6 December 2019

Overview

GCube Accounting allows to collect information of the infrastructure usage and expose them for the interested consumer.

Key Features

open and extensible accounting model
the underlying accounting model is flexible to adapt to diverse provider needs;
highly modular and extensible architecture
the entire subsystem comprise a large number of components clearly separating the functional constituents
diverse options for storage
the subsystem can rely on an array of diverse solutions for actually storing records

Design

Architecture

The following picture shows the gCube Accounting architecture, that is logically divided in three different layers:

Moreover, a fourth layer (which is not gCube-based) exist to take care of the storage of the collected information:

Accounting-architecture.png

All the developed component must respect this architecture. In particular, the design and the implementation has to respect the following rules:

  • Each enabling layer has its own correspondent back-end implementation.
  • Each back-end implementation must be dynamically discovered at run-time. This allow to decouple the deployment of a different back-end from the development of the enabling layer. In other word each component on the enabling layer MUST NOT have any dependency over a certain back-end implementation.

The only entitled software to depend to the back-end component are:

  • bundles (e.g. smartgear-bundle, portal-bundle)
  • provisioning script

Accounting Enabling Layer

Accounting Storage Layer

This layer is not developed by gCube. Rather it relies on a technology guaranteeing HA (High Availability). In the current settings, it is implemented by relying on CouchDB. MongoDB, OrientDB and CouchBase have been investigated. CouchBase has been selected as next storage for accounting Data.

Accounting Backend Layer

  • document-store-lib-BACKEND (implementations available: document-store-lib-couchdb, document-store-lib-couchbase, document-store-lib-mongodb)
  • accounting-analytics-persistence-BACKEND (implementations available: accounting-analytics-persistence-couchdb, accounting-analytics-persistence-couchbase)
  • accounting-aggregator-persistence-BACKEND (e.g. accounting-aggregator-persistence-couchdb, accounting-aggregator-persistence-couchbase)

Each component in this layer has been explicitly developed over a certain storage technology. Each component MUST NOT rely on IS to discover the information needed to connect to the underling storage. In other words the component MUST NOT have an hard-cabled connection information or a local configuration files. To retrieve the storage connection information the following parameter must be part of the query:

  • underlying storage technology
  • enabling component

The first constraint allow to switch to different storage avoiding to switch all nodes together (two underling storage can co-exist). The second allow to keep separated the connection information for each component. This allow to provide create different access policy for different component (e.g. write only for accounting-lib connection and read only for accounting-analytics).

Accounting Consumer Layer

In this layer we found not only the accounting consumer which retrieve information related to collected data (e.g. Accounting Portlet) but also all the component which collect data and use accounting-lib to persist in a common pattern.

Client Library

There are two libraries to interact with the Accounting subsystem:

Installation

In the reality the entire subsystem consists of a series of software libraries, thus no installation is actually needed. Such a set of libraries are available by default in any gCube node.

The "external" component actually realising the storage layer (currently CouchDB) is expected to be deployed and registered in the infrastructure. It will be transparently used via the Document Store Lib.