Difference between revisions of "Accounting"
From Gcube Wiki
Luca.frosini (Talk | contribs) (→Accounting Backend Layer) |
Luca.frosini (Talk | contribs) (→Accounting Enabling Layer) |
||
| (27 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
| + | {| align="right" | ||
| + | ||__TOC__ | ||
| + | |} | ||
| + | |||
| + | == Overview == | ||
| + | |||
GCube Accounting allows to collect information of the infrastructure usage and expose them for the interested consumer. | GCube Accounting allows to collect information of the infrastructure usage and expose them for the interested consumer. | ||
| − | == | + | == Key Features == |
| − | + | ;open and extensible accounting model | |
| + | :the underlying accounting model is flexible to adapt to diverse provider needs; | ||
| − | + | ;highly modular and extensible architecture | |
| + | :the entire subsystem comprise a large number of components clearly separating the functional constituents | ||
| − | The Accounting | + | ;diverse options for storage |
| + | :the subsystem can rely on an array of diverse solutions for actually storing records | ||
| + | |||
| + | == Design == | ||
| + | |||
| + | === Architecture === | ||
| + | |||
| + | The following picture shows the gCube Accounting architecture, that is logically divided in three different layers: | ||
* [[Accounting#Accounting Consumer Layer | Accounting Consumer Layer]] | * [[Accounting#Accounting Consumer Layer | Accounting Consumer Layer]] | ||
* [[Accounting#Accounting Enabling Layer | Accounting Enabling layer]] | * [[Accounting#Accounting Enabling Layer | Accounting Enabling layer]] | ||
* [[Accounting#Accounting Backend Layer | Accounting Backend layer]] | * [[Accounting#Accounting Backend Layer | Accounting Backend layer]] | ||
| − | + | Moreover, a fourth layer (which is not gCube-based) exist to take care of the storage of the collected information: | |
| − | Moreover | + | |
* [[Accounting#Accounting Storage Layer | Accounting Storage layer]] | * [[Accounting#Accounting Storage Layer | Accounting Storage layer]] | ||
| − | + | [[File:accounting-architecture.png]] | |
| − | + | All the developed component must respect this architecture. In particular, the design and the implementation has to respect the following rules: | |
| − | * | + | * Each enabling layer has its own correspondent back-end implementation. |
| − | * | + | * Each back-end implementation must be dynamically discovered at run-time. This allow to decouple the deployment of a different back-end from the development of the enabling layer. In other word each component on the enabling layer MUST NOT have any dependency over a certain back-end implementation. |
| − | + | ||
| − | + | The only entitled software to depend to the back-end component are: | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | The only entitled software | + | |
* bundles (e.g. smartgear-bundle, portal-bundle) | * bundles (e.g. smartgear-bundle, portal-bundle) | ||
* provisioning script | * provisioning script | ||
| + | === Accounting Enabling Layer === | ||
| + | |||
| + | * [[Accounting Library | Accounting Library]] is responsible of collecting, harmonizing and storing '''Accounting''' data. [[Accounting Library | Accounting Library]] is mainly based and developed on top of facilities provided by [[Document Store Lib | Document Store Lib]] | ||
| + | * [[Accounting Analytics | Accounting Analytics]] is responsible of exposing a common access point interface to query the collected '''Accounting''' data. | ||
| + | * [[Accounting Aggregator | Accounting Aggregator]] is responsible of aggregating the collected '''Accounting''' data. | ||
=== Accounting Storage Layer === | === Accounting Storage Layer === | ||
| − | This layer is not developed by gCube | + | This layer is not developed by gCube. Rather it relies on a technology guaranteeing [https://en.wikipedia.org/wiki/High_availability HA (High Availability)]. |
| − | + | In the current settings, it is implemented by relying on [http://couchdb.apache.org/ CouchDB]. MongoDB, OrientDB and CouchBase have been investigated. CouchBase has been selected as next storage for accounting Data. | |
=== Accounting Backend Layer === | === Accounting Backend Layer === | ||
| − | * | + | * document-store-lib-BACKEND (implementations available: document-store-lib-couchdb, document-store-lib-couchbase, document-store-lib-mongodb) |
| − | * accounting-analytics-persistence-BACKEND ( | + | * accounting-analytics-persistence-BACKEND (implementations available: accounting-analytics-persistence-couchdb, accounting-analytics-persistence-couchbase) |
| − | * accounting-aggregator-persistence-BACKEND ( | + | * accounting-aggregator-persistence-BACKEND (e.g. accounting-aggregator-persistence-couchdb, accounting-aggregator-persistence-couchbase) |
Each component in this layer has been explicitly developed over a certain storage technology. | Each component in this layer has been explicitly developed over a certain storage technology. | ||
| Line 51: | Line 64: | ||
The first constraint allow to switch to different storage avoiding to switch all nodes together (two underling storage can co-exist). | The first constraint allow to switch to different storage avoiding to switch all nodes together (two underling storage can co-exist). | ||
| − | The second allow to keep separated the connection information for each component. This allow to provide create different access policy for different component. | + | The second allow to keep separated the connection information for each component. This allow to provide create different access policy for different component (e.g. write only for accounting-lib connection and read only for accounting-analytics). |
=== Accounting Consumer Layer === | === Accounting Consumer Layer === | ||
| − | In this layer we found not only the accounting consumer which retrieve information related to collected data (e.g. Accounting | + | In this layer we found not only the accounting consumer which retrieve information related to collected data (e.g. [[Accounting Portlet]]) but also all the component which collect data and use accounting-lib to persist in a common pattern. |
| + | |||
| + | == Client Library == | ||
| + | There are two libraries to interact with the Accounting subsystem: | ||
| + | * [[Accounting Lib]] that is used by data producers to "store" records in accordance with the [[Accounting Model]]; | ||
| + | * [[Accounting Analytics]] that enacts to consume accounting records; | ||
| + | |||
| + | == Installation == | ||
| + | In the reality the entire subsystem consists of a series of software libraries, thus no installation is actually needed. Such a set of libraries are available by default in any gCube node. | ||
| + | |||
| + | The "external" component actually realising the storage layer (currently CouchDB) is expected to be deployed and registered in the infrastructure. It will be transparently used via the [[Document Store Lib]]. | ||
Latest revision as of 16:11, 6 December 2019
Overview
GCube Accounting allows to collect information of the infrastructure usage and expose them for the interested consumer.
Key Features
- open and extensible accounting model
- the underlying accounting model is flexible to adapt to diverse provider needs;
- highly modular and extensible architecture
- the entire subsystem comprise a large number of components clearly separating the functional constituents
- diverse options for storage
- the subsystem can rely on an array of diverse solutions for actually storing records
Design
Architecture
The following picture shows the gCube Accounting architecture, that is logically divided in three different layers:
Moreover, a fourth layer (which is not gCube-based) exist to take care of the storage of the collected information:
All the developed component must respect this architecture. In particular, the design and the implementation has to respect the following rules:
- Each enabling layer has its own correspondent back-end implementation.
- Each back-end implementation must be dynamically discovered at run-time. This allow to decouple the deployment of a different back-end from the development of the enabling layer. In other word each component on the enabling layer MUST NOT have any dependency over a certain back-end implementation.
The only entitled software to depend to the back-end component are:
- bundles (e.g. smartgear-bundle, portal-bundle)
- provisioning script
Accounting Enabling Layer
- Accounting Library is responsible of collecting, harmonizing and storing Accounting data. Accounting Library is mainly based and developed on top of facilities provided by Document Store Lib
- Accounting Analytics is responsible of exposing a common access point interface to query the collected Accounting data.
- Accounting Aggregator is responsible of aggregating the collected Accounting data.
Accounting Storage Layer
This layer is not developed by gCube. Rather it relies on a technology guaranteeing HA (High Availability). In the current settings, it is implemented by relying on CouchDB. MongoDB, OrientDB and CouchBase have been investigated. CouchBase has been selected as next storage for accounting Data.
Accounting Backend Layer
- document-store-lib-BACKEND (implementations available: document-store-lib-couchdb, document-store-lib-couchbase, document-store-lib-mongodb)
- accounting-analytics-persistence-BACKEND (implementations available: accounting-analytics-persistence-couchdb, accounting-analytics-persistence-couchbase)
- accounting-aggregator-persistence-BACKEND (e.g. accounting-aggregator-persistence-couchdb, accounting-aggregator-persistence-couchbase)
Each component in this layer has been explicitly developed over a certain storage technology. Each component MUST NOT rely on IS to discover the information needed to connect to the underling storage. In other words the component MUST NOT have an hard-cabled connection information or a local configuration files. To retrieve the storage connection information the following parameter must be part of the query:
- underlying storage technology
- enabling component
The first constraint allow to switch to different storage avoiding to switch all nodes together (two underling storage can co-exist). The second allow to keep separated the connection information for each component. This allow to provide create different access policy for different component (e.g. write only for accounting-lib connection and read only for accounting-analytics).
Accounting Consumer Layer
In this layer we found not only the accounting consumer which retrieve information related to collected data (e.g. Accounting Portlet) but also all the component which collect data and use accounting-lib to persist in a common pattern.
Client Library
There are two libraries to interact with the Accounting subsystem:
- Accounting Lib that is used by data producers to "store" records in accordance with the Accounting Model;
- Accounting Analytics that enacts to consume accounting records;
Installation
In the reality the entire subsystem consists of a series of software libraries, thus no installation is actually needed. Such a set of libraries are available by default in any gCube node.
The "external" component actually realising the storage layer (currently CouchDB) is expected to be deployed and registered in the infrastructure. It will be transparently used via the Document Store Lib.
