Difference between revisions of "Accounting Library"

From Gcube Wiki
Jump to: navigation, search
(→‎Library)
(→‎Development)
 
(24 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 +
{| align="right"
 +
||__TOC__
 +
|}
 +
 +
[[Category: Developer's Guide]]
 +
 
This component is responsible of collecting, harmonizing and storing '''Accounting''' data.
 
This component is responsible of collecting, harmonizing and storing '''Accounting''' data.
  
−
== Accounting Lib in the Accounting Architecture ==
+
== Overview ==
  
 
The following image evidences the accounting-lib component as part of [[Accounting | Accounting Architecture]]:
 
The following image evidences the accounting-lib component as part of [[Accounting | Accounting Architecture]]:
Line 8: Line 14:
  
 
== Library ==
 
== Library ==
−
The library has been tough to provide a common way to elaborate '''Usage Record''' (''UR'') (see [[Accounting Model]] to check which Usage Records are supported by default). In particular is responsible for:
+
The library has been implemented to provide a common way to elaborate '''Usage Record''' (''UR'') (see [[Accounting Model]] to check which Usage Records are supported by default). In particular is responsible for:
 
* Collecting Accounting Data
 
* Collecting Accounting Data
  
−
It uses facilities provided by [[Document Store Lib | Document Store Lib]] for:
+
It uses facilities provided by [[Document Store Library | Document Store Library]] for:
  
 
* Aggregating
 
* Aggregating
 
* Buffering
 
* Buffering
 
* Storing
 
* Storing
−
* Providing '''Fallback and Retry''' capability
+
* Providing ''Fallback and Retry'' capability
  
 
=== Collecting Usage Record ===
 
=== Collecting Usage Record ===
Line 34: Line 40:
 
Moreover you can provide a Map<String,Comparable<? extends Serializable>> to set all properties at once.
 
Moreover you can provide a Map<String,Comparable<? extends Serializable>> to set all properties at once.
  
−
==== Usage Record Record Validation and Harmonization ====
+
==== Usage Record Validation and Harmonization ====
  
 
This library provides support to validate and harmonize the values to a common type (e.g. the string representation on an integer to a integer type, the string representation of an enum to the enum type).
 
This library provides support to validate and harmonize the values to a common type (e.g. the string representation on an integer to a integer type, the string representation of an enum to the enum type).
Line 41: Line 47:
 
For further details see [[Accounting Model]].
 
For further details see [[Accounting Model]].
  
−
=== Buffering & Aggregating Usage Record ===
+
== Code Example ==
  
−
The library provides buffering capabilities.
+
The following snippet of code provide an example of '''ServiceUsageRecord''' account which is automatically made from wHN and gHN.
−
For each new request of accounting a new record, the library buffers them and check is is possible to aggregate with the ones has already buffered. The aggregation made in this phase is lossless. In other words, the ''UsageRecord'' aggregated are only the ones having the same values for required fields.
+
See [[Accounting Model#ServiceUsageRecord | Service Usage Record Model]]
  
−
=== Storing Usage Record ===
 
  
−
The library discover dynamically the '''Accounting Persistence Lib''' to persist the ''UsageRecord''s.
 
−
If no library is discovered or if the storing fails (e.g. for network issue) the ''UsageRecord''s are persisted using a '''Fallback Persistence''' (on filesystem).
 
−
There is a recurrent task which periodically (10 min and only if an '''Accounting Persistence Lib''' has been discovered on classpath) retry to persist ''UsageRecord''s persisted from '''FallBack Persistence''' (on filesystem).
 
  
−
== Development ==
+
<syntaxhighlight lang="java">
  
−
The library is identified by the following maven coordinates:
+
// This code auto-discover the available persistence and retrieve the configuration from IS to correctly initialize it.
 +
AccountingPersistence accountingPersistence = AccountingPersistenceFactory.getPersistence();
  
−
''<groupId>'''''org.gcube.accounting'''''</groupId>''
+
// Create a new Service Usage Record
 +
ServiceUsageRecord usageRecord = new ServiceUsageRecord();
  
−
''<artifactId>'''''accounting-lib'''''</artifactId>''
+
/*
 +
* Commented to show the use of key-value setResourceProperty function
 +
* instead of the convenient setter
 +
* usageRecord.setConsumerId("luca.frosini");
 +
* Please note that as KEY is used the provided static variable. Always behave like this.
 +
*/
 +
usageRecord.setResourceProperty(ServiceUsageRecord.CONSUMER_ID, "luca.frosini");
  
 +
usageRecord.setOperationResult(OperationResult.SUCCESS);
 +
usageRecord.setCallerHost("node13.d4science.org");
 +
usageRecord.setHost("node22.d4science.org");
 +
usageRecord.setServiceClass("VREManagement");
 +
usageRecord.setServiceName("SmartExecutor");
 +
usageRecord.setCalledMethod("launch");
  
−
Every components has to account ''UsageRecord''s must depend from this library.
 
  
−
In some cases you may also to depended from [[Document Store  Lib | Document Store Lib]]. The maven coordinates of  Document Store  Lib are:
+
// usageRecord.setDuration(300)
 +
// Duration should be an integer. Anyway the string is converted from the validator.
 +
usageRecord.setResourceProperty(ServiceUsageRecord.DURATION, "300");
  
−
''<groupId>'''''org.gcube.data-publishing'''''</groupId>''
+
accountingPersistence.account(usageRecord);
  
−
''<artifactId>'''''document-store-lib'''''</artifactId>''
+
</syntaxhighlight>
  
  
−
Please set the scope of both library to PROVIDED if your component runs on a container gHN or vHN.
+
Please note that ServiceUsage record is accounted automatically from smartgears. Smartgears auto calculate '''calledMethod''' by using the relative path of the URL.
 +
This led to have records which cannot be aggregated from [[Accounting Aggregator]]. Please refer to the section [[Accounting_Aggregator#How_to_manage_calledMethod_cannot_be_aggregated | How to manage calledMethod lead to ServiceUsageRecord cannot be aggregated ]] to solve such an issue.
  
−
== Accounting Persistence Lib ==
+
== Development ==
  
−
The current implementation of persistence for this library is for CouchDB storage.
+
The library is identified by the following maven coordinates:
  
 
''<groupId>'''''org.gcube.accounting'''''</groupId>''
 
''<groupId>'''''org.gcube.accounting'''''</groupId>''
  
−
''<artifactId>'''''accounting-lib-persistence-couchdb'''''</artifactId>''
+
''<artifactId>'''''accounting-lib'''''</artifactId>''
  
−
The developer MUST not depend from this library. This library is already available on container because has been included in gCore and smartgears bundles.
+
Every components has to account ''UsageRecord''s must depend from this library.
−
Also, when a component does not run on a container , the component MUST not include this library as dependency. It will be responsibility of the person in charge of deploying the component to provide also that library.  
+
  
−
This library must be available on every node of the infrastructure. To respect as much possible the smartgears ZERO-DEPENDENCY paradigm a simple couchdb-connector (simple HTTP client library) has been developed.
 
  
−
Other implementations are:
+
In some cases (e.g. Exception management) you may also depended from [[Document Store  Library | Document Store Library]]. The maven coordinates of  Document Store  Lib are:
  
 +
''<groupId>'''''org.gcube.data-publishing'''''</groupId>''
  
−
''<groupId>'''''org.gcube.accounting'''''</groupId>''
+
''<artifactId>'''''document-store-lib'''''</artifactId>''
−
 
+
−
''<artifactId>'''''accounting-lib-persistence-couchbase'''''</artifactId>''
+
−
 
+
−
 
+
−
''<groupId>'''''org.gcube.accounting'''''</groupId>''
+
−
 
+
−
''<artifactId>'''''accounting-lib-persistence-mongodb'''''</artifactId>''
+
  
−
== Deployment ==
 
  
−
'''Accounting Lib''' and the '''Accounting Persistence Lib''' are already available on gCore and smartgears bundles (gHN and wHN).
+
Please set the scope of both library to '''PROVIDED''' if your component runs on a container gHN or vHN.

Latest revision as of 16:12, 6 December 2019

This component is responsible of collecting, harmonizing and storing Accounting data.

Overview

The following image evidences the accounting-lib component as part of Accounting Architecture:

Accounting-architecture-accounting-lib.png

Library

The library has been implemented to provide a common way to elaborate Usage Record (UR) (see Accounting Model to check which Usage Records are supported by default). In particular is responsible for:

  • Collecting Accounting Data

It uses facilities provided by Document Store Library for:

  • Aggregating
  • Buffering
  • Storing
  • Providing Fallback and Retry capability

Collecting Usage Record

The library allows to collect Usage Records providing the following functionality:

  • Attribute Validation and Harmonization
  • Usage Record Validation

To set the various properties of an usage records two different methods are available

  • Developer Friendly methods (getter and setter)
  • Generic Key-Value method (key:String, value:Comparable<? extends Serializable> )

The methods are idempotent. This means that any of the way you are using to set a properties the value is always validated and harmonized in the same way (i.e. from the the same code). You can mix different methods for different properties.

Moreover you can provide a Map<String,Comparable<? extends Serializable>> to set all properties at once.

Usage Record Validation and Harmonization

This library provides support to validate and harmonize the values to a common type (e.g. the string representation on an integer to a integer type, the string representation of an enum to the enum type). This functionality is very useful especially when the developer use a the key-value method.

For further details see Accounting Model.

Code Example

The following snippet of code provide an example of ServiceUsageRecord account which is automatically made from wHN and gHN. See Service Usage Record Model 


// This code auto-discover the available persistence and retrieve the configuration from IS to correctly initialize it.
AccountingPersistence accountingPersistence = AccountingPersistenceFactory.getPersistence();
 
// Create a new Service Usage Record
ServiceUsageRecord usageRecord = new ServiceUsageRecord();
 
/* 
 * Commented to show the use of key-value setResourceProperty function 
 * instead of the convenient setter
 * usageRecord.setConsumerId("luca.frosini");
 * Please note that as KEY is used the provided static variable. Always behave like this.
 */
usageRecord.setResourceProperty(ServiceUsageRecord.CONSUMER_ID, "luca.frosini");
 
usageRecord.setOperationResult(OperationResult.SUCCESS);
usageRecord.setCallerHost("node13.d4science.org");
usageRecord.setHost("node22.d4science.org");
usageRecord.setServiceClass("VREManagement");
usageRecord.setServiceName("SmartExecutor");
usageRecord.setCalledMethod("launch");
 
 
// usageRecord.setDuration(300)
// Duration should be an integer. Anyway the string is converted from the validator.
usageRecord.setResourceProperty(ServiceUsageRecord.DURATION, "300");
 
accountingPersistence.account(usageRecord);


Please note that ServiceUsage record is accounted automatically from smartgears. Smartgears auto calculate calledMethod by using the relative path of the URL. This led to have records which cannot be aggregated from Accounting Aggregator. Please refer to the section How to manage calledMethod lead to ServiceUsageRecord cannot be aggregated to solve such an issue.

Development

The library is identified by the following maven coordinates:

<groupId>org.gcube.accounting</groupId>

<artifactId>accounting-lib</artifactId>

Every components has to account UsageRecords must depend from this library.


In some cases (e.g. Exception management) you may also depended from Document Store Library. The maven coordinates of Document Store Lib are:

<groupId>org.gcube.data-publishing</groupId>

<artifactId>document-store-lib</artifactId>


Please set the scope of both library to PROVIDED if your component runs on a container gHN or vHN.

Retrieved from "https://gcube.wiki.gcube-system.org/index.php?title=Accounting_Library&oldid=33090"