Resource Accounting ABANDONED

Overview

The goal of the Resource Accounting is to allow gCube services to account their own custom information within the iMarine infrastructure

Key Features

Flexible and customizable data-model

In order to have a customizable accounting data-model, with the possibility to "easily" add new resources to take into account, the Resource Accounting sub-system relies on a flexible data-model to represent accounting information coming from the services, as described in common-accounting-model page.

RESTful interface

decouples the module from gCube infrastructure according to the zero-dependencies model.

High performance and scalability

REST communication and NoSQL DB for the backend.

Report-oriented portlet

Resource Accounting portlet complies with the Visual Information-Seeking Mantra, showing summarized views at a first glance, allowing the users to dive into more detailed data.

Design

Philosophy

Resource Accounting exploits the Messaging Infrastructure to deliver the message containing the accounting info to the proper instance of the Consumer for storage. In particular each time a new Resource Accounting record is generated a new message is enqueued into the local Producer which is in charge to dispatch it to the proper Message Broker instance. The Consumer(s) in charge to process this message, will then be notified and will store the information on the Resource Accounting DB.

Architecture

Resource Accounting sub-system is built around the Usage Tracker service whose goal is to keep track of resource usage by receiving and archiving accounting records. It provides create and query operations on the accounting records exposing RESTful API. The Usage Tracker adopts a document-oriented database (MongoDB in the current implementation) to persist accounting records. The adopted data model to represent accounting records is implemented by the common-accounting-model library.

The common-accounting-lib allows to produce/consume Resource Accounting records. At the service-side, each service could use the library to publish accounting information to the proper Message Broker instance, which is retrieved by querying the IS. At the Resource Accounting sub-system side, the Usage Tracker use the library to consume the accounting information published by the services. All the Resource Accounting records will be persisted using MongoDB.

The accounting information will be available both as reports (aggregated accounting records) and as raw data (single accounting record) through the Resource Accounting portlet, which is integrated with the Resource Accounting sub-system through the Usage Tracker RESTful API. The Resource Accounting portlet retrieves the proper Usage Tracker endpoint by querying the IS.

Deployment

Since we have a scope-based hierarchy in the infrastructure, there are two options for the deployment. First is to deploy the Resource Accounting sub-system at infrastructure level. Second is to deploy the Resource Accounting sub-system at VO level.

The following picture shows the deployment:

Use case

An example of a use case for the Resource Accounting is a service running jobs on the infrastructure (e.g. Workflow Engine) that integrates the common accounting library in order to produce the accounting information on the resources consumption. Those data will be published by the library on the Messaging Broker and will be consumed by the Usage Tracker, and then stored on the Resource Accounting backend (MongoDB). Finally, the infrastructure admins using the Resource Accounting portlet could navigate through the accounting information, which are provided by Resource Accounting sub-system.