Resource Management Specification

Overview

Resource Management focuses on the optimal exploitation of software and hardware resources. Software resources include a range of entities from packages ready to be deployed to already activated services, while hardware resources model platforms hosting a service suitable to be exploited by the e-infrastructure.

Resource Management covers the entire lifecycle of such resources through a set of services and libraries. This document outlines their design rationale, key features, and high-level architecture as well as the opportunities they collectively offer.

Key features

Dynamic Deployment

First-class features for deploying gCube software and external software packaged following the gCube policies, support for deployment of software hosted on recognized Maven repositories

Optimal Resource Allocation

Optimal selection and usage of resources during the deployment phase

Monitoring

Tools for pro-actively monitoring the current state of nodes and active services. Local information are collected and published in the Information System as well as returned on demand

Easy Remote Administration

Remotely reconfiguration of nodes and active services upon the e-infrastructure/manager's needs

Failover capabilities

Self-recovering from erroneous states

Extensible Bridging over Virtual Platforms

A flexible model for transparently interfacing a potentially unlimited number of hosting environments

Maven Integration

Any software available in a Maven repository is eligible to become part of the Data e-Infrastructure

Coordination and Elastic Management

New services can be fired up on demand and through self-government

Cloud-orientation

Gateway to Cloud platforms with Java support, looking forward for the proposed Topology and Orchestration Specification for Cloud Applications (or TOSCA) for short, standards specification

Scalability

e-Infrastructures composed by thousands of nodes and software can be concurrently managed

Design

Philosophy

The Data e-Infrastructure requires dynamic allocation of services and nodes and a strong capability to adapt itself to changes occurring during its operation. Components of Resource Management subsystem address these requirements by providing a robust ground for the overall infrastructure lifetime.

Software has different externalizations at different level, ranging from a tarball of binaries to an activated instance hosted on a platform. Resource management has to be able to deal with such different forms and the designed architecture reflect this role.

Different components have different perceptions of the resources to manage. There are components operating at a logical level (VO and VRE) that only matter about how to optimize the VO/VRE delivered functionality by replicating/duplicating services when they see this need. These components do not obviously care of the physical form of the software and will never change if it will. Other components work at service level (the constituents of the VO/VRE) and know how to optimize their allocation. Finally, there exist components that operates at local level (node) and manage only local deployed software.

Other principles on which the design of the presented architecture is piled are:

• components must be generic with respect to the resource managed
• fault recovery is a crucial feature: nodes and services must be guaranteed to have in a consistent state under any condition
• external software has to be as transparently as possible managed in the same way of the gCube software

Architecture

Resource Management Architecture

• VRE-Creation-Wizard: this graphical wizard guides a VRE-manager in the creation of a VRE in a step-by-step fashion
• VRE-Modeler: this service receives the high level requirements and criteria produced by the wizards and translates them in a concrete set of services to deploy or join
• Resource-Management-Portlet: integrated in the portal and accessible to authorized users, this portlet exposes a graphical visualization of the current state of the resources belonging the e-infrastructure and provides instruments for corrective and management actions (such as restarting a node, adding/removing node and service to/from a VO/VRE, etc.)
• Resource-Manager: this service stays on top of many other services and coordinates them to satisfy the VRE-Modeler requests.More specifically, the service manages resources belonging the Scope and creates new instances of services within it. It interacts with the Software gateway (for dependencies resolution), the Resource Broker (for allocating services on gHNs), the Deployer (for deploying services), the gHN Manager (for managing node's scopes) and the Information System (for publishing and retrieving resource's profiles
• Resource-Broker: this service promotes and supports the optimal selection and usage of resources during the deployment phase. In particular, it is invoked to select the most appropriate pool of gHNs to be used, among those available in the context of the deployment operation
• Resource-Broker-Serialization: a collection of structures and definitions needed to (de-)serialize data managed by the Resource-Broker
• Software-Gateway: a gateway over a cluster of Maven Repositories granting access to the stored information and software for deployment purposes
• gHN-Manager: a service providing operations for the remote management of a node
• Deployer: a service providing facilities for deploying, managing and un-deploying software components on a node
• Virtual-Platform: a library with classes and interfaces to be extended/implemented to bridge and manage the lifecycle of software running on a different platform than gCore
• Virtual-Platform for Tomcat6: an implementation of the Virtual-Platform model to manage Web Applications on Apache Tomcat 6.x
• RainyCloud

Deployment

Resource Management components have almost a fixed deployment scheme (with a few variations) because of their nature and responsibilities.

Large deployment

VRE creations have their natural place inside a VO. Therefore, VRE-Wizard, VRE-Modeler and Resource Broker, which play their respective role at VRE creation, are mandatory to be deployed in each VO aiming at supporting dynamic VREs.

Resource-Management-Portlet is also suggested to be deployed at VO level. VO views are more restricted that the whole infrastructure and can be better administrated by VO managers.

Each organized subset of resources (VO or VRE) requires an instance (and only one) of the Resource Manager that is charge of moving resource inside and outside such a subset.

Finally, the class of components (gHN-Manager, Deployer, Virtual Platforms) working at Package/Node level, are optimally deployed in each gHN supporting their respective platform (gCore, Tomcat, Venus-C or future bridged platforms). This way, the functionality they expose are available on every and each node by maximizing the possible targets of deployments, replica creation and balancing decided by managers or by the self-government of the system itself.

Small deployment

There exist few variations of the optimal scheme presented above. They can be adopted in some specific situations where a set of management functions is not needed or supported.

Use Cases

The subsystem has been conceived to support a number of use cases moreover it will be used to serve a number of scenarios. This area will collect these "success stories".