Resource Broker

From Gcube Wiki
Revision as of 17:50, 10 April 2010 by Daniele.strollo (Talk | contribs) (→‎Requiring a deployment plan)

Jump to: navigation, search

The Resource Broker represents an ad hoc gCube service promoting and supporting the optimal selection and usage of resources during the VRE deployment phase. In particular, it is invoked to select the most appropriate pool of gHNs to be used, among those available in the context of the VRE, during the deployment of the services needed to operate the VRE. Because of this, it interacts with:

  • the Information System (IS) to be aware of the available gHNs as well as of their distinguishing features (e.g. the number of Running Instances currently hosted by it, the RAM the machine is equipped with) and
  • with the Virtual Organisation Management (VO-Management) to act securely and filter out the gHNs that falls out of the operational context of the VRE. From an architectural point of view it mainly consists of a service implementing the matchmaking algorithm.

Complete Protocol Overview

Several steps are involved during the deployment of a VRE. The green circles are labeled with numbers representing the natural succession of actions. The labeled boxes are the actors by means of GHNs and the services instantiated on them that take part to the protocol (the other services published on such GHNs are omitted for a better readability). Finally the white box containing several GHNs depicts a typical situation where (a group of) generic GHNs are dedicated to host the freshly deployed VRE instances.


Figure 1. The complete VRE deployment phase.


The Figure 1 reports a scenario of VRE instantiation. Here, we briefly recall the main steps that are performed (we assume to start from a request of VRE instantiation coming from the portal interface and that VREa is the name to assign to it):

  1. The portal requires the VRE Modeler to create a VRE (we call it VREa).
  2. Internally the VRE Modeler compiles a list of mandatory services plus optional services (e.g. GeoSpatial Search, support for data annotation). Actually these information are statically bound into the DB.
  3. The VRE Modeler accesses the VO Manager (a particular instance of VRE Manager defined at VO scope level) requesting to create an instance of the VRE Manager (Resource Manager of VREa) that will handle the requests for the resources for the scope of VREa.
    1. the VO Manager asks the Software Repository the list of packages needed to make the deploy (the ones required to instantiate the VRE Manager of VREa).
  4. The VO Manager asks the Deployer on the gHN of VRE Manager (Resource Manager of VREa) to create the instance of VRE Manager (at VREa Scope).
    1. The Deployer of VREa gets from the Software Repository the required packages needed for deployment. Installs packets. Activates service instances (in this case the only VRE manager).
    2. The Deployer of VREa creates and activates the VRE Manager of VREa (now the gHN of VREa is ready - VREa is instantiated).
  5. The VRE Modeler contacts the VREa Manager trough VRE Manager AddResources for allocating resources for VREa.
  6. VREa Manager asks the list of packages needed to all the services of VREa (just instantiated).
  7. VREa Manager distributes to the generic gHN the services to deploy and to run (services will be properly distributed among such nodes). A workflow similar to steps 4.1 and 4.2 will starts on the local deployers of generic gHN for getting the required gCube packages from the Software Repository.

Project Structure

This Section will introduce the main concepts and the development strategies adopted at Resource Broker design phase.

Service architecture and code design

The whole project consists of several parts (reflected in the structure of packages) here summarized:

  • configuration: various parameters characterizing the execution of BrokerService. Up to now they are grouped inside a single file. As future plan they are supposed to be partially moved in the jndi configuration file (so that they can be directly configured by end user). The access to all configurable properties will be demanded to an intermediate component responsible to retrieve and store them transparently to where they are located.
  • contexts: a constraint for building Stateful WS-Resources in the gCube framework is that each service declares its own StatefulBrokerContext. Aside, the ServiceContext represents the context of each service instance and keeps no persistence. Both the contexts are managed by a ResourceBinder that properly instantiates and maintain them.
  • resources: the SingletonResourceStorage is the singleton instance keeping the persistence of ResourceBroker status while the BrokerHome class is mandatory for storing internally the persistent resource and for retrieving it. For further details on how the state is mantained in gCube, refer to Building gCube Services Tutorial.
  • types: contains the data structures acting as support for the ResourceBroker. Here we find:
    • CustomXPathQueries: a set of customized queries to the IS. Manly introduced to retrieve from the IS a compact description of GHNs searched, with the only information needed on this side (e.g. ID of the GHN and the number (a counter) of Running Instances actually on it).
    • GHNDescriptor: the local description of a GHN, with the subset of information needed to apply decision planning.
    • GHNReservation: each request to the ResourceBroker is coupled with an unique identifier and a reservation status where the reserved GHNs are stored. Each reservation has an expiration date and keeps track of all GHNs (their descriptors) assigned during the workflow planning. Once a reservation fails, or the expiration time elapses or a feedback is sent back, the reservation is released (and, accordingly, all the reserved GHNs unlocked). Additionally, the GHNReservation contains the initial PlanRequest received during getPlan invocation so that in further stages (e.g. feedback handling) it can be retrieved and the failure can be associated to the specific node of the request that has failed.
    • SortableElement: generic container for sortable elements. Wraps generic elements inside it and is built up an index that is used to apply sorting algorithms. Sortable elements consist essentially of templates <T,V> where the first parameter (T) identifies the type for sortable indexes (e.g. int, long, ...) and the second parameter (V) constitutes the type of wrapped element.
    • Tuple: a general purpose definition of tuples. A tuple is a sequence (or ordered list) of finite length. Typical operations allowed on tuples are: union, append and equals (to compare them).
  • services: the components implementing the core functionalities of ResourceBroker. Here are defined: GHNReservationHandler acting as proxy in behalf of the other services and the persistent resource maintaining the state of reservations. ISClientRequester that intermediates with the IS making the queries in behalf of Broker Service and returns the results encapsulated inside proper data structures defined in types package. BrokerService provides the implementation of the externally published web service allowing to request planning decisions during the VRE deployment phase.
  • threads: the deployment plans are made in accordance to some parameters relative to the GHNs that are remotely retrieved from the IS. Once retrieved the set of GHNs active in a precise Scope, they are locally stored inside the persistent state of the service in order to avoid multiple requests to the IS for each request. For a such reason the locally stored information of GHNs must be refreshed at some intervals of time. TUpdateGHNProfiles is a timed thread that is responsible to update the profiles of GHNs defined in a given scope (or to remove from the local cache the ones no more available on the IS). TRevokeReservations is demanded to remove from the list of reserved GHNs the ones for which the time has expired. Both these threads are defined as subclasses of TimedThread, consisting of a scheduled thread that is frequently activated on a delay of time. It applies a passive wait during idle time. As support class here we find TimedThreadsStorage where these kinds of thread can be registered so that the exit of all of them can be directly required to this class.
  • notifications: over the scheduled threads here, as additional support, we have two particular services that are subscribed on the IS-Registry for the notifications sent by IS-Notifier regarding modifications to the GHNs relevant at BrokerService Scope. The notification subscribers play a complementary role with respect to threads so that once the registry is not available or down for a while threads supply its functionality and, conversely, if the notifications arrive before the thread refresh delay, they are as soon as possible applied.
  • assertions: support library for assertions.
  • serialization: here all the structures and components needed to (de-)serialize data managed by the BrokerService.
  • planbuilders: once a requester asks the BrokerService to compile a deploy plan a PlanBuilderWorkflow is instantiated. For a sake of modularity, a workflow is built up to a chain of PlanBuilderTask elements each of one is designed to accomplish a specific goal (e.g. take the suggested GHNs that come with the request, check consistence of request...). The response to send to the requester is the result of these steps or, in case of failure, the representation of the PlanBuilderException raised during internal computation (e.g. the required GHNs are not defined on the IS or are locked by other plans). All the possible error messages that can be thrown are grouped inside the PlanExceptionMessages enum.

Configuration

The configuration of ResourceBroker is stored in a property file called broker.properties located inside the etc configuration folder (e.g. $BUILD_LOCATION/etc/org.gcube.vremanagement.resourcebroker/broker.properties). A possible configuration is:

# Delay in seconds for retrying operations.
SLEEP_TIME=10

# The JNDI name used to publish the BrokerService
# Corresponds to the name set in the deploy-jndi.
JNDI_SERVICE_NAME = gcube/vremanagement/ResourceBroker

# Max tries for publishing the resource.
RESOURCE_PUBLICATION_MAX_ATTEMPTS = 20

# The max number of minutes from last profile update to consider a GHN alive.
LIVE_GHN_MAX_MINUTES = 40

# The namespace used for BrokerService.
NS_CONTEXT = http://gcube-system.org/namespaces/resourcebroker/ResourceBroker

# The namespace associated to the Registry service used to subscribe for notifications.
NS_REGISTRY = http://gcube-system.org/namespaces/informationsystem/registry

# The key to retrieve/publish the resource used in a singleton pattern.
SINGLETON_RESOURCE_KEY = BrokerService

# The max time a GHN reservation expires.
GHN_RESERVATION_TTL_MINUTES = 1

# The delay for next ghn update notification.
GHN_PROFILE_UPDATER_TTL_MINUTES = 2

# The default {@link GCUBEScope} the {@link BrokerService} operates.
# This scope is also used as default scope for publishing the service
# and its related resources.
CONTEXT_SCOPE = /gcube/devsec

# The package name as specified in build.properties.
PRJ_PACKAGE_NAME = org.gcube.vremanagement.resourcebroker

# To enable subscription to notifications for modifications to the GHNs.
ENABLE_GHN_NOTIFICATIONS = false
ENABLE_RI_NOTIFICATIONS = false

# TIMED THREADS - default true
ENABLE_REVOKE_RESERVATION_HANDLER = true
ENABLE_UPDATE_GHN_HANDLER = true

# The prefix to use in all classes of this project for logging.
LOGGING_PREFIX = BMM

# Each single node (corresponding to a PackageGroup block)
# can be considered accepted or failed according to the score
# associated in the Feedback.
# The FEEDBACK_NODE_TRESHOLD defines the minimum value
# allowed in percentage (0..100) for
# accepting such node.
FEEDBACK_NODE_TRESHOLD = 30

#
# The threshold for global score of a Feedback.
#
FEEDBACK_GLOBAL_TRESHOLD = 40

# How much the score influences the choice of competing GHNs.
GHN_ACCURACY_WEIGHT = 5

#
# how much a reservation influences the sorting of GHNs.
#
GHN_RESERVATION_WEIGHT = 0.4f


Implementation more in detail

GetPlan: Requiring a deployment plan

The entry point of ResourceBroker service is the getPlan operation. In Figure 2 is given a sketch of the internal logics (some aspects such as the interaction with the IS have been omitted for simplicity). Here is given an outline of the main implementation choices took in the getPlan.

  • The plan requester (VREManager) sends a PlanRequest (whose structure is given below in Interface) to the ResourceBroker.
  • The ResourceBroker retrieves the scope in which the deployment should be done (this information is contained inside the PlanRequest).
  • In a such scope are instantiated the threads and the notification handlers responsible to monitor changes to the GHNs defined at such level (at IS side). Notice that a single instance of such components is done per scope. So further requests of creations of threads and notifications handlers for an already monitored scope are ignored.
  • Now the plan request can be served. A new instance of plan workflow is created and associated to the current request.
Figure 2. getPlan.
  • The workflow is uniquely identified by its PlanBuilderIdentifier and receives as input a couple of (PlanRequest, PlanResponse) elements. Such a input in passed to all the tasks (PlanBuilderTask in Figure 2 represented as orange boxes (T1..T4)) constituting the workflow chain. Each task can

gHNs priority, reliability and reservation

Reservations

To support the deployment of decision planning tasks (PlanBuilderTask), the GHNReservationHandler class exposes high level capabilities such as the getNextGHN. This method automatically takes the GHN "better" for a workflow session (PlanBuilderIdentifier). This ensures that:

  • the returned GHN is reservable by requester workflow instance.
  • the access to the element is guaranteed to be mutually exclusive to avoid conflicts that can arise during get/reserve operations. It is enough to set the third parameter (reserve) to true.
  • if no GHN are available (e.g. all the available GHNs are already reserved by other instances of workflow) the failure is signaled to the caller.

GHNReservationHandler reserver = GHNReservationHandler.getInstance();
ghnToUse = reserver.getNextGHN(scope, this.identifier, true);

  • the property "GHN_RESERVATION_TTL_MINUTES" specifies the keep alive delay of a reservation. Reservations living from more that this slot are automatically revoked by low level support functions and the corresponding GHN made available for further requests.

Building and Managing the Internal Knowledge

Concurrent plan management

Interface

GetPlan: Requiring a Deployment plan

At requester side the access to the getPlan functionality can be done in two ways:

  1. by accessing the Java API (see PlanRequest API). In this case an utility is provided to transform from java representation of a PlanRequest to its XML Representation.
  2. by passing to it the corresponding XML representation of a plan request (see PlanRequest XML).

PlanRequest API

  1. import org.gcube.vremanagement.resourcebroker.utils.serialization.parser.xstream.XStreamTransformer;
  2. import org.gcube.vremanagement.resourcebroker.utils.serialization.types.Package;
  3. import org.gcube.vremanagement.resourcebroker.utils.serialization.types.PackageGroup;
  4. import org.gcube.vremanagement.resourcebroker.utils.serialization.types.PlanRequest;
  5. import org.gcube.vremanagement.resourcebroker.utils.serialization.types.PlanResponse;
  6.  
  7. .........
  8.  
  9. // REQUEST
  10. PlanRequest planReq = new PlanRequest("/gcube/devsec");
  11.  
  12. // Creates, inside the request, a group of packages to deploy.
  13. // This group is associated to the deployment of service1 (the input parameter).
  14. PackageGroup group = planReq.createPackageGroup("service1");
  15. // Defines a package group with a pre-defined ghn
  16. group.addPackage(new Package(true, "PkgServiceClass", "PkgServiceName", "PkgServiceVersion", "PkgPackageName", "PkgPackageVersion"));
  17. // Declares a pre-selected GHN to use for the deployment of current group of packages.
  18. group.setGHN("d5a5af20-ac50-11de-a928-ab32081f9f00");
  19.  
  20. PackageGroup group1 = planReq.createPackageGroup("service2");
  21. // Defines a package group without a pre-defined ghn
  22. group1.addPackage(new Package(true, "PkgServiceClass2", "PkgServiceName2", "PkgServiceVersion2", "PkgPackageName2", "PkgPackageVersion2"));
  23. group1.addPackage(new Package(true, "PkgServiceClass3", "PkgServiceName3", "PkgServiceVersion3", "PkgPackageName3", "PkgPackageVersion3"));
  24.  
  25. planReq.getGHNList().addGHN("1f251d60-215f-11df-b9c5-8ea2ab6d6650");
  26. planReq.getGHNList().addGHN("09ee4a70-1723-11df-adce-80cd45adc17d");
  27.  
  28. XStreamTransformer transformer = new XStreamTransformer();
  29. String xml = transformer.toXML(planReq);


Observations:

  • The instantiation of a PlanRequest, as shown in Line 10, requires a gCube scope name. Notice that all the deployment will predicate on a single instance of gCube scope.
  • PackageGroup elements (Line 14) are instantiated in a factory builder fashion. Their constructor is hidden and the instantiation demanded to the PlanRequest instance. Once instantiated these elements are automatically assigned to the proper PlanRequest owner (so that no explicit add operation is needed). Each PackageGroup is related to a service (the parameter passed at instantiation) so that all packages defined inside it will be considered the (sub-)set of packages constituting the software required to realize that service to be deployed.
  • Package elements are instead instantiated as usual with their constructor and added to their container as shown in Line 16.
  • To a PackageGroup' it is possible to assign a pre-selected GHN as in Line 18.
  • Analogously GHNs can be assigned to PackageGroups as shown in Lines 25-26. This set defines the suggested GHNs of a package group (it is used to declare a set of GHNs where the service must be deployed).
  • PlanRequest objects can be serialized in XML as given in Lines 28-29.

PlanRequest XML

The PlanRequest presented before is so serialized as follows, it is essentially made up to three parts:

  • An header declaring the scope where packages to be deployed will act (Line 3).
  • Some PackageGroup blocks (Lines 4-13 and Lines 14-29). It holds that:
    • they are numbered by a serial ID that is used to distinguish several nodes in order to handle the feedback procedure properly. Notice that using the PlanRequest API it is not required to assign an ID to the PackageGroup it will be automatically assigned at serialization phase.
    • entries of PackageGroup are blocks declaring a non empty lists of Package elements.
    • the structure of Package is alway a tuple of 5 elements.
    • the GHN part of a PackageGroup instead is optional. This tag represents the preselected GHN on which the set of packages must be deployed.
  • An optional GHNList block (Lines 30-33) containing a list of GHN identifiers that are called suggested GHNs. When present, the plan should be arranged around the suggested GHNs so that no other GHNs (over the preselected, declared inside PackageGroup, or suggested ones) can be used during deployment plan building.
  1. <?xml version="1.0" encoding="UTF-8"?>
  2. <PlanRequest xmlns="http://gcube-system.org/namespaces/resourcebroker/broker/xsd/deployRequest">
  3.   <Scope>/gcube/devsec</Scope>
  4.   <PackageGroup ID="1" service="service1">
  5.     <Package reuse="true">
  6.       <ServiceClass>PkgServiceClass</ServiceClass>
  7.       <ServiceName>PkgServiceName</ServiceName>
  8.       <ServiceVersion>PkgServiceVersion</ServiceVersion>
  9.       <PackageName>PkgPackageName</PackageName>
  10.       <PackageVersion>PkgPackageVersion</PackageVersion>
  11.     </Package>
  12.     <GHN>d5a5af20-ac50-11de-a928-ab32081f9f00</GHN>
  13.   </PackageGroup>
  14.   <PackageGroup ID="2" service="service2">
  15.     <Package reuse="true">
  16.       <ServiceClass>PkgServiceClass2</ServiceClass>
  17.       <ServiceName>PkgServiceName2</ServiceName>
  18.       <ServiceVersion>PkgServiceVersion2</ServiceVersion>
  19.       <PackageName>PkgPackageName2</PackageName>
  20.       <PackageVersion>PkgPackageVersion2</PackageVersion>
  21.     </Package>
  22.     <Package reuse="true">
  23.       <ServiceClass>PkgServiceClass3</ServiceClass>
  24.       <ServiceName>PkgServiceName3</ServiceName>
  25.       <ServiceVersion>PkgServiceVersion3</ServiceVersion>
  26.       <PackageName>PkgPackageName3</PackageName>
  27.       <PackageVersion>PkgPackageVersion3</PackageVersion>
  28.     </Package>
  29.   </PackageGroup>
  30.   <GHNList>
  31.     <GHN>1f251d60-215f-11df-b9c5-8ea2ab6d6650</GHN>
  32.     <GHN>09ee4a70-1723-11df-adce-80cd45adc17d</GHN>
  33.   </GHNList>
  34. </PlanRequest>
Suggested and Preselected GHNs

The intended meaning of GHNs entries inside a PlanRequest (suggested GHNs) or inside a PackageGroup (preselected GHN) is the following:

  • the PackageGroup ** must ** be deployed on its preselected GHN. If the preselected GHN cannot be used by the ResourceBroker (e.g. it is locked by another plan or not available) a failure happens.
  • if there are any suggested GHNs the complete set of GHNs on which the packages can be deployed is given by the union of the sets of suggested and preselected GHNs.

PlanResponse

Analogously for a PlanRequest, there are two ways for handling a PlanResponse:

PlanResponse API

Once invoked the getPlan returns a String consisting of the XML representation of PlanResponse. Its object representation can be obtained as follows:

import org.gcube.vremanagement.resourcebroker.utils.serialization.parser.xstream.XStreamTransformer;

...

XStreamTransformer transformer = new XStreamTransformer();
// Gets the response without validating against the response XSD - put true if validation needed.
PlanResponse resp = transformer.getResponsefromXML(xmlResponse, false);

PlanResponse XML

Here is an illustrative XML representation of a PlanResponse: 

  1. <?xml version="1.0" encoding="UTF-8"?>
  2. <PlanResponse xmlns="http://gcube-system.org/namespaces/resourcebroker/broker/xsd/deployResponse"
  3.       deployID="jvpubespfnqui04ii50p0ukgf9">
  4.   <Scope>/gcube/devsec</Scope>
  5.  
  6.   <PackageGroup>
  7.     <Package reuse="true" ID="1" service="service1">
  8.       <ServiceClass>PkgServiceClass</ServiceClass>
  9.       <ServiceName>PkgServiceName</ServiceName>
  10.       <ServiceVersion>PkgServiceVersion</ServiceVersion>
  11.       <PackageName>PkgPackageName</PackageName>
  12.       <PackageVersion>PkgPackageVersion</PackageVersion>
  13.     </Package>
  14.     <GHN>d5a5af20-ac50-11de-a928-ab32081f9f00</GHN>
  15.   </PackageGroup>
  16.  
  17.   <PackageGroup ID="2" service="service2">
  18.     <Package reuse="true">
  19.       <ServiceClass>PkgServiceClass2</ServiceClass>
  20.       <ServiceName>PkgServiceName2</ServiceName>
  21.       <ServiceVersion>PkgServiceVersion2</ServiceVersion>
  22.       <PackageName>PkgPackageName2</PackageName>
  23.       <PackageVersion>PkgPackageVersion2</PackageVersion>
  24.     </Package>
  25.     <Package reuse="true">
  26.       <ServiceClass>PkgServiceClass3</ServiceClass>
  27.       <ServiceName>PkgServiceName3</ServiceName>
  28.       <ServiceVersion>PkgServiceVersion3</ServiceVersion>
  29.       <PackageName>PkgPackageName3</PackageName>
  30.       <PackageVersion>PkgPackageVersion3</PackageVersion>
  31.     </Package>
  32.     <GHN>1f251d60-215f-11df-b9c5-8ea2ab6d6650</GHN>
  33.   </PackageGroup>
  34.  
  35.   <Status value="SUCCESS"/>
  36. </PlanResponse>

The main elements of the response are:

  • deployID [mandatory]: is the unique plan identifier (built at creation of a PlanBuilderWorkflow). Feedback or revocations of plans will need this parameter.
  • Scope: the value will be the same of the original PlanRequest scope provided at getPlan invocation.
  • A list of PackageGroup elements: the original PackageGroup elements present in the request are now assigned to a (single and mandatory) GHN. Note: the first group has been assigned to the preselected GHN declared in the request while the second one has been assigned to one of the suggested GHNs defined in the request.
  • Status: the status of deployment planning execution. In case of failure it will contain a description message as follows:

<Status value="FAILED">
 <ErrorMsg>The required GHN is already locked by another plan.
  The ghn: d5a5af20-ac50-11de-a928-ab32081f9f00 cannot be
  reserved by hihd0qtf5bsk2vmdqjahalm61i it is already locked
  by plan: jvpubespfnqui04ii50p0ukgf9
 </ErrorMsg>
</Status>

Sending feedback

A sample of feedback that can be sent to the ResourceBroker is:


  1. <?xml version="1.0" encoding="UTF-8"?>
  2. <Feedback planID="r1u82qb9d9e0c5k2ue43cbq26f" xmlns="http://gcube-system.org/namespaces/resourcebroker/broker/xsd/deployFeedback">
  3.   <Scope>/gcube/devsec</Scope>
  4.   <DeployNode>
  5.     <PackageGroup ID="1" service="service1">
  6.       <Package reuse="true" status="FAILED">
  7.         <ServiceClass>PkgServiceClass</ServiceClass>
  8.         <ServiceName>PkgServiceName</ServiceName>
  9.         <ServiceVersion>PkgServiceVersion</ServiceVersion>
  10.         <PackageName>PkgPackageName</PackageName>
  11.         <PackageVersion>PkgPackageVersion</PackageVersion>
  12.       </Package>
  13.       <Package reuse="true" status="FAILED">
  14.         <ServiceClass>PkgServiceClass2</ServiceClass>
  15.         <ServiceName>PkgServiceName2</ServiceName>
  16.         <ServiceVersion>PkgServiceVersion2</ServiceVersion>
  17.         <PackageName>PkgPackageName2</PackageName>
  18.         <PackageVersion>PkgPackageVersion2</PackageVersion>
  19.       </Package>
  20.       <GHN>10609b30-1d75-11df-a8a9-e63121c030b8</GHN>
  21.     </PackageGroup>
  22.   </DeployNode>
  23.   <DeployNode>
  24.     <PackageGroup ID="2" service="service2">
  25.       <Package reuse="true" status="PARTIAL">
  26.         <ServiceClass>PkgServiceClass3</ServiceClass>
  27.         <ServiceName>PkgServiceName3</ServiceName>
  28.         <ServiceVersion>PkgServiceVersion3</ServiceVersion>
  29.         <PackageName>PkgPackageName3</PackageName>
  30.         <PackageVersion>PkgPackageVersion3</PackageVersion>
  31.       </Package>
  32.       <GHN>e81f6b20-3934-11df-b2aa-a600d1910064</GHN>
  33.     </PackageGroup>
  34.   </DeployNode>
  35.   <DeployNode>
  36.     <PackageGroup ID="3" service="service1">
  37.       <Package reuse="true" status="SUCCESS">
  38.         <ServiceClass>PkgServiceClass3</ServiceClass>
  39.         <ServiceName>PkgServiceName3</ServiceName>
  40.         <ServiceVersion>PkgServiceVersion3</ServiceVersion>
  41.         <PackageName>PkgPackageName3</PackageName>
  42.         <PackageVersion>PkgPackageVersion3</PackageVersion>
  43.       </Package>
  44.       <GHN>6cd37690-2de6-11df-8cff-c060e194e270</GHN>
  45.     </PackageGroup>
  46.   </DeployNode>
  47.   <DeployNode>
  48.     <PackageGroup ID="4" service="service2">
  49.       <Package reuse="true" status="PARTIAL">
  50.         <ServiceClass>PkgServiceClass3</ServiceClass>
  51.         <ServiceName>PkgServiceName3</ServiceName>
  52.         <ServiceVersion>PkgServiceVersion3</ServiceVersion>
  53.         <PackageName>PkgPackageName3</PackageName>
  54.         <PackageVersion>PkgPackageVersion3</PackageVersion>
  55.       </Package>
  56.       <GHN>10609b30-1d75-11df-a8a9-e63121c030b8</GHN>
  57.     </PackageGroup>
  58.   </DeployNode>
  59.   <DeployNode>
  60.     <PackageGroup ID="5" service="service3">
  61.       <Package reuse="true" status="FAILED">
  62.         <ServiceClass>PkgServiceClass3</ServiceClass>
  63.         <ServiceName>PkgServiceName3</ServiceName>
  64.         <ServiceVersion>PkgServiceVersion3</ServiceVersion>
  65.         <PackageName>PkgPackageName3</PackageName>
  66.         <PackageVersion>PkgPackageVersion3</PackageVersion>
  67.       </Package>
  68.       <GHN>0807af10-17f2-11df-9a98-f67cee73373b</GHN>
  69.     </PackageGroup>
  70.   </DeployNode>
  71.   <DeployNode>
  72.     <PackageGroup ID="6" service="service1">
  73.       <Package reuse="true" status="PARTIAL">
  74.         <ServiceClass>PkgServiceClass3</ServiceClass>
  75.         <ServiceName>PkgServiceName3</ServiceName>
  76.         <ServiceVersion>PkgServiceVersion3</ServiceVersion>
  77.         <PackageName>PkgPackageName3</PackageName>
  78.         <PackageVersion>PkgPackageVersion3</PackageVersion>
  79.       </Package>
  80.       <GHN>cff736a0-1728-11df-ab5d-81441e51d390</GHN>
  81.     </PackageGroup>
  82.   </DeployNode>
  83.   <DeployNode>
  84.     <PackageGroup ID="7" service="service1">
  85.       <Package reuse="true" status="FAILED">
  86.         <ServiceClass>PkgServiceClass3</ServiceClass>
  87.         <ServiceName>PkgServiceName3</ServiceName>
  88.         <ServiceVersion>PkgServiceVersion3</ServiceVersion>
  89.         <PackageName>PkgPackageName3</PackageName>
  90.         <PackageVersion>PkgPackageVersion3</PackageVersion>
  91.       </Package>
  92.       <GHN>b0c37430-de9b-11de-bd7f-8a98e4e5bcf0</GHN>
  93.     </PackageGroup>
  94.   </DeployNode>
  95. </Feedback>

Test-suite

Local test-suite

The package org.gcube.vremanagement.resourcebroker.local.testsuite contains several classes defined for internal use and for testing functionalities of internally defined data structures and services. In particular the class StandaloneISQueryTestConf is used to test custom XPath queries to the IS service. Typical scenario:

java org.gcube.vremanagement.resourcebroker.local.testsuite.StandaloneISQueryTest -s /gcube/devsec File.query

where the parameter -s specifies the gCube scope in which the query should be executed and File.query is the input file containing the queries to submit to the IS.

A sample of File.query is:

# A query is a multi-line text block separeted by empty line
# And these are comments.
#
let $now := current-dateTime()
for $GHNs in collection("/db/Profiles/GHN")//Document/Data/child::*[local-name()='Profile']/Resource
let $RIs := collection("/db/Profiles/RunningInstance")//Document/Data/child::*[local-name()='Profile']/Resource[Profile/GHN/string(@UniqueID)=$GHNs/ID]
let $lastTestTimeDuration := xs:dateTime($now) - xs:dateTime($GHNs/Profile/GHNDescription/LastUpdate)
let $totalminutes := ceiling($lastTestTimeDuration div xdt:dayTimeDuration('PT1M'))
order by count($RIs)                                     
return
concat('<RIONGHN>','<GHNID>',$GHNs/ID, '</GHNID>','<AllocatedRI>', count($RIs),'</AllocatedRI>', '<LastUpdate>', $GHNs/Profile/GHNDescription/LastUpdate,'</LastUpdate>','<UpdateMinutesElapsed>',$totalminutes,'</UpdateMinutesElapsed>','</RIONGHN>')

whose result is a set of elements of the form:

<RIONGHN>
 <GHNID>59fed2c0-2dbf-11df-9784-a6c7b139143c</GHNID>
 <AllocatedRI>15</AllocatedRI>
 <LastUpdate>2010-04-06T15:56:27+01:00</LastUpdate>
 <UpdateMinutesElapsed>2</UpdateMinutesElapsed>
</RIONGHN>

Implementation details

Adding a PlanBuilderTask

  1. import org.gcube.common.core.faults.GCUBEFault;
  2. import org.gcube.vremanagement.resourcebroker.impl.planbuilders.PlanBuilderElem;
  3. import org.gcube.vremanagement.resourcebroker.impl.planbuilders.PlanBuilderException;
  4. import org.gcube.vremanagement.resourcebroker.impl.planbuilders.PlanBuilderTask;
  5. import org.gcube.vremanagement.resourcebroker.utils.serialization.parser.xstream.XStreamTransformer;
  6.  
  7. public class PrintRequestTask extends PlanBuilderTask {
  8.  @Override
  9.  public PlanBuilderElem makeDecision(PlanBuilderElem input)
  10.   throws PlanBuilderException {
  11.   XStreamTransformer transformer = new XStreamTransformer();
  12.   try {
  13.    logger.debug(transformer.toXML(input.getRequest()));
  14.   } catch (GCUBEFault e) {
  15.    e.printStackTrace();
  16.   }
  17.   return input;
  18.  }
  19. }

The code here reported consists of a dummy PlanBuilderTask configuration that simply prints (Line 13) the XML representation of input request (PlanRequest) received by the getPlan method. A task must extends the abstract class PlanBuilderTask (Line 7) and override its abstract method makeDecision. The input must be returned back as given in Line 17.

Additional features

Here is given an outline of main features that can be accessed by tasks for their decision making:

  • obtaining and reserving a new available GHN (that better fits load balance requirements):
    • access the singleton instance of reservation handler that ensures the mutual exclusive access to the reservations.
    • use the reserver to retrieve the next GHN available for current request (GHNs are ordered on the number of allocated resource instances). The parameters scope and identifier guarantee that the GHN is available for the required scope and that are reservable for the workflow identifier (so that it is not reserved or already reserved in such workflow) and the last parameter "true" required the reserver to make an atomic reservation for the retrieved GHN.

GHNReservationHandler reserver = GHNReservationHandler.getInstance();
ghnToUse = reserver.getNextGHN(scope, this.identifier, true);

  • signaling a failure in the response:

// If from the IS no GHN have been retrieved
// nothing to do!!!
if (ghnToUse == null) {
        throw new PlanBuilderException(PlanExceptionMessages.NO_GHNS_AVAILABLE, pkgGroupToUse);
}

Failure Handling

Retrieved from "https://gcube.wiki.gcube-system.org/index.php?title=Resource_Broker&oldid=9512"