Difference between revisions of "Resource Management for the gCube Model"

From Gcube Wiki
Jump to: navigation, search
m (Manuele.simi moved page Information System Resource Manager to Resource Management for the gCube Model: Does not belong to the IS)
(→‎Contexts and Resources)
 
(91 intermediate revisions by 2 users not shown)
Line 1: Line 1:
−
This component is under development.
+
[[Category: Developer's Guide]]
 +
[[Category: Resource Management]]
 +
{| align="right"
 +
||__TOC__
 +
|}
  
−
Resource Manager is part of [[Information System]].
+
'''This component is under development.'''
  
−
Resource Manager is responsible for supporting VRE creation and management.
+
'''Here some [[gcube/Resource_Management_for_the_gCube_Model/discussions | preliminary discussions]] that guided the design.'''
 +
 
 +
== Role and Responsibilities ==
 +
 
 +
Resource Management refers to a set of software components revolving around the Resource Manager service. They are collectively responsible for delivering the following operations:
 +
* creating [[Facet_Based_Resource_Model#Context | ''contexts'']]
 +
* moving, editing, cloning, deleting contexts
 +
* creating a dump of users, resources and information (everything except data) to allow rebuilding a deleted context
 +
* managing resources within a context according to the [[Facet_Based_Resource_Model#gCube_Model|'''gCube Resource Model''']].
 +
 
 +
=== Contexts and Resources ===
 +
Some remarks on contexts and resources that affect the Resource Management:
 +
* contexts have an arbitrary hierarchy
 +
* a context can be empty
 +
* a context can have no parent
 +
* a context cannot have two children with the same name
 +
* a resource in a context can be private or shared with other contexts
 +
* resources can join a context from any source context (not only from the parent), if permitted.
 +
 
 +
RM should manage generic rules on the source/target of resources. For example:
 +
* resources from context C1 can join only context C3
 +
* resources from context C1 cannot join context C2
 +
* resources in context C1 are private
 +
* resources in context C1 can be shared with any other context
 +
...and so on.
 +
 
 +
 
 +
Concerning the resources, the Resource Manager operates at [[Facet_Based_Resource_Model#Virtual_Service | Virtual Service]] level. A Virtual Service is a bag of Services that delivers a logical group of functionalities. A request to RM specifies which Virtual Service to add/remove/edit. The RM translates this request to the list of concrete services (by querying the Resource Registry) and then instructs the Resource Registry accordingly.
 +
 
 +
=== Encryption keys for contexts ===
 +
To evaluate if Resource Management is also responsible for creating/distributing RSA keys to encrypt/decrypt sensitive information in the Facets.
 +
 
 +
== Design requirements ==
 +
 
 +
=== On the interaction with other components ===
 +
The Resource Management software is expected to deliver its operations by collaborating with the following components:
 +
 
 +
* [[Information_System_Resource_Registry | Resource Registry]] and its client libraries to add/remove/edit/merge contexts and their resources. The scope of this interaction is to make the resources discoverable in a new context (or un-discoverable if removed).
 +
* [[Authorization_Framework | Authorization Framework]] to create new context tokens
 +
* WHN Manager: to add/remove HostingNodes/EServices from a context. Once a new context is available, the HostingNodes hosting the EServices belonging to the context need to be added to it. The RM has to contact the WHN Manager (docs?) to notify the changes in the context for that node. Requesting to WHN Manager to add an Hosting Nodes or EServices to a context has the following effect:
 +
** Smartgears requests to AuthorizationFramework a new token for the new Context for the HostingNode;
 +
** Smartgears requests to AuthorizationFramework a new token for the new Context for the each EService hosted in the HostingNode.
 +
** Smartgears invokes addToContext of HostingNode to the Resource Registry. Thanks to the propagation constraints also the hosted EServices are added to the new Context
 +
** Smartgears starts to accept requests for the new Context for all hosted EServices.
 +
 
 +
It is expected that a portlet is built atop the Resource Management (and integrates with its client library) to request the operations.
 +
 
 +
=== On the context it operates ===
 +
 
 +
Considering that each token is bound to a single context, while the Resource Management needs to work across contexts, the interactions with the Resource Registry have to be trusted because of the caller identity. I.e. the RR allows the Resource Management to do whatever it wants just because of its identity and regardless the context attached to the call. We would need to set up new policies for this authZ.
 +
 
 +
The service token with the identity of the Resource Manager service is obtained by the local SmartGears lib when the webapp with the service is activated by the container.
 +
 
 +
=== On the deployment ===
 +
* The service has to be stateless. Multiple instances will be deployed behind a HAProxy.
 +
* The service has to be able to manage more than one context (a.k.a. scope). In principle, we will deploy just one set of instances of the service (behind the proxy) to manage the infrastructure context, the VOs, and VREs. However, it should also be possible to deploy one instance for a specific VO.
 +
* A new instance of the service can be added at any time to the set of instances already deployed without manual configuration of the new instance or changes to the configuration of the pre-existing ones.
 +
 
 +
== Architecture ==
 +
 
 +
The Resource Management is divided in the following software components:
 +
 
 +
;Resource Manager service
 +
:A webapp hosted on a wHN that provides the main logic accessible through a RESTful interface
 +
;Resource Manager client
 +
:an abstraction over the JSON-based details of the interaction with the service
 +
;Resource I/O library
 +
:common definitions and utilities shared between the service and the client
 +
 
 +
== Service: Context resource ==
 +
When reporting error conditions, these resource methods make use of [[GxRest/Responses#Code_Exceptions | Code Exceptions]].
 +
 
 +
=== Create a Context ===
 +
<pre>POST /resource-manager/context?rrURL=http://host:port/resource-registry</pre>
 +
 
 +
==== Parameters ====
 +
 
 +
{|class="wikitable"
 +
! Name
 +
! Type
 +
! Required
 +
! Description
 +
|-
 +
| '''body of the post request'''
 +
| String
 +
| true
 +
| The JSON serialization of the context to create.
 +
|-
 +
| '''rrUrl'''
 +
| String
 +
| false
 +
| The URL of the Resource Registry instance to contact.
 +
|}
 +
 
 +
Sample body for the request:
 +
<pre>
 +
{
 +
  "@class": "Context",
 +
  "header": {
 +
    "@class": "Header",
 +
    "uuid": "ba3003da-a394-4d71-872f-63d96e37a2b6",
 +
    "creator": "manuele simi",
 +
    "modifiedBy": null,
 +
    "creationTime": null,
 +
    "lastUpdateTime": null
 +
  },
 +
  "name": "firstContext",
 +
  "parent": null,
 +
  "children": []
 +
}
 +
</pre>
 +
 
 +
==== Returned Values ====
 +
In case of success: HTTP code [http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.2.3 201] (CREATED)
 +
 
 +
In case of failure: HTTP code [http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.7 406] (NOT_ACCEPTABLE)
 +
 
 +
The reason of the failure is specified as [[GxRest/Responses#With_Error_Codes | Error Code]] the response's entity:
 +
 
 +
*INVALID_METHOD_REQUEST(0, "The request is invalid."),
 +
*MISSING_PARAMETER(1,"Required query parameter is missing."),
 +
*MISSING_HEADER(2, "Required header is missing."),
 +
*CONTEXT_ALREADY_EXISTS(3, "Context already exists at the same level of the hierarchy."),
 +
*CONTEXT_PARENT_DOES_NOT_EXIST(4, "Failed to validate the request. The request was not submitted to the Resource Registry."),
 +
*INVALID_REQUEST_FOR_RR(5, "Failed to validate the request. The request was not submitted to the Resource Registry."),
 +
*GENERIC_ERROR_FROM_RR(6, "The Resource Registry returned an error.");
 +
 
 +
==== Description ====
 +
 
 +
Steps for creating a new context:
 +
* Validation: the new context must be properly framed in the pre-existing context hierarchy. For example:
 +
** its parent context must exist
 +
** there must be no other context with the same name at the same level in the hierarchy.
 +
* Invoke the [[Interacting_with_Resource_Registry_Service_-_Context_and_Schema_Port_Type#Create | context]] method of Resource Registry to create the context.
 +
* Contact the Authorization Service by mean of its client lib to create a new context token.
 +
* Add the token to local node, i.e. the node on which the RM is running automatically joins the new context.
 +
 
 +
=== Delete a Context ===
 +
 
 +
<pre>DELETE /resource-manager/context/{{UUID}}?rrURL=http://host:port/resource-registry</pre>
 +
 
 +
==== Parameters ====
 +
 
 +
{|class="wikitable"
 +
! Name
 +
! Type
 +
! Required
 +
! Description
 +
|-
 +
| '''id'''
 +
| String
 +
| true
 +
| The UUID of the context to delete.
 +
|-
 +
| '''force'''
 +
| Boolean
 +
| false
 +
| Delete a context and all its descendants. When false or not set, a context is deleted only if it does not have descendants. Default is set to false.
 +
|-
 +
| '''dryRun'''
 +
| Boolean
 +
| false
 +
| Does not delete the context but returns all the contexts that would be deleted by the operation. Default is set to false.
 +
|-
 +
| '''rrUrl'''
 +
| String
 +
| false
 +
| The URL of the Resource Registry instance to contact.
 +
|}
 +
 
 +
==== Returned Status ====
 +
 
 +
In case of success: HTTP code [https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.2.1 200] (OK)
 +
 
 +
In case of failure:
 +
* HTTP code [http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.5 304] (NOT_MODIFIED), if the operation fails
 +
* HTTP code [http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.5 404] (NOT_FOUND), if the context in the request does not exist
 +
 
 +
==== Description ====
 +
The operation is expected to:
 +
* check if all the conditions to delete the context apply
 +
* delete the context
 +
 
 +
=== Rename a Context ===
 +
<pre>PUT /resource-manager/context/{{UUID}}?newName=value&&rrURL=value</pre>
 +
 
 +
==== Parameters ====
 +
 
 +
{|class="wikitable"
 +
! Name
 +
! Type
 +
! Required
 +
! Description
 +
|-
 +
| '''id'''
 +
| String
 +
| true
 +
| The UUID of the context to delete.
 +
|-
 +
| '''newName'''
 +
| String
 +
| true
 +
| The new name to assign to the context.
 +
|-
 +
| '''rrUrl'''
 +
| String
 +
| false
 +
| The URL of the Resource Registry instance to contact.
 +
|}
 +
 
 +
==== Returned Status ====
 +
 
 +
In case of success: HTTP code [https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.2.1 200] (OK)
 +
 
 +
In case of failure:
 +
* HTTP code [http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.5 304] (NOT_MODIFIED), if the operation fails
 +
* HTTP code [http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.5 404] (NOT_FOUND), if the context in the request does not exist
 +
 
 +
==== Description ====
 +
The operation renames an existing context.
 +
 
 +
=== Edit a Context ===
 +
 
 +
=== Move a Context ===
 +
 
 +
== Service: Virtual Service resource ==
 +
 
 +
=== Add to Context ===
 +
 
 +
<pre>PUT resource-manager/resource/{{UUID}}</pre>
 +
 
 +
==== Parameters ====
 +
 
 +
{|class="wikitable"
 +
! Name
 +
! Type
 +
! Required
 +
! Description
 +
|-
 +
| '''UUID'''
 +
| String
 +
| true
 +
| The UUID of the [[Facet_Based_Resource_Model#Virtual_Service | Virtual Service]] and/or [[Facet_Based_Resource_Model#Software | Software]] to add.
 +
|-
 +
| '''contextID'''
 +
| String
 +
| true
 +
| The UUID of the context.
 +
|}
 +
 
 +
==== Description ====
 +
 
 +
Steps to perform:
 +
* Validation:
 +
** the input context must exist
 +
** Virtual Services and Services must not be already part of the context (unless replication is allowed).
 +
* Add the Virtual Services to the context. To do this, RM contacts the RR to:
 +
** [[Interacting_with_Resource_Registry_Service_-_Query_%26_Access | query ]] and navigate the parent context and select the pre-existing (already deployed) Virtual Services to add to the context (is that correct?)
 +
** add the Virtual Services to the context. With a cascade policy on the [[Facet_Based_Resource_Model#callsFor | callFor]] relation, all the Services related to each Virtual Service are automatically added to the context by the RR
 +
** [[Interacting_with_Resource_Registry_Service_-_Query_%26_Access | query ]] and navigate the [[Facet_Based_Resource_Model#demands | demands]] relations and discover the Software required by each Virtual Service (typically not running on [[SmartGears|SmartGears]], e.g. a DBMS). Then add this Software to the context.
 +
* Add the Software received as parameter to the context. To do this, RM contacts the RR to add each Software to the new context. Because this Software does not run on SmartGears, the only operation to do is to make it visible in a context on the [[Information_System|IS]].
 +
 
 +
== Related documentation ==
 +
 
 +
* [[SmartGears_Web_Hosting_Node_(wHN)_Installation| Web Hosting Node (wHN)]]
 +
* [[SmartGears | SmartGears]]
 +
* [[Facet_Based_Resource_Model#gCube_Model| gCube Resource Model]]
 +
* [[Information_System_Resource_Registry | Resource Registry]]
 +
* [[Interacting_with_Resource_Registry_Service_-_Context_and_Schema_Port_Type | Interacting with the Resource Registry]]
 +
* [[Authorization_Framework | Authorization Framework]]

Latest revision as of 02:32, 26 June 2018

This component is under development.

Here some preliminary discussions that guided the design.

Role and Responsibilities

Resource Management refers to a set of software components revolving around the Resource Manager service. They are collectively responsible for delivering the following operations:

  • creating contexts
  • moving, editing, cloning, deleting contexts
  • creating a dump of users, resources and information (everything except data) to allow rebuilding a deleted context
  • managing resources within a context according to the gCube Resource Model.

Contexts and Resources

Some remarks on contexts and resources that affect the Resource Management:

  • contexts have an arbitrary hierarchy
  • a context can be empty
  • a context can have no parent
  • a context cannot have two children with the same name
  • a resource in a context can be private or shared with other contexts
  • resources can join a context from any source context (not only from the parent), if permitted.

RM should manage generic rules on the source/target of resources. For example:

  • resources from context C1 can join only context C3
  • resources from context C1 cannot join context C2
  • resources in context C1 are private
  • resources in context C1 can be shared with any other context

...and so on.


Concerning the resources, the Resource Manager operates at Virtual Service level. A Virtual Service is a bag of Services that delivers a logical group of functionalities. A request to RM specifies which Virtual Service to add/remove/edit. The RM translates this request to the list of concrete services (by querying the Resource Registry) and then instructs the Resource Registry accordingly.

Encryption keys for contexts

To evaluate if Resource Management is also responsible for creating/distributing RSA keys to encrypt/decrypt sensitive information in the Facets.

Design requirements

On the interaction with other components

The Resource Management software is expected to deliver its operations by collaborating with the following components:

  • Resource Registry and its client libraries to add/remove/edit/merge contexts and their resources. The scope of this interaction is to make the resources discoverable in a new context (or un-discoverable if removed).
  • Authorization Framework to create new context tokens
  • WHN Manager: to add/remove HostingNodes/EServices from a context. Once a new context is available, the HostingNodes hosting the EServices belonging to the context need to be added to it. The RM has to contact the WHN Manager (docs?) to notify the changes in the context for that node. Requesting to WHN Manager to add an Hosting Nodes or EServices to a context has the following effect:
    • Smartgears requests to AuthorizationFramework a new token for the new Context for the HostingNode;
    • Smartgears requests to AuthorizationFramework a new token for the new Context for the each EService hosted in the HostingNode.
    • Smartgears invokes addToContext of HostingNode to the Resource Registry. Thanks to the propagation constraints also the hosted EServices are added to the new Context
    • Smartgears starts to accept requests for the new Context for all hosted EServices.

It is expected that a portlet is built atop the Resource Management (and integrates with its client library) to request the operations.

On the context it operates

Considering that each token is bound to a single context, while the Resource Management needs to work across contexts, the interactions with the Resource Registry have to be trusted because of the caller identity. I.e. the RR allows the Resource Management to do whatever it wants just because of its identity and regardless the context attached to the call. We would need to set up new policies for this authZ.

The service token with the identity of the Resource Manager service is obtained by the local SmartGears lib when the webapp with the service is activated by the container.

On the deployment

  • The service has to be stateless. Multiple instances will be deployed behind a HAProxy.
  • The service has to be able to manage more than one context (a.k.a. scope). In principle, we will deploy just one set of instances of the service (behind the proxy) to manage the infrastructure context, the VOs, and VREs. However, it should also be possible to deploy one instance for a specific VO.
  • A new instance of the service can be added at any time to the set of instances already deployed without manual configuration of the new instance or changes to the configuration of the pre-existing ones.

Architecture

The Resource Management is divided in the following software components:

Resource Manager service
A webapp hosted on a wHN that provides the main logic accessible through a RESTful interface
Resource Manager client
an abstraction over the JSON-based details of the interaction with the service
Resource I/O library
common definitions and utilities shared between the service and the client

Service: Context resource

When reporting error conditions, these resource methods make use of Code Exceptions.

Create a Context

POST /resource-manager/context?rrURL=http://host:port/resource-registry

Parameters

Name Type Required Description
body of the post request String true The JSON serialization of the context to create.
rrUrl String false The URL of the Resource Registry instance to contact.

Sample body for the request: 

{
  "@class": "Context",
  "header": {
    "@class": "Header",
    "uuid": "ba3003da-a394-4d71-872f-63d96e37a2b6",
    "creator": "manuele simi",
    "modifiedBy": null,
    "creationTime": null,
    "lastUpdateTime": null
  },
  "name": "firstContext",
  "parent": null,
  "children": []
}

Returned Values

In case of success: HTTP code 201 (CREATED)

In case of failure: HTTP code 406 (NOT_ACCEPTABLE)

The reason of the failure is specified as Error Code the response's entity:

  • INVALID_METHOD_REQUEST(0, "The request is invalid."),
  • MISSING_PARAMETER(1,"Required query parameter is missing."),
  • MISSING_HEADER(2, "Required header is missing."),
  • CONTEXT_ALREADY_EXISTS(3, "Context already exists at the same level of the hierarchy."),
  • CONTEXT_PARENT_DOES_NOT_EXIST(4, "Failed to validate the request. The request was not submitted to the Resource Registry."),
  • INVALID_REQUEST_FOR_RR(5, "Failed to validate the request. The request was not submitted to the Resource Registry."),
  • GENERIC_ERROR_FROM_RR(6, "The Resource Registry returned an error.");

Description

Steps for creating a new context:

  • Validation: the new context must be properly framed in the pre-existing context hierarchy. For example:
    • its parent context must exist
    • there must be no other context with the same name at the same level in the hierarchy.
  • Invoke the context method of Resource Registry to create the context.
  • Contact the Authorization Service by mean of its client lib to create a new context token.
  • Add the token to local node, i.e. the node on which the RM is running automatically joins the new context.

Delete a Context

DELETE /resource-manager/context/{{UUID}}?rrURL=http://host:port/resource-registry

Parameters

Name Type Required Description
id String true The UUID of the context to delete.
force Boolean false Delete a context and all its descendants. When false or not set, a context is deleted only if it does not have descendants. Default is set to false.
dryRun Boolean false Does not delete the context but returns all the contexts that would be deleted by the operation. Default is set to false.
rrUrl String false The URL of the Resource Registry instance to contact.

Returned Status

In case of success: HTTP code 200 (OK)

In case of failure:

  • HTTP code 304 (NOT_MODIFIED), if the operation fails
  • HTTP code 404 (NOT_FOUND), if the context in the request does not exist

Description

The operation is expected to:

  • check if all the conditions to delete the context apply
  • delete the context

Rename a Context

PUT /resource-manager/context/{{UUID}}?newName=value&&rrURL=value

Parameters

Name Type Required Description
id String true The UUID of the context to delete.
newName String true The new name to assign to the context.
rrUrl String false The URL of the Resource Registry instance to contact.

Returned Status

In case of success: HTTP code 200 (OK)

In case of failure:

  • HTTP code 304 (NOT_MODIFIED), if the operation fails
  • HTTP code 404 (NOT_FOUND), if the context in the request does not exist

Description

The operation renames an existing context.

Edit a Context

Move a Context

Service: Virtual Service resource

Add to Context

PUT resource-manager/resource/{{UUID}}

Parameters

Name Type Required Description
UUID String true The UUID of the Virtual Service and/or Software to add.
contextID String true The UUID of the context.

Description

Steps to perform:

  • Validation:
    • the input context must exist
    • Virtual Services and Services must not be already part of the context (unless replication is allowed).
  • Add the Virtual Services to the context. To do this, RM contacts the RR to:
    • query and navigate the parent context and select the pre-existing (already deployed) Virtual Services to add to the context (is that correct?)
    • add the Virtual Services to the context. With a cascade policy on the callFor relation, all the Services related to each Virtual Service are automatically added to the context by the RR
    • query and navigate the demands relations and discover the Software required by each Virtual Service (typically not running on SmartGears, e.g. a DBMS). Then add this Software to the context.
  • Add the Software received as parameter to the context. To do this, RM contacts the RR to add each Software to the new context. Because this Software does not run on SmartGears, the only operation to do is to make it visible in a context on the IS.

Related documentation

Retrieved from "https://gcube.wiki.gcube-system.org/index.php?title=Resource_Management_for_the_gCube_Model&oldid=31394"