Difference between revisions of "Executor"

Revision as of 17:50, 26 August 2009

The Executor acts as a container for gCube tasks, i.e. functionally unconstrained bodies of code that lack a network interface but can be dynamically deployed into the service and executed through its interface. In particular, gCube tasks are designed, packaged, and deployed as plugins of the Executor service.

An instance of the Executor publishes descriptive information about the co-deployed tasks, can execute them on demand on behalf of clients, and can inform clients on the state of their execution. Clients may interact with the Executor service through a library of high-level facilities that subsume standard service stubs to simplify the discovery of service instances and the execution of tasks through those instances.

Design

Like all services that can be dynamically extended with plugins, the Executor has a plugin manager that accepts requests to register or deregister plugins with gCube tasks. The requests are not issued by service clients, however. They are issued by the Deployer in response to the availability of tasks in the infrastructure. The manager persists plugin profiles to autonomically re-register them at each container restarts.

Clients interact instead with either one of two port-types: the Engine and the Task.

The Engine port-type is the point of contact for clients that wish to launch the execution of registered tasks. The port-type is stateful, in that it maintains descriptions of the available tasks in a single resource, the engine. The engine is created at service startup, when it subscribes with the plugin manager to be notified of plugin registration and de-registration events. It is then bound to the port-type into a WS-Resource accessible to clients via the implied resource pattern of WSRF. The task descriptions are modelled as a single, multi-valued Resource Property (RP) of the WS-Resource and published in all the scopes of the service instance at regular intervals. Task descriptions include the name of the task, a textual description, a set of arbitrary-valued properties, prototypical examples of the task inputs, and prototypical examples of the task outputs. The precise definition of the RP and the signature of the launch operation can be found in the WSDL of the Engine port-type.

The Task port-type is the the point of contacts that wish to monitor the execution of tasks. The port-type is stateful, in that it maintains the execution state of tasks in dedicated resources. Task execution resources are created by the engine when tasks are launched and are bound to the port-type into WS-Resources available via the implied resource pattern of WSRF. The execution state is modelled as RPs of the WS-Resources and published in all the scopes of the WS-Resources at regular intervals. RPs include the start time, inputs, and current state of the execution, as well as the logs, outputs and errors produced by the task. The port-type does not expose ad-hoc operations for monitoring purpose but relies on the standard operations of the gCube Notification Provider.

Sample Usage

The examples below use the high-level facilities of the client library of the Executor, partly because it is the recommended way to interact with the service and partly because the use of plain stubs (also included in the library) can be inferred from the public interfaces of the service.

Conceptually, most clients engage in the following interactions:

• discover service instances that can execute the target task. This requires interaction with the Information System.
• launch the execution of the task with one the discovered instances. This requires interaction with the Engine port-type of the Executor.
• monitor the execution of the task. This requires interaction with the Task port-type of the Executor.

These interactions are conveniently subsumed by instances of ExecutorCall, a class that model high-level calls to the Executor service. ExecutorCall is instantiated with the name of the target task, the intended scope of the call, and, optionally, security settings. Scope information may be provided with a GCUBEScope or a GCUBEScopeManagers, while security settings are provided by a GCUBEScopeManager. If the call is issued from another service, scope and security information can also be provided by a GCUBEServiceContexts. The example below illustrates the instantiation possibilites:

String name = ....
GCUBEScope scope = ....
GCUBEScopeManager smanager = .....
GCUBESecurityManager secmanager = .....
GCUBEServiceContext context = ....

//some call
Executor call;
call = new ExecutorCall(name,scope);
call = new ExecutorCall(name,smanager);
call = new ExecutorCall(name,scope, secmanager);
call = new ExecutorCall(name,smanager, secmanager);
call = new ExecutorCall(name,context);

The call is now configured to transparently discover instances of the Executor service that can execute the target task. Published properties of the target task can be set on the call to further disambiguate discovery:

String propertyName = ...
String propertyValue = ...
call.setTaskProperty(propertyName,propertyValue);

Discovery, on the other hand, can be entirely bypassed if the endpoint of a suitable Executor instance is already known:

String hostname = ...
String port = ...
call.setEndpoint(hostname,port);

The method launch can then be invoked on the call to execute the target task. This may require the preliminary definition of task inputs as a Map of string keys and arbitrary object values, e.g.:

Map<String,Object> inputs = ...
String inputName = ...
String inputValue = ...
inputs.put(inputName,inputValue)
...

The name and value the inputs must of course align with task expectations (specified in the task documentation and also manifest is in the RP published by the Engine port-type of all service instances that can execute the target task). Here we assume a string valued input, though any input type provided by the plugin is allowed, e.g.:

String input2name = ....
MyType input2value = ...
inputs.put(input2Name, input2Value);

In this case, the call must be configured with the type mapping required to serialise MyType instances (a type mapping is a correspondence between a type and its serialisation on the wire). As type mappings are explicitly provided by the context of the plugin of the target task (see below), e.g. an instance of MyPluginContext, the client can conveniently set them on the call as follows:

MyPluginContext pcontext = new MyPluginContext();
call.addTypeMapping(pcontext.getTypeMappings());

Do notice that clients that use task-specific types have an explicit dependency on the plugin of the task in addition to the service client libraries.

The target task can finally be executed as follows:

ExecutorCall.TaskProxy proxy = call.launch(inputs);

where TaskProxy is the type of a local proxy of the running task. Clients can use it to poll the execution state (consult the documentatio for the full list of methods that can be invoked on a task proxy):

System.out.format("Task invoked started at %Tc with %s state",proxy.getStartTime(),proxy.getState());

The task proxy reflects the value of the RPs of the WS-Resource that models the execution of the target task. Its methods, however, execute against a local cache of the RP values and do not engage the remote WS-Resource. The cache is first populated immediately after the execution of the task but must be explicitly synchronized by clients whenever fresh information is required:

proxy.synchronize();

Typically, clients will wish to synchronise proxies when there is some change to the execution of the target task. The Executor allows clients to subscribe for changes to the overall state of the execution and to its output, and the client library offers a convenient abstraction for this purpose. TaskMonitor is an abstract class that defines callbacks for event notifications and clients can subclass it to implement the callbacks for the events of interest. One common way of doing so is with an anonymous class, e.g.:

TaskMonitor monitor = new TaskMonitor() {
      public void onStateChange(String newState) throws Exception {
        //state values are the string conversion of handler's states
        if (state.equals(State.Failed.INSTANCE.toString())) { 
	  this.getProxy().synchronize();//synchronise to get error
          System.out.println("task has failed with error "+this.getProxy().getError());
        }
        else if (state.equals(State.Done.INSTANCE.toString())) {
          this.getProxy().synchronize();//synchronise to get output
	   System.out.println("task has completed with: "+this.getProxy().getOutput().get("endresult")));
        }
        else logger.info("task has moved to status "+state);
     }

     public void onOutputChange(Map<String, Object> output) { 
        if (output.containsKey("endresult"))
	logger.info("output message is "+output.get("endresult")));
     }
};

This monitor defines callbacks for both type of events and will thus receive both. Optionally, a monitor can express interest in either type of event by passing a TaskMonitor.TaskTopic to the constructor of TaskMonitor. With the anonymous class approach used above this can be accomplished as follows:

TaskMonitor monitor = new TaskMonitor(TaskMonitor.STATECHANGE) {
      public void onStateChange(String newState) throws Exception {...}
};

The task monitor can finally be passed to the ExecutorCall as a parameter of the launch:

ExecutorCall.TaskProxy proxy = call.launch(inputs, monitor);

The call will then transparently subscribe the monitor with the WS-Resource that models the execution of the target task. It will also inject the local proxy in the monitor (as well as returning it from the launch method as we have already seen). This is why the callback implementations can retrieve it with this.getProxy(), as can be seen above.

Finally, note that tasks that require no inputs can be simply invoked as follows:

proxy = call.launch();
proxy = call.launch(monitor);

Plugin Development

(in progress)