Difference between revisions of "IS-Publisher"

Latest revision as of 13:56, 19 October 2016

In conjunction with the IS-Client and the IS-Notification, the IS-Publisher represents the mediation layer gCube Services will rely on to interact with the Information Service as a whole.

Design

The IS-Publisher is a Java library providing a reference implementation for a group of interfaces defined in the gCore Framework. The purpose of these interfaces is to define the behavior of providers of information to the gCore Based Information System.

Figure 1. ISPublisher Design

More specifically:

• by implementing the org.gcube.common.core.informationsystem.publisher.ISPublisher interface, the library allows gCube services to publish GCUBEResources and instances' states as of WS-ResourceProperty documents;
• by implementing the org.gcube.common.core.informationsystem.publisher.ISGenericPublisher and org.gcube.common.core.informationsystem.publisher.ISResource interfaces, the library provides a way to publish generic XML documents in the IS;
• by implementing the org.gcube.common.core.informationsystem.publisher.ISLocalPublisher interface, it provides a subscription/notification mechanism based on local events.

At runtime, all the above interfaces are dynamically bound by gCore to the implementation provided by the library.

Each registration request creates an internal resource sent to the appropriate IS service. Instance states and generic XML documents are wrapped as ISResources and sent to the Information Collector. GCUBEResources are instead sent to the IS-Registry service for validation and approval.

Figure 2. ISPublisher Interactions

Managed Resources

Instance State

An instance state is the set stateful WS-Resource created by that instance following the WSRF patterns.

In order to be published, a WS-Resource has to expose a view of its state as a mean of ResourceProperty and declare a registration file for that properties in its JNDI (<service folder>/etc/deploy-jndi-config.xml). This declaration is obtained through the publicationProfile element inside the service section. This is an example of such declaration:

<service name="...">
 
	<resource name="publicationProfile" type="org.gcube.common.core.state.GCUBEPublicationProfile">	
            <resourceParams>
                <parameter>
                    <name>factory</name>
                    <value>org.globus.wsrf.jndi.BeanFactory</value>
                </parameter>
                <parameter>
                    <name>mode</name>
                    <value>push</value>
                </parameter>
                <parameter>
                    <name>fileName</name>
                    <value>Registration.xml</value>
                </parameter>
            </resourceParams>
        </resource>
</service>

The registration file specifies which properties have to be published, when and how following the syntax defined for the WS-MDS Aggregator Source registrations. The following example shows a registration file for 3 resource properties (RPString, RPDate, RPAny):

<ServiceGroupRegistrationParameters
    xmlns:sgc="http://mds.globus.org/servicegroup/client"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:agg="http://mds.globus.org/aggregator/types"
    xmlns="http://mds.globus.org/servicegroup/client">
 
    <Content xsi:type="agg:AggregatorContent"
        xmlns:agg="http://mds.globus.org/aggregator/types">        
        <agg:AggregatorConfig>
            <agg:GetMultipleResourcePropertiesPollType 
                xmlns:stateful="http://gcube-system.org/namespaces/test/stateful">
                <agg:PollIntervalMillis>60000</agg:PollIntervalMillis>
                <agg:ResourcePropertyNames>stateful:RPString</agg:ResourcePropertyNames>
                <agg:ResourcePropertyNames>stateful:RPDate</agg:ResourcePropertyNames>
                <agg:ResourcePropertyNames>stateful:RPAny</agg:ResourcePropertyNames> 
            </agg:GetMultipleResourcePropertiesPollType>
        </agg:AggregatorConfig>        
       <agg:AggregatorData/>
    </Content>
 
</ServiceGroupRegistrationParameters>

Note that the ResourcePropertyNames have to be fully qualified with the namespace declared in the service's WSDL (http//gcube-system.org/namespaces/test/stateful in the example above).

Name of the file aside, the other parameter to consider in the JNDI file is the mode. Clients may select the push or pull to publish their instance state. The chosen mode heavily impacts on the behavior of the ISPublisher.

Publishing with push mode

With the push mode, the resource properties are (re)published whenever the values of one of them changes. If the RPs change every now and then, intermittently or there are peaks of changes in the RPs but longer periods without any change, this is the preferred way to go.

Publishing with pull mode

When RPs change quite frequently and constantly and it's not critical to have them refreshed immediately, the pull mode has to be selected. In this modality, the ISPublisher periodically harvests and collects the RP values from the WS-Resource and publishes them in the Information Collector service. The polling period is indicated in the PollIntervalMillis parameter inside the registration file.

GCUBEResource

GCUBEResource profiles are managed by interacting with the ResourceRegistration portType of IS-Registry service. The ISPublisher here acts as a simple mediator by selecting the IS-Registry instance in the publishing scope and invoking its operations.

XML Document

Starting from the release 3.0 (Feb 2011), the ISPublisher offers the possibility to publish well-formed XML documents in the IS. This feature exploits the new XMLCollectionAccess portType of the Information Collector compliant with the WS-DAIX specification. This raises significant opportunities for clients to create their own collections of indexed documents that can be queried through the IS-Client library.

Library Implementation Notes

Behind the exposed interfaces, there are three main paths of execution inside the library depending on the type of resources to manage.

If the published resource is a GCUBEResource, it is synchronously sent to the IS-Registry instance in the given scope. The selection of the appropriate instance is done by a set of handlers defined in org.gcube.common.informationsystem.publisher.impl.registrations.resources.

A completely different approach if followed when an Instance State is published. Each registered instance state requires a dedicated running task that reacts to the state changes. An instance of the org.gcube.common.informationsystem.publisher.impl.instancestates.InstanceStatePublisher class keeps track of the activated tasks. If the pull mode has been selected, an instance of RegisterInstanceStatePullHandler class periodically harvests the resource properties document and creates an ISResource to send to the InformationCollector for indexing. If the push mode has been chosen, an instance of RegisterInstanceStatePushHandler acts as an observer of the resource properties values and each time one of them changes receives a notification from the RPSet (gCore) and then sends the resource properties document to the InformationCollector.

The third path is related to the publication of generic XML Documents. The received documents (wrapped as ISResource) are simply sent to the InformationCollector for indexing.

Configuration

The behavior of the ISPublisher can be partially configured via a properties file. This file is located in $GLOBUS_LOCATION/config/ISPublisher.properties. This is an example of such a file with a bit of explanation on the configurable properties:

# Timeout in the communications with the IS-Registry
REGISTRY_CHANNEL_TIMEOUT=60000

# Timeout in the communications with the IS-InformationCollector
COLLECTOR_CHANNEL_TIMEOUT=120000

# Max number of parallel registrations for this gHN
MAX_PARALLEL_REGISTRATIONS=100

# Max tries for publishing the resource (-1 means unlimited attempts)
RESOURCE_PUBLICATION_MAX_ATTEMPTS=3

# Interval between two bulk publications
BULK_PUBLICATIONS_INTERVAL=20000

Bulked Publications

To support an optimization of the load at infrastructure level, from release 3.0 on, instance states and XML documents to be sent to the InformationCollector are queued and periodically sent in a bulk way (i.e. in a single invocation). This is achieved with a new internal publisher named GCUBEGenericBulkPublisher. The interval between two bulk publications is specified in the BULK_PUBLICATIONS_INTERVAL of the configuration file.

Sample Usage

GCUBEResources (ISPublisher interface)

How to publish a GCUBEResource

The registerGCUBEResource operation has to be invoked to publish a profile in the IS. The operation takes the following parameters as input:

• the resource to publish, an instance of a sub-class of the GCUBEResource class
• the operational scope, an object representing a scope in the gCube model of VO/VREs
• the security manager, the component keeping track of credentials, and returns a string containing the registered profile if the registration is successful.

Here it is an example of GCUBEGenericResource registration:

try {
	ISPublisher publisher = GHNContext.getImplementation(ISPublisher.class); //dynamically load the reference implementation of the interface
	GCUBEGenericResource resource = GHNContext.getImplementation(GCUBEGenericResource.class);
        //load the resource here
	publisher.registerGCUBEResource(resource, this.scope, ServiceContext.getContext());
} catch (Exception e) {			
	logger.error(e);
	throw new Exception("Unable to register the Resource");
}

How to update a GCUBEResource

The updateGCUBEResource operation has to be invoked to update a profile in the IS. The operation takes the following parameters as input:

• the resource to update, an instance of a sub-class of the GCUBEResource class
• the operational scope, an object representing a scope in the gCube model of VO/VREs
• the security manager, the component keeping track of credentials, and returns a string containing the registered profile if the registration is successful.

Here it is an example of GCUBEGenericResource update:

try {
	ISPublisher publisher = GHNContext.getImplementation(ISPublisher.class); //dynamically load the reference implementation of the interface
	GCUBEGenericResource resource = GHNContext.getImplementation(GCUBEGenericResource.class);
        //load the resource here...
	publisher.updateGCUBEResource(resource, this.scope, ServiceContext.getContext());
} catch (Exception e) {			
	logger.error(e);
	throw new Exception("Unable to register the Resource");
}

How to remove a GCUBEResource

The removeGCUBEResource operation has to be invoked to remove a profile from the IS. The operation takes the following parameters as input:

• the resource ID, the unique identifier of the resource to delete
• the resource type, the type of the resource to delete
• the operational scope, an object representing a scope in the gCube model of VO/VREs
• the security manager, the component keeping track of credentials, and returns a string containing the registered profile if the registration is successful.

Here it is an example of GCUBEGenericResource removal:

try {
	ISPublisher publisher = GHNContext.getImplementation(ISPublisher.class); //dynamically load the reference implementation of the interface
	GCUBEGenericResource resource = ....;// the resource could be obtained as a results of a query, loaded from a file/string or passed as input by someone else...
	publisher.removeGCUBEResource(resource.getID(), resource.getType() this.scope, ServiceContext.getContext());
} catch (Exception e) {			
	logger.error(e);
	throw new Exception("Unable to register the Resource");
}

Instance states (ISPublisher interface)

Published Structure of an instance state

The ISPublisher automatically enriches the instance state document with a set of information about the instance and gHN publishing the document. The resulting structured document sent to the IS-InformationCollector looks like in this example:

   <Data>
       <!-- sample RP --> 
      <ns1:RPString xmlns:ns2="http://www.globus.org/foobar" xmlns:ns1="http://gcube-system.org/namespaces/test/stateful" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ns2:QueryExpressionDialect">RP value</ns1:RPString>
        <!-- end sample RP -->
 
      <ns3:GHN xmlns:ns4="http://www.globus.org/foobar" xmlns:ns3="http://gcube-system.org/namespaces/common/core/porttypes/GCUBEProvider" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ns4:QueryExpressionDialect">380cdb40-4b6d-11e0-a429-d9deb44f0111</ns3:GHN>
 
      <ns5:RI xmlns:ns5="http://gcube-system.org/namespaces/common/core/porttypes/GCUBEProvider" xmlns:ns6="http://www.globus.org/foobar" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ns6:QueryExpressionDialect">f6c17bb0-5435-11e0-9a8c-a4b5e12ad05b</ns5:RI>
 
      <ns7:ServiceClass xmlns:ns7="http://gcube-system.org/namespaces/common/core/porttypes/GCUBEProvider" xmlns:ns8="http://www.globus.org/foobar" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ns8:QueryExpressionDialect">Test</ns7:ServiceClass>
 
      <ns9:Scope xmlns:ns9="http://gcube-system.org/namespaces/common/core/porttypes/GCUBEProvider" xmlns:ns10="http://www.globus.org/foobar" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ns10:QueryExpressionDialect">/gcube/devNext</ns9:Scope>
 
      <ns11:ServiceID xmlns:ns12="http://www.globus.org/foobar" xmlns:ns11="http://gcube-system.org/namespaces/common/core/porttypes/GCUBEProvider" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ns12:QueryExpressionDialect">deccebb0-5657-11e0-a91a-8c7e5ba2721e</ns11:ServiceID>
 
      <ns13:ServiceName xmlns:ns14="http://www.globus.org/foobar" xmlns:ns13="http://gcube-system.org/namespaces/common/core/porttypes/GCUBEProvider" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ns14:QueryExpressionDialect">StatefulService</ns13:ServiceName>
 
   </Data>

Publishing an instance state

To publish an instance state, the registerWSResource operation has to be invoked. It takes as input the following parameters:

• the stateful resource
• the scope
• optionally the registration name

Here it is an example of WSResource registration:

try {
	ISPublisher publisher=GHNContext.getImplementation(ISPublisher.class);
	publisher.registerWSResource(<instance extends GCUBEWSResource>,scope);
} catch(Exception e) {
	logger.error(e);
	logger.warn("could not publish RPs for "+GCUBEWSResource.this.getClass().getSimpleName()+"("+getID()+") in "+scope,e);}		
}

Removing an instance state

To delete an instance state from the IS, the removeWSResource has to be invoked. It takes the following parameters:

• the stateful resource
• the scope
• optionally the registration name

If the resource was published in the pull mode, the correspondent update task is also cancelled.

Here it is an example of WSResource unregistration:

try {
	ISPublisher publisher=GHNContext.getImplementation(ISPublisher.class);
	publisher.unregisterWSResource(<instance extends GCUBEWSResource>,scope);
} catch(Exception e) {
	logger.error(e);
	logger.warn("could not remove RPs for "+GCUBEWSResource.this.getClass().getSimpleName()+"("+getID()+") in "+scope,e);}		
}

XML Documents (ISGenericPublisher interface)

Dealing with XML Documents requires more knowledge of the underlying implementation. The client has to:

• create an ISResource
• set the resource identifier
• set the target collection (the collection that will contain the document once indexed in the InformationCollector), eventually a full path separated by slashes (the entire path will be created if it does not exist)
• set the resource type, allowed types are defined in org.gcube.common.core.informationsystem.publisher.ISRESOURCETYPE. For generic resources, the type ISRESOURCETYPE.WSDAIX must be selected

Publishing an XML Document

ISGenericPublisher publisher = GHNContext.getImplementation(ISGenericPublisher.class);
ISResource resource = GHNContext.getImplementation(ISResource.class);
String id = "...";
resource.setID(id);
resource.setType(ISRESOURCETYPE.WSDAIX);
resource.setCollection("ParentCollection/MyCollection");
logger.debug("Registering document: " + id);
try {
	publisher.register(resource, scope)
} catch (Exception e) {
	logger.error("Failed to register document: " + id, e);
}

Removing an XML Document

ISGenericPublisher publisher = GHNContext.getImplementation(ISGenericPublisher.class);
ISResource resource = GHNContext.getImplementation(ISResource.class);
String id = "...";
resource.setID(id);
resource.setType(ISRESOURCETYPE.WSDAIX);
resource.setCollection("ParentCollection/MyCollection");
logger.debug("Removing document: " + id);
try {
	publisher.remove(resource, scope);
} catch (Exception e) {
	logger.error("Failed to remove document: " + id, e);
}

Local Events (ISLocalPublisher Interface)

The interface provides a subscription/notification mechanism based on local events allowing consumers to be notified about changes in any GCUBEResource published by others within the same GHN. This mechanism is widely exploited by the IS services whenever they are themselves hosted on the local GHN.

Subscribing for local events

The operation subscribeLocalProfileEvents(LocalProfileConsumer consumer) subscribes a new consumer as listener of local event. A consumer is an instance of a class extending the LocalProfileConsumer class also defined in the interface. The following methods of the class can be overridden to receive the appropriate callbacks after a modification on any of the local profiles occurred:

• • onProfileRemoved(String resourceID, String type, GCUBEScope scope) - removed profile event callback
  • onProfileUpdated(GCUBEResource resource, GCUBEScope scope) - updated profile event callback.
  • onProfileRegistered(GCUBEResource resource, GCUBEScope scope) - new registered profile event callback.

• isEnabled(GCUBEScope scope) - it checks if in the given scope, the notification mechanism based on local events is enabled or not.

The following example shows how to register a consumer and be notified for local events occurring in the hosting node:

import org.gcube.common.core.informationsystem.publisher.ISLocalPublisher.LocalProfileConsumer;
 
 
private void subscribeToLocalRegistrationEvents() throws Exception{
	ISLocalPublisher pub = GHNContext.getImplementation(ISLocalPublisher.class);
	LocalProfileConsumer cons = new LocalProfileConsumer() {
 
	        /* (non-Javadoc)
	         * @see org.gcube.common.core.informationsystem.publisher.ISLocalPublisher.LocalProfileConsumer#onProfileRegistered(org.gcube.common.core.resources.GCUBEResource)
	         */
	        @Override
	        protected void onProfileRegistered(GCUBEResource resource, GCUBEScope scope) {
	            logger.debug("onProfileRegistered event received" );                                   
	            //manage the event...
 
	        }
 
	        /* (non-Javadoc)
	         * @see org.gcube.common.core.informationsystem.publisher.ISLocalPublisher.LocalProfileConsumer#onProfileRemoved(java.lang.String, java.lang.String)
	         */
	        @Override
	        protected void onProfileRemoved(String resourceID, String type, GCUBEScope scope) {
	            logger.trace("onProfileRemoved event received");
	            //manage the event...	        		                               
	        }
 
	        /* (non-Javadoc)
	         * @see org.gcube.common.core.informationsystem.publisher.ISLocalPublisher.LocalProfileConsumer#onProfileUpdated(org.gcube.common.core.resources.GCUBEResource)
	         */
	        @Override
	        protected void onProfileUpdated(GCUBEResource resource, GCUBEScope scope) {
	            logger.trace("onProfileUpdated event received");
	            //manage the event...	        		                               
	        }
 
 
	    };
 
    pub.subscribeLocalProfileEvents(cons);
}

On GCUBEResources loading

There are several ways to obtain an object representing a GCUBEResource. However, none of them includes the direct instantiation of a sub-class of GCUBEResource.

The examples below illustrate how to load the GCUBERunningInstance resource but they can be applied to any other class extending GCUBEResource.

• The resource is serialized in the IS, then returned as a result from a query:

ISClient client = GHNContext.getImplementation(ISClient.class);
GCUBERIQuery query = client.getQuery(GCUBERIQuery.class);
List<GCUBERunningInstance> results = client.execute(query, GCUBEScope.getScope("AScope"));

• The resource is serialized in file:

File fileserialization = new File("..");
GCUBERunningInstance resource = GHNContext.getImplementation(GCUBERunningInstance.class);
resource.load(new FileReader(fileserialization));

• The resource is serialized in a String:

String xmlserialization = "...";
GCUBERunningInstance resource = GHNContext.getImplementation(GCUBERunningInstance.class);
resource.load(new StringReader(xmlserialization));

• The resource is created from scratch:

GCUBERunningInstance resource = GHNContext.getImplementation(GCUBERunningInstance.class);
//invoke the setters on the resource here...

In all the cases, if the resource does not have an identifier, the loading procedure automatically assigns it a new identifier.