ResultSet Framework
From Gcube Wiki
THIS SECTION OF GCUBE DOCUMENTATION IS CURRENTLY UNDER UPDATE.
Contents
ResultSet Framework
Introduction
The ResultSet Framework provides the enabling mechanism to pass by reference data between components and / or services that can be hosted in the same or remote nodes. It utilizes the WSRF Framework for keeping state and offers value adding opeartions for encapsulating remote and local calls, paging of content, data movement, remote processing and more described in available code documentation.
Implementation Overview
The framework consists of 4 sub components :
- the ResultSet library
- the ResultSet Service
- the ResultSet Client
- the ResultSet Garbage Collector
- The ResultSet is the enabling element that creates a linked list of pages holding the records that are part of the specific result set. It can iterate over the created parts and perform various operations on the structure of the list and the contents themselves. Its operations are mosty part specific bu they can also affect the entire result chain. Once a page of content has been created and the next in line has been initialized the part cannot be altered. Once it has been declared that the authoring of a result set has finished, the entire chain of it is immutible.
- The ResultSet Service is merely a WS front-end to the ResultSet library. It creates and manages WS-Resources holding rferences to instances of the ResultSet components. These instances are responsible to hold state regarding the iteration over the chain of pages and for accessing / modifying / querying the data held.
- The ResultSet Client is a library giving access to both high and low level operations that can be performed on a specific ResultSet component instance and its underlying data. It enables handling of ResultSets that are both wrapped through a WS front-end and that are locally manipulated though direct java invocations (within JVM).
- The ResultSet Garbage Collector is a deamon-like program that can be initialized to clear up resources used by the ResultSet Framework (removing filesystem resources and destroying created WS-Resources)
Dependencies
- ResultSetLibrary
- jdk 1.5
- WS-Core
- ResultSetService
- jdk 1.5
- WS-Core
- ResultSetLibrary
- ResultSetClient
- jdk 1.5
- WS-Core
- ResultSetLibrary
- ResultSetService
- ResultSetGarbageCollector
- jdk 1.5
- WS-Core
- ResultSetLibrary
- ResultSetService stubs
Usage Example
The ResultSets that can be created are content independent but depending on the content inserted different operations can be performed. So different readers and writers have been implemented to offer different levels of versatility depending on the content. Here he will mainly focus on the XML authoring and retrieving which is most commonly used and thouroughly tested up to this point. As more types of readers / writers become popular additional documentation will be made available
The code examples are not exaustive in error checking and should only be considered as usage templates
Creating an RS
/*A method that creates a new RSXMLWriter and populates it with some results*/
public static RSXMLWriter createRSWriter() throws Exception{
/*Create a new writer with the default parametrizable values*/
RSXMLWriter writer=RSXMLWriter.getRSXMLWriter();
/*Create a new writer which should have as a condition for paging inserted results either having
30 records per page or having a total size surpasing 1024 bytes */
//writer=RSXMLWriter.getRSXMLWriter(30,1024);
/*Create a new writer and set a lifetime property of 1000 millisecs. Also set that the paging
condition will be having 10 records per page or a total size surpasing teh default value*/
//writer=RSXMLWriter.getRSXMLWriter(new PropertyElementBase[]{new PropertyElementLifeSpanGC(1000)});
//writer.setRecsPerPart(10);
/*Add a new result with the given id, collection, rank and payload*/
writer.addResults(new ResultElementGeneric("id1","collection1","rank1","payload"));
/*Add a new result with the given id, collection, rank and payload*/
writer.addResults(new ResultElementGeneric("id2","collection1","rank2","payload"));
/*Add an array of results with the given id, collection, rank and payload*/
writer.addResults(new ResultElementBase[]{new ResultElementGeneric("id3","collection1","rank3","payload"),
new ResultElementGeneric("id4","collection1","rank4","payload")});
/*Give back the writer*/
/*One should ALWAYS close the RS he is creating in order to make the full payload available to readers*/
writer.close();
return writer;
}
Retrieving a locator
From a writer
/*This method retrieves an instance of an RSLocator capable of identifying a ResultSet authored by the provided RSXMLWriter. Depending on the type of
resource requested, this identifier can either be an identifier pointing to the local node (filesystem) or an idenitfier capable of pinpointing the RS
through a web service using the WS-Resource pattern. One can create many locators (of the same or different) type for the same authored RS*/
public static RSLocator getLocator(RSXMLWriter writer) throws Exception{
RSLocator locator=null;
/*Retrieves a locator identifying the RS in the local node*/
//locator=writer.getRSLocator(new RSResourceLocalType());
/*Retrieves a locator identifying the RS through WS-Resource pattern accesible from any node. The
The service pointed to must be in the localhost. This method can be used when manipulating the RSXMLWriter
outside container context. Otherwise the following method can be used*/
//locator=writer.getRSLocator(new RSResourceWSRFType("http://localhost:8080/wsrf/services/diligentproject/searchservice/ResultSetService"));
/*Retrieves a locator identifying the RS through WS-Resource pattern accesible from any node. The
default constructor utilizes the container context (which must be available) to compose the localy hosted
ResultSetService that must be deployed*/
locator=writer.getRSLocator(new RSResourceWSRFType());
return locator;
}
From a reader
/*This method retrieves an instance of an RSLocator capable of identifying the ResultSet read by the provided RSXMLReader. Everythime the method is called
the same resource is identified. Both types of locators can be serialized to string and utilized to initialize a new RSLocator*/
public static RSLocator getLocator(RSXMLReader reader) throws Exception{
RSLocator locator=null;
/*Retrieves a locator identifying the RS. The type is the same as the one used to initialize the RSXMLReader*/
locator=reader.getRSLocator();
return new RSLocator(locator.getLocator());
}
Reading an RS
Page by Page
/*This method instantiates a new RSXMLReader to point to the RS identified by the provided RSLocator and iterates over each page retrieving the contents*/
public static void readRS(RSLocator locator) throws Exception {
PropertyElementBase []props=reader.getProperties(PropertyElementEstimationCount.class,PropertyElementEstimationCount.propertyType);
System.out.println(((PropertyElementEstimationCount)props[0]).toXML());
RSXMLReader reader=RSXMLReader.getRSXMLReader(locator);
while(!reader.isLast()){
reader.getNumberOfResults();
/*Retrieve the records in the current page as instances of the provided class that extends the ResultElementBase*/
ResultElementBase []res=reader.getResults(ResultElementGeneric.class);
reader.getNextPart();
}
reader.getFirstPart();
do{
reader.getFullPayload();
}while(reader.getNextPart());
}
Iterating
/*This method instantiates a new RSXMLReader to point to the RS identified by the provided RSLocator and iterates over each record retrieving the
contents*/
public static void iterateRS(RSLocator locator) throws Exception {
RSXMLIterator iter=RSXMLReader.getRSXMLReader(locator).getRSIterator();
while(iter.hasNext()){
ResultElementGeneric elem=(ResultElementGeneric)iter.next(ResultElementGeneric.class);
/*Retrieve the DocID of the retrieved record*/
if(elem!=null) elem.getRecordAttributes(ResultElementGeneric.RECORD_ID_NAME);
}
}
Extending Base elements
Two types of elements are currently extendable in the context of the Framework.
- PropertyElementBase - Which is used to add / retrieve property elements describing the authored RSs
- ResultElementBase - Which is used to add / retrieve records from the RS
These elements must be available to both the producer and the consumer as library elements in order to be used. of course this is not mandatory and a consumer can ustilize a differnt element to retrieve the records that a consumer inserted using some other element. It is in the repsonsibility of the producer / consumer to synchronize the elements they use. These elements serve mainly as containers for the actual payload that should be inserted / retrieved possibly adding some common handling functionality on top of that payload.
Extending PropertyElementBase
/*This is ane example of extending the PropertyElementBase to derive Property Element that is used to add WSRF EndpointReferenceType serializations to
the RS head page*/
public class PropertyElementWSEPR extends PropertyElementBase{
/* The Type of the Property this Property element produces*/
public static String propertyType="WS-EPR";
private String epr=null;
/* Default contructor required by the PropertyElementBase in order to instantiate the class using reflection*/
public PropertyElementWSEPR(){}
/* Initializes a new PropertyElementWSEPR with the given EndpointReferenceType serialization */
public PropertyElementWSEPR(String epr) throws Exception{
this.epr=epr;
setType(PropertyElementWSEPR.propertyType);
}
/* The usefull property payload from the PropertyElementWSEPR point of view as valid xml */
public String toXML() throws Exception{
return this.epr;
}
/* The usefull property payload from the PropertyElementWSEPR point of view as returned by
PropertyElementWSEPR.fromXML */
public void fromXML(String xml) throws Exception{
this.epr=xml;
}
}
Extending ResultElementBase
/* A Result element that extends the ResultElementBase and can be used to insert and retrieve xml records from an RS*/
public class ResultElementFoo extends ResultElementBase{
/* The name of the attribute holding a desired record attribute */
public static final String RECORD_FOO_NAME="Foo";
private String payload;
/* Default contructor nessecary for the framework to instantiate the ResultElementFoo though reflection */
public ResultElementFoo(){}
/* Creates a new link ResultElementFoo with the provided Foo attribute value and payload */
public ResultElementFoo(String foo,String payload) throws Exception{
Vector<RecordAttribute> at=new Vector<RecordAttribute>();
at.add(new RecordAttribute(ResultElementFoo.RECORD_FOO_NAME,foo));
setRecordAttributes(at.toArray(new RecordAttribute[0]));
this.payload=payload;
}
private void setPayload(String payload){
this.payload=payload;
}
public String getPayload(){
return this.payload;
}
/* The Attributes that can be defined for a record are handled internally by the ResultElementBase and can be
set / accessed using the Base elements respective methods*/
public String getFooValue(){
return getRecordAttributes(ResultElementFoo.RECORD_FOO_NAME)[0].getAttrValue();
}
/* The xml representation of the record payload */
public String toXML() throws Exception{
return payload;
}
/* The payload of the result element as returned by the ResultElementFoo.toXML */
public void fromXML(String xml) throws Exception{
setPayload(xml);
}
}
Complex Operations
RS creation within Workflows
In the context of a workflow it might be desirable to send to the asking component a locator capable of identifying the RS that the producer will be populating. This is best done in a non blocking manner in a background thread.
/*This method creates an RS and starts a background thread populating the RS. It then returns the locator to the RS that is beeing populated*/
public static RSLocator populateRS() throws Exception{
RSXMLWriter writer=RSXMLWriter.getRSXMLWriter();
BackgroundPopulating rst=new BackgroundPopulating(writer);
rst.start();
return writer.getRSLocator(new RSResourceLocalType());
}
/*This extends Thread in order to enable populating of the RS the RSXMLWriters it is initialized with points to*/
public class BackgroundPopulating extends Thread{
RSXMLWriter writer=null;
/*The RSXMLWriter to use for populating the RS*/
public BackgroundPopulating(RSXMLWriter writer){
this.writer=writer;
}
/*Populate the RS*/
public void run(){
try{
writer.addResults(new ResultElementGeneric("id1","collection1","rank1","payload"));
writer.addResults(new ResultElementGeneric("id2","collection1","rank2","payload"));
writer.addResults(new ResultElementBase[]{new ResultElementGeneric("id3","collection1","rank3","payload"),
new ResultElementGeneric("id4","collection1","rank4","payload")});
writer.close();
}catch(Exception e){
e.printStackTrace();
}
}
}
Partial Localization
/*This method localizes a ResultSet keeping the top count records of the identified throught the RSLocator RS. The localization is in terms of the node
that hosts the RS content and not the Resource type. This can be further defined by the RSResourceType*/
public static RSLocator localizePart(RSLocator locator,int count) throws Exception {
RSXMLReader reader=RSXMLReader.getRSXMLReader(locator);
reader.isLocal(); //false
//RSXMLReader readerTop=reader.keepTop(count);
//readerTop.isLocal(); //false
//RSXMLReader readerLocalJVM=readerTop.makeLocal(new RSResourceLocalType());
//readerLocalJVM.isLocal(); //true
//RSXMLReader readerLocalWS=readerTop.makeLocal(new RSResourceWSRFType());
//readerLocalWS.isLocal(); //true
RSXMLReader readerLocalTop=reader.makeLocal(new RSResourceLocalType(),count);
readerLocalTop.isLocal(); //true
return readerLocalTop.getRSLocator();
}
RS xPath filtering
/*This method scans through every record of the RS and evaluates the provided xPath expression against each. If the evaluation returns some result, the
record is inserted in a new RS. A new RSXMLReader is created holding the matching record. The xPath expression must start with a reference to the current
node (eg .//[..])*/
public static RSLocator filterRS(RSLocator locator,String xPath) throws Exception {
RSXMLReader reader=RSXMLReader.getRSXMLReader(locator);
return reader.filter(xPath).getRSLocator();
}
RS as Flow Control Mechanism
The ResultSet Framework offers the additional functionality of synchronizing result production with result consumption. This functionality is simulated in the following attached code
ResultSet as Flow Contol Mechanism Example code
Pool Of RS Writers
Another feature that is available for usage is a pool of pre-created writers with already created locators. In cases of container heavy load and of WS based locators, this pool can save a significant amount of time in the creation of a writer. In the context of a service for example on can staticly define a factory of writers that include a configurable pool of precreated writers and use this factory to retrieve the writers to use. The code below shows this in sudo-code.
import org.diligentproject.searchservice.searchlibrary.rsclient.elements.pool.PoolConfig;
import org.diligentproject.searchservice.searchlibrary.rsclient.elements.pool.PoolObjectConfig;
import org.diligentproject.searchservice.searchlibrary.rsclient.elements.pool.RSPoolObject;
import org.diligentproject.searchservice.searchlibrary.rswriter.RSWriterFactory;
import org.diligentproject.searchservice.searchlibrary.rswriter.RSXMLWriter;
public class ExampleService implements gCubeRIStateICallback {
/**
* Factory of rs writers
*/
private static RSWriterFactory factory=null; //the factory that will be used and include the pool
static{
PoolConfig config=new PoolConfig(); //the pool configuration
PoolObjectConfig oConf=new PoolObjectConfig(); //configuration item
try{
oConf.FlowControl=false; //the specific writers should not allow flow control
oConf.MaxSize=20; //the maximum size of the specific writer pool
oConf.MinSize=8; //the minimum size (threshlod to repopulate)
oConf.ObjectType=RSPoolObject.PoolObjectType.WriterXML; //the type of writer
oConf.ResourceType=RSPoolObject.PoolObjectResourceType.WSRFType; //the type of locator to pre-contsruct
oConf.ServiceEndPoint=null; //not staticly created but in service context
config.add(oConf); //add this object in the pool
}catch(Exception e){
log.error("Could not initialize factory pool. Continuing",e);
}
factory=new RSWriterFactory(config);
}
public String doWork(DoWork params) throws SomeBaseFault{
Vector<String> results=MyLibrary.search(params.getQuery);
//background add records
PropertyElementBase [] props=new PropertyElementBase []{
new PropertyElementEstimationCount(results.size(),results.size(),results.size())};
// without the factory this call would have been RSXMLWriter.getRSXMLWriter(props)
return writer=factory.getRSXMLWriter(props).getRSLocator(new RSResourceWSRFType();
}
}
--Gpapanikos 13:14, 14 November 2007 (EET)
