Difference between revisions of "Application Support Layer"

Latest revision as of 14:22, 2 February 2016

Introduction

The Application Support Layer (ASL) is a middleware between gCube Lower level Services and the gCube Presentation Layer. Its main goal is to provide a unified interface for accessing gCube and to allow application developers to deal only with how to render and present the corresponding functionality. Furthermore, ASL is responsible for covering all the application logic needed in order to fulfill users’ requirements. Since ASL aims at addressing a wide audience - not only gCube developers, it is designed to wrap gCube’s complexity by packing and interfacing standard procedures like retrieving service running instances from IS or accessing information the services or the IS holds. It should be mentioned that ASL users need no knowledge about where the services are running and how to access them in order to accomplish their goal. In particular, ASL consists of ASL Core and ASL Extensions. ASL Core is the group of basic functionality on which developers can build their own extensions. It provides a session mechanism as well as easy ways to retrieve running instances and invoke services. It also deals with security and authentication. On the other hand, ASL Extensions are libraries that extend ASL Core in order to cover the usage of other (new / non core) services. ASL Core functionality facilitates the development of these libraries. They aim to provide the GUI developer with a full package that will cover all the application logic so as to create the corresponding Graphical Interfaces. For each new functionality offered by gCube, a corresponding ASL Extension should be developed. Offering both Java and HTTP APIs, ASL supports multiple ways to contact and interact with gCube lower level services. In essence, ASL acts as a gateway for other technologies to access gCube advanced facilities. This means that it receives all the requests to call gCube services and, acting as a proxy, it invokes the corresponding services.

Reference Architecture

In brief, the design of ASL is motivated by the following objectives:

• To remove the application logic from the presentation layer.
• To enable external developers to create Graphical User Interfaces (GUIs) for gCube system without any knowledge of its internal design and architecture.
• To speed up gCube system’s performance by caching frequently used, and/or rarely modified resources.
• To provide an easy way to manage credentials.
• To offer a global session management that enforces interoperability and intercommunication between application components.

The aforementioned objectives and design decisions are analyzed in the following sub-sections.

As it is depicted in the figure above, ASL consists of two main sections: ASL Core and ASL Extensions. ASL core is internally organized in three main groups: session management, cache, and the core functionality. ASL extensions currently contain the packages for ASL UserProfile and ASL Search.

ASL Core Functionality

ASL Core provides the fundamentals for building a gCube Graphical Application. It is the API that ASL consumers use in order to interact with gCube. It acts as a proxy that hides complexity of several steps needed for the invocations. It also contains the application logic for interpreting information received by gCube and transforming it into a presentable form. It also facilitates ASL extension developers by dealing with security issues and user credentials, session management and retrieval of generic resources.

Session Management

An important part of ASL Core is the session management. In order to consider ASL as a framework that completely supports users’ needs, it should implement its own mechanisms for providing sessions. Session is of crucial importance for the presentation layer since it provides means of storing information related to each user and means of intercommunication between application components. For this reason, a Session Manager has been implemented, based on the singleton design pattern. Its role is to administrate session objects, and to provide each user with his/her session. By using it, an application component can intercommunicate with another component by storing information in the session, and reading it from the other one. In fact, session is a hash map containing name – value pairs for various properties.

Resources Retrieval from IS

An functionality offered by the ASL Core, is the retrieval of Generic resources from the IS. It exposes an API to read and write GenericResource objects on the IS. Then, by using extensions the GenericResource content can even be parsed to extract it's information (i.e. the asl content extension offers such a functionality).

The ASL core component can be found at the following maven coordinates:

 
<groupId>org.gcube.applicationsupportlayer</groupId>
<artifactId>aslcore</artifactId>
<version>...</version>

ASL Extensions

ASL Content Extension

ASL Content extension supports the management of Resources and Digital Objects which reside within the IS and hold information regarding all the available collections (data sources) and their linked data. It offers methods to extract information from IS objects, parse the content of gCube collection objects, modify the content and replace it on the IS.

The ASL Content component can be found at the following maven coordinates:

 
<groupId>org.gcube.applicationsupportlayer</groupId>
<artifactId>aslcontent</artifactId>
<version>...</version>

ASL Search Extension

ASL Search Extension supports search functionality in order to query collections and retrieve the results. It contacts the SearchSystem Service and the ResourceRegistry in order to retrieve information about the collections and the available fields, joining the knowledge acquired from their dynamic and static configuration. Graphical User Interface can build the query step by step by first selecting the collections, then filling in the criteria, and finally submitting the query. Once the user's selections are made, a CQL Query is built from within the component, using the GCQLParser library. Then the query is submitted to the SearchMaster in order to be executed. The results will be retrieved in pages either in a personalized html format or as plain metadata (e.g. xml). Additionally, the user can navigate through results be requesting the previous, next or first page.

The ASL search extension can be found at the following maven coordinates:

 
<groupId>org.gcube.applicationsupportlayer</groupId>
<artifactId>aslsearch</artifactId>
<version>...</version>

ASL VRE Management Extension

In order to manage a Virtual Research Environment, ASL VRE Management extension offers a rich API so as to administer the infrastructure. Part of the functionality provided is to create/ update/ delete a Generic Resource on IS, to query IS in order to retrieve information such as running instances, available GHNs, in addition to creating / updating a VRE, and dynamically deploying a service.

ASL Social Extension

gCube applications use a common library for exploiting the Social Data Sharing facilities. gCube Applications do not interact with the Social Library directly. Rather they exploit this ASL-Extension capability. Please refer to the Social Networking Library wiki page for more information about how to exploit it.

ASL Access Logger Library

The Access Logger library provides methods in order to log the most common actions that are performed when using the D4Science portal. The actions that can be logged using the corresponding methods are:

• The login to a VO/VRE
• All types of search (Simple, Advanced and collection Browsing)
• Content Retrieval

For any other information the GenericLogger can be used to log any given message

Furthermore portlets can extend AccessLogger's interface and create their own specific entries that will be logged.

How to use the Access Logger

In order to use the AccessLogger first you need to create an instance of the AccessLogger object that is based on the Singleton resource pattern.

AccessLogger log = AccessLogger.getAccessLogger();

Then depending on the action you would like to log you need to create an instance of the object that logs the desired action.
The sample code below shows how to log the login to a VO/VRE and a simple search query:

LoginToVreAccessLogEntry loginEntry = new LoginToVreAccessLogEntry();
/*
* The first argument is the username of the user that performs the action
* The second argument is the name of the working VO/VRE
*/
log.logEntry("testUsername", "ECOSYSTEM/TryIt", loginEntry);
 
String collections[][] = new String[2][2];
collections[0][0] = "Earth images";
collections[0][1] = "12345";
collections[1][0] = "Landsat 7";
collections[1][1] = "54321";
SimpleSearchAccessLogEntry simpleEntry = new SimpleSearchAccessLogEntry(collections, "satellite, sea");
/*
* The first argument is the username of the user that performs the action
* The second argument is the name of the working VO/VRE
*/
log.logEntry("testUsername", "ECOSYSTEM/TryIt", simpleEntry);

The other actions follow the same pattern.

The logs are saved in the tmp/accessLogs directory. A logger file is created for every day following this pattern: accessLog"DATE".log

How to create your own entries by extending the Access Logger

The AccessLogger library can be used and extended by any component in order to log its own entries to the access log.

In order to create your own entry you should:

• Create your own class that extends the AccessLogEntry class
• The constructor of your class must invoke the super constructor and pass the name of your entry that represents the Type of the log (see the example below)
• The getLogMessage should return the message that will be logged

public class ContentRetrievalLogEntry extends AccessLogEntry {
 
	private String objectID;
 
	private String objectName;
 
	/**
	 * Constructor 
	 * 
	 * @param objectID The ID of the object that is retrieved
	 * @param objectName The name of the object that is retrieved
	 */
	public ContentRetrievalLogEntry(String objectID, String objectName) {
		super(EntryTypeConstants.RETRIEVE_CONTENT_ENTRY);
 
		this.objectID = replaceReservedChars(objectID);
		this.objectName = replaceReservedChars(objectName);
	}
 
	/**
	 * @return The log message for a Content retrieval entry
	 */
	public String getLogMessage() {
		String message = "";
		message += TemplateConstants.CONTENT_ID + TemplateConstants.eqChar + 
			this.objectID + TemplateConstants.separateCharacters + TemplateConstants.CONTENT_NAME +  
			TemplateConstants.eqChar + this.objectName;
 
		return message;
	}
 
}

• NOTE: The AccessLogEntry class provides the static method replaceReservedChars that should be invoked on the message to be logged so as to ensure that none of the reserved characters used by the logger will be part of the message

Home Library

The Home Library manage and persist the users homes. More informations can be found here.

Caching for Performance

Cache plays a main role within ASL. Its existence is considered essential in order to improve gCube’s performance and to give the impression to the end-user that the application layer is always ready to promptly respond to their requests. Therefore, a variety of resources that are frequently requested are cached in order to speed up response time. Depending on the type of the cached resource, different refresh and storage policies can be applied. In particular, one can configure the expiration time of the cached resources, the maximum number of these objects in memory, whether or not to support overflow on disk, the maximum size of objects on the disk, etc. Furthermore, examples of resources that are cached are:

• The available collections per VRE: ASL Content extension caches the names of the available collections in a hierarchical structure together with information about the available languages, searchable fields, and the existence of any special kind of index, like full text, geospatial, or similarity index.

• Content information, it contains information about the digital objects, like their mime type, length, the existence of annotations over the object, the number of associated metadata, etc. This cache is implemented by ASL Content extension.

• Endpoint Resources for various service nodes (e.g. discovered search service endpoints). This cache is implemented by ASL Search extension.

• Generic Resources, it caches generic resources from IS since they are frequently requested and yet rarely modified. Examples of such resources are the metadata presentation xslts, the record presentation xslts, etc. This cache is placed in ASL Core, as the generic resources are frequently asked, their use is destined for a large variety of reasons and they should be provided by the core unit of ASL.

• Profiles, user’s profile is used in many cases in order to personalize the layout based on user’s preferences. Thus, having a replica in main memory (or in the local disk) speeds up the procedure. This caching mechanism is placed inside ASL User Profile extension.

Download

ASL jars can be found in the ETICS latest build.

How to use ASL

ASL Core - Retrieving Session

Graphical User Interface developers should use the ASL internal session in order to store information. In order to use the ASL's session, you need to take an instance of the SessionManager object that is based on the Singleton resource pattern. Through this instance a unique session for each user can be retrieved. If GUI is a Web Application (portlet, servlet, jsp etc), then the proposed way to retrieve user's session is:

ASLSession aslSession = SessionManager.getInstance().getASLSession(session.getId(), username);

Where session is either a portlet-session or a servlet-session, and username is the username of the logged in user.

Needed imports:

import org.gcube.application.framework.session.ASLSession;
import org.gcube.application.framework.session.SessionManager;

Once you have the ASLSession object, you can use ASL API to communicate with gCube infrastructure.

VRE Management extension - Querying IS

By using ISInfo class, you can directly query the IS in order to retrieve information.
Additionally, you can add / remove a GHN and a Running Instance to a specific scope, you can retrieve the profiles of the collections of a given scope, you can get the metadata-collections associated with a given collection as well as the indices associated with a collection or metadata collection.

ASL Core - Managing Generic Resources

You can manage gCube generic resource by using the GenericResource class of ASL. Through this class you can create, update, delete and retrieve a generic resource from IS.

The sample code below represents how you can create, update and delete a generic resource:

ASLSession aslSession = SessionManager.getInstance().getASLSession(session.getId(), username);
GenericResource gR = new GenericResource(aslSession);
ISGenericResource genResource = new ISGenericResource();
try {
	genResource.setName(SessionConstants.ScenarioSchemaInfo);
	genResource.setDescription("Bla foo test");
	genResource.setBody("<DL>the dl :-)</DL>");
} catch (FileNotFoundException e) {
	e.printStackTrace();
} catch (Exception e) {
	e.printStackTrace();
}		
try {
	gR.createGenericResource(genResource); // Creating the generic resource!!!
	genResource.setDescription("this is different description!!!"); // Changing the description (locally)
	gR.updateDiligentGenericResourceByID(genResource); // Updating the generic resource on IS
	gR.remove(genResource); // Removing the generic resource
} catch (RemoteException e) {
		e.printStackTrace();
	}
}

The sample code below represents how you can retrieve information for generic resources:

/* Get the "name - id" pairs of the available generic resources */
 
ASLSession aslSession = SessionManager.getInstance().getASLSession(session.getId(), username);
 
LinkedHashMap<String,String> existingGRs = new LinkedHashMap<String,String>();
GenericResource gResources = new GenericResource(aslSession);
List<Pair> pairs = gResources.getAvailableGenericResourceNames();
 
for (int i=0; i<pairs.size(); i++) {
        String GRID = pairs.get(i).getValue();
        String GRName = pairs.get(i).getName();
        existingGRs.put(GRID, GRName);
}

/* Retrieve the list of the generic resources with the given name / id */
 
ASLSession aslSession = SessionManager.getInstance().getASLSession(session.getId(), username);
GenericResource gResources = new GenericResource(aslSession);
 
List<ISGenericResource> genericResources = gResources.getGenericResourceByID(“resourceId”);
 
genericResources = gResources.getGenericResourceByName(“resourceName”);

Additionally, ASL provides methods to retrieve from IS all tree collections and all Opensearch collections for the current scope.

ASLSession aslSession = SessionManager.getInstance().getASLSession(session.getId(), username);
GenericResource gResources = new GenericResource(aslSession);
//retrieve the Generic Resources of all Tree Collections from IS (hashmap of {collectionID,GenericResource})
HashMap <String,org.gcube.common.resources.gcore.GenericResource> treeColGenResource = gResources.getAllTreeCollections(false);
//retrieve the Generic Resources of all OpenSearch Collections from IS (hashmap of {collectionID,GenericResource})
HashMap <String,org.gcube.common.resources.gcore.GenericResource> osColGenResource = gResources.getAllOpenSearchCollections(false);
// retrieve the Cardinality of a Tree Resource (hashmap of {collectionID, cardinallity})
HashMap<String, String> cardinalites = getTreeResourcesCardinalities();

A sample generic resource:

<Resource version="0.4.x">
 
   <ID>666cc610-dab3-11df-9db7-e8211f3b1465</ID>
 
   <Type>GenericResource</Type>
 
   <Scopes>
 
      <Scope>/d4science.research-infrastructures.eu/Ecosystem/DRIVER</Scope>
 
   </Scopes>
 
   <Profile>
 
      <SecondaryType>GenericResource</SecondaryType>
 
      <Name>ScenarioCollectionInfo</Name>
 
      <Description>Available collection groups and collections</Description>
 
      <Body>
 
         <VRE name="/d4science.research-infrastructures.eu/Ecosystem/DRIVER">
 
            <collectionsGroup description="" id="ext-gen319" name="DRIVER" reference="">
 
               <collection creationDate="04-02-2011 06:48:17" description="DRIVER Collection" id="4bc54b10-9661-11e0-b316-9c541a3a1781" name="DRIVER Collection" recno="-1" reference="" />
 
            </collectionsGroup>
 
         </VRE>
 
      </Body>
 
   </Profile>
 
</Resource>

Note:

Generic Reources are cached so as to avoid multiple calls to IS in order to retrieve the same resource many times.

Search Extension - Getting available Collections & Search Queries

In order to retrieve the available collections regarding the active VRE, or to retrieve a search query, you have to use the SearchHelper class.

The collections are returned in a hierarchical way. A HashMap contains the collections that are organized as level 2 tree. Each key of the map containes the id of a collection group that assembles collections logically packed together. The value of each key in the hash map is an array list with the collections belonging to that group.

By using SearchHelper class you can also retrieve the available languages supported by the collections in addition to the searchable and presentable fields that each of them offers. For each field offered by a collection, the languages, the presentable fields and the index capabilities are stored in the model of the internal architecture representing it.

How to retrieve available collections, search query and search fields per metadata schema:

ASLSession aslSession = SessionManager.getInstance().getASLSession(session.getId(), username);
 
 
SearchHelper searchH = new SearchHelper(aslSession);
// Retrieving the available collections:
HashMap<org.gcube.application.framework.search.library.model.CollectionInfo, ArrayList<org.gcube.application.framework.search.library.model.CollectionInfo>> colls = searchH.getAvailableCollections();
if(colls != null)
{
	for (org.gcube.application.framework.search.library.model.CollectionInfo key:collectionInfos.keySet()) {
		ArrayList<org.gcube.application.framework.search.library.model.CollectionInfo> cols = collectionInfos.get(key);
		for (int i = 0; i < cols.size(); i++) {
			org.gcube.application.framework.search.library.model.CollectionInfo col = cols.get(i);
			System.out.println("The collection id is: " + col.getId() + ", the collection name is: " + col.getName() " and the number of contained documents is: " + col.getRecNo());
		}
	}
}
 
 
/** The queries are organized in "query groups". 
 * A query group actually contains the main query, but more queries can be added later
 * to the group that are corresponding to the initial query. For example when a user submits
 * a query and chooses to perform the search "by collection", then a set of queries will be
 * performed, one for every different collection. All those queries will belong to the same
 * query group. When asking for the "default" query, you actually get the first (and maybe only)
 * query of the group */
 
// Retrieving the first query (from a set of open queries):
// Get the current query group
QueryGroup queryGroup = sHelper.getActiveQueryGroup();
// Get the first query of the current group. 
Query queryObj = queryGroup.getQuery(DefaultQuery);

Needed imports:

import org.gcube.application.framework.search.library.impl.SearchHelper;
org.gcube.application.framework.search.library.model.CollectionInfo

Search Extension - Building & Submitting a Search Query

Once having created or obtained the query object you can form or update it, performing operations on the selected collections, the criteria set by the user, the selected schema, the condition type and the sorting order.

ASLSession aslSession = SessionManager.getInstance().getASLSession(session.getId(), username);
SearchHelper sHelper = new SearchHelper(aslSession);
 
 
/* Refresh the information for the available collections */
 
// The information about the collections is retrieved directly from the gCube system
shelper.refreshAvailableCollections();
 
 
// Get the current query group
QueryGroup queryGroup = sHelper.getActiveQueryGroup();
// Get the first query of the current group. 
Query queryObj = queryGroup.getQuery(DefaultQuery);
 
/* Add new collections to the selected ones */
/* Set second parameter to true if you want to select the collections or false if you want to deselect them */
queryObj.selectCollections(collections, true, session);
 
 
/* Add / remove a new criterion to the query */
 
queryObj.addCriterion(new Criterion(searchFieldId, searchFieldName, searchFieldValue));
queryObj.removeCriterion(searchFieldIntegerNo)
 
/* Reset all the current criteria */
queryObj.reset();
 
/* Get the selected real collections (the selected collections without the collection groups) */
List<String>selectedCollections = queryObj.getSelectedRealCollections();
 
 
/* Get / set the current selected condition type */
queryObj.setOperator(Operator.AND); //or Operator.OR
Operator op = queryObj.getOperator();
 
/* Get the Languages for the selected collections */
List<String> languages = queryObj.getAvailableLanguages();
 
/* Get the fields for the selected collections */
List<SearchField> sFields = queryObj.getAvailableSearchFields();  // searchable fields
List<String> browsableFields = queryObj.getAvailableBrowseFields(); // browsable fields
List<String> sortFields = queryObj.getAvailableSortFields();  //sortable fields
 
 
/* Update a crierion */
List<Criterion> criteria = queryObj.getCriteria();
queryObj.updateCriterionName(internalNo, newName);  //update a criterion name
queryObj.updateCriterionValue(internalNo, newValue); //update a criterion value
queryObj.updateCriterionName(internalNo, newId); //update a criterion id
 
/* See available indexes */
boolean hasFts = queryObj.isFTSAvailable();
boolean hasGeo = queryObj.isGeoAvailable();
 
/* Get Query Description - in User friendly language */
String qDescription = queryObj.getQueryDescription();
 
/* Get Query in CQL */
String qCQL = queryObj.getQueryString();