DJRA2.1 Report on Scenario Specific Technology Development
Contents
Introduction
The content of this report captures the development status of all software components that are direct or indirect consequence of the work performed by the Joint Research Activity 2 (JRA2) work package of the D4ScienceII project.
The objective of this work package is to cover the design and implementation case-specific needs for each particular interoperable infrastructure. Each case is analyzed and the system to implement it, on the side of the interoperating infrastructure, is designed in detail. Following the initial design, each case implements the specific components required so that it can exploit the facilities offered by the evolving ecosystem core and it can export the identified interoperable features to the rest of the ecosystem cases.
The cases involved in these scenarios are the ones defined by the high level needs and requirements exposed by the one of the following infrastructures and have been specialized through the NA4 and NA5 process. These interoperability cases through this process evolved to specific implementation requirements whose definitions and status of implementation is reported in this document. The implementation is performed in the context of the following tasks:
- INSPIRE specific implementation
- DRIVER specific implementation
- AquaMaps specific implementation
- FCPPS specific implementation
- ICIS specific implementation
This document reports on the implementation progress made in the context of these tasks. Progress is reported both in the context of each individual task as well as in the direction of ecosystem interoperability. The report groups related functionalities (from an implementation point of view), with respect to their impact in components that are involved for their provision. Thus, the status of development is presented by functional group, whereas this can be the result of the combined status of various elements involved in a particular functional scope. Furthermore, as a single component might expose more than a single functional facet, it might be implicitly or explicitly involved in several functional groups.
As a result of a requirement (or an entire requirement group), development can be aiming to:
- An entirely new software suite (Service including libraries and user interface elements if applicable) for extending gCube potential in a new domain.
- A minor new component (of any of the aforementioned types) that plugs into the infrastructure (or to a particular existing component) in well defined areas, adding to its behavior.
- Modification/extension of existing components, for provision of new functionality or modification/improvement of the existing one.
In this stage of D4ScienceII project, the status of a component can be any of the following:
- Canceled (components that their design and implementation has been canceled due to shift of interest)
- Design (components that are in the stage of design and the best implementation approach is yet to be decided)
- Prototypes (components that are already designed and implemented as prototypes but have not been delivered yet as stable versions).
- Delivered (components that have been completely delivered, i.e. are deployable on the infrastructure, but further refinements are performed)
- Completed (delivered components that their formal development cycle is finished and no updates are planned for next versions)
A further clarification is given with respect to development so as to make evident which is the degree to which a component can be included in a production infrastructure in its current state.
It is important to note that a component can be a product of joint community requirements, and as such it is co-referenced in the respective sections. In this report, a clear reference to the request originator is made wherever suited, with a potential clarification on whether this is a “diffusion” adoption or an original request. In the former case a reference is made in the respective section to detail the role of the component in the collaboration among ecosystems.
Functionality Status Sum-up
This section contains definition and reporting information on the functionalities implemented in the context of JRA2 as well as the involved components, be it existing, modified or new. If the list becomes to big to be hosted in a single page, each one of the sub sections can be moved to a new page.
INSPIRE
| Functionality | Related TRACked Requirements | Support | Diffusion | Status | Stage Completion (%) | Completion Date | Production Release Date |
| In-document reference to detailed description | Track URI | List of partners involved | List of in-document references to related functionalities where this component is diffused | The current status of the component | Percentage completed | The envisaged or actual completion date | The envisaged or actual release date |
| DAG-JDL Job Execution on gCube nodes | Track URI | NKUA | List of in-document references to related functionalities where this component is diffused | Prototype | 50% | N/A | N/A |
| Grid Job Execution on gLite Grid | Track URI | NKUA | List of in-document references to related functionalities where this component is diffused | Prototype | 50% | N/A | N/A |
| Map-Reduce modeled execution | Track URI | NKUA | List of in-document references to related functionalities where this component is diffused | Prototype | 50% | N/A | N/A |
| Access of OpenSearch interface on INSPIRE content | Track URI | NKUA | List of in-document references to related functionalities where this component is diffused | Prototype | 20% | N/A | N/A |
Functionality details
Requirements
The requirements that raised the need for this functionality
Functionalities
The functionalities that are expected to be provided
Status
The current status of the the overall functionality
Main Components Involved
The main components that either pre-exist, are extended or are completly new and are implemented in this context
More Information
Links to more information
DAG-JDL Job Execution on gCube nodes
Requirements
The requirement raised to necessitate the development of this component was the need of execution arbitrary, validated, of various technologies in the gCube infrastructure
Functionalities
This adaptor as part of the adaptors offered by the WorkflowEngine constructs an Execution Plan based on the description of a job defined in JDL syntax. This description can be of a single job or it can include a Directed Acyclic Graph (DAG) of jobs. This adaptor parses a Job Description Language (JDL) definition block and translates the described job or DAG of jobs into an Execution Plan which can be submitted to the ExecutionEngine for execution.
Status
Prototype
Main Components Involved
More Information
Grid Job Execution on gLite Grid
Requirements
The requirement raised to necessitate the development of this component was the need of forwarding execution of resource intensive jobs to the gLite grid
Functionalities
This adaptor as part of the adaptors offered by the WorkflowEngine constructs an Execution Plan that can mediate to submit a job described through a JDL file using a gLite Grid UI node. After its submission the job is monitored for its status and the final output of the job is retrieved.
Status
Prototype
Main Components Involved
More Information
Map-Reduce modeled execution
Requirements
The requirement raised to necessitate the development of this component was the need of supporting the Map Reduce paradigm and offer an effective environment for the jobs to operate on
Functionalities
This adaptor as part of the adaptors offered by the WorkflowEngine constructs an Execution Plan that can mediate to submit a job writen under the Map Reduce design pattern and written against the utilities offered by the Hadoop infrastructure. This adaptor constructs an Execution Plan that can contact a Hadoop UI node, submit, monitor and retrieve the output of a Map Reduce job. The Hadoop infrastructure utilized resides in a cloud infrastructure with which the ExecutionEngine negotiates the resource availability.
Status
Prototype
Main Components Involved
More Information
Links to more information
Access OpenSearch Inspire
Requirements
The requirement raised to necessitate the development of this component was the need of integrating content provided from INSPIRE platform though OpenSearch interface
Functionalities
This Operator will provide core functionality to searching INpsire content through an OpenSearch interface provided from the INSPIRE platform
Status
Prototype
Main Components Involved
Page not Available
More Information
Links to more information
- Page not Available
DRIVER
| Functionality | Related TRACked Requirements | Support | Diffusion | Status | Stage Completion (%) | Completion Date | Production Release Date |
| In-document reference to detailed description | Track URI | List of partners involved | List of in-document references to related functionalities where this component is diffused | The current status of the component | Percentage completed | The envisaged or actual completion date | The envisaged or actual release date |
Functionality details
Requirements
The requirements that raised the need for this functionality
Functionalities
The functionalities that are expected to be provided
Status
The current status of the the overall functionality
Main Components Involved
The main components that either pre-exist, are extended or are completly new and are implemented in this context
More Information
Links to more information
AquaMaps
| Functionality | Related TRACked Requirements | Support | Diffusion | Status | Stage Completion (%) | Completion Date | Production Release Date |
| In-document reference to detailed description | Track URI | List of partners involved | List of in-document references to related functionalities where this component is diffused | The current status of the component | Percentage completed | The envisaged or actual completion date | The envisaged or actual release date |
Functionality details
Requirements
The requirements that raised the need for this functionality
Functionalities
The functionalities that are expected to be provided
Status
The current status of the the overall functionality
Main Components Involved
The main components that either pre-exist, are extended or are completly new and are implemented in this context
More Information
Links to more information
FCPPS
| Functionality | Related TRACked Requirements | Support | Diffusion | Status | Stage Completion (%) | Completion Date | Production Release Date |
| In-document reference to detailed description | Track URI | List of partners involved | List of in-document references to related functionalities where this component is diffused | The current status of the component | Percentage completed | The envisaged or actual completion date | The envisaged or actual release date |
Functionality details
Requirements
The requirements that raised the need for this functionality
Functionalities
The functionalities that are expected to be provided
Status
The current status of the the overall functionality
Main Components Involved
The main components that either pre-exist, are extended or are completly new and are implemented in this context
More Information
Links to more information
ICIS
| Functionality | Related TRACked Requirements | Support | Diffusion | Status | Stage Completion (%) | Completion Date | Production Release Date |
| In-document reference to detailed description | Track URI | List of partners involved | List of in-document references to related functionalities where this component is diffused | The current status of the component | Percentage completed | The envisaged or actual completion date | The envisaged or actual release date |
Functionality details
Requirements
The requirements that raised the need for this functionality
Functionalities
The functionalities that are expected to be provided
Status
The current status of the the overall functionality
Main Components Involved
The main components that either pre-exist, are extended or are completly new and are implemented in this context
More Information
Links to more information
Diffusion
This section summarizes the interoperability resolving implementation by coupling the described functionality requirements. Although this section introduces a level of repetition, since some of the information detailed in previous section of the document, it serves also as a validation index for the implemented work and its deadlines. Additionally, the status and completion percentage reported in this section is targeting the integration of functionalities which may differ from the ones described in the above sections. This is mainly due to the fact that a functionality requirement may need in order to be satisfied not only integration with other ecosystem modules, but also with platform components as well as external ones.
| Required Functionality | Provided Functionality | Integration Status | Integration Completion (%) | Integration Completion Date |
| In-document reference to detailed description of the required functionality and the partner requesting it | List of in-document references to detailed description of the provided functionalities and the partners providing it | The current status of the integration procedure | Integration procedure percentage completed | The envisaged or actual integration procedure completion date |
