Difference between revisions of "DJRA1.1 Report on Knowledge Ecosystem Supporting Technology Development"

From Gcube Wiki
Jump to: navigation, search
(Introduction)
(Code Statistics)
 
(54 intermediate revisions by 2 users not shown)
Line 5: Line 5:
  
 
== Introduction ==
 
== Introduction ==
The goals of  the work package ''JRA1 Knowledge Ecosystem Implementation'' are to provide the technology realising the common practices, standards and solutions identified by the NA4 work package and enhance and consolidate the gCube system to match the needs of the ecosystem approach. A primary role of JRA1 (played within ''TJRA1.2'') is to address the various interoperability issues inherent in the construction of a knowledge ecosystem, and then identify and implement proper solutions. Ranging from content-related problems to architecture-oriented issues. Major inputs for this part of the work come from [https://networking.wiki.d4science-ii.research-infrastructures.eu/networking/index.php/Interoperability_Solutions '''DNA4.1'''].
+
The goals of  the work package ''JRA1 Knowledge Ecosystem Implementation'' are ''(i)'' to provide the technology realising the common practices, standards and solutions identified by the NA4 work package and ''(ii)'' to enhance and consolidate the gCube system to match the needs of the ecosystem approach. A primary role of JRA1 (played within ''TJRA1.2'') is to address the various interoperability issues -- ranging from content-related problems to architecture-oriented issues -- inherent in the construction of a knowledge ecosystem, and then identify and implement proper solutions. Major inputs for this part of the work come from [https://networking.wiki.d4science-ii.research-infrastructures.eu/networking/index.php/Interoperability_Solutions '''DNA4.1'''].
  
An ''Agilish'' development methodology (referring to no specific Agile development platforms, but to the general idea of quickly responding to change) is driving the teams in JRA1. Plans are sketched and verified with the management boards and user communities; ''seeing and playing with the system'' being the most important thing and preferred to over-planning.
+
An ''Agilish'' development methodology (referring to no specific Agile development platforms, but to the general idea of quickly responding to change) is driving the teams in JRA1. Plans are sketched and verified with the management boards and user communities; ''seeing and playing with the system'' being the most important thing and is preferred to over-planning.
  
 
This deliverable documents the gCube technology by reporting on the overall status of the system architecture, the per service tasks and enhancements, and other technical details needed to have a comprehensive understanding of the developed knowledge ecosystem enabling technology.  
 
This deliverable documents the gCube technology by reporting on the overall status of the system architecture, the per service tasks and enhancements, and other technical details needed to have a comprehensive understanding of the developed knowledge ecosystem enabling technology.  
Line 15: Line 15:
 
== State of Software Development Activities ==
 
== State of Software Development Activities ==
  
This section reports on the current maintenance and upgrade activities on the gCube system.  
+
This section reports on the current maintenance and upgrade activities performed and to be performed on the gCube system.  
  
To have a living picture of the continuous evolution of the system, the project's [https://issue.d4science-ii.research-infrastructures.eu/ TRAC instance] has been properly exploited in the following manner:
+
Traditional approaches to planning have proved to do not fit Agile methodology. The fast moving forward of the software makes obsolete every plan very quickly and huge documentation should be continuously updated with a considerable waste of effort. The innovative approach followed by D4ScienceII is based upon a proper exploitation of the project's [https://issue.d4science-ii.research-infrastructures.eu/ '''TRAC instance''']:
  
* firstly, four dedicated milestones have been defined:
+
* firstly, four '''''dedicated milestones''''' were defined:
  
 
:* ''Short-Term gCube plan'', used when the activities described in a ticket are planned to be completed within the next month;
 
:* ''Short-Term gCube plan'', used when the activities described in a ticket are planned to be completed within the next month;
  
:* ''Medium-Term gCube plan'' , used when the activities described in a ticket are planned to be completed within the next 3 months;
+
:* ''Medium-Term gCube plan'' , used when the activities described in a ticket are planned to be completed within the next 3 months;
  
:* ''Long-Term gCube plan'', , used when the activities described in a ticket do not have a starting and ending date yet;
+
:* ''Long-Term gCube plan'', used when the activities described in a ticket do not have a starting and ending date yet;
  
:* ''Ready To Release'', used when the activities described in a ticket are completed and passed to SA.
+
:* ''Ready To Release'', used when the activities described in a ticket are completed and the resulting software is passed to SA.
  
 
: The idea behind these milestones is that a developer moves his/her tickets among them according to the work performed or to be performed on each ticket;
 
: The idea behind these milestones is that a developer moves his/her tickets among them according to the work performed or to be performed on each ticket;
Line 33: Line 33:
 
* then, JRA1 management asked to each developer to insert his/her own tasks and give an estimation of delivery by assigning one of the milestones above;
 
* then, JRA1 management asked to each developer to insert his/her own tasks and give an estimation of delivery by assigning one of the milestones above;
  
* finally, in order to stand over the activities, four TRAC's living reports have been created for this purpose. These reports are introduced in the following sub-sections.
+
* finally, in order to stand over the activities, four TRAC's '''''living reports''''' have been created for this purpose. These reports are introduced in the following sub-sections.
 +
 
 +
This flexible strategy gives the opportunity to have a living picture of the continuous evolution of the system. The reports are automatically updated each time a change occurs in a ticket or whenever a new ticket is submitted.  
  
 
=== Upgrade Activities ===
 
=== Upgrade Activities ===
  
Upgrade activities describe tasks and enhancements that lead to future releases of the gCube technology.
+
Upgrade activities refer to tasks and enhancements that lead to future releases of the gCube technology:
  
The planned actions are recorded in [https://issue.d4science-ii.research-infrastructures.eu/report/37 report #37].
+
: → the '''''planned upgrade activities''''' are recorded in [https://issue.d4science-ii.research-infrastructures.eu/report/37 '''report #37''']
  
The closed actions are recorded in [https://issue.d4science-ii.research-infrastructures.eu/report/38 report #38]  
+
: → the '''''completed upgrade activities''''' are recorded in [https://issue.d4science-ii.research-infrastructures.eu/report/38 '''report #38''']
  
 
=== Maintenance Activities ===
 
=== Maintenance Activities ===
  
Maintenance activities are dedicated to fix defects found in the technology delivered so far and the related documentation.
+
Maintenance activities are dedicated to fix defects found in the technology delivered so far and the related documentation:
  
The open maintenance activities are recorded in [https://issue.d4science-ii.research-infrastructures.eu/report/39 report #39].
+
: →  the '''''planned maintenance activities''''' are recorded in [https://issue.d4science-ii.research-infrastructures.eu/report/39 '''report #39''']
  
The closed maintenance activities are recorded in [https://issue.d4science-ii.research-infrastructures.eu/report/40 report #40].
+
: →  the '''''completed maintenance activities''''' are recorded in [https://issue.d4science-ii.research-infrastructures.eu/report/40 '''report #40''']
  
== Code Metrics ==
 
''This section will describe and link the [https://networking.wiki.d4science-ii.research-infrastructures.eu/networking/index.php/QATF_Indicators#JRA_gCube_Development JRA Indicators]''
 
  
[TBC]
+
In addition to the development activities captured by the reports above, other stems from incidents and malfunctions discovered while exploiting the technology in the production infrastructure. They are captured by two specific living reports, '''current incidents''' ([https://support.d4science-ii.research-infrastructures.eu/report/1 '''Report #1''']) and '''resolved incidents''' ([https://support.d4science-ii.research-infrastructures.eu/report/2 '''Report #2'''] and [https://issue.d4science-ii.research-infrastructures.eu/report/11 '''Report #11''']). Yet, these reports include both JRA1 and JRA2 incidents.
 +
 
 +
== Software Metrics ==
 +
Software metrics are a way to measure the software in order to provide managers and developers better insight into the code they are releasing. By taking advantage of metrics:
 +
* managers identify potential risks, understand the current state of a project, and track progress during software development
 +
* developers can understand which components should be reworked or more thoroughly tested
 +
 
 +
Therefore, monthly, starting from M4 (the first month of JRA), a set of software metrics are calculated and collected in the [https://networking.wiki.d4science-ii.research-infrastructures.eu/networking/index.php/QATF_Indicators#JRA_gCube_Development '''Indicators page'''] (Login is required).
 +
 
 +
The selected indicators are intended to give a measure of the quality of the code produced along the project.
  
 
== Code Statistics ==
 
== Code Statistics ==
''This section will report various tables and charts describing the project development.''
 
  
[TBC]
+
D4ScienceII uses [http://www.statsvn.org/ '''StatSVN'''] to retrieve information from the project ''source code repository'' and to generate statistics describing the project development. The project ''source code repository'' consists of two separated [http://subversion.tigris.org/ '''subversion'''] instances: one instance is dedicated to the ''gCube'' components, while the other one is specific for the ''gCore'' development stream.
 +
 
 +
Both of them have been operative since March 2008, thus the produced data documents the evolution of the software from such period on.
 +
 
 +
The following reports, tables and charts provide statistics about the development activity within D4ScienceII:
 +
 
 +
{| align="center" width="80%"
 +
! bgcolor="gray"|gCube !! bgcolor="gray"|gCore
 +
|-
 +
| align="center"| [http://issue.d4science-ii.research-infrastructures.eu/chrome/site/statsvn-static/loc.html '''Timeline for the Lines of Code (LOC)''']  || align="center"| [https://issue.gcore.research-infrastructures.eu/chrome/site/statsvn/loc.html '''Timeline for the Lines of Code (LOC)''']
 +
|-
 +
| align="center"| [http://issue.d4science-ii.research-infrastructures.eu/chrome/site/statsvn-static/developers.html '''Lines of Code per Developer''']  || align="center"| [https://issue.gcore.research-infrastructures.eu/chrome/site/statsvn/developers.html '''Lines of Code per Developer''']
 +
|-
 +
| align="center"| [http://issue.d4science-ii.research-infrastructures.eu/chrome/site/statsvn-static/churn.html '''LOC Evolution and Changes per Day'''] || align="center"| [https://issue.gcore.research-infrastructures.eu/chrome/site/statsvn/churn.html '''LOC Evolution and Changes per Day''']
 +
|-
 +
| align="center"| [http://issue.d4science-ii.research-infrastructures.eu/chrome/site/statsvn-static/repomap.html '''Software Repository Hierarchical View'''] || align="center"| [https://issue.gcore.research-infrastructures.eu/chrome/site/statsvn/repomap.html '''Software Repository Hierarchical View''']
 +
|}
  
 
== Documentation ==
 
== Documentation ==
''This section will describe the work on the Development and Administrator's Guide''
+
The software delivered is properly documented by the Developer's and Administrator's Guides. These two guides represent the main source of information for whoever has to technically deal with the gCube system and they run deeply at single component level.
  
* [https://gcube.wiki.gcube-system.org/gcube/index.php/Developer%27s_Guide Developer's Guide]
+
* [https://gcube.wiki.gcube-system.org/gcube/index.php/Developer%27s_Guide '''Developer's Guide''']: this guide targets two audiences:
 +
:* developers aiming at producing software that interacts with a gCube components;
 +
:* developers that take over a component from another one and start a new development stream.
  
* [https://gcube.wiki.gcube-system.org/gcube/index.php/Administrator%27s_Guide Administrator's Guide]
+
* [https://gcube.wiki.gcube-system.org/gcube/index.php/Administrator%27s_Guide '''Administrator's Guide''']: this guide mainly targets Site Managers for installing gCube nodes and deploying components. However, it can be used also by developers to have local instances of the components they interact with.
  
 
== Software ==
 
== Software ==
  
 
=== Development Version ===
 
=== Development Version ===
D4ScienceII relies on the [http://etics.web.cern.ch/etics ETICS] system to automate the way its software (gCube and gCore) is built and tested. In particular, the project has put in place mechanisms to perform ''daily builds'' of the latest version of the code committed in the project source code repository. This activity leads to the production of (''i'') a report of the build activity and (''ii'') a set of software artifacts including the software package, the source code and the documentation.  
+
D4ScienceII relies on the [http://etics.web.cern.ch/etics '''ETICS'''] system to automate the building and testing phases of the software. In particular, the project has put in place mechanisms to perform ''daily builds'' of the latest version of the code committed in the project source code repository. This activity leads to the production of (''i'') a report of the build activity and (''ii'') a set of software artifacts including the software package, the source code and the documentation.  
  
 
JRA1 is responsible for managing the development version (namely HEAD) of these reports and artifacts whose main purpose is to provide a way to
 
JRA1 is responsible for managing the development version (namely HEAD) of these reports and artifacts whose main purpose is to provide a way to
Line 78: Line 104:
  
  
{| align="center" width="50%"  
+
{| align="center" width="80%"  
! bgcolor="yellow"|gCube !! bgcolor="yellow"|gCore
+
! bgcolor="gray"|gCube !! bgcolor="gray"|gCore
 
|-
 
|-
| align="center"|[http://grids16.eng.it/BuildReport/browse/Recent_Builds/org.gcube.HEAD Build Report] || align="center"|[http://grids16.eng.it/BuildReport/browse/Recent_Builds/org.gcore.HEAD Build Report]
+
| align="center"|[http://grids16.eng.it/BuildReport/browse/Recent_Builds/org.gcube.HEAD '''Build Reports'''] || align="center"|[http://grids16.eng.it/BuildReport/browse/Recent_Builds/org.gcore.HEAD '''Build Reports''']
 
|-
 
|-
| align="center"|[http://www.gcube-system.org/index.php?option=com_distribution&view=distribution&Itemid=23&release=HEAD Development Version] (Login is required) || align="center"|[http://www.gcube-system.org/index.php?option=com_content&view=article&id=85&Itemid=47 Latest Version] (Login is required)
+
| align="center"|[http://www.gcube-system.org/index.php?option=com_distribution&view=distribution&Itemid=23&release=HEAD '''Development Version'''] (Login is required) || align="center"|[http://www.gcube-system.org/index.php?option=com_content&view=article&id=85&Itemid=47 '''Latest Version'''] (Login is required)
 
|}
 
|}
 
 
[TBC]
 

Latest revision as of 15:41, 3 November 2011


Introduction

The goals of the work package JRA1 Knowledge Ecosystem Implementation are (i) to provide the technology realising the common practices, standards and solutions identified by the NA4 work package and (ii) to enhance and consolidate the gCube system to match the needs of the ecosystem approach. A primary role of JRA1 (played within TJRA1.2) is to address the various interoperability issues -- ranging from content-related problems to architecture-oriented issues -- inherent in the construction of a knowledge ecosystem, and then identify and implement proper solutions. Major inputs for this part of the work come from DNA4.1.

An Agilish development methodology (referring to no specific Agile development platforms, but to the general idea of quickly responding to change) is driving the teams in JRA1. Plans are sketched and verified with the management boards and user communities; seeing and playing with the system being the most important thing and is preferred to over-planning.

This deliverable documents the gCube technology by reporting on the overall status of the system architecture, the per service tasks and enhancements, and other technical details needed to have a comprehensive understanding of the developed knowledge ecosystem enabling technology.

The deliverable is implemented through a Wiki page to be easily and promptly adapted to reflect the ever-evolving status of the developed technologies. For the same reasons, performed and planned activities are reported via dedicated TRAC reports, created for this very purpose.

State of Software Development Activities

This section reports on the current maintenance and upgrade activities performed and to be performed on the gCube system.

Traditional approaches to planning have proved to do not fit Agile methodology. The fast moving forward of the software makes obsolete every plan very quickly and huge documentation should be continuously updated with a considerable waste of effort. The innovative approach followed by D4ScienceII is based upon a proper exploitation of the project's TRAC instance:

  • firstly, four dedicated milestones were defined:
  • Short-Term gCube plan, used when the activities described in a ticket are planned to be completed within the next month;
  • Medium-Term gCube plan , used when the activities described in a ticket are planned to be completed within the next 3 months;
  • Long-Term gCube plan, used when the activities described in a ticket do not have a starting and ending date yet;
  • Ready To Release, used when the activities described in a ticket are completed and the resulting software is passed to SA.
The idea behind these milestones is that a developer moves his/her tickets among them according to the work performed or to be performed on each ticket;
  • then, JRA1 management asked to each developer to insert his/her own tasks and give an estimation of delivery by assigning one of the milestones above;
  • finally, in order to stand over the activities, four TRAC's living reports have been created for this purpose. These reports are introduced in the following sub-sections.

This flexible strategy gives the opportunity to have a living picture of the continuous evolution of the system. The reports are automatically updated each time a change occurs in a ticket or whenever a new ticket is submitted.

Upgrade Activities

Upgrade activities refer to tasks and enhancements that lead to future releases of the gCube technology:

→ the planned upgrade activities are recorded in report #37
→ the completed upgrade activities are recorded in report #38

Maintenance Activities

Maintenance activities are dedicated to fix defects found in the technology delivered so far and the related documentation:

→ the planned maintenance activities are recorded in report #39
→ the completed maintenance activities are recorded in report #40


In addition to the development activities captured by the reports above, other stems from incidents and malfunctions discovered while exploiting the technology in the production infrastructure. They are captured by two specific living reports, current incidents (Report #1) and resolved incidents (Report #2 and Report #11). Yet, these reports include both JRA1 and JRA2 incidents.

Software Metrics

Software metrics are a way to measure the software in order to provide managers and developers better insight into the code they are releasing. By taking advantage of metrics:

  • managers identify potential risks, understand the current state of a project, and track progress during software development
  • developers can understand which components should be reworked or more thoroughly tested

Therefore, monthly, starting from M4 (the first month of JRA), a set of software metrics are calculated and collected in the Indicators page (Login is required).

The selected indicators are intended to give a measure of the quality of the code produced along the project.

Code Statistics

D4ScienceII uses StatSVN to retrieve information from the project source code repository and to generate statistics describing the project development. The project source code repository consists of two separated subversion instances: one instance is dedicated to the gCube components, while the other one is specific for the gCore development stream.

Both of them have been operative since March 2008, thus the produced data documents the evolution of the software from such period on.

The following reports, tables and charts provide statistics about the development activity within D4ScienceII:

gCube gCore
Timeline for the Lines of Code (LOC) Timeline for the Lines of Code (LOC)
Lines of Code per Developer Lines of Code per Developer
LOC Evolution and Changes per Day LOC Evolution and Changes per Day
Software Repository Hierarchical View Software Repository Hierarchical View

Documentation

The software delivered is properly documented by the Developer's and Administrator's Guides. These two guides represent the main source of information for whoever has to technically deal with the gCube system and they run deeply at single component level.

  • developers aiming at producing software that interacts with a gCube components;
  • developers that take over a component from another one and start a new development stream.
  • Administrator's Guide: this guide mainly targets Site Managers for installing gCube nodes and deploying components. However, it can be used also by developers to have local instances of the components they interact with.

Software

Development Version

D4ScienceII relies on the ETICS system to automate the building and testing phases of the software. In particular, the project has put in place mechanisms to perform daily builds of the latest version of the code committed in the project source code repository. This activity leads to the production of (i) a report of the build activity and (ii) a set of software artifacts including the software package, the source code and the documentation.

JRA1 is responsible for managing the development version (namely HEAD) of these reports and artifacts whose main purpose is to provide a way to

  • verify the dependencies among the artifacts
  • exchange artifacts among the development teams


gCube gCore
Build Reports Build Reports
Development Version (Login is required) Latest Version (Login is required)