Difference between revisions of "DJRA1.1 Report on Knowledge Ecosystem Supporting Technology Development"
Manuele.simi (Talk | contribs) |
(→Code Statistics) |
||
(71 intermediate revisions by 2 users not shown) | |||
Line 5: | Line 5: | ||
== Introduction == | == Introduction == | ||
+ | The goals of the work package ''JRA1 Knowledge Ecosystem Implementation'' are ''(i)'' to provide the technology realising the common practices, standards and solutions identified by the NA4 work package and ''(ii)'' to enhance and consolidate the gCube system to match the needs of the ecosystem approach. A primary role of JRA1 (played within ''TJRA1.2'') is to address the various interoperability issues -- ranging from content-related problems to architecture-oriented issues -- inherent in the construction of a knowledge ecosystem, and then identify and implement proper solutions. Major inputs for this part of the work come from [https://networking.wiki.d4science-ii.research-infrastructures.eu/networking/index.php/Interoperability_Solutions '''DNA4.1''']. | ||
− | + | An ''Agilish'' development methodology (referring to no specific Agile development platforms, but to the general idea of quickly responding to change) is driving the teams in JRA1. Plans are sketched and verified with the management boards and user communities; ''seeing and playing with the system'' being the most important thing and is preferred to over-planning. | |
− | + | This deliverable documents the gCube technology by reporting on the overall status of the system architecture, the per service tasks and enhancements, and other technical details needed to have a comprehensive understanding of the developed knowledge ecosystem enabling technology. | |
− | + | The deliverable is implemented through a Wiki page to be easily and promptly adapted to reflect the ever-evolving status of the developed technologies. For the same reasons, performed and planned activities are reported via dedicated TRAC reports, created for this very purpose. | |
− | + | == State of Software Development Activities == | |
− | + | This section reports on the current maintenance and upgrade activities performed and to be performed on the gCube system. | |
− | The | + | Traditional approaches to planning have proved to do not fit Agile methodology. The fast moving forward of the software makes obsolete every plan very quickly and huge documentation should be continuously updated with a considerable waste of effort. The innovative approach followed by D4ScienceII is based upon a proper exploitation of the project's [https://issue.d4science-ii.research-infrastructures.eu/ '''TRAC instance''']: |
− | + | * firstly, four '''''dedicated milestones''''' were defined: | |
− | + | :* ''Short-Term gCube plan'', used when the activities described in a ticket are planned to be completed within the next month; | |
− | + | :* ''Medium-Term gCube plan'' , used when the activities described in a ticket are planned to be completed within the next 3 months; | |
− | + | :* ''Long-Term gCube plan'', used when the activities described in a ticket do not have a starting and ending date yet; | |
− | '' | + | |
− | [ | + | :* ''Ready To Release'', used when the activities described in a ticket are completed and the resulting software is passed to SA. |
+ | |||
+ | : The idea behind these milestones is that a developer moves his/her tickets among them according to the work performed or to be performed on each ticket; | ||
+ | |||
+ | * then, JRA1 management asked to each developer to insert his/her own tasks and give an estimation of delivery by assigning one of the milestones above; | ||
+ | |||
+ | * finally, in order to stand over the activities, four TRAC's '''''living reports''''' have been created for this purpose. These reports are introduced in the following sub-sections. | ||
+ | |||
+ | This flexible strategy gives the opportunity to have a living picture of the continuous evolution of the system. The reports are automatically updated each time a change occurs in a ticket or whenever a new ticket is submitted. | ||
+ | |||
+ | === Upgrade Activities === | ||
+ | |||
+ | Upgrade activities refer to tasks and enhancements that lead to future releases of the gCube technology: | ||
+ | |||
+ | : → the '''''planned upgrade activities''''' are recorded in [https://issue.d4science-ii.research-infrastructures.eu/report/37 '''report #37'''] | ||
+ | |||
+ | : → the '''''completed upgrade activities''''' are recorded in [https://issue.d4science-ii.research-infrastructures.eu/report/38 '''report #38'''] | ||
+ | |||
+ | === Maintenance Activities === | ||
+ | |||
+ | Maintenance activities are dedicated to fix defects found in the technology delivered so far and the related documentation: | ||
+ | |||
+ | : → the '''''planned maintenance activities''''' are recorded in [https://issue.d4science-ii.research-infrastructures.eu/report/39 '''report #39'''] | ||
+ | |||
+ | : → the '''''completed maintenance activities''''' are recorded in [https://issue.d4science-ii.research-infrastructures.eu/report/40 '''report #40'''] | ||
+ | |||
+ | |||
+ | In addition to the development activities captured by the reports above, other stems from incidents and malfunctions discovered while exploiting the technology in the production infrastructure. They are captured by two specific living reports, '''current incidents''' ([https://support.d4science-ii.research-infrastructures.eu/report/1 '''Report #1''']) and '''resolved incidents''' ([https://support.d4science-ii.research-infrastructures.eu/report/2 '''Report #2'''] and [https://issue.d4science-ii.research-infrastructures.eu/report/11 '''Report #11''']). Yet, these reports include both JRA1 and JRA2 incidents. | ||
+ | |||
+ | == Software Metrics == | ||
+ | Software metrics are a way to measure the software in order to provide managers and developers better insight into the code they are releasing. By taking advantage of metrics: | ||
+ | * managers identify potential risks, understand the current state of a project, and track progress during software development | ||
+ | * developers can understand which components should be reworked or more thoroughly tested | ||
+ | |||
+ | Therefore, monthly, starting from M4 (the first month of JRA), a set of software metrics are calculated and collected in the [https://networking.wiki.d4science-ii.research-infrastructures.eu/networking/index.php/QATF_Indicators#JRA_gCube_Development '''Indicators page'''] (Login is required). | ||
+ | |||
+ | The selected indicators are intended to give a measure of the quality of the code produced along the project. | ||
== Code Statistics == | == Code Statistics == | ||
− | |||
− | [ | + | D4ScienceII uses [http://www.statsvn.org/ '''StatSVN'''] to retrieve information from the project ''source code repository'' and to generate statistics describing the project development. The project ''source code repository'' consists of two separated [http://subversion.tigris.org/ '''subversion'''] instances: one instance is dedicated to the ''gCube'' components, while the other one is specific for the ''gCore'' development stream. |
+ | |||
+ | Both of them have been operative since March 2008, thus the produced data documents the evolution of the software from such period on. | ||
+ | |||
+ | The following reports, tables and charts provide statistics about the development activity within D4ScienceII: | ||
+ | |||
+ | {| align="center" width="80%" | ||
+ | ! bgcolor="gray"|gCube !! bgcolor="gray"|gCore | ||
+ | |- | ||
+ | | align="center"| [http://issue.d4science-ii.research-infrastructures.eu/chrome/site/statsvn-static/loc.html '''Timeline for the Lines of Code (LOC)'''] || align="center"| [https://issue.gcore.research-infrastructures.eu/chrome/site/statsvn/loc.html '''Timeline for the Lines of Code (LOC)'''] | ||
+ | |- | ||
+ | | align="center"| [http://issue.d4science-ii.research-infrastructures.eu/chrome/site/statsvn-static/developers.html '''Lines of Code per Developer'''] || align="center"| [https://issue.gcore.research-infrastructures.eu/chrome/site/statsvn/developers.html '''Lines of Code per Developer'''] | ||
+ | |- | ||
+ | | align="center"| [http://issue.d4science-ii.research-infrastructures.eu/chrome/site/statsvn-static/churn.html '''LOC Evolution and Changes per Day'''] || align="center"| [https://issue.gcore.research-infrastructures.eu/chrome/site/statsvn/churn.html '''LOC Evolution and Changes per Day'''] | ||
+ | |- | ||
+ | | align="center"| [http://issue.d4science-ii.research-infrastructures.eu/chrome/site/statsvn-static/repomap.html '''Software Repository Hierarchical View'''] || align="center"| [https://issue.gcore.research-infrastructures.eu/chrome/site/statsvn/repomap.html '''Software Repository Hierarchical View'''] | ||
+ | |} | ||
+ | |||
+ | == Documentation == | ||
+ | The software delivered is properly documented by the Developer's and Administrator's Guides. These two guides represent the main source of information for whoever has to technically deal with the gCube system and they run deeply at single component level. | ||
+ | |||
+ | * [https://gcube.wiki.gcube-system.org/gcube/index.php/Developer%27s_Guide '''Developer's Guide''']: this guide targets two audiences: | ||
+ | :* developers aiming at producing software that interacts with a gCube components; | ||
+ | :* developers that take over a component from another one and start a new development stream. | ||
+ | |||
+ | * [https://gcube.wiki.gcube-system.org/gcube/index.php/Administrator%27s_Guide '''Administrator's Guide''']: this guide mainly targets Site Managers for installing gCube nodes and deploying components. However, it can be used also by developers to have local instances of the components they interact with. | ||
== Software == | == Software == | ||
=== Development Version === | === Development Version === | ||
− | D4ScienceII relies on the [http://etics.web.cern.ch/etics ETICS] system to automate the | + | D4ScienceII relies on the [http://etics.web.cern.ch/etics '''ETICS'''] system to automate the building and testing phases of the software. In particular, the project has put in place mechanisms to perform ''daily builds'' of the latest version of the code committed in the project source code repository. This activity leads to the production of (''i'') a report of the build activity and (''ii'') a set of software artifacts including the software package, the source code and the documentation. |
− | JRA1 is responsible for managing the development version (namely HEAD) of these reports and artifacts | + | JRA1 is responsible for managing the development version (namely HEAD) of these reports and artifacts whose main purpose is to provide a way to |
+ | * verify the dependencies among the artifacts | ||
+ | * exchange artifacts among the development teams | ||
− | {| align="center" width=" | + | |
− | ! bgcolor=" | + | {| align="center" width="80%" |
+ | ! bgcolor="gray"|gCube !! bgcolor="gray"|gCore | ||
|- | |- | ||
− | | align="center"|[http://grids16.eng.it/BuildReport/browse/Recent_Builds/org.gcube.HEAD Build | + | | align="center"|[http://grids16.eng.it/BuildReport/browse/Recent_Builds/org.gcube.HEAD '''Build Reports'''] || align="center"|[http://grids16.eng.it/BuildReport/browse/Recent_Builds/org.gcore.HEAD '''Build Reports'''] |
|- | |- | ||
− | | align="center"|[http://www.gcube-system.org/index.php?option=com_distribution&view=distribution&Itemid=23&release=HEAD Development Version] (Login is required) || align="center"|[http://www.gcube-system.org/index.php?option=com_content&view=article&id=85&Itemid=47 Latest Version] (Login is required) | + | | align="center"|[http://www.gcube-system.org/index.php?option=com_distribution&view=distribution&Itemid=23&release=HEAD '''Development Version'''] (Login is required) || align="center"|[http://www.gcube-system.org/index.php?option=com_content&view=article&id=85&Itemid=47 '''Latest Version'''] (Login is required) |
|} | |} | ||
− | |||
− | |||
− |
Latest revision as of 15:41, 3 November 2011
Introduction
The goals of the work package JRA1 Knowledge Ecosystem Implementation are (i) to provide the technology realising the common practices, standards and solutions identified by the NA4 work package and (ii) to enhance and consolidate the gCube system to match the needs of the ecosystem approach. A primary role of JRA1 (played within TJRA1.2) is to address the various interoperability issues -- ranging from content-related problems to architecture-oriented issues -- inherent in the construction of a knowledge ecosystem, and then identify and implement proper solutions. Major inputs for this part of the work come from DNA4.1.
An Agilish development methodology (referring to no specific Agile development platforms, but to the general idea of quickly responding to change) is driving the teams in JRA1. Plans are sketched and verified with the management boards and user communities; seeing and playing with the system being the most important thing and is preferred to over-planning.
This deliverable documents the gCube technology by reporting on the overall status of the system architecture, the per service tasks and enhancements, and other technical details needed to have a comprehensive understanding of the developed knowledge ecosystem enabling technology.
The deliverable is implemented through a Wiki page to be easily and promptly adapted to reflect the ever-evolving status of the developed technologies. For the same reasons, performed and planned activities are reported via dedicated TRAC reports, created for this very purpose.
State of Software Development Activities
This section reports on the current maintenance and upgrade activities performed and to be performed on the gCube system.
Traditional approaches to planning have proved to do not fit Agile methodology. The fast moving forward of the software makes obsolete every plan very quickly and huge documentation should be continuously updated with a considerable waste of effort. The innovative approach followed by D4ScienceII is based upon a proper exploitation of the project's TRAC instance:
- firstly, four dedicated milestones were defined:
- Short-Term gCube plan, used when the activities described in a ticket are planned to be completed within the next month;
- Medium-Term gCube plan , used when the activities described in a ticket are planned to be completed within the next 3 months;
- Long-Term gCube plan, used when the activities described in a ticket do not have a starting and ending date yet;
- Ready To Release, used when the activities described in a ticket are completed and the resulting software is passed to SA.
- The idea behind these milestones is that a developer moves his/her tickets among them according to the work performed or to be performed on each ticket;
- then, JRA1 management asked to each developer to insert his/her own tasks and give an estimation of delivery by assigning one of the milestones above;
- finally, in order to stand over the activities, four TRAC's living reports have been created for this purpose. These reports are introduced in the following sub-sections.
This flexible strategy gives the opportunity to have a living picture of the continuous evolution of the system. The reports are automatically updated each time a change occurs in a ticket or whenever a new ticket is submitted.
Upgrade Activities
Upgrade activities refer to tasks and enhancements that lead to future releases of the gCube technology:
- → the planned upgrade activities are recorded in report #37
- → the completed upgrade activities are recorded in report #38
Maintenance Activities
Maintenance activities are dedicated to fix defects found in the technology delivered so far and the related documentation:
- → the planned maintenance activities are recorded in report #39
- → the completed maintenance activities are recorded in report #40
In addition to the development activities captured by the reports above, other stems from incidents and malfunctions discovered while exploiting the technology in the production infrastructure. They are captured by two specific living reports, current incidents (Report #1) and resolved incidents (Report #2 and Report #11). Yet, these reports include both JRA1 and JRA2 incidents.
Software Metrics
Software metrics are a way to measure the software in order to provide managers and developers better insight into the code they are releasing. By taking advantage of metrics:
- managers identify potential risks, understand the current state of a project, and track progress during software development
- developers can understand which components should be reworked or more thoroughly tested
Therefore, monthly, starting from M4 (the first month of JRA), a set of software metrics are calculated and collected in the Indicators page (Login is required).
The selected indicators are intended to give a measure of the quality of the code produced along the project.
Code Statistics
D4ScienceII uses StatSVN to retrieve information from the project source code repository and to generate statistics describing the project development. The project source code repository consists of two separated subversion instances: one instance is dedicated to the gCube components, while the other one is specific for the gCore development stream.
Both of them have been operative since March 2008, thus the produced data documents the evolution of the software from such period on.
The following reports, tables and charts provide statistics about the development activity within D4ScienceII:
Documentation
The software delivered is properly documented by the Developer's and Administrator's Guides. These two guides represent the main source of information for whoever has to technically deal with the gCube system and they run deeply at single component level.
- Developer's Guide: this guide targets two audiences:
- developers aiming at producing software that interacts with a gCube components;
- developers that take over a component from another one and start a new development stream.
- Administrator's Guide: this guide mainly targets Site Managers for installing gCube nodes and deploying components. However, it can be used also by developers to have local instances of the components they interact with.
Software
Development Version
D4ScienceII relies on the ETICS system to automate the building and testing phases of the software. In particular, the project has put in place mechanisms to perform daily builds of the latest version of the code committed in the project source code repository. This activity leads to the production of (i) a report of the build activity and (ii) a set of software artifacts including the software package, the source code and the documentation.
JRA1 is responsible for managing the development version (namely HEAD) of these reports and artifacts whose main purpose is to provide a way to
- verify the dependencies among the artifacts
- exchange artifacts among the development teams
gCube | gCore |
---|---|
Build Reports | Build Reports |
Development Version (Login is required) | Latest Version (Login is required) |