https://gcube.wiki.gcube-system.org/api.php?action=feedcontributions&user=Gabriele.giammatteo&feedformat=atomGcube Wiki - User contributions [en]2024-03-28T14:14:23ZUser contributionsMediaWiki 1.25.1https://gcube.wiki.gcube-system.org/index.php?title=ETICS&diff=32095ETICS2019-03-08T09:44:12Z<p>Gabriele.giammatteo: /* ETICS ModelSync */</p>
<hr />
<div>= Overview =<br />
ETICS stands for '''e'''-Infrastructure for '''T'''esting, '''I'''ntegration and '''C'''onfiguration of '''S'''oftware. ETICS is an on-line collaborative service for managing software projects by managing their configuration, enforcing quality standards, building packages and testing them in environments as close as possible to real-world infrastructures. The ETICS tools can be used to run remote builds and tests on multiple platforms at the same time. <br />
<br />
[[File:Etics-overview.png|center|500px]]<br />
<br />
The main architectural components of ETICS (depicted in the figure above) are:<br />
<br />
* '''ETICS Web Portal''' is the main access point to ETICS. It allows to access (view and edit) the configuration of gCube components, submit builds and tests and check the reports;<br />
* '''ETICS Worker Nodes''' are the nodes were builds and tests take place. Users do not need to access to these nodes;<br />
* '''ETICS Command-Line Client''' is a set of command-line tools to access ETICS services. The Command-Line Client provides similar functionalities as the web application. In addition, it allows to run local builds on the developer's machine and is expecially suitable for scripting and batch executions;<br />
<br />
<br />
= How to create ETICS configurations for gCube components =<br />
This sections is meant to provide operative instructions about most relevant aspects of creation of ETICS configurations for Maven-based components.<br />
<br />
== Versioning ==<br />
Version field of the ETICS configuration '''must match''' the version specified for the Maven project in the ''pom.xml'', excepting for the "-SNAPSHOT" postfix. The match will be checked during the invocation of mvn to guarantee that ETICS versions are always in sync with Maven versions.<br />
<br />
== Build Commands ==<br />
<br />
* the ''compile'' target of their ''Build Commands'' must invoke Maven to build the components '''up to the deploy phase''' (e.g. <code>mvn deploy</code>). This allows the deployment of components into Maven repositories directly from ETICS builds<br />
<br />
* the ''install'' target of their ''Build Commands'' must '''copy the gCube profile of the component in the <code>${prefix}</code>''' directory. This allows the registration of the profile with the Software Gateway when the component is released. It remains a good practice to copy also the outcome of the compilation (usually jar files) in <code>${prefix}</code> in order to include them in packages generated by ETICS<br />
<br />
<br />
Sample build commands for a maven-based component should look like the one in the picture below:<br />
<br />
[[Image:maven-component-configuration.png|600px|center|Example of maven-component's build commands]]<br />
<br />
'''Note''': please note that configuration's commands (mainly <code>install</code> target) could change slightly depending on the actual project structure.<br />
<br />
=== Build Commands for gCube Services ===<br />
<br />
gCube services components are usually multi-modules maven projects (see [[Developing_gCube_Maven_Components#Services| gCyve Service development guide]] for more information) containing a module for the stubs, one for the service and optionally one for test-suites. In ETICS a service must be modelled with one ETICS component for each Maven project's module. <br />
<br />
For each ETICS component, build commands should contains commands to deploy only the correspondent Maven module, and, if needed, compile the other modules needed by the first.<br />
<br />
Picture below shows the typical build commands for the stubs component. First the <code>mvn deploy -N</code> is invoked to deploy the project's parent pom. If not executed, stubs' compilation will fail because parent's pom will not be resolved. Then the <code>mvn deploy -pl exampleservice-stubs</code> is invoked to build and compile '''only''' the stubs module. <br />
[[Image:Mvn_services_stubs.png|600px|center|Example of stubs build commands]]<br />
<br />
<br />
Finally, picture below shows the typical build commands for the service component. Since the service module of a gCube service needs the stubs to be compiled locally in order to generate the gar archive, before execute the <code>mvn deploy -pl exampleservice-service</code> it is needed to also build the stubs <code>mvn install -pl exampleservice-stubs</code>. Note that it is not needed to invoke target deploy for stubs, since they are deployed already by the stubs ETICS' component. The parent's pom will already exist in the local repository (deployed by the stubs' component) and, thereofore, it is not needed to compile it again.<br />
[[Image:Mvn_services_service.png|600px|center|Example of service build commands]]<br />
<br />
== Dependencies ==<br />
The configuration must directly or indirectly '''depend on the ETICS configurations of all gCube components that are specified in the POM''', including ''maven-builder''. This guarantees that at build-time all dependencies requested by the pom.xml have been already compiled and installed on the Maven's local reposiotry.<br />
<br />
Dependencies on '''third-party components''' available in Maven Central (or other Maven repository specified in the POM) do '''not''' need to be configured in ETICS (i.e. Maven repositories replace ETICS ''externals'').<br />
<br />
All dependencies must be declared of type '''dynamic''' and the actual version will be resolved at project level.<br />
<br />
<br />
Example of dependencies for a Maven component:<br />
[[Image:Maven-deps.png|700px|center|Example of maven-component's dependency set]]<br />
<br />
== Environment ==<br />
Unlike Ant-based components, Maven components do not use the <code>CLASSPATH</code> variable for compile-time dependency resolution, but rely on the Maven local repository. This means that the Environment section of Maven component's configurations '''will be empty''' (unless other environment variables has to be set for other purposes).<br />
<br />
There exist an exception, though. If the component's artifact is meant to be used as compile-time dependency by others Ant-based components, the artifact produced by the Maven build needs to be ''exported'' in the CLASSPATH to allows the Ant-based component to find it. Only in this case the Environment section will contain an entry to add to CLASSPATH the artifact produced. It is suggested to use following convention:<br />
<pre><br />
CLASSPATH: ${moduleDir}/target/<artifactid>-${version}-[SNAPSHOT|${maven.system-version}].jar<br />
</pre><br />
where:<br />
* <artifactId> must be replaced with the name of the artifact produced by Maven compilation. By default it is the same of <artifactId> value in the pom<br />
* <code>SNAPSHOT</code> must be added only if you are setting up a configuration for org.gcube.HEAD otherwise<br />
* you should use the <code>${maven.system-version}</code> property<br />
* <code>${moduleDir}</code> and <code>${version}</code> are ETICS properties and will be resolved by ETICS itself<br />
<br />
Valid examples of CLASSPATH:<br />
<pre><br />
CLASSPATH: ${moduleDir}/target/home-library-${version}-${maven.system-version}.jar<br />
CLASSPATH: ${moduleDir}/target/home-library-${version}-SNAPSHOT.jar<br />
CLASSPATH: ${moduleDir}/SoftwareGateway-stubs/target/softwaregateway-stubs-${version}-${maven.system-version}.jar<br />
CLASSPATH: ${moduleDir}/SoftwareGateway-stubs/target/softwaregateway-stubs-${version}-SNAPSHOT.jar <br />
...<br />
</pre><br />
<br />
'''Tip''': you can easily find exactly which artifact is created during an ETICS build by searching in the build report of the component for the line "''Installing /home/gbuilder/catalogueoflife-spd-plugin/<span style="background-color:#E6E6FA">target/catalogue-of-life-spd-plugin-1.0.0-2.12.0.jar</span> to...''"<br />
<br />
<br />
= How to submit remote builds of gcube subsystems/components =<br />
A pre-requisite to submit a build is to have right to do that. [[Role Developer|Developers]] have rights to build their components, [[Role Subsystem Manager|Subsystem Managers]] have rights to build their subsystem and the [[Role Release Manager|Release Manager]] have the rights to build the entire project.<br />
<br />
<br />
In ETICS Portal, to access the Remote Build submitting form (see picture below) just '''right-click''' on the configuration you wish to build and then select '''"Remote Build..."''' from the contextual menu that will appear. The submission form is shown below:<br />
<br />
[[File:Etic-build-submission.png|center|frame]]<br />
<br />
<br />
<br />
The most common options to set for gCube builds are listed in the table below.<br />
{| border="1" cellpadding="4" cellspacing="0"<br />
|- style="background-color:#F2F2F2"<br />
! width=200 | option <br />
! width=500 | description<br />
! width=80 | usage<br />
|-<br />
| '''verbose''' <br />
| useful to debug the build process<br />
| recommended<br />
|-<br />
| '''Ignore Locking''' <br />
| A locked configuration in ETICS has a "snapshot" of the gCube configuration in which it has been locked. When a locked configuration is built in a different version of gCube of the one in which has been locked, it will try to bring in the build all the configurations belonging to the "snapshot"; most likely this will result in multiple configurations for the same component that will break the build. The usage of "Ignore Locking" option will prevent configurations from the snapshots of locked configuration to be brought in the build.<br />
| mandatory<br />
|-<br />
| '''Propagate environment and ...'''<br />
| essential. It will instruct ETICS about the gCube version in which context (gCube's version) build the configuration.<br />
| mandatory<br />
|-<br />
| '''Checkout from Volatile area'''<br />
| every ETICS build stores the binary artifacts in a remote area. By setting this option the build can use, if they exist, binary artifacts from past builds instead of re-compiling it from source (it only applies to dependencies, not on the build target component). <br />
| recommended<br />
|-<br />
| '''Do not stop on errors''' <br />
| ETICS will try to complete the build also in case the build of one or more components fails.<br />
| recommended<br />
|-<br />
| '''Host selection'''<br />
| The platform against which the build will be performed. gCube is completely in java and, thus, it is completely cross-platform and, theoretically, any of available platforms can be chosen. However, '''currently only CentOS 6.10 is available for gCube builds.'''<br />
| mandatory<br />
|}<br />
<br />
<br />
Once options have been set, just click on the ''submit'' icon (top-left of the submission form) and the build will start. You can keep trace of your builds in the '''Submissions'' tab of ETICS Portal.<br />
<br />
= ETICS Issues =<br />
The ''ETICS Issues'' section (reachable from the top-bar, on the left) lists the issues found in components during the build. Issues are related not only to the result of the builds but also to quality guidelines defined by projects. Some issues specific for gCube have been defined (e.g. compliance of README files) and are reported in this list.<br />
<br />
As depicted in the figure below, the user can group (in the top-right of the page) the issue list by different attributes: module, priority, type and owner. A summary for each group is displayed at the top of the page reporting the type and subtype of the issue, the total number of issues in the group and the sum of the technical debt computed for the issues.<br />
<br />
[[File:etics-issues-tutorial-1.png|center|800px]]<br />
<br />
<br />
<br />
Under the groups summary, there is the list of all issues ordered by group. By clicking on the issue, the details are displayed (see figure below).<br />
<br />
[[File:etics-issues-tutorial-2.png|center|800px]]<br />
<br />
<br />
== Issue Owners ==<br />
For each issue, ETICS computes a list of owners. It is the list of users that have permissions on the component associated to the issue order by proximity: users that have permissions at component-level are listed first, then the users that have permission on the component's subystem and finally users that have permission at level of project.<br />
<br />
<br />
== Issue Lifecycle ==<br />
ETICS runs a number of checkers during the remote builds and creates the discovered issues at the end of each build. An issue is automatically closed by ETICS when it runs again the build of the affected component and does not find the issue anymore. Closed issues appears with in Straight-through style for few days and then disappear completely.<br />
<br />
Some checks run only during BTRT builds and, therefore, are updated only during BTRT builds.<br />
<br />
= ETICS Continuous Builds =<br />
Along the remote builds submitted by the users on demand (see [[#How_to_submit_remote_builds_of_gcube_subsystems.2Fcomponents|here]]) ETICS has also the capability to trigger builds when specific events occurred.<br />
<br />
There are two types of events monitored:<br />
* changes in the ETICS data model (e.g. the user modify build commands, the user modify a dependency);<br />
* changes in the VCS: new commits in the source code repository;<br />
<br />
When one of these two events occurs, ETICS look for the ETICS Configuration(s) associated with the event and request a build for that configuration. The build submission is delayed of 5 minutes from the event. During this time, any other request of building the same configuration is ignored. This avoid multiple builds when several changes and/or several commits are done in a very short period of time.<br />
<br />
== Limitations ==<br />
At the moment the user that made the change is not notified when the build is launched and/or finish.<br />
<br />
= Dismissed Modules =<br />
ETICS does not support the removal of modules (projects, subsystems and components) if they (or they children) have one or more locked configurations.<br />
<br />
However, in big and long running projects some modules could be abandoned, replaced by new ones or dismissed. The number of dismissed modules can increase a lot and "pollute" the GUI with several unneeded modules and configurations.<br />
<br />
For this reason, there is an option to hide dismissed modules in the ETICS GUI (only hide, they will remain in the database).<br />
<br />
== How to define dismissed modules ==<br />
To flag a module as dismissed, it is enough to select the "Set Dismissed" action in the contextual menù in the project tree. The action will be enabled only if the user has the edit privileges on the module.<br />
<br />
[[image:set-dismissed.png||alt="etics-dismiss-module.png"||center||width="350"]]<br />
<br />
== How to show/hide dismissed modules ==<br />
To show/hide dismissed modules in the ETICS GUI there is a toggle option. It is located in the settings menù, in the toolbar as shown in the figure below.<br />
<br />
[[image:ETICS-GUI-dismissed-option.png||height="197"||center]]<br />
<br />
The option is per-user and persisted server side. The default for users and for not authenticated users is to hide dismissed modules.<br />
<br />
This option affects the ProjectTree (on the left-side of the GUI) and the list of submodules shown when editing the subconfigurations. In the latter case, please note that even if a module is flagged as dismissed, but one of its configurations is attached as subconfiguration, it will be shown in any case.<br />
<br />
== Under the hood ==<br />
<br />
Dismissed modules are flagged by using the "#dismissed" tag in the '''description''' of the module. Any module that contains this tag in the description will be considered as dismissed. The option "Set Dismissed" is a shortcut to add such a tag in the description. If the tag is removed, the module will not be considered dismissed anymore.<br />
<br />
= ETICS ModelSync =<br />
<br />
This functionality allows to automatically update ETICS model and keep it in sync with changes in the source code.<br />
<br />
In general terms, it works in the following way:<br />
# each ETICS Configuration is associated with a particular location on VCS (e.g. SVN folder, Git commit, ...) computed using the checkout commands and properties defined in ETICS by the users;<br />
# a specific module (namely ProjectParser) crawls the VCS location looking for Maven project descriptors (i.e. pom.xml files). The ProjectParser is general enough to allows also other project types (e.g. Ant, Python distuitls, Ivy, ...), but currently only parsing of Maven pom.xml files is supported;<br />
# The ProjectParser extract from the pom.xml the component information like coordinates, version, dependencies, description, etc, ...<br />
# The ModelSync sets the version of the ETICS Configuration to the version found in the pom.xml;<br />
# The ModelSync adds (or updates, if it already exists) the ".coords" property in the ETICS Configuration with the coordinates found in the pom.xml analysed. This realizes a mapping between ETICS Configurations and Maven coordinates. This mapping will be useful later when setting ETICS dependencies. Moreover, this mapping enables Maven-aware search operations like: "What ETICS Configuration provides a given maven coordinates?", "What artifact is produced by a given ETICS Configuration?";<br />
# The ETICS ModelSync adds the dependencies in the ETICS Configuration. For each dependency found in the pom.xml, a search is performed in ETICS to find the component that provides the dependency based on its coordinates. If one or more components are found, they are added as dependency in ETICS. This works not only for dependencies but also for the parent and dependencyManagement (e.g. boms) section in the pom.xml;<br />
# ETICS dependencies that are not found in the pom.xml are deleted;<br />
# It may happens that a given dependency found in the pom.xml is not added by the ETICS ModelSync because no ETICS Configurations that provides the dependency are found. This could be simply caused by the fact that the ProjectParser still did not analysed the pom.xml of the dependency component. In this case, ETICS remembers this "unresolved" dependency and as soon as the searched coordinates are found in an ETICS Configuration, the dependency is added;<br />
# In case multiple projects (i.e. pom.xml files) are detected in VCS location (e.g. multi-module projects like in case of gCube services), the build command is analysed to understand which Maven projects are actually built by the component;<br />
<br />
This process is triggered automatically every time a change in ETICS or in the VCS is detected (e.g. the ETICS Configuration has been modified, a new commit in SVN occurred).<br />
<br />
<br />
<br />
By default, this process is enabled for all configurations. However, its behaviour can be controlled through a set of ETICS properties that can be set in the ETICS Configurations:<br />
* <code>reactor.ignore=true</code> disables any action on the configuration (including automatic builds). It is usually automatically set when ETICS finds anomalies on the configuration (e.g. a configuration that points to the trunk)<br />
* <code>model.sync.enabled=false</code> disables completely the synchronization<br />
* <code>model.sync.coordinates.enabled=false</code> disables synchronization of the ".coords" property<br />
* <code>model.sync.version.enabled=false</code> disable synchronization of version<br />
* <code>model.sync.dependencies.enabled=false</code> disables the synchronization of dependencies<br />
* <code>model.sync.dependencies.ignore=<etics-module-name-1>;<etics-module-name-2>;...</code> a ";" separated list of ETICS module names to ignore while synchronizing dependencies. Dependencies on ETICS modules in this list are neither added nor deleted automatically; <br />
<br />
In addition, the label <code>#model.sync.ignorefordependencies</code> added at module level (in the module description) excludes that module from being set as dependency in other modules. It is commonly used for dismissed modules when they have locked configurations that have the ''.coords'' property (and hence are considered when looking for modules that provides the given maven project).<br />
<br />
<br />
== Current limitations ==<br />
* currently only Maven projects are synchronised. For Ant components sync is not (and will be not) implemented;<br />
<br />
== How to use it ==<br />
The rational behind this functionality is to automatically react to changes in the VCS and to update ETICS model accordingly. This is intended to be completely automatic without the interaction of users. The only way users can interact with the synchronisation mechanism is through setting of the control properties. When the synchronisation is enabled, users should not modify manually ETICS Configuration version and dependencies because they will be revert by the synchronisation mechanism.<br />
<br />
What to do when a new component is created on ETICS? It is enough to set the ETICS checkout and build commands for the ETICS Configuration and the ModelSync will be able to automatically set (and keep aligned) the version and the dependencies.<br />
<br />
= External Links =<br />
[https://etics.esl.eng.it/etics-gui/ Etics Web Portal]<br />
<br />
[http://eticsbuild2.research-infrastructures.eu/BuildReport/home/AllBuilds Etics Build Report]<br />
<br />
[https://twiki.cern.ch/twiki/bin/view/ETICS/ClientHowTo Etics Client Installation HowTo]</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=ETICS&diff=32094ETICS2019-03-08T09:42:55Z<p>Gabriele.giammatteo: /* ETICS ModelSync */</p>
<hr />
<div>= Overview =<br />
ETICS stands for '''e'''-Infrastructure for '''T'''esting, '''I'''ntegration and '''C'''onfiguration of '''S'''oftware. ETICS is an on-line collaborative service for managing software projects by managing their configuration, enforcing quality standards, building packages and testing them in environments as close as possible to real-world infrastructures. The ETICS tools can be used to run remote builds and tests on multiple platforms at the same time. <br />
<br />
[[File:Etics-overview.png|center|500px]]<br />
<br />
The main architectural components of ETICS (depicted in the figure above) are:<br />
<br />
* '''ETICS Web Portal''' is the main access point to ETICS. It allows to access (view and edit) the configuration of gCube components, submit builds and tests and check the reports;<br />
* '''ETICS Worker Nodes''' are the nodes were builds and tests take place. Users do not need to access to these nodes;<br />
* '''ETICS Command-Line Client''' is a set of command-line tools to access ETICS services. The Command-Line Client provides similar functionalities as the web application. In addition, it allows to run local builds on the developer's machine and is expecially suitable for scripting and batch executions;<br />
<br />
<br />
= How to create ETICS configurations for gCube components =<br />
This sections is meant to provide operative instructions about most relevant aspects of creation of ETICS configurations for Maven-based components.<br />
<br />
== Versioning ==<br />
Version field of the ETICS configuration '''must match''' the version specified for the Maven project in the ''pom.xml'', excepting for the "-SNAPSHOT" postfix. The match will be checked during the invocation of mvn to guarantee that ETICS versions are always in sync with Maven versions.<br />
<br />
== Build Commands ==<br />
<br />
* the ''compile'' target of their ''Build Commands'' must invoke Maven to build the components '''up to the deploy phase''' (e.g. <code>mvn deploy</code>). This allows the deployment of components into Maven repositories directly from ETICS builds<br />
<br />
* the ''install'' target of their ''Build Commands'' must '''copy the gCube profile of the component in the <code>${prefix}</code>''' directory. This allows the registration of the profile with the Software Gateway when the component is released. It remains a good practice to copy also the outcome of the compilation (usually jar files) in <code>${prefix}</code> in order to include them in packages generated by ETICS<br />
<br />
<br />
Sample build commands for a maven-based component should look like the one in the picture below:<br />
<br />
[[Image:maven-component-configuration.png|600px|center|Example of maven-component's build commands]]<br />
<br />
'''Note''': please note that configuration's commands (mainly <code>install</code> target) could change slightly depending on the actual project structure.<br />
<br />
=== Build Commands for gCube Services ===<br />
<br />
gCube services components are usually multi-modules maven projects (see [[Developing_gCube_Maven_Components#Services| gCyve Service development guide]] for more information) containing a module for the stubs, one for the service and optionally one for test-suites. In ETICS a service must be modelled with one ETICS component for each Maven project's module. <br />
<br />
For each ETICS component, build commands should contains commands to deploy only the correspondent Maven module, and, if needed, compile the other modules needed by the first.<br />
<br />
Picture below shows the typical build commands for the stubs component. First the <code>mvn deploy -N</code> is invoked to deploy the project's parent pom. If not executed, stubs' compilation will fail because parent's pom will not be resolved. Then the <code>mvn deploy -pl exampleservice-stubs</code> is invoked to build and compile '''only''' the stubs module. <br />
[[Image:Mvn_services_stubs.png|600px|center|Example of stubs build commands]]<br />
<br />
<br />
Finally, picture below shows the typical build commands for the service component. Since the service module of a gCube service needs the stubs to be compiled locally in order to generate the gar archive, before execute the <code>mvn deploy -pl exampleservice-service</code> it is needed to also build the stubs <code>mvn install -pl exampleservice-stubs</code>. Note that it is not needed to invoke target deploy for stubs, since they are deployed already by the stubs ETICS' component. The parent's pom will already exist in the local repository (deployed by the stubs' component) and, thereofore, it is not needed to compile it again.<br />
[[Image:Mvn_services_service.png|600px|center|Example of service build commands]]<br />
<br />
== Dependencies ==<br />
The configuration must directly or indirectly '''depend on the ETICS configurations of all gCube components that are specified in the POM''', including ''maven-builder''. This guarantees that at build-time all dependencies requested by the pom.xml have been already compiled and installed on the Maven's local reposiotry.<br />
<br />
Dependencies on '''third-party components''' available in Maven Central (or other Maven repository specified in the POM) do '''not''' need to be configured in ETICS (i.e. Maven repositories replace ETICS ''externals'').<br />
<br />
All dependencies must be declared of type '''dynamic''' and the actual version will be resolved at project level.<br />
<br />
<br />
Example of dependencies for a Maven component:<br />
[[Image:Maven-deps.png|700px|center|Example of maven-component's dependency set]]<br />
<br />
== Environment ==<br />
Unlike Ant-based components, Maven components do not use the <code>CLASSPATH</code> variable for compile-time dependency resolution, but rely on the Maven local repository. This means that the Environment section of Maven component's configurations '''will be empty''' (unless other environment variables has to be set for other purposes).<br />
<br />
There exist an exception, though. If the component's artifact is meant to be used as compile-time dependency by others Ant-based components, the artifact produced by the Maven build needs to be ''exported'' in the CLASSPATH to allows the Ant-based component to find it. Only in this case the Environment section will contain an entry to add to CLASSPATH the artifact produced. It is suggested to use following convention:<br />
<pre><br />
CLASSPATH: ${moduleDir}/target/<artifactid>-${version}-[SNAPSHOT|${maven.system-version}].jar<br />
</pre><br />
where:<br />
* <artifactId> must be replaced with the name of the artifact produced by Maven compilation. By default it is the same of <artifactId> value in the pom<br />
* <code>SNAPSHOT</code> must be added only if you are setting up a configuration for org.gcube.HEAD otherwise<br />
* you should use the <code>${maven.system-version}</code> property<br />
* <code>${moduleDir}</code> and <code>${version}</code> are ETICS properties and will be resolved by ETICS itself<br />
<br />
Valid examples of CLASSPATH:<br />
<pre><br />
CLASSPATH: ${moduleDir}/target/home-library-${version}-${maven.system-version}.jar<br />
CLASSPATH: ${moduleDir}/target/home-library-${version}-SNAPSHOT.jar<br />
CLASSPATH: ${moduleDir}/SoftwareGateway-stubs/target/softwaregateway-stubs-${version}-${maven.system-version}.jar<br />
CLASSPATH: ${moduleDir}/SoftwareGateway-stubs/target/softwaregateway-stubs-${version}-SNAPSHOT.jar <br />
...<br />
</pre><br />
<br />
'''Tip''': you can easily find exactly which artifact is created during an ETICS build by searching in the build report of the component for the line "''Installing /home/gbuilder/catalogueoflife-spd-plugin/<span style="background-color:#E6E6FA">target/catalogue-of-life-spd-plugin-1.0.0-2.12.0.jar</span> to...''"<br />
<br />
<br />
= How to submit remote builds of gcube subsystems/components =<br />
A pre-requisite to submit a build is to have right to do that. [[Role Developer|Developers]] have rights to build their components, [[Role Subsystem Manager|Subsystem Managers]] have rights to build their subsystem and the [[Role Release Manager|Release Manager]] have the rights to build the entire project.<br />
<br />
<br />
In ETICS Portal, to access the Remote Build submitting form (see picture below) just '''right-click''' on the configuration you wish to build and then select '''"Remote Build..."''' from the contextual menu that will appear. The submission form is shown below:<br />
<br />
[[File:Etic-build-submission.png|center|frame]]<br />
<br />
<br />
<br />
The most common options to set for gCube builds are listed in the table below.<br />
{| border="1" cellpadding="4" cellspacing="0"<br />
|- style="background-color:#F2F2F2"<br />
! width=200 | option <br />
! width=500 | description<br />
! width=80 | usage<br />
|-<br />
| '''verbose''' <br />
| useful to debug the build process<br />
| recommended<br />
|-<br />
| '''Ignore Locking''' <br />
| A locked configuration in ETICS has a "snapshot" of the gCube configuration in which it has been locked. When a locked configuration is built in a different version of gCube of the one in which has been locked, it will try to bring in the build all the configurations belonging to the "snapshot"; most likely this will result in multiple configurations for the same component that will break the build. The usage of "Ignore Locking" option will prevent configurations from the snapshots of locked configuration to be brought in the build.<br />
| mandatory<br />
|-<br />
| '''Propagate environment and ...'''<br />
| essential. It will instruct ETICS about the gCube version in which context (gCube's version) build the configuration.<br />
| mandatory<br />
|-<br />
| '''Checkout from Volatile area'''<br />
| every ETICS build stores the binary artifacts in a remote area. By setting this option the build can use, if they exist, binary artifacts from past builds instead of re-compiling it from source (it only applies to dependencies, not on the build target component). <br />
| recommended<br />
|-<br />
| '''Do not stop on errors''' <br />
| ETICS will try to complete the build also in case the build of one or more components fails.<br />
| recommended<br />
|-<br />
| '''Host selection'''<br />
| The platform against which the build will be performed. gCube is completely in java and, thus, it is completely cross-platform and, theoretically, any of available platforms can be chosen. However, '''currently only CentOS 6.10 is available for gCube builds.'''<br />
| mandatory<br />
|}<br />
<br />
<br />
Once options have been set, just click on the ''submit'' icon (top-left of the submission form) and the build will start. You can keep trace of your builds in the '''Submissions'' tab of ETICS Portal.<br />
<br />
= ETICS Issues =<br />
The ''ETICS Issues'' section (reachable from the top-bar, on the left) lists the issues found in components during the build. Issues are related not only to the result of the builds but also to quality guidelines defined by projects. Some issues specific for gCube have been defined (e.g. compliance of README files) and are reported in this list.<br />
<br />
As depicted in the figure below, the user can group (in the top-right of the page) the issue list by different attributes: module, priority, type and owner. A summary for each group is displayed at the top of the page reporting the type and subtype of the issue, the total number of issues in the group and the sum of the technical debt computed for the issues.<br />
<br />
[[File:etics-issues-tutorial-1.png|center|800px]]<br />
<br />
<br />
<br />
Under the groups summary, there is the list of all issues ordered by group. By clicking on the issue, the details are displayed (see figure below).<br />
<br />
[[File:etics-issues-tutorial-2.png|center|800px]]<br />
<br />
<br />
== Issue Owners ==<br />
For each issue, ETICS computes a list of owners. It is the list of users that have permissions on the component associated to the issue order by proximity: users that have permissions at component-level are listed first, then the users that have permission on the component's subystem and finally users that have permission at level of project.<br />
<br />
<br />
== Issue Lifecycle ==<br />
ETICS runs a number of checkers during the remote builds and creates the discovered issues at the end of each build. An issue is automatically closed by ETICS when it runs again the build of the affected component and does not find the issue anymore. Closed issues appears with in Straight-through style for few days and then disappear completely.<br />
<br />
Some checks run only during BTRT builds and, therefore, are updated only during BTRT builds.<br />
<br />
= ETICS Continuous Builds =<br />
Along the remote builds submitted by the users on demand (see [[#How_to_submit_remote_builds_of_gcube_subsystems.2Fcomponents|here]]) ETICS has also the capability to trigger builds when specific events occurred.<br />
<br />
There are two types of events monitored:<br />
* changes in the ETICS data model (e.g. the user modify build commands, the user modify a dependency);<br />
* changes in the VCS: new commits in the source code repository;<br />
<br />
When one of these two events occurs, ETICS look for the ETICS Configuration(s) associated with the event and request a build for that configuration. The build submission is delayed of 5 minutes from the event. During this time, any other request of building the same configuration is ignored. This avoid multiple builds when several changes and/or several commits are done in a very short period of time.<br />
<br />
== Limitations ==<br />
At the moment the user that made the change is not notified when the build is launched and/or finish.<br />
<br />
= Dismissed Modules =<br />
ETICS does not support the removal of modules (projects, subsystems and components) if they (or they children) have one or more locked configurations.<br />
<br />
However, in big and long running projects some modules could be abandoned, replaced by new ones or dismissed. The number of dismissed modules can increase a lot and "pollute" the GUI with several unneeded modules and configurations.<br />
<br />
For this reason, there is an option to hide dismissed modules in the ETICS GUI (only hide, they will remain in the database).<br />
<br />
== How to define dismissed modules ==<br />
To flag a module as dismissed, it is enough to select the "Set Dismissed" action in the contextual menù in the project tree. The action will be enabled only if the user has the edit privileges on the module.<br />
<br />
[[image:set-dismissed.png||alt="etics-dismiss-module.png"||center||width="350"]]<br />
<br />
== How to show/hide dismissed modules ==<br />
To show/hide dismissed modules in the ETICS GUI there is a toggle option. It is located in the settings menù, in the toolbar as shown in the figure below.<br />
<br />
[[image:ETICS-GUI-dismissed-option.png||height="197"||center]]<br />
<br />
The option is per-user and persisted server side. The default for users and for not authenticated users is to hide dismissed modules.<br />
<br />
This option affects the ProjectTree (on the left-side of the GUI) and the list of submodules shown when editing the subconfigurations. In the latter case, please note that even if a module is flagged as dismissed, but one of its configurations is attached as subconfiguration, it will be shown in any case.<br />
<br />
== Under the hood ==<br />
<br />
Dismissed modules are flagged by using the "#dismissed" tag in the '''description''' of the module. Any module that contains this tag in the description will be considered as dismissed. The option "Set Dismissed" is a shortcut to add such a tag in the description. If the tag is removed, the module will not be considered dismissed anymore.<br />
<br />
= ETICS ModelSync =<br />
<br />
This functionality allows to automatically update ETICS model and keep it in sync with changes in the source code.<br />
<br />
In general terms, it works in the following way:<br />
# each ETICS Configuration is associated with a particular location on VCS (e.g. SVN folder, Git commit, ...) computed using the checkout commands and properties defined in ETICS by the users;<br />
# a specific module (namely ProjectParser) crawls the VCS location looking for Maven project descriptors (i.e. pom.xml files). The ProjectParser is general enough to allows also other project types (e.g. Ant, Python distuitls, Ivy, ...), but currently only parsing of Maven pom.xml files is supported;<br />
# The ProjectParser extract from the pom.xml the component information like coordinates, version, dependencies, description, etc, ...<br />
# The ModelSync sets the version of the ETICS Configuration to the version found in the pom.xml;<br />
# The ModelSync adds (or updates, if it already exists) the ".coords" property in the ETICS Configuration with the coordinates found in the pom.xml analysed. This realizes a mapping between ETICS Configurations and Maven coordinates. This mapping will be useful later when setting ETICS dependencies. Moreover, this mapping enables Maven-aware search operations like: "What ETICS Configuration provides a given maven coordinates?", "What artifact is produced by a given ETICS Configuration?";<br />
# The ETICS ModelSync adds the dependencies in the ETICS Configuration. For each dependency found in the pom.xml, a search is performed in ETICS to find the component that provides the dependency based on its coordinates. If one or more components are found, they are added as dependency in ETICS. This works not only for dependencies but also for the parent and dependencyManagement (e.g. boms) section in the pom.xml;<br />
# ETICS dependencies that are not found in the pom.xml are deleted;<br />
# It may happens that a given dependency found in the pom.xml is not added by the ETICS ModelSync because no ETICS Configurations that provides the dependency are found. This could be simply caused by the fact that the ProjectParser still did not analysed the pom.xml of the dependency component. In this case, ETICS remembers this "unresolved" dependency and as soon as the searched coordinates are found in an ETICS Configuration, the dependency is added;<br />
# In case multiple projects (i.e. pom.xml files) are detected in VCS location (e.g. multi-module projects like in case of gCube services), the build command is analysed to understand which Maven projects are actually built by the component;<br />
<br />
This process is triggered automatically every time a change in ETICS or in the VCS is detected (e.g. the ETICS Configuration has been modified, a new commit in SVN occurred).<br />
<br />
<br />
<br />
By default, this process is enabled for all configurations. However, its behaviour can be controlled through a set of ETICS properties that can be set in the ETICS Configurations:<br />
* "reactor.ignore=true" disables any action on the configuration (including automatic builds). It is usually automatically set when ETICS finds anomalies on the configuration (e.g. a configuration that points to the trunk)<br />
* "model.sync.enabled=false" disables completely the synchronization<br />
* "model.sync.coordinates.enabled=false" disables synchronization of the ".coords" property<br />
* "model.sync.version.enabled=false" disable synchronization of version<br />
* "model.sync.dependencies.enabled=false" disables the synchronization of dependencies<br />
* "model.sync.dependencies.ignore=<etics-module-name-1>;<etics-module-name-2>;..." a ";" separated list of ETICS module names to ignore while synchronizing dependencies. Dependencies on ETICS modules in this list are neither added nor deleted automatically; <br />
<br />
In addition, the label <code>#model.sync.ignorefordependencies</code> added at module level (in the module description) excludes that module from being set as dependency in other modules. It is commonly used for dismissed modules when they have locked configurations that have the ''.coords'' property (and hence are considered when looking for modules that provides the given maven project).<br />
<br />
<br />
== Current limitations ==<br />
* currently only Maven projects are synchronised. For Ant components sync is not (and will be not) implemented;<br />
<br />
== How to use it ==<br />
The rational behind this functionality is to automatically react to changes in the VCS and to update ETICS model accordingly. This is intended to be completely automatic without the interaction of users. The only way users can interact with the synchronisation mechanism is through setting of the control properties. When the synchronisation is enabled, users should not modify manually ETICS Configuration version and dependencies because they will be revert by the synchronisation mechanism.<br />
<br />
What to do when a new component is created on ETICS? It is enough to set the ETICS checkout and build commands for the ETICS Configuration and the ModelSync will be able to automatically set (and keep aligned) the version and the dependencies.<br />
<br />
= External Links =<br />
[https://etics.esl.eng.it/etics-gui/ Etics Web Portal]<br />
<br />
[http://eticsbuild2.research-infrastructures.eu/BuildReport/home/AllBuilds Etics Build Report]<br />
<br />
[https://twiki.cern.ch/twiki/bin/view/ETICS/ClientHowTo Etics Client Installation HowTo]</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=ETICS&diff=31458ETICS2018-10-10T12:59:19Z<p>Gabriele.giammatteo: /* How to submit remote builds of gcube subsystems/components */</p>
<hr />
<div>= Overview =<br />
ETICS stands for '''e'''-Infrastructure for '''T'''esting, '''I'''ntegration and '''C'''onfiguration of '''S'''oftware. ETICS is an on-line collaborative service for managing software projects by managing their configuration, enforcing quality standards, building packages and testing them in environments as close as possible to real-world infrastructures. The ETICS tools can be used to run remote builds and tests on multiple platforms at the same time. <br />
<br />
[[File:Etics-overview.png|center|500px]]<br />
<br />
The main architectural components of ETICS (depicted in the figure above) are:<br />
<br />
* '''ETICS Web Portal''' is the main access point to ETICS. It allows to access (view and edit) the configuration of gCube components, submit builds and tests and check the reports;<br />
* '''ETICS Worker Nodes''' are the nodes were builds and tests take place. Users do not need to access to these nodes;<br />
* '''ETICS Command-Line Client''' is a set of command-line tools to access ETICS services. The Command-Line Client provides similar functionalities as the web application. In addition, it allows to run local builds on the developer's machine and is expecially suitable for scripting and batch executions;<br />
<br />
<br />
= How to create ETICS configurations for gCube components =<br />
This sections is meant to provide operative instructions about most relevant aspects of creation of ETICS configurations for Maven-based components.<br />
<br />
== Versioning ==<br />
Version field of the ETICS configuration '''must match''' the version specified for the Maven project in the ''pom.xml'', excepting for the "-SNAPSHOT" postfix. The match will be checked during the invocation of mvn to guarantee that ETICS versions are always in sync with Maven versions.<br />
<br />
== Build Commands ==<br />
<br />
* the ''compile'' target of their ''Build Commands'' must invoke Maven to build the components '''up to the deploy phase''' (e.g. <code>mvn deploy</code>). This allows the deployment of components into Maven repositories directly from ETICS builds<br />
<br />
* the ''install'' target of their ''Build Commands'' must '''copy the gCube profile of the component in the <code>${prefix}</code>''' directory. This allows the registration of the profile with the Software Gateway when the component is released. It remains a good practice to copy also the outcome of the compilation (usually jar files) in <code>${prefix}</code> in order to include them in packages generated by ETICS<br />
<br />
<br />
Sample build commands for a maven-based component should look like the one in the picture below:<br />
<br />
[[Image:maven-component-configuration.png|600px|center|Example of maven-component's build commands]]<br />
<br />
'''Note''': please note that configuration's commands (mainly <code>install</code> target) could change slightly depending on the actual project structure.<br />
<br />
=== Build Commands for gCube Services ===<br />
<br />
gCube services components are usually multi-modules maven projects (see [[Developing_gCube_Maven_Components#Services| gCyve Service development guide]] for more information) containing a module for the stubs, one for the service and optionally one for test-suites. In ETICS a service must be modelled with one ETICS component for each Maven project's module. <br />
<br />
For each ETICS component, build commands should contains commands to deploy only the correspondent Maven module, and, if needed, compile the other modules needed by the first.<br />
<br />
Picture below shows the typical build commands for the stubs component. First the <code>mvn deploy -N</code> is invoked to deploy the project's parent pom. If not executed, stubs' compilation will fail because parent's pom will not be resolved. Then the <code>mvn deploy -pl exampleservice-stubs</code> is invoked to build and compile '''only''' the stubs module. <br />
[[Image:Mvn_services_stubs.png|600px|center|Example of stubs build commands]]<br />
<br />
<br />
Finally, picture below shows the typical build commands for the service component. Since the service module of a gCube service needs the stubs to be compiled locally in order to generate the gar archive, before execute the <code>mvn deploy -pl exampleservice-service</code> it is needed to also build the stubs <code>mvn install -pl exampleservice-stubs</code>. Note that it is not needed to invoke target deploy for stubs, since they are deployed already by the stubs ETICS' component. The parent's pom will already exist in the local repository (deployed by the stubs' component) and, thereofore, it is not needed to compile it again.<br />
[[Image:Mvn_services_service.png|600px|center|Example of service build commands]]<br />
<br />
== Dependencies ==<br />
The configuration must directly or indirectly '''depend on the ETICS configurations of all gCube components that are specified in the POM''', including ''maven-builder''. This guarantees that at build-time all dependencies requested by the pom.xml have been already compiled and installed on the Maven's local reposiotry.<br />
<br />
Dependencies on '''third-party components''' available in Maven Central (or other Maven repository specified in the POM) do '''not''' need to be configured in ETICS (i.e. Maven repositories replace ETICS ''externals'').<br />
<br />
All dependencies must be declared of type '''dynamic''' and the actual version will be resolved at project level.<br />
<br />
<br />
Example of dependencies for a Maven component:<br />
[[Image:Maven-deps.png|700px|center|Example of maven-component's dependency set]]<br />
<br />
== Environment ==<br />
Unlike Ant-based components, Maven components do not use the <code>CLASSPATH</code> variable for compile-time dependency resolution, but rely on the Maven local repository. This means that the Environment section of Maven component's configurations '''will be empty''' (unless other environment variables has to be set for other purposes).<br />
<br />
There exist an exception, though. If the component's artifact is meant to be used as compile-time dependency by others Ant-based components, the artifact produced by the Maven build needs to be ''exported'' in the CLASSPATH to allows the Ant-based component to find it. Only in this case the Environment section will contain an entry to add to CLASSPATH the artifact produced. It is suggested to use following convention:<br />
<pre><br />
CLASSPATH: ${moduleDir}/target/<artifactid>-${version}-[SNAPSHOT|${maven.system-version}].jar<br />
</pre><br />
where:<br />
* <artifactId> must be replaced with the name of the artifact produced by Maven compilation. By default it is the same of <artifactId> value in the pom<br />
* <code>SNAPSHOT</code> must be added only if you are setting up a configuration for org.gcube.HEAD otherwise<br />
* you should use the <code>${maven.system-version}</code> property<br />
* <code>${moduleDir}</code> and <code>${version}</code> are ETICS properties and will be resolved by ETICS itself<br />
<br />
Valid examples of CLASSPATH:<br />
<pre><br />
CLASSPATH: ${moduleDir}/target/home-library-${version}-${maven.system-version}.jar<br />
CLASSPATH: ${moduleDir}/target/home-library-${version}-SNAPSHOT.jar<br />
CLASSPATH: ${moduleDir}/SoftwareGateway-stubs/target/softwaregateway-stubs-${version}-${maven.system-version}.jar<br />
CLASSPATH: ${moduleDir}/SoftwareGateway-stubs/target/softwaregateway-stubs-${version}-SNAPSHOT.jar <br />
...<br />
</pre><br />
<br />
'''Tip''': you can easily find exactly which artifact is created during an ETICS build by searching in the build report of the component for the line "''Installing /home/gbuilder/catalogueoflife-spd-plugin/<span style="background-color:#E6E6FA">target/catalogue-of-life-spd-plugin-1.0.0-2.12.0.jar</span> to...''"<br />
<br />
<br />
= How to submit remote builds of gcube subsystems/components =<br />
A pre-requisite to submit a build is to have right to do that. [[Role Developer|Developers]] have rights to build their components, [[Role Subsystem Manager|Subsystem Managers]] have rights to build their subsystem and the [[Role Release Manager|Release Manager]] have the rights to build the entire project.<br />
<br />
<br />
In ETICS Portal, to access the Remote Build submitting form (see picture below) just '''right-click''' on the configuration you wish to build and then select '''"Remote Build..."''' from the contextual menu that will appear. The submission form is shown below:<br />
<br />
[[File:Etic-build-submission.png|center|frame]]<br />
<br />
<br />
<br />
The most common options to set for gCube builds are listed in the table below.<br />
{| border="1" cellpadding="4" cellspacing="0"<br />
|- style="background-color:#F2F2F2"<br />
! width=200 | option <br />
! width=500 | description<br />
! width=80 | usage<br />
|-<br />
| '''verbose''' <br />
| useful to debug the build process<br />
| recommended<br />
|-<br />
| '''Ignore Locking''' <br />
| A locked configuration in ETICS has a "snapshot" of the gCube configuration in which it has been locked. When a locked configuration is built in a different version of gCube of the one in which has been locked, it will try to bring in the build all the configurations belonging to the "snapshot"; most likely this will result in multiple configurations for the same component that will break the build. The usage of "Ignore Locking" option will prevent configurations from the snapshots of locked configuration to be brought in the build.<br />
| mandatory<br />
|-<br />
| '''Propagate environment and ...'''<br />
| essential. It will instruct ETICS about the gCube version in which context (gCube's version) build the configuration.<br />
| mandatory<br />
|-<br />
| '''Checkout from Volatile area'''<br />
| every ETICS build stores the binary artifacts in a remote area. By setting this option the build can use, if they exist, binary artifacts from past builds instead of re-compiling it from source (it only applies to dependencies, not on the build target component). <br />
| recommended<br />
|-<br />
| '''Do not stop on errors''' <br />
| ETICS will try to complete the build also in case the build of one or more components fails.<br />
| recommended<br />
|-<br />
| '''Host selection'''<br />
| The platform against which the build will be performed. gCube is completely in java and, thus, it is completely cross-platform and, theoretically, any of available platforms can be chosen. However, '''currently only CentOS 6.10 is available for gCube builds.'''<br />
| mandatory<br />
|}<br />
<br />
<br />
Once options have been set, just click on the ''submit'' icon (top-left of the submission form) and the build will start. You can keep trace of your builds in the '''Submissions'' tab of ETICS Portal.<br />
<br />
= ETICS Issues =<br />
The ''ETICS Issues'' section (reachable from the top-bar, on the left) lists the issues found in components during the build. Issues are related not only to the result of the builds but also to quality guidelines defined by projects. Some issues specific for gCube have been defined (e.g. compliance of README files) and are reported in this list.<br />
<br />
As depicted in the figure below, the user can group (in the top-right of the page) the issue list by different attributes: module, priority, type and owner. A summary for each group is displayed at the top of the page reporting the type and subtype of the issue, the total number of issues in the group and the sum of the technical debt computed for the issues.<br />
<br />
[[File:etics-issues-tutorial-1.png|center|800px]]<br />
<br />
<br />
<br />
Under the groups summary, there is the list of all issues ordered by group. By clicking on the issue, the details are displayed (see figure below).<br />
<br />
[[File:etics-issues-tutorial-2.png|center|800px]]<br />
<br />
<br />
== Issue Owners ==<br />
For each issue, ETICS computes a list of owners. It is the list of users that have permissions on the component associated to the issue order by proximity: users that have permissions at component-level are listed first, then the users that have permission on the component's subystem and finally users that have permission at level of project.<br />
<br />
<br />
== Issue Lifecycle ==<br />
ETICS runs a number of checkers during the remote builds and creates the discovered issues at the end of each build. An issue is automatically closed by ETICS when it runs again the build of the affected component and does not find the issue anymore. Closed issues appears with in Straight-through style for few days and then disappear completely.<br />
<br />
Some checks run only during BTRT builds and, therefore, are updated only during BTRT builds.<br />
<br />
= ETICS Continuous Builds =<br />
Along the remote builds submitted by the users on demand (see [[#How_to_submit_remote_builds_of_gcube_subsystems.2Fcomponents|here]]) ETICS has also the capability to trigger builds when specific events occurred.<br />
<br />
There are two types of events monitored:<br />
* changes in the ETICS data model (e.g. the user modify build commands, the user modify a dependency);<br />
* changes in the VCS: new commits in the source code repository;<br />
<br />
When one of these two events occurs, ETICS look for the ETICS Configuration(s) associated with the event and request a build for that configuration. The build submission is delayed of 5 minutes from the event. During this time, any other request of building the same configuration is ignored. This avoid multiple builds when several changes and/or several commits are done in a very short period of time.<br />
<br />
== Limitations ==<br />
At the moment the user that made the change is not notified when the build is launched and/or finish.<br />
<br />
= Dismissed Modules =<br />
ETICS does not support the removal of modules (projects, subsystems and components) if they (or they children) have one or more locked configurations.<br />
<br />
However, in big and long running projects some modules could be abandoned, replaced by new ones or dismissed. The number of dismissed modules can increase a lot and "pollute" the GUI with several unneeded modules and configurations.<br />
<br />
For this reason, there is an option to hide dismissed modules in the ETICS GUI (only hide, they will remain in the database).<br />
<br />
== How to define dismissed modules ==<br />
To flag a module as dismissed, it is enough to select the "Set Dismissed" action in the contextual menù in the project tree. The action will be enabled only if the user has the edit privileges on the module.<br />
<br />
[[image:set-dismissed.png||alt="etics-dismiss-module.png"||center||width="350"]]<br />
<br />
== How to show/hide dismissed modules ==<br />
To show/hide dismissed modules in the ETICS GUI there is a toggle option. It is located in the settings menù, in the toolbar as shown in the figure below.<br />
<br />
[[image:ETICS-GUI-dismissed-option.png||height="197"||center]]<br />
<br />
The option is per-user and persisted server side. The default for users and for not authenticated users is to hide dismissed modules.<br />
<br />
This option affects the ProjectTree (on the left-side of the GUI) and the list of submodules shown when editing the subconfigurations. In the latter case, please note that even if a module is flagged as dismissed, but one of its configurations is attached as subconfiguration, it will be shown in any case.<br />
<br />
== Under the hood ==<br />
<br />
Dismissed modules are flagged by using the "#dismissed" tag in the '''description''' of the module. Any module that contains this tag in the description will be considered as dismissed. The option "Set Dismissed" is a shortcut to add such a tag in the description. If the tag is removed, the module will not be considered dismissed anymore.<br />
<br />
= ETICS ModelSync =<br />
<br />
This functionality allows to automatically update ETICS model and keep it in sync with changes in the source code.<br />
<br />
In general terms, it works in the following way:<br />
# each ETICS Configuration is associated with a particular location on VCS (e.g. SVN folder, Git commit, ...) computed using the checkout commands and properties defined in ETICS by the users;<br />
# a specific module (namely ProjectParser) crawls the VCS location looking for Maven project descriptors (i.e. pom.xml files). The ProjectParser is general enough to allows also other project types (e.g. Ant, Python distuitls, Ivy, ...), but currently only parsing of Maven pom.xml files is supported;<br />
# The ProjectParser extract from the pom.xml the component information like coordinates, version, dependencies, description, etc, ...<br />
# The ModelSync sets the version of the ETICS Configuration to the version found in the pom.xml;<br />
# The ModelSync adds (or updates, if it already exists) the ".coords" property in the ETICS Configuration with the coordinates found in the pom.xml analysed. This realizes a mapping between ETICS Configurations and Maven coordinates. This mapping will be useful later when setting ETICS dependencies. Moreover, this mapping enables Maven-aware search operations like: "What ETICS Configuration provides a given maven coordinates?", "What artifact is produced by a given ETICS Configuration?";<br />
# The ETICS ModelSync adds the dependencies in the ETICS Configuration. For each dependency found in the pom.xml, a search is performed in ETICS to find the component that provides the dependency based on its coordinates. If one or more components are found, they are added as dependency in ETICS. This works not only for dependencies but also for the parent and dependencyManagement (e.g. boms) section in the pom.xml;<br />
# ETICS dependencies that are not found in the pom.xml are deleted;<br />
# It may happens that a given dependency found in the pom.xml is not added by the ETICS ModelSync because no ETICS Configurations that provides the dependency are found. This could be simply caused by the fact that the ProjectParser still did not analysed the pom.xml of the dependency component. In this case, ETICS remembers this "unresolved" dependency and as soon as the searched coordinates are found in an ETICS Configuration, the dependency is added;<br />
# In case multiple projects (i.e. pom.xml files) are detected in VCS location (e.g. multi-module projects like in case of gCube services), the build command is analysed to understand which Maven projects are actually built by the component;<br />
<br />
This process is triggered automatically every time a change in ETICS or in the VCS is detected (e.g. the ETICS Configuration has been modified, a new commit in SVN occurred).<br />
<br />
<br />
<br />
By default, this process is enabled for all configurations. However, its behaviour can be controlled through a set of ETICS properties that can be set in the ETICS Configurations:<br />
* "reactor.ignore=true" disables any action on the configuration (including automatic builds). It is usually automatically set when ETICS finds anomalies on the configuration (e.g. a configuration that points to the trunk)<br />
* "model.sync.enabled=false" disables completely the synchronization<br />
* "model.sync.coordinates.enabled=false" disables synchronization of the ".coords" property<br />
* "model.sync.version.enabled=false" disable synchronization of version<br />
* "model.sync.dependencies.enabled=false" disables the synchronization of dependencies<br />
* "model.sync.dependencies.ignore=<etics-module-name-1>;<etics-module-name-2>;..." a ";" separated list of ETICS module names to ignore while synchronizing dependencies. Dependencies on ETICS modules in this list are neither added nor deleted automatically; <br />
<br />
== Current limitations ==<br />
* currently only Maven projects are synchronised. For Ant components sync is not (and will be not) implemented;<br />
<br />
== How to use it ==<br />
The rational behind this functionality is to automatically react to changes in the VCS and to update ETICS model accordingly. This is intended to be completely automatic without the interaction of users. The only way users can interact with the synchronisation mechanism is through setting of the control properties. When the synchronisation is enabled, users should not modify manually ETICS Configuration version and dependencies because they will be revert by the synchronisation mechanism.<br />
<br />
What to do when a new component is created on ETICS? It is enough to set the ETICS checkout and build commands for the ETICS Configuration and the ModelSync will be able to automatically set (and keep aligned) the version and the dependencies.<br />
<br />
= External Links =<br />
[https://etics.esl.eng.it/etics-gui/ Etics Web Portal]<br />
<br />
[http://eticsbuild2.research-infrastructures.eu/BuildReport/home/AllBuilds Etics Build Report]<br />
<br />
[https://twiki.cern.ch/twiki/bin/view/ETICS/ClientHowTo Etics Client Installation HowTo]</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Software_Integration_and_Distribution:_Release_Log&diff=31421Software Integration and Distribution: Release Log2018-08-09T07:30:43Z<p>Gabriele.giammatteo: </p>
<hr />
<div>This page publishes information and statistics of all gCube releases.<br />
<br />
{| border="1" cellpadding="4" cellspacing="0"<br />
|- style="background-color:#F2F2F2"<br />
! rowspan=2|Release !! rowspan=2|Patch !! colspan=3|Duration !! colspan=5|Size<br />
|- style="background-color:#F2F2F2"<br />
! width=100|Start !! width=100|End !! width=100|Number Days !! width=100|Total SubSys !! width=100|New+Updated SubSys || width=100|Total SAs || width=100|Total Comps || width=150|New+Updated Comps<br />
|-<br />
| rowspan=1|1.6 || 1.6.1 || 27 Jan 2010 || 27 Jan 2010 || 1 || 21 || 1 || 181 || n/a || n/a<br />
|-<br />
| rowspan=5|1.7 || 1.7.0 || 14 Jan 2010 || 11 Feb 2010 || 29 || 21 || 18 || 179|| n/a || n/a<br />
|-<br />
| 1.7.1 || 12 Feb 2010 || 17 Feb 2010 || 6 || 21 || 5 || 179 || n/a || n/a<br />
|-<br />
| 1.7.2 || 22 Feb 2010 || 24 Feb 2010 || 3 || 21 || 6 || 179 || n/a || n/a<br />
|-<br />
| 1.7.3 || 09 Apr 2010 || 16 Apr 2010 || 8 || 21 || 3 || 179|| n/a || n/a<br />
|-<br />
| 1.7.4 || 30 Apr 2010 || 06 May 2010 || 7 || 21 || 4 || 179 || n/a || n/a<br />
|-<br />
| rowspan=2 |1.8 || 1.8.0 [[Image:changelog_icon.png|16px|link=Release_.1-8-0_Changelog]] || 14 May 2010 || 01 Jun 2010 || 19 || 22 || 19 || 189 || n/a || n/a<br />
|-<br />
| 1.8.1 [[Image:changelog_icon.png|16px|link=Release_.1-8-1_Changelog]] || 25 June 2010 || 01 Jul 2010 || 07 || 22 || 11 || 189 || n/a || n/a<br />
|-<br />
| rowspan=2 | 1.9 || 1.9.0 [[Image:changelog_icon.png|16px|link=Release_.1-9-0]] || 16 Jul 2010 || 28 Jul 2010 || 13 || 23 || 14 || 197 || n/a || n/a<br />
|-<br />
| 1.9.1 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.1-9-1]] || 13 Sep 2010 || 13 Sep 2010 || 1 || 23 || 5 || 200 || n/a || n/a<br />
|-<br />
| rowspan=3 | 2.0 || 2.0.0 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-0-0]] || 03 Sep 2010 || 15 Sep 2010 || 13 || 23 || 12 || 207 || n/a || n/a<br />
|-<br />
| 2.0.1 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2-0-1]] || 29 Sep 2010 || 05 Oct 2010 || 7 || 23 || 3 || 207 || n/a || n/a<br />
|-<br />
| 2.0.2 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2-0-2]] || 15 Oct 2010 || 21 Oct 2010 || 7 || 23 || 4 || 208 || n/a || n/a<br />
|-<br />
| rowspan=1 | 2.1 || 2.1.0 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-1-0]] || 23 Oct 2010 || 03 Nov 2010 || 12 || 23 || 15 || 209 || n/a || n/a<br />
|-<br />
| rowspan=3 | 2.2 || 2.2.0 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-2-0]] || 09 Nov 2010 || 12 Nov 2010 || 4 || 23 || 8 || 212 || n/a || n/a<br />
|-<br />
| 2.2.1 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-2-1]] || 26 Nov 2010 || 02 Dec 2010 || 7 || 23 || 12 || 212 || n/a || n/a<br />
|-<br />
| 2.2.2 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-2-2]] || 14 Jan 2011 || 17 Jan 2011 || 4 || 23 || 3 || 212 || n/a || n/a<br />
|-<br />
| rowspan=2 | 2.3 || 2.3.0 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-3-0]] || 07 Feb 2011 || 03 Mar 2011 || 24 || 23 || 16 || 229 || n/a || n/a<br />
|-<br />
| 2.3.1 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-3-1]] || 25 Mar 2011 || 04 Apr 2011 || 10 || 23 || 9 || 230 || n/a || n/a<br />
|-<br />
| rowspan=2 | 2.4 || 2.4.0 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-4-0]] || 06 May 2011 || 06 June 2011 || 31 || 21 || 19 || 237 || n/a || n/a<br />
|-<br />
| 2.4.1 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-4-1]] || 10 Jun 2011 || 01 Jul 2011 || 21 || 21 || 9 || 238 || n/a || n/a<br />
|-<br />
| rowspan=2 | 2.5 || 2.5.0 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-5-0]] || 01 Jul 2011 || 29 Jul 2011 || 29 || 21 || 18 || 243 || n/a || n/a<br />
|-<br />
| 2.5.1 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-5-1]] || 2 Aug 2011 || 02 Sept 2011 || 31 || 21 || 10 || 243 || n/a || n/a<br />
|-<br />
| rowspan=1 | 2.6 || 2.6.0 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-6-0]] || 29 Jul 2011 || 30 Sept 2011 || 62 || 21 || 10 || 249 || n/a || n/a<br />
|-<br />
| rowspan=3 | 2.7 || 2.7.0 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-7-0]] || 02 Sept 2011 || 30 Sept 2011 || 28 || 21 || 16 || 257 || n/a || n/a<br />
|-<br />
| 2.7.1 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-7-1]] || 5 Nov 2011 || 21 Nov 2011 || 17 || 21 || 9 || 257 || n/a || n/a<br />
|-<br />
| 2.7.2 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-7-2]] || 16 Dec 2011 || 23 Dec 2011 || 8 || 21 || 7 || 258 || n/a || n/a<br />
|- style="background-color:#F2F2F2"<br />
! rowspan=2|Release !! rowspan=2|Patch !! colspan=3|Duration !! colspan=5|Size<br />
|- style="background-color:#F2F2F2"<br />
! width=100|Start !! width=100|End !! width=100|Number Days !! width=100|Total SubSys !! width=100|New+Updated SubSys || width=100|Total SAs || width=100|Total Comps || width=150|New+Updated Comps<br />
|-<br />
| rowspan=1 |2.7 || 2.7.3 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-7-3]] || 28 Jan 2012 || 06 Feb 2012 || 10 || 21 || 7 || 258 || 526 || 18<br />
|-<br />
| rowspan=2 |2.8 || 2.8.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2-8-0]] || 20 Feb 2012 || 08 Mar 2012 || 18 || 23 || 16 || 268 || 548 || 148<br />
|-<br />
| 2.8.1 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2-8-1]] || 6 Apr 2012 || 20 Apr 2012 || 14 || 23 || 14 || 271 || 554 || 51<br />
|-<br />
| rowspan=2 |2.9 || 2.9.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2.9.0]] || 04 May 2012 || 10 Jul 2012 || 67 || 23 || 22 || n/a || 512 || 245<br />
|-<br />
| 2.9.1 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2-9-1]] || 24 Jul 2012 || 27 Jul 2012 || 4 || 23 || 4 || n/a || 511 || 22<br />
|-<br />
| rowspan=1 |2.10 || 2.10.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2.10.0]] || 14 Sept 2012 || 17 Oct 2012 || 32 || 28 || 24 || n/a || 524 || 246<br />
|-<br />
| rowspan=2 |2.11 || 2.11.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2.11.0]] || 19 Oct 2012 || 13 Nov 2012 || 24 || 29 || 15 || n/a || 547 || 118<br />
|-<br />
| 2.11.1 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2.11.1]] || 1 Dec 2012 || 8 Dec 2012 || 8 || 29 || 10 || n/a || 548 || 60<br />
|-<br />
| rowspan=1 |2.12 || 2.12.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2.12.0]] || 17 Jan 2013 || 26 Feb 2013 || 41 || 30 || 23 || n/a || 561 || 233<br />
|-<br />
| rowspan=1 |2.13 || 2.13.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2.13.0]] || 05 Mar 2013 || 24 Mar 2013 || 20 || 31 || 11 || n/a || 558 || 80<br />
|-<br />
| rowspan=1 |2.14 || 2.14.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2.14.0]] || 22 Apr 2013 || 21 May 2013 || 30 || 31 || 24 || n/a || 551 || 172<br />
|-<br />
| rowspan=1 |2.15 || 2.15.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2.15.0]] || 31 May 2013 || 18 Jun 2013 || 19 || 31 || 15 || n/a || 550 || 69<br />
|-<br />
| rowspan=2 |2.16 || 2.16.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2.16.0]] || 9 July 2013 || 30 July 2013 || 22 || 29 || 22 || n/a || 461 || 161<br />
|-<br />
| 2.16.1 [[Image:changelog_icon.png|16px|link=Release_Notes_.2.16.1]] || 13 Sep 2013 || 24 Sep 2013 || 12 || 29 || 12 || n/a || 461 || 42<br />
|-<br />
| rowspan=3 |2.17 || 2.17.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2.17.0]] || 21 Oct 2013 || 29 Nov 2013 || 40 || 29 || 27 || n/a || 470 || 251<br />
|-<br />
| 2.17.1 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2.17.1]] || 14 Dec 2013 || 8 Jan 2014 || 26 || 29 || 9 || n/a || 472 || 24<br />
|-<br />
| 2.17.2 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2.17.2]] || 30 Jan 2014 || 3 Feb 2014 || 4 || 29 || 10 || n/a || 472 || 14<br />
|-<br />
| rowspan=1 |3.0 || 3.0.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.3.0.0]] || 27 Jan 2014 || 30 Mar 2014 || 63 || 29 || 25 || n/a || 497 || 237<br />
|-<br />
| rowspan=2 |3.1 || 3.1.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.3.1.0]] || 31 Mar 2014 || 21 May 2014 || 52 || 29 || 21 || n/a || 511 || 183<br />
|-<br />
| 3.1.1 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.3.1.1]] || 22 May 2014 || 03 June 2014 || 13 || 29 || 6 || n/a || 511 || 14<br />
|-<br />
| rowspan=1 |3.2 || 3.2.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.3.2.0]] || 04 Jun 2014 || 19 Jun 2014 || 16 || 29 || 17 || n/a || 518 || 184<br />
|-<br />
| rowspan=1 |3.3 || 3.3.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.3.3.0]] || 30 Jun 2014 || 17 July 2014 || 18 || 29 || 17 || n/a || 537 || 137<br />
|-<br />
| rowspan=1 |3.4 || 3.4.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.3.4.0]] || 10 Sep 2014 ||25 Sep 2014 || 16 || 29 || 18 || n/a || 546 || 122<br />
|-<br />
| rowspan=2 |3.5 || 3.5.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.3.5.0]] || 24 Oct 2014 ||12 Nov 2014 || 20 || 29 || 20 || n/a || 556 || 114<br />
|-<br />
| 3.5.1 [[Image:changelog_icon.png|16px|link=Release_Notes_.3.5.1]] || 05 Dec 2014 || 16 Dec 2014 || 12 || 29 || 12 || n/a || 558 || 65<br />
|-<br />
| rowspan=1 |3.6 || 3.6.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.3.6.0]] || 16 Feb 2015 || 26 Feb 2015 || 11 || 29 || 6 || n/a || 562 || 65<br />
|-<br />
| rowspan=1 |3.7 || 3.7.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.3.7.0]] || 30 Apr 2015 || 14 May 2015 || 15 || 29 || 18 (18 upd) || n/a || 567 || 98 (11 new + 87 upd)<br />
|-<br />
| rowspan=1 |3.8 || 3.8.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.3.8.0]] || 03 Jul 2015 || 21 Jul 2015 || 19 || 29 || 15 (15 upd) || n/a || 575 || 118 (9 new + 109 upd)<br />
|- style="background-color:#F2F2F2"<br />
! rowspan=2 | Release !! rowspan=2|Patch !! Start || End || Number Days || Total SubSys || New+Updated SubSys || Total SAs || Total Comps || New+Updated Comps<br />
|- style="background-color:#F2F2F2"<br />
! colspan=3|Duration !! colspan=5|Size<br />
|-<br />
| rowspan=1 |3.9 || 3.9.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.3.9.0]] || 28 Sep 2015 || 02 Dec 2015 || 64 || 28 || 17 (17 upd) || n/a || 574 || 134 (16 new + 118 upd)<br />
|-<br />
| rowspan=2 |3.10 || 3.10.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.3.10.0]] || 04 Dec 2015 || 08 Feb 2016 || 67 || 27 || 22 (22 upd)|| n/a || 536 || 146 (49 new + 86 upd - 11 drp)<br />
|-<br />
| 3.10.1 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.3.10.1]] || 17 Feb 2016|| 07 Apr 2016 || 51 || 26 || 26 || n/a || 525 || 510 (4 new + 487 upd + 19 drp)<br />
|-<br />
| rowspan=1 |3.11 || 3.11.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.3.11.0]] || 04 Apr 2016 || 24 May 2016 || 51 || 24 ||24 (24 upd)|| n/a || 535 || 108 (17 new + 84 upd (new version) + 7 drp)<br />
|-<br />
| rowspan=1 |4.0 || 4.0.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.0.0]] || 20 June 2016 ||26 July 2016 ||37||21 ||18(17 upd(new version),1 new ||n/a ||544||210(29 new, 161 upd(new version), 20 drp) <br />
|-<br />
| rowspan=2 |4.1 || 4.1.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.1.0]] || 12 September 2016 || 04 November 2016||53||26 ||26 (22 upd (new version), 2 new) ||n/a||575||231(170 upd(new version), 46 new, 15 drp) <br />
|-<br />
| 4.1.1 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.1.1]] || 21 November 2016 || 22 November 2016||2||26 || 2 upd (new version) ||n/a||575||2 upd (new version)<br />
|-<br />
| rowspan=2 |4.2 ||4.2.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.2.0]] || 25 November 2016 || 13 December 2016|| 19 || 26 || 11 upd ||n/a|| 585 ||11 new, 109 upd, 1 del<br />
|-<br />
| 4.2.1 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.2.1]] || 2 February 2017|| 10 February 2017 || 8 || 26 || 7 upd||n/a|| 586 || 26 (1 new, 25 upd, 0 del)<br />
|-<br />
| rowspan=1 |4.3 ||4.3.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.3.0]] || 20 January 2017 || 17 March 2017|| 59 || 29 || 17 upd, 1 del [org..data-harmonization], 1 new [org..resource-management]||n/a|| 582 || 207 (12 new, 189 upd (new version), 16 drp)<br />
|-<br />
| rowspan=1 |4.4 ||4.4.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.4.0]] || 20 March 2017 || 28 April 2017|| 40 || 26 || 26 upd||n/a|| 491 || 214 (9 new, 95 upd (new version), 100 drp)<br />
|-<br />
| rowspan=1 |4.5 ||4.5.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.5.0]] || 02 May 2017 || 09 June 2017|| 40 || 22 || 22 upd||n/a|| 485 || 485 (10 new, 94 upd (new version), 16 drp)<br />
|-<br />
| rowspan=2 |4.6 ||4.6.0 [[Image:changelog_icon.png|16px|link=Release_Notes_.4.6.0]] || 12 June 2017 || 24 July 2017|| 43 days || 30 || 17 subsystems update||n/a|| 495 components || 82 (13 new, 63 upd (new version), 3 drp)<br />
|-<br />
| 4.6.1 [[Image:changelog_icon.png|16px|link=Release_Notes_.4.6.1]] || 25 July 2017|| 19 September 2017 || 23 days || 30 || 20 upd||n/a|| 502 || 502 (14 new, 69 upd, 0 del)<br />
|-<br />
| rowspan=2 |4.7 ||4.7.0 [[Image:changelog_icon.png|16px|link=Release_Notes_.4.7.0]] || 28 August 2017 || 06 October 2017|| 40 days || 30 || 16 subsystems update||n/a|| 497 components || 497 (1 new, 58 upd, 6 drp)<br />
|-<br />
| 4.7.1 [[Image:changelog_icon.png|16px|link=Release_Notes_.4.7.1]] || 16 October 2017|| 18 October 2017 || 3 days || 30 || 1 upd||n/a|| 497 || 497 (0 new, 1 upd, 0 del)<br />
|-<br />
| rowspan=1 |4.8 ||4.8.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.8.0]] || 11 October 2017 || 29 November 2017|| 49 || 30 || 18 upd||n/a|| 501 || 501( 4 new, 47 upd , 0 del)<br />
|-<br />
| rowspan=1 |4.9 ||4.9.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.9.0]] || 28 November 2017 || 19 December 2017|| 22 || 30 || 11 upd||n/a|| 504 || 504 (3 new, 35 upd (new version))<br />
|-<br />
| rowspan=1 |4.10.0 ||4.10.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.10.0]] || 18 December 2017 || 14 February 2018|| 58 || 30|| 15||n/a|| 519|| 519 (15 new, 57 upd (new version), 0 drp)<br />
|-<br />
| rowspan=2 |4.11 ||4.11.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.11.0]] || 12 February 2018 || 12 April 2018|| 60 || 30|| 13||n/a|| 521|| 48 (2 new, 46 upd (new version), 0 drp)<br />
|-<br />
| 4.11.1 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.11.1]] || 16 April 2018 || 29 May 2018|| 43|| 30|| 13||n/a|| 526|| 56 (7 new, 49 upd (new version), 0 drp)<br />
|-<br />
| rowspane= 1 | 4.12.0 || 4.12.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.12.0]] || xx April 2018 || 01 August 2018|| number of days|| 30 || 12||n/a|| 530|| xy component (8 new, 46 upd (new version), 4 drp)<br />
|-<br />
| rowspane= 1 | 4.13.0 || 4.13.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.13.0]] || xx July 2018 || xx month 2018|| number of days|| total subsystems|| xx subsystem update||n/a|| xxx total component|| xy component (x new, x upd (new version), x drp)<br />
|}<br />
<br />
<br />
''Clicking on [[Image:changelog_icon.png|16px]] icon, where available, you can access release changelog''</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Software_Integration_and_Distribution:_Release_Log&diff=31420Software Integration and Distribution: Release Log2018-08-09T07:29:08Z<p>Gabriele.giammatteo: </p>
<hr />
<div>This page publishes information and statistics of all gCube releases.<br />
<br />
{| border="1" cellpadding="4" cellspacing="0"<br />
|- style="background-color:#F2F2F2"<br />
! rowspan=2|Release !! rowspan=2|Patch !! colspan=3|Duration !! colspan=5|Size<br />
|- style="background-color:#F2F2F2"<br />
! width=100|Start !! width=100|End !! width=100|Number Days !! width=100|Total SubSys !! width=100|New+Updated SubSys || width=100|Total SAs || width=100|Total Comps || width=150|New+Updated Comps<br />
|-<br />
| rowspan=1|1.6 || 1.6.1 || 27 Jan 2010 || 27 Jan 2010 || 1 || 21 || 1 || 181 || n/a || n/a<br />
|-<br />
| rowspan=5|1.7 || 1.7.0 || 14 Jan 2010 || 11 Feb 2010 || 29 || 21 || 18 || 179|| n/a || n/a<br />
|-<br />
| 1.7.1 || 12 Feb 2010 || 17 Feb 2010 || 6 || 21 || 5 || 179 || n/a || n/a<br />
|-<br />
| 1.7.2 || 22 Feb 2010 || 24 Feb 2010 || 3 || 21 || 6 || 179 || n/a || n/a<br />
|-<br />
| 1.7.3 || 09 Apr 2010 || 16 Apr 2010 || 8 || 21 || 3 || 179|| n/a || n/a<br />
|-<br />
| 1.7.4 || 30 Apr 2010 || 06 May 2010 || 7 || 21 || 4 || 179 || n/a || n/a<br />
|-<br />
| rowspan=2 |1.8 || 1.8.0 [[Image:changelog_icon.png|16px|link=Release_.1-8-0_Changelog]] || 14 May 2010 || 01 Jun 2010 || 19 || 22 || 19 || 189 || n/a || n/a<br />
|-<br />
| 1.8.1 [[Image:changelog_icon.png|16px|link=Release_.1-8-1_Changelog]] || 25 June 2010 || 01 Jul 2010 || 07 || 22 || 11 || 189 || n/a || n/a<br />
|-<br />
| rowspan=2 | 1.9 || 1.9.0 [[Image:changelog_icon.png|16px|link=Release_.1-9-0]] || 16 Jul 2010 || 28 Jul 2010 || 13 || 23 || 14 || 197 || n/a || n/a<br />
|-<br />
| 1.9.1 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.1-9-1]] || 13 Sep 2010 || 13 Sep 2010 || 1 || 23 || 5 || 200 || n/a || n/a<br />
|-<br />
| rowspan=3 | 2.0 || 2.0.0 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-0-0]] || 03 Sep 2010 || 15 Sep 2010 || 13 || 23 || 12 || 207 || n/a || n/a<br />
|-<br />
| 2.0.1 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2-0-1]] || 29 Sep 2010 || 05 Oct 2010 || 7 || 23 || 3 || 207 || n/a || n/a<br />
|-<br />
| 2.0.2 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2-0-2]] || 15 Oct 2010 || 21 Oct 2010 || 7 || 23 || 4 || 208 || n/a || n/a<br />
|-<br />
| rowspan=1 | 2.1 || 2.1.0 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-1-0]] || 23 Oct 2010 || 03 Nov 2010 || 12 || 23 || 15 || 209 || n/a || n/a<br />
|-<br />
| rowspan=3 | 2.2 || 2.2.0 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-2-0]] || 09 Nov 2010 || 12 Nov 2010 || 4 || 23 || 8 || 212 || n/a || n/a<br />
|-<br />
| 2.2.1 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-2-1]] || 26 Nov 2010 || 02 Dec 2010 || 7 || 23 || 12 || 212 || n/a || n/a<br />
|-<br />
| 2.2.2 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-2-2]] || 14 Jan 2011 || 17 Jan 2011 || 4 || 23 || 3 || 212 || n/a || n/a<br />
|-<br />
| rowspan=2 | 2.3 || 2.3.0 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-3-0]] || 07 Feb 2011 || 03 Mar 2011 || 24 || 23 || 16 || 229 || n/a || n/a<br />
|-<br />
| 2.3.1 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-3-1]] || 25 Mar 2011 || 04 Apr 2011 || 10 || 23 || 9 || 230 || n/a || n/a<br />
|-<br />
| rowspan=2 | 2.4 || 2.4.0 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-4-0]] || 06 May 2011 || 06 June 2011 || 31 || 21 || 19 || 237 || n/a || n/a<br />
|-<br />
| 2.4.1 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-4-1]] || 10 Jun 2011 || 01 Jul 2011 || 21 || 21 || 9 || 238 || n/a || n/a<br />
|-<br />
| rowspan=2 | 2.5 || 2.5.0 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-5-0]] || 01 Jul 2011 || 29 Jul 2011 || 29 || 21 || 18 || 243 || n/a || n/a<br />
|-<br />
| 2.5.1 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-5-1]] || 2 Aug 2011 || 02 Sept 2011 || 31 || 21 || 10 || 243 || n/a || n/a<br />
|-<br />
| rowspan=1 | 2.6 || 2.6.0 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-6-0]] || 29 Jul 2011 || 30 Sept 2011 || 62 || 21 || 10 || 249 || n/a || n/a<br />
|-<br />
| rowspan=3 | 2.7 || 2.7.0 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-7-0]] || 02 Sept 2011 || 30 Sept 2011 || 28 || 21 || 16 || 257 || n/a || n/a<br />
|-<br />
| 2.7.1 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-7-1]] || 5 Nov 2011 || 21 Nov 2011 || 17 || 21 || 9 || 257 || n/a || n/a<br />
|-<br />
| 2.7.2 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-7-2]] || 16 Dec 2011 || 23 Dec 2011 || 8 || 21 || 7 || 258 || n/a || n/a<br />
|- style="background-color:#F2F2F2"<br />
! rowspan=2|Release !! rowspan=2|Patch !! colspan=3|Duration !! colspan=5|Size<br />
|- style="background-color:#F2F2F2"<br />
! width=100|Start !! width=100|End !! width=100|Number Days !! width=100|Total SubSys !! width=100|New+Updated SubSys || width=100|Total SAs || width=100|Total Comps || width=150|New+Updated Comps<br />
|-<br />
| rowspan=1 |2.7 || 2.7.3 [[Image:changelog_icon.png|16px|link=Release_Notes_.2-7-3]] || 28 Jan 2012 || 06 Feb 2012 || 10 || 21 || 7 || 258 || 526 || 18<br />
|-<br />
| rowspan=2 |2.8 || 2.8.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2-8-0]] || 20 Feb 2012 || 08 Mar 2012 || 18 || 23 || 16 || 268 || 548 || 148<br />
|-<br />
| 2.8.1 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2-8-1]] || 6 Apr 2012 || 20 Apr 2012 || 14 || 23 || 14 || 271 || 554 || 51<br />
|-<br />
| rowspan=2 |2.9 || 2.9.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2.9.0]] || 04 May 2012 || 10 Jul 2012 || 67 || 23 || 22 || n/a || 512 || 245<br />
|-<br />
| 2.9.1 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2-9-1]] || 24 Jul 2012 || 27 Jul 2012 || 4 || 23 || 4 || n/a || 511 || 22<br />
|-<br />
| rowspan=1 |2.10 || 2.10.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2.10.0]] || 14 Sept 2012 || 17 Oct 2012 || 32 || 28 || 24 || n/a || 524 || 246<br />
|-<br />
| rowspan=2 |2.11 || 2.11.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2.11.0]] || 19 Oct 2012 || 13 Nov 2012 || 24 || 29 || 15 || n/a || 547 || 118<br />
|-<br />
| 2.11.1 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2.11.1]] || 1 Dec 2012 || 8 Dec 2012 || 8 || 29 || 10 || n/a || 548 || 60<br />
|-<br />
| rowspan=1 |2.12 || 2.12.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2.12.0]] || 17 Jan 2013 || 26 Feb 2013 || 41 || 30 || 23 || n/a || 561 || 233<br />
|-<br />
| rowspan=1 |2.13 || 2.13.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2.13.0]] || 05 Mar 2013 || 24 Mar 2013 || 20 || 31 || 11 || n/a || 558 || 80<br />
|-<br />
| rowspan=1 |2.14 || 2.14.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2.14.0]] || 22 Apr 2013 || 21 May 2013 || 30 || 31 || 24 || n/a || 551 || 172<br />
|-<br />
| rowspan=1 |2.15 || 2.15.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2.15.0]] || 31 May 2013 || 18 Jun 2013 || 19 || 31 || 15 || n/a || 550 || 69<br />
|-<br />
| rowspan=2 |2.16 || 2.16.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2.16.0]] || 9 July 2013 || 30 July 2013 || 22 || 29 || 22 || n/a || 461 || 161<br />
|-<br />
| 2.16.1 [[Image:changelog_icon.png|16px|link=Release_Notes_.2.16.1]] || 13 Sep 2013 || 24 Sep 2013 || 12 || 29 || 12 || n/a || 461 || 42<br />
|-<br />
| rowspan=3 |2.17 || 2.17.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2.17.0]] || 21 Oct 2013 || 29 Nov 2013 || 40 || 29 || 27 || n/a || 470 || 251<br />
|-<br />
| 2.17.1 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2.17.1]] || 14 Dec 2013 || 8 Jan 2014 || 26 || 29 || 9 || n/a || 472 || 24<br />
|-<br />
| 2.17.2 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.2.17.2]] || 30 Jan 2014 || 3 Feb 2014 || 4 || 29 || 10 || n/a || 472 || 14<br />
|-<br />
| rowspan=1 |3.0 || 3.0.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.3.0.0]] || 27 Jan 2014 || 30 Mar 2014 || 63 || 29 || 25 || n/a || 497 || 237<br />
|-<br />
| rowspan=2 |3.1 || 3.1.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.3.1.0]] || 31 Mar 2014 || 21 May 2014 || 52 || 29 || 21 || n/a || 511 || 183<br />
|-<br />
| 3.1.1 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.3.1.1]] || 22 May 2014 || 03 June 2014 || 13 || 29 || 6 || n/a || 511 || 14<br />
|-<br />
| rowspan=1 |3.2 || 3.2.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.3.2.0]] || 04 Jun 2014 || 19 Jun 2014 || 16 || 29 || 17 || n/a || 518 || 184<br />
|-<br />
| rowspan=1 |3.3 || 3.3.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.3.3.0]] || 30 Jun 2014 || 17 July 2014 || 18 || 29 || 17 || n/a || 537 || 137<br />
|-<br />
| rowspan=1 |3.4 || 3.4.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.3.4.0]] || 10 Sep 2014 ||25 Sep 2014 || 16 || 29 || 18 || n/a || 546 || 122<br />
|-<br />
| rowspan=2 |3.5 || 3.5.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.3.5.0]] || 24 Oct 2014 ||12 Nov 2014 || 20 || 29 || 20 || n/a || 556 || 114<br />
|-<br />
| 3.5.1 [[Image:changelog_icon.png|16px|link=Release_Notes_.3.5.1]] || 05 Dec 2014 || 16 Dec 2014 || 12 || 29 || 12 || n/a || 558 || 65<br />
|-<br />
| rowspan=1 |3.6 || 3.6.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.3.6.0]] || 16 Feb 2015 || 26 Feb 2015 || 11 || 29 || 6 || n/a || 562 || 65<br />
|-<br />
| rowspan=1 |3.7 || 3.7.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.3.7.0]] || 30 Apr 2015 || 14 May 2015 || 15 || 29 || 18 (18 upd) || n/a || 567 || 98 (11 new + 87 upd)<br />
|-<br />
| rowspan=1 |3.8 || 3.8.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.3.8.0]] || 03 Jul 2015 || 21 Jul 2015 || 19 || 29 || 15 (15 upd) || n/a || 575 || 118 (9 new + 109 upd)<br />
|- style="background-color:#F2F2F2"<br />
! rowspan=2 | Release !! rowspan=2|Patch !! Start || End || Number Days || Total SubSys || New+Updated SubSys || Total SAs || Total Comps || New+Updated Comps<br />
|- style="background-color:#F2F2F2"<br />
! colspan=3|Duration !! colspan=5|Size<br />
|-<br />
| rowspan=1 |3.9 || 3.9.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.3.9.0]] || 28 Sep 2015 || 02 Dec 2015 || 64 || 28 || 17 (17 upd) || n/a || 574 || 134 (16 new + 118 upd)<br />
|-<br />
| rowspan=2 |3.10 || 3.10.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.3.10.0]] || 04 Dec 2015 || 08 Feb 2016 || 67 || 27 || 22 (22 upd)|| n/a || 536 || 146 (49 new + 86 upd - 11 drp)<br />
|-<br />
| 3.10.1 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.3.10.1]] || 17 Feb 2016|| 07 Apr 2016 || 51 || 26 || 26 || n/a || 525 || 510 (4 new + 487 upd + 19 drp)<br />
|-<br />
| rowspan=1 |3.11 || 3.11.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.3.11.0]] || 04 Apr 2016 || 24 May 2016 || 51 || 24 ||24 (24 upd)|| n/a || 535 || 108 (17 new + 84 upd (new version) + 7 drp)<br />
|-<br />
| rowspan=1 |4.0 || 4.0.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.0.0]] || 20 June 2016 ||26 July 2016 ||37||21 ||18(17 upd(new version),1 new ||n/a ||544||210(29 new, 161 upd(new version), 20 drp) <br />
|-<br />
| rowspan=2 |4.1 || 4.1.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.1.0]] || 12 September 2016 || 04 November 2016||53||26 ||26 (22 upd (new version), 2 new) ||n/a||575||231(170 upd(new version), 46 new, 15 drp) <br />
|-<br />
| 4.1.1 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.1.1]] || 21 November 2016 || 22 November 2016||2||26 || 2 upd (new version) ||n/a||575||2 upd (new version)<br />
|-<br />
| rowspan=2 |4.2 ||4.2.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.2.0]] || 25 November 2016 || 13 December 2016|| 19 || 26 || 11 upd ||n/a|| 585 ||11 new, 109 upd, 1 del<br />
|-<br />
| 4.2.1 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.2.1]] || 2 February 2017|| 10 February 2017 || 8 || 26 || 7 upd||n/a|| 586 || 26 (1 new, 25 upd, 0 del)<br />
|-<br />
| rowspan=1 |4.3 ||4.3.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.3.0]] || 20 January 2017 || 17 March 2017|| 59 || 29 || 17 upd, 1 del [org..data-harmonization], 1 new [org..resource-management]||n/a|| 582 || 207 (12 new, 189 upd (new version), 16 drp)<br />
|-<br />
| rowspan=1 |4.4 ||4.4.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.4.0]] || 20 March 2017 || 28 April 2017|| 40 || 26 || 26 upd||n/a|| 491 || 214 (9 new, 95 upd (new version), 100 drp)<br />
|-<br />
| rowspan=1 |4.5 ||4.5.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.5.0]] || 02 May 2017 || 09 June 2017|| 40 || 22 || 22 upd||n/a|| 485 || 485 (10 new, 94 upd (new version), 16 drp)<br />
|-<br />
| rowspan=2 |4.6 ||4.6.0 [[Image:changelog_icon.png|16px|link=Release_Notes_.4.6.0]] || 12 June 2017 || 24 July 2017|| 43 days || 30 || 17 subsystems update||n/a|| 495 components || 82 (13 new, 63 upd (new version), 3 drp)<br />
|-<br />
| 4.6.1 [[Image:changelog_icon.png|16px|link=Release_Notes_.4.6.1]] || 25 July 2017|| 19 September 2017 || 23 days || 30 || 20 upd||n/a|| 502 || 502 (14 new, 69 upd, 0 del)<br />
|-<br />
| rowspan=2 |4.7 ||4.7.0 [[Image:changelog_icon.png|16px|link=Release_Notes_.4.7.0]] || 28 August 2017 || 06 October 2017|| 40 days || 30 || 16 subsystems update||n/a|| 497 components || 497 (1 new, 58 upd, 6 drp)<br />
|-<br />
| 4.7.1 [[Image:changelog_icon.png|16px|link=Release_Notes_.4.7.1]] || 16 October 2017|| 18 October 2017 || 3 days || 30 || 1 upd||n/a|| 497 || 497 (0 new, 1 upd, 0 del)<br />
|-<br />
| rowspan=1 |4.8 ||4.8.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.8.0]] || 11 October 2017 || 29 November 2017|| 49 || 30 || 18 upd||n/a|| 501 || 501( 4 new, 47 upd , 0 del)<br />
|-<br />
| rowspan=1 |4.9 ||4.9.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.9.0]] || 28 November 2017 || 19 December 2017|| 22 || 30 || 11 upd||n/a|| 504 || 504 (3 new, 35 upd (new version))<br />
|-<br />
| rowspan=1 |4.10.0 ||4.10.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.10.0]] || 18 December 2017 || 14 February 2018|| 58 || 30|| 15||n/a|| 519|| 519 (15 new, 57 upd (new version), 0 drp)<br />
|-<br />
| rowspan=2 |4.11 ||4.11.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.11.0]] || 12 February 2018 || 12 April 2018|| 60 || 30|| 13||n/a|| 521|| 48 (2 new, 46 upd (new version), 0 drp)<br />
|-<br />
| 4.11.1 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.11.1]] || 16 April 2018 || 29 May 2018|| 43|| 30|| 13||n/a|| 526|| 56 (7 new, 49 upd (new version), 0 drp)<br />
|-<br />
| rowspane= 1 | 4.12.0 || 4.12.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.12.0]] || xx April 2018 || xx month 2018|| number of days|| 30 || 12||n/a|| 530|| xy component (8 new, 46 upd (new version), 4 drp)<br />
|-<br />
| rowspane= 1 | 4.13.0 || 4.13.0 [[Image:changelog_icon.png|16px|link=Release_Notes_gCube.4.13.0]] || xx July 2018 || xx month 2018|| number of days|| total subsystems|| xx subsystem update||n/a|| xxx total component|| xy component (x new, x upd (new version), x drp)<br />
|}<br />
<br />
<br />
''Clicking on [[Image:changelog_icon.png|16px]] icon, where available, you can access release changelog''</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Zenodo_Publication&diff=31075Zenodo Publication2018-04-12T13:56:23Z<p>Gabriele.giammatteo: </p>
<hr />
<div>Zenodo is a portal (launched in May 2013) that collects outputs from researches in all fields of science to promote open access and open data. It is supported by the OpenAIRE initiative and developed and hosted by CERN.<br />
<br />
Each outcome uploaded (called ''Deposition'' in the Zenodo gergo) is stored by Zenodo along with a rich set of metadata, searchable via the portal and harvestable via the OAI-PMH protocol.<br />
<br />
Zenodo assigns to each deposition a unique Digital Object Identifier ('''DOI''') to to make the upload easily and uniquely citeable.<br />
<br />
<br />
= gCube Community =<br />
The gCube Community is a community created with the objective of collecting all the depositions related to '''gCube software'''. New software artifacts are automatically uploaded at every gCube release and metadata is generated automatically (see [[#Automatic Metadata Generation]]).<br />
<br />
<br />
[[File:zenodo-gcube-community.png|500px|center]]<br />
<br />
= Automatic Metadata Generation =<br />
<br />
The content for each deposition is generated automatically by the Zenodo Publisher from the information in the component source code and in ETICS.<br />
<br />
<div style="background:#fff8ce;padding:2px; border: 1px solid #827623;margin: 4px">'''Important''': Starting from gCube 4.9.0 (December 2017), the information uploaded has been heavily changed to improve the readability of the depositions created and to add support for ORCID identifiers and Zenodo versioning. All depositions created prior gCube 4.9.0 are still in Zenodo, but with the old format.</div><br />
<br />
The figure below shows the structure and the information published for each software artifact.<br />
<br />
[[File:zenodo.png|500px|center]]<br />
<br />
== 1. Title ==<br />
The title of the deposition is built as follow:<br />
<pre><br />
gCube <name><br />
</pre><br />
<br />
where:<br />
* name is the content of <name> tag in the pom.xml. If not present, the artifactId is used<br />
<br />
Previously also the component version and the gcube release were added in the title, but since the versioning of depositions is supported, this information has been removed because redundant.<br />
<br />
== 2. Authors ==<br />
Built from the content of "vcs.authors" property in ETICS configuration. That property is computed by ETICS looking at the VCS history of the component, If the user in ETICS has associated an [https://orcid.org/ ORCID] identifier, it is added (Zenodo shows it as a green circle before the Author name).<br />
<br />
== 3. Description ==<br />
It is generated starting from the README and changelog.xml files. The two file are parsed removing redundant (e.g. license because it is published already in [[#14. License]], source code, because it is published already in [[#5. GitHub link]]) or useless information (e.g. links to root page of the gCube wiki).<br />
<br />
== 4. Files ==<br />
The only file uploaded is the source package. It is generated during ETICS builds and contains everything checked-out from the VCS <br />
<br />
== 5. GitHub link ==<br />
The link to the position of the source code in GitHub<br />
<br />
== 6. DOI ==<br />
This is the unique identifier assigned by Zenodo to this deposition. <br />
<br />
== 7. Keywords ==<br />
Keywords for each component are stored in ETICS in the the description field of the corresponding ETICS module (not the configuration). For instance, keywords for "Common Authorization" are stored in the description of "org.gcube.common.authorization-common" module.<br />
They are encoded as follow:<br />
<pre><br />
#keywords=keyword1, keyword 2, key word3, ...<br />
</pre><br />
<br />
If keywords are not found in the component, they are searched in the subsystem and if also in the subsystem are not found, the keywords for the project are used. org.gcube project define these keywords:<br />
<pre><br />
#keywords=gCube, Java, Data Management System, Hybrid Data Infrastructure<br />
</pre><br />
<br />
<br />
== 8. Grants ==<br />
Static list of grants. The same for all depositions<br />
<br />
== 9. Binary artifact reference ==<br />
A link to the binary package (stored in Nexus) generated from the compilation of the source package uploaded in [[#4. Files]].<br />
<br />
== 10. Documentation Reference ==<br />
A reference to the documentation of the binary artifact. If present, this is a link to the jar file that contains the javadoc<br />
<br />
== 11. GitHub Reference ==<br />
(same as 5)<br />
<br />
== 12. PackageId ==<br />
internal identifier to uniquely identify the depositions<br />
<br />
== 13. Commiunity ==<br />
All depositions belongs to the gCube community<br />
<br />
== 14. License ==<br />
Statically set to EUPL-1.1<br />
<br />
== 15. Versions ==<br />
This is automatically generated by Zenodo to list all the other versions of the same component published in Zenodo.<br />
<br />
= Support for .zenodo.json file =<br />
<br />
It is possible to override all the metadata information generated automatically by adding a '''.zenodo.json''' file in the root of the source code of the component. This is a mechanism that Zenodo offers for the publication through GitHub and that we implemented also in our publisher. <br />
<br />
The .zenodo.json file will be read after the metadata generation from the source package and each metadata field generated will be replaced by the same field in the file (if present).<br />
<br />
Some examples of the format of the .zenodo.json file are [https://github.com/nipy/nipype/blob/master/.zenodo.json here] [https://github.com/numenta/htmresearch/blob/master/.zenodo.json here]<br />
<br />
<br />
= Troubleshooting =<br />
<br />
== Wrong authors published ==<br />
<br />
The automated publisher takes the list of authors from the <code>vcs.authors</code> property in the ETICS configuration. In turn, ETICS generated the <code>vcs.authors</code> property analysing the SVN/Git commits in the component repository. The analysis is based on an implementation of the [https://arxiv.org/pdf/1604.06766.pdf truck factor] that takes the authors of all the commits and weights them on the basis of the type of commits they did. It also uses thresholds to try to remove people that contributed less to the component.<br />
<br />
It could happen for some components that authors published in Zenodo do not match the actual authors of the components. This can be caused by<br />
# a not updated <code>vcs.authors</code> property,<br />
# not appropriate thresholds in the algorithm implementation or <br />
# the authors did not contributed (enough) to the code (e.g. because they just designed the component, because the code has been imported in SVN by someone else). <br />
<br />
'''Solutions'''<br />
<br />
For already published depositions, contact the release manager that will take care of fixing the depositions on the Zenodo portal directly<br />
<br />
For future releases of the component: <br />
# try to trigger an update of the <code>vcs.authors</code> property by modifying something in the configuration or in the code. If the configuration is locked, you need to clone the configuration and release it (attach to the current gCube release)<br />
# if after the update the property still report wrong authors, contact ETICS support team: they will try to adjust algorithm thresholds<br />
# if the algorithm cannot manage to extract correct autors (case 3), you can override the list of authors providing a .zenodo.json file in the component codebase root (see [[#Support for .zenodo.json file]]).</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Zenodo_Publication&diff=31074Zenodo Publication2018-04-12T13:55:32Z<p>Gabriele.giammatteo: /* Wrong authors published */</p>
<hr />
<div>Zenodo is a portal (launched in May 2013) that collects outputs from researches in all fields of science to promote open access and open data. It is supported by the OpenAIRE initiative and developed and hosted by CERN.<br />
<br />
Each outcome uploaded (called ''Deposition'' in the Zenodo gergo) is stored by Zenodo along with a rich set of metadata, searchable via the portal and harvestable via the OAI-PMH protocol.<br />
<br />
Zenodo assigns to each deposition a unique Digital Object Identifier ('''DOI''') to to make the upload easily and uniquely citeable.<br />
<br />
<br />
= gCube Community =<br />
The gCube Community is a community created with the objective of collecting all the depositions related to '''gCube software'''. New software artifacts are automatically uploaded at every gCube release and metadata is generated automatically (see [[#Automatic Metadata Generation]]).<br />
<br />
<br />
[[File:zenodo-gcube-community.png|500px|center]]<br />
<br />
= Automatic Metadata Generation =<br />
<br />
The content for each deposition is generated automatically by the Zenodo Publisher from the information in the component source code and in ETICS.<br />
<br />
<div style="background:#fff8ce;padding:2px; border: 1px solid #827623;margin: 4px">'''Important''': Starting from gCube 4.9.0 (December 2017), the information uploaded has been heavily changed to improve the readability of the depositions created and to add support for ORCID identifiers and Zenodo versioning. All depositions created prior gCube 4.9.0 are still in Zenodo, but with the old format.</div><br />
<br />
The figure below shows the structure and the information published for each software artifact.<br />
<br />
[[File:zenodo.png|500px|center]]<br />
<br />
== 1. Title ==<br />
The title of the deposition is built as follow:<br />
<pre><br />
gCube <name><br />
</pre><br />
<br />
where:<br />
* name is the content of <name> tag in the pom.xml. If not present, the artifactId is used<br />
<br />
Previously also the component version and the gcube release were added in the title, but since the versioning of depositions is supported, this information has been removed because redundant.<br />
<br />
== 2. Authors ==<br />
Built from the content of "vcs.authors" property in ETICS configuration. That property is computed by ETICS looking at the VCS history of the component, If the user in ETICS has associated an [https://orcid.org/ ORCID] identifier, it is added (Zenodo shows it as a green circle before the Author name).<br />
<br />
== 3. Description ==<br />
It is generated starting from the README and changelog.xml files. The two file are parsed removing redundant (e.g. license because it is published already in [[#14. License]], source code, because it is published already in [[#5. GitHub link]]) or useless information (e.g. links to root page of the gCube wiki).<br />
<br />
== 4. Files ==<br />
The only file uploaded is the source package. It is generated during ETICS builds and contains everything checked-out from the VCS <br />
<br />
== 5. GitHub link ==<br />
The link to the position of the source code in GitHub<br />
<br />
== 6. DOI ==<br />
This is the unique identifier assigned by Zenodo to this deposition. <br />
<br />
== 7. Keywords ==<br />
Keywords for each component are stored in ETICS in the the description field of the corresponding ETICS module (not the configuration). For instance, keywords for "Common Authorization" are stored in the description of "org.gcube.common.authorization-common" module.<br />
They are encoded as follow:<br />
<pre><br />
#keywords=keyword1, keyword 2, key word3, ...<br />
</pre><br />
<br />
If keywords are not found in the component, they are searched in the subsystem and if also in the subsystem are not found, the keywords for the project are used. org.gcube project define these keywords:<br />
<pre><br />
#keywords=gCube, Java, Data Management System, Hybrid Data Infrastructure<br />
</pre><br />
<br />
<br />
== 8. Grants ==<br />
Static list of grants. The same for all depositions<br />
<br />
== 9. Binary artifact reference ==<br />
A link to the binary package (stored in Nexus) generated from the compilation of the source package uploaded in [[#4. Files]].<br />
<br />
== 10. Documentation Reference ==<br />
A reference to the documentation of the binary artifact. If present, this is a link to the jar file that contains the javadoc<br />
<br />
== 11. GitHub Reference ==<br />
(same as 5)<br />
<br />
== 12. PackageId ==<br />
internal identifier to uniquely identify the depositions<br />
<br />
== 13. Commiunity ==<br />
All depositions belongs to the gCube community<br />
<br />
== 14. License ==<br />
Statically set to EUPL-1.1<br />
<br />
== 15. Versions ==<br />
This is automatically generated by Zenodo to list all the other versions of the same component published in Zenodo.<br />
<br />
= Support for .zenodo.json file =<br />
<br />
It is possible to override all the metadata information generated automatically by adding a '''.zenodo.json''' file in the root of the source code of the component. This is a mechanism that Zenodo offers for the publication through GitHub and that we implemented also in our publisher. <br />
<br />
The .zenodo.json file will be read after the metadata generation from the source package and each metadata field generated will be replaced by the same field in the file (if present).<br />
<br />
Some examples of the format of the .zenodo.json file are [https://github.com/nipy/nipype/blob/master/.zenodo.json here] [https://github.com/numenta/htmresearch/blob/master/.zenodo.json here]<br />
<br />
<br />
= Troubleshooting =<br />
<br />
== Wrong authors published ==<br />
<br />
The automated publisher takes the list of authors from the <code>vcs.authors</code> property in the ETICS configuration. In turn, ETICS generated the <code>vcs.authors</code> property analysing the SVN/Git commits in the component repository. The analysis is based on an implementation of the [https://arxiv.org/pdf/1604.06766.pdf truck factor] that takes the authors of all the commits and weights them on the basis of the type of commits they did. It also uses thresholds to try to remove people that contributed less to the component.<br />
<br />
It could happen for some components that authors published in Zenodo do not match the actual authors of the components. This can be caused by<br />
# a not updated <code>vcs.authors</code> property,<br />
# not appropriate thresholds in the algorithm implementation or <br />
# the authors did not contributed (enough) to the code (e.g. because they just designed the component, because the code has been imported in SVN by someone else). <br />
<br />
'''Solutions'''<br />
<br />
For already published depositions, contact the release manager that will take care of fixing the depositions on the Zenodo portal directly<br />
<br />
For future releases of the component: <br />
# try to trigger an update of the <code>vcs.authors</code> property by modifying something in the configuration or in the code. If the configuration is locked, you need to clone the configuration and release it (attach to the current gCube release)<br />
# if after the update the property still report wrong authors, contact ETICS support team: they will try to adjust algorithm thresholds<br />
# if the algorithm cannot manage to extract correct autors (case 3), you can override the list of authors providing a .zenodo.json file in the component codebase root (se</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Zenodo_Publication&diff=31073Zenodo Publication2018-04-12T13:46:59Z<p>Gabriele.giammatteo: /* Wrong authors published */</p>
<hr />
<div>Zenodo is a portal (launched in May 2013) that collects outputs from researches in all fields of science to promote open access and open data. It is supported by the OpenAIRE initiative and developed and hosted by CERN.<br />
<br />
Each outcome uploaded (called ''Deposition'' in the Zenodo gergo) is stored by Zenodo along with a rich set of metadata, searchable via the portal and harvestable via the OAI-PMH protocol.<br />
<br />
Zenodo assigns to each deposition a unique Digital Object Identifier ('''DOI''') to to make the upload easily and uniquely citeable.<br />
<br />
<br />
= gCube Community =<br />
The gCube Community is a community created with the objective of collecting all the depositions related to '''gCube software'''. New software artifacts are automatically uploaded at every gCube release and metadata is generated automatically (see [[#Automatic Metadata Generation]]).<br />
<br />
<br />
[[File:zenodo-gcube-community.png|500px|center]]<br />
<br />
= Automatic Metadata Generation =<br />
<br />
The content for each deposition is generated automatically by the Zenodo Publisher from the information in the component source code and in ETICS.<br />
<br />
<div style="background:#fff8ce;padding:2px; border: 1px solid #827623;margin: 4px">'''Important''': Starting from gCube 4.9.0 (December 2017), the information uploaded has been heavily changed to improve the readability of the depositions created and to add support for ORCID identifiers and Zenodo versioning. All depositions created prior gCube 4.9.0 are still in Zenodo, but with the old format.</div><br />
<br />
The figure below shows the structure and the information published for each software artifact.<br />
<br />
[[File:zenodo.png|500px|center]]<br />
<br />
== 1. Title ==<br />
The title of the deposition is built as follow:<br />
<pre><br />
gCube <name><br />
</pre><br />
<br />
where:<br />
* name is the content of <name> tag in the pom.xml. If not present, the artifactId is used<br />
<br />
Previously also the component version and the gcube release were added in the title, but since the versioning of depositions is supported, this information has been removed because redundant.<br />
<br />
== 2. Authors ==<br />
Built from the content of "vcs.authors" property in ETICS configuration. That property is computed by ETICS looking at the VCS history of the component, If the user in ETICS has associated an [https://orcid.org/ ORCID] identifier, it is added (Zenodo shows it as a green circle before the Author name).<br />
<br />
== 3. Description ==<br />
It is generated starting from the README and changelog.xml files. The two file are parsed removing redundant (e.g. license because it is published already in [[#14. License]], source code, because it is published already in [[#5. GitHub link]]) or useless information (e.g. links to root page of the gCube wiki).<br />
<br />
== 4. Files ==<br />
The only file uploaded is the source package. It is generated during ETICS builds and contains everything checked-out from the VCS <br />
<br />
== 5. GitHub link ==<br />
The link to the position of the source code in GitHub<br />
<br />
== 6. DOI ==<br />
This is the unique identifier assigned by Zenodo to this deposition. <br />
<br />
== 7. Keywords ==<br />
Keywords for each component are stored in ETICS in the the description field of the corresponding ETICS module (not the configuration). For instance, keywords for "Common Authorization" are stored in the description of "org.gcube.common.authorization-common" module.<br />
They are encoded as follow:<br />
<pre><br />
#keywords=keyword1, keyword 2, key word3, ...<br />
</pre><br />
<br />
If keywords are not found in the component, they are searched in the subsystem and if also in the subsystem are not found, the keywords for the project are used. org.gcube project define these keywords:<br />
<pre><br />
#keywords=gCube, Java, Data Management System, Hybrid Data Infrastructure<br />
</pre><br />
<br />
<br />
== 8. Grants ==<br />
Static list of grants. The same for all depositions<br />
<br />
== 9. Binary artifact reference ==<br />
A link to the binary package (stored in Nexus) generated from the compilation of the source package uploaded in [[#4. Files]].<br />
<br />
== 10. Documentation Reference ==<br />
A reference to the documentation of the binary artifact. If present, this is a link to the jar file that contains the javadoc<br />
<br />
== 11. GitHub Reference ==<br />
(same as 5)<br />
<br />
== 12. PackageId ==<br />
internal identifier to uniquely identify the depositions<br />
<br />
== 13. Commiunity ==<br />
All depositions belongs to the gCube community<br />
<br />
== 14. License ==<br />
Statically set to EUPL-1.1<br />
<br />
== 15. Versions ==<br />
This is automatically generated by Zenodo to list all the other versions of the same component published in Zenodo.<br />
<br />
= Support for .zenodo.json file =<br />
<br />
It is possible to override all the metadata information generated automatically by adding a '''.zenodo.json''' file in the root of the source code of the component. This is a mechanism that Zenodo offers for the publication through GitHub and that we implemented also in our publisher. <br />
<br />
The .zenodo.json file will be read after the metadata generation from the source package and each metadata field generated will be replaced by the same field in the file (if present).<br />
<br />
Some examples of the format of the .zenodo.json file are [https://github.com/nipy/nipype/blob/master/.zenodo.json here] [https://github.com/numenta/htmresearch/blob/master/.zenodo.json here]<br />
<br />
<br />
= Troubleshooting =<br />
<br />
== Wrong authors published ==<br />
<br />
The automated publisher takes the list of authors from the <code>vcs.authors</code> property in the ETICS configuration. In turn, ETICS generated the <code>vcs.authors</code> property analysing the SVN/Git commits in the component repository. The analysis is based on an implementation of the truck factor [1] that takes the authors of all the commits and weights them on the basis of the type of commits they did. It also uses thresholds to try to remove people that contributed less to the component.<br />
<br />
It could happen for some components that authors published in Zenodo do not match the actual authors of the components. This can be caused by<br />
# a not updated <code>vcs.authors</code> property,<br />
# not appropriate thresholds in the algorithm implementation or <br />
# the authors did not contributed (enough) to the code (e.g. because they just designed the component, because the code has been imported in SVN by someone else). <br />
<br />
For already published depositions, contact the release manager that will take care of fixing the depositions on the Zenodo portal directly<br />
<br />
For future releases of the component: <br />
# try to trigger an update of the <code>vcs.authors</code> property by modifying something in the configuration or in the code. If the configuration is locked, you need to clone the configuration and release it (attach to the current gCube release)<br />
# if after the update, the property still report wrong authors, contact that <br />
# if the algorithm cannot handle this, you can override the list of authors providing a .zenodo.json file in the component codebase root (seee <br />
<br />
<br />
<br />
[1] https://arxiv.org/pdf/1604.06766.pdf</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Zenodo_Publication&diff=31070Zenodo Publication2018-04-12T13:30:31Z<p>Gabriele.giammatteo: </p>
<hr />
<div>Zenodo is a portal (launched in May 2013) that collects outputs from researches in all fields of science to promote open access and open data. It is supported by the OpenAIRE initiative and developed and hosted by CERN.<br />
<br />
Each outcome uploaded (called ''Deposition'' in the Zenodo gergo) is stored by Zenodo along with a rich set of metadata, searchable via the portal and harvestable via the OAI-PMH protocol.<br />
<br />
Zenodo assigns to each deposition a unique Digital Object Identifier ('''DOI''') to to make the upload easily and uniquely citeable.<br />
<br />
<br />
= gCube Community =<br />
The gCube Community is a community created with the objective of collecting all the depositions related to '''gCube software'''. New software artifacts are automatically uploaded at every gCube release and metadata is generated automatically (see [[#Automatic Metadata Generation]]).<br />
<br />
<br />
[[File:zenodo-gcube-community.png|500px|center]]<br />
<br />
= Automatic Metadata Generation =<br />
<br />
The content for each deposition is generated automatically by the Zenodo Publisher from the information in the component source code and in ETICS.<br />
<br />
<div style="background:#fff8ce;padding:2px; border: 1px solid #827623;margin: 4px">'''Important''': Starting from gCube 4.9.0 (December 2017), the information uploaded has been heavily changed to improve the readability of the depositions created and to add support for ORCID identifiers and Zenodo versioning. All depositions created prior gCube 4.9.0 are still in Zenodo, but with the old format.</div><br />
<br />
The figure below shows the structure and the information published for each software artifact.<br />
<br />
[[File:zenodo.png|500px|center]]<br />
<br />
== 1. Title ==<br />
The title of the deposition is built as follow:<br />
<pre><br />
gCube <name><br />
</pre><br />
<br />
where:<br />
* name is the content of <name> tag in the pom.xml. If not present, the artifactId is used<br />
<br />
Previously also the component version and the gcube release were added in the title, but since the versioning of depositions is supported, this information has been removed because redundant.<br />
<br />
== 2. Authors ==<br />
Built from the content of "vcs.authors" property in ETICS configuration. That property is computed by ETICS looking at the VCS history of the component, If the user in ETICS has associated an [https://orcid.org/ ORCID] identifier, it is added (Zenodo shows it as a green circle before the Author name).<br />
<br />
== 3. Description ==<br />
It is generated starting from the README and changelog.xml files. The two file are parsed removing redundant (e.g. license because it is published already in [[#14. License]], source code, because it is published already in [[#5. GitHub link]]) or useless information (e.g. links to root page of the gCube wiki).<br />
<br />
== 4. Files ==<br />
The only file uploaded is the source package. It is generated during ETICS builds and contains everything checked-out from the VCS <br />
<br />
== 5. GitHub link ==<br />
The link to the position of the source code in GitHub<br />
<br />
== 6. DOI ==<br />
This is the unique identifier assigned by Zenodo to this deposition. <br />
<br />
== 7. Keywords ==<br />
Keywords for each component are stored in ETICS in the the description field of the corresponding ETICS module (not the configuration). For instance, keywords for "Common Authorization" are stored in the description of "org.gcube.common.authorization-common" module.<br />
They are encoded as follow:<br />
<pre><br />
#keywords=keyword1, keyword 2, key word3, ...<br />
</pre><br />
<br />
If keywords are not found in the component, they are searched in the subsystem and if also in the subsystem are not found, the keywords for the project are used. org.gcube project define these keywords:<br />
<pre><br />
#keywords=gCube, Java, Data Management System, Hybrid Data Infrastructure<br />
</pre><br />
<br />
<br />
== 8. Grants ==<br />
Static list of grants. The same for all depositions<br />
<br />
== 9. Binary artifact reference ==<br />
A link to the binary package (stored in Nexus) generated from the compilation of the source package uploaded in [[#4. Files]].<br />
<br />
== 10. Documentation Reference ==<br />
A reference to the documentation of the binary artifact. If present, this is a link to the jar file that contains the javadoc<br />
<br />
== 11. GitHub Reference ==<br />
(same as 5)<br />
<br />
== 12. PackageId ==<br />
internal identifier to uniquely identify the depositions<br />
<br />
== 13. Commiunity ==<br />
All depositions belongs to the gCube community<br />
<br />
== 14. License ==<br />
Statically set to EUPL-1.1<br />
<br />
== 15. Versions ==<br />
This is automatically generated by Zenodo to list all the other versions of the same component published in Zenodo.<br />
<br />
= Support for .zenodo.json file =<br />
<br />
It is possible to override all the metadata information generated automatically by adding a '''.zenodo.json''' file in the root of the source code of the component. This is a mechanism that Zenodo offers for the publication through GitHub and that we implemented also in our publisher. <br />
<br />
The .zenodo.json file will be read after the metadata generation from the source package and each metadata field generated will be replaced by the same field in the file (if present).<br />
<br />
Some examples of the format of the .zenodo.json file are [https://github.com/nipy/nipype/blob/master/.zenodo.json here] [https://github.com/numenta/htmresearch/blob/master/.zenodo.json here]<br />
<br />
<br />
= Troubleshooting =<br />
<br />
== Wrong authors published ==<br />
It could happen for some components that authors published in Zenodo do not match the actual authors of the components. The automated publisher takes the list of authors from the <code>vcs.authors</code> property in the ETICS configuration. In turn, ETICS generates the list analysing the SVN/Git commits in the component repository. The analysis is based on an implementation of the truck factor [1] that takes the authors of all the commits and weights them on the basis of the type of commits they did. <br />
<br />
<br />
[1] https://arxiv.org/pdf/1604.06766.pdf</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Zenodo_Publication&diff=31064Zenodo Publication2018-04-09T08:18:18Z<p>Gabriele.giammatteo: /* Possible Improvements */</p>
<hr />
<div>Zenodo is a portal (launched in May 2013) that collects outputs from researches in all fields of science to promote open access and open data. It is supported by the OpenAIRE initiative and developed and hosted by CERN.<br />
<br />
Each outcome uploaded (called ''Deposition'' in the Zenodo gergo) is stored by Zenodo along with a rich set of metadata, searchable via the portal and harvestable via the OAI-PMH protocol.<br />
<br />
Zenodo assigns to each deposition a unique Digital Object Identifier ('''DOI''') to to make the upload easily and uniquely citeable.<br />
<br />
<br />
= gCube Community =<br />
The gCube Community is a community created with the objective of collecting all the depositions related to '''gCube software'''. New software artifacts are automatically uploaded at every gCube release and metadata is generated automatically (see [[#Automatic Metadata Generation]]).<br />
<br />
<br />
[[File:zenodo-gcube-community.png|500px|center]]<br />
<br />
= Automatic Metadata Generation =<br />
<br />
The content for each deposition is generated automatically by the Zenodo Publisher from the information in the component source code and in ETICS.<br />
<br />
<div style="background:#fff8ce;padding:2px; border: 1px solid #827623;margin: 4px">'''Important''': Starting from gCube 4.9.0 (December 2017), the information uploaded has been heavily changed to improve the readability of the depositions created and to add support for ORCID identifiers and Zenodo versioning. All depositions created prior gCube 4.9.0 are still in Zenodo, but with the old format.</div><br />
<br />
The figure below shows the structure and the information published for each software artifact.<br />
<br />
[[File:zenodo.png|500px|center]]<br />
<br />
== 1. Title ==<br />
The title of the deposition is built as follow:<br />
<pre><br />
gCube <name><br />
</pre><br />
<br />
where:<br />
* name is the content of <name> tag in the pom.xml. If not present, the artifactId is used<br />
<br />
Previously also the component version and the gcube release were added in the title, but since the versioning of depositions is supported, this information has been removed because redundant.<br />
<br />
== 2. Authors ==<br />
Built from the content of "vcs.authors" property in ETICS configuration. That property is computed by ETICS looking at the VCS history of the component, If the user in ETICS has associated an [https://orcid.org/ ORCID] identifier, it is added (Zenodo shows it as a green circle before the Author name).<br />
<br />
== 3. Description ==<br />
It is generated starting from the README and changelog.xml files. The two file are parsed removing redundant (e.g. license because it is published already in [[#14. License]], source code, because it is published already in [[#5. GitHub link]]) or useless information (e.g. links to root page of the gCube wiki).<br />
<br />
== 4. Files ==<br />
The only file uploaded is the source package. It is generated during ETICS builds and contains everything checked-out from the VCS <br />
<br />
== 5. GitHub link ==<br />
The link to the position of the source code in GitHub<br />
<br />
== 6. DOI ==<br />
This is the unique identifier assigned by Zenodo to this deposition. <br />
<br />
== 7. Keywords ==<br />
Keywords for each component are stored in ETICS in the the description field of the corresponding ETICS module (not the configuration). For instance, keywords for "Common Authorization" are stored in the description of "org.gcube.common.authorization-common" module.<br />
They are encoded as follow:<br />
<pre><br />
#keywords=keyword1, keyword 2, key word3, ...<br />
</pre><br />
<br />
If keywords are not found in the component, they are searched in the subsystem and if also in the subsystem are not found, the keywords for the project are used. org.gcube project define these keywords:<br />
<pre><br />
#keywords=gCube, Java, Data Management System, Hybrid Data Infrastructure<br />
</pre><br />
<br />
<br />
== 8. Grants ==<br />
Static list of grants. The same for all depositions<br />
<br />
== 9. Binary artifact reference ==<br />
A link to the binary package (stored in Nexus) generated from the compilation of the source package uploaded in [[#4. Files]].<br />
<br />
== 10. Documentation Reference ==<br />
A reference to the documentation of the binary artifact. If present, this is a link to the jar file that contains the javadoc<br />
<br />
== 11. GitHub Reference ==<br />
(same as 5)<br />
<br />
== 12. PackageId ==<br />
internal identifier to uniquely identify the depositions<br />
<br />
== 13. Commiunity ==<br />
All depositions belongs to the gCube community<br />
<br />
== 14. License ==<br />
Statically set to EUPL-1.1<br />
<br />
== 15. Versions ==<br />
This is automatically generated by Zenodo to list all the other versions of the same component published in Zenodo.<br />
<br />
= Support for .zenodo.json file =<br />
<br />
It is possible to override all the metadata information generated automatically by adding a '''.zenodo.json''' file in the root of the source code of the component. This is a mechanism that Zenodo offers for the publication through GitHub and that we implemented also in our publisher. <br />
<br />
The .zenodo.json file will be read after the metadata generation from the source package and each metadata field generated will be replaced by the same field in the file (if present).<br />
<br />
Some examples of the format of the .zenodo.json file are [https://github.com/nipy/nipype/blob/master/.zenodo.json here] [https://github.com/numenta/htmresearch/blob/master/.zenodo.json here]</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Zenodo_Publication&diff=31063Zenodo Publication2018-04-09T08:12:17Z<p>Gabriele.giammatteo: /* Automatic Metadata Generation */</p>
<hr />
<div>Zenodo is a portal (launched in May 2013) that collects outputs from researches in all fields of science to promote open access and open data. It is supported by the OpenAIRE initiative and developed and hosted by CERN.<br />
<br />
Each outcome uploaded (called ''Deposition'' in the Zenodo gergo) is stored by Zenodo along with a rich set of metadata, searchable via the portal and harvestable via the OAI-PMH protocol.<br />
<br />
Zenodo assigns to each deposition a unique Digital Object Identifier ('''DOI''') to to make the upload easily and uniquely citeable.<br />
<br />
<br />
= gCube Community =<br />
The gCube Community is a community created with the objective of collecting all the depositions related to '''gCube software'''. New software artifacts are automatically uploaded at every gCube release and metadata is generated automatically (see [[#Automatic Metadata Generation]]).<br />
<br />
<br />
[[File:zenodo-gcube-community.png|500px|center]]<br />
<br />
= Automatic Metadata Generation =<br />
<br />
The content for each deposition is generated automatically by the Zenodo Publisher from the information in the component source code and in ETICS.<br />
<br />
<div style="background:#fff8ce;padding:2px; border: 1px solid #827623;margin: 4px">'''Important''': Starting from gCube 4.9.0 (December 2017), the information uploaded has been heavily changed to improve the readability of the depositions created and to add support for ORCID identifiers and Zenodo versioning. All depositions created prior gCube 4.9.0 are still in Zenodo, but with the old format.</div><br />
<br />
The figure below shows the structure and the information published for each software artifact.<br />
<br />
[[File:zenodo.png|500px|center]]<br />
<br />
== 1. Title ==<br />
The title of the deposition is built as follow:<br />
<pre><br />
gCube <name><br />
</pre><br />
<br />
where:<br />
* name is the content of <name> tag in the pom.xml. If not present, the artifactId is used<br />
<br />
Previously also the component version and the gcube release were added in the title, but since the versioning of depositions is supported, this information has been removed because redundant.<br />
<br />
== 2. Authors ==<br />
Built from the content of "vcs.authors" property in ETICS configuration. That property is computed by ETICS looking at the VCS history of the component, If the user in ETICS has associated an [https://orcid.org/ ORCID] identifier, it is added (Zenodo shows it as a green circle before the Author name).<br />
<br />
== 3. Description ==<br />
It is generated starting from the README and changelog.xml files. The two file are parsed removing redundant (e.g. license because it is published already in [[#14. License]], source code, because it is published already in [[#5. GitHub link]]) or useless information (e.g. links to root page of the gCube wiki).<br />
<br />
== 4. Files ==<br />
The only file uploaded is the source package. It is generated during ETICS builds and contains everything checked-out from the VCS <br />
<br />
== 5. GitHub link ==<br />
The link to the position of the source code in GitHub<br />
<br />
== 6. DOI ==<br />
This is the unique identifier assigned by Zenodo to this deposition. <br />
<br />
== 7. Keywords ==<br />
Keywords for each component are stored in ETICS in the the description field of the corresponding ETICS module (not the configuration). For instance, keywords for "Common Authorization" are stored in the description of "org.gcube.common.authorization-common" module.<br />
They are encoded as follow:<br />
<pre><br />
#keywords=keyword1, keyword 2, key word3, ...<br />
</pre><br />
<br />
If keywords are not found in the component, they are searched in the subsystem and if also in the subsystem are not found, the keywords for the project are used. org.gcube project define these keywords:<br />
<pre><br />
#keywords=gCube, Java, Data Management System, Hybrid Data Infrastructure<br />
</pre><br />
<br />
<br />
== 8. Grants ==<br />
Static list of grants. The same for all depositions<br />
<br />
== 9. Binary artifact reference ==<br />
A link to the binary package (stored in Nexus) generated from the compilation of the source package uploaded in [[#4. Files]].<br />
<br />
== 10. Documentation Reference ==<br />
A reference to the documentation of the binary artifact. If present, this is a link to the jar file that contains the javadoc<br />
<br />
== 11. GitHub Reference ==<br />
(same as 5)<br />
<br />
== 12. PackageId ==<br />
internal identifier to uniquely identify the depositions<br />
<br />
== 13. Commiunity ==<br />
All depositions belongs to the gCube community<br />
<br />
== 14. License ==<br />
Statically set to EUPL-1.1<br />
<br />
== 15. Versions ==<br />
This is automatically generated by Zenodo to list all the other versions of the same component published in Zenodo.<br />
<br />
= Possible Improvements =<br />
<br />
* Add Changelog in the description<br />
<br />
* Add links to wiki (parsed from the README)<br />
<br />
* Support for a ".zenodo" file in the projet root that override metadata generated automatically<br />
<br />
* Make a Smartgears service<br />
<br />
* More metadata can be added:<br />
** ''Subjects'' (however Subjects does not appear in Zenodo)</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Zenodo_Publication&diff=31062Zenodo Publication2018-04-09T08:06:54Z<p>Gabriele.giammatteo: /* Automatic Metadata Generation */</p>
<hr />
<div>Zenodo is a portal (launched in May 2013) that collects outputs from researches in all fields of science to promote open access and open data. It is supported by the OpenAIRE initiative and developed and hosted by CERN.<br />
<br />
Each outcome uploaded (called ''Deposition'' in the Zenodo gergo) is stored by Zenodo along with a rich set of metadata, searchable via the portal and harvestable via the OAI-PMH protocol.<br />
<br />
Zenodo assigns to each deposition a unique Digital Object Identifier ('''DOI''') to to make the upload easily and uniquely citeable.<br />
<br />
<br />
= gCube Community =<br />
The gCube Community is a community created with the objective of collecting all the depositions related to '''gCube software'''. New software artifacts are automatically uploaded at every gCube release and metadata is generated automatically (see [[#Automatic Metadata Generation]]).<br />
<br />
<br />
[[File:zenodo-gcube-community.png|500px|center]]<br />
<br />
= Automatic Metadata Generation =<br />
<br />
The content for each deposition is generated automatically by the Zenodo Publisher from the information in the component source code and in ETICS.<br />
<br />
<div style="background:#fff8ce;padding:2px; border: 1px solid #827623;margin: 4px">'''Important''': Starting from gCube 4.9.0 (December 2017), the information uploaded has been heavily changed to improve the readability of the depositions created and to add support for ORCID identifiers and Zenodo versioning. All depositions created prior gCube 4.9.0 are still in Zenodo, but with the old format.</div><br />
<br />
The figure below shows the structure and the information published for each software artifact.<br />
<br />
[[File:zenodo.png|500px|center]]<br />
<br />
== 1. Title ==<br />
The title of the deposition is built as follow:<br />
<pre><br />
gCube <name><br />
</pre><br />
<br />
where:<br />
* name is the content of <name> tag in the pom.xml. If not present, the artifactId is used<br />
<br />
Previously also the component version and the gcube release were added in the title, but since the versioning of depositions is supported, this information has been removed because redundant.<br />
<br />
== 2. Authors ==<br />
Built from the content of "vcs.authors" property in ETICS configuration. That property is computed by ETICS looking at the VCS history of the component, If the user in ETICS has associated an [https://orcid.org/ ORCID] identifier, it is added (Zenodo shows it as a green circle before the Author name).<br />
<br />
== 3. Description ==<br />
It is generated starting from the README and changelog.xml files. The two file are parsed removing redundant (e.g. license because it is published already in [[#14. License]], source code, because it is published already in [[#5. GitHub link]]) or useless information (e.g. links to root page of the gCube wiki).<br />
<br />
== 4. Files ==<br />
The only file uploaded is the source package. It is generated during ETICS builds and contains everything checked-out from the VCS <br />
<br />
== 5. GitHub link ==<br />
The link to the position of the source code in GitHub<br />
<br />
== 6. DOI ==<br />
assigned by Zenodo to this deposition. <br />
<br />
== 7. Keywords ==<br />
Keywords for each component are stored in ETICS in the the description field of the corresponding ETICS module (not the configuration). For instance, keywords for "Common Authorization" are stored in the description of "org.gcube.common.authorization-common" module.<br />
They are encoded as follow:<br />
<pre><br />
#keywords=keyword1, keyword 2, key word3, ...<br />
</pre><br />
<br />
If keywords are not found in the component, they are searched in the subsystem and if also in the subsystem are not found, the keywords for the project are used. org.gcube project define these keywords:<br />
<pre><br />
#keywords=gCube, Java, Data Management System, Hybrid Data Infrastructure<br />
</pre><br />
<br />
<br />
== 8. Grants ==<br />
Static list of grants. The same for all depositions<br />
<br />
== 9. Previous Version Reference ==<br />
The DOI of the previous version of this component<br />
<br />
== 10. GitHub Reference ==<br />
(same as 5)<br />
<br />
== 11. PackageId ==<br />
internal identifier to uniquely identify the depositions<br />
<br />
== 12. Commiunity ==<br />
All depositions belongs to the gCube community<br />
<br />
== 14. License ==<br />
Statically set to EUPL-1.1<br />
<br />
= Possible Improvements =<br />
<br />
* Add Changelog in the description<br />
<br />
* Add links to wiki (parsed from the README)<br />
<br />
* Support for a ".zenodo" file in the projet root that override metadata generated automatically<br />
<br />
* Make a Smartgears service<br />
<br />
* More metadata can be added:<br />
** ''Subjects'' (however Subjects does not appear in Zenodo)</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Zenodo_Publication&diff=31061Zenodo Publication2018-04-09T08:03:31Z<p>Gabriele.giammatteo: /* Automatic Metadata Generation */</p>
<hr />
<div>Zenodo is a portal (launched in May 2013) that collects outputs from researches in all fields of science to promote open access and open data. It is supported by the OpenAIRE initiative and developed and hosted by CERN.<br />
<br />
Each outcome uploaded (called ''Deposition'' in the Zenodo gergo) is stored by Zenodo along with a rich set of metadata, searchable via the portal and harvestable via the OAI-PMH protocol.<br />
<br />
Zenodo assigns to each deposition a unique Digital Object Identifier ('''DOI''') to to make the upload easily and uniquely citeable.<br />
<br />
<br />
= gCube Community =<br />
The gCube Community is a community created with the objective of collecting all the depositions related to '''gCube software'''. New software artifacts are automatically uploaded at every gCube release and metadata is generated automatically (see [[#Automatic Metadata Generation]]).<br />
<br />
<br />
[[File:zenodo-gcube-community.png|500px|center]]<br />
<br />
= Automatic Metadata Generation =<br />
<br />
The content for each deposition is generated automatically by the Zenodo Publisher from the information in the component source code and in ETICS.<br />
<br />
<div style="background:#fff8ce;padding:2px; border: 1px solid #827623;margin: 4px">'''Important''': Starting from gCube 4.9.0 (December 2017), the information uploaded has been heavily changed to improve the readability of the depositions created and to add support for ORCID identifiers and Zenodo versioning. All depositions created prior gCube 4.9.0 are still in Zenodo, but with the old format.</div><br />
<br />
The figure below shows the structure and the information published for each software artifact.<br />
<br />
[[File:zenodo.png|500px|center]]<br />
<br />
== 1. Title ==<br />
The title of the deposition is built as follow:<br />
<pre><br />
gCube <name><br />
</pre><br />
<br />
where:<br />
* name is the content of <name> tag in the pom.xml. If not present, the artifactId is used<br />
<br />
Previously also the component version and the gcube release were added in the title, but since the versioning of depositions is supported, this information has been removed because redundant.<br />
<br />
== 2. Authors ==<br />
Built from the content of "vcs.authors" property in ETICS configuration. That property is computed by ETICS looking at the VCS history of the component, If the user in ETICS has associated an [https://orcid.org/ ORCID] identifier, it is added (Zenodo shows it as a green circle before the Author name).<br />
<br />
== 3. Description ==<br />
It is generated starting from the README and changelog.xml files. The two file are parsed removing redundant (e.g. license, because it is published already in <br />
<br />
== 4. Files ==<br />
The only file uploaded is the source package. It is generated during ETICS builds and contains everything checked-out from the VCS <br />
<br />
== 5. GitHub link ==<br />
The link to the position of the source code in GitHub<br />
<br />
== 6. DOI ==<br />
assigned by Zenodo to this deposition. <br />
<br />
== 7. Keywords ==<br />
Keywords for each component are stored in ETICS in the the description field of the corresponding ETICS module (not the configuration). For instance, keywords for "Common Authorization" are stored in the description of "org.gcube.common.authorization-common" module.<br />
They are encoded as follow:<br />
<pre><br />
#keywords=keyword1, keyword 2, key word3, ...<br />
</pre><br />
<br />
If keywords are not found in the component, they are searched in the subsystem and if also in the subsystem are not found, the keywords for the project are used. org.gcube project define these keywords:<br />
<pre><br />
#keywords=gCube, Java, Data Management System, Hybrid Data Infrastructure<br />
</pre><br />
<br />
<br />
== 8. Grants ==<br />
Static list of grants. The same for all depositions<br />
<br />
== 9. Previous Version Reference ==<br />
The DOI of the previous version of this component<br />
<br />
== 10. GitHub Reference ==<br />
(same as 5)<br />
<br />
== 11. PackageId ==<br />
internal identifier to uniquely identify the depositions<br />
<br />
== 12. Commiunity ==<br />
All depositions belongs to the gCube community<br />
<br />
== 13. License ==<br />
Statically set to EUPL-1.1<br />
<br />
= Possible Improvements =<br />
<br />
* Add Changelog in the description<br />
<br />
* Add links to wiki (parsed from the README)<br />
<br />
* Support for a ".zenodo" file in the projet root that override metadata generated automatically<br />
<br />
* Make a Smartgears service<br />
<br />
* More metadata can be added:<br />
** ''Subjects'' (however Subjects does not appear in Zenodo)</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Zenodo_Publication&diff=31060Zenodo Publication2018-04-09T08:00:21Z<p>Gabriele.giammatteo: /* Automatic Metadata Generation */</p>
<hr />
<div>Zenodo is a portal (launched in May 2013) that collects outputs from researches in all fields of science to promote open access and open data. It is supported by the OpenAIRE initiative and developed and hosted by CERN.<br />
<br />
Each outcome uploaded (called ''Deposition'' in the Zenodo gergo) is stored by Zenodo along with a rich set of metadata, searchable via the portal and harvestable via the OAI-PMH protocol.<br />
<br />
Zenodo assigns to each deposition a unique Digital Object Identifier ('''DOI''') to to make the upload easily and uniquely citeable.<br />
<br />
<br />
= gCube Community =<br />
The gCube Community is a community created with the objective of collecting all the depositions related to '''gCube software'''. New software artifacts are automatically uploaded at every gCube release and metadata is generated automatically (see [[#Automatic Metadata Generation]]).<br />
<br />
<br />
[[File:zenodo-gcube-community.png|500px|center]]<br />
<br />
= Automatic Metadata Generation =<br />
<br />
The content for each deposition is generated automatically by the Zenodo Publisher from the information in the component source code and in ETICS.<br />
<br />
<div style="background:#fff196;padding:2px; border: 1px solid #827623;margin: 4px">'''Important''': Starting from gCube 4.9.0 (December 2017), the information uploaded has been heavily changed to improve the readability of the depositions created and to add support for ORCID identifiers and Zenodo versioning. All depositions created prior gCube 4.9.0 are still in Zenodo, but with the old format.</div><br />
<br />
The figure below shows the structure and the information published for each software artifact.<br />
<br />
[[File:zenodo.png|500px|center]]<br />
<br />
== 1. Title ==<br />
The title of the deposition is built as follow:<br />
<pre><br />
gCube <name><br />
</pre><br />
<br />
where:<br />
* name is the content of <name> tag in the pom.xml. If not present, the artifactId is used<br />
<br />
Previously also the component version and the gcube release were added in the title, but since the versioning of depositions is supported, this information has been removed because redundant.<br />
<br />
== 2. Authors ==<br />
Built from the content of "vcs.authors" property in ETICS configuration. That property is computed by ETICS looking at the VCS history of the component, If the user in ETICS has associated an [https://orcid.org/ ORCID] identifier, it is added (Zenodo shows it as a green circle before the Author name).<br />
<br />
== 3. Description ==<br />
It is generated starting from the README and changelog.xml files. <br />
<br />
== 4. Files ==<br />
The only file uploaded is the source package. It is generated during ETICS builds and contains everything checked-out from the VCS <br />
<br />
== 5. GitHub link ==<br />
The link to the position of the source code in GitHub<br />
<br />
== 6. DOI ==<br />
assigned by Zenodo to this deposition. <br />
<br />
== 7. Keywords ==<br />
Keywords for each component are stored in ETICS in the the description field of the corresponding ETICS module (not the configuration). For instance, keywords for "Common Authorization" are stored in the description of "org.gcube.common.authorization-common" module.<br />
They are encoded as follow:<br />
<pre><br />
#keywords=keyword1, keyword 2, key word3, ...<br />
</pre><br />
<br />
If keywords are not found in the component, they are searched in the subsystem and if also in the subsystem are not found, the keywords for the project are used. org.gcube project define these keywords:<br />
<pre><br />
#keywords=gCube, Java, Data Management System, Hybrid Data Infrastructure<br />
</pre><br />
<br />
<br />
== 8. Grants ==<br />
Static list of grants. The same for all depositions<br />
<br />
== 9. Previous Version Reference ==<br />
The DOI of the previous version of this component<br />
<br />
== 10. GitHub Reference ==<br />
(same as 5)<br />
<br />
== 11. PackageId ==<br />
internal identifier to uniquely identify the depositions<br />
<br />
== 12. Commiunity ==<br />
All depositions belongs to the gCube community<br />
<br />
== 13. License ==<br />
Statically set to EUPL-1.1<br />
<br />
= Possible Improvements =<br />
<br />
* Add Changelog in the description<br />
<br />
* Add links to wiki (parsed from the README)<br />
<br />
* Support for a ".zenodo" file in the projet root that override metadata generated automatically<br />
<br />
* Make a Smartgears service<br />
<br />
* More metadata can be added:<br />
** ''Subjects'' (however Subjects does not appear in Zenodo)</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Zenodo_Publication&diff=31059Zenodo Publication2018-04-09T07:56:09Z<p>Gabriele.giammatteo: /* Automatic Metadata Generation */</p>
<hr />
<div>Zenodo is a portal (launched in May 2013) that collects outputs from researches in all fields of science to promote open access and open data. It is supported by the OpenAIRE initiative and developed and hosted by CERN.<br />
<br />
Each outcome uploaded (called ''Deposition'' in the Zenodo gergo) is stored by Zenodo along with a rich set of metadata, searchable via the portal and harvestable via the OAI-PMH protocol.<br />
<br />
Zenodo assigns to each deposition a unique Digital Object Identifier ('''DOI''') to to make the upload easily and uniquely citeable.<br />
<br />
<br />
= gCube Community =<br />
The gCube Community is a community created with the objective of collecting all the depositions related to '''gCube software'''. New software artifacts are automatically uploaded at every gCube release and metadata is generated automatically (see [[#Automatic Metadata Generation]]).<br />
<br />
<br />
[[File:zenodo-gcube-community.png|500px|center]]<br />
<br />
= Automatic Metadata Generation =<br />
<br />
The content for each deposition is generated automatically by the Zenodo Publisher from the information in the component source code and in ETICS.<br />
<br />
<div style="background:#fff196;padding:2px; border: 1px solid #827623;margin: 4px">'''Important''': Starting from gCube 4.9.0 (December 2017), the information uploaded has been heavily changed to improve the readability of the depositions created and to add support for ORCID identifiers and Zenodo versioning. All depositions created prior gCube 4.9.0 are still in Zenodo, but with the old format.</div><br />
<br />
The figure below shows the structure and the information published for each software artifact.<br />
<br />
[[File:zenodo.png|500px|center]]<br />
<br />
== 1. Title ==<br />
The title of the deposition is built as follow:<br />
<pre><br />
gCube <gCube Version> - <name> v. <version><br />
</pre><br />
<br />
where:<br />
* gCube Version is the version of gCube (e.g. 3.11.1, 4.0.0, ...)<br />
* name is the content of <name> tag in the pom.xml. If not present, the artifactId is used<br />
* version is the version in the pom<br />
<br />
== 2. Authors ==<br />
Built from the content of "vcs.authors" property in ETICS configuration. That property is computed by ETICS looking at the VCS history of the component<br />
<br />
== 3. Description ==<br />
It corresponds to the content of README file.<br />
<br />
== 4. Files ==<br />
The only file uploaded is the source package. It is generated during ETICS builds and contains everything checked-out from the VCS <br />
<br />
== 5. GitHub link ==<br />
The link to the position of the source code in GitHub<br />
<br />
== 6. DOI ==<br />
assigned by Zenodo to this deposition. <br />
<br />
== 7. Keywords ==<br />
Keywords for each component are stored in ETICS in the the description field of the corresponding ETICS module (not the configuration). For instance, keywords for "Common Authorization" are stored in the description of "org.gcube.common.authorization-common" module.<br />
They are encoded as follow:<br />
<pre><br />
#keywords=keyword1, keyword 2, key word3, ...<br />
</pre><br />
<br />
If keywords are not found in the component, they are searched in the subsystem and if also in the subsystem are not found, the keywords for the project are used. org.gcube project define these keywords:<br />
<pre><br />
#keywords=gCube, Java, Data Management System, Hybrid Data Infrastructure<br />
</pre><br />
<br />
<br />
== 8. Grants ==<br />
Static list of grants. The same for all depositions<br />
<br />
== 9. Previous Version Reference ==<br />
The DOI of the previous version of this component<br />
<br />
== 10. GitHub Reference ==<br />
(same as 5)<br />
<br />
== 11. PackageId ==<br />
internal identifier to uniquely identify the depositions<br />
<br />
== 12. Commiunity ==<br />
All depositions belongs to the gCube community<br />
<br />
== 13. License ==<br />
Statically set to EUPL-1.1<br />
<br />
= Possible Improvements =<br />
<br />
* Add Changelog in the description<br />
<br />
* Add links to wiki (parsed from the README)<br />
<br />
* Support for a ".zenodo" file in the projet root that override metadata generated automatically<br />
<br />
* Make a Smartgears service<br />
<br />
* More metadata can be added:<br />
** ''Subjects'' (however Subjects does not appear in Zenodo)</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Zenodo_Publication&diff=31058Zenodo Publication2018-04-09T07:55:06Z<p>Gabriele.giammatteo: /* Automatic Metadata Generation */</p>
<hr />
<div>Zenodo is a portal (launched in May 2013) that collects outputs from researches in all fields of science to promote open access and open data. It is supported by the OpenAIRE initiative and developed and hosted by CERN.<br />
<br />
Each outcome uploaded (called ''Deposition'' in the Zenodo gergo) is stored by Zenodo along with a rich set of metadata, searchable via the portal and harvestable via the OAI-PMH protocol.<br />
<br />
Zenodo assigns to each deposition a unique Digital Object Identifier ('''DOI''') to to make the upload easily and uniquely citeable.<br />
<br />
<br />
= gCube Community =<br />
The gCube Community is a community created with the objective of collecting all the depositions related to '''gCube software'''. New software artifacts are automatically uploaded at every gCube release and metadata is generated automatically (see [[#Automatic Metadata Generation]]).<br />
<br />
<br />
[[File:zenodo-gcube-community.png|500px|center]]<br />
<br />
= Automatic Metadata Generation =<br />
<br />
The content for each deposition is generated automatically by the Zenodo Publisher from the information in the component source code and in ETICS.<br />
<br />
<div style="background:#fff196;padding:2px; border: 1px solid #827623;margin: 4px">'''Important''': Starting from gCube 4.9.0 (December 2017), the information uploaded has been heavily changed to improve the readability of the depositions created and to add support for ORCID identifiers and Zenodo versioning. All depositions created prior gCube 4.9.0 are still in Zenodo, but with the old format.</div><br />
<br />
[[File:zenodo.png|500px|center]]<br />
<br />
== 1. Title ==<br />
The title of the deposition is built as follow:<br />
<pre><br />
gCube <gCube Version> - <name> v. <version><br />
</pre><br />
<br />
where:<br />
* gCube Version is the version of gCube (e.g. 3.11.1, 4.0.0, ...)<br />
* name is the content of <name> tag in the pom.xml. If not present, the artifactId is used<br />
* version is the version in the pom<br />
<br />
== 2. Authors ==<br />
Built from the content of "vcs.authors" property in ETICS configuration. That property is computed by ETICS looking at the VCS history of the component<br />
<br />
== 3. Description ==<br />
It corresponds to the content of README file.<br />
<br />
== 4. Files ==<br />
The only file uploaded is the source package. It is generated during ETICS builds and contains everything checked-out from the VCS <br />
<br />
== 5. GitHub link ==<br />
The link to the position of the source code in GitHub<br />
<br />
== 6. DOI ==<br />
assigned by Zenodo to this deposition. <br />
<br />
== 7. Keywords ==<br />
Keywords for each component are stored in ETICS in the the description field of the corresponding ETICS module (not the configuration). For instance, keywords for "Common Authorization" are stored in the description of "org.gcube.common.authorization-common" module.<br />
They are encoded as follow:<br />
<pre><br />
#keywords=keyword1, keyword 2, key word3, ...<br />
</pre><br />
<br />
If keywords are not found in the component, they are searched in the subsystem and if also in the subsystem are not found, the keywords for the project are used. org.gcube project define these keywords:<br />
<pre><br />
#keywords=gCube, Java, Data Management System, Hybrid Data Infrastructure<br />
</pre><br />
<br />
<br />
== 8. Grants ==<br />
Static list of grants. The same for all depositions<br />
<br />
== 9. Previous Version Reference ==<br />
The DOI of the previous version of this component<br />
<br />
== 10. GitHub Reference ==<br />
(same as 5)<br />
<br />
== 11. PackageId ==<br />
internal identifier to uniquely identify the depositions<br />
<br />
== 12. Commiunity ==<br />
All depositions belongs to the gCube community<br />
<br />
== 13. License ==<br />
Statically set to EUPL-1.1<br />
<br />
= Possible Improvements =<br />
<br />
* Add Changelog in the description<br />
<br />
* Add links to wiki (parsed from the README)<br />
<br />
* Support for a ".zenodo" file in the projet root that override metadata generated automatically<br />
<br />
* Make a Smartgears service<br />
<br />
* More metadata can be added:<br />
** ''Subjects'' (however Subjects does not appear in Zenodo)</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Zenodo_Publication&diff=31057Zenodo Publication2018-04-09T07:45:57Z<p>Gabriele.giammatteo: /* gCube Community */</p>
<hr />
<div>Zenodo is a portal (launched in May 2013) that collects outputs from researches in all fields of science to promote open access and open data. It is supported by the OpenAIRE initiative and developed and hosted by CERN.<br />
<br />
Each outcome uploaded (called ''Deposition'' in the Zenodo gergo) is stored by Zenodo along with a rich set of metadata, searchable via the portal and harvestable via the OAI-PMH protocol.<br />
<br />
Zenodo assigns to each deposition a unique Digital Object Identifier ('''DOI''') to to make the upload easily and uniquely citeable.<br />
<br />
<br />
= gCube Community =<br />
The gCube Community is a community created with the objective of collecting all the depositions related to '''gCube software'''. New software artifacts are automatically uploaded at every gCube release and metadata is generated automatically (see [[#Automatic Metadata Generation]]).<br />
<br />
<br />
[[File:zenodo-gcube-community.png|500px|center]]<br />
<br />
= Automatic Metadata Generation =<br />
<br />
The content for each deposition is generated automatically by the Zenodo Published from the information in the component source code and in ETICS.<br />
<br />
[[File:zenodo.png|500px|center]]<br />
<br />
== 1. Title ==<br />
The title of the deposition is built as follow:<br />
<pre><br />
gCube <gCube Version> - <name> v. <version><br />
</pre><br />
<br />
where:<br />
* gCube Version is the version of gCube (e.g. 3.11.1, 4.0.0, ...)<br />
* name is the content of <name> tag in the pom.xml. If not present, the artifactId is used<br />
* version is the version in the pom<br />
<br />
== 2. Authors ==<br />
Built from the content of "vcs.authors" property in ETICS configuration. That property is computed by ETICS looking at the VCS history of the component<br />
<br />
== 3. Description ==<br />
It corresponds to the content of README file.<br />
<br />
== 4. Files ==<br />
The only file uploaded is the source package. It is generated during ETICS builds and contains everything checked-out from the VCS <br />
<br />
== 5. GitHub link ==<br />
The link to the position of the source code in GitHub<br />
<br />
== 6. DOI ==<br />
assigned by Zenodo to this deposition. <br />
<br />
== 7. Keywords ==<br />
Keywords for each component are stored in ETICS in the the description field of the corresponding ETICS module (not the configuration). For instance, keywords for "Common Authorization" are stored in the description of "org.gcube.common.authorization-common" module.<br />
They are encoded as follow:<br />
<pre><br />
#keywords=keyword1, keyword 2, key word3, ...<br />
</pre><br />
<br />
If keywords are not found in the component, they are searched in the subsystem and if also in the subsystem are not found, the keywords for the project are used. org.gcube project define these keywords:<br />
<pre><br />
#keywords=gCube, Java, Data Management System, Hybrid Data Infrastructure<br />
</pre><br />
<br />
<br />
== 8. Grants ==<br />
Static list of grants. The same for all depositions<br />
<br />
== 9. Previous Version Reference ==<br />
The DOI of the previous version of this component<br />
<br />
== 10. GitHub Reference ==<br />
(same as 5)<br />
<br />
== 11. PackageId ==<br />
internal identifier to uniquely identify the depositions<br />
<br />
== 12. Commiunity ==<br />
All depositions belongs to the gCube community<br />
<br />
== 13. License ==<br />
Statically set to EUPL-1.1<br />
<br />
= Possible Improvements =<br />
<br />
* Add Changelog in the description<br />
<br />
* Add links to wiki (parsed from the README)<br />
<br />
* Support for a ".zenodo" file in the projet root that override metadata generated automatically<br />
<br />
* Make a Smartgears service<br />
<br />
* More metadata can be added:<br />
** ''Subjects'' (however Subjects does not appear in Zenodo)</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Zenodo_Publication&diff=31056Zenodo Publication2018-04-09T07:43:37Z<p>Gabriele.giammatteo: /* gCube Community */</p>
<hr />
<div>Zenodo is a portal (launched in May 2013) that collects outputs from researches in all fields of science to promote open access and open data. It is supported by the OpenAIRE initiative and developed and hosted by CERN.<br />
<br />
Each outcome uploaded (called ''Deposition'' in the Zenodo gergo) is stored by Zenodo along with a rich set of metadata, searchable via the portal and harvestable via the OAI-PMH protocol.<br />
<br />
Zenodo assigns to each deposition a unique Digital Object Identifier ('''DOI''') to to make the upload easily and uniquely citeable.<br />
<br />
<br />
= gCube Community =<br />
The gCube Community is a community created with the objective of collecting all the depositions related to '''gCube software'''. New software artifacts are automatically uploaded at every gCube release and metadata is generated automatically <br />
<br />
[[File:zenodo-gcube-community.png|500px|center]]<br />
<br />
= Automatic Metadata Generation =<br />
<br />
The content for each deposition is generated automatically by the Zenodo Published from the information in the component source code and in ETICS.<br />
<br />
[[File:zenodo.png|500px|center]]<br />
<br />
== 1. Title ==<br />
The title of the deposition is built as follow:<br />
<pre><br />
gCube <gCube Version> - <name> v. <version><br />
</pre><br />
<br />
where:<br />
* gCube Version is the version of gCube (e.g. 3.11.1, 4.0.0, ...)<br />
* name is the content of <name> tag in the pom.xml. If not present, the artifactId is used<br />
* version is the version in the pom<br />
<br />
== 2. Authors ==<br />
Built from the content of "vcs.authors" property in ETICS configuration. That property is computed by ETICS looking at the VCS history of the component<br />
<br />
== 3. Description ==<br />
It corresponds to the content of README file.<br />
<br />
== 4. Files ==<br />
The only file uploaded is the source package. It is generated during ETICS builds and contains everything checked-out from the VCS <br />
<br />
== 5. GitHub link ==<br />
The link to the position of the source code in GitHub<br />
<br />
== 6. DOI ==<br />
assigned by Zenodo to this deposition. <br />
<br />
== 7. Keywords ==<br />
Keywords for each component are stored in ETICS in the the description field of the corresponding ETICS module (not the configuration). For instance, keywords for "Common Authorization" are stored in the description of "org.gcube.common.authorization-common" module.<br />
They are encoded as follow:<br />
<pre><br />
#keywords=keyword1, keyword 2, key word3, ...<br />
</pre><br />
<br />
If keywords are not found in the component, they are searched in the subsystem and if also in the subsystem are not found, the keywords for the project are used. org.gcube project define these keywords:<br />
<pre><br />
#keywords=gCube, Java, Data Management System, Hybrid Data Infrastructure<br />
</pre><br />
<br />
<br />
== 8. Grants ==<br />
Static list of grants. The same for all depositions<br />
<br />
== 9. Previous Version Reference ==<br />
The DOI of the previous version of this component<br />
<br />
== 10. GitHub Reference ==<br />
(same as 5)<br />
<br />
== 11. PackageId ==<br />
internal identifier to uniquely identify the depositions<br />
<br />
== 12. Commiunity ==<br />
All depositions belongs to the gCube community<br />
<br />
== 13. License ==<br />
Statically set to EUPL-1.1<br />
<br />
= Possible Improvements =<br />
<br />
* Add Changelog in the description<br />
<br />
* Add links to wiki (parsed from the README)<br />
<br />
* Support for a ".zenodo" file in the projet root that override metadata generated automatically<br />
<br />
* Make a Smartgears service<br />
<br />
* More metadata can be added:<br />
** ''Subjects'' (however Subjects does not appear in Zenodo)</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Zenodo_Publication&diff=31055Zenodo Publication2018-04-09T07:40:16Z<p>Gabriele.giammatteo: </p>
<hr />
<div>Zenodo is a portal (launched in May 2013) that collects outputs from researches in all fields of science to promote open access and open data. It is supported by the OpenAIRE initiative and developed and hosted by CERN.<br />
<br />
Each outcome uploaded (called ''Deposition'' in the Zenodo gergo) is stored by Zenodo along with a rich set of metadata, searchable via the portal and harvestable via the OAI-PMH protocol.<br />
<br />
Zenodo assigns to each deposition a unique Digital Object Identifier ('''DOI''') to to make the upload easily and uniquely citeable.<br />
<br />
<br />
= gCube Community =<br />
The gCube Community is a community created with the objective of collecting all the depositions related to gCube software.<br />
<br />
[[File:zenodo-gcube-community.png|500px|center]]<br />
<br />
<br />
= Automatic Metadata Generation =<br />
<br />
The content for each deposition is generated automatically by the Zenodo Published from the information in the component source code and in ETICS.<br />
<br />
[[File:zenodo.png|500px|center]]<br />
<br />
== 1. Title ==<br />
The title of the deposition is built as follow:<br />
<pre><br />
gCube <gCube Version> - <name> v. <version><br />
</pre><br />
<br />
where:<br />
* gCube Version is the version of gCube (e.g. 3.11.1, 4.0.0, ...)<br />
* name is the content of <name> tag in the pom.xml. If not present, the artifactId is used<br />
* version is the version in the pom<br />
<br />
== 2. Authors ==<br />
Built from the content of "vcs.authors" property in ETICS configuration. That property is computed by ETICS looking at the VCS history of the component<br />
<br />
== 3. Description ==<br />
It corresponds to the content of README file.<br />
<br />
== 4. Files ==<br />
The only file uploaded is the source package. It is generated during ETICS builds and contains everything checked-out from the VCS <br />
<br />
== 5. GitHub link ==<br />
The link to the position of the source code in GitHub<br />
<br />
== 6. DOI ==<br />
assigned by Zenodo to this deposition. <br />
<br />
== 7. Keywords ==<br />
Keywords for each component are stored in ETICS in the the description field of the corresponding ETICS module (not the configuration). For instance, keywords for "Common Authorization" are stored in the description of "org.gcube.common.authorization-common" module.<br />
They are encoded as follow:<br />
<pre><br />
#keywords=keyword1, keyword 2, key word3, ...<br />
</pre><br />
<br />
If keywords are not found in the component, they are searched in the subsystem and if also in the subsystem are not found, the keywords for the project are used. org.gcube project define these keywords:<br />
<pre><br />
#keywords=gCube, Java, Data Management System, Hybrid Data Infrastructure<br />
</pre><br />
<br />
<br />
== 8. Grants ==<br />
Static list of grants. The same for all depositions<br />
<br />
== 9. Previous Version Reference ==<br />
The DOI of the previous version of this component<br />
<br />
== 10. GitHub Reference ==<br />
(same as 5)<br />
<br />
== 11. PackageId ==<br />
internal identifier to uniquely identify the depositions<br />
<br />
== 12. Commiunity ==<br />
All depositions belongs to the gCube community<br />
<br />
== 13. License ==<br />
Statically set to EUPL-1.1<br />
<br />
= Possible Improvements =<br />
<br />
* Add Changelog in the description<br />
<br />
* Add links to wiki (parsed from the README)<br />
<br />
* Support for a ".zenodo" file in the projet root that override metadata generated automatically<br />
<br />
* Make a Smartgears service<br />
<br />
* More metadata can be added:<br />
** ''Subjects'' (however Subjects does not appear in Zenodo)</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=File:Zenodo-gcube-community.png&diff=31054File:Zenodo-gcube-community.png2018-04-09T07:36:58Z<p>Gabriele.giammatteo: Gabriele.giammatteo uploaded a new version of File:Zenodo-gcube-community.png</p>
<hr />
<div></div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=File:Zenodo.png&diff=31053File:Zenodo.png2018-04-09T07:36:23Z<p>Gabriele.giammatteo: Gabriele.giammatteo uploaded a new version of File:Zenodo.png</p>
<hr />
<div></div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=FHNManager_Installation&diff=31040FHNManager Installation2018-03-22T08:04:50Z<p>Gabriele.giammatteo: </p>
<hr />
<div>The Federated Hosting Node Manager (FHNM) is the core part of the gCube-FedCloud integration.<br />
<br />
<br />
= Usage =<br />
== Maven coordinates ==<br />
The maven artifact coordinates are: <br />
<dependency><br />
<groupId>org.gcube.resources.federation</groupId><br />
<artifactId>fhn-manager-service</artifactId><br />
<version>1.0.0-SNAPSHOT</version> <br />
<packaging>war</packaging><br />
</dependency><br />
<br />
= Deployment =<br />
<br />
Components related to the D4Science-FedCloud integration are essentially assembled in two packages that are deployed independently:<br />
* the Federated Hosting Node Manager, embedding cloud libraries and connectors, is deployed in a SmartGears container<br />
* the FHNManagerPortlet is deployed in the gCube portal.<br />
<br />
Please deploy the war file related to the service into the SmartGears Container (tomcat Webapp folder)<br />
<br />
= Authentication =<br />
<br />
In order to use the Federated Cloud, it is necessary to register a certificate to join a Virtual Organisation (VO) of the cloud.<br />
In case of integration with EGI it is necessary to generate a VOMS proxy in order to obtain authorization attributes to be embedded in X.509 proxy certificates, needed to access FedCloud sites.<br />
<br />
Ideally, the VRE Manager should have to import the credentials related to own VRE directly from the gCube IS; actually the service foresees an interface for the future integration with the IS, but currently we're using VRE Manager certificates (actually a second-level proxy of the manager's cert. The first one is stored, encrypted, in the gCube information system; the second one is generated by the service just before accessing FedCloud and it's really short-lived; usually it is available at ''/tmp'' folder).<br />
<br />
Most, if not all, gCube users do not own a x509 certificate; they access the gCube portal with username/password and the usage of external infrastructures is completely transparent to them.<br />
Basically the gCube Information System will contain the .pem certificated associated to each VRE or Infrastructure Manager. Such certificates will be used by the service to create a runtime VOMS proxy in order to interact with FedCloud. If no certificates are provided, no interaction with FedCloud will be allowed.<br />
<br />
Please refer to [https://wiki.egi.eu/wiki/Fedcloud-tf:CLI_Environment How to use the rOCCI Client] page for the correct VOMS configuration steps to follow.<br />
<br />
To create the first level proxy:<br />
voms-proxy-init -voms d4science.org -rfc -cert INFN_2018.p12 -out ggproxy.pem --valid 8760:00<br />
<br />
<br />
= Configuration Steps =<br />
<br />
<span style="background:yellow">'''OUTDATED SECTION BECAUSE THE NEW VERSION OF THE FHN-MANAGER USES THE IS TO STORE/RETRIEVE VMPROVIDERS, NODES AND TEMPLATES'''.</span><br />
<br />
Currently, the data persistency useful to configuration of the service is file-based; in the future, such data (e.g., Service Profiles, Target definition, etc..) will be stored/extracted directly from the gCube Information System repository. <br />
For a correct configuration of the service, please refer to subsections below.<br />
<br />
== Target definition ==<br />
First, it is necessary to configure the target cloud platforms to add to configuration.<br />
To achieve that, please edit ''vmproviders.yml'' file (available to ../classes/ folder) and insert data (a list if you desire to add multiple resources) as follows: <br />
<source lang="html4strict"><br />
---<br />
credentials: {type: x509, <br />
encodedCredentails: /tmp/x509up_u1004}<br />
endpoint: https://carach5.ics.muni.cz:11443/<br />
id: 4-1<br />
name: Cesnet-Metacloud<br />
resourceTemplates: <br />
nodeTemplates: <br />
- refId: 3-1<br />
---<br />
</source><br />
<br />
Credentials will specify:<br />
* the path to the second level Proxy certificate<br />
* the typology (x509)<br />
Other fields are related to:<br />
* the endpoint (please refer to [https://appdb.egi.eu/ AppDb] for a list of sites exposing occi and supporting gCube smartExecutor Virtual Appliance)<br />
* the provider ID<br />
* the provider name<br />
* the reference to node template file<br />
The resource template describe the resources (in terms of memory, number of cores, etc..).<br />
In FedCloud case, such field is left to blank; so far, the list of templates extracted from the [https://gcube.wiki.gcube-system.org/gcube/OCCI_library fhn-occi-connector] is stored in ''ResourceTemplates.yml'' file (as GUI requirement) in order to allow the user to collect the template Id and use it as parameter when using APIs; In the next releases, with the introduction of new providers to integrate with gCube, the development of a caching system linking a resource templates repository could be considered. Thus, also the ''resourceTemplates'' attibute will refer an Id.<br />
An example of resource template representation is shown in the following:<br />
<source lang="xml"><br />
<resourceTemplates><br />
<resourceTemplate><br />
<id>http://fedcloud.egi.eu/occi/compute/flavour/1.0#mem_small</id><br />
<cores>1</cores><br />
<memory>4294967296</memory><br />
<name>Small Instance - 1 core and 4 GB RAM</name><br />
<vmProvider refId="4-1"/><br />
</resourceTemplate><br />
</source><br />
<br />
== Define Node Template ==<br />
Node Templates must be defined in ''nodetemplates.yml'' file (available to ../classes/ folder).<br />
<br />
<source lang="html4strict"><br />
---<br />
id: 3-1<br />
os: <br />
osVersion: <br />
name: <br />
description:<br />
version:<br />
diskSize: <br />
script: https://appdb.egi.eu/storage/cs/vapp/15819120-7ee4-4b85-818a-d9bd755a61f0/devsec-init<br />
osTemplateId: http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_gcubesmartexecutor_fedcloud_warg_139<br />
serviceProfile: <br />
refId: 2-1<br />
</source><br />
<br />
Fields of interest are the following:<br />
* the Node template ID<br />
* the URL of the contextualisation script that will be used to instantiate a virtual machine able to be monitored by the d4Science infrastructure<br />
* the id related to the image (OS template) obtained by [https://appdb.egi.eu/ AppDb] (in FedCloud case, just the one related to SmartExecutor is considered); In the next releases, with the introduction of new providers to integrate with gCube, the development of a caching system linking an osTemplates repository could be considered. Thus, also the osTemplate will refer an Id.<br />
* the reference to a service profile<br />
<br />
== Define Service Profile ==<br />
<br />
According to domain model, the service profile defines the software specifications running on the instance.<br />
It must be defined in ''serviceprofiles.yml'' file (available to ../classes/ folder). <br />
An example of service profiles definition can be found below:<br />
<br />
<source lang="html4strict"><br />
---<br />
deployedSoftware:<br />
- refId: s2<br />
description: gCube Smart Executor<br />
creationDate: 06-Oct-2015<br />
version: 1.2.0<br />
id: 2-1<br />
minRam: 4294967296<br />
minCores: 1<br />
suggestedRam: 8589934592<br />
suggestedCores: 2<br />
---<br />
</source><br />
<br />
== Edit properties file ==<br />
<br />
Please edit the YAML service.properties file available in WEB.INF/classes folder and replace the ''STORAGE_DIR: /home/"user_name"/fhnmanager'' path with user home path (e.g., ''STORAGE_DIR: /home/ngalante/fhnmanager''); such folder will contain the persistency of nodes created by using the service.<br />
An example of nodes created by the service and stored in ''nodes.yml'' file is shown below:<br />
<br />
<source lang="html4strict"><br />
---<br />
!!org.gcube.resources.federation.fhnmanager.api.type.Node<br />
hostname: hostname<br />
id: 4-1@https://carach5.ics.muni.cz:11443/compute/68051<br />
nodeTemplate: {refId: 3-1}<br />
resourceTemplate: {refId: 'http://fedcloud.egi.eu/occi/compute/flavour/1.0#small'}<br />
serviceProfile: {refId: 2-1}<br />
status: waiting<br />
vmProvider: {refId: 4-1}<br />
workload: {allTimeAverageWorkload: 0.5, lastDayWorkload: 0.7, lastHourWorkload: 0.2,<br />
nowWorkload: 0.8}<br />
---<br />
!!org.gcube.resources.federation.fhnmanager.api.type.Node<br />
hostname: stoor180.meta.zcu.cz<br />
id: 4-1@https://carach5.ics.muni.cz:11443/compute/68102<br />
nodeTemplate: {refId: 3-1}<br />
resourceTemplate: {refId: 'http://fedcloud.egi.eu/occi/compute/flavour/1.0#small'}<br />
serviceProfile: {refId: 2-1}<br />
status: active<br />
vmProvider: {refId: 4-1}<br />
workload: {allTimeAverageWorkload: 0.5, lastDayWorkload: 0.7, lastHourWorkload: 0.2,<br />
nowWorkload: 0.8}<br />
---<br />
!!org.gcube.resources.federation.fhnmanager.api.type.Node<br />
hostname: stoor154.meta.zcu.cz<br />
id: 4-1@https://carach5.ics.muni.cz:11443/compute/68942<br />
nodeTemplate: {refId: 3-1}<br />
resourceTemplate: {refId: 'http://fedcloud.egi.eu/occi/compute/flavour/1.0#small'}<br />
serviceProfile: {refId: 2-1}<br />
status: suspended<br />
vmProvider: {refId: 4-1}<br />
#workload: null<br />
---<br />
</source><br />
<br />
<br />
Since the nodes are created/started/stopped/deleted via [https://wiki.gcube-system.org/gcube/OCCI_library fhn-occi-connector], it could take time to retrieve consistent information (e.g., the ''status'' attribute, is initially populated as "waiting" and just actually later it becomes "active").<br />
To solve this kind of issue, a mechanism of synchronization between the data retrieved from the connector and the persistency currently stored locally from the service has been implemented (in the future, such persistency will be stored directly in the gCube Information System repository).<br />
<br />
= Nodes Monitoring =<br />
<br />
The created instances will appear in the [https://next.d4science.org/web/guest/monitor D4Science Monitoring system].<br />
Data collected is analysed to produce alarms and/or take countermeasures in case of problems with one or more resources.<br />
<br />
[[File:Monitor.png]]<br />
<br />
By selecting an hosting node, many information concerning the resource will be available.<br />
<br />
[[File:host.png]]</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Software_Integration_and_Distribution:_Overview&diff=30796Software Integration and Distribution: Overview2018-01-19T07:50:10Z<p>Gabriele.giammatteo: </p>
<hr />
<div>__NOTOC__<br />
<br />
This section documents all the procedures and tools used for releasing the gCube system. It is intended as the central access point to information about the integration, validation, and distribution of the system for developers, software testers, release managers, and for any other actor involved in the development of the system (see [[Software_Integration_and_Distribution:_Roles|Roles]]).<br />
<br />
Given the large size of the gCube system, its high degree of modularity and the number of developers involved in the implementation, a set of procedures that formalize and simplify the integration, test, documentation and distribution of the software have been defined. These procedures coordinate the actions and the timing of the different actors to achieve the desired result. Section [[Release Procedures]] provides a detailed description of each procedure by listing actors and their responsibilities for each step of each procedure.<br />
<br />
Activities in release procedures are highly automated and supported by several tools described in section [[Release Tools|Release Tools]].<br />
<br />
==== [[Software Integration and Distribution: Roles|Release Roles]] ====<br />
* [[Role Release Manager|Release Manager]]<br />
* [[Role Subsystem Manager|Subsystem Manager]]<br />
* [[Role Developer|Developer]]<br />
* [[Role Tester|Tester]]<br />
* [[Role Documentation Editor|Documentation Editor]]<br />
* [[Role Documentation Author|Documentation Author]]<br />
<br />
==== [[Release Tools|Release Tools]] ====<br />
* [[Redmine|Tracking System]]<br />
* [[Integration of Maven Components|Maven]]<br />
** [[Developing gCube Maven Components|gCube Maven Components]]<br />
** [[GCube_Maven_BOMs|gCube Maven BOMs]]<br />
* [[Integration Infrastructure]]<br />
** <span style="color:DarkGreen">'''''NEW'''''</span> [[GCube_Git_Migration|gCube Git Migration State Of the Art]]<br />
** [[gcube subversion repository|SVN Repository]]<br />
** [[ETICS|ETICS]]<br />
*** [[MavenBuilder|MavenBuilder]]<br />
** [[BTRT|BTRT]]<br />
** [[gCube Web Site and Distribution Site|Distribution Sites]]<br />
*** [[GitHub|GitHub]]<br />
* [[Testing Infrastructure|Testing Infrastructure]]<br />
* [[gCube System documentation wiki|Documentation]]<br />
<br />
==== [[Release Procedures|Release Procedures]] ====<br />
* [[Continuous Integration procedure|Continuous Integration]]<br />
* [[Major/Minor Release Cycle procedure|Release Cycle]]<br />
* [[Maintenance Release Cycle procedure|Maintenance Release Cycle]]<br />
* [[Documentation Writing guidelines|Documentation Writing Guidelines]]<br />
** [[Changelog Generation procedure|Changelog Generation Procedure]]<br />
* [[Distribution procedure|Distribution Procedure]]<br />
** [[Zenodo Publication|Zenodo Publication]]<br />
<br />
==== [[Test Plan|Test Plan]] ====<br />
* [[Static Code Analysis|Static Code Analysis]]<br />
* [[Unit Testing|Unit Testing]]<br />
* [[Functional Testing|Functional Testing]]<br />
* [[Deployment Testing|Deployment Testing]]<br />
<br />
==== [[Naming Conventions|Naming Conventions]] ====<br />
<br />
==== [[Software Integration and Distribution: Release Log|Release Log]] ====</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Software_Integration_and_Distribution:_Overview&diff=30749Software Integration and Distribution: Overview2018-01-15T19:06:48Z<p>Gabriele.giammatteo: </p>
<hr />
<div>__NOTOC__<br />
<br />
This section documents all the procedures and tools used for releasing the gCube system. It is intended as the central access point to information about the integration, validation, and distribution of the system for developers, software testers, release managers, and for any other actor involved in the development of the system (see [[Software_Integration_and_Distribution:_Roles|Roles]]).<br />
<br />
Given the large size of the gCube system, its high degree of modularity and the number of developers involved in the implementation, a set of procedures that formalize and simplify the integration, test, documentation and distribution of the software have been defined. These procedures coordinate the actions and the timing of the different actors to achieve the desired result. Section [[Release Procedures]] provides a detailed description of each procedure by listing actors and their responsibilities for each step of each procedure.<br />
<br />
Activities in release procedures are highly automated and supported by several tools described in section [[Release Tools|Release Tools]].<br />
<br />
==== [[Software Integration and Distribution: Roles|Release Roles]] ====<br />
* [[Role Release Manager|Release Manager]]<br />
* [[Role Subsystem Manager|Subsystem Manager]]<br />
* [[Role Developer|Developer]]<br />
* [[Role Tester|Tester]]<br />
* [[Role Documentation Editor|Documentation Editor]]<br />
* [[Role Documentation Author|Documentation Author]]<br />
<br />
==== [[Release Tools|Release Tools]] ====<br />
* [[Redmine|Tracking System]]<br />
* [[Integration of Maven Components|Maven]]<br />
** [[Developing gCube Maven Components|gCube Maven Components]]<br />
** [[GCube_Maven_BOMs|gCube Maven BOMs]]<br />
* [[Integration Infrastructure]]<br />
** <span style="color:DarkGreen">'''NEW'''</span> [[GCube_Git_Migration|gCube Git Migration State Of the Art]]<br />
** [[gcube subversion repository|SVN Repository]]<br />
** [[ETICS|ETICS]]<br />
*** [[MavenBuilder|MavenBuilder]]<br />
** [[BTRT|BTRT]]<br />
** [[gCube Web Site and Distribution Site|Distribution Sites]]<br />
*** [[GitHub|GitHub]]<br />
* [[Testing Infrastructure|Testing Infrastructure]]<br />
* [[gCube System documentation wiki|Documentation]]<br />
<br />
==== [[Release Procedures|Release Procedures]] ====<br />
* [[Continuous Integration procedure|Continuous Integration]]<br />
* [[Major/Minor Release Cycle procedure|Release Cycle]]<br />
* [[Maintenance Release Cycle procedure|Maintenance Release Cycle]]<br />
* [[Documentation Writing guidelines|Documentation Writing Guidelines]]<br />
** [[Changelog Generation procedure|Changelog Generation Procedure]]<br />
* [[Distribution procedure|Distribution Procedure]]<br />
** [[Zenodo Publication|Zenodo Publication]]<br />
<br />
==== [[Test Plan|Test Plan]] ====<br />
* [[Static Code Analysis|Static Code Analysis]]<br />
* [[Unit Testing|Unit Testing]]<br />
* [[Functional Testing|Functional Testing]]<br />
* [[Deployment Testing|Deployment Testing]]<br />
<br />
==== [[Naming Conventions|Naming Conventions]] ====<br />
<br />
==== [[Software Integration and Distribution: Release Log|Release Log]] ====</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Software_Integration_and_Distribution:_Overview&diff=30748Software Integration and Distribution: Overview2018-01-15T19:01:32Z<p>Gabriele.giammatteo: /* Release Tools */</p>
<hr />
<div>__NOTOC__<br />
<br />
This section documents all the procedures and tools used for releasing the gCube system. It is intended as the central access point to information about the integration, validation, and distribution of the system for developers, software testers, release managers, and for any other actor involved in the development of the system (see [[Software_Integration_and_Distribution:_Roles|Roles]]).<br />
<br />
Given the large size of the gCube system, its high degree of modularity and the number of developers involved in the implementation, a set of procedures that formalize and simplify the integration, test, documentation and distribution of the software have been defined. These procedures coordinate the actions and the timing of the different actors to achieve the desired result. Section [[Release Procedures]] provides a detailed description of each procedure by listing actors and their responsibilities for each step of each procedure.<br />
<br />
Activities in release procedures are highly automated and supported by several tools described in section [[Release Tools|Release Tools]].<br />
<br />
==== [[Software Integration and Distribution: Roles|Release Roles]] ====<br />
* [[Role Release Manager|Release Manager]]<br />
* [[Role Subsystem Manager|Subsystem Manager]]<br />
* [[Role Developer|Developer]]<br />
* [[Role Tester|Tester]]<br />
* [[Role Documentation Editor|Documentation Editor]]<br />
* [[Role Documentation Author|Documentation Author]]<br />
<br />
==== [[Release Tools|Release Tools]] ====<br />
* [[Redmine|Tracking System]]<br />
* [[Integration of Maven Components|Maven]]<br />
** [[Developing gCube Maven Components|gCube Maven Components]]<br />
** [[GCube_Maven_BOMs|gCube Maven BOMs]]<br />
* [[Integration Infrastructure]]<br />
** [[gcube subversion repository|SVN Repository]]<br />
** [[ETICS|ETICS]]<br />
*** [[MavenBuilder|MavenBuilder]]<br />
** [[BTRT|BTRT]]<br />
** [[gCube Web Site and Distribution Site|Distribution Sites]]<br />
*** [[GitHub|GitHub]]<br />
* [[Testing Infrastructure|Testing Infrastructure]]<br />
* [[gCube System documentation wiki|Documentation]]<br />
* [[GCube_Git_Migration|gCube Git Migration]]<br />
<br />
==== [[Release Procedures|Release Procedures]] ====<br />
* [[Continuous Integration procedure|Continuous Integration]]<br />
* [[Major/Minor Release Cycle procedure|Release Cycle]]<br />
* [[Maintenance Release Cycle procedure|Maintenance Release Cycle]]<br />
* [[Documentation Writing guidelines|Documentation Writing Guidelines]]<br />
** [[Changelog Generation procedure|Changelog Generation Procedure]]<br />
* [[Distribution procedure|Distribution Procedure]]<br />
** [[Zenodo Publication|Zenodo Publication]]<br />
<br />
==== [[Test Plan|Test Plan]] ====<br />
* [[Static Code Analysis|Static Code Analysis]]<br />
* [[Unit Testing|Unit Testing]]<br />
* [[Functional Testing|Functional Testing]]<br />
* [[Deployment Testing|Deployment Testing]]<br />
<br />
==== [[Naming Conventions|Naming Conventions]] ====<br />
<br />
==== [[Software Integration and Distribution: Release Log|Release Log]] ====</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=GCube_Git_Migration&diff=30747GCube Git Migration2018-01-15T18:59:56Z<p>Gabriele.giammatteo: /* Updates to Release Procedure */</p>
<hr />
<div>__TOC__<br />
<br />
During the BlueBRIDGE project, we started an activity for migrating the gCube source code from the Subversion (SVN) technology to Git. An analysis of the requirements and the design of a proper process and tools to governate the migration has been conducted. However, due to resources constraints, the actual migration has not been implemented.<br />
<br />
This document collects the output of all the work done and wants to be an helpful starting point for the future implementation of the migration of gCube source code from SVN to Git.<br />
<br />
= Requirements =<br />
<br />
The migration will affect the tools and the procedures currently used in the gCube development, integration and release. We started our analysis by considering some basic requirements that the final solution should address.<br />
<br />
'''Gradual implementation'''<br />
<br />
The migration should be designed to allow a gradual migration of gCube components allowing for continuing the normal development, integration and release activities even if some components have been already migrated and some others not yet. In this way, no major delays will be introduced in the gCube release scheduling. To address this requirement, the procedures and the tools used in the integration of gCube must support both SVN and Git repositories and allow to seamless integration between components stored in SVN and Git.<br />
<br />
'''History of files'''<br />
<br />
The migration should keep the history of the source code. This means that not only the latest version of each source file should be available in the new Git repository, but also all the single commits of each single source file with, possibly, the associated meta-data (i.e. date, author, commit message) should be preserved.<br />
<br />
'''Branching schema'''<br />
<br />
In the current SVN repository, each component has multiple branches: a) the development branch (i.e. ''trunk''), b) one release branch for each release and, optionally, c) one or more private branch (or feature branch). This allows developers to organize better their code and to maintain multiple versions of their component in parallel. The same functionality should be preserved when migrating to Git.<br />
<br />
'''GitHub publication'''<br />
<br />
Currently the source code of each gCube release (composed by the source code of all components that constitute the release) is replicated in a repository on GitHub. The current solution takes the source code to publish on GitHub from the source packages built by Maven during the builds and published on Nexus. This was the only viable solution to publish the source code from SVN, but it has the disadvantage that the history and the authors of the source files are not kept when copied on GitHub. After the migration, since the technology of gCube source code repositories and GitHub will be the same (Git), it should be possible also to publish on GitHub the authors and the history of each source file.<br />
<br />
'''Git repositories management software'''<br />
<br />
The software that will be used to manage the Git repositories should guarantee at least the following functionalities:<br />
<br />
* integration with the ''research infrastructures LDAP'': this will maintain the developers identity in the new Git repositories;<br />
* REST interface to create new repositories: this will increase the automation of the migration operation;<br />
* management of roles: the source code should be readable to everybody without authentication, but only authorized users should be able to make changes;<br />
* web interface for browsing the source code and the commits;<br />
<br />
Additionally, the tool should also implement collaboration functionalities like fork of repositories, pull requests, code review. This would greatly improve the collaboration between gCube developers and external community developers.<br />
<br />
= Git Repository Layout =<br />
<br />
Currently the gCube source code is maintained in a single SVN repository available at http://svn.research-infrastructures.eu/public/d4science/gcube/. The repository has the following layout:<br />
<br />
* ''trunk'' for the development version of components<br />
* ''branches'' for the release branches of components<br />
* ''private'' for feature branches of components<br />
* ''tags'' to keep a copy of gCube source code for each gCube release<br />
<br />
More information on the current SVN layout can be found in the [https://wiki.gcube-system.org/gcube/Gcube_subversion_repository gCube wiki].<br />
<br />
Switching to Git, the single repository layout will be abandoned in favor of a '''per-component repository layout'''. Each component will have its own Git repository that will contain all the branches for that component. This approach is not only more in-line with the Git philosophy, but, mostly important, more practical.<br />
<br />
In fact, while SVN allows for checkout and commit of single files/folders, Git (by design) does not provide this functionality. This mean that each operation (e.g. checkout of source code to execute a build) involves the entire repository. Given the size of gCube system (hundred of components and more than 1GB of source code), it is not practical to have all the source code in a single repository. In addition, Git allows to assign permissions only at repository level and having a per-component repository layout can be useful if we decide to assign permissions on component basis.<br />
<br />
The following table maps the current SVN layout with the new proposed Git layout also considering the workflow that we want to adopt (see [[#Updates to Release Procedure|Updates to Release Procedure]]).<br />
<br />
{|border="1" cellpadding="4" cellspacing="0"<br />
|- style="background-color:#F2F2F2"<br />
! type<br />
! SVN<br />
! Git<br />
|-<br />
| development branch<br />
| the developments are all located under the /trunk directory organized per component (e.g. <code>/trunk/common/common-client</code>)<br />
| the <code>development</code> branch<br />
|-<br />
| release branch<br />
| the releases are all located under the /branches directory organized per component and per version (e.g. <code>/branches/common/common-client/1.1</code>)<br />
| a branch named accordingly with the version (e.g. <code>1.1</code>)<br />
|-<br />
| feature branch<br />
| users place their feature branches in the /private directory organized per component (e.g. <code>/private/common/common-client/my-feature-1</code>)<br />
| a branch named accordingly with the feature (e.g. <code>my-feature-1</code>)<br />
|-<br />
| tags<br />
| tags are copies of the source code under the /tags directory organized per gCube release, component and version (e.g. <code>/tags/gcube-4.5.0/common/common-client/1.1</code>)<br />
| a tag of the most recent commit in the release branch is created with the full version of the release (e.g. <code>1.1.0-4.5.0</code>)<br />
|}<br />
<br />
= Import of source code from SVN =<br />
<br />
A strong requirement of the migration to Git is to maintain the history of the source code for each component. To achieve this, it is not sufficient to simply create an empty Git repository and copy the latest version of files from SVN.<br />
<br />
Since this is a very common requirement when migration from SVN to Git, there is a command line tool distributed directly with Git to address this problem. This was the base for our importing tests described below. There are also few alternatives reported at the end of the section.<br />
<br />
'''[https://git-scm.com/docs/git-svn git-svn]''' is a tool that given a SVN repository URL is able to discover all the files and all the commits in the repository and replicate them in a new Git repository. It is very complete and highly customizable. It supports different repository layouts, can distinguish between branches and tags and tries to follow the history of each file even if it has been copied or moved in different locations at certain point in the repository history. It is also able to keep user identities (by providing a mapping file of SVN and Git users) and can ignore specific directories and/or revisions. It is distributed under the open source GPLv2 license.<br />
<br />
The tests we executed highlighted that given the size (both in terms of number of files and number of revisions) of the gCube SVN repository the import can take several time (hours) even for a single component because the tool will try to scan the entire repository looking for branches and tags of that component. However, using some options, it is possible to limit the tool to only scan the folders where we know the source files are and only for the revisions we know the source files exist. This reduces drastically the importing times (from hours to minutes). However, it has to be considered that this depends highly from the &quot;age&quot; of the component. In fact old components, for which we need to scan more revisions, will be slower that new components (with fewer revisions to scan) and from the number of commits/releases existing.<br />
<br />
The following command imports the code of the home-library component (trunk and branches) from revision 82000 to the current one:<br />
<br />
<source lang="bash">> git svn clone http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/Common/<br />
home-library/ --trunk http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/Common/home-library/ --branches http://svn.research-infrastructures.eu/public/d4science/gcube/branches/common/home-library/ --authors-file=users.txt --no-metadata --no-minimize-url --revision 82000:HEAD<br />
</source><br />
This test took about 10 minutes (tried in July 2017).<br />
<br />
This command can be used (setting the correct parameters) for importing all gCube components. To facilitate the users, it can be wrapped in a more high-level script that is able to determine (or, at least, to guess) the values for almost all command parameters:<br />
<br />
* the min and max revisions can be found by running <code>svn log</code> and extract the revision of the first and the latest commits;<br />
* the branches and tags directory can be guessed from the trunk directory using the gCube naming conventions;<br />
* the authors-file can be generated upfront (see [https://john.albin.net/git/convert-subversion-to-git here]) and it is the stame for all the components.<br />
<br />
In addition the wrapper script can create and publish the new Git repository using the repository software APIs.<br />
<br />
== Alternative tools ==<br />
<br />
'''[https://github.com/JohnAlbin/git-svn-migrate git-svn-migrate]''' is just a wrapper script around git-svn and does not add any functionality useful for us.<br />
<br />
'''[https://subgit.com/ SubGit]''' is a java-based importer that promises to be faster than git-svn. It is not an open source product, but the ''import'' functionality can be used for [https://subgit.com/pricing.html free] (while the ''mirror'' functionality has to be purchased). However, from the small-scale tests we executed on few components there were no relevant improvements with respect to the git-svn times.<br />
<br />
= Integration with release tools =<br />
<br />
Git is a very popular software and it is already supported by all the '''development tools''' used by gCube developers (e.g. Eclipse, IntellijIDEA) on all platforms. Its usage is also very well documented and several tutorials and instructions on how to use Git can be found online.<br />
<br />
Support for Git has been also added and tested in '''ETICS'''. The commands to checkout the code from Git are quite straightforward. Supposing that the URL of the repository is kept in the <code>vcsroot</code> property and the branch name in <code>tag</code>, the checkout command to fetch the latest version of the code can be:<br />
<br />
<source lang="bash">git clone --depth 1 ${vcsroot} --branch ${tag} ${moduleName}</source><br />
while for checking out a specific commit:<br />
<br />
<source lang="bash">git clone ${vcsroot} --branch ${tag} ${moduleName} && cd ${moduleName} && git reset 104491f60c14f09124806272b88a605d5a324735 --hard</source><br />
However, ETICS partially misses the support for Git in the module for the automatic [https://wiki.gcube-system.org/gcube/ETICS#ETICS_ModelSync synchronization of the model]. This should be added to fully exploit with the automation that ETICS offers.<br />
<br />
The only other tool dependent form SVN is the distribution script used to create a tag on SVN with all the software for the new gCube releases. However, this tool will be abandoned because, once the code will be on Git, there will be no necessity to create a tag on SVN since this functionality will be fully replaced by the publication of the release on GitHub.<br />
<br />
= Updates to Release Procedure =<br />
<br />
gCube follows a strict [https://wiki.gcube-system.org/gcube/Major/Minor_Release_Cycle_procedure release procedure] that defines also how developers should use SVN to develop and release their software (e.g. creation and naming of release branches). This procedure will, of course, be updated to match the (slightly) different Git functionalities.<br />
<br />
Of particular interest in this context is to have a look at the different ''workflow'' that developers uses with Git. There are several of them documented: [https://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows Centralized workflow], [https://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows Integration-Manger workflow], [https://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows Dictator and Lieutenants Workflow], [https://www.atlassian.com/git/tutorials/comparing-workflows/feature-branch-workflow Feature Branch Workflow], [http://nvie.com/posts/a-successful-git-branching-model/ GitFlow], [https://guides.github.com/introduction/flow/ GitHub Flow] and [https://www.endpoint.com/blog/2014/05/02/git-workflows-that-work others].<br />
<br />
From a first analysis comparing these workflows with the one currently used (with SVN) by gCube, [http://nvie.com/posts/a-successful-git-branching-model/ GitFlow] is the closest one and therefore is the one that most probably would match the gCube requirements. <br />
<br />
The essential points of this workflow (in comparisons with our current workflow) are:<br />
<br />
* there is a development branch where the new developments are done, like in our current workflow;<br />
* a new branch is created for each new release, like in our current workflow;<br />
* integration issues are solved in the release branch, like in our current workflow;<br />
* the release branch is deleted after the release is closed, differently from our current workflow where the release branch remain and is used for patches and future releases;<br />
* when a patch is required, it is done in a new branch, differently from our current workflow where the patches are done in the release branch. However, in both cases, the patches are merged back in the development branch;<br />
* a tag is created after each release/patch before deleting the branch. This is equivalent (has the same objective) of the tags done at the end of each release in our current workflow.<br />
<br />
The diagram below describes the main steps of GitFlow.<br />
<br />
[[File:git-model@2x.png|400px|center]]<br />
<br />
We are not obliged to adopt GitFlow ''in toto'', we will probably build our own workflow, based on GitFlow with some minor modifications like:<br />
<br />
* release branches are not deleted: they are used to “generate” different component releases for each gCube release (e.g. 1.0.0-3.10.0, 1.0.0-3.11.0, …) and for patch releases;<br />
* before tagging a release a new commit with “resolved” pom.xml and distro files is done. This commit is not merged back to develop;<br />
* patches (hotfix) are branched from the release branch;<br />
* all components tags are merged into the gCube repository (this would realize the GitHub publication we do with ad-hoc scripts now).<br />
<br />
A better and more in depth analysis will be done on this before defining the final workflow that gCube will adopt.<br />
<br />
= Migration Strategy =<br />
<br />
The migration of the entire code base of gCube from SVN to Git is a very big and time consuming task given the size of gCube. Making this migration on a single shot for the entire gCube system, would stop the development of the system since all developers should be committed to migrate their (plus the ''orphan'') components. <br />
<br />
A more meaningful strategy is to articulate the migration in different phases to distribute the effort of migrating the component over a large (months) time period. A possible four-phases approach could be:<br />
<br />
# Initially, the migration will be done on &quot;volunteering&quot; basis. If a developer wants to migrate her component, she can do it following the new procedures and using the new tools. This ''early adopter'' group will be useful also to collect feedback on the tools and procedures before applying them on large-scale;<br />
# Then, developers will be forced to use Git for new component only, while for new releases of old components they can continue to use SVN;<br />
# Then, developers will be forced to use Git for any component they release, also old ones;<br />
# Finally, developers will be forced to migrate to Git any remaining component in the gCube release (even if there would be no need of releasing them).<br />
<br />
Of course, the migration can start only after all the tools and procedures are ready to manage Git-based components. In addition, all the old tools and procedures (SVN-based) will remain in place and co-exists with the new ones until all the gCube components have been migrated.</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=GCube_Git_Migration&diff=30746GCube Git Migration2018-01-15T18:59:31Z<p>Gabriele.giammatteo: /* Updates to Release Procedure */</p>
<hr />
<div>__TOC__<br />
<br />
During the BlueBRIDGE project, we started an activity for migrating the gCube source code from the Subversion (SVN) technology to Git. An analysis of the requirements and the design of a proper process and tools to governate the migration has been conducted. However, due to resources constraints, the actual migration has not been implemented.<br />
<br />
This document collects the output of all the work done and wants to be an helpful starting point for the future implementation of the migration of gCube source code from SVN to Git.<br />
<br />
= Requirements =<br />
<br />
The migration will affect the tools and the procedures currently used in the gCube development, integration and release. We started our analysis by considering some basic requirements that the final solution should address.<br />
<br />
'''Gradual implementation'''<br />
<br />
The migration should be designed to allow a gradual migration of gCube components allowing for continuing the normal development, integration and release activities even if some components have been already migrated and some others not yet. In this way, no major delays will be introduced in the gCube release scheduling. To address this requirement, the procedures and the tools used in the integration of gCube must support both SVN and Git repositories and allow to seamless integration between components stored in SVN and Git.<br />
<br />
'''History of files'''<br />
<br />
The migration should keep the history of the source code. This means that not only the latest version of each source file should be available in the new Git repository, but also all the single commits of each single source file with, possibly, the associated meta-data (i.e. date, author, commit message) should be preserved.<br />
<br />
'''Branching schema'''<br />
<br />
In the current SVN repository, each component has multiple branches: a) the development branch (i.e. ''trunk''), b) one release branch for each release and, optionally, c) one or more private branch (or feature branch). This allows developers to organize better their code and to maintain multiple versions of their component in parallel. The same functionality should be preserved when migrating to Git.<br />
<br />
'''GitHub publication'''<br />
<br />
Currently the source code of each gCube release (composed by the source code of all components that constitute the release) is replicated in a repository on GitHub. The current solution takes the source code to publish on GitHub from the source packages built by Maven during the builds and published on Nexus. This was the only viable solution to publish the source code from SVN, but it has the disadvantage that the history and the authors of the source files are not kept when copied on GitHub. After the migration, since the technology of gCube source code repositories and GitHub will be the same (Git), it should be possible also to publish on GitHub the authors and the history of each source file.<br />
<br />
'''Git repositories management software'''<br />
<br />
The software that will be used to manage the Git repositories should guarantee at least the following functionalities:<br />
<br />
* integration with the ''research infrastructures LDAP'': this will maintain the developers identity in the new Git repositories;<br />
* REST interface to create new repositories: this will increase the automation of the migration operation;<br />
* management of roles: the source code should be readable to everybody without authentication, but only authorized users should be able to make changes;<br />
* web interface for browsing the source code and the commits;<br />
<br />
Additionally, the tool should also implement collaboration functionalities like fork of repositories, pull requests, code review. This would greatly improve the collaboration between gCube developers and external community developers.<br />
<br />
= Git Repository Layout =<br />
<br />
Currently the gCube source code is maintained in a single SVN repository available at http://svn.research-infrastructures.eu/public/d4science/gcube/. The repository has the following layout:<br />
<br />
* ''trunk'' for the development version of components<br />
* ''branches'' for the release branches of components<br />
* ''private'' for feature branches of components<br />
* ''tags'' to keep a copy of gCube source code for each gCube release<br />
<br />
More information on the current SVN layout can be found in the [https://wiki.gcube-system.org/gcube/Gcube_subversion_repository gCube wiki].<br />
<br />
Switching to Git, the single repository layout will be abandoned in favor of a '''per-component repository layout'''. Each component will have its own Git repository that will contain all the branches for that component. This approach is not only more in-line with the Git philosophy, but, mostly important, more practical.<br />
<br />
In fact, while SVN allows for checkout and commit of single files/folders, Git (by design) does not provide this functionality. This mean that each operation (e.g. checkout of source code to execute a build) involves the entire repository. Given the size of gCube system (hundred of components and more than 1GB of source code), it is not practical to have all the source code in a single repository. In addition, Git allows to assign permissions only at repository level and having a per-component repository layout can be useful if we decide to assign permissions on component basis.<br />
<br />
The following table maps the current SVN layout with the new proposed Git layout also considering the workflow that we want to adopt (see [[#Updates to Release Procedure|Updates to Release Procedure]]).<br />
<br />
{|border="1" cellpadding="4" cellspacing="0"<br />
|- style="background-color:#F2F2F2"<br />
! type<br />
! SVN<br />
! Git<br />
|-<br />
| development branch<br />
| the developments are all located under the /trunk directory organized per component (e.g. <code>/trunk/common/common-client</code>)<br />
| the <code>development</code> branch<br />
|-<br />
| release branch<br />
| the releases are all located under the /branches directory organized per component and per version (e.g. <code>/branches/common/common-client/1.1</code>)<br />
| a branch named accordingly with the version (e.g. <code>1.1</code>)<br />
|-<br />
| feature branch<br />
| users place their feature branches in the /private directory organized per component (e.g. <code>/private/common/common-client/my-feature-1</code>)<br />
| a branch named accordingly with the feature (e.g. <code>my-feature-1</code>)<br />
|-<br />
| tags<br />
| tags are copies of the source code under the /tags directory organized per gCube release, component and version (e.g. <code>/tags/gcube-4.5.0/common/common-client/1.1</code>)<br />
| a tag of the most recent commit in the release branch is created with the full version of the release (e.g. <code>1.1.0-4.5.0</code>)<br />
|}<br />
<br />
= Import of source code from SVN =<br />
<br />
A strong requirement of the migration to Git is to maintain the history of the source code for each component. To achieve this, it is not sufficient to simply create an empty Git repository and copy the latest version of files from SVN.<br />
<br />
Since this is a very common requirement when migration from SVN to Git, there is a command line tool distributed directly with Git to address this problem. This was the base for our importing tests described below. There are also few alternatives reported at the end of the section.<br />
<br />
'''[https://git-scm.com/docs/git-svn git-svn]''' is a tool that given a SVN repository URL is able to discover all the files and all the commits in the repository and replicate them in a new Git repository. It is very complete and highly customizable. It supports different repository layouts, can distinguish between branches and tags and tries to follow the history of each file even if it has been copied or moved in different locations at certain point in the repository history. It is also able to keep user identities (by providing a mapping file of SVN and Git users) and can ignore specific directories and/or revisions. It is distributed under the open source GPLv2 license.<br />
<br />
The tests we executed highlighted that given the size (both in terms of number of files and number of revisions) of the gCube SVN repository the import can take several time (hours) even for a single component because the tool will try to scan the entire repository looking for branches and tags of that component. However, using some options, it is possible to limit the tool to only scan the folders where we know the source files are and only for the revisions we know the source files exist. This reduces drastically the importing times (from hours to minutes). However, it has to be considered that this depends highly from the &quot;age&quot; of the component. In fact old components, for which we need to scan more revisions, will be slower that new components (with fewer revisions to scan) and from the number of commits/releases existing.<br />
<br />
The following command imports the code of the home-library component (trunk and branches) from revision 82000 to the current one:<br />
<br />
<source lang="bash">> git svn clone http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/Common/<br />
home-library/ --trunk http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/Common/home-library/ --branches http://svn.research-infrastructures.eu/public/d4science/gcube/branches/common/home-library/ --authors-file=users.txt --no-metadata --no-minimize-url --revision 82000:HEAD<br />
</source><br />
This test took about 10 minutes (tried in July 2017).<br />
<br />
This command can be used (setting the correct parameters) for importing all gCube components. To facilitate the users, it can be wrapped in a more high-level script that is able to determine (or, at least, to guess) the values for almost all command parameters:<br />
<br />
* the min and max revisions can be found by running <code>svn log</code> and extract the revision of the first and the latest commits;<br />
* the branches and tags directory can be guessed from the trunk directory using the gCube naming conventions;<br />
* the authors-file can be generated upfront (see [https://john.albin.net/git/convert-subversion-to-git here]) and it is the stame for all the components.<br />
<br />
In addition the wrapper script can create and publish the new Git repository using the repository software APIs.<br />
<br />
== Alternative tools ==<br />
<br />
'''[https://github.com/JohnAlbin/git-svn-migrate git-svn-migrate]''' is just a wrapper script around git-svn and does not add any functionality useful for us.<br />
<br />
'''[https://subgit.com/ SubGit]''' is a java-based importer that promises to be faster than git-svn. It is not an open source product, but the ''import'' functionality can be used for [https://subgit.com/pricing.html free] (while the ''mirror'' functionality has to be purchased). However, from the small-scale tests we executed on few components there were no relevant improvements with respect to the git-svn times.<br />
<br />
= Integration with release tools =<br />
<br />
Git is a very popular software and it is already supported by all the '''development tools''' used by gCube developers (e.g. Eclipse, IntellijIDEA) on all platforms. Its usage is also very well documented and several tutorials and instructions on how to use Git can be found online.<br />
<br />
Support for Git has been also added and tested in '''ETICS'''. The commands to checkout the code from Git are quite straightforward. Supposing that the URL of the repository is kept in the <code>vcsroot</code> property and the branch name in <code>tag</code>, the checkout command to fetch the latest version of the code can be:<br />
<br />
<source lang="bash">git clone --depth 1 ${vcsroot} --branch ${tag} ${moduleName}</source><br />
while for checking out a specific commit:<br />
<br />
<source lang="bash">git clone ${vcsroot} --branch ${tag} ${moduleName} && cd ${moduleName} && git reset 104491f60c14f09124806272b88a605d5a324735 --hard</source><br />
However, ETICS partially misses the support for Git in the module for the automatic [https://wiki.gcube-system.org/gcube/ETICS#ETICS_ModelSync synchronization of the model]. This should be added to fully exploit with the automation that ETICS offers.<br />
<br />
The only other tool dependent form SVN is the distribution script used to create a tag on SVN with all the software for the new gCube releases. However, this tool will be abandoned because, once the code will be on Git, there will be no necessity to create a tag on SVN since this functionality will be fully replaced by the publication of the release on GitHub.<br />
<br />
= Updates to Release Procedure =<br />
<br />
gCube follows a strict [https://wiki.gcube-system.org/gcube/Major/Minor_Release_Cycle_procedure release procedure] that defines also how developers should use SVN to develop and release their software (e.g. creation and naming of release branches). This procedure will, of course, be updated to match the (slightly) different Git functionalities.<br />
<br />
Of particular interest in this context is to have a look at the different ''workflow'' that developers uses with Git. There are several of them documented: [https://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows Centralized workflow], [https://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows Integration-Manger workflow], [https://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows Dictator and Lieutenants Workflow], [https://www.atlassian.com/git/tutorials/comparing-workflows/feature-branch-workflow Feature Branch Workflow], [http://nvie.com/posts/a-successful-git-branching-model/ GitFlow], [https://guides.github.com/introduction/flow/ GitHub Flow] and [https://www.endpoint.com/blog/2014/05/02/git-workflows-that-work others].<br />
<br />
From a first analysis comparing these workflows with the one currently used (with SVN) by gCube, [http://nvie.com/posts/a-successful-git-branching-model/ GitFlow] is the closest one and therefore is the one that most probably would match the gCube requirements. <br />
<br />
The essential points of this workflow (in comparisons with our current workflow) are:<br />
<br />
* there is a development branch where the new developments are done, like in our current workflow;<br />
* a new branch is created for each new release, like in our current workflow;<br />
* integration issues are solved in the release branch, like in our current workflow;<br />
* the release branch is deleted after the release is closed, differently from our current workflow where the release branch remain and is used for patches and future releases;<br />
* when a patch is required, it is done in a new branch, differently from our current workflow where the patches are done in the release branch. However, in both cases, the patches are merged back in the development branch;<br />
* a tag is created after each release/patch before deleting the branch. This is equivalent (has the same objective) of the tags done at the end of each release in our current workflow.<br />
<br />
The diagram below describes the main steps of GitFlow.<br />
<br />
[[File:git-model@2x.png|frame|none|alt=|caption|800px ]]<br />
<br />
We are not obliged to adopt GitFlow ''in toto'', we will probably build our own workflow, based on GitFlow with some minor modifications like:<br />
<br />
* release branches are not deleted: they are used to “generate” different component releases for each gCube release (e.g. 1.0.0-3.10.0, 1.0.0-3.11.0, …) and for patch releases;<br />
* before tagging a release a new commit with “resolved” pom.xml and distro files is done. This commit is not merged back to develop;<br />
* patches (hotfix) are branched from the release branch;<br />
* all components tags are merged into the gCube repository (this would realize the GitHub publication we do with ad-hoc scripts now).<br />
<br />
A better and more in depth analysis will be done on this before defining the final workflow that gCube will adopt.<br />
<br />
= Migration Strategy =<br />
<br />
The migration of the entire code base of gCube from SVN to Git is a very big and time consuming task given the size of gCube. Making this migration on a single shot for the entire gCube system, would stop the development of the system since all developers should be committed to migrate their (plus the ''orphan'') components. <br />
<br />
A more meaningful strategy is to articulate the migration in different phases to distribute the effort of migrating the component over a large (months) time period. A possible four-phases approach could be:<br />
<br />
# Initially, the migration will be done on &quot;volunteering&quot; basis. If a developer wants to migrate her component, she can do it following the new procedures and using the new tools. This ''early adopter'' group will be useful also to collect feedback on the tools and procedures before applying them on large-scale;<br />
# Then, developers will be forced to use Git for new component only, while for new releases of old components they can continue to use SVN;<br />
# Then, developers will be forced to use Git for any component they release, also old ones;<br />
# Finally, developers will be forced to migrate to Git any remaining component in the gCube release (even if there would be no need of releasing them).<br />
<br />
Of course, the migration can start only after all the tools and procedures are ready to manage Git-based components. In addition, all the old tools and procedures (SVN-based) will remain in place and co-exists with the new ones until all the gCube components have been migrated.</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=GCube_Git_Migration&diff=30745GCube Git Migration2018-01-15T18:57:48Z<p>Gabriele.giammatteo: /* Updates to Release Procedure */</p>
<hr />
<div>__TOC__<br />
<br />
During the BlueBRIDGE project, we started an activity for migrating the gCube source code from the Subversion (SVN) technology to Git. An analysis of the requirements and the design of a proper process and tools to governate the migration has been conducted. However, due to resources constraints, the actual migration has not been implemented.<br />
<br />
This document collects the output of all the work done and wants to be an helpful starting point for the future implementation of the migration of gCube source code from SVN to Git.<br />
<br />
= Requirements =<br />
<br />
The migration will affect the tools and the procedures currently used in the gCube development, integration and release. We started our analysis by considering some basic requirements that the final solution should address.<br />
<br />
'''Gradual implementation'''<br />
<br />
The migration should be designed to allow a gradual migration of gCube components allowing for continuing the normal development, integration and release activities even if some components have been already migrated and some others not yet. In this way, no major delays will be introduced in the gCube release scheduling. To address this requirement, the procedures and the tools used in the integration of gCube must support both SVN and Git repositories and allow to seamless integration between components stored in SVN and Git.<br />
<br />
'''History of files'''<br />
<br />
The migration should keep the history of the source code. This means that not only the latest version of each source file should be available in the new Git repository, but also all the single commits of each single source file with, possibly, the associated meta-data (i.e. date, author, commit message) should be preserved.<br />
<br />
'''Branching schema'''<br />
<br />
In the current SVN repository, each component has multiple branches: a) the development branch (i.e. ''trunk''), b) one release branch for each release and, optionally, c) one or more private branch (or feature branch). This allows developers to organize better their code and to maintain multiple versions of their component in parallel. The same functionality should be preserved when migrating to Git.<br />
<br />
'''GitHub publication'''<br />
<br />
Currently the source code of each gCube release (composed by the source code of all components that constitute the release) is replicated in a repository on GitHub. The current solution takes the source code to publish on GitHub from the source packages built by Maven during the builds and published on Nexus. This was the only viable solution to publish the source code from SVN, but it has the disadvantage that the history and the authors of the source files are not kept when copied on GitHub. After the migration, since the technology of gCube source code repositories and GitHub will be the same (Git), it should be possible also to publish on GitHub the authors and the history of each source file.<br />
<br />
'''Git repositories management software'''<br />
<br />
The software that will be used to manage the Git repositories should guarantee at least the following functionalities:<br />
<br />
* integration with the ''research infrastructures LDAP'': this will maintain the developers identity in the new Git repositories;<br />
* REST interface to create new repositories: this will increase the automation of the migration operation;<br />
* management of roles: the source code should be readable to everybody without authentication, but only authorized users should be able to make changes;<br />
* web interface for browsing the source code and the commits;<br />
<br />
Additionally, the tool should also implement collaboration functionalities like fork of repositories, pull requests, code review. This would greatly improve the collaboration between gCube developers and external community developers.<br />
<br />
= Git Repository Layout =<br />
<br />
Currently the gCube source code is maintained in a single SVN repository available at http://svn.research-infrastructures.eu/public/d4science/gcube/. The repository has the following layout:<br />
<br />
* ''trunk'' for the development version of components<br />
* ''branches'' for the release branches of components<br />
* ''private'' for feature branches of components<br />
* ''tags'' to keep a copy of gCube source code for each gCube release<br />
<br />
More information on the current SVN layout can be found in the [https://wiki.gcube-system.org/gcube/Gcube_subversion_repository gCube wiki].<br />
<br />
Switching to Git, the single repository layout will be abandoned in favor of a '''per-component repository layout'''. Each component will have its own Git repository that will contain all the branches for that component. This approach is not only more in-line with the Git philosophy, but, mostly important, more practical.<br />
<br />
In fact, while SVN allows for checkout and commit of single files/folders, Git (by design) does not provide this functionality. This mean that each operation (e.g. checkout of source code to execute a build) involves the entire repository. Given the size of gCube system (hundred of components and more than 1GB of source code), it is not practical to have all the source code in a single repository. In addition, Git allows to assign permissions only at repository level and having a per-component repository layout can be useful if we decide to assign permissions on component basis.<br />
<br />
The following table maps the current SVN layout with the new proposed Git layout also considering the workflow that we want to adopt (see [[#Updates to Release Procedure|Updates to Release Procedure]]).<br />
<br />
{|border="1" cellpadding="4" cellspacing="0"<br />
|- style="background-color:#F2F2F2"<br />
! type<br />
! SVN<br />
! Git<br />
|-<br />
| development branch<br />
| the developments are all located under the /trunk directory organized per component (e.g. <code>/trunk/common/common-client</code>)<br />
| the <code>development</code> branch<br />
|-<br />
| release branch<br />
| the releases are all located under the /branches directory organized per component and per version (e.g. <code>/branches/common/common-client/1.1</code>)<br />
| a branch named accordingly with the version (e.g. <code>1.1</code>)<br />
|-<br />
| feature branch<br />
| users place their feature branches in the /private directory organized per component (e.g. <code>/private/common/common-client/my-feature-1</code>)<br />
| a branch named accordingly with the feature (e.g. <code>my-feature-1</code>)<br />
|-<br />
| tags<br />
| tags are copies of the source code under the /tags directory organized per gCube release, component and version (e.g. <code>/tags/gcube-4.5.0/common/common-client/1.1</code>)<br />
| a tag of the most recent commit in the release branch is created with the full version of the release (e.g. <code>1.1.0-4.5.0</code>)<br />
|}<br />
<br />
= Import of source code from SVN =<br />
<br />
A strong requirement of the migration to Git is to maintain the history of the source code for each component. To achieve this, it is not sufficient to simply create an empty Git repository and copy the latest version of files from SVN.<br />
<br />
Since this is a very common requirement when migration from SVN to Git, there is a command line tool distributed directly with Git to address this problem. This was the base for our importing tests described below. There are also few alternatives reported at the end of the section.<br />
<br />
'''[https://git-scm.com/docs/git-svn git-svn]''' is a tool that given a SVN repository URL is able to discover all the files and all the commits in the repository and replicate them in a new Git repository. It is very complete and highly customizable. It supports different repository layouts, can distinguish between branches and tags and tries to follow the history of each file even if it has been copied or moved in different locations at certain point in the repository history. It is also able to keep user identities (by providing a mapping file of SVN and Git users) and can ignore specific directories and/or revisions. It is distributed under the open source GPLv2 license.<br />
<br />
The tests we executed highlighted that given the size (both in terms of number of files and number of revisions) of the gCube SVN repository the import can take several time (hours) even for a single component because the tool will try to scan the entire repository looking for branches and tags of that component. However, using some options, it is possible to limit the tool to only scan the folders where we know the source files are and only for the revisions we know the source files exist. This reduces drastically the importing times (from hours to minutes). However, it has to be considered that this depends highly from the &quot;age&quot; of the component. In fact old components, for which we need to scan more revisions, will be slower that new components (with fewer revisions to scan) and from the number of commits/releases existing.<br />
<br />
The following command imports the code of the home-library component (trunk and branches) from revision 82000 to the current one:<br />
<br />
<source lang="bash">> git svn clone http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/Common/<br />
home-library/ --trunk http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/Common/home-library/ --branches http://svn.research-infrastructures.eu/public/d4science/gcube/branches/common/home-library/ --authors-file=users.txt --no-metadata --no-minimize-url --revision 82000:HEAD<br />
</source><br />
This test took about 10 minutes (tried in July 2017).<br />
<br />
This command can be used (setting the correct parameters) for importing all gCube components. To facilitate the users, it can be wrapped in a more high-level script that is able to determine (or, at least, to guess) the values for almost all command parameters:<br />
<br />
* the min and max revisions can be found by running <code>svn log</code> and extract the revision of the first and the latest commits;<br />
* the branches and tags directory can be guessed from the trunk directory using the gCube naming conventions;<br />
* the authors-file can be generated upfront (see [https://john.albin.net/git/convert-subversion-to-git here]) and it is the stame for all the components.<br />
<br />
In addition the wrapper script can create and publish the new Git repository using the repository software APIs.<br />
<br />
== Alternative tools ==<br />
<br />
'''[https://github.com/JohnAlbin/git-svn-migrate git-svn-migrate]''' is just a wrapper script around git-svn and does not add any functionality useful for us.<br />
<br />
'''[https://subgit.com/ SubGit]''' is a java-based importer that promises to be faster than git-svn. It is not an open source product, but the ''import'' functionality can be used for [https://subgit.com/pricing.html free] (while the ''mirror'' functionality has to be purchased). However, from the small-scale tests we executed on few components there were no relevant improvements with respect to the git-svn times.<br />
<br />
= Integration with release tools =<br />
<br />
Git is a very popular software and it is already supported by all the '''development tools''' used by gCube developers (e.g. Eclipse, IntellijIDEA) on all platforms. Its usage is also very well documented and several tutorials and instructions on how to use Git can be found online.<br />
<br />
Support for Git has been also added and tested in '''ETICS'''. The commands to checkout the code from Git are quite straightforward. Supposing that the URL of the repository is kept in the <code>vcsroot</code> property and the branch name in <code>tag</code>, the checkout command to fetch the latest version of the code can be:<br />
<br />
<source lang="bash">git clone --depth 1 ${vcsroot} --branch ${tag} ${moduleName}</source><br />
while for checking out a specific commit:<br />
<br />
<source lang="bash">git clone ${vcsroot} --branch ${tag} ${moduleName} && cd ${moduleName} && git reset 104491f60c14f09124806272b88a605d5a324735 --hard</source><br />
However, ETICS partially misses the support for Git in the module for the automatic [https://wiki.gcube-system.org/gcube/ETICS#ETICS_ModelSync synchronization of the model]. This should be added to fully exploit with the automation that ETICS offers.<br />
<br />
The only other tool dependent form SVN is the distribution script used to create a tag on SVN with all the software for the new gCube releases. However, this tool will be abandoned because, once the code will be on Git, there will be no necessity to create a tag on SVN since this functionality will be fully replaced by the publication of the release on GitHub.<br />
<br />
= Updates to Release Procedure =<br />
<br />
gCube follows a strict [https://wiki.gcube-system.org/gcube/Major/Minor_Release_Cycle_procedure release procedure] that defines also how developers should use SVN to develop and release their software (e.g. creation and naming of release branches). This procedure will, of course, be updated to match the (slightly) different Git functionalities.<br />
<br />
Of particular interest in this context is to have a look at the different ''workflow'' that developers uses with Git. There are several of them documented: [https://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows Centralized workflow], [https://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows Integration-Manger workflow], [https://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows Dictator and Lieutenants Workflow], [https://www.atlassian.com/git/tutorials/comparing-workflows/feature-branch-workflow Feature Branch Workflow], [http://nvie.com/posts/a-successful-git-branching-model/ GitFlow], [https://guides.github.com/introduction/flow/ GitHub Flow] and [https://www.endpoint.com/blog/2014/05/02/git-workflows-that-work others].<br />
<br />
From a first analysis comparing these workflows with the one currently used (with SVN) by gCube, [http://nvie.com/posts/a-successful-git-branching-model/ GitFlow] is the closest one and therefore is the one that most probably would match the gCube requirements. <br />
<br />
The essential points of this workflow (in comparisons with our current workflow) are:<br />
<br />
* there is a development branch where the new developments are done, like in our current workflow;<br />
* a new branch is created for each new release, like in our current workflow;<br />
* integration issues are solved in the release branch, like in our current workflow;<br />
* the release branch is deleted after the release is closed, differently from our current workflow where the release branch remain and is used for patches and future releases;<br />
* when a patch is required, it is done in a new branch, differently from our current workflow where the patches are done in the release branch. However, in both cases, the patches are merged back in the development branch;<br />
* a tag is created after each release/patch before deleting the branch. This is equivalent (has the same objective) of the tags done at the end of each release in our current workflow.<br />
<br />
The diagram below describes the main steps of GitFlow.<br />
<br />
[[File:git-model@2x.png|frame|none|alt=|caption|width:800px ]]<br />
<br />
We are not obliged to adopt GitFlow ''in toto'', we will probably build our own workflow, based on GitFlow with some minor modifications like:<br />
<br />
* release branches are not deleted: they are used to “generate” different component releases for each gCube release (e.g. 1.0.0-3.10.0, 1.0.0-3.11.0, …) and for patch releases;<br />
* before tagging a release a new commit with “resolved” pom.xml and distro files is done. This commit is not merged back to develop;<br />
* patches (hotfix) are branched from the release branch;<br />
* all components tags are merged into the gCube repository (this would realize the GitHub publication we do with ad-hoc scripts now).<br />
<br />
A better and more in depth analysis will be done on this before defining the final workflow that gCube will adopt.<br />
<br />
= Migration Strategy =<br />
<br />
The migration of the entire code base of gCube from SVN to Git is a very big and time consuming task given the size of gCube. Making this migration on a single shot for the entire gCube system, would stop the development of the system since all developers should be committed to migrate their (plus the ''orphan'') components. <br />
<br />
A more meaningful strategy is to articulate the migration in different phases to distribute the effort of migrating the component over a large (months) time period. A possible four-phases approach could be:<br />
<br />
# Initially, the migration will be done on &quot;volunteering&quot; basis. If a developer wants to migrate her component, she can do it following the new procedures and using the new tools. This ''early adopter'' group will be useful also to collect feedback on the tools and procedures before applying them on large-scale;<br />
# Then, developers will be forced to use Git for new component only, while for new releases of old components they can continue to use SVN;<br />
# Then, developers will be forced to use Git for any component they release, also old ones;<br />
# Finally, developers will be forced to migrate to Git any remaining component in the gCube release (even if there would be no need of releasing them).<br />
<br />
Of course, the migration can start only after all the tools and procedures are ready to manage Git-based components. In addition, all the old tools and procedures (SVN-based) will remain in place and co-exists with the new ones until all the gCube components have been migrated.</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=File:Git-model@2x.png&diff=30744File:Git-model@2x.png2018-01-15T18:56:09Z<p>Gabriele.giammatteo: </p>
<hr />
<div></div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=GCube_Git_Migration&diff=30743GCube Git Migration2018-01-15T18:55:45Z<p>Gabriele.giammatteo: /* Updates to Release Procedure */</p>
<hr />
<div>__TOC__<br />
<br />
During the BlueBRIDGE project, we started an activity for migrating the gCube source code from the Subversion (SVN) technology to Git. An analysis of the requirements and the design of a proper process and tools to governate the migration has been conducted. However, due to resources constraints, the actual migration has not been implemented.<br />
<br />
This document collects the output of all the work done and wants to be an helpful starting point for the future implementation of the migration of gCube source code from SVN to Git.<br />
<br />
= Requirements =<br />
<br />
The migration will affect the tools and the procedures currently used in the gCube development, integration and release. We started our analysis by considering some basic requirements that the final solution should address.<br />
<br />
'''Gradual implementation'''<br />
<br />
The migration should be designed to allow a gradual migration of gCube components allowing for continuing the normal development, integration and release activities even if some components have been already migrated and some others not yet. In this way, no major delays will be introduced in the gCube release scheduling. To address this requirement, the procedures and the tools used in the integration of gCube must support both SVN and Git repositories and allow to seamless integration between components stored in SVN and Git.<br />
<br />
'''History of files'''<br />
<br />
The migration should keep the history of the source code. This means that not only the latest version of each source file should be available in the new Git repository, but also all the single commits of each single source file with, possibly, the associated meta-data (i.e. date, author, commit message) should be preserved.<br />
<br />
'''Branching schema'''<br />
<br />
In the current SVN repository, each component has multiple branches: a) the development branch (i.e. ''trunk''), b) one release branch for each release and, optionally, c) one or more private branch (or feature branch). This allows developers to organize better their code and to maintain multiple versions of their component in parallel. The same functionality should be preserved when migrating to Git.<br />
<br />
'''GitHub publication'''<br />
<br />
Currently the source code of each gCube release (composed by the source code of all components that constitute the release) is replicated in a repository on GitHub. The current solution takes the source code to publish on GitHub from the source packages built by Maven during the builds and published on Nexus. This was the only viable solution to publish the source code from SVN, but it has the disadvantage that the history and the authors of the source files are not kept when copied on GitHub. After the migration, since the technology of gCube source code repositories and GitHub will be the same (Git), it should be possible also to publish on GitHub the authors and the history of each source file.<br />
<br />
'''Git repositories management software'''<br />
<br />
The software that will be used to manage the Git repositories should guarantee at least the following functionalities:<br />
<br />
* integration with the ''research infrastructures LDAP'': this will maintain the developers identity in the new Git repositories;<br />
* REST interface to create new repositories: this will increase the automation of the migration operation;<br />
* management of roles: the source code should be readable to everybody without authentication, but only authorized users should be able to make changes;<br />
* web interface for browsing the source code and the commits;<br />
<br />
Additionally, the tool should also implement collaboration functionalities like fork of repositories, pull requests, code review. This would greatly improve the collaboration between gCube developers and external community developers.<br />
<br />
= Git Repository Layout =<br />
<br />
Currently the gCube source code is maintained in a single SVN repository available at http://svn.research-infrastructures.eu/public/d4science/gcube/. The repository has the following layout:<br />
<br />
* ''trunk'' for the development version of components<br />
* ''branches'' for the release branches of components<br />
* ''private'' for feature branches of components<br />
* ''tags'' to keep a copy of gCube source code for each gCube release<br />
<br />
More information on the current SVN layout can be found in the [https://wiki.gcube-system.org/gcube/Gcube_subversion_repository gCube wiki].<br />
<br />
Switching to Git, the single repository layout will be abandoned in favor of a '''per-component repository layout'''. Each component will have its own Git repository that will contain all the branches for that component. This approach is not only more in-line with the Git philosophy, but, mostly important, more practical.<br />
<br />
In fact, while SVN allows for checkout and commit of single files/folders, Git (by design) does not provide this functionality. This mean that each operation (e.g. checkout of source code to execute a build) involves the entire repository. Given the size of gCube system (hundred of components and more than 1GB of source code), it is not practical to have all the source code in a single repository. In addition, Git allows to assign permissions only at repository level and having a per-component repository layout can be useful if we decide to assign permissions on component basis.<br />
<br />
The following table maps the current SVN layout with the new proposed Git layout also considering the workflow that we want to adopt (see [[#Updates to Release Procedure|Updates to Release Procedure]]).<br />
<br />
{|border="1" cellpadding="4" cellspacing="0"<br />
|- style="background-color:#F2F2F2"<br />
! type<br />
! SVN<br />
! Git<br />
|-<br />
| development branch<br />
| the developments are all located under the /trunk directory organized per component (e.g. <code>/trunk/common/common-client</code>)<br />
| the <code>development</code> branch<br />
|-<br />
| release branch<br />
| the releases are all located under the /branches directory organized per component and per version (e.g. <code>/branches/common/common-client/1.1</code>)<br />
| a branch named accordingly with the version (e.g. <code>1.1</code>)<br />
|-<br />
| feature branch<br />
| users place their feature branches in the /private directory organized per component (e.g. <code>/private/common/common-client/my-feature-1</code>)<br />
| a branch named accordingly with the feature (e.g. <code>my-feature-1</code>)<br />
|-<br />
| tags<br />
| tags are copies of the source code under the /tags directory organized per gCube release, component and version (e.g. <code>/tags/gcube-4.5.0/common/common-client/1.1</code>)<br />
| a tag of the most recent commit in the release branch is created with the full version of the release (e.g. <code>1.1.0-4.5.0</code>)<br />
|}<br />
<br />
= Import of source code from SVN =<br />
<br />
A strong requirement of the migration to Git is to maintain the history of the source code for each component. To achieve this, it is not sufficient to simply create an empty Git repository and copy the latest version of files from SVN.<br />
<br />
Since this is a very common requirement when migration from SVN to Git, there is a command line tool distributed directly with Git to address this problem. This was the base for our importing tests described below. There are also few alternatives reported at the end of the section.<br />
<br />
'''[https://git-scm.com/docs/git-svn git-svn]''' is a tool that given a SVN repository URL is able to discover all the files and all the commits in the repository and replicate them in a new Git repository. It is very complete and highly customizable. It supports different repository layouts, can distinguish between branches and tags and tries to follow the history of each file even if it has been copied or moved in different locations at certain point in the repository history. It is also able to keep user identities (by providing a mapping file of SVN and Git users) and can ignore specific directories and/or revisions. It is distributed under the open source GPLv2 license.<br />
<br />
The tests we executed highlighted that given the size (both in terms of number of files and number of revisions) of the gCube SVN repository the import can take several time (hours) even for a single component because the tool will try to scan the entire repository looking for branches and tags of that component. However, using some options, it is possible to limit the tool to only scan the folders where we know the source files are and only for the revisions we know the source files exist. This reduces drastically the importing times (from hours to minutes). However, it has to be considered that this depends highly from the &quot;age&quot; of the component. In fact old components, for which we need to scan more revisions, will be slower that new components (with fewer revisions to scan) and from the number of commits/releases existing.<br />
<br />
The following command imports the code of the home-library component (trunk and branches) from revision 82000 to the current one:<br />
<br />
<source lang="bash">> git svn clone http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/Common/<br />
home-library/ --trunk http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/Common/home-library/ --branches http://svn.research-infrastructures.eu/public/d4science/gcube/branches/common/home-library/ --authors-file=users.txt --no-metadata --no-minimize-url --revision 82000:HEAD<br />
</source><br />
This test took about 10 minutes (tried in July 2017).<br />
<br />
This command can be used (setting the correct parameters) for importing all gCube components. To facilitate the users, it can be wrapped in a more high-level script that is able to determine (or, at least, to guess) the values for almost all command parameters:<br />
<br />
* the min and max revisions can be found by running <code>svn log</code> and extract the revision of the first and the latest commits;<br />
* the branches and tags directory can be guessed from the trunk directory using the gCube naming conventions;<br />
* the authors-file can be generated upfront (see [https://john.albin.net/git/convert-subversion-to-git here]) and it is the stame for all the components.<br />
<br />
In addition the wrapper script can create and publish the new Git repository using the repository software APIs.<br />
<br />
== Alternative tools ==<br />
<br />
'''[https://github.com/JohnAlbin/git-svn-migrate git-svn-migrate]''' is just a wrapper script around git-svn and does not add any functionality useful for us.<br />
<br />
'''[https://subgit.com/ SubGit]''' is a java-based importer that promises to be faster than git-svn. It is not an open source product, but the ''import'' functionality can be used for [https://subgit.com/pricing.html free] (while the ''mirror'' functionality has to be purchased). However, from the small-scale tests we executed on few components there were no relevant improvements with respect to the git-svn times.<br />
<br />
= Integration with release tools =<br />
<br />
Git is a very popular software and it is already supported by all the '''development tools''' used by gCube developers (e.g. Eclipse, IntellijIDEA) on all platforms. Its usage is also very well documented and several tutorials and instructions on how to use Git can be found online.<br />
<br />
Support for Git has been also added and tested in '''ETICS'''. The commands to checkout the code from Git are quite straightforward. Supposing that the URL of the repository is kept in the <code>vcsroot</code> property and the branch name in <code>tag</code>, the checkout command to fetch the latest version of the code can be:<br />
<br />
<source lang="bash">git clone --depth 1 ${vcsroot} --branch ${tag} ${moduleName}</source><br />
while for checking out a specific commit:<br />
<br />
<source lang="bash">git clone ${vcsroot} --branch ${tag} ${moduleName} && cd ${moduleName} && git reset 104491f60c14f09124806272b88a605d5a324735 --hard</source><br />
However, ETICS partially misses the support for Git in the module for the automatic [https://wiki.gcube-system.org/gcube/ETICS#ETICS_ModelSync synchronization of the model]. This should be added to fully exploit with the automation that ETICS offers.<br />
<br />
The only other tool dependent form SVN is the distribution script used to create a tag on SVN with all the software for the new gCube releases. However, this tool will be abandoned because, once the code will be on Git, there will be no necessity to create a tag on SVN since this functionality will be fully replaced by the publication of the release on GitHub.<br />
<br />
= Updates to Release Procedure =<br />
<br />
gCube follows a strict [https://wiki.gcube-system.org/gcube/Major/Minor_Release_Cycle_procedure release procedure] that defines also how developers should use SVN to develop and release their software (e.g. creation and naming of release branches). This procedure will, of course, be updated to match the (slightly) different Git functionalities.<br />
<br />
Of particular interest in this context is to have a look at the different ''workflow'' that developers uses with Git. There are several of them documented: [https://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows Centralized workflow], [https://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows Integration-Manger workflow], [https://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows Dictator and Lieutenants Workflow], [https://www.atlassian.com/git/tutorials/comparing-workflows/feature-branch-workflow Feature Branch Workflow], [http://nvie.com/posts/a-successful-git-branching-model/ GitFlow], [https://guides.github.com/introduction/flow/ GitHub Flow] and [https://www.endpoint.com/blog/2014/05/02/git-workflows-that-work others].<br />
<br />
From a first analysis comparing these workflows with the one currently used (with SVN) by gCube, [http://nvie.com/posts/a-successful-git-branching-model/ GitFlow] is the closest one and therefore is the one that most probably would match the gCube requirements. <br />
<br />
The essential points of this workflow (in comparisons with our current workflow) are:<br />
<br />
* there is a development branch where the new developments are done, like in our current workflow;<br />
* a new branch is created for each new release, like in our current workflow;<br />
* integration issues are solved in the release branch, like in our current workflow;<br />
* the release branch is deleted after the release is closed, differently from our current workflow where the release branch remain and is used for patches and future releases;<br />
* when a patch is required, it is done in a new branch, differently from our current workflow where the patches are done in the release branch. However, in both cases, the patches are merged back in the development branch;<br />
* a tag is created after each release/patch before deleting the branch. This is equivalent (has the same objective) of the tags done at the end of each release in our current workflow.<br />
<br />
The diagram below describes the main steps of GitFlow.<br />
<br />
[[File:git-model@2x.png|frame|none|alt=|caption ]]<br />
<br />
We are not obliged to adopt GitFlow ''in toto'', we will probably build our own workflow, based on GitFlow with some minor modifications like:<br />
<br />
* release branches are not deleted: they are used to “generate” different component releases for each gCube release (e.g. 1.0.0-3.10.0, 1.0.0-3.11.0, …) and for patch releases;<br />
* before tagging a release a new commit with “resolved” pom.xml and distro files is done. This commit is not merged back to develop;<br />
* patches (hotfix) are branched from the release branch;<br />
* all components tags are merged into the gCube repository (this would realize the GitHub publication we do with ad-hoc scripts now).<br />
<br />
A better and more in depth analysis will be done on this before defining the final workflow that gCube will adopt.<br />
<br />
= Migration Strategy =<br />
<br />
The migration of the entire code base of gCube from SVN to Git is a very big and time consuming task given the size of gCube. Making this migration on a single shot for the entire gCube system, would stop the development of the system since all developers should be committed to migrate their (plus the ''orphan'') components. <br />
<br />
A more meaningful strategy is to articulate the migration in different phases to distribute the effort of migrating the component over a large (months) time period. A possible four-phases approach could be:<br />
<br />
# Initially, the migration will be done on &quot;volunteering&quot; basis. If a developer wants to migrate her component, she can do it following the new procedures and using the new tools. This ''early adopter'' group will be useful also to collect feedback on the tools and procedures before applying them on large-scale;<br />
# Then, developers will be forced to use Git for new component only, while for new releases of old components they can continue to use SVN;<br />
# Then, developers will be forced to use Git for any component they release, also old ones;<br />
# Finally, developers will be forced to migrate to Git any remaining component in the gCube release (even if there would be no need of releasing them).<br />
<br />
Of course, the migration can start only after all the tools and procedures are ready to manage Git-based components. In addition, all the old tools and procedures (SVN-based) will remain in place and co-exists with the new ones until all the gCube components have been migrated.</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=GCube_Git_Migration&diff=30742GCube Git Migration2018-01-15T18:54:07Z<p>Gabriele.giammatteo: /* Integration with release tools */</p>
<hr />
<div>__TOC__<br />
<br />
During the BlueBRIDGE project, we started an activity for migrating the gCube source code from the Subversion (SVN) technology to Git. An analysis of the requirements and the design of a proper process and tools to governate the migration has been conducted. However, due to resources constraints, the actual migration has not been implemented.<br />
<br />
This document collects the output of all the work done and wants to be an helpful starting point for the future implementation of the migration of gCube source code from SVN to Git.<br />
<br />
= Requirements =<br />
<br />
The migration will affect the tools and the procedures currently used in the gCube development, integration and release. We started our analysis by considering some basic requirements that the final solution should address.<br />
<br />
'''Gradual implementation'''<br />
<br />
The migration should be designed to allow a gradual migration of gCube components allowing for continuing the normal development, integration and release activities even if some components have been already migrated and some others not yet. In this way, no major delays will be introduced in the gCube release scheduling. To address this requirement, the procedures and the tools used in the integration of gCube must support both SVN and Git repositories and allow to seamless integration between components stored in SVN and Git.<br />
<br />
'''History of files'''<br />
<br />
The migration should keep the history of the source code. This means that not only the latest version of each source file should be available in the new Git repository, but also all the single commits of each single source file with, possibly, the associated meta-data (i.e. date, author, commit message) should be preserved.<br />
<br />
'''Branching schema'''<br />
<br />
In the current SVN repository, each component has multiple branches: a) the development branch (i.e. ''trunk''), b) one release branch for each release and, optionally, c) one or more private branch (or feature branch). This allows developers to organize better their code and to maintain multiple versions of their component in parallel. The same functionality should be preserved when migrating to Git.<br />
<br />
'''GitHub publication'''<br />
<br />
Currently the source code of each gCube release (composed by the source code of all components that constitute the release) is replicated in a repository on GitHub. The current solution takes the source code to publish on GitHub from the source packages built by Maven during the builds and published on Nexus. This was the only viable solution to publish the source code from SVN, but it has the disadvantage that the history and the authors of the source files are not kept when copied on GitHub. After the migration, since the technology of gCube source code repositories and GitHub will be the same (Git), it should be possible also to publish on GitHub the authors and the history of each source file.<br />
<br />
'''Git repositories management software'''<br />
<br />
The software that will be used to manage the Git repositories should guarantee at least the following functionalities:<br />
<br />
* integration with the ''research infrastructures LDAP'': this will maintain the developers identity in the new Git repositories;<br />
* REST interface to create new repositories: this will increase the automation of the migration operation;<br />
* management of roles: the source code should be readable to everybody without authentication, but only authorized users should be able to make changes;<br />
* web interface for browsing the source code and the commits;<br />
<br />
Additionally, the tool should also implement collaboration functionalities like fork of repositories, pull requests, code review. This would greatly improve the collaboration between gCube developers and external community developers.<br />
<br />
= Git Repository Layout =<br />
<br />
Currently the gCube source code is maintained in a single SVN repository available at http://svn.research-infrastructures.eu/public/d4science/gcube/. The repository has the following layout:<br />
<br />
* ''trunk'' for the development version of components<br />
* ''branches'' for the release branches of components<br />
* ''private'' for feature branches of components<br />
* ''tags'' to keep a copy of gCube source code for each gCube release<br />
<br />
More information on the current SVN layout can be found in the [https://wiki.gcube-system.org/gcube/Gcube_subversion_repository gCube wiki].<br />
<br />
Switching to Git, the single repository layout will be abandoned in favor of a '''per-component repository layout'''. Each component will have its own Git repository that will contain all the branches for that component. This approach is not only more in-line with the Git philosophy, but, mostly important, more practical.<br />
<br />
In fact, while SVN allows for checkout and commit of single files/folders, Git (by design) does not provide this functionality. This mean that each operation (e.g. checkout of source code to execute a build) involves the entire repository. Given the size of gCube system (hundred of components and more than 1GB of source code), it is not practical to have all the source code in a single repository. In addition, Git allows to assign permissions only at repository level and having a per-component repository layout can be useful if we decide to assign permissions on component basis.<br />
<br />
The following table maps the current SVN layout with the new proposed Git layout also considering the workflow that we want to adopt (see [[#Updates to Release Procedure|Updates to Release Procedure]]).<br />
<br />
{|border="1" cellpadding="4" cellspacing="0"<br />
|- style="background-color:#F2F2F2"<br />
! type<br />
! SVN<br />
! Git<br />
|-<br />
| development branch<br />
| the developments are all located under the /trunk directory organized per component (e.g. <code>/trunk/common/common-client</code>)<br />
| the <code>development</code> branch<br />
|-<br />
| release branch<br />
| the releases are all located under the /branches directory organized per component and per version (e.g. <code>/branches/common/common-client/1.1</code>)<br />
| a branch named accordingly with the version (e.g. <code>1.1</code>)<br />
|-<br />
| feature branch<br />
| users place their feature branches in the /private directory organized per component (e.g. <code>/private/common/common-client/my-feature-1</code>)<br />
| a branch named accordingly with the feature (e.g. <code>my-feature-1</code>)<br />
|-<br />
| tags<br />
| tags are copies of the source code under the /tags directory organized per gCube release, component and version (e.g. <code>/tags/gcube-4.5.0/common/common-client/1.1</code>)<br />
| a tag of the most recent commit in the release branch is created with the full version of the release (e.g. <code>1.1.0-4.5.0</code>)<br />
|}<br />
<br />
= Import of source code from SVN =<br />
<br />
A strong requirement of the migration to Git is to maintain the history of the source code for each component. To achieve this, it is not sufficient to simply create an empty Git repository and copy the latest version of files from SVN.<br />
<br />
Since this is a very common requirement when migration from SVN to Git, there is a command line tool distributed directly with Git to address this problem. This was the base for our importing tests described below. There are also few alternatives reported at the end of the section.<br />
<br />
'''[https://git-scm.com/docs/git-svn git-svn]''' is a tool that given a SVN repository URL is able to discover all the files and all the commits in the repository and replicate them in a new Git repository. It is very complete and highly customizable. It supports different repository layouts, can distinguish between branches and tags and tries to follow the history of each file even if it has been copied or moved in different locations at certain point in the repository history. It is also able to keep user identities (by providing a mapping file of SVN and Git users) and can ignore specific directories and/or revisions. It is distributed under the open source GPLv2 license.<br />
<br />
The tests we executed highlighted that given the size (both in terms of number of files and number of revisions) of the gCube SVN repository the import can take several time (hours) even for a single component because the tool will try to scan the entire repository looking for branches and tags of that component. However, using some options, it is possible to limit the tool to only scan the folders where we know the source files are and only for the revisions we know the source files exist. This reduces drastically the importing times (from hours to minutes). However, it has to be considered that this depends highly from the &quot;age&quot; of the component. In fact old components, for which we need to scan more revisions, will be slower that new components (with fewer revisions to scan) and from the number of commits/releases existing.<br />
<br />
The following command imports the code of the home-library component (trunk and branches) from revision 82000 to the current one:<br />
<br />
<source lang="bash">> git svn clone http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/Common/<br />
home-library/ --trunk http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/Common/home-library/ --branches http://svn.research-infrastructures.eu/public/d4science/gcube/branches/common/home-library/ --authors-file=users.txt --no-metadata --no-minimize-url --revision 82000:HEAD<br />
</source><br />
This test took about 10 minutes (tried in July 2017).<br />
<br />
This command can be used (setting the correct parameters) for importing all gCube components. To facilitate the users, it can be wrapped in a more high-level script that is able to determine (or, at least, to guess) the values for almost all command parameters:<br />
<br />
* the min and max revisions can be found by running <code>svn log</code> and extract the revision of the first and the latest commits;<br />
* the branches and tags directory can be guessed from the trunk directory using the gCube naming conventions;<br />
* the authors-file can be generated upfront (see [https://john.albin.net/git/convert-subversion-to-git here]) and it is the stame for all the components.<br />
<br />
In addition the wrapper script can create and publish the new Git repository using the repository software APIs.<br />
<br />
== Alternative tools ==<br />
<br />
'''[https://github.com/JohnAlbin/git-svn-migrate git-svn-migrate]''' is just a wrapper script around git-svn and does not add any functionality useful for us.<br />
<br />
'''[https://subgit.com/ SubGit]''' is a java-based importer that promises to be faster than git-svn. It is not an open source product, but the ''import'' functionality can be used for [https://subgit.com/pricing.html free] (while the ''mirror'' functionality has to be purchased). However, from the small-scale tests we executed on few components there were no relevant improvements with respect to the git-svn times.<br />
<br />
= Integration with release tools =<br />
<br />
Git is a very popular software and it is already supported by all the '''development tools''' used by gCube developers (e.g. Eclipse, IntellijIDEA) on all platforms. Its usage is also very well documented and several tutorials and instructions on how to use Git can be found online.<br />
<br />
Support for Git has been also added and tested in '''ETICS'''. The commands to checkout the code from Git are quite straightforward. Supposing that the URL of the repository is kept in the <code>vcsroot</code> property and the branch name in <code>tag</code>, the checkout command to fetch the latest version of the code can be:<br />
<br />
<source lang="bash">git clone --depth 1 ${vcsroot} --branch ${tag} ${moduleName}</source><br />
while for checking out a specific commit:<br />
<br />
<source lang="bash">git clone ${vcsroot} --branch ${tag} ${moduleName} && cd ${moduleName} && git reset 104491f60c14f09124806272b88a605d5a324735 --hard</source><br />
However, ETICS partially misses the support for Git in the module for the automatic [https://wiki.gcube-system.org/gcube/ETICS#ETICS_ModelSync synchronization of the model]. This should be added to fully exploit with the automation that ETICS offers.<br />
<br />
The only other tool dependent form SVN is the distribution script used to create a tag on SVN with all the software for the new gCube releases. However, this tool will be abandoned because, once the code will be on Git, there will be no necessity to create a tag on SVN since this functionality will be fully replaced by the publication of the release on GitHub.<br />
<br />
= Updates to Release Procedure =<br />
<br />
gCube follows a strict [https://wiki.gcube-system.org/gcube/Major/Minor_Release_Cycle_procedure release procedure] that defines also how developers should use SVN to develop and release their software (e.g. creation and naming of release branches). This procedure will, of course, be updated to match the (slightly) different Git functionalities.<br />
<br />
Of particular interest in this context is to have a look at the different ''workflow'' that developers uses with Git. There are several of them documented: [https://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows Centralized workflow], [https://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows Integration-Manger workflow], [https://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows Dictator and Lieutenants Workflow], [https://www.atlassian.com/git/tutorials/comparing-workflows/feature-branch-workflow Feature Branch Workflow], [http://nvie.com/posts/a-successful-git-branching-model/ GitFlow], [https://guides.github.com/introduction/flow/ GitHub Flow] and [https://www.endpoint.com/blog/2014/05/02/git-workflows-that-work others].<br />
<br />
From a first analysis comparing these workflows with the one currently used (with SVN) by gCube, [http://nvie.com/posts/a-successful-git-branching-model/ GitFlow] is the closest one and therefore is the one that most probably would match the gCube requirements. <br />
<br />
The essential points of this workflow (in comparisons with our current workflow) are:<br />
<br />
* there is a development branch where the new developments are done, like in our current workflow;<br />
* a new branch is created for each new release, like in our current workflow;<br />
* integration issues are solved in the release branch, like in our current workflow;<br />
* the release branch is deleted after the release is closed, differently from our current workflow where the release branch remain and is used for patches and future releases;<br />
* when a patch is required, it is done in a new branch, differently from our current workflow where the patches are done in the release branch. However, in both cases, the patches are merged back in the development branch;<br />
* a tag is created after each release/patch before deleting the branch. This is equivalent (has the same objective) of the tags done at the end of each release in our current workflow.<br />
<br />
The diagram below describes the main steps of GitFlow.<br />
<br />
[[File:/home/ggiammat/OneDrive/BlueBRIDGE/GitMigration/final document/git-model@2x.png|frame|none|alt=|caption ]]<br />
<br />
We are not obliged to adopt GitFlow ''in toto'', we will probably build our own workflow, based on GitFlow with some minor modifications like:<br />
<br />
* release branches are not deleted: they are used to “generate” different component releases for each gCube release (e.g. 1.0.0-3.10.0, 1.0.0-3.11.0, …) and for patch releases;<br />
* before tagging a release a new commit with “resolved” pom.xml and distro files is done. This commit is not merged back to develop;<br />
* patches (hotfix) are branched from the release branch;<br />
* all components tags are merged into the gCube repository (this would realize the GitHub publication we do with ad-hoc scripts now).<br />
<br />
A better and more in depth analysis will be done on this before defining the final workflow that gCube will adopt.<br />
<br />
= Migration Strategy =<br />
<br />
The migration of the entire code base of gCube from SVN to Git is a very big and time consuming task given the size of gCube. Making this migration on a single shot for the entire gCube system, would stop the development of the system since all developers should be committed to migrate their (plus the ''orphan'') components. <br />
<br />
A more meaningful strategy is to articulate the migration in different phases to distribute the effort of migrating the component over a large (months) time period. A possible four-phases approach could be:<br />
<br />
# Initially, the migration will be done on &quot;volunteering&quot; basis. If a developer wants to migrate her component, she can do it following the new procedures and using the new tools. This ''early adopter'' group will be useful also to collect feedback on the tools and procedures before applying them on large-scale;<br />
# Then, developers will be forced to use Git for new component only, while for new releases of old components they can continue to use SVN;<br />
# Then, developers will be forced to use Git for any component they release, also old ones;<br />
# Finally, developers will be forced to migrate to Git any remaining component in the gCube release (even if there would be no need of releasing them).<br />
<br />
Of course, the migration can start only after all the tools and procedures are ready to manage Git-based components. In addition, all the old tools and procedures (SVN-based) will remain in place and co-exists with the new ones until all the gCube components have been migrated.</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=GCube_Git_Migration&diff=30741GCube Git Migration2018-01-15T18:52:52Z<p>Gabriele.giammatteo: /* Git Repository Layout */</p>
<hr />
<div>__TOC__<br />
<br />
During the BlueBRIDGE project, we started an activity for migrating the gCube source code from the Subversion (SVN) technology to Git. An analysis of the requirements and the design of a proper process and tools to governate the migration has been conducted. However, due to resources constraints, the actual migration has not been implemented.<br />
<br />
This document collects the output of all the work done and wants to be an helpful starting point for the future implementation of the migration of gCube source code from SVN to Git.<br />
<br />
= Requirements =<br />
<br />
The migration will affect the tools and the procedures currently used in the gCube development, integration and release. We started our analysis by considering some basic requirements that the final solution should address.<br />
<br />
'''Gradual implementation'''<br />
<br />
The migration should be designed to allow a gradual migration of gCube components allowing for continuing the normal development, integration and release activities even if some components have been already migrated and some others not yet. In this way, no major delays will be introduced in the gCube release scheduling. To address this requirement, the procedures and the tools used in the integration of gCube must support both SVN and Git repositories and allow to seamless integration between components stored in SVN and Git.<br />
<br />
'''History of files'''<br />
<br />
The migration should keep the history of the source code. This means that not only the latest version of each source file should be available in the new Git repository, but also all the single commits of each single source file with, possibly, the associated meta-data (i.e. date, author, commit message) should be preserved.<br />
<br />
'''Branching schema'''<br />
<br />
In the current SVN repository, each component has multiple branches: a) the development branch (i.e. ''trunk''), b) one release branch for each release and, optionally, c) one or more private branch (or feature branch). This allows developers to organize better their code and to maintain multiple versions of their component in parallel. The same functionality should be preserved when migrating to Git.<br />
<br />
'''GitHub publication'''<br />
<br />
Currently the source code of each gCube release (composed by the source code of all components that constitute the release) is replicated in a repository on GitHub. The current solution takes the source code to publish on GitHub from the source packages built by Maven during the builds and published on Nexus. This was the only viable solution to publish the source code from SVN, but it has the disadvantage that the history and the authors of the source files are not kept when copied on GitHub. After the migration, since the technology of gCube source code repositories and GitHub will be the same (Git), it should be possible also to publish on GitHub the authors and the history of each source file.<br />
<br />
'''Git repositories management software'''<br />
<br />
The software that will be used to manage the Git repositories should guarantee at least the following functionalities:<br />
<br />
* integration with the ''research infrastructures LDAP'': this will maintain the developers identity in the new Git repositories;<br />
* REST interface to create new repositories: this will increase the automation of the migration operation;<br />
* management of roles: the source code should be readable to everybody without authentication, but only authorized users should be able to make changes;<br />
* web interface for browsing the source code and the commits;<br />
<br />
Additionally, the tool should also implement collaboration functionalities like fork of repositories, pull requests, code review. This would greatly improve the collaboration between gCube developers and external community developers.<br />
<br />
= Git Repository Layout =<br />
<br />
Currently the gCube source code is maintained in a single SVN repository available at http://svn.research-infrastructures.eu/public/d4science/gcube/. The repository has the following layout:<br />
<br />
* ''trunk'' for the development version of components<br />
* ''branches'' for the release branches of components<br />
* ''private'' for feature branches of components<br />
* ''tags'' to keep a copy of gCube source code for each gCube release<br />
<br />
More information on the current SVN layout can be found in the [https://wiki.gcube-system.org/gcube/Gcube_subversion_repository gCube wiki].<br />
<br />
Switching to Git, the single repository layout will be abandoned in favor of a '''per-component repository layout'''. Each component will have its own Git repository that will contain all the branches for that component. This approach is not only more in-line with the Git philosophy, but, mostly important, more practical.<br />
<br />
In fact, while SVN allows for checkout and commit of single files/folders, Git (by design) does not provide this functionality. This mean that each operation (e.g. checkout of source code to execute a build) involves the entire repository. Given the size of gCube system (hundred of components and more than 1GB of source code), it is not practical to have all the source code in a single repository. In addition, Git allows to assign permissions only at repository level and having a per-component repository layout can be useful if we decide to assign permissions on component basis.<br />
<br />
The following table maps the current SVN layout with the new proposed Git layout also considering the workflow that we want to adopt (see [[#Updates to Release Procedure|Updates to Release Procedure]]).<br />
<br />
{|border="1" cellpadding="4" cellspacing="0"<br />
|- style="background-color:#F2F2F2"<br />
! type<br />
! SVN<br />
! Git<br />
|-<br />
| development branch<br />
| the developments are all located under the /trunk directory organized per component (e.g. <code>/trunk/common/common-client</code>)<br />
| the <code>development</code> branch<br />
|-<br />
| release branch<br />
| the releases are all located under the /branches directory organized per component and per version (e.g. <code>/branches/common/common-client/1.1</code>)<br />
| a branch named accordingly with the version (e.g. <code>1.1</code>)<br />
|-<br />
| feature branch<br />
| users place their feature branches in the /private directory organized per component (e.g. <code>/private/common/common-client/my-feature-1</code>)<br />
| a branch named accordingly with the feature (e.g. <code>my-feature-1</code>)<br />
|-<br />
| tags<br />
| tags are copies of the source code under the /tags directory organized per gCube release, component and version (e.g. <code>/tags/gcube-4.5.0/common/common-client/1.1</code>)<br />
| a tag of the most recent commit in the release branch is created with the full version of the release (e.g. <code>1.1.0-4.5.0</code>)<br />
|}<br />
<br />
= Import of source code from SVN =<br />
<br />
A strong requirement of the migration to Git is to maintain the history of the source code for each component. To achieve this, it is not sufficient to simply create an empty Git repository and copy the latest version of files from SVN.<br />
<br />
Since this is a very common requirement when migration from SVN to Git, there is a command line tool distributed directly with Git to address this problem. This was the base for our importing tests described below. There are also few alternatives reported at the end of the section.<br />
<br />
'''[https://git-scm.com/docs/git-svn git-svn]''' is a tool that given a SVN repository URL is able to discover all the files and all the commits in the repository and replicate them in a new Git repository. It is very complete and highly customizable. It supports different repository layouts, can distinguish between branches and tags and tries to follow the history of each file even if it has been copied or moved in different locations at certain point in the repository history. It is also able to keep user identities (by providing a mapping file of SVN and Git users) and can ignore specific directories and/or revisions. It is distributed under the open source GPLv2 license.<br />
<br />
The tests we executed highlighted that given the size (both in terms of number of files and number of revisions) of the gCube SVN repository the import can take several time (hours) even for a single component because the tool will try to scan the entire repository looking for branches and tags of that component. However, using some options, it is possible to limit the tool to only scan the folders where we know the source files are and only for the revisions we know the source files exist. This reduces drastically the importing times (from hours to minutes). However, it has to be considered that this depends highly from the &quot;age&quot; of the component. In fact old components, for which we need to scan more revisions, will be slower that new components (with fewer revisions to scan) and from the number of commits/releases existing.<br />
<br />
The following command imports the code of the home-library component (trunk and branches) from revision 82000 to the current one:<br />
<br />
<source lang="bash">> git svn clone http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/Common/<br />
home-library/ --trunk http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/Common/home-library/ --branches http://svn.research-infrastructures.eu/public/d4science/gcube/branches/common/home-library/ --authors-file=users.txt --no-metadata --no-minimize-url --revision 82000:HEAD<br />
</source><br />
This test took about 10 minutes (tried in July 2017).<br />
<br />
This command can be used (setting the correct parameters) for importing all gCube components. To facilitate the users, it can be wrapped in a more high-level script that is able to determine (or, at least, to guess) the values for almost all command parameters:<br />
<br />
* the min and max revisions can be found by running <code>svn log</code> and extract the revision of the first and the latest commits;<br />
* the branches and tags directory can be guessed from the trunk directory using the gCube naming conventions;<br />
* the authors-file can be generated upfront (see [https://john.albin.net/git/convert-subversion-to-git here]) and it is the stame for all the components.<br />
<br />
In addition the wrapper script can create and publish the new Git repository using the repository software APIs.<br />
<br />
== Alternative tools ==<br />
<br />
'''[https://github.com/JohnAlbin/git-svn-migrate git-svn-migrate]''' is just a wrapper script around git-svn and does not add any functionality useful for us.<br />
<br />
'''[https://subgit.com/ SubGit]''' is a java-based importer that promises to be faster than git-svn. It is not an open source product, but the ''import'' functionality can be used for [https://subgit.com/pricing.html free] (while the ''mirror'' functionality has to be purchased). However, from the small-scale tests we executed on few components there were no relevant improvements with respect to the git-svn times.<br />
<br />
= Integration with release tools =<br />
<br />
Git is a very popular software and it is already supported by all the '''development tools''' used by gCube developers (e.g. Eclipse, IntellijIDEA) on all platforms. Its usage is also very well documented and several tutorials and instructions on how to use Git can be found online.<br />
<br />
Support for Git has been also added and tested in '''ETICS'''. The commands to checkout the code from Git are quite straightforward. Supposing that the URL of the repository is kept in the <code>vcsroot</code> property and the branch name in <code>tag</code>, the checkout command to fetch the latest version of the code can be:<br />
<br />
<source lang="">git clone --depth 1 ${vcsroot} --branch ${tag} ${moduleName}</source><br />
while for checking out a specific commit:<br />
<br />
<source lang="">git clone ${vcsroot} --branch ${tag} ${moduleName} && cd ${moduleName} && git reset 104491f60c14f09124806272b88a605d5a324735 --hard</source><br />
However, ETICS partially misses the support for Git in the module for the automatic [https://wiki.gcube-system.org/gcube/ETICS#ETICS_ModelSync synchronization of the model]. This should be added to fully exploit with the automation that ETICS offers.<br />
<br />
The only other tool dependent form SVN is the distribution script used to create a tag on SVN with all the software for the new gCube releases. However, this tool will be abandoned because, once the code will be on Git, there will be no necessity to create a tag on SVN since this functionality will be fully replaced by the publication of the release on GitHub.<br />
<br />
= Updates to Release Procedure =<br />
<br />
gCube follows a strict [https://wiki.gcube-system.org/gcube/Major/Minor_Release_Cycle_procedure release procedure] that defines also how developers should use SVN to develop and release their software (e.g. creation and naming of release branches). This procedure will, of course, be updated to match the (slightly) different Git functionalities.<br />
<br />
Of particular interest in this context is to have a look at the different ''workflow'' that developers uses with Git. There are several of them documented: [https://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows Centralized workflow], [https://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows Integration-Manger workflow], [https://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows Dictator and Lieutenants Workflow], [https://www.atlassian.com/git/tutorials/comparing-workflows/feature-branch-workflow Feature Branch Workflow], [http://nvie.com/posts/a-successful-git-branching-model/ GitFlow], [https://guides.github.com/introduction/flow/ GitHub Flow] and [https://www.endpoint.com/blog/2014/05/02/git-workflows-that-work others].<br />
<br />
From a first analysis comparing these workflows with the one currently used (with SVN) by gCube, [http://nvie.com/posts/a-successful-git-branching-model/ GitFlow] is the closest one and therefore is the one that most probably would match the gCube requirements. <br />
<br />
The essential points of this workflow (in comparisons with our current workflow) are:<br />
<br />
* there is a development branch where the new developments are done, like in our current workflow;<br />
* a new branch is created for each new release, like in our current workflow;<br />
* integration issues are solved in the release branch, like in our current workflow;<br />
* the release branch is deleted after the release is closed, differently from our current workflow where the release branch remain and is used for patches and future releases;<br />
* when a patch is required, it is done in a new branch, differently from our current workflow where the patches are done in the release branch. However, in both cases, the patches are merged back in the development branch;<br />
* a tag is created after each release/patch before deleting the branch. This is equivalent (has the same objective) of the tags done at the end of each release in our current workflow.<br />
<br />
The diagram below describes the main steps of GitFlow.<br />
<br />
[[File:/home/ggiammat/OneDrive/BlueBRIDGE/GitMigration/final document/git-model@2x.png|frame|none|alt=|caption ]]<br />
<br />
We are not obliged to adopt GitFlow ''in toto'', we will probably build our own workflow, based on GitFlow with some minor modifications like:<br />
<br />
* release branches are not deleted: they are used to “generate” different component releases for each gCube release (e.g. 1.0.0-3.10.0, 1.0.0-3.11.0, …) and for patch releases;<br />
* before tagging a release a new commit with “resolved” pom.xml and distro files is done. This commit is not merged back to develop;<br />
* patches (hotfix) are branched from the release branch;<br />
* all components tags are merged into the gCube repository (this would realize the GitHub publication we do with ad-hoc scripts now).<br />
<br />
A better and more in depth analysis will be done on this before defining the final workflow that gCube will adopt.<br />
<br />
= Migration Strategy =<br />
<br />
The migration of the entire code base of gCube from SVN to Git is a very big and time consuming task given the size of gCube. Making this migration on a single shot for the entire gCube system, would stop the development of the system since all developers should be committed to migrate their (plus the ''orphan'') components. <br />
<br />
A more meaningful strategy is to articulate the migration in different phases to distribute the effort of migrating the component over a large (months) time period. A possible four-phases approach could be:<br />
<br />
# Initially, the migration will be done on &quot;volunteering&quot; basis. If a developer wants to migrate her component, she can do it following the new procedures and using the new tools. This ''early adopter'' group will be useful also to collect feedback on the tools and procedures before applying them on large-scale;<br />
# Then, developers will be forced to use Git for new component only, while for new releases of old components they can continue to use SVN;<br />
# Then, developers will be forced to use Git for any component they release, also old ones;<br />
# Finally, developers will be forced to migrate to Git any remaining component in the gCube release (even if there would be no need of releasing them).<br />
<br />
Of course, the migration can start only after all the tools and procedures are ready to manage Git-based components. In addition, all the old tools and procedures (SVN-based) will remain in place and co-exists with the new ones until all the gCube components have been migrated.</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=GCube_Git_Migration&diff=30740GCube Git Migration2018-01-15T18:52:11Z<p>Gabriele.giammatteo: /* Git Repository Layout */</p>
<hr />
<div>__TOC__<br />
<br />
During the BlueBRIDGE project, we started an activity for migrating the gCube source code from the Subversion (SVN) technology to Git. An analysis of the requirements and the design of a proper process and tools to governate the migration has been conducted. However, due to resources constraints, the actual migration has not been implemented.<br />
<br />
This document collects the output of all the work done and wants to be an helpful starting point for the future implementation of the migration of gCube source code from SVN to Git.<br />
<br />
= Requirements =<br />
<br />
The migration will affect the tools and the procedures currently used in the gCube development, integration and release. We started our analysis by considering some basic requirements that the final solution should address.<br />
<br />
'''Gradual implementation'''<br />
<br />
The migration should be designed to allow a gradual migration of gCube components allowing for continuing the normal development, integration and release activities even if some components have been already migrated and some others not yet. In this way, no major delays will be introduced in the gCube release scheduling. To address this requirement, the procedures and the tools used in the integration of gCube must support both SVN and Git repositories and allow to seamless integration between components stored in SVN and Git.<br />
<br />
'''History of files'''<br />
<br />
The migration should keep the history of the source code. This means that not only the latest version of each source file should be available in the new Git repository, but also all the single commits of each single source file with, possibly, the associated meta-data (i.e. date, author, commit message) should be preserved.<br />
<br />
'''Branching schema'''<br />
<br />
In the current SVN repository, each component has multiple branches: a) the development branch (i.e. ''trunk''), b) one release branch for each release and, optionally, c) one or more private branch (or feature branch). This allows developers to organize better their code and to maintain multiple versions of their component in parallel. The same functionality should be preserved when migrating to Git.<br />
<br />
'''GitHub publication'''<br />
<br />
Currently the source code of each gCube release (composed by the source code of all components that constitute the release) is replicated in a repository on GitHub. The current solution takes the source code to publish on GitHub from the source packages built by Maven during the builds and published on Nexus. This was the only viable solution to publish the source code from SVN, but it has the disadvantage that the history and the authors of the source files are not kept when copied on GitHub. After the migration, since the technology of gCube source code repositories and GitHub will be the same (Git), it should be possible also to publish on GitHub the authors and the history of each source file.<br />
<br />
'''Git repositories management software'''<br />
<br />
The software that will be used to manage the Git repositories should guarantee at least the following functionalities:<br />
<br />
* integration with the ''research infrastructures LDAP'': this will maintain the developers identity in the new Git repositories;<br />
* REST interface to create new repositories: this will increase the automation of the migration operation;<br />
* management of roles: the source code should be readable to everybody without authentication, but only authorized users should be able to make changes;<br />
* web interface for browsing the source code and the commits;<br />
<br />
Additionally, the tool should also implement collaboration functionalities like fork of repositories, pull requests, code review. This would greatly improve the collaboration between gCube developers and external community developers.<br />
<br />
= Git Repository Layout =<br />
<br />
Currently the gCube source code is maintained in a single SVN repository available at http://svn.research-infrastructures.eu/public/d4science/gcube/. The repository has the following layout:<br />
<br />
* ''trunk'' for the development version of components<br />
* ''branches'' for the release branches of components<br />
* ''private'' for feature branches of components<br />
* ''tags'' to keep a copy of gCube source code for each gCube release<br />
<br />
More information on the current SVN layout can be found in the [https://wiki.gcube-system.org/gcube/Gcube_subversion_repository gCube wiki].<br />
<br />
Switching to Git, the single repository layout will be abandoned in favor of a '''per-component repository layout'''. Each component will have its own Git repository that will contain all the branches for that component. This approach is not only more in-line with the Git philosophy, but, mostly important, more practical.<br />
<br />
In fact, while SVN allows for checkout and commit of single files/folders, Git (by design) does not provide this functionality. This mean that each operation (e.g. checkout of source code to execute a build) involves the entire repository. Given the size of gCube system (hundred of components and more than 1GB of source code), it is not practical to have all the source code in a single repository. In addition, Git allows to assign permissions only at repository level and having a per-component repository layout can be useful if we decide to assign permissions on component basis.<br />
<br />
The following table maps the current SVN layout with the new proposed Git layout also considering the workflow that we want to adopt (see [[#header-n135|Updates to Release Procedure]]).<br />
<br />
{|border="1" cellpadding="4" cellspacing="0"<br />
|- style="background-color:#F2F2F2"<br />
! type<br />
! SVN<br />
! Git<br />
|-<br />
| development branch<br />
| the developments are all located under the /trunk directory organized per component (e.g. <code>/trunk/common/common-client</code>)<br />
| the <code>development</code> branch<br />
|-<br />
| release branch<br />
| the releases are all located under the /branches directory organized per component and per version (e.g. <code>/branches/common/common-client/1.1</code>)<br />
| a branch named accordingly with the version (e.g. <code>1.1</code>)<br />
|-<br />
| feature branch<br />
| users place their feature branches in the /private directory organized per component (e.g. <code>/private/common/common-client/my-feature-1</code>)<br />
| a branch named accordingly with the feature (e.g. <code>my-feature-1</code>)<br />
|-<br />
| tags<br />
| tags are copies of the source code under the /tags directory organized per gCube release, component and version (e.g. <code>/tags/gcube-4.5.0/common/common-client/1.1</code>)<br />
| a tag of the most recent commit in the release branch is created with the full version of the release (e.g. <code>1.1.0-4.5.0</code>)<br />
|}<br />
<br />
= Import of source code from SVN =<br />
<br />
A strong requirement of the migration to Git is to maintain the history of the source code for each component. To achieve this, it is not sufficient to simply create an empty Git repository and copy the latest version of files from SVN.<br />
<br />
Since this is a very common requirement when migration from SVN to Git, there is a command line tool distributed directly with Git to address this problem. This was the base for our importing tests described below. There are also few alternatives reported at the end of the section.<br />
<br />
'''[https://git-scm.com/docs/git-svn git-svn]''' is a tool that given a SVN repository URL is able to discover all the files and all the commits in the repository and replicate them in a new Git repository. It is very complete and highly customizable. It supports different repository layouts, can distinguish between branches and tags and tries to follow the history of each file even if it has been copied or moved in different locations at certain point in the repository history. It is also able to keep user identities (by providing a mapping file of SVN and Git users) and can ignore specific directories and/or revisions. It is distributed under the open source GPLv2 license.<br />
<br />
The tests we executed highlighted that given the size (both in terms of number of files and number of revisions) of the gCube SVN repository the import can take several time (hours) even for a single component because the tool will try to scan the entire repository looking for branches and tags of that component. However, using some options, it is possible to limit the tool to only scan the folders where we know the source files are and only for the revisions we know the source files exist. This reduces drastically the importing times (from hours to minutes). However, it has to be considered that this depends highly from the &quot;age&quot; of the component. In fact old components, for which we need to scan more revisions, will be slower that new components (with fewer revisions to scan) and from the number of commits/releases existing.<br />
<br />
The following command imports the code of the home-library component (trunk and branches) from revision 82000 to the current one:<br />
<br />
<source lang="bash">> git svn clone http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/Common/<br />
home-library/ --trunk http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/Common/home-library/ --branches http://svn.research-infrastructures.eu/public/d4science/gcube/branches/common/home-library/ --authors-file=users.txt --no-metadata --no-minimize-url --revision 82000:HEAD<br />
</source><br />
This test took about 10 minutes (tried in July 2017).<br />
<br />
This command can be used (setting the correct parameters) for importing all gCube components. To facilitate the users, it can be wrapped in a more high-level script that is able to determine (or, at least, to guess) the values for almost all command parameters:<br />
<br />
* the min and max revisions can be found by running <code>svn log</code> and extract the revision of the first and the latest commits;<br />
* the branches and tags directory can be guessed from the trunk directory using the gCube naming conventions;<br />
* the authors-file can be generated upfront (see [https://john.albin.net/git/convert-subversion-to-git here]) and it is the stame for all the components.<br />
<br />
In addition the wrapper script can create and publish the new Git repository using the repository software APIs.<br />
<br />
== Alternative tools ==<br />
<br />
'''[https://github.com/JohnAlbin/git-svn-migrate git-svn-migrate]''' is just a wrapper script around git-svn and does not add any functionality useful for us.<br />
<br />
'''[https://subgit.com/ SubGit]''' is a java-based importer that promises to be faster than git-svn. It is not an open source product, but the ''import'' functionality can be used for [https://subgit.com/pricing.html free] (while the ''mirror'' functionality has to be purchased). However, from the small-scale tests we executed on few components there were no relevant improvements with respect to the git-svn times.<br />
<br />
= Integration with release tools =<br />
<br />
Git is a very popular software and it is already supported by all the '''development tools''' used by gCube developers (e.g. Eclipse, IntellijIDEA) on all platforms. Its usage is also very well documented and several tutorials and instructions on how to use Git can be found online.<br />
<br />
Support for Git has been also added and tested in '''ETICS'''. The commands to checkout the code from Git are quite straightforward. Supposing that the URL of the repository is kept in the <code>vcsroot</code> property and the branch name in <code>tag</code>, the checkout command to fetch the latest version of the code can be:<br />
<br />
<source lang="">git clone --depth 1 ${vcsroot} --branch ${tag} ${moduleName}</source><br />
while for checking out a specific commit:<br />
<br />
<source lang="">git clone ${vcsroot} --branch ${tag} ${moduleName} && cd ${moduleName} && git reset 104491f60c14f09124806272b88a605d5a324735 --hard</source><br />
However, ETICS partially misses the support for Git in the module for the automatic [https://wiki.gcube-system.org/gcube/ETICS#ETICS_ModelSync synchronization of the model]. This should be added to fully exploit with the automation that ETICS offers.<br />
<br />
The only other tool dependent form SVN is the distribution script used to create a tag on SVN with all the software for the new gCube releases. However, this tool will be abandoned because, once the code will be on Git, there will be no necessity to create a tag on SVN since this functionality will be fully replaced by the publication of the release on GitHub.<br />
<br />
= Updates to Release Procedure =<br />
<br />
gCube follows a strict [https://wiki.gcube-system.org/gcube/Major/Minor_Release_Cycle_procedure release procedure] that defines also how developers should use SVN to develop and release their software (e.g. creation and naming of release branches). This procedure will, of course, be updated to match the (slightly) different Git functionalities.<br />
<br />
Of particular interest in this context is to have a look at the different ''workflow'' that developers uses with Git. There are several of them documented: [https://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows Centralized workflow], [https://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows Integration-Manger workflow], [https://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows Dictator and Lieutenants Workflow], [https://www.atlassian.com/git/tutorials/comparing-workflows/feature-branch-workflow Feature Branch Workflow], [http://nvie.com/posts/a-successful-git-branching-model/ GitFlow], [https://guides.github.com/introduction/flow/ GitHub Flow] and [https://www.endpoint.com/blog/2014/05/02/git-workflows-that-work others].<br />
<br />
From a first analysis comparing these workflows with the one currently used (with SVN) by gCube, [http://nvie.com/posts/a-successful-git-branching-model/ GitFlow] is the closest one and therefore is the one that most probably would match the gCube requirements. <br />
<br />
The essential points of this workflow (in comparisons with our current workflow) are:<br />
<br />
* there is a development branch where the new developments are done, like in our current workflow;<br />
* a new branch is created for each new release, like in our current workflow;<br />
* integration issues are solved in the release branch, like in our current workflow;<br />
* the release branch is deleted after the release is closed, differently from our current workflow where the release branch remain and is used for patches and future releases;<br />
* when a patch is required, it is done in a new branch, differently from our current workflow where the patches are done in the release branch. However, in both cases, the patches are merged back in the development branch;<br />
* a tag is created after each release/patch before deleting the branch. This is equivalent (has the same objective) of the tags done at the end of each release in our current workflow.<br />
<br />
The diagram below describes the main steps of GitFlow.<br />
<br />
[[File:/home/ggiammat/OneDrive/BlueBRIDGE/GitMigration/final document/git-model@2x.png|frame|none|alt=|caption ]]<br />
<br />
We are not obliged to adopt GitFlow ''in toto'', we will probably build our own workflow, based on GitFlow with some minor modifications like:<br />
<br />
* release branches are not deleted: they are used to “generate” different component releases for each gCube release (e.g. 1.0.0-3.10.0, 1.0.0-3.11.0, …) and for patch releases;<br />
* before tagging a release a new commit with “resolved” pom.xml and distro files is done. This commit is not merged back to develop;<br />
* patches (hotfix) are branched from the release branch;<br />
* all components tags are merged into the gCube repository (this would realize the GitHub publication we do with ad-hoc scripts now).<br />
<br />
A better and more in depth analysis will be done on this before defining the final workflow that gCube will adopt.<br />
<br />
= Migration Strategy =<br />
<br />
The migration of the entire code base of gCube from SVN to Git is a very big and time consuming task given the size of gCube. Making this migration on a single shot for the entire gCube system, would stop the development of the system since all developers should be committed to migrate their (plus the ''orphan'') components. <br />
<br />
A more meaningful strategy is to articulate the migration in different phases to distribute the effort of migrating the component over a large (months) time period. A possible four-phases approach could be:<br />
<br />
# Initially, the migration will be done on &quot;volunteering&quot; basis. If a developer wants to migrate her component, she can do it following the new procedures and using the new tools. This ''early adopter'' group will be useful also to collect feedback on the tools and procedures before applying them on large-scale;<br />
# Then, developers will be forced to use Git for new component only, while for new releases of old components they can continue to use SVN;<br />
# Then, developers will be forced to use Git for any component they release, also old ones;<br />
# Finally, developers will be forced to migrate to Git any remaining component in the gCube release (even if there would be no need of releasing them).<br />
<br />
Of course, the migration can start only after all the tools and procedures are ready to manage Git-based components. In addition, all the old tools and procedures (SVN-based) will remain in place and co-exists with the new ones until all the gCube components have been migrated.</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=GCube_Git_Migration&diff=30739GCube Git Migration2018-01-15T18:50:29Z<p>Gabriele.giammatteo: Created page with "__TOC__ During the BlueBRIDGE project, we started an activity for migrating the gCube source code from the Subversion (SVN) technology to Git. An analysis of the requirements..."</p>
<hr />
<div>__TOC__<br />
<br />
During the BlueBRIDGE project, we started an activity for migrating the gCube source code from the Subversion (SVN) technology to Git. An analysis of the requirements and the design of a proper process and tools to governate the migration has been conducted. However, due to resources constraints, the actual migration has not been implemented.<br />
<br />
This document collects the output of all the work done and wants to be an helpful starting point for the future implementation of the migration of gCube source code from SVN to Git.<br />
<br />
= Requirements =<br />
<br />
The migration will affect the tools and the procedures currently used in the gCube development, integration and release. We started our analysis by considering some basic requirements that the final solution should address.<br />
<br />
'''Gradual implementation'''<br />
<br />
The migration should be designed to allow a gradual migration of gCube components allowing for continuing the normal development, integration and release activities even if some components have been already migrated and some others not yet. In this way, no major delays will be introduced in the gCube release scheduling. To address this requirement, the procedures and the tools used in the integration of gCube must support both SVN and Git repositories and allow to seamless integration between components stored in SVN and Git.<br />
<br />
'''History of files'''<br />
<br />
The migration should keep the history of the source code. This means that not only the latest version of each source file should be available in the new Git repository, but also all the single commits of each single source file with, possibly, the associated meta-data (i.e. date, author, commit message) should be preserved.<br />
<br />
'''Branching schema'''<br />
<br />
In the current SVN repository, each component has multiple branches: a) the development branch (i.e. ''trunk''), b) one release branch for each release and, optionally, c) one or more private branch (or feature branch). This allows developers to organize better their code and to maintain multiple versions of their component in parallel. The same functionality should be preserved when migrating to Git.<br />
<br />
'''GitHub publication'''<br />
<br />
Currently the source code of each gCube release (composed by the source code of all components that constitute the release) is replicated in a repository on GitHub. The current solution takes the source code to publish on GitHub from the source packages built by Maven during the builds and published on Nexus. This was the only viable solution to publish the source code from SVN, but it has the disadvantage that the history and the authors of the source files are not kept when copied on GitHub. After the migration, since the technology of gCube source code repositories and GitHub will be the same (Git), it should be possible also to publish on GitHub the authors and the history of each source file.<br />
<br />
'''Git repositories management software'''<br />
<br />
The software that will be used to manage the Git repositories should guarantee at least the following functionalities:<br />
<br />
* integration with the ''research infrastructures LDAP'': this will maintain the developers identity in the new Git repositories;<br />
* REST interface to create new repositories: this will increase the automation of the migration operation;<br />
* management of roles: the source code should be readable to everybody without authentication, but only authorized users should be able to make changes;<br />
* web interface for browsing the source code and the commits;<br />
<br />
Additionally, the tool should also implement collaboration functionalities like fork of repositories, pull requests, code review. This would greatly improve the collaboration between gCube developers and external community developers.<br />
<br />
= Git Repository Layout =<br />
<br />
Currently the gCube source code is maintained in a single SVN repository available at http://svn.research-infrastructures.eu/public/d4science/gcube/. The repository has the following layout:<br />
<br />
* ''trunk'' for the development version of components<br />
* ''branches'' for the release branches of components<br />
* ''private'' for feature branches of components<br />
* ''tags'' to keep a copy of gCube source code for each gCube release<br />
<br />
More information on the current SVN layout can be found in the [https://wiki.gcube-system.org/gcube/Gcube_subversion_repository gCube wiki].<br />
<br />
Switching to Git, the single repository layout will be abandoned in favor of a '''per-component repository layout'''. Each component will have its own Git repository that will contain all the branches for that component. This approach is not only more in-line with the Git philosophy, but, mostly important, more practical.<br />
<br />
In fact, while SVN allows for checkout and commit of single files/folders, Git (by design) does not provide this functionality. This mean that each operation (e.g. checkout of source code to execute a build) involves the entire repository. Given the size of gCube system (hundred of components and more than 1GB of source code), it is not practical to have all the source code in a single repository. In addition, Git allows to assign permissions only at repository level and having a per-component repository layout can be useful if we decide to assign permissions on component basis.<br />
<br />
The following table maps the current SVN layout with the new proposed Git layout also considering the workflow that we want to adopt (see [[#header-n135|Updates to Release Procedure]]).<br />
<br />
{|<br />
! type<br />
! SVN<br />
! Git<br />
|-<br />
| development branch<br />
| the developments are all located under the /trunk directory organized per component (e.g. <code>/trunk/common/common-client</code>)<br />
| the <code>development</code> branch<br />
|-<br />
| release branch<br />
| the releases are all located under the /branches directory organized per component and per version (e.g. <code>/branches/common/common-client/1.1</code>)<br />
| a branch named accordingly with the version (e.g. <code>1.1</code>)<br />
|-<br />
| feature branch<br />
| users place their feature branches in the /private directory organized per component (e.g. <code>/private/common/common-client/my-feature-1</code>)<br />
| a branch named accordingly with the feature (e.g. <code>my-feature-1</code>)<br />
|-<br />
| tags<br />
| tags are copies of the source code under the /tags directory organized per gCube release, component and version (e.g. <code>/tags/gcube-4.5.0/common/common-client/1.1</code>)<br />
| a tag of the most recent commit in the release branch is created with the full version of the release (e.g. <code>1.1.0-4.5.0</code>)<br />
|}<br />
<br />
= Import of source code from SVN =<br />
<br />
A strong requirement of the migration to Git is to maintain the history of the source code for each component. To achieve this, it is not sufficient to simply create an empty Git repository and copy the latest version of files from SVN.<br />
<br />
Since this is a very common requirement when migration from SVN to Git, there is a command line tool distributed directly with Git to address this problem. This was the base for our importing tests described below. There are also few alternatives reported at the end of the section.<br />
<br />
'''[https://git-scm.com/docs/git-svn git-svn]''' is a tool that given a SVN repository URL is able to discover all the files and all the commits in the repository and replicate them in a new Git repository. It is very complete and highly customizable. It supports different repository layouts, can distinguish between branches and tags and tries to follow the history of each file even if it has been copied or moved in different locations at certain point in the repository history. It is also able to keep user identities (by providing a mapping file of SVN and Git users) and can ignore specific directories and/or revisions. It is distributed under the open source GPLv2 license.<br />
<br />
The tests we executed highlighted that given the size (both in terms of number of files and number of revisions) of the gCube SVN repository the import can take several time (hours) even for a single component because the tool will try to scan the entire repository looking for branches and tags of that component. However, using some options, it is possible to limit the tool to only scan the folders where we know the source files are and only for the revisions we know the source files exist. This reduces drastically the importing times (from hours to minutes). However, it has to be considered that this depends highly from the &quot;age&quot; of the component. In fact old components, for which we need to scan more revisions, will be slower that new components (with fewer revisions to scan) and from the number of commits/releases existing.<br />
<br />
The following command imports the code of the home-library component (trunk and branches) from revision 82000 to the current one:<br />
<br />
<source lang="bash">> git svn clone http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/Common/<br />
home-library/ --trunk http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/Common/home-library/ --branches http://svn.research-infrastructures.eu/public/d4science/gcube/branches/common/home-library/ --authors-file=users.txt --no-metadata --no-minimize-url --revision 82000:HEAD<br />
</source><br />
This test took about 10 minutes (tried in July 2017).<br />
<br />
This command can be used (setting the correct parameters) for importing all gCube components. To facilitate the users, it can be wrapped in a more high-level script that is able to determine (or, at least, to guess) the values for almost all command parameters:<br />
<br />
* the min and max revisions can be found by running <code>svn log</code> and extract the revision of the first and the latest commits;<br />
* the branches and tags directory can be guessed from the trunk directory using the gCube naming conventions;<br />
* the authors-file can be generated upfront (see [https://john.albin.net/git/convert-subversion-to-git here]) and it is the stame for all the components.<br />
<br />
In addition the wrapper script can create and publish the new Git repository using the repository software APIs.<br />
<br />
== Alternative tools ==<br />
<br />
'''[https://github.com/JohnAlbin/git-svn-migrate git-svn-migrate]''' is just a wrapper script around git-svn and does not add any functionality useful for us.<br />
<br />
'''[https://subgit.com/ SubGit]''' is a java-based importer that promises to be faster than git-svn. It is not an open source product, but the ''import'' functionality can be used for [https://subgit.com/pricing.html free] (while the ''mirror'' functionality has to be purchased). However, from the small-scale tests we executed on few components there were no relevant improvements with respect to the git-svn times.<br />
<br />
= Integration with release tools =<br />
<br />
Git is a very popular software and it is already supported by all the '''development tools''' used by gCube developers (e.g. Eclipse, IntellijIDEA) on all platforms. Its usage is also very well documented and several tutorials and instructions on how to use Git can be found online.<br />
<br />
Support for Git has been also added and tested in '''ETICS'''. The commands to checkout the code from Git are quite straightforward. Supposing that the URL of the repository is kept in the <code>vcsroot</code> property and the branch name in <code>tag</code>, the checkout command to fetch the latest version of the code can be:<br />
<br />
<source lang="">git clone --depth 1 ${vcsroot} --branch ${tag} ${moduleName}</source><br />
while for checking out a specific commit:<br />
<br />
<source lang="">git clone ${vcsroot} --branch ${tag} ${moduleName} && cd ${moduleName} && git reset 104491f60c14f09124806272b88a605d5a324735 --hard</source><br />
However, ETICS partially misses the support for Git in the module for the automatic [https://wiki.gcube-system.org/gcube/ETICS#ETICS_ModelSync synchronization of the model]. This should be added to fully exploit with the automation that ETICS offers.<br />
<br />
The only other tool dependent form SVN is the distribution script used to create a tag on SVN with all the software for the new gCube releases. However, this tool will be abandoned because, once the code will be on Git, there will be no necessity to create a tag on SVN since this functionality will be fully replaced by the publication of the release on GitHub.<br />
<br />
= Updates to Release Procedure =<br />
<br />
gCube follows a strict [https://wiki.gcube-system.org/gcube/Major/Minor_Release_Cycle_procedure release procedure] that defines also how developers should use SVN to develop and release their software (e.g. creation and naming of release branches). This procedure will, of course, be updated to match the (slightly) different Git functionalities.<br />
<br />
Of particular interest in this context is to have a look at the different ''workflow'' that developers uses with Git. There are several of them documented: [https://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows Centralized workflow], [https://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows Integration-Manger workflow], [https://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows Dictator and Lieutenants Workflow], [https://www.atlassian.com/git/tutorials/comparing-workflows/feature-branch-workflow Feature Branch Workflow], [http://nvie.com/posts/a-successful-git-branching-model/ GitFlow], [https://guides.github.com/introduction/flow/ GitHub Flow] and [https://www.endpoint.com/blog/2014/05/02/git-workflows-that-work others].<br />
<br />
From a first analysis comparing these workflows with the one currently used (with SVN) by gCube, [http://nvie.com/posts/a-successful-git-branching-model/ GitFlow] is the closest one and therefore is the one that most probably would match the gCube requirements. <br />
<br />
The essential points of this workflow (in comparisons with our current workflow) are:<br />
<br />
* there is a development branch where the new developments are done, like in our current workflow;<br />
* a new branch is created for each new release, like in our current workflow;<br />
* integration issues are solved in the release branch, like in our current workflow;<br />
* the release branch is deleted after the release is closed, differently from our current workflow where the release branch remain and is used for patches and future releases;<br />
* when a patch is required, it is done in a new branch, differently from our current workflow where the patches are done in the release branch. However, in both cases, the patches are merged back in the development branch;<br />
* a tag is created after each release/patch before deleting the branch. This is equivalent (has the same objective) of the tags done at the end of each release in our current workflow.<br />
<br />
The diagram below describes the main steps of GitFlow.<br />
<br />
[[File:/home/ggiammat/OneDrive/BlueBRIDGE/GitMigration/final document/git-model@2x.png|frame|none|alt=|caption ]]<br />
<br />
We are not obliged to adopt GitFlow ''in toto'', we will probably build our own workflow, based on GitFlow with some minor modifications like:<br />
<br />
* release branches are not deleted: they are used to “generate” different component releases for each gCube release (e.g. 1.0.0-3.10.0, 1.0.0-3.11.0, …) and for patch releases;<br />
* before tagging a release a new commit with “resolved” pom.xml and distro files is done. This commit is not merged back to develop;<br />
* patches (hotfix) are branched from the release branch;<br />
* all components tags are merged into the gCube repository (this would realize the GitHub publication we do with ad-hoc scripts now).<br />
<br />
A better and more in depth analysis will be done on this before defining the final workflow that gCube will adopt.<br />
<br />
= Migration Strategy =<br />
<br />
The migration of the entire code base of gCube from SVN to Git is a very big and time consuming task given the size of gCube. Making this migration on a single shot for the entire gCube system, would stop the development of the system since all developers should be committed to migrate their (plus the ''orphan'') components. <br />
<br />
A more meaningful strategy is to articulate the migration in different phases to distribute the effort of migrating the component over a large (months) time period. A possible four-phases approach could be:<br />
<br />
# Initially, the migration will be done on &quot;volunteering&quot; basis. If a developer wants to migrate her component, she can do it following the new procedures and using the new tools. This ''early adopter'' group will be useful also to collect feedback on the tools and procedures before applying them on large-scale;<br />
# Then, developers will be forced to use Git for new component only, while for new releases of old components they can continue to use SVN;<br />
# Then, developers will be forced to use Git for any component they release, also old ones;<br />
# Finally, developers will be forced to migrate to Git any remaining component in the gCube release (even if there would be no need of releasing them).<br />
<br />
Of course, the migration can start only after all the tools and procedures are ready to manage Git-based components. In addition, all the old tools and procedures (SVN-based) will remain in place and co-exists with the new ones until all the gCube components have been migrated.</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Developing_gCube_Maven_Components&diff=29743Developing gCube Maven Components2017-10-30T15:49:07Z<p>Gabriele.giammatteo: /* Service Archive (SA) */</p>
<hr />
<div>[[Category: Developer's Guide]]<br />
__TOC__<br />
<br />
= Common =<br />
<br />
== Maven Coordinates ==<br />
Maven components' coordinates must follow these conventions:<br />
<br />
* their <code>groupId</code> must be of the following form:<br />
<br />
<pre>org.gcube.<class></pre><br />
<br />
: where ''class'' identifies a functionally correlated set of components and helps a range of users (developers, administrator) to discover components in various UIs/reports used within the system and in tools. Logically, the class corresponds to the gCube class of the component's, as specified in its gCube Software Profile. However, more specific, or else cross-class groups may be agreed upon among developers. In all cases, the group should align with official Maven [http://maven.apache.org/guides/mini/guide-naming-conventions.html guidelines]. By agreement, for example, components in gCube class <code>DataAccess</code> declare a <code>groupId</code> of <code>org.gcube.data.access</code> and those in gCube class <code>DataTransfer</code> declare a <code>groupId</code> of <code>org.gcube.data.transfer</code>. On the other hand, also by agreement, components in gCube class <code>InformationSystem</code> declare a <code>groupId</code> of <code>org.gcube.informationsystem</code>.<br />
<br />
* their <code>artifactId</code> must follow Maven conventions (e.g. ''avoid capital letters or punctuation other than hyphens'').<br />
<br />
== Parent ==<br />
All gCube components must inherit from '''maven-parent'''. This ensure and promotes compliance with system-wide requirements, from the enforcement of minimal Java and Maven versions to generation and packaging of Javadoc documentation and component sources.<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
<br />
The maven-parent defines:<br />
* several system-wide properties like <code>gcube.license</code>, <code>gcube.wikiRoot</code>, <code>distroDirectory</code><br />
* the default Java version (see more in [[Developing_gCube_Maven_Components#Java_Version|Java Version]])<br />
* the execution of some plugins that run for every component like: <code>maven-javadoc-plugin</code>, <code>maven-source-plugin</code><br />
* the configuration of the <code>maven-assembly-plugin</code> to generate some gCube-specific packages: <code>servicearchive</code>, <code>uberjar</code> and <code>source-package</code> (see more in [[Developing_gCube_Maven_Components#Artifact_Types|Artifact Types]])<br />
* the logic to enable the generation of the source package<br />
<br />
The '''maven-parent''' is at http://maven.research-infrastructures.eu/nexus/service/local/repositories/gcube-externals/content/org/gcube/tools/maven-parent/1.0.0/maven-parent-1.0.0.pom<br />
<br />
== The distro folder ==<br />
This folder contains files that will be distributed along with the code in the distribution packages. Further details are in the [[Maven distro directory layout]] page.<br />
<br />
<br />
== Dependencies ==<br />
Service and libraries that depend on other gCube components (Maven-based or Ant-based) can find them in the gCube Maven Repositories: <code>gcube-snapshots</code> <strike>and <code>gcube-releases</code></strike>. <br />
<br />
=== gCube Maven Repositories ===<br />
As a one-off task, developers must configure these repositories in the <code>~/.m2/settings.xml</code> file of their local Maven installation, as follows:<br />
<br />
<source lang="xml"><br />
<settings xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"<br />
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd"><br />
<br />
<servers><br />
<server><br />
<id>gcube-snapshots</id><br />
<username>gcube-user</username><br />
<password>maven</password><br />
</server><br />
</servers><br />
<br />
<br />
<profiles><br />
<profile><br />
<id>gcube</id><br />
<repositories><br />
<repository><br />
<id>gcube-snapshots</id><br />
<name>gCube Snapshots</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
</snapshots><br />
</repository><br />
<repository><br />
<id>gcube-externals</id><br />
<name>gCube Externals</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals</url><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
</repository><br />
</repositories><br />
<br />
<pluginRepositories><br />
<pluginRepository><br />
<id>gcube-snapshots</id><br />
<name>gCube Snapshots</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
</snapshots><br />
</pluginRepository><br />
<pluginRepository><br />
<id>gcube-externals</id><br />
<name>gCube Externals</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals</url><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
</pluginRepository><br />
</pluginRepositories><br />
<br />
</profile><br />
</profiles><br />
<br />
<activeProfiles><br />
<activeProfile>gcube</activeProfile><br />
</activeProfiles><br />
</settings><br />
</source><br />
<br />
<br />
'''Note:Developers must not configure these repositories in their POMs''', only in <code>settings.xml</code>. <br />
<br />
This is to:<br />
* keep their projects independent from future changes in repository locations; <br />
* avoid interference with ETICS builds, where access to these repositories may comprise correct system integration;<br />
<br />
With this settings.xml, developers will have access on their local development environment to development versions (gcube-snapshots) and externals (gcube-externals). This is the correct behaviour for most of use cases. If access to staging or releases repositories is necessary (e.g. patching a component already released in the branch where dependencies versions in the release are not in snapshots), developer should modify his settings.xml to point to gcube-staging(releases) instead of gcube-snapshots.<br />
<br />
=== Third-party Repositories ===<br />
Using the the <code>settings.xml</code> aforementioned, any dependency declared in pom.xml that can be found in one of gCube repositories or in Maven Central will be resolved at build time and deployment time without any further configuration. Frequently it may happens that a gCube component needs software stored on a public Maven repository not in on of those repositories. In this case, Maven provides an easy solution for declaring additional repositories for dependency resolution by adding a <repositories> section in the pom.xml of the project. E.g.:<br />
<br />
<source lang="xml" highlight="3,4"><br />
<repositories><br />
<repository><br />
<id>[repository-id]</id><br />
<url>[repository URL]</url><br />
</repository><br />
</repositories><br />
</source><br />
<br />
Though technically possible to add any Maven repository in this way, in order to preserve the quality of gCube software it is required to check trustworthiness of its dependencies disallowing usage of certain external software. To enforce such a control over public Maven repositories, '''gCube components are allowed to use external repository not directly but only through a proxy repository''' (a.k.a. shadow repository) created on CNR's Nexus Repository Portal (http://maven.research-infrastructures.eu/nexus/).<br />
<br />
List of all repositories in Nexus Portal is available at http://maven.research-infrastructures.eu/nexus/index.html#view-repositories<br />
<br />
'''Note''': if a component uses a public Maven repository directly, it will work for building but would fail in deployment phase. In fact, Software Gateway is only able to download software accessible from Nexus.<br />
<br />
==== Request new proxy repository in Nexus ====<br />
Proxy Repositories on Nexus are created by the Release Manager. To request a new proxy a new ticket must be created.<br />
<br />
Details of ticket are:<br />
# open a ticket in tracker "Support" in gCube ticketing system (at https://support.d4science.org/projects/gcube/)<br />
# ticket's '''description''' must report:<br />
#* URL of repository to add<br />
#* organization which hosts the repository<br />
#* which artifacts are needed from that repository<br />
#* which components needs those artifacts<br />
# '''Sprint''' field should be set to the gCube release in which the component is expected to be released <br />
# '''Category''' field must report the gCube subsystem that needs the external repository<br />
# '''Assignee''' is the Release Manager<br />
<br />
An example of ticket for requesting a new repository mirror is reported in the figure below.<br />
[[Image:New_maven_mirror_repo.png|600px|center|Ticket for requesting a new Mirror Repository]]<br />
<br />
Release Manager will process the ticket checking that no issues are foreseen from the usage of software stored in the repository. If request is accepted, a new proxy repository is created and the repository's URL will be communicated publicly on gCube's developers mailinglist (a.k.a. TCOM mailinglist).<br />
<br />
Knowing the mirror repository URL, developer can use it adding following lines in pom.xml. E.g.<br />
<br />
<source lang="xml" highlight="3,4"><br />
<repositories><br />
<repository><br />
<id>gcube-ext-osgeo</id><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/osgeo/</url><br />
</repository><br />
</repositories><br />
</source><br />
<br />
'''Note''': it is possible that software stored on proxy repositories is not included in search results on Nexus. This happens for those repositories that does not have indexes (or have indexes in a format that Nexus is not able to download). Anyway, once Nexus has downloaded artifacts from the original repository for the first time (e.g. because a compilation of a pom that uses the proxy repository), creates its own indexes and those artifacts will become available also for local search.<br />
<br />
=== Dependency Versions ===<br />
<br />
ETICS builds ensure correct integration of gCube components, regardless of the versions their dependencies. It also ensures that components are released with fixed versions for their dependencies, i.e. ensures build reproducibility.<br />
<br />
Accordingly, developers can be liberal in the choice of dependency versions. i.e. rely on version ranges to:<br />
<br />
* minimise the requirements for version management;<br />
* be readily informed of any mis-alignment with their dependencies;<br />
<br />
As we start adopting Maven, the lower-bound of version ranges will be a SNAPSHOT version. For example, a dependency on the <code>streams</code> library can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>streams</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
</dependency><br />
</source><br />
<br />
With this dependency, a component will be soon notified of a non-retro compatible change that, contrary to conventions, <code>streams</code> may have introduced in, say, version <code>1.3.0</code> (typically the day after at the latest). The notification will be delivered in the IDE, assuming the availability of Maven integration (e.g. <code>m2e</code> in Eclipse). <br />
The component is in fact free to open the range further if it wants to be aligned with its dependencies across major versions.<br />
<br />
=== Common Dependencies ===<br />
<br />
Many gCube components depend on <code>gcf</code>, which is now available as a Maven component since version <code>1.4</code>. At the time of writing, the component has not been released yet. A dependency on it is then specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
The scope <code>provided</code> indicates that these dependency will be satisfied in the development environment (the gHN) and should not be included in GAR and SA archives.<br />
<br />
Some components depend on the libraries available in all gHNs, though they do not depend on <code>gcf</code> itself. The subset of these libraries that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called <code>ghn-core-runtime</code>. A dependency on this Jar can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ghn-core-runtime</artifactId><br />
<version>1.0.0</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Note: a component that depends on <code>gcf</code> indirectly depends on <code>gun-core-runtime</code>.<br />
<br />
Another common dependency is on the gCube gRS2, an Ant component that the build infrastructure make available in Maven repositories. A dependency on this component can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.execution</groupId><br />
<artifactId>grs2library</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
</dependency><br />
</source><br />
<br />
== Java Version ==<br />
<br />
In Maven the Java version of the source code and of the generated classes can be controlled using the configuration of the <code>maven-compiler-plugin</code>. E.g.:<br />
<syntaxhighlight lang="xml" highlight="5,6"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-compiler-plugin</artifactId><br />
<configuration><br />
<source>1.7</source><br />
<target>1.8</target><br />
</configuration><br />
</plugin><br />
</syntaxhighlight><br />
<br />
<br />
In gCube, this mechanism is exploited to enforce the usage of the same Java version for all components. In particular, the maven-parent defines the properties <code>java_version</code>, <code>maven.compiler.source</code> and <code>maven.compiler.target</code> and use these properties in the configuration of the <code>maven-compiler-plugin</code>.<br />
<syntaxhighlight lang="xml" highlight="15,16"><br />
...<br />
<properties><br />
<java_version>1.8</java_version><br />
<maven.compiler.source>${java_version}</maven.compiler.source><br />
<maven.compiler.target>${java_version}</maven.compiler.target><br />
...<br />
</properties><br />
<br />
...<br />
<br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-compiler-plugin</artifactId><br />
<configuration><br />
<target>${maven.compiler.target}</target><br />
<source>${maven.compiler.source}</source><br />
</configuration><br />
</plugin><br />
...<br />
</syntaxhighlight><br />
<br />
'''All the gCube components inherit this default configuration from <code>maven-parent</code> and it is not recommended to override it in the component's pom.xml'''.<br />
<br />
If it is really needed to override these values at component's pom.xml level, instead of specify a new configuration for the maven-compiler-plugin, the value of the property should be re-defined:<br />
<br />
<syntaxhighlight lang="xml" highlight="3"><br />
...<br />
<properties><br />
<maven.compiler.source>1.7</maven.compiler.source><br />
...<br />
</properties><br />
<br />
</syntaxhighlight><br />
<br />
<br />
This configuration is flexible enough to allow to customize the Java compiler source and target values depending on different factors in the integration builds (e.g. during ETICS builds gCore components are built with Java 7, all the others with Java 8).<br />
<br />
<br />
<br />
== Artifact Types ==<br />
<br />
There are four gCube-specific artifact types that are generated during the build of the gCube components. Each of them is summarized in the following subsections.<br />
<br />
=== Source Package ===<br />
The source package is an artifact with classifier <code>src</code> that contains the source code of the component plus the files in the distro directory (see [[Maven distro directory layout]] for more).<br />
<br />
Its descriptor is at [http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/build-utils/shared-assemblies/src/main/resources/assemblies/gcube-source-package.xml link].<br />
<br />
It is automatically generated if the <code>generateDistribution</code> property is ''true''.<br />
<br />
=== Uberjar ===<br />
<br />
The Uberjar is an artifact with classifier <code>uberjar</code> that contains the component classes plus all the jar files of the dependencies. This is not mandatory for all components, but only for those that needs to be deployed along with their dependenices.<br />
<br />
Its descriptor is at [http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/build-utils/shared-assemblies/src/main/resources/assemblies/gcube-uberjar.xml link].<br />
<br />
The generation of this package is '''not enabled by default'''. To enable it, bind its configuration to a build phase in the component pom.xml:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<executions><br />
<execution><br />
<id>make-uberjar</id><br />
<phase>package</phase><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
=== Service Archive (SA) ===<br />
Whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary <code>tar.gz</code> artefact with classifier <code>servicearchive</code>. Note that the SA of Maven components does ''not'' need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);<br />
<br />
* for each deployable artefact they produce (primary or secondary), they must include the Maven coordinates of the arteact in the corresponding package section of the gCube Software Profile. Specifying package dependencies in the Software Profile, on the other hand, is no longer required;<br />
<br />
* whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary <code>tar.gz</code> artefact with classifier <code>servicearchive</code>. Note that the SA of Maven components does ''not'' need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);<br />
<br />
<br />
The '''maven-parent''' defines a default descriptor to build a servicearchive ([http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/build-utils/shared-assemblies/src/main/resources/assemblies/gcube-default-servicearchive.xml link]). This descriptor fits most of single module gCube components that meet the requirements in [[Maven distro directory layout]]. If it does not fit your component, you will have to provide a custom descriptor.xml file and a custom configuration as described [[Maven_distro_directory_layout#descriptor.xml|here]].<br />
<br />
The generation of the default servicearchvie is disabled by default (beacuse it is not fits all cases, please see above). To enable it, bind its configuration to a build phase in the component pom.xml:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<executions><br />
<execution><br />
<id>make-servicearchive</id><br />
<phase>package</phase><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
=== Javadoc ===<br />
It is the default artifact (with classifier <code>javadoc</code>) generated by the <code>maven-javadoc-plugin</code>. It is automatically generated for all components.<br />
<br />
= Libraries =<br />
The <code>streams</code> library is a Maven component in gCube class <code>DataAccess</code>. We will use it as example of a library component in Maven.<br />
<br />
=== Profile ===<br />
<br />
The gCube Software profile of the library is as follows:<br />
<br />
<source lang="xml" highlight="14-18"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>Embedded Domain-Specific Language for Stream Transformations</Description><br />
<Class>DataAccess</Class><br />
<Name>streams</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>streams</Name><br />
<Version>1.0.0-SNAPSHOT</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>streams</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<Files><br />
<File>streams-1.0.0-SNAPSHOT.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the profile includes a single package and this package corresponds to the main build artefact of the component; <br />
* the whole profile and the package have usual gCube coordinates;<br />
* the package name is aligned with the Maven <code>artifactId</code> of the component;<br />
* the package version is aligned with the Maven <code>version</code> of the component;<br />
* the package includes <code>MavenCoordinates</code> that can be directly copied and pasted from the POM;<br />
* the package does '''not''' specify dependencies;<br />
* the package points to the main artefact of the Maven build;<br />
<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>streams</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>DataAccess</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<Files><br />
<File>${build.finalName}.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
=== Service Archive ===<br />
<br />
<code>streams</code> is responsible for generating its own SA as part of its Maven build. This requires dedicated logic in the POM but has the advantage that:<br />
<br />
* SA validity can be verified locally;<br />
* no SA configurations are required in ETICS;<br />
* the required build logic can be easily reused across components;<br />
<br />
<code>streams</code> uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation in the profile and other static files. The approach is best illustrated with a reference to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/streams sources]] of the <code>streams</code> library.<br />
<br />
<br />
As we discuss [[#Creating the ETICS Configurations|below]], <code>streams</code> has the final requirement to produce an interpolated profile during ETICS builds, outside the context of the SA.<br />
<code>streams</code> meets this requirement with the [http://maven.apache.org/plugins/maven-resources-plugin/ Maven Resources Plugin]. Again, consult the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/streams sources]] of the <code>streams</code> library to reuse this approach.<br />
<br />
= Services =<br />
The <code>tree-manager</code> service is a Maven component in gCube class <code>DataAccess</code>. <br />
<br />
== Project Structure ==<br />
<br />
Differently from a library, the service is associated with a number of distinguished build artefacts:<br />
<br />
* the service implementation (a Jar);<br />
* the stubs automatically generated from the WSDL interfaces (a Jar);<br />
* the archive required for deployment in gCore (a Gar);<br />
* the archive required for registration in the system (a SA);<br />
<br />
Producing these artefacts requires sharing a number of static and dynamic files (from profiles to WSDLs to pre-processed WSDLs). The usual Maven approach of mapping the artefacts onto different Maven projects is then genuinely difficult. It is instead more practical to use a single multi-module Maven project. The <code>tree-manager</code> uses this module structure:<br />
<br />
<pre><br />
tree-manager (parent module)------- (produces) --- (POM only)<br />
|<br />
|----- tree-manager-service (service impl module)------- (produces) --- jar, gar, SA<br />
| |<br />
| | (depends on)<br />
| |<br />
| \/<br />
|----- tree-manager-stubs (service stubs module)------- (produces) --- jar<br />
</pre><br />
<br />
As customary, the parent does not produce any artefact (is a POM-only project). It declares these coordinates in its POM:<br />
<br />
<source lang="xml"><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
<packaging>pom</packaging><br />
</source><br />
<br />
The parent physically includes the sub-modules;<br />
<br />
<source lang="xml"><br />
<modules><br />
<module>tree-manager-stubs</module><br />
<module>tree-manager-service</module><br />
</modules><br />
</source> <br />
<br />
Finally, the parent inherits from '''maven-parent''':<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
The submodules inherit from the parent:<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>tree-manager</artifactId><br />
<groupId>org.gcube.data.access</groupId><br />
<version>1.0.0-SNAPSHOT</version><br />
<relativePath>..</relativePath><br />
</parent><br />
</source><br />
<br />
and share its <code>version</code> and <code>groupId</code>. Their coordinates are then minimal:<br />
<br />
<source lang="xml"><br />
<artifactId>tree-manager-service</artifactId><br />
</source><br />
<br />
<source lang="xml"><br />
<artifactId>tree-manager-stubs</artifactId><br />
</source><br />
<br />
The <code>tree-manager-service</code> module produces a Jar of its own implementation, as well the GAR and the SA of the whole services as secondary artefacts with classifiers of, respectively, <code>gar</code> and <code>service archive</code>.<br />
<br />
The <code>tree-manager-stubs</code> module produces a Jar of its own implementation.<br />
<br />
Finally, the parent module coordinates the build of its sub-modules and centralises the declaration of common dependencies and configuration. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] to reuse this approach.<br />
<br />
== Generating Stubs and Gars ==<br />
<br />
The <code>tree-manager-stubs</code> uses the '''gCube Service Plugin''' to generate its classes from WSDL interfaces. <br />
<br />
Similarly, the <code>tree-manager-service</code> uses the plugin to generate its secondary GAR artefact. <br />
<br />
The configuration of the plugin is conveniently shared in the parent module.<br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] of the <code>streams</code> for details.<br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the service is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>DataAccess</Class><br />
<Name>tree-manager-service</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Main><br />
<Name>tree-manager-service</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager-service</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<GARArchive>tree-manager-service-1.0.0-SNAPSHOT.gar</GARArchive><br />
<PortType><br />
<Name>gcube/data/tm/binder</Name><br />
</PortType><br />
<PortType><br />
<Name>gcube/data/tm/reader</Name><br />
</PortType><br />
<PortType><br />
<Name>gcube/data/tm/writer</Name><br />
</PortType><br />
</Main><br />
<Software><br />
<Name>tree-manager-stubs</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager-stubs</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<Type>library</Type><br />
<Files><br />
<File>tree-manager-stubs-1.0.0-SNAPSHOT.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the profile includes two packages and this package corresponds to the main build artefacts of the submodules; <br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code>s of the submodules;<br />
* package versions are aligned with the Maven <code>version</code>s of the submodules;<br />
* package include <code>MavenCoordinates</code>s that can be directly copied and pasted from the POM of the submodules;<br />
* packages do '''not''' specify dependencies;<br />
* packages points to the main artefact of the Maven build of the submodules;<br />
<br />
<strike>Notice that, unlike libraries, services cannot easily use Maven variables in their profiles (as these artefacts are used for different purposes by different modules). Accordingly, developers needs to maintain versions used in profiles across releases. In particular, they need to:<br />
<br />
* replace SNAPSHOT versions with release versions in release branches;<br />
* advance SNAPSHOT versions in HEAD;</strike><br />
<br />
=== Profile Interpolation ===<br />
The interpolation of profiles is required in order to keep information in profile in sync with <code>pom.xml</code>, in particular versions and filenames (that contain versions) that change at each release.<br />
<br />
In order to generate <code>profile.xml</code> during build process you need to:<br />
# add in the service's parent <code>pom.xml</code> the invocation to maven-resources-plugin<br />
# place a ''template'' <code>profile.xml</code> in the <code>${distroDirectory}</code><br />
<br />
Invocation to <code>maven-resources-plugin</code> looks like the following (to be placed in the <plugins> section inside the <build> section of the pom):<br />
<source lang="xml"><br />
<!-- interpolates profiles and copies from distribution location to configuration <br />
location, where it is need for embedding into stub artifact, gar generation, <br />
and service archive. --><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-resources-plugin</artifactId><br />
<version>2.6</version><br />
<executions><br />
<execution><br />
<id>copy-profile</id><br />
<goals><br />
<goal>copy-resources</goal><br />
</goals><br />
<phase>process-resources</phase><br />
<configuration><br />
<outputDirectory>${configDirectory}</outputDirectory><br />
<resources><br />
<resource><br />
<directory>${distroDirectory}</directory><br />
<includes><br />
<include>profile.xml</include><br />
</includes><br />
<filtering>true</filtering><br />
</resource><br />
</resources><br />
</configuration><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
The ''template'' <code>profile.xml</code> looks like the following:<br />
<source lang="xml" highlight="12,16,18,22,26,30"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>TestClass</Class><br />
<Name>TestService</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Main><br />
<Name>TestService</Name><br />
<Version>${project.version}</Version><br />
<MavenCoordinates><br />
<groupId>org.example</groupId><br />
<artifactId>test-service</artifactId><br />
<version>${project.version}</version><br />
</MavenCoordinates><br />
<GARArchive>>test-service-${project.version}.gar</GARArchive><br />
</Main><br />
<Software><br />
<Name>test-stubs</Name><br />
<Version>${project.version}</Version><br />
<MavenCoordinates><br />
<groupId>org.example</groupId><br />
<artifactId>test-stubs</artifactId><br />
<version>${project.version}</version><br />
</MavenCoordinates><br />
<Type>library</Type><br />
<Files><br />
<File>test-stubs-${project.version}.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
In the example, property <code>${project.version}</code> will be substituted with the actual service's version at build-time. It is also possible, but not required, to parametrize also other values of <code>profile.xml</code> such as maven's coordinates and package names. You can see a more complex <code>profile.xml</code> ''template'' in the <code>tree-manager</code> project: http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-access/tree-manager/distro/profile.xml<br />
<br />
Maven will copy the final <code>profile.xml</code> in <code>${configDirectory}</code> during the <code>process-resources</code> phase of the build. From there it will be copied in the service-archive package.<br />
<br />
== Service Archive ==<br />
<br />
<code>tree-manager-service</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the module and its parent.<br />
<br />
As shown for [[#Libraries|libraries]], the service uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] for details.<br />
<br />
= Portlets =<br />
<br />
The <code>sample-portlet</code> portlet is a Maven based sample portlet based on GWT. Refer to the [http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources] for details.<br />
<br />
== Project Structure ==<br />
<br />
As for a library project, the portlet is associated with 2 distinguished build artefacts:<br />
<br />
* the archive required for deployment in a Liferay Container (a WAR);<br />
* the archive required for registration in the system (a SA);<br />
<br />
In order to generate automatically the project structure and the base pom, we start by customizing the maven archetype liferay-portlet-archetype v. 6.0.6.<br />
<br />
The generation of the project structure and the pom can be performed either from Eclipse ( by using the m2eclipse plugin) or by command line using the Maven SDK.<br />
<br />
On Eclipse the following steps must be followed:<br />
<br />
* FIle -> New -> Project<br />
* From the wizard select Maven->Maven Project<br />
* From the Select Archetype step, Select 'All Catalogs' from the Catalogs combobox and type 'portlet' on the Filter field.<br />
* Select the '''liferay-portlet-archetype''' version 6.0.6 archetype ( if does not appear on the list just unselect 'show the last version of the Archetype only")<br />
* In the next step you have to provide Maven coordinates for your project<br />
* groupid : <your group id> e.g org.gcube.portlets.admin<br />
* artifactId : <your artifactId> e.g. sample-admin-portlet<br />
* version : <your project version> e.g. 1.0.0-SNAPSHOT ( use always SNAPSHOT versions for development versions of a project)<br />
* package: <your portlet code package> e.g. org.gcube.portlets.admin.sample<br />
<br />
while from shell the following mvn command should be issued :<br />
<br />
<pre><br />
mvn archetype:generate \<br />
-DarchetypeArtifactId=liferay-theme-archetype \<br />
-DarchetypeGroupId=com.liferay.maven.archetypes \<br />
-DarchetypeVersion=6.0.6 \<br />
-DartifactId=<your artifactId> \<br />
-DgroupId=<your group id> \<br />
-Dversion=1.0.0-SNAPSHOT<br />
</pre><br />
<br />
Both approaches generate a project structure similar to the picture below:<br />
<br />
[[image:sample-portlet-project.png|Sample portlet project structure]]<br />
<br />
In order to customize the build procedure to the gCube needs we should open the pom.xml and apply the following changes:<br />
<br />
Configure the inheritance from '''maven-parent''' :<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
Configure some project properties:<br />
<br />
<source lang="xml"><br />
<properties><br />
<liferay.version>6.0.6</liferay.version><br />
<liferay.auto.deploy.dir>${system.CATALINA_HOME}/../deploy</liferay.auto.deploy.dir><br />
<distroDirectory>${project.basedir}/distro</distroDirectory><br />
<configDirectory>${project.basedir}/config</configDirectory><br />
</properties><br />
</source><br />
<br />
Add the plugin for the generation of the SA, which has to be included under the build plugins section:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<version>2.2</version><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
In the case of a GWT portlet the [http://mojo.codehaus.org/gwt-maven-plugin gwt-maven-plugin] should also be configured ( within build plugins section as well), by also selecting the proper gwt version ( in our example 2.4)<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.codehaus.mojo</groupId><br />
<artifactId>gwt-maven-plugin</artifactId><br />
<version>${gwt.version}</version><br />
<executions><br />
<execution><br />
<goals><br />
<goal>compile</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
The proper GWT version should be added to the pom properties :<br />
<br />
<source lang="xml"><br />
<gwt.version>2.4.0</gwt.version><br />
</source><br />
<br />
and GWT dependencies should be also configured:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-user</artifactId><br />
<version>${gwt.version}</version><br />
</dependency><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-servlet</artifactId><br />
<version>${gwt.version}</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet/ project sources]] for further details on the project configuration.<br />
<br />
== Generating Artifacts ==<br />
<br />
The <code>sample-portlet</code> uses the '''maven-compiler-plugin''' to compile the portlet code and the '''gwt-maven-plugin''' as well for GWT compilation.<br />
<br />
The generation of a war package can be triggered by typing :<br />
<br />
<pre><br />
mvn package <br />
</pre><br />
<br />
on your shell or running this build task within Eclipse.<br />
<br />
In both cases the war package will be generated under the ''target'' folder, e.g.:<br />
<br />
<pre><br />
target/sample-admin-portlet-1.0.0-SNAPSHOT.war<br />
</pre><br />
<br />
The generation of all project artifacts ( war, source jar, javadoc, SA) can be triggered by the build task :<br />
<br />
<pre><br />
mvn install <br />
</pre><br />
<br />
this will install as well all artifacts in your local repository, while the deployment of the artifacts to the remote gCube Maven repos can triggered by the build task:<br />
<br />
<pre><br />
mvn deploy <br />
</pre><br />
<br />
== Portlet dependencies ==<br />
<br />
Specific portlet dependencies ( e.g. GWT, service stubs ) should be packaged within the portlet war. All dependencies specified within the pom file without the ''provided'' scope will be copied inside the WEB-INF/lib folder at packaging time.<br />
<br />
For this reason all portlet specific dependencies should not specified that scope e.g:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.externals</groupId><br />
<artifactId>gwt-ext-patched</artifactId><br />
<version>2.0.5</version><br />
</dependency><br />
</source><br />
<br />
On the other hand the system dependencies ( the one contained within the [[GCube_Portal_Installation#Portal_Bundle_Libraries|Portal bundle]] ) should not be included inside the portlet war. As said in this case the ''provided'' scope should be specified, e.g.<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portal</groupId><br />
<artifactId>custom-portal-handler</artifactId><br />
<version>1.1.0-SNAPSHOT</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.4.0,1.5.0]</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
<br />
A subset of these libraries ( contained in a GHN) that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called <code>ghn-core-runtime</code>. A dependency on this Jar can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ghn-core-runtime</artifactId><br />
<version>1.0.0</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Note: a component that depends on <code>gcf</code> indirectly depends on <code>gun-core-runtime</code>.<br />
<br />
== Include gCube Guided Tour using Maven ==<br />
<br />
This particular component is not yet ported to maven so in order to be included in a maven based project it should be declared as system dependency as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portletsuser</groupId><br />
<artifactId>gcube-guidedtour-widget</artifactId><br />
<version>1.0.0</version><br />
<scope>system</scope><br />
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/gcube-guidedtour-widget.jar</systemPath> <br />
</dependency><br />
</source><br />
<br />
together with the gCube Widget Library:<br />
<br />
<source lang="xml"> <br />
<dependency><br />
<groupId>org.gcube.portlets.user</groupId><br />
<artifactId>gcubewidgets</artifactId><br />
<version>1.2.0</version><br />
<scope>system</scope><br />
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/gCube-Widgets-Library.jar</systemPath><br />
</dependency><br />
</source><br />
<br />
both components can be downloaded from the distribution repository [[http://grids16.eng.it:8080/BuildReport/browse/Recent_Builds]]. <br />
<br />
In case the portlet depends on the ''gcf'' component, some dependency resolution changes are required to your portlet in order to properly compile ( because of libraries clashes)<br />
<br />
The ghn-core-runtime component ( gcf dependency) should be excluded:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.5.1-SNAPSHOT,2.0.0)</version><br />
<scope>provided</scope><br />
<exclusions><br />
<exclusion><br />
<artifactId>ghn-core-runtime</artifactId><br />
<groupId>org.gcube.distribution</groupId><br />
</exclusion><br />
</exclusions><br />
</dependency><br />
</source><br />
<br />
<br />
and some new dependencies should be added:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>xerces</groupId><br />
<artifactId>xercesImpl</artifactId><br />
<version>2.9.1</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ws-addressing</artifactId><br />
<version>1.0</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>axis-patched</artifactId><br />
<version>1.2RC</version><br />
<scope>provided</scope><br />
</dependency><br />
<br />
</source><br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the portlet is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>PortletAdmin</Class><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.portles.admin</groupId><br />
<artifactId>sample-admin-portlet</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/sample-admin-portlet-1.0.0-SNAPSHOT.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code> of the project;<br />
* package versions are aligned with the Maven <code>version</code> of the project;<br />
* package include <code>MavenCoordinates</code> that can be directly copied and pasted from the POM of the project;<br />
* packages do '''not''' specify dependencies;<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>sample-portlet</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>PortletAdmin</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/${build.finalName}.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
== Service Archive ==<br />
<br />
<code>sample-portlet</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the project.<br />
<br />
As shown for [[#Libraries|libraries]], the portlet uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources]] for details.<br />
<br />
== Use Case: Create a new Mavenized GWT Portlet ==<br />
<br />
In order to create a new Mavenized GWT Portlet this detailed guide can be also followed [https://gcube.wiki.gcube-system.org/gcube/index.php/Create_a_new_Mavenized_gCube_GWT_Portlet_Guide]<br />
<br />
= Widgets =<br />
<br />
Falls in this category of components mainly GWT widgets or so called GWT Libraries, which can be used to package classes for mutualization and/or modularization. A Widget is a java archive (JAR) containing both classes and java sources for later GWT compilation and a gwt.xml module descriptor. <br />
<br />
In this section we refer to a sample Widget with maven structure (https://svn.research-infrastructures.eu/d4science/gcube/trunk/portal/mvn-widget-sample/)<br />
<br />
== Project Structure ==<br />
<br />
As for a other project, the portlet is associated with 2 distinguished build artefacts:<br />
<br />
* the archive required for deployment (a JAR);<br />
* the archive required for registration in the system (a SA);<br />
<br />
In order to generate automatically the project structure and the base pom, we start by customizing the maven archetype liferay-portlet-archetype v. 6.0.6.<br />
<br />
The generation of the project structure and the pom can be performed either from Eclipse ( by using the m2eclipse plugin) or by command line using the Maven SDK.<br />
<br />
On Eclipse the following steps must be followed:<br />
<br />
* FIle -> New -> Project<br />
* From the wizard select Maven->Maven Project<br />
* From the Select Archetype step, Select 'All Catalogs' from the Catalogs combobox and type 'portlet' on the Filter field.<br />
* Select the '''maven-archetype-quickstart''' version 1.1.0 archetype ( if does not appear on the list just unselect 'show the last version of the Archetype only")<br />
* In the next step you have to provide Maven coordinates for your project<br />
* groupid : <your group id> e.g org.gcube.portlets.admin<br />
* artifactId : <your artifactId> e.g. sample-admin-portlet<br />
* version : <your project version> e.g. 1.0.0-SNAPSHOT ( use always SNAPSHOT versions for development versions of a project)<br />
* package: <your portlet code package> e.g. org.gcube.portlets.admin.sample<br />
<br />
while from shell the following mvn command should be issued :<br />
<br />
<pre><br />
mvn archetype:generate \<br />
-DarchetypeArtifactId=maven-archetype-quickstart \<br />
-DarchetypeGroupId=com.liferay.maven.archetypes \<br />
-DarchetypeVersion=1.1.0 \<br />
-DartifactId=<your artifactId> \<br />
-DgroupId=<your group id> \<br />
-Dversion=1.0.0-SNAPSHOT<br />
</pre><br />
<br />
Both approaches generate a project structure similar to the picture below:<br />
<br />
[[image:sample-widget-project.png|Sample widget project structure]]<br />
<br />
In order to customize the build procedure to the gCube needs we should open the pom.xml and apply the following changes:<br />
<br />
Configure the inheritance from '''maven-parent''' :<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
Configure some project properties:<br />
<br />
<source lang="xml"><br />
<properties><br />
<distroDirectory>${project.basedir}/distro</distroDirectory><br />
<configDirectory>${project.basedir}/config</configDirectory><br />
</properties><br />
</source><br />
<br />
Add the plugin for the generation of the SA, which has to be included under the build plugins section:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<version>2.2</version><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
In the case of a GWT widget the [http://mojo.codehaus.org/gwt-maven-plugin gwt-maven-plugin] should also be configured ( within build plugins section as well), by also selecting the proper gwt version ( in our example 2.4)<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.codehaus.mojo</groupId><br />
<artifactId>gwt-maven-plugin</artifactId><br />
<version>${gwt.version}</version><br />
<executions><br />
<execution><br />
<goals><br />
<goal>compile</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
The proper GWT version should be added to the pom properties :<br />
<br />
<source lang="xml"><br />
<gwt.version>2.4.0</gwt.version><br />
</source><br />
<br />
and GWT dependencies should be also configured:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-user</artifactId><br />
<version>${gwt.version}</version><br />
</dependency><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-servlet</artifactId><br />
<version>${gwt.version}</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/mvn-widget-sample/ project sources]] for further details on the project configuration.<br />
<br />
== Generating Artifacts ==<br />
<br />
The <code>sample-portlet</code> uses the '''maven-compiler-plugin''' to compile the portlet code and the '''gwt-maven-plugin''' as well for GWT compilation.<br />
<br />
The generation of a war package can be triggered by typing :<br />
<br />
<pre><br />
mvn package <br />
</pre><br />
<br />
on your shell or running this build task within Eclipse.<br />
<br />
In both cases the war package will be generated under the ''target'' folder, e.g.:<br />
<br />
<pre><br />
target/sample-admin-portlet-1.0.0-SNAPSHOT.war<br />
</pre><br />
<br />
The generation of all project artifacts ( war, source jar, javadoc, SA) can be triggered by the build task :<br />
<br />
<pre><br />
mvn install <br />
</pre><br />
<br />
this will install as well all artifacts in your local repository, while the deployment of the artifacts to the remote gCube Maven repos can triggered by the build task:<br />
<br />
<pre><br />
mvn deploy <br />
</pre><br />
<br />
== Widget dependencies ==<br />
<br />
The system dependencies ( the one contained within the [[GCube_Portal_Installation#Portal_Bundle_Libraries|Portal bundle]] ) should not be included inside the portlet war. As said in this case the ''provided'' scope should be specified, e.g.<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portal</groupId><br />
<artifactId>custom-portal-handler</artifactId><br />
<version>1.1.0-SNAPSHOT</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.4.0,1.5.0]</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the widget is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>PortletAdmin</Class><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.portles.admin</groupId><br />
<artifactId>sample-admin-portlet</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/sample-admin-portlet-1.0.0-SNAPSHOT.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code> of the project;<br />
* package versions are aligned with the Maven <code>version</code> of the project;<br />
* package include <code>MavenCoordinates</code> that can be directly copied and pasted from the POM of the project;<br />
* packages do '''not''' specify dependencies;<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>sample-portlet</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>PortletAdmin</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/${build.finalName}.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
== Service Archive ==<br />
<br />
<code>sample-portlet</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the project.<br />
<br />
As shown for [[#Libraries|libraries]], the portlet uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources]] for details.</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Developing_gCube_Maven_Components&diff=29739Developing gCube Maven Components2017-10-30T14:41:19Z<p>Gabriele.giammatteo: /* Service Archive (SA) */</p>
<hr />
<div>[[Category: Developer's Guide]]<br />
__TOC__<br />
<br />
= Common =<br />
<br />
== Maven Coordinates ==<br />
Maven components' coordinates must follow these conventions:<br />
<br />
* their <code>groupId</code> must be of the following form:<br />
<br />
<pre>org.gcube.<class></pre><br />
<br />
: where ''class'' identifies a functionally correlated set of components and helps a range of users (developers, administrator) to discover components in various UIs/reports used within the system and in tools. Logically, the class corresponds to the gCube class of the component's, as specified in its gCube Software Profile. However, more specific, or else cross-class groups may be agreed upon among developers. In all cases, the group should align with official Maven [http://maven.apache.org/guides/mini/guide-naming-conventions.html guidelines]. By agreement, for example, components in gCube class <code>DataAccess</code> declare a <code>groupId</code> of <code>org.gcube.data.access</code> and those in gCube class <code>DataTransfer</code> declare a <code>groupId</code> of <code>org.gcube.data.transfer</code>. On the other hand, also by agreement, components in gCube class <code>InformationSystem</code> declare a <code>groupId</code> of <code>org.gcube.informationsystem</code>.<br />
<br />
* their <code>artifactId</code> must follow Maven conventions (e.g. ''avoid capital letters or punctuation other than hyphens'').<br />
<br />
== Parent ==<br />
All gCube components must inherit from '''maven-parent'''. This ensure and promotes compliance with system-wide requirements, from the enforcement of minimal Java and Maven versions to generation and packaging of Javadoc documentation and component sources.<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
<br />
The maven-parent defines:<br />
* several system-wide properties like <code>gcube.license</code>, <code>gcube.wikiRoot</code>, <code>distroDirectory</code><br />
* the default Java version (see more in [[Developing_gCube_Maven_Components#Java_Version|Java Version]])<br />
* the execution of some plugins that run for every component like: <code>maven-javadoc-plugin</code>, <code>maven-source-plugin</code><br />
* the configuration of the <code>maven-assembly-plugin</code> to generate some gCube-specific packages: <code>servicearchive</code>, <code>uberjar</code> and <code>source-package</code> (see more in [[Developing_gCube_Maven_Components#Artifact_Types|Artifact Types]])<br />
* the logic to enable the generation of the source package<br />
<br />
The '''maven-parent''' is at http://maven.research-infrastructures.eu/nexus/service/local/repositories/gcube-externals/content/org/gcube/tools/maven-parent/1.0.0/maven-parent-1.0.0.pom<br />
<br />
== The distro folder ==<br />
This folder contains files that will be distributed along with the code in the distribution packages. Further details are in the [[Maven distro directory layout]] page.<br />
<br />
<br />
== Dependencies ==<br />
Service and libraries that depend on other gCube components (Maven-based or Ant-based) can find them in the gCube Maven Repositories: <code>gcube-snapshots</code> <strike>and <code>gcube-releases</code></strike>. <br />
<br />
=== gCube Maven Repositories ===<br />
As a one-off task, developers must configure these repositories in the <code>~/.m2/settings.xml</code> file of their local Maven installation, as follows:<br />
<br />
<source lang="xml"><br />
<settings xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"<br />
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd"><br />
<br />
<servers><br />
<server><br />
<id>gcube-snapshots</id><br />
<username>gcube-user</username><br />
<password>maven</password><br />
</server><br />
</servers><br />
<br />
<br />
<profiles><br />
<profile><br />
<id>gcube</id><br />
<repositories><br />
<repository><br />
<id>gcube-snapshots</id><br />
<name>gCube Snapshots</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
</snapshots><br />
</repository><br />
<repository><br />
<id>gcube-externals</id><br />
<name>gCube Externals</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals</url><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
</repository><br />
</repositories><br />
<br />
<pluginRepositories><br />
<pluginRepository><br />
<id>gcube-snapshots</id><br />
<name>gCube Snapshots</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
</snapshots><br />
</pluginRepository><br />
<pluginRepository><br />
<id>gcube-externals</id><br />
<name>gCube Externals</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals</url><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
</pluginRepository><br />
</pluginRepositories><br />
<br />
</profile><br />
</profiles><br />
<br />
<activeProfiles><br />
<activeProfile>gcube</activeProfile><br />
</activeProfiles><br />
</settings><br />
</source><br />
<br />
<br />
'''Note:Developers must not configure these repositories in their POMs''', only in <code>settings.xml</code>. <br />
<br />
This is to:<br />
* keep their projects independent from future changes in repository locations; <br />
* avoid interference with ETICS builds, where access to these repositories may comprise correct system integration;<br />
<br />
With this settings.xml, developers will have access on their local development environment to development versions (gcube-snapshots) and externals (gcube-externals). This is the correct behaviour for most of use cases. If access to staging or releases repositories is necessary (e.g. patching a component already released in the branch where dependencies versions in the release are not in snapshots), developer should modify his settings.xml to point to gcube-staging(releases) instead of gcube-snapshots.<br />
<br />
=== Third-party Repositories ===<br />
Using the the <code>settings.xml</code> aforementioned, any dependency declared in pom.xml that can be found in one of gCube repositories or in Maven Central will be resolved at build time and deployment time without any further configuration. Frequently it may happens that a gCube component needs software stored on a public Maven repository not in on of those repositories. In this case, Maven provides an easy solution for declaring additional repositories for dependency resolution by adding a <repositories> section in the pom.xml of the project. E.g.:<br />
<br />
<source lang="xml" highlight="3,4"><br />
<repositories><br />
<repository><br />
<id>[repository-id]</id><br />
<url>[repository URL]</url><br />
</repository><br />
</repositories><br />
</source><br />
<br />
Though technically possible to add any Maven repository in this way, in order to preserve the quality of gCube software it is required to check trustworthiness of its dependencies disallowing usage of certain external software. To enforce such a control over public Maven repositories, '''gCube components are allowed to use external repository not directly but only through a proxy repository''' (a.k.a. shadow repository) created on CNR's Nexus Repository Portal (http://maven.research-infrastructures.eu/nexus/).<br />
<br />
List of all repositories in Nexus Portal is available at http://maven.research-infrastructures.eu/nexus/index.html#view-repositories<br />
<br />
'''Note''': if a component uses a public Maven repository directly, it will work for building but would fail in deployment phase. In fact, Software Gateway is only able to download software accessible from Nexus.<br />
<br />
==== Request new proxy repository in Nexus ====<br />
Proxy Repositories on Nexus are created by the Release Manager. To request a new proxy a new ticket must be created.<br />
<br />
Details of ticket are:<br />
# open a ticket in tracker "Support" in gCube ticketing system (at https://support.d4science.org/projects/gcube/)<br />
# ticket's '''description''' must report:<br />
#* URL of repository to add<br />
#* organization which hosts the repository<br />
#* which artifacts are needed from that repository<br />
#* which components needs those artifacts<br />
# '''Sprint''' field should be set to the gCube release in which the component is expected to be released <br />
# '''Category''' field must report the gCube subsystem that needs the external repository<br />
# '''Assignee''' is the Release Manager<br />
<br />
An example of ticket for requesting a new repository mirror is reported in the figure below.<br />
[[Image:New_maven_mirror_repo.png|600px|center|Ticket for requesting a new Mirror Repository]]<br />
<br />
Release Manager will process the ticket checking that no issues are foreseen from the usage of software stored in the repository. If request is accepted, a new proxy repository is created and the repository's URL will be communicated publicly on gCube's developers mailinglist (a.k.a. TCOM mailinglist).<br />
<br />
Knowing the mirror repository URL, developer can use it adding following lines in pom.xml. E.g.<br />
<br />
<source lang="xml" highlight="3,4"><br />
<repositories><br />
<repository><br />
<id>gcube-ext-osgeo</id><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/osgeo/</url><br />
</repository><br />
</repositories><br />
</source><br />
<br />
'''Note''': it is possible that software stored on proxy repositories is not included in search results on Nexus. This happens for those repositories that does not have indexes (or have indexes in a format that Nexus is not able to download). Anyway, once Nexus has downloaded artifacts from the original repository for the first time (e.g. because a compilation of a pom that uses the proxy repository), creates its own indexes and those artifacts will become available also for local search.<br />
<br />
=== Dependency Versions ===<br />
<br />
ETICS builds ensure correct integration of gCube components, regardless of the versions their dependencies. It also ensures that components are released with fixed versions for their dependencies, i.e. ensures build reproducibility.<br />
<br />
Accordingly, developers can be liberal in the choice of dependency versions. i.e. rely on version ranges to:<br />
<br />
* minimise the requirements for version management;<br />
* be readily informed of any mis-alignment with their dependencies;<br />
<br />
As we start adopting Maven, the lower-bound of version ranges will be a SNAPSHOT version. For example, a dependency on the <code>streams</code> library can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>streams</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
</dependency><br />
</source><br />
<br />
With this dependency, a component will be soon notified of a non-retro compatible change that, contrary to conventions, <code>streams</code> may have introduced in, say, version <code>1.3.0</code> (typically the day after at the latest). The notification will be delivered in the IDE, assuming the availability of Maven integration (e.g. <code>m2e</code> in Eclipse). <br />
The component is in fact free to open the range further if it wants to be aligned with its dependencies across major versions.<br />
<br />
=== Common Dependencies ===<br />
<br />
Many gCube components depend on <code>gcf</code>, which is now available as a Maven component since version <code>1.4</code>. At the time of writing, the component has not been released yet. A dependency on it is then specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
The scope <code>provided</code> indicates that these dependency will be satisfied in the development environment (the gHN) and should not be included in GAR and SA archives.<br />
<br />
Some components depend on the libraries available in all gHNs, though they do not depend on <code>gcf</code> itself. The subset of these libraries that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called <code>ghn-core-runtime</code>. A dependency on this Jar can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ghn-core-runtime</artifactId><br />
<version>1.0.0</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Note: a component that depends on <code>gcf</code> indirectly depends on <code>gun-core-runtime</code>.<br />
<br />
Another common dependency is on the gCube gRS2, an Ant component that the build infrastructure make available in Maven repositories. A dependency on this component can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.execution</groupId><br />
<artifactId>grs2library</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
</dependency><br />
</source><br />
<br />
== Java Version ==<br />
<br />
In Maven the Java version of the source code and of the generated classes can be controlled using the configuration of the <code>maven-compiler-plugin</code>. E.g.:<br />
<syntaxhighlight lang="xml" highlight="5,6"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-compiler-plugin</artifactId><br />
<configuration><br />
<source>1.7</source><br />
<target>1.8</target><br />
</configuration><br />
</plugin><br />
</syntaxhighlight><br />
<br />
<br />
In gCube, this mechanism is exploited to enforce the usage of the same Java version for all components. In particular, the maven-parent defines the properties <code>java_version</code>, <code>maven.compiler.source</code> and <code>maven.compiler.target</code> and use these properties in the configuration of the <code>maven-compiler-plugin</code>.<br />
<syntaxhighlight lang="xml" highlight="15,16"><br />
...<br />
<properties><br />
<java_version>1.8</java_version><br />
<maven.compiler.source>${java_version}</maven.compiler.source><br />
<maven.compiler.target>${java_version}</maven.compiler.target><br />
...<br />
</properties><br />
<br />
...<br />
<br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-compiler-plugin</artifactId><br />
<configuration><br />
<target>${maven.compiler.target}</target><br />
<source>${maven.compiler.source}</source><br />
</configuration><br />
</plugin><br />
...<br />
</syntaxhighlight><br />
<br />
'''All the gCube components inherit this default configuration from <code>maven-parent</code> and it is not recommended to override it in the component's pom.xml'''.<br />
<br />
If it is really needed to override these values at component's pom.xml level, instead of specify a new configuration for the maven-compiler-plugin, the value of the property should be re-defined:<br />
<br />
<syntaxhighlight lang="xml" highlight="3"><br />
...<br />
<properties><br />
<maven.compiler.source>1.7</maven.compiler.source><br />
...<br />
</properties><br />
<br />
</syntaxhighlight><br />
<br />
<br />
This configuration is flexible enough to allow to customize the Java compiler source and target values depending on different factors in the integration builds (e.g. during ETICS builds gCore components are built with Java 7, all the others with Java 8).<br />
<br />
<br />
<br />
== Artifact Types ==<br />
<br />
There are four gCube-specific artifact types that are generated during the build of the gCube components. Each of them is summarized in the following subsections.<br />
<br />
=== Source Package ===<br />
The source package is an artifact with classifier <code>src</code> that contains the source code of the component plus the files in the distro directory (see [[Maven distro directory layout]] for more).<br />
<br />
Its descriptor is at [http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/build-utils/shared-assemblies/src/main/resources/assemblies/gcube-source-package.xml link].<br />
<br />
It is automatically generated if the <code>generateDistribution</code> property is ''true''.<br />
<br />
=== Uberjar ===<br />
<br />
The Uberjar is an artifact with classifier <code>uberjar</code> that contains the component classes plus all the jar files of the dependencies. This is not mandatory for all components, but only for those that needs to be deployed along with their dependenices.<br />
<br />
Its descriptor is at [http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/build-utils/shared-assemblies/src/main/resources/assemblies/gcube-uberjar.xml link].<br />
<br />
The generation of this package is '''not enabled by default'''. To enable it, bind its configuration to a build phase in the component pom.xml:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<executions><br />
<execution><br />
<id>make-uberjar</id><br />
<phase>package</phase><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
=== Service Archive (SA) ===<br />
Whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary <code>tar.gz</code> artefact with classifier <code>servicearchive</code>. Note that the SA of Maven components does ''not'' need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);<br />
<br />
* for each deployable artefact they produce (primary or secondary), they must include the Maven coordinates of the arteact in the corresponding package section of the gCube Software Profile. Specifying package dependencies in the Software Profile, on the other hand, is no longer required;<br />
<br />
* whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary <code>tar.gz</code> artefact with classifier <code>servicearchive</code>. Note that the SA of Maven components does ''not'' need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);<br />
<br />
<br />
The '''maven-parent''' defines a default descriptor to build a servicearchive ([http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/build-utils/shared-assemblies/src/main/resources/assemblies/gcube-default-servicearchive.xml link]). This descriptor fits most of single module gCube components that meet the requirements in [[Maven distro directory layout]]. If it does not fit your component, you will have to provide a custom descriptor.xml file and a custom configuration as described [[Maven_distro_directory_layout#descriptor.xml|here]].<br />
<br />
<br />
The generation of the default servicearchvie is disabled by default (beacuse it is not fits all cases, please see above). To enable it, bind its configuration to a build phase in the component pom.xml:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<executions><br />
<execution><br />
<id>make-servicearchive</id><br />
<phase>package</phase><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
=== Javadoc ===<br />
It is the default artifact (with classifier <code>javadoc</code>) generated by the <code>maven-javadoc-plugin</code>. It is automatically generated for all components.<br />
<br />
= Libraries =<br />
The <code>streams</code> library is a Maven component in gCube class <code>DataAccess</code>. We will use it as example of a library component in Maven.<br />
<br />
=== Profile ===<br />
<br />
The gCube Software profile of the library is as follows:<br />
<br />
<source lang="xml" highlight="14-18"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>Embedded Domain-Specific Language for Stream Transformations</Description><br />
<Class>DataAccess</Class><br />
<Name>streams</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>streams</Name><br />
<Version>1.0.0-SNAPSHOT</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>streams</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<Files><br />
<File>streams-1.0.0-SNAPSHOT.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the profile includes a single package and this package corresponds to the main build artefact of the component; <br />
* the whole profile and the package have usual gCube coordinates;<br />
* the package name is aligned with the Maven <code>artifactId</code> of the component;<br />
* the package version is aligned with the Maven <code>version</code> of the component;<br />
* the package includes <code>MavenCoordinates</code> that can be directly copied and pasted from the POM;<br />
* the package does '''not''' specify dependencies;<br />
* the package points to the main artefact of the Maven build;<br />
<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>streams</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>DataAccess</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<Files><br />
<File>${build.finalName}.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
=== Service Archive ===<br />
<br />
<code>streams</code> is responsible for generating its own SA as part of its Maven build. This requires dedicated logic in the POM but has the advantage that:<br />
<br />
* SA validity can be verified locally;<br />
* no SA configurations are required in ETICS;<br />
* the required build logic can be easily reused across components;<br />
<br />
<code>streams</code> uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation in the profile and other static files. The approach is best illustrated with a reference to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/streams sources]] of the <code>streams</code> library.<br />
<br />
<br />
As we discuss [[#Creating the ETICS Configurations|below]], <code>streams</code> has the final requirement to produce an interpolated profile during ETICS builds, outside the context of the SA.<br />
<code>streams</code> meets this requirement with the [http://maven.apache.org/plugins/maven-resources-plugin/ Maven Resources Plugin]. Again, consult the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/streams sources]] of the <code>streams</code> library to reuse this approach.<br />
<br />
= Services =<br />
The <code>tree-manager</code> service is a Maven component in gCube class <code>DataAccess</code>. <br />
<br />
== Project Structure ==<br />
<br />
Differently from a library, the service is associated with a number of distinguished build artefacts:<br />
<br />
* the service implementation (a Jar);<br />
* the stubs automatically generated from the WSDL interfaces (a Jar);<br />
* the archive required for deployment in gCore (a Gar);<br />
* the archive required for registration in the system (a SA);<br />
<br />
Producing these artefacts requires sharing a number of static and dynamic files (from profiles to WSDLs to pre-processed WSDLs). The usual Maven approach of mapping the artefacts onto different Maven projects is then genuinely difficult. It is instead more practical to use a single multi-module Maven project. The <code>tree-manager</code> uses this module structure:<br />
<br />
<pre><br />
tree-manager (parent module)------- (produces) --- (POM only)<br />
|<br />
|----- tree-manager-service (service impl module)------- (produces) --- jar, gar, SA<br />
| |<br />
| | (depends on)<br />
| |<br />
| \/<br />
|----- tree-manager-stubs (service stubs module)------- (produces) --- jar<br />
</pre><br />
<br />
As customary, the parent does not produce any artefact (is a POM-only project). It declares these coordinates in its POM:<br />
<br />
<source lang="xml"><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
<packaging>pom</packaging><br />
</source><br />
<br />
The parent physically includes the sub-modules;<br />
<br />
<source lang="xml"><br />
<modules><br />
<module>tree-manager-stubs</module><br />
<module>tree-manager-service</module><br />
</modules><br />
</source> <br />
<br />
Finally, the parent inherits from '''maven-parent''':<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
The submodules inherit from the parent:<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>tree-manager</artifactId><br />
<groupId>org.gcube.data.access</groupId><br />
<version>1.0.0-SNAPSHOT</version><br />
<relativePath>..</relativePath><br />
</parent><br />
</source><br />
<br />
and share its <code>version</code> and <code>groupId</code>. Their coordinates are then minimal:<br />
<br />
<source lang="xml"><br />
<artifactId>tree-manager-service</artifactId><br />
</source><br />
<br />
<source lang="xml"><br />
<artifactId>tree-manager-stubs</artifactId><br />
</source><br />
<br />
The <code>tree-manager-service</code> module produces a Jar of its own implementation, as well the GAR and the SA of the whole services as secondary artefacts with classifiers of, respectively, <code>gar</code> and <code>service archive</code>.<br />
<br />
The <code>tree-manager-stubs</code> module produces a Jar of its own implementation.<br />
<br />
Finally, the parent module coordinates the build of its sub-modules and centralises the declaration of common dependencies and configuration. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] to reuse this approach.<br />
<br />
== Generating Stubs and Gars ==<br />
<br />
The <code>tree-manager-stubs</code> uses the '''gCube Service Plugin''' to generate its classes from WSDL interfaces. <br />
<br />
Similarly, the <code>tree-manager-service</code> uses the plugin to generate its secondary GAR artefact. <br />
<br />
The configuration of the plugin is conveniently shared in the parent module.<br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] of the <code>streams</code> for details.<br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the service is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>DataAccess</Class><br />
<Name>tree-manager-service</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Main><br />
<Name>tree-manager-service</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager-service</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<GARArchive>tree-manager-service-1.0.0-SNAPSHOT.gar</GARArchive><br />
<PortType><br />
<Name>gcube/data/tm/binder</Name><br />
</PortType><br />
<PortType><br />
<Name>gcube/data/tm/reader</Name><br />
</PortType><br />
<PortType><br />
<Name>gcube/data/tm/writer</Name><br />
</PortType><br />
</Main><br />
<Software><br />
<Name>tree-manager-stubs</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager-stubs</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<Type>library</Type><br />
<Files><br />
<File>tree-manager-stubs-1.0.0-SNAPSHOT.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the profile includes two packages and this package corresponds to the main build artefacts of the submodules; <br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code>s of the submodules;<br />
* package versions are aligned with the Maven <code>version</code>s of the submodules;<br />
* package include <code>MavenCoordinates</code>s that can be directly copied and pasted from the POM of the submodules;<br />
* packages do '''not''' specify dependencies;<br />
* packages points to the main artefact of the Maven build of the submodules;<br />
<br />
<strike>Notice that, unlike libraries, services cannot easily use Maven variables in their profiles (as these artefacts are used for different purposes by different modules). Accordingly, developers needs to maintain versions used in profiles across releases. In particular, they need to:<br />
<br />
* replace SNAPSHOT versions with release versions in release branches;<br />
* advance SNAPSHOT versions in HEAD;</strike><br />
<br />
=== Profile Interpolation ===<br />
The interpolation of profiles is required in order to keep information in profile in sync with <code>pom.xml</code>, in particular versions and filenames (that contain versions) that change at each release.<br />
<br />
In order to generate <code>profile.xml</code> during build process you need to:<br />
# add in the service's parent <code>pom.xml</code> the invocation to maven-resources-plugin<br />
# place a ''template'' <code>profile.xml</code> in the <code>${distroDirectory}</code><br />
<br />
Invocation to <code>maven-resources-plugin</code> looks like the following (to be placed in the <plugins> section inside the <build> section of the pom):<br />
<source lang="xml"><br />
<!-- interpolates profiles and copies from distribution location to configuration <br />
location, where it is need for embedding into stub artifact, gar generation, <br />
and service archive. --><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-resources-plugin</artifactId><br />
<version>2.6</version><br />
<executions><br />
<execution><br />
<id>copy-profile</id><br />
<goals><br />
<goal>copy-resources</goal><br />
</goals><br />
<phase>process-resources</phase><br />
<configuration><br />
<outputDirectory>${configDirectory}</outputDirectory><br />
<resources><br />
<resource><br />
<directory>${distroDirectory}</directory><br />
<includes><br />
<include>profile.xml</include><br />
</includes><br />
<filtering>true</filtering><br />
</resource><br />
</resources><br />
</configuration><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
The ''template'' <code>profile.xml</code> looks like the following:<br />
<source lang="xml" highlight="12,16,18,22,26,30"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>TestClass</Class><br />
<Name>TestService</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Main><br />
<Name>TestService</Name><br />
<Version>${project.version}</Version><br />
<MavenCoordinates><br />
<groupId>org.example</groupId><br />
<artifactId>test-service</artifactId><br />
<version>${project.version}</version><br />
</MavenCoordinates><br />
<GARArchive>>test-service-${project.version}.gar</GARArchive><br />
</Main><br />
<Software><br />
<Name>test-stubs</Name><br />
<Version>${project.version}</Version><br />
<MavenCoordinates><br />
<groupId>org.example</groupId><br />
<artifactId>test-stubs</artifactId><br />
<version>${project.version}</version><br />
</MavenCoordinates><br />
<Type>library</Type><br />
<Files><br />
<File>test-stubs-${project.version}.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
In the example, property <code>${project.version}</code> will be substituted with the actual service's version at build-time. It is also possible, but not required, to parametrize also other values of <code>profile.xml</code> such as maven's coordinates and package names. You can see a more complex <code>profile.xml</code> ''template'' in the <code>tree-manager</code> project: http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-access/tree-manager/distro/profile.xml<br />
<br />
Maven will copy the final <code>profile.xml</code> in <code>${configDirectory}</code> during the <code>process-resources</code> phase of the build. From there it will be copied in the service-archive package.<br />
<br />
== Service Archive ==<br />
<br />
<code>tree-manager-service</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the module and its parent.<br />
<br />
As shown for [[#Libraries|libraries]], the service uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] for details.<br />
<br />
= Portlets =<br />
<br />
The <code>sample-portlet</code> portlet is a Maven based sample portlet based on GWT. Refer to the [http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources] for details.<br />
<br />
== Project Structure ==<br />
<br />
As for a library project, the portlet is associated with 2 distinguished build artefacts:<br />
<br />
* the archive required for deployment in a Liferay Container (a WAR);<br />
* the archive required for registration in the system (a SA);<br />
<br />
In order to generate automatically the project structure and the base pom, we start by customizing the maven archetype liferay-portlet-archetype v. 6.0.6.<br />
<br />
The generation of the project structure and the pom can be performed either from Eclipse ( by using the m2eclipse plugin) or by command line using the Maven SDK.<br />
<br />
On Eclipse the following steps must be followed:<br />
<br />
* FIle -> New -> Project<br />
* From the wizard select Maven->Maven Project<br />
* From the Select Archetype step, Select 'All Catalogs' from the Catalogs combobox and type 'portlet' on the Filter field.<br />
* Select the '''liferay-portlet-archetype''' version 6.0.6 archetype ( if does not appear on the list just unselect 'show the last version of the Archetype only")<br />
* In the next step you have to provide Maven coordinates for your project<br />
* groupid : <your group id> e.g org.gcube.portlets.admin<br />
* artifactId : <your artifactId> e.g. sample-admin-portlet<br />
* version : <your project version> e.g. 1.0.0-SNAPSHOT ( use always SNAPSHOT versions for development versions of a project)<br />
* package: <your portlet code package> e.g. org.gcube.portlets.admin.sample<br />
<br />
while from shell the following mvn command should be issued :<br />
<br />
<pre><br />
mvn archetype:generate \<br />
-DarchetypeArtifactId=liferay-theme-archetype \<br />
-DarchetypeGroupId=com.liferay.maven.archetypes \<br />
-DarchetypeVersion=6.0.6 \<br />
-DartifactId=<your artifactId> \<br />
-DgroupId=<your group id> \<br />
-Dversion=1.0.0-SNAPSHOT<br />
</pre><br />
<br />
Both approaches generate a project structure similar to the picture below:<br />
<br />
[[image:sample-portlet-project.png|Sample portlet project structure]]<br />
<br />
In order to customize the build procedure to the gCube needs we should open the pom.xml and apply the following changes:<br />
<br />
Configure the inheritance from '''maven-parent''' :<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
Configure some project properties:<br />
<br />
<source lang="xml"><br />
<properties><br />
<liferay.version>6.0.6</liferay.version><br />
<liferay.auto.deploy.dir>${system.CATALINA_HOME}/../deploy</liferay.auto.deploy.dir><br />
<distroDirectory>${project.basedir}/distro</distroDirectory><br />
<configDirectory>${project.basedir}/config</configDirectory><br />
</properties><br />
</source><br />
<br />
Add the plugin for the generation of the SA, which has to be included under the build plugins section:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<version>2.2</version><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
In the case of a GWT portlet the [http://mojo.codehaus.org/gwt-maven-plugin gwt-maven-plugin] should also be configured ( within build plugins section as well), by also selecting the proper gwt version ( in our example 2.4)<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.codehaus.mojo</groupId><br />
<artifactId>gwt-maven-plugin</artifactId><br />
<version>${gwt.version}</version><br />
<executions><br />
<execution><br />
<goals><br />
<goal>compile</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
The proper GWT version should be added to the pom properties :<br />
<br />
<source lang="xml"><br />
<gwt.version>2.4.0</gwt.version><br />
</source><br />
<br />
and GWT dependencies should be also configured:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-user</artifactId><br />
<version>${gwt.version}</version><br />
</dependency><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-servlet</artifactId><br />
<version>${gwt.version}</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet/ project sources]] for further details on the project configuration.<br />
<br />
== Generating Artifacts ==<br />
<br />
The <code>sample-portlet</code> uses the '''maven-compiler-plugin''' to compile the portlet code and the '''gwt-maven-plugin''' as well for GWT compilation.<br />
<br />
The generation of a war package can be triggered by typing :<br />
<br />
<pre><br />
mvn package <br />
</pre><br />
<br />
on your shell or running this build task within Eclipse.<br />
<br />
In both cases the war package will be generated under the ''target'' folder, e.g.:<br />
<br />
<pre><br />
target/sample-admin-portlet-1.0.0-SNAPSHOT.war<br />
</pre><br />
<br />
The generation of all project artifacts ( war, source jar, javadoc, SA) can be triggered by the build task :<br />
<br />
<pre><br />
mvn install <br />
</pre><br />
<br />
this will install as well all artifacts in your local repository, while the deployment of the artifacts to the remote gCube Maven repos can triggered by the build task:<br />
<br />
<pre><br />
mvn deploy <br />
</pre><br />
<br />
== Portlet dependencies ==<br />
<br />
Specific portlet dependencies ( e.g. GWT, service stubs ) should be packaged within the portlet war. All dependencies specified within the pom file without the ''provided'' scope will be copied inside the WEB-INF/lib folder at packaging time.<br />
<br />
For this reason all portlet specific dependencies should not specified that scope e.g:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.externals</groupId><br />
<artifactId>gwt-ext-patched</artifactId><br />
<version>2.0.5</version><br />
</dependency><br />
</source><br />
<br />
On the other hand the system dependencies ( the one contained within the [[GCube_Portal_Installation#Portal_Bundle_Libraries|Portal bundle]] ) should not be included inside the portlet war. As said in this case the ''provided'' scope should be specified, e.g.<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portal</groupId><br />
<artifactId>custom-portal-handler</artifactId><br />
<version>1.1.0-SNAPSHOT</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.4.0,1.5.0]</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
<br />
A subset of these libraries ( contained in a GHN) that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called <code>ghn-core-runtime</code>. A dependency on this Jar can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ghn-core-runtime</artifactId><br />
<version>1.0.0</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Note: a component that depends on <code>gcf</code> indirectly depends on <code>gun-core-runtime</code>.<br />
<br />
== Include gCube Guided Tour using Maven ==<br />
<br />
This particular component is not yet ported to maven so in order to be included in a maven based project it should be declared as system dependency as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portletsuser</groupId><br />
<artifactId>gcube-guidedtour-widget</artifactId><br />
<version>1.0.0</version><br />
<scope>system</scope><br />
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/gcube-guidedtour-widget.jar</systemPath> <br />
</dependency><br />
</source><br />
<br />
together with the gCube Widget Library:<br />
<br />
<source lang="xml"> <br />
<dependency><br />
<groupId>org.gcube.portlets.user</groupId><br />
<artifactId>gcubewidgets</artifactId><br />
<version>1.2.0</version><br />
<scope>system</scope><br />
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/gCube-Widgets-Library.jar</systemPath><br />
</dependency><br />
</source><br />
<br />
both components can be downloaded from the distribution repository [[http://grids16.eng.it:8080/BuildReport/browse/Recent_Builds]]. <br />
<br />
In case the portlet depends on the ''gcf'' component, some dependency resolution changes are required to your portlet in order to properly compile ( because of libraries clashes)<br />
<br />
The ghn-core-runtime component ( gcf dependency) should be excluded:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.5.1-SNAPSHOT,2.0.0)</version><br />
<scope>provided</scope><br />
<exclusions><br />
<exclusion><br />
<artifactId>ghn-core-runtime</artifactId><br />
<groupId>org.gcube.distribution</groupId><br />
</exclusion><br />
</exclusions><br />
</dependency><br />
</source><br />
<br />
<br />
and some new dependencies should be added:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>xerces</groupId><br />
<artifactId>xercesImpl</artifactId><br />
<version>2.9.1</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ws-addressing</artifactId><br />
<version>1.0</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>axis-patched</artifactId><br />
<version>1.2RC</version><br />
<scope>provided</scope><br />
</dependency><br />
<br />
</source><br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the portlet is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>PortletAdmin</Class><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.portles.admin</groupId><br />
<artifactId>sample-admin-portlet</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/sample-admin-portlet-1.0.0-SNAPSHOT.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code> of the project;<br />
* package versions are aligned with the Maven <code>version</code> of the project;<br />
* package include <code>MavenCoordinates</code> that can be directly copied and pasted from the POM of the project;<br />
* packages do '''not''' specify dependencies;<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>sample-portlet</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>PortletAdmin</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/${build.finalName}.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
== Service Archive ==<br />
<br />
<code>sample-portlet</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the project.<br />
<br />
As shown for [[#Libraries|libraries]], the portlet uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources]] for details.<br />
<br />
== Use Case: Create a new Mavenized GWT Portlet ==<br />
<br />
In order to create a new Mavenized GWT Portlet this detailed guide can be also followed [https://gcube.wiki.gcube-system.org/gcube/index.php/Create_a_new_Mavenized_gCube_GWT_Portlet_Guide]<br />
<br />
= Widgets =<br />
<br />
Falls in this category of components mainly GWT widgets or so called GWT Libraries, which can be used to package classes for mutualization and/or modularization. A Widget is a java archive (JAR) containing both classes and java sources for later GWT compilation and a gwt.xml module descriptor. <br />
<br />
In this section we refer to a sample Widget with maven structure (https://svn.research-infrastructures.eu/d4science/gcube/trunk/portal/mvn-widget-sample/)<br />
<br />
== Project Structure ==<br />
<br />
As for a other project, the portlet is associated with 2 distinguished build artefacts:<br />
<br />
* the archive required for deployment (a JAR);<br />
* the archive required for registration in the system (a SA);<br />
<br />
In order to generate automatically the project structure and the base pom, we start by customizing the maven archetype liferay-portlet-archetype v. 6.0.6.<br />
<br />
The generation of the project structure and the pom can be performed either from Eclipse ( by using the m2eclipse plugin) or by command line using the Maven SDK.<br />
<br />
On Eclipse the following steps must be followed:<br />
<br />
* FIle -> New -> Project<br />
* From the wizard select Maven->Maven Project<br />
* From the Select Archetype step, Select 'All Catalogs' from the Catalogs combobox and type 'portlet' on the Filter field.<br />
* Select the '''maven-archetype-quickstart''' version 1.1.0 archetype ( if does not appear on the list just unselect 'show the last version of the Archetype only")<br />
* In the next step you have to provide Maven coordinates for your project<br />
* groupid : <your group id> e.g org.gcube.portlets.admin<br />
* artifactId : <your artifactId> e.g. sample-admin-portlet<br />
* version : <your project version> e.g. 1.0.0-SNAPSHOT ( use always SNAPSHOT versions for development versions of a project)<br />
* package: <your portlet code package> e.g. org.gcube.portlets.admin.sample<br />
<br />
while from shell the following mvn command should be issued :<br />
<br />
<pre><br />
mvn archetype:generate \<br />
-DarchetypeArtifactId=maven-archetype-quickstart \<br />
-DarchetypeGroupId=com.liferay.maven.archetypes \<br />
-DarchetypeVersion=1.1.0 \<br />
-DartifactId=<your artifactId> \<br />
-DgroupId=<your group id> \<br />
-Dversion=1.0.0-SNAPSHOT<br />
</pre><br />
<br />
Both approaches generate a project structure similar to the picture below:<br />
<br />
[[image:sample-widget-project.png|Sample widget project structure]]<br />
<br />
In order to customize the build procedure to the gCube needs we should open the pom.xml and apply the following changes:<br />
<br />
Configure the inheritance from '''maven-parent''' :<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
Configure some project properties:<br />
<br />
<source lang="xml"><br />
<properties><br />
<distroDirectory>${project.basedir}/distro</distroDirectory><br />
<configDirectory>${project.basedir}/config</configDirectory><br />
</properties><br />
</source><br />
<br />
Add the plugin for the generation of the SA, which has to be included under the build plugins section:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<version>2.2</version><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
In the case of a GWT widget the [http://mojo.codehaus.org/gwt-maven-plugin gwt-maven-plugin] should also be configured ( within build plugins section as well), by also selecting the proper gwt version ( in our example 2.4)<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.codehaus.mojo</groupId><br />
<artifactId>gwt-maven-plugin</artifactId><br />
<version>${gwt.version}</version><br />
<executions><br />
<execution><br />
<goals><br />
<goal>compile</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
The proper GWT version should be added to the pom properties :<br />
<br />
<source lang="xml"><br />
<gwt.version>2.4.0</gwt.version><br />
</source><br />
<br />
and GWT dependencies should be also configured:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-user</artifactId><br />
<version>${gwt.version}</version><br />
</dependency><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-servlet</artifactId><br />
<version>${gwt.version}</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/mvn-widget-sample/ project sources]] for further details on the project configuration.<br />
<br />
== Generating Artifacts ==<br />
<br />
The <code>sample-portlet</code> uses the '''maven-compiler-plugin''' to compile the portlet code and the '''gwt-maven-plugin''' as well for GWT compilation.<br />
<br />
The generation of a war package can be triggered by typing :<br />
<br />
<pre><br />
mvn package <br />
</pre><br />
<br />
on your shell or running this build task within Eclipse.<br />
<br />
In both cases the war package will be generated under the ''target'' folder, e.g.:<br />
<br />
<pre><br />
target/sample-admin-portlet-1.0.0-SNAPSHOT.war<br />
</pre><br />
<br />
The generation of all project artifacts ( war, source jar, javadoc, SA) can be triggered by the build task :<br />
<br />
<pre><br />
mvn install <br />
</pre><br />
<br />
this will install as well all artifacts in your local repository, while the deployment of the artifacts to the remote gCube Maven repos can triggered by the build task:<br />
<br />
<pre><br />
mvn deploy <br />
</pre><br />
<br />
== Widget dependencies ==<br />
<br />
The system dependencies ( the one contained within the [[GCube_Portal_Installation#Portal_Bundle_Libraries|Portal bundle]] ) should not be included inside the portlet war. As said in this case the ''provided'' scope should be specified, e.g.<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portal</groupId><br />
<artifactId>custom-portal-handler</artifactId><br />
<version>1.1.0-SNAPSHOT</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.4.0,1.5.0]</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the widget is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>PortletAdmin</Class><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.portles.admin</groupId><br />
<artifactId>sample-admin-portlet</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/sample-admin-portlet-1.0.0-SNAPSHOT.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code> of the project;<br />
* package versions are aligned with the Maven <code>version</code> of the project;<br />
* package include <code>MavenCoordinates</code> that can be directly copied and pasted from the POM of the project;<br />
* packages do '''not''' specify dependencies;<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>sample-portlet</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>PortletAdmin</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/${build.finalName}.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
== Service Archive ==<br />
<br />
<code>sample-portlet</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the project.<br />
<br />
As shown for [[#Libraries|libraries]], the portlet uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources]] for details.</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Developing_gCube_Maven_Components&diff=29738Developing gCube Maven Components2017-10-30T14:37:38Z<p>Gabriele.giammatteo: /* Uberjar */</p>
<hr />
<div>[[Category: Developer's Guide]]<br />
__TOC__<br />
<br />
= Common =<br />
<br />
== Maven Coordinates ==<br />
Maven components' coordinates must follow these conventions:<br />
<br />
* their <code>groupId</code> must be of the following form:<br />
<br />
<pre>org.gcube.<class></pre><br />
<br />
: where ''class'' identifies a functionally correlated set of components and helps a range of users (developers, administrator) to discover components in various UIs/reports used within the system and in tools. Logically, the class corresponds to the gCube class of the component's, as specified in its gCube Software Profile. However, more specific, or else cross-class groups may be agreed upon among developers. In all cases, the group should align with official Maven [http://maven.apache.org/guides/mini/guide-naming-conventions.html guidelines]. By agreement, for example, components in gCube class <code>DataAccess</code> declare a <code>groupId</code> of <code>org.gcube.data.access</code> and those in gCube class <code>DataTransfer</code> declare a <code>groupId</code> of <code>org.gcube.data.transfer</code>. On the other hand, also by agreement, components in gCube class <code>InformationSystem</code> declare a <code>groupId</code> of <code>org.gcube.informationsystem</code>.<br />
<br />
* their <code>artifactId</code> must follow Maven conventions (e.g. ''avoid capital letters or punctuation other than hyphens'').<br />
<br />
== Parent ==<br />
All gCube components must inherit from '''maven-parent'''. This ensure and promotes compliance with system-wide requirements, from the enforcement of minimal Java and Maven versions to generation and packaging of Javadoc documentation and component sources.<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
<br />
The maven-parent defines:<br />
* several system-wide properties like <code>gcube.license</code>, <code>gcube.wikiRoot</code>, <code>distroDirectory</code><br />
* the default Java version (see more in [[Developing_gCube_Maven_Components#Java_Version|Java Version]])<br />
* the execution of some plugins that run for every component like: <code>maven-javadoc-plugin</code>, <code>maven-source-plugin</code><br />
* the configuration of the <code>maven-assembly-plugin</code> to generate some gCube-specific packages: <code>servicearchive</code>, <code>uberjar</code> and <code>source-package</code> (see more in [[Developing_gCube_Maven_Components#Artifact_Types|Artifact Types]])<br />
* the logic to enable the generation of the source package<br />
<br />
The '''maven-parent''' is at http://maven.research-infrastructures.eu/nexus/service/local/repositories/gcube-externals/content/org/gcube/tools/maven-parent/1.0.0/maven-parent-1.0.0.pom<br />
<br />
== The distro folder ==<br />
This folder contains files that will be distributed along with the code in the distribution packages. Further details are in the [[Maven distro directory layout]] page.<br />
<br />
<br />
== Dependencies ==<br />
Service and libraries that depend on other gCube components (Maven-based or Ant-based) can find them in the gCube Maven Repositories: <code>gcube-snapshots</code> <strike>and <code>gcube-releases</code></strike>. <br />
<br />
=== gCube Maven Repositories ===<br />
As a one-off task, developers must configure these repositories in the <code>~/.m2/settings.xml</code> file of their local Maven installation, as follows:<br />
<br />
<source lang="xml"><br />
<settings xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"<br />
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd"><br />
<br />
<servers><br />
<server><br />
<id>gcube-snapshots</id><br />
<username>gcube-user</username><br />
<password>maven</password><br />
</server><br />
</servers><br />
<br />
<br />
<profiles><br />
<profile><br />
<id>gcube</id><br />
<repositories><br />
<repository><br />
<id>gcube-snapshots</id><br />
<name>gCube Snapshots</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
</snapshots><br />
</repository><br />
<repository><br />
<id>gcube-externals</id><br />
<name>gCube Externals</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals</url><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
</repository><br />
</repositories><br />
<br />
<pluginRepositories><br />
<pluginRepository><br />
<id>gcube-snapshots</id><br />
<name>gCube Snapshots</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
</snapshots><br />
</pluginRepository><br />
<pluginRepository><br />
<id>gcube-externals</id><br />
<name>gCube Externals</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals</url><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
</pluginRepository><br />
</pluginRepositories><br />
<br />
</profile><br />
</profiles><br />
<br />
<activeProfiles><br />
<activeProfile>gcube</activeProfile><br />
</activeProfiles><br />
</settings><br />
</source><br />
<br />
<br />
'''Note:Developers must not configure these repositories in their POMs''', only in <code>settings.xml</code>. <br />
<br />
This is to:<br />
* keep their projects independent from future changes in repository locations; <br />
* avoid interference with ETICS builds, where access to these repositories may comprise correct system integration;<br />
<br />
With this settings.xml, developers will have access on their local development environment to development versions (gcube-snapshots) and externals (gcube-externals). This is the correct behaviour for most of use cases. If access to staging or releases repositories is necessary (e.g. patching a component already released in the branch where dependencies versions in the release are not in snapshots), developer should modify his settings.xml to point to gcube-staging(releases) instead of gcube-snapshots.<br />
<br />
=== Third-party Repositories ===<br />
Using the the <code>settings.xml</code> aforementioned, any dependency declared in pom.xml that can be found in one of gCube repositories or in Maven Central will be resolved at build time and deployment time without any further configuration. Frequently it may happens that a gCube component needs software stored on a public Maven repository not in on of those repositories. In this case, Maven provides an easy solution for declaring additional repositories for dependency resolution by adding a <repositories> section in the pom.xml of the project. E.g.:<br />
<br />
<source lang="xml" highlight="3,4"><br />
<repositories><br />
<repository><br />
<id>[repository-id]</id><br />
<url>[repository URL]</url><br />
</repository><br />
</repositories><br />
</source><br />
<br />
Though technically possible to add any Maven repository in this way, in order to preserve the quality of gCube software it is required to check trustworthiness of its dependencies disallowing usage of certain external software. To enforce such a control over public Maven repositories, '''gCube components are allowed to use external repository not directly but only through a proxy repository''' (a.k.a. shadow repository) created on CNR's Nexus Repository Portal (http://maven.research-infrastructures.eu/nexus/).<br />
<br />
List of all repositories in Nexus Portal is available at http://maven.research-infrastructures.eu/nexus/index.html#view-repositories<br />
<br />
'''Note''': if a component uses a public Maven repository directly, it will work for building but would fail in deployment phase. In fact, Software Gateway is only able to download software accessible from Nexus.<br />
<br />
==== Request new proxy repository in Nexus ====<br />
Proxy Repositories on Nexus are created by the Release Manager. To request a new proxy a new ticket must be created.<br />
<br />
Details of ticket are:<br />
# open a ticket in tracker "Support" in gCube ticketing system (at https://support.d4science.org/projects/gcube/)<br />
# ticket's '''description''' must report:<br />
#* URL of repository to add<br />
#* organization which hosts the repository<br />
#* which artifacts are needed from that repository<br />
#* which components needs those artifacts<br />
# '''Sprint''' field should be set to the gCube release in which the component is expected to be released <br />
# '''Category''' field must report the gCube subsystem that needs the external repository<br />
# '''Assignee''' is the Release Manager<br />
<br />
An example of ticket for requesting a new repository mirror is reported in the figure below.<br />
[[Image:New_maven_mirror_repo.png|600px|center|Ticket for requesting a new Mirror Repository]]<br />
<br />
Release Manager will process the ticket checking that no issues are foreseen from the usage of software stored in the repository. If request is accepted, a new proxy repository is created and the repository's URL will be communicated publicly on gCube's developers mailinglist (a.k.a. TCOM mailinglist).<br />
<br />
Knowing the mirror repository URL, developer can use it adding following lines in pom.xml. E.g.<br />
<br />
<source lang="xml" highlight="3,4"><br />
<repositories><br />
<repository><br />
<id>gcube-ext-osgeo</id><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/osgeo/</url><br />
</repository><br />
</repositories><br />
</source><br />
<br />
'''Note''': it is possible that software stored on proxy repositories is not included in search results on Nexus. This happens for those repositories that does not have indexes (or have indexes in a format that Nexus is not able to download). Anyway, once Nexus has downloaded artifacts from the original repository for the first time (e.g. because a compilation of a pom that uses the proxy repository), creates its own indexes and those artifacts will become available also for local search.<br />
<br />
=== Dependency Versions ===<br />
<br />
ETICS builds ensure correct integration of gCube components, regardless of the versions their dependencies. It also ensures that components are released with fixed versions for their dependencies, i.e. ensures build reproducibility.<br />
<br />
Accordingly, developers can be liberal in the choice of dependency versions. i.e. rely on version ranges to:<br />
<br />
* minimise the requirements for version management;<br />
* be readily informed of any mis-alignment with their dependencies;<br />
<br />
As we start adopting Maven, the lower-bound of version ranges will be a SNAPSHOT version. For example, a dependency on the <code>streams</code> library can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>streams</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
</dependency><br />
</source><br />
<br />
With this dependency, a component will be soon notified of a non-retro compatible change that, contrary to conventions, <code>streams</code> may have introduced in, say, version <code>1.3.0</code> (typically the day after at the latest). The notification will be delivered in the IDE, assuming the availability of Maven integration (e.g. <code>m2e</code> in Eclipse). <br />
The component is in fact free to open the range further if it wants to be aligned with its dependencies across major versions.<br />
<br />
=== Common Dependencies ===<br />
<br />
Many gCube components depend on <code>gcf</code>, which is now available as a Maven component since version <code>1.4</code>. At the time of writing, the component has not been released yet. A dependency on it is then specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
The scope <code>provided</code> indicates that these dependency will be satisfied in the development environment (the gHN) and should not be included in GAR and SA archives.<br />
<br />
Some components depend on the libraries available in all gHNs, though they do not depend on <code>gcf</code> itself. The subset of these libraries that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called <code>ghn-core-runtime</code>. A dependency on this Jar can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ghn-core-runtime</artifactId><br />
<version>1.0.0</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Note: a component that depends on <code>gcf</code> indirectly depends on <code>gun-core-runtime</code>.<br />
<br />
Another common dependency is on the gCube gRS2, an Ant component that the build infrastructure make available in Maven repositories. A dependency on this component can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.execution</groupId><br />
<artifactId>grs2library</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
</dependency><br />
</source><br />
<br />
== Java Version ==<br />
<br />
In Maven the Java version of the source code and of the generated classes can be controlled using the configuration of the <code>maven-compiler-plugin</code>. E.g.:<br />
<syntaxhighlight lang="xml" highlight="5,6"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-compiler-plugin</artifactId><br />
<configuration><br />
<source>1.7</source><br />
<target>1.8</target><br />
</configuration><br />
</plugin><br />
</syntaxhighlight><br />
<br />
<br />
In gCube, this mechanism is exploited to enforce the usage of the same Java version for all components. In particular, the maven-parent defines the properties <code>java_version</code>, <code>maven.compiler.source</code> and <code>maven.compiler.target</code> and use these properties in the configuration of the <code>maven-compiler-plugin</code>.<br />
<syntaxhighlight lang="xml" highlight="15,16"><br />
...<br />
<properties><br />
<java_version>1.8</java_version><br />
<maven.compiler.source>${java_version}</maven.compiler.source><br />
<maven.compiler.target>${java_version}</maven.compiler.target><br />
...<br />
</properties><br />
<br />
...<br />
<br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-compiler-plugin</artifactId><br />
<configuration><br />
<target>${maven.compiler.target}</target><br />
<source>${maven.compiler.source}</source><br />
</configuration><br />
</plugin><br />
...<br />
</syntaxhighlight><br />
<br />
'''All the gCube components inherit this default configuration from <code>maven-parent</code> and it is not recommended to override it in the component's pom.xml'''.<br />
<br />
If it is really needed to override these values at component's pom.xml level, instead of specify a new configuration for the maven-compiler-plugin, the value of the property should be re-defined:<br />
<br />
<syntaxhighlight lang="xml" highlight="3"><br />
...<br />
<properties><br />
<maven.compiler.source>1.7</maven.compiler.source><br />
...<br />
</properties><br />
<br />
</syntaxhighlight><br />
<br />
<br />
This configuration is flexible enough to allow to customize the Java compiler source and target values depending on different factors in the integration builds (e.g. during ETICS builds gCore components are built with Java 7, all the others with Java 8).<br />
<br />
<br />
<br />
== Artifact Types ==<br />
<br />
There are four gCube-specific artifact types that are generated during the build of the gCube components. Each of them is summarized in the following subsections.<br />
<br />
=== Source Package ===<br />
The source package is an artifact with classifier <code>src</code> that contains the source code of the component plus the files in the distro directory (see [[Maven distro directory layout]] for more).<br />
<br />
Its descriptor is at [http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/build-utils/shared-assemblies/src/main/resources/assemblies/gcube-source-package.xml link].<br />
<br />
It is automatically generated if the <code>generateDistribution</code> property is ''true''.<br />
<br />
=== Uberjar ===<br />
<br />
The Uberjar is an artifact with classifier <code>uberjar</code> that contains the component classes plus all the jar files of the dependencies. This is not mandatory for all components, but only for those that needs to be deployed along with their dependenices.<br />
<br />
Its descriptor is at [http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/build-utils/shared-assemblies/src/main/resources/assemblies/gcube-uberjar.xml link].<br />
<br />
The generation of this package is '''not enabled by default'''. To enable it, bind its configuration to a build phase in the component pom.xml:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<executions><br />
<execution><br />
<id>make-uberjar</id><br />
<phase>package</phase><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
=== Service Archive (SA) ===<br />
Whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary <code>tar.gz</code> artefact with classifier <code>servicearchive</code>. Note that the SA of Maven components does ''not'' need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);<br />
<br />
* for each deployable artefact they produce (primary or secondary), they must include the Maven coordinates of the arteact in the corresponding package section of the gCube Software Profile. Specifying package dependencies in the Software Profile, on the other hand, is no longer required;<br />
<br />
* whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary <code>tar.gz</code> artefact with classifier <code>servicearchive</code>. Note that the SA of Maven components does ''not'' need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);<br />
<br />
<br />
The '''maven-parent''' defines a default descriptor to build a servicearchive ([http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/build-utils/shared-assemblies/src/main/resources/assemblies/gcube-default-servicearchive.xml link]).<br />
<br />
This Please not that, while this fits most of the single module projects that meet the requirements in [[Maven distro directory layout]], for multi-module projects and/or projects that need to include custom files a custom servicearchive might be needed. In the latter case, the default assembly configuration cannot be used and a custom maven-assembly-plugin configuration must be defined in the component's pom.xml.<br />
<br />
The generation of the default servicearchvie is disabled by default. To enable it, bind its configuration to a build phase in the component pom.xml:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<executions><br />
<execution><br />
<id>make-servicearchive</id><br />
<phase>package</phase><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
=== Javadoc ===<br />
It is the default artifact (with classifier <code>javadoc</code>) generated by the <code>maven-javadoc-plugin</code>. It is automatically generated for all components.<br />
<br />
= Libraries =<br />
The <code>streams</code> library is a Maven component in gCube class <code>DataAccess</code>. We will use it as example of a library component in Maven.<br />
<br />
=== Profile ===<br />
<br />
The gCube Software profile of the library is as follows:<br />
<br />
<source lang="xml" highlight="14-18"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>Embedded Domain-Specific Language for Stream Transformations</Description><br />
<Class>DataAccess</Class><br />
<Name>streams</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>streams</Name><br />
<Version>1.0.0-SNAPSHOT</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>streams</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<Files><br />
<File>streams-1.0.0-SNAPSHOT.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the profile includes a single package and this package corresponds to the main build artefact of the component; <br />
* the whole profile and the package have usual gCube coordinates;<br />
* the package name is aligned with the Maven <code>artifactId</code> of the component;<br />
* the package version is aligned with the Maven <code>version</code> of the component;<br />
* the package includes <code>MavenCoordinates</code> that can be directly copied and pasted from the POM;<br />
* the package does '''not''' specify dependencies;<br />
* the package points to the main artefact of the Maven build;<br />
<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>streams</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>DataAccess</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<Files><br />
<File>${build.finalName}.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
=== Service Archive ===<br />
<br />
<code>streams</code> is responsible for generating its own SA as part of its Maven build. This requires dedicated logic in the POM but has the advantage that:<br />
<br />
* SA validity can be verified locally;<br />
* no SA configurations are required in ETICS;<br />
* the required build logic can be easily reused across components;<br />
<br />
<code>streams</code> uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation in the profile and other static files. The approach is best illustrated with a reference to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/streams sources]] of the <code>streams</code> library.<br />
<br />
<br />
As we discuss [[#Creating the ETICS Configurations|below]], <code>streams</code> has the final requirement to produce an interpolated profile during ETICS builds, outside the context of the SA.<br />
<code>streams</code> meets this requirement with the [http://maven.apache.org/plugins/maven-resources-plugin/ Maven Resources Plugin]. Again, consult the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/streams sources]] of the <code>streams</code> library to reuse this approach.<br />
<br />
= Services =<br />
The <code>tree-manager</code> service is a Maven component in gCube class <code>DataAccess</code>. <br />
<br />
== Project Structure ==<br />
<br />
Differently from a library, the service is associated with a number of distinguished build artefacts:<br />
<br />
* the service implementation (a Jar);<br />
* the stubs automatically generated from the WSDL interfaces (a Jar);<br />
* the archive required for deployment in gCore (a Gar);<br />
* the archive required for registration in the system (a SA);<br />
<br />
Producing these artefacts requires sharing a number of static and dynamic files (from profiles to WSDLs to pre-processed WSDLs). The usual Maven approach of mapping the artefacts onto different Maven projects is then genuinely difficult. It is instead more practical to use a single multi-module Maven project. The <code>tree-manager</code> uses this module structure:<br />
<br />
<pre><br />
tree-manager (parent module)------- (produces) --- (POM only)<br />
|<br />
|----- tree-manager-service (service impl module)------- (produces) --- jar, gar, SA<br />
| |<br />
| | (depends on)<br />
| |<br />
| \/<br />
|----- tree-manager-stubs (service stubs module)------- (produces) --- jar<br />
</pre><br />
<br />
As customary, the parent does not produce any artefact (is a POM-only project). It declares these coordinates in its POM:<br />
<br />
<source lang="xml"><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
<packaging>pom</packaging><br />
</source><br />
<br />
The parent physically includes the sub-modules;<br />
<br />
<source lang="xml"><br />
<modules><br />
<module>tree-manager-stubs</module><br />
<module>tree-manager-service</module><br />
</modules><br />
</source> <br />
<br />
Finally, the parent inherits from '''maven-parent''':<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
The submodules inherit from the parent:<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>tree-manager</artifactId><br />
<groupId>org.gcube.data.access</groupId><br />
<version>1.0.0-SNAPSHOT</version><br />
<relativePath>..</relativePath><br />
</parent><br />
</source><br />
<br />
and share its <code>version</code> and <code>groupId</code>. Their coordinates are then minimal:<br />
<br />
<source lang="xml"><br />
<artifactId>tree-manager-service</artifactId><br />
</source><br />
<br />
<source lang="xml"><br />
<artifactId>tree-manager-stubs</artifactId><br />
</source><br />
<br />
The <code>tree-manager-service</code> module produces a Jar of its own implementation, as well the GAR and the SA of the whole services as secondary artefacts with classifiers of, respectively, <code>gar</code> and <code>service archive</code>.<br />
<br />
The <code>tree-manager-stubs</code> module produces a Jar of its own implementation.<br />
<br />
Finally, the parent module coordinates the build of its sub-modules and centralises the declaration of common dependencies and configuration. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] to reuse this approach.<br />
<br />
== Generating Stubs and Gars ==<br />
<br />
The <code>tree-manager-stubs</code> uses the '''gCube Service Plugin''' to generate its classes from WSDL interfaces. <br />
<br />
Similarly, the <code>tree-manager-service</code> uses the plugin to generate its secondary GAR artefact. <br />
<br />
The configuration of the plugin is conveniently shared in the parent module.<br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] of the <code>streams</code> for details.<br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the service is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>DataAccess</Class><br />
<Name>tree-manager-service</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Main><br />
<Name>tree-manager-service</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager-service</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<GARArchive>tree-manager-service-1.0.0-SNAPSHOT.gar</GARArchive><br />
<PortType><br />
<Name>gcube/data/tm/binder</Name><br />
</PortType><br />
<PortType><br />
<Name>gcube/data/tm/reader</Name><br />
</PortType><br />
<PortType><br />
<Name>gcube/data/tm/writer</Name><br />
</PortType><br />
</Main><br />
<Software><br />
<Name>tree-manager-stubs</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager-stubs</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<Type>library</Type><br />
<Files><br />
<File>tree-manager-stubs-1.0.0-SNAPSHOT.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the profile includes two packages and this package corresponds to the main build artefacts of the submodules; <br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code>s of the submodules;<br />
* package versions are aligned with the Maven <code>version</code>s of the submodules;<br />
* package include <code>MavenCoordinates</code>s that can be directly copied and pasted from the POM of the submodules;<br />
* packages do '''not''' specify dependencies;<br />
* packages points to the main artefact of the Maven build of the submodules;<br />
<br />
<strike>Notice that, unlike libraries, services cannot easily use Maven variables in their profiles (as these artefacts are used for different purposes by different modules). Accordingly, developers needs to maintain versions used in profiles across releases. In particular, they need to:<br />
<br />
* replace SNAPSHOT versions with release versions in release branches;<br />
* advance SNAPSHOT versions in HEAD;</strike><br />
<br />
=== Profile Interpolation ===<br />
The interpolation of profiles is required in order to keep information in profile in sync with <code>pom.xml</code>, in particular versions and filenames (that contain versions) that change at each release.<br />
<br />
In order to generate <code>profile.xml</code> during build process you need to:<br />
# add in the service's parent <code>pom.xml</code> the invocation to maven-resources-plugin<br />
# place a ''template'' <code>profile.xml</code> in the <code>${distroDirectory}</code><br />
<br />
Invocation to <code>maven-resources-plugin</code> looks like the following (to be placed in the <plugins> section inside the <build> section of the pom):<br />
<source lang="xml"><br />
<!-- interpolates profiles and copies from distribution location to configuration <br />
location, where it is need for embedding into stub artifact, gar generation, <br />
and service archive. --><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-resources-plugin</artifactId><br />
<version>2.6</version><br />
<executions><br />
<execution><br />
<id>copy-profile</id><br />
<goals><br />
<goal>copy-resources</goal><br />
</goals><br />
<phase>process-resources</phase><br />
<configuration><br />
<outputDirectory>${configDirectory}</outputDirectory><br />
<resources><br />
<resource><br />
<directory>${distroDirectory}</directory><br />
<includes><br />
<include>profile.xml</include><br />
</includes><br />
<filtering>true</filtering><br />
</resource><br />
</resources><br />
</configuration><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
The ''template'' <code>profile.xml</code> looks like the following:<br />
<source lang="xml" highlight="12,16,18,22,26,30"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>TestClass</Class><br />
<Name>TestService</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Main><br />
<Name>TestService</Name><br />
<Version>${project.version}</Version><br />
<MavenCoordinates><br />
<groupId>org.example</groupId><br />
<artifactId>test-service</artifactId><br />
<version>${project.version}</version><br />
</MavenCoordinates><br />
<GARArchive>>test-service-${project.version}.gar</GARArchive><br />
</Main><br />
<Software><br />
<Name>test-stubs</Name><br />
<Version>${project.version}</Version><br />
<MavenCoordinates><br />
<groupId>org.example</groupId><br />
<artifactId>test-stubs</artifactId><br />
<version>${project.version}</version><br />
</MavenCoordinates><br />
<Type>library</Type><br />
<Files><br />
<File>test-stubs-${project.version}.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
In the example, property <code>${project.version}</code> will be substituted with the actual service's version at build-time. It is also possible, but not required, to parametrize also other values of <code>profile.xml</code> such as maven's coordinates and package names. You can see a more complex <code>profile.xml</code> ''template'' in the <code>tree-manager</code> project: http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-access/tree-manager/distro/profile.xml<br />
<br />
Maven will copy the final <code>profile.xml</code> in <code>${configDirectory}</code> during the <code>process-resources</code> phase of the build. From there it will be copied in the service-archive package.<br />
<br />
== Service Archive ==<br />
<br />
<code>tree-manager-service</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the module and its parent.<br />
<br />
As shown for [[#Libraries|libraries]], the service uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] for details.<br />
<br />
= Portlets =<br />
<br />
The <code>sample-portlet</code> portlet is a Maven based sample portlet based on GWT. Refer to the [http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources] for details.<br />
<br />
== Project Structure ==<br />
<br />
As for a library project, the portlet is associated with 2 distinguished build artefacts:<br />
<br />
* the archive required for deployment in a Liferay Container (a WAR);<br />
* the archive required for registration in the system (a SA);<br />
<br />
In order to generate automatically the project structure and the base pom, we start by customizing the maven archetype liferay-portlet-archetype v. 6.0.6.<br />
<br />
The generation of the project structure and the pom can be performed either from Eclipse ( by using the m2eclipse plugin) or by command line using the Maven SDK.<br />
<br />
On Eclipse the following steps must be followed:<br />
<br />
* FIle -> New -> Project<br />
* From the wizard select Maven->Maven Project<br />
* From the Select Archetype step, Select 'All Catalogs' from the Catalogs combobox and type 'portlet' on the Filter field.<br />
* Select the '''liferay-portlet-archetype''' version 6.0.6 archetype ( if does not appear on the list just unselect 'show the last version of the Archetype only")<br />
* In the next step you have to provide Maven coordinates for your project<br />
* groupid : <your group id> e.g org.gcube.portlets.admin<br />
* artifactId : <your artifactId> e.g. sample-admin-portlet<br />
* version : <your project version> e.g. 1.0.0-SNAPSHOT ( use always SNAPSHOT versions for development versions of a project)<br />
* package: <your portlet code package> e.g. org.gcube.portlets.admin.sample<br />
<br />
while from shell the following mvn command should be issued :<br />
<br />
<pre><br />
mvn archetype:generate \<br />
-DarchetypeArtifactId=liferay-theme-archetype \<br />
-DarchetypeGroupId=com.liferay.maven.archetypes \<br />
-DarchetypeVersion=6.0.6 \<br />
-DartifactId=<your artifactId> \<br />
-DgroupId=<your group id> \<br />
-Dversion=1.0.0-SNAPSHOT<br />
</pre><br />
<br />
Both approaches generate a project structure similar to the picture below:<br />
<br />
[[image:sample-portlet-project.png|Sample portlet project structure]]<br />
<br />
In order to customize the build procedure to the gCube needs we should open the pom.xml and apply the following changes:<br />
<br />
Configure the inheritance from '''maven-parent''' :<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
Configure some project properties:<br />
<br />
<source lang="xml"><br />
<properties><br />
<liferay.version>6.0.6</liferay.version><br />
<liferay.auto.deploy.dir>${system.CATALINA_HOME}/../deploy</liferay.auto.deploy.dir><br />
<distroDirectory>${project.basedir}/distro</distroDirectory><br />
<configDirectory>${project.basedir}/config</configDirectory><br />
</properties><br />
</source><br />
<br />
Add the plugin for the generation of the SA, which has to be included under the build plugins section:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<version>2.2</version><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
In the case of a GWT portlet the [http://mojo.codehaus.org/gwt-maven-plugin gwt-maven-plugin] should also be configured ( within build plugins section as well), by also selecting the proper gwt version ( in our example 2.4)<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.codehaus.mojo</groupId><br />
<artifactId>gwt-maven-plugin</artifactId><br />
<version>${gwt.version}</version><br />
<executions><br />
<execution><br />
<goals><br />
<goal>compile</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
The proper GWT version should be added to the pom properties :<br />
<br />
<source lang="xml"><br />
<gwt.version>2.4.0</gwt.version><br />
</source><br />
<br />
and GWT dependencies should be also configured:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-user</artifactId><br />
<version>${gwt.version}</version><br />
</dependency><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-servlet</artifactId><br />
<version>${gwt.version}</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet/ project sources]] for further details on the project configuration.<br />
<br />
== Generating Artifacts ==<br />
<br />
The <code>sample-portlet</code> uses the '''maven-compiler-plugin''' to compile the portlet code and the '''gwt-maven-plugin''' as well for GWT compilation.<br />
<br />
The generation of a war package can be triggered by typing :<br />
<br />
<pre><br />
mvn package <br />
</pre><br />
<br />
on your shell or running this build task within Eclipse.<br />
<br />
In both cases the war package will be generated under the ''target'' folder, e.g.:<br />
<br />
<pre><br />
target/sample-admin-portlet-1.0.0-SNAPSHOT.war<br />
</pre><br />
<br />
The generation of all project artifacts ( war, source jar, javadoc, SA) can be triggered by the build task :<br />
<br />
<pre><br />
mvn install <br />
</pre><br />
<br />
this will install as well all artifacts in your local repository, while the deployment of the artifacts to the remote gCube Maven repos can triggered by the build task:<br />
<br />
<pre><br />
mvn deploy <br />
</pre><br />
<br />
== Portlet dependencies ==<br />
<br />
Specific portlet dependencies ( e.g. GWT, service stubs ) should be packaged within the portlet war. All dependencies specified within the pom file without the ''provided'' scope will be copied inside the WEB-INF/lib folder at packaging time.<br />
<br />
For this reason all portlet specific dependencies should not specified that scope e.g:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.externals</groupId><br />
<artifactId>gwt-ext-patched</artifactId><br />
<version>2.0.5</version><br />
</dependency><br />
</source><br />
<br />
On the other hand the system dependencies ( the one contained within the [[GCube_Portal_Installation#Portal_Bundle_Libraries|Portal bundle]] ) should not be included inside the portlet war. As said in this case the ''provided'' scope should be specified, e.g.<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portal</groupId><br />
<artifactId>custom-portal-handler</artifactId><br />
<version>1.1.0-SNAPSHOT</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.4.0,1.5.0]</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
<br />
A subset of these libraries ( contained in a GHN) that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called <code>ghn-core-runtime</code>. A dependency on this Jar can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ghn-core-runtime</artifactId><br />
<version>1.0.0</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Note: a component that depends on <code>gcf</code> indirectly depends on <code>gun-core-runtime</code>.<br />
<br />
== Include gCube Guided Tour using Maven ==<br />
<br />
This particular component is not yet ported to maven so in order to be included in a maven based project it should be declared as system dependency as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portletsuser</groupId><br />
<artifactId>gcube-guidedtour-widget</artifactId><br />
<version>1.0.0</version><br />
<scope>system</scope><br />
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/gcube-guidedtour-widget.jar</systemPath> <br />
</dependency><br />
</source><br />
<br />
together with the gCube Widget Library:<br />
<br />
<source lang="xml"> <br />
<dependency><br />
<groupId>org.gcube.portlets.user</groupId><br />
<artifactId>gcubewidgets</artifactId><br />
<version>1.2.0</version><br />
<scope>system</scope><br />
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/gCube-Widgets-Library.jar</systemPath><br />
</dependency><br />
</source><br />
<br />
both components can be downloaded from the distribution repository [[http://grids16.eng.it:8080/BuildReport/browse/Recent_Builds]]. <br />
<br />
In case the portlet depends on the ''gcf'' component, some dependency resolution changes are required to your portlet in order to properly compile ( because of libraries clashes)<br />
<br />
The ghn-core-runtime component ( gcf dependency) should be excluded:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.5.1-SNAPSHOT,2.0.0)</version><br />
<scope>provided</scope><br />
<exclusions><br />
<exclusion><br />
<artifactId>ghn-core-runtime</artifactId><br />
<groupId>org.gcube.distribution</groupId><br />
</exclusion><br />
</exclusions><br />
</dependency><br />
</source><br />
<br />
<br />
and some new dependencies should be added:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>xerces</groupId><br />
<artifactId>xercesImpl</artifactId><br />
<version>2.9.1</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ws-addressing</artifactId><br />
<version>1.0</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>axis-patched</artifactId><br />
<version>1.2RC</version><br />
<scope>provided</scope><br />
</dependency><br />
<br />
</source><br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the portlet is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>PortletAdmin</Class><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.portles.admin</groupId><br />
<artifactId>sample-admin-portlet</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/sample-admin-portlet-1.0.0-SNAPSHOT.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code> of the project;<br />
* package versions are aligned with the Maven <code>version</code> of the project;<br />
* package include <code>MavenCoordinates</code> that can be directly copied and pasted from the POM of the project;<br />
* packages do '''not''' specify dependencies;<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>sample-portlet</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>PortletAdmin</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/${build.finalName}.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
== Service Archive ==<br />
<br />
<code>sample-portlet</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the project.<br />
<br />
As shown for [[#Libraries|libraries]], the portlet uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources]] for details.<br />
<br />
== Use Case: Create a new Mavenized GWT Portlet ==<br />
<br />
In order to create a new Mavenized GWT Portlet this detailed guide can be also followed [https://gcube.wiki.gcube-system.org/gcube/index.php/Create_a_new_Mavenized_gCube_GWT_Portlet_Guide]<br />
<br />
= Widgets =<br />
<br />
Falls in this category of components mainly GWT widgets or so called GWT Libraries, which can be used to package classes for mutualization and/or modularization. A Widget is a java archive (JAR) containing both classes and java sources for later GWT compilation and a gwt.xml module descriptor. <br />
<br />
In this section we refer to a sample Widget with maven structure (https://svn.research-infrastructures.eu/d4science/gcube/trunk/portal/mvn-widget-sample/)<br />
<br />
== Project Structure ==<br />
<br />
As for a other project, the portlet is associated with 2 distinguished build artefacts:<br />
<br />
* the archive required for deployment (a JAR);<br />
* the archive required for registration in the system (a SA);<br />
<br />
In order to generate automatically the project structure and the base pom, we start by customizing the maven archetype liferay-portlet-archetype v. 6.0.6.<br />
<br />
The generation of the project structure and the pom can be performed either from Eclipse ( by using the m2eclipse plugin) or by command line using the Maven SDK.<br />
<br />
On Eclipse the following steps must be followed:<br />
<br />
* FIle -> New -> Project<br />
* From the wizard select Maven->Maven Project<br />
* From the Select Archetype step, Select 'All Catalogs' from the Catalogs combobox and type 'portlet' on the Filter field.<br />
* Select the '''maven-archetype-quickstart''' version 1.1.0 archetype ( if does not appear on the list just unselect 'show the last version of the Archetype only")<br />
* In the next step you have to provide Maven coordinates for your project<br />
* groupid : <your group id> e.g org.gcube.portlets.admin<br />
* artifactId : <your artifactId> e.g. sample-admin-portlet<br />
* version : <your project version> e.g. 1.0.0-SNAPSHOT ( use always SNAPSHOT versions for development versions of a project)<br />
* package: <your portlet code package> e.g. org.gcube.portlets.admin.sample<br />
<br />
while from shell the following mvn command should be issued :<br />
<br />
<pre><br />
mvn archetype:generate \<br />
-DarchetypeArtifactId=maven-archetype-quickstart \<br />
-DarchetypeGroupId=com.liferay.maven.archetypes \<br />
-DarchetypeVersion=1.1.0 \<br />
-DartifactId=<your artifactId> \<br />
-DgroupId=<your group id> \<br />
-Dversion=1.0.0-SNAPSHOT<br />
</pre><br />
<br />
Both approaches generate a project structure similar to the picture below:<br />
<br />
[[image:sample-widget-project.png|Sample widget project structure]]<br />
<br />
In order to customize the build procedure to the gCube needs we should open the pom.xml and apply the following changes:<br />
<br />
Configure the inheritance from '''maven-parent''' :<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
Configure some project properties:<br />
<br />
<source lang="xml"><br />
<properties><br />
<distroDirectory>${project.basedir}/distro</distroDirectory><br />
<configDirectory>${project.basedir}/config</configDirectory><br />
</properties><br />
</source><br />
<br />
Add the plugin for the generation of the SA, which has to be included under the build plugins section:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<version>2.2</version><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
In the case of a GWT widget the [http://mojo.codehaus.org/gwt-maven-plugin gwt-maven-plugin] should also be configured ( within build plugins section as well), by also selecting the proper gwt version ( in our example 2.4)<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.codehaus.mojo</groupId><br />
<artifactId>gwt-maven-plugin</artifactId><br />
<version>${gwt.version}</version><br />
<executions><br />
<execution><br />
<goals><br />
<goal>compile</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
The proper GWT version should be added to the pom properties :<br />
<br />
<source lang="xml"><br />
<gwt.version>2.4.0</gwt.version><br />
</source><br />
<br />
and GWT dependencies should be also configured:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-user</artifactId><br />
<version>${gwt.version}</version><br />
</dependency><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-servlet</artifactId><br />
<version>${gwt.version}</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/mvn-widget-sample/ project sources]] for further details on the project configuration.<br />
<br />
== Generating Artifacts ==<br />
<br />
The <code>sample-portlet</code> uses the '''maven-compiler-plugin''' to compile the portlet code and the '''gwt-maven-plugin''' as well for GWT compilation.<br />
<br />
The generation of a war package can be triggered by typing :<br />
<br />
<pre><br />
mvn package <br />
</pre><br />
<br />
on your shell or running this build task within Eclipse.<br />
<br />
In both cases the war package will be generated under the ''target'' folder, e.g.:<br />
<br />
<pre><br />
target/sample-admin-portlet-1.0.0-SNAPSHOT.war<br />
</pre><br />
<br />
The generation of all project artifacts ( war, source jar, javadoc, SA) can be triggered by the build task :<br />
<br />
<pre><br />
mvn install <br />
</pre><br />
<br />
this will install as well all artifacts in your local repository, while the deployment of the artifacts to the remote gCube Maven repos can triggered by the build task:<br />
<br />
<pre><br />
mvn deploy <br />
</pre><br />
<br />
== Widget dependencies ==<br />
<br />
The system dependencies ( the one contained within the [[GCube_Portal_Installation#Portal_Bundle_Libraries|Portal bundle]] ) should not be included inside the portlet war. As said in this case the ''provided'' scope should be specified, e.g.<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portal</groupId><br />
<artifactId>custom-portal-handler</artifactId><br />
<version>1.1.0-SNAPSHOT</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.4.0,1.5.0]</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the widget is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>PortletAdmin</Class><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.portles.admin</groupId><br />
<artifactId>sample-admin-portlet</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/sample-admin-portlet-1.0.0-SNAPSHOT.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code> of the project;<br />
* package versions are aligned with the Maven <code>version</code> of the project;<br />
* package include <code>MavenCoordinates</code> that can be directly copied and pasted from the POM of the project;<br />
* packages do '''not''' specify dependencies;<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>sample-portlet</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>PortletAdmin</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/${build.finalName}.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
== Service Archive ==<br />
<br />
<code>sample-portlet</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the project.<br />
<br />
As shown for [[#Libraries|libraries]], the portlet uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources]] for details.</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Developing_gCube_Maven_Components&diff=29737Developing gCube Maven Components2017-10-30T14:37:28Z<p>Gabriele.giammatteo: /* Service Archive (SA) */</p>
<hr />
<div>[[Category: Developer's Guide]]<br />
__TOC__<br />
<br />
= Common =<br />
<br />
== Maven Coordinates ==<br />
Maven components' coordinates must follow these conventions:<br />
<br />
* their <code>groupId</code> must be of the following form:<br />
<br />
<pre>org.gcube.<class></pre><br />
<br />
: where ''class'' identifies a functionally correlated set of components and helps a range of users (developers, administrator) to discover components in various UIs/reports used within the system and in tools. Logically, the class corresponds to the gCube class of the component's, as specified in its gCube Software Profile. However, more specific, or else cross-class groups may be agreed upon among developers. In all cases, the group should align with official Maven [http://maven.apache.org/guides/mini/guide-naming-conventions.html guidelines]. By agreement, for example, components in gCube class <code>DataAccess</code> declare a <code>groupId</code> of <code>org.gcube.data.access</code> and those in gCube class <code>DataTransfer</code> declare a <code>groupId</code> of <code>org.gcube.data.transfer</code>. On the other hand, also by agreement, components in gCube class <code>InformationSystem</code> declare a <code>groupId</code> of <code>org.gcube.informationsystem</code>.<br />
<br />
* their <code>artifactId</code> must follow Maven conventions (e.g. ''avoid capital letters or punctuation other than hyphens'').<br />
<br />
== Parent ==<br />
All gCube components must inherit from '''maven-parent'''. This ensure and promotes compliance with system-wide requirements, from the enforcement of minimal Java and Maven versions to generation and packaging of Javadoc documentation and component sources.<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
<br />
The maven-parent defines:<br />
* several system-wide properties like <code>gcube.license</code>, <code>gcube.wikiRoot</code>, <code>distroDirectory</code><br />
* the default Java version (see more in [[Developing_gCube_Maven_Components#Java_Version|Java Version]])<br />
* the execution of some plugins that run for every component like: <code>maven-javadoc-plugin</code>, <code>maven-source-plugin</code><br />
* the configuration of the <code>maven-assembly-plugin</code> to generate some gCube-specific packages: <code>servicearchive</code>, <code>uberjar</code> and <code>source-package</code> (see more in [[Developing_gCube_Maven_Components#Artifact_Types|Artifact Types]])<br />
* the logic to enable the generation of the source package<br />
<br />
The '''maven-parent''' is at http://maven.research-infrastructures.eu/nexus/service/local/repositories/gcube-externals/content/org/gcube/tools/maven-parent/1.0.0/maven-parent-1.0.0.pom<br />
<br />
== The distro folder ==<br />
This folder contains files that will be distributed along with the code in the distribution packages. Further details are in the [[Maven distro directory layout]] page.<br />
<br />
<br />
== Dependencies ==<br />
Service and libraries that depend on other gCube components (Maven-based or Ant-based) can find them in the gCube Maven Repositories: <code>gcube-snapshots</code> <strike>and <code>gcube-releases</code></strike>. <br />
<br />
=== gCube Maven Repositories ===<br />
As a one-off task, developers must configure these repositories in the <code>~/.m2/settings.xml</code> file of their local Maven installation, as follows:<br />
<br />
<source lang="xml"><br />
<settings xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"<br />
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd"><br />
<br />
<servers><br />
<server><br />
<id>gcube-snapshots</id><br />
<username>gcube-user</username><br />
<password>maven</password><br />
</server><br />
</servers><br />
<br />
<br />
<profiles><br />
<profile><br />
<id>gcube</id><br />
<repositories><br />
<repository><br />
<id>gcube-snapshots</id><br />
<name>gCube Snapshots</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
</snapshots><br />
</repository><br />
<repository><br />
<id>gcube-externals</id><br />
<name>gCube Externals</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals</url><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
</repository><br />
</repositories><br />
<br />
<pluginRepositories><br />
<pluginRepository><br />
<id>gcube-snapshots</id><br />
<name>gCube Snapshots</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
</snapshots><br />
</pluginRepository><br />
<pluginRepository><br />
<id>gcube-externals</id><br />
<name>gCube Externals</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals</url><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
</pluginRepository><br />
</pluginRepositories><br />
<br />
</profile><br />
</profiles><br />
<br />
<activeProfiles><br />
<activeProfile>gcube</activeProfile><br />
</activeProfiles><br />
</settings><br />
</source><br />
<br />
<br />
'''Note:Developers must not configure these repositories in their POMs''', only in <code>settings.xml</code>. <br />
<br />
This is to:<br />
* keep their projects independent from future changes in repository locations; <br />
* avoid interference with ETICS builds, where access to these repositories may comprise correct system integration;<br />
<br />
With this settings.xml, developers will have access on their local development environment to development versions (gcube-snapshots) and externals (gcube-externals). This is the correct behaviour for most of use cases. If access to staging or releases repositories is necessary (e.g. patching a component already released in the branch where dependencies versions in the release are not in snapshots), developer should modify his settings.xml to point to gcube-staging(releases) instead of gcube-snapshots.<br />
<br />
=== Third-party Repositories ===<br />
Using the the <code>settings.xml</code> aforementioned, any dependency declared in pom.xml that can be found in one of gCube repositories or in Maven Central will be resolved at build time and deployment time without any further configuration. Frequently it may happens that a gCube component needs software stored on a public Maven repository not in on of those repositories. In this case, Maven provides an easy solution for declaring additional repositories for dependency resolution by adding a <repositories> section in the pom.xml of the project. E.g.:<br />
<br />
<source lang="xml" highlight="3,4"><br />
<repositories><br />
<repository><br />
<id>[repository-id]</id><br />
<url>[repository URL]</url><br />
</repository><br />
</repositories><br />
</source><br />
<br />
Though technically possible to add any Maven repository in this way, in order to preserve the quality of gCube software it is required to check trustworthiness of its dependencies disallowing usage of certain external software. To enforce such a control over public Maven repositories, '''gCube components are allowed to use external repository not directly but only through a proxy repository''' (a.k.a. shadow repository) created on CNR's Nexus Repository Portal (http://maven.research-infrastructures.eu/nexus/).<br />
<br />
List of all repositories in Nexus Portal is available at http://maven.research-infrastructures.eu/nexus/index.html#view-repositories<br />
<br />
'''Note''': if a component uses a public Maven repository directly, it will work for building but would fail in deployment phase. In fact, Software Gateway is only able to download software accessible from Nexus.<br />
<br />
==== Request new proxy repository in Nexus ====<br />
Proxy Repositories on Nexus are created by the Release Manager. To request a new proxy a new ticket must be created.<br />
<br />
Details of ticket are:<br />
# open a ticket in tracker "Support" in gCube ticketing system (at https://support.d4science.org/projects/gcube/)<br />
# ticket's '''description''' must report:<br />
#* URL of repository to add<br />
#* organization which hosts the repository<br />
#* which artifacts are needed from that repository<br />
#* which components needs those artifacts<br />
# '''Sprint''' field should be set to the gCube release in which the component is expected to be released <br />
# '''Category''' field must report the gCube subsystem that needs the external repository<br />
# '''Assignee''' is the Release Manager<br />
<br />
An example of ticket for requesting a new repository mirror is reported in the figure below.<br />
[[Image:New_maven_mirror_repo.png|600px|center|Ticket for requesting a new Mirror Repository]]<br />
<br />
Release Manager will process the ticket checking that no issues are foreseen from the usage of software stored in the repository. If request is accepted, a new proxy repository is created and the repository's URL will be communicated publicly on gCube's developers mailinglist (a.k.a. TCOM mailinglist).<br />
<br />
Knowing the mirror repository URL, developer can use it adding following lines in pom.xml. E.g.<br />
<br />
<source lang="xml" highlight="3,4"><br />
<repositories><br />
<repository><br />
<id>gcube-ext-osgeo</id><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/osgeo/</url><br />
</repository><br />
</repositories><br />
</source><br />
<br />
'''Note''': it is possible that software stored on proxy repositories is not included in search results on Nexus. This happens for those repositories that does not have indexes (or have indexes in a format that Nexus is not able to download). Anyway, once Nexus has downloaded artifacts from the original repository for the first time (e.g. because a compilation of a pom that uses the proxy repository), creates its own indexes and those artifacts will become available also for local search.<br />
<br />
=== Dependency Versions ===<br />
<br />
ETICS builds ensure correct integration of gCube components, regardless of the versions their dependencies. It also ensures that components are released with fixed versions for their dependencies, i.e. ensures build reproducibility.<br />
<br />
Accordingly, developers can be liberal in the choice of dependency versions. i.e. rely on version ranges to:<br />
<br />
* minimise the requirements for version management;<br />
* be readily informed of any mis-alignment with their dependencies;<br />
<br />
As we start adopting Maven, the lower-bound of version ranges will be a SNAPSHOT version. For example, a dependency on the <code>streams</code> library can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>streams</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
</dependency><br />
</source><br />
<br />
With this dependency, a component will be soon notified of a non-retro compatible change that, contrary to conventions, <code>streams</code> may have introduced in, say, version <code>1.3.0</code> (typically the day after at the latest). The notification will be delivered in the IDE, assuming the availability of Maven integration (e.g. <code>m2e</code> in Eclipse). <br />
The component is in fact free to open the range further if it wants to be aligned with its dependencies across major versions.<br />
<br />
=== Common Dependencies ===<br />
<br />
Many gCube components depend on <code>gcf</code>, which is now available as a Maven component since version <code>1.4</code>. At the time of writing, the component has not been released yet. A dependency on it is then specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
The scope <code>provided</code> indicates that these dependency will be satisfied in the development environment (the gHN) and should not be included in GAR and SA archives.<br />
<br />
Some components depend on the libraries available in all gHNs, though they do not depend on <code>gcf</code> itself. The subset of these libraries that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called <code>ghn-core-runtime</code>. A dependency on this Jar can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ghn-core-runtime</artifactId><br />
<version>1.0.0</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Note: a component that depends on <code>gcf</code> indirectly depends on <code>gun-core-runtime</code>.<br />
<br />
Another common dependency is on the gCube gRS2, an Ant component that the build infrastructure make available in Maven repositories. A dependency on this component can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.execution</groupId><br />
<artifactId>grs2library</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
</dependency><br />
</source><br />
<br />
== Java Version ==<br />
<br />
In Maven the Java version of the source code and of the generated classes can be controlled using the configuration of the <code>maven-compiler-plugin</code>. E.g.:<br />
<syntaxhighlight lang="xml" highlight="5,6"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-compiler-plugin</artifactId><br />
<configuration><br />
<source>1.7</source><br />
<target>1.8</target><br />
</configuration><br />
</plugin><br />
</syntaxhighlight><br />
<br />
<br />
In gCube, this mechanism is exploited to enforce the usage of the same Java version for all components. In particular, the maven-parent defines the properties <code>java_version</code>, <code>maven.compiler.source</code> and <code>maven.compiler.target</code> and use these properties in the configuration of the <code>maven-compiler-plugin</code>.<br />
<syntaxhighlight lang="xml" highlight="15,16"><br />
...<br />
<properties><br />
<java_version>1.8</java_version><br />
<maven.compiler.source>${java_version}</maven.compiler.source><br />
<maven.compiler.target>${java_version}</maven.compiler.target><br />
...<br />
</properties><br />
<br />
...<br />
<br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-compiler-plugin</artifactId><br />
<configuration><br />
<target>${maven.compiler.target}</target><br />
<source>${maven.compiler.source}</source><br />
</configuration><br />
</plugin><br />
...<br />
</syntaxhighlight><br />
<br />
'''All the gCube components inherit this default configuration from <code>maven-parent</code> and it is not recommended to override it in the component's pom.xml'''.<br />
<br />
If it is really needed to override these values at component's pom.xml level, instead of specify a new configuration for the maven-compiler-plugin, the value of the property should be re-defined:<br />
<br />
<syntaxhighlight lang="xml" highlight="3"><br />
...<br />
<properties><br />
<maven.compiler.source>1.7</maven.compiler.source><br />
...<br />
</properties><br />
<br />
</syntaxhighlight><br />
<br />
<br />
This configuration is flexible enough to allow to customize the Java compiler source and target values depending on different factors in the integration builds (e.g. during ETICS builds gCore components are built with Java 7, all the others with Java 8).<br />
<br />
<br />
<br />
== Artifact Types ==<br />
<br />
There are four gCube-specific artifact types that are generated during the build of the gCube components. Each of them is summarized in the following subsections.<br />
<br />
=== Source Package ===<br />
The source package is an artifact with classifier <code>src</code> that contains the source code of the component plus the files in the distro directory (see [[Maven distro directory layout]] for more).<br />
<br />
Its descriptor is at [http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/build-utils/shared-assemblies/src/main/resources/assemblies/gcube-source-package.xml link].<br />
<br />
It is automatically generated if the <code>generateDistribution</code> property is ''true''.<br />
<br />
=== Uberjar ===<br />
<br />
The Uberjar is an artifact with classifier <code>uberjar</code> that contains the component classes plus all the jar files of the dependencies. This is not mandatory for all components, but only for those that needs to be deployed along with their dependenices.<br />
<br />
Its descriptor is at [http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/build-utils/shared-assemblies/src/main/resources/assemblies/gcube-uberjar.xml link].<br />
<br />
The generation of this package is '''not enabled by default'''. To enable it, bind its configuration to a build phase in the component pom.xml:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<executions><br />
<execution><br />
<id>make-uberjar</id><br />
<phase>install</phase><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
=== Service Archive (SA) ===<br />
Whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary <code>tar.gz</code> artefact with classifier <code>servicearchive</code>. Note that the SA of Maven components does ''not'' need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);<br />
<br />
* for each deployable artefact they produce (primary or secondary), they must include the Maven coordinates of the arteact in the corresponding package section of the gCube Software Profile. Specifying package dependencies in the Software Profile, on the other hand, is no longer required;<br />
<br />
* whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary <code>tar.gz</code> artefact with classifier <code>servicearchive</code>. Note that the SA of Maven components does ''not'' need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);<br />
<br />
<br />
The '''maven-parent''' defines a default descriptor to build a servicearchive ([http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/build-utils/shared-assemblies/src/main/resources/assemblies/gcube-default-servicearchive.xml link]).<br />
<br />
This Please not that, while this fits most of the single module projects that meet the requirements in [[Maven distro directory layout]], for multi-module projects and/or projects that need to include custom files a custom servicearchive might be needed. In the latter case, the default assembly configuration cannot be used and a custom maven-assembly-plugin configuration must be defined in the component's pom.xml.<br />
<br />
The generation of the default servicearchvie is disabled by default. To enable it, bind its configuration to a build phase in the component pom.xml:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<executions><br />
<execution><br />
<id>make-servicearchive</id><br />
<phase>package</phase><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
=== Javadoc ===<br />
It is the default artifact (with classifier <code>javadoc</code>) generated by the <code>maven-javadoc-plugin</code>. It is automatically generated for all components.<br />
<br />
= Libraries =<br />
The <code>streams</code> library is a Maven component in gCube class <code>DataAccess</code>. We will use it as example of a library component in Maven.<br />
<br />
=== Profile ===<br />
<br />
The gCube Software profile of the library is as follows:<br />
<br />
<source lang="xml" highlight="14-18"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>Embedded Domain-Specific Language for Stream Transformations</Description><br />
<Class>DataAccess</Class><br />
<Name>streams</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>streams</Name><br />
<Version>1.0.0-SNAPSHOT</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>streams</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<Files><br />
<File>streams-1.0.0-SNAPSHOT.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the profile includes a single package and this package corresponds to the main build artefact of the component; <br />
* the whole profile and the package have usual gCube coordinates;<br />
* the package name is aligned with the Maven <code>artifactId</code> of the component;<br />
* the package version is aligned with the Maven <code>version</code> of the component;<br />
* the package includes <code>MavenCoordinates</code> that can be directly copied and pasted from the POM;<br />
* the package does '''not''' specify dependencies;<br />
* the package points to the main artefact of the Maven build;<br />
<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>streams</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>DataAccess</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<Files><br />
<File>${build.finalName}.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
=== Service Archive ===<br />
<br />
<code>streams</code> is responsible for generating its own SA as part of its Maven build. This requires dedicated logic in the POM but has the advantage that:<br />
<br />
* SA validity can be verified locally;<br />
* no SA configurations are required in ETICS;<br />
* the required build logic can be easily reused across components;<br />
<br />
<code>streams</code> uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation in the profile and other static files. The approach is best illustrated with a reference to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/streams sources]] of the <code>streams</code> library.<br />
<br />
<br />
As we discuss [[#Creating the ETICS Configurations|below]], <code>streams</code> has the final requirement to produce an interpolated profile during ETICS builds, outside the context of the SA.<br />
<code>streams</code> meets this requirement with the [http://maven.apache.org/plugins/maven-resources-plugin/ Maven Resources Plugin]. Again, consult the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/streams sources]] of the <code>streams</code> library to reuse this approach.<br />
<br />
= Services =<br />
The <code>tree-manager</code> service is a Maven component in gCube class <code>DataAccess</code>. <br />
<br />
== Project Structure ==<br />
<br />
Differently from a library, the service is associated with a number of distinguished build artefacts:<br />
<br />
* the service implementation (a Jar);<br />
* the stubs automatically generated from the WSDL interfaces (a Jar);<br />
* the archive required for deployment in gCore (a Gar);<br />
* the archive required for registration in the system (a SA);<br />
<br />
Producing these artefacts requires sharing a number of static and dynamic files (from profiles to WSDLs to pre-processed WSDLs). The usual Maven approach of mapping the artefacts onto different Maven projects is then genuinely difficult. It is instead more practical to use a single multi-module Maven project. The <code>tree-manager</code> uses this module structure:<br />
<br />
<pre><br />
tree-manager (parent module)------- (produces) --- (POM only)<br />
|<br />
|----- tree-manager-service (service impl module)------- (produces) --- jar, gar, SA<br />
| |<br />
| | (depends on)<br />
| |<br />
| \/<br />
|----- tree-manager-stubs (service stubs module)------- (produces) --- jar<br />
</pre><br />
<br />
As customary, the parent does not produce any artefact (is a POM-only project). It declares these coordinates in its POM:<br />
<br />
<source lang="xml"><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
<packaging>pom</packaging><br />
</source><br />
<br />
The parent physically includes the sub-modules;<br />
<br />
<source lang="xml"><br />
<modules><br />
<module>tree-manager-stubs</module><br />
<module>tree-manager-service</module><br />
</modules><br />
</source> <br />
<br />
Finally, the parent inherits from '''maven-parent''':<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
The submodules inherit from the parent:<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>tree-manager</artifactId><br />
<groupId>org.gcube.data.access</groupId><br />
<version>1.0.0-SNAPSHOT</version><br />
<relativePath>..</relativePath><br />
</parent><br />
</source><br />
<br />
and share its <code>version</code> and <code>groupId</code>. Their coordinates are then minimal:<br />
<br />
<source lang="xml"><br />
<artifactId>tree-manager-service</artifactId><br />
</source><br />
<br />
<source lang="xml"><br />
<artifactId>tree-manager-stubs</artifactId><br />
</source><br />
<br />
The <code>tree-manager-service</code> module produces a Jar of its own implementation, as well the GAR and the SA of the whole services as secondary artefacts with classifiers of, respectively, <code>gar</code> and <code>service archive</code>.<br />
<br />
The <code>tree-manager-stubs</code> module produces a Jar of its own implementation.<br />
<br />
Finally, the parent module coordinates the build of its sub-modules and centralises the declaration of common dependencies and configuration. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] to reuse this approach.<br />
<br />
== Generating Stubs and Gars ==<br />
<br />
The <code>tree-manager-stubs</code> uses the '''gCube Service Plugin''' to generate its classes from WSDL interfaces. <br />
<br />
Similarly, the <code>tree-manager-service</code> uses the plugin to generate its secondary GAR artefact. <br />
<br />
The configuration of the plugin is conveniently shared in the parent module.<br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] of the <code>streams</code> for details.<br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the service is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>DataAccess</Class><br />
<Name>tree-manager-service</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Main><br />
<Name>tree-manager-service</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager-service</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<GARArchive>tree-manager-service-1.0.0-SNAPSHOT.gar</GARArchive><br />
<PortType><br />
<Name>gcube/data/tm/binder</Name><br />
</PortType><br />
<PortType><br />
<Name>gcube/data/tm/reader</Name><br />
</PortType><br />
<PortType><br />
<Name>gcube/data/tm/writer</Name><br />
</PortType><br />
</Main><br />
<Software><br />
<Name>tree-manager-stubs</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager-stubs</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<Type>library</Type><br />
<Files><br />
<File>tree-manager-stubs-1.0.0-SNAPSHOT.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the profile includes two packages and this package corresponds to the main build artefacts of the submodules; <br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code>s of the submodules;<br />
* package versions are aligned with the Maven <code>version</code>s of the submodules;<br />
* package include <code>MavenCoordinates</code>s that can be directly copied and pasted from the POM of the submodules;<br />
* packages do '''not''' specify dependencies;<br />
* packages points to the main artefact of the Maven build of the submodules;<br />
<br />
<strike>Notice that, unlike libraries, services cannot easily use Maven variables in their profiles (as these artefacts are used for different purposes by different modules). Accordingly, developers needs to maintain versions used in profiles across releases. In particular, they need to:<br />
<br />
* replace SNAPSHOT versions with release versions in release branches;<br />
* advance SNAPSHOT versions in HEAD;</strike><br />
<br />
=== Profile Interpolation ===<br />
The interpolation of profiles is required in order to keep information in profile in sync with <code>pom.xml</code>, in particular versions and filenames (that contain versions) that change at each release.<br />
<br />
In order to generate <code>profile.xml</code> during build process you need to:<br />
# add in the service's parent <code>pom.xml</code> the invocation to maven-resources-plugin<br />
# place a ''template'' <code>profile.xml</code> in the <code>${distroDirectory}</code><br />
<br />
Invocation to <code>maven-resources-plugin</code> looks like the following (to be placed in the <plugins> section inside the <build> section of the pom):<br />
<source lang="xml"><br />
<!-- interpolates profiles and copies from distribution location to configuration <br />
location, where it is need for embedding into stub artifact, gar generation, <br />
and service archive. --><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-resources-plugin</artifactId><br />
<version>2.6</version><br />
<executions><br />
<execution><br />
<id>copy-profile</id><br />
<goals><br />
<goal>copy-resources</goal><br />
</goals><br />
<phase>process-resources</phase><br />
<configuration><br />
<outputDirectory>${configDirectory}</outputDirectory><br />
<resources><br />
<resource><br />
<directory>${distroDirectory}</directory><br />
<includes><br />
<include>profile.xml</include><br />
</includes><br />
<filtering>true</filtering><br />
</resource><br />
</resources><br />
</configuration><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
The ''template'' <code>profile.xml</code> looks like the following:<br />
<source lang="xml" highlight="12,16,18,22,26,30"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>TestClass</Class><br />
<Name>TestService</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Main><br />
<Name>TestService</Name><br />
<Version>${project.version}</Version><br />
<MavenCoordinates><br />
<groupId>org.example</groupId><br />
<artifactId>test-service</artifactId><br />
<version>${project.version}</version><br />
</MavenCoordinates><br />
<GARArchive>>test-service-${project.version}.gar</GARArchive><br />
</Main><br />
<Software><br />
<Name>test-stubs</Name><br />
<Version>${project.version}</Version><br />
<MavenCoordinates><br />
<groupId>org.example</groupId><br />
<artifactId>test-stubs</artifactId><br />
<version>${project.version}</version><br />
</MavenCoordinates><br />
<Type>library</Type><br />
<Files><br />
<File>test-stubs-${project.version}.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
In the example, property <code>${project.version}</code> will be substituted with the actual service's version at build-time. It is also possible, but not required, to parametrize also other values of <code>profile.xml</code> such as maven's coordinates and package names. You can see a more complex <code>profile.xml</code> ''template'' in the <code>tree-manager</code> project: http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-access/tree-manager/distro/profile.xml<br />
<br />
Maven will copy the final <code>profile.xml</code> in <code>${configDirectory}</code> during the <code>process-resources</code> phase of the build. From there it will be copied in the service-archive package.<br />
<br />
== Service Archive ==<br />
<br />
<code>tree-manager-service</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the module and its parent.<br />
<br />
As shown for [[#Libraries|libraries]], the service uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] for details.<br />
<br />
= Portlets =<br />
<br />
The <code>sample-portlet</code> portlet is a Maven based sample portlet based on GWT. Refer to the [http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources] for details.<br />
<br />
== Project Structure ==<br />
<br />
As for a library project, the portlet is associated with 2 distinguished build artefacts:<br />
<br />
* the archive required for deployment in a Liferay Container (a WAR);<br />
* the archive required for registration in the system (a SA);<br />
<br />
In order to generate automatically the project structure and the base pom, we start by customizing the maven archetype liferay-portlet-archetype v. 6.0.6.<br />
<br />
The generation of the project structure and the pom can be performed either from Eclipse ( by using the m2eclipse plugin) or by command line using the Maven SDK.<br />
<br />
On Eclipse the following steps must be followed:<br />
<br />
* FIle -> New -> Project<br />
* From the wizard select Maven->Maven Project<br />
* From the Select Archetype step, Select 'All Catalogs' from the Catalogs combobox and type 'portlet' on the Filter field.<br />
* Select the '''liferay-portlet-archetype''' version 6.0.6 archetype ( if does not appear on the list just unselect 'show the last version of the Archetype only")<br />
* In the next step you have to provide Maven coordinates for your project<br />
* groupid : <your group id> e.g org.gcube.portlets.admin<br />
* artifactId : <your artifactId> e.g. sample-admin-portlet<br />
* version : <your project version> e.g. 1.0.0-SNAPSHOT ( use always SNAPSHOT versions for development versions of a project)<br />
* package: <your portlet code package> e.g. org.gcube.portlets.admin.sample<br />
<br />
while from shell the following mvn command should be issued :<br />
<br />
<pre><br />
mvn archetype:generate \<br />
-DarchetypeArtifactId=liferay-theme-archetype \<br />
-DarchetypeGroupId=com.liferay.maven.archetypes \<br />
-DarchetypeVersion=6.0.6 \<br />
-DartifactId=<your artifactId> \<br />
-DgroupId=<your group id> \<br />
-Dversion=1.0.0-SNAPSHOT<br />
</pre><br />
<br />
Both approaches generate a project structure similar to the picture below:<br />
<br />
[[image:sample-portlet-project.png|Sample portlet project structure]]<br />
<br />
In order to customize the build procedure to the gCube needs we should open the pom.xml and apply the following changes:<br />
<br />
Configure the inheritance from '''maven-parent''' :<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
Configure some project properties:<br />
<br />
<source lang="xml"><br />
<properties><br />
<liferay.version>6.0.6</liferay.version><br />
<liferay.auto.deploy.dir>${system.CATALINA_HOME}/../deploy</liferay.auto.deploy.dir><br />
<distroDirectory>${project.basedir}/distro</distroDirectory><br />
<configDirectory>${project.basedir}/config</configDirectory><br />
</properties><br />
</source><br />
<br />
Add the plugin for the generation of the SA, which has to be included under the build plugins section:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<version>2.2</version><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
In the case of a GWT portlet the [http://mojo.codehaus.org/gwt-maven-plugin gwt-maven-plugin] should also be configured ( within build plugins section as well), by also selecting the proper gwt version ( in our example 2.4)<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.codehaus.mojo</groupId><br />
<artifactId>gwt-maven-plugin</artifactId><br />
<version>${gwt.version}</version><br />
<executions><br />
<execution><br />
<goals><br />
<goal>compile</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
The proper GWT version should be added to the pom properties :<br />
<br />
<source lang="xml"><br />
<gwt.version>2.4.0</gwt.version><br />
</source><br />
<br />
and GWT dependencies should be also configured:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-user</artifactId><br />
<version>${gwt.version}</version><br />
</dependency><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-servlet</artifactId><br />
<version>${gwt.version}</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet/ project sources]] for further details on the project configuration.<br />
<br />
== Generating Artifacts ==<br />
<br />
The <code>sample-portlet</code> uses the '''maven-compiler-plugin''' to compile the portlet code and the '''gwt-maven-plugin''' as well for GWT compilation.<br />
<br />
The generation of a war package can be triggered by typing :<br />
<br />
<pre><br />
mvn package <br />
</pre><br />
<br />
on your shell or running this build task within Eclipse.<br />
<br />
In both cases the war package will be generated under the ''target'' folder, e.g.:<br />
<br />
<pre><br />
target/sample-admin-portlet-1.0.0-SNAPSHOT.war<br />
</pre><br />
<br />
The generation of all project artifacts ( war, source jar, javadoc, SA) can be triggered by the build task :<br />
<br />
<pre><br />
mvn install <br />
</pre><br />
<br />
this will install as well all artifacts in your local repository, while the deployment of the artifacts to the remote gCube Maven repos can triggered by the build task:<br />
<br />
<pre><br />
mvn deploy <br />
</pre><br />
<br />
== Portlet dependencies ==<br />
<br />
Specific portlet dependencies ( e.g. GWT, service stubs ) should be packaged within the portlet war. All dependencies specified within the pom file without the ''provided'' scope will be copied inside the WEB-INF/lib folder at packaging time.<br />
<br />
For this reason all portlet specific dependencies should not specified that scope e.g:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.externals</groupId><br />
<artifactId>gwt-ext-patched</artifactId><br />
<version>2.0.5</version><br />
</dependency><br />
</source><br />
<br />
On the other hand the system dependencies ( the one contained within the [[GCube_Portal_Installation#Portal_Bundle_Libraries|Portal bundle]] ) should not be included inside the portlet war. As said in this case the ''provided'' scope should be specified, e.g.<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portal</groupId><br />
<artifactId>custom-portal-handler</artifactId><br />
<version>1.1.0-SNAPSHOT</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.4.0,1.5.0]</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
<br />
A subset of these libraries ( contained in a GHN) that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called <code>ghn-core-runtime</code>. A dependency on this Jar can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ghn-core-runtime</artifactId><br />
<version>1.0.0</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Note: a component that depends on <code>gcf</code> indirectly depends on <code>gun-core-runtime</code>.<br />
<br />
== Include gCube Guided Tour using Maven ==<br />
<br />
This particular component is not yet ported to maven so in order to be included in a maven based project it should be declared as system dependency as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portletsuser</groupId><br />
<artifactId>gcube-guidedtour-widget</artifactId><br />
<version>1.0.0</version><br />
<scope>system</scope><br />
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/gcube-guidedtour-widget.jar</systemPath> <br />
</dependency><br />
</source><br />
<br />
together with the gCube Widget Library:<br />
<br />
<source lang="xml"> <br />
<dependency><br />
<groupId>org.gcube.portlets.user</groupId><br />
<artifactId>gcubewidgets</artifactId><br />
<version>1.2.0</version><br />
<scope>system</scope><br />
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/gCube-Widgets-Library.jar</systemPath><br />
</dependency><br />
</source><br />
<br />
both components can be downloaded from the distribution repository [[http://grids16.eng.it:8080/BuildReport/browse/Recent_Builds]]. <br />
<br />
In case the portlet depends on the ''gcf'' component, some dependency resolution changes are required to your portlet in order to properly compile ( because of libraries clashes)<br />
<br />
The ghn-core-runtime component ( gcf dependency) should be excluded:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.5.1-SNAPSHOT,2.0.0)</version><br />
<scope>provided</scope><br />
<exclusions><br />
<exclusion><br />
<artifactId>ghn-core-runtime</artifactId><br />
<groupId>org.gcube.distribution</groupId><br />
</exclusion><br />
</exclusions><br />
</dependency><br />
</source><br />
<br />
<br />
and some new dependencies should be added:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>xerces</groupId><br />
<artifactId>xercesImpl</artifactId><br />
<version>2.9.1</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ws-addressing</artifactId><br />
<version>1.0</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>axis-patched</artifactId><br />
<version>1.2RC</version><br />
<scope>provided</scope><br />
</dependency><br />
<br />
</source><br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the portlet is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>PortletAdmin</Class><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.portles.admin</groupId><br />
<artifactId>sample-admin-portlet</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/sample-admin-portlet-1.0.0-SNAPSHOT.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code> of the project;<br />
* package versions are aligned with the Maven <code>version</code> of the project;<br />
* package include <code>MavenCoordinates</code> that can be directly copied and pasted from the POM of the project;<br />
* packages do '''not''' specify dependencies;<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>sample-portlet</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>PortletAdmin</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/${build.finalName}.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
== Service Archive ==<br />
<br />
<code>sample-portlet</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the project.<br />
<br />
As shown for [[#Libraries|libraries]], the portlet uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources]] for details.<br />
<br />
== Use Case: Create a new Mavenized GWT Portlet ==<br />
<br />
In order to create a new Mavenized GWT Portlet this detailed guide can be also followed [https://gcube.wiki.gcube-system.org/gcube/index.php/Create_a_new_Mavenized_gCube_GWT_Portlet_Guide]<br />
<br />
= Widgets =<br />
<br />
Falls in this category of components mainly GWT widgets or so called GWT Libraries, which can be used to package classes for mutualization and/or modularization. A Widget is a java archive (JAR) containing both classes and java sources for later GWT compilation and a gwt.xml module descriptor. <br />
<br />
In this section we refer to a sample Widget with maven structure (https://svn.research-infrastructures.eu/d4science/gcube/trunk/portal/mvn-widget-sample/)<br />
<br />
== Project Structure ==<br />
<br />
As for a other project, the portlet is associated with 2 distinguished build artefacts:<br />
<br />
* the archive required for deployment (a JAR);<br />
* the archive required for registration in the system (a SA);<br />
<br />
In order to generate automatically the project structure and the base pom, we start by customizing the maven archetype liferay-portlet-archetype v. 6.0.6.<br />
<br />
The generation of the project structure and the pom can be performed either from Eclipse ( by using the m2eclipse plugin) or by command line using the Maven SDK.<br />
<br />
On Eclipse the following steps must be followed:<br />
<br />
* FIle -> New -> Project<br />
* From the wizard select Maven->Maven Project<br />
* From the Select Archetype step, Select 'All Catalogs' from the Catalogs combobox and type 'portlet' on the Filter field.<br />
* Select the '''maven-archetype-quickstart''' version 1.1.0 archetype ( if does not appear on the list just unselect 'show the last version of the Archetype only")<br />
* In the next step you have to provide Maven coordinates for your project<br />
* groupid : <your group id> e.g org.gcube.portlets.admin<br />
* artifactId : <your artifactId> e.g. sample-admin-portlet<br />
* version : <your project version> e.g. 1.0.0-SNAPSHOT ( use always SNAPSHOT versions for development versions of a project)<br />
* package: <your portlet code package> e.g. org.gcube.portlets.admin.sample<br />
<br />
while from shell the following mvn command should be issued :<br />
<br />
<pre><br />
mvn archetype:generate \<br />
-DarchetypeArtifactId=maven-archetype-quickstart \<br />
-DarchetypeGroupId=com.liferay.maven.archetypes \<br />
-DarchetypeVersion=1.1.0 \<br />
-DartifactId=<your artifactId> \<br />
-DgroupId=<your group id> \<br />
-Dversion=1.0.0-SNAPSHOT<br />
</pre><br />
<br />
Both approaches generate a project structure similar to the picture below:<br />
<br />
[[image:sample-widget-project.png|Sample widget project structure]]<br />
<br />
In order to customize the build procedure to the gCube needs we should open the pom.xml and apply the following changes:<br />
<br />
Configure the inheritance from '''maven-parent''' :<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
Configure some project properties:<br />
<br />
<source lang="xml"><br />
<properties><br />
<distroDirectory>${project.basedir}/distro</distroDirectory><br />
<configDirectory>${project.basedir}/config</configDirectory><br />
</properties><br />
</source><br />
<br />
Add the plugin for the generation of the SA, which has to be included under the build plugins section:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<version>2.2</version><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
In the case of a GWT widget the [http://mojo.codehaus.org/gwt-maven-plugin gwt-maven-plugin] should also be configured ( within build plugins section as well), by also selecting the proper gwt version ( in our example 2.4)<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.codehaus.mojo</groupId><br />
<artifactId>gwt-maven-plugin</artifactId><br />
<version>${gwt.version}</version><br />
<executions><br />
<execution><br />
<goals><br />
<goal>compile</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
The proper GWT version should be added to the pom properties :<br />
<br />
<source lang="xml"><br />
<gwt.version>2.4.0</gwt.version><br />
</source><br />
<br />
and GWT dependencies should be also configured:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-user</artifactId><br />
<version>${gwt.version}</version><br />
</dependency><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-servlet</artifactId><br />
<version>${gwt.version}</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/mvn-widget-sample/ project sources]] for further details on the project configuration.<br />
<br />
== Generating Artifacts ==<br />
<br />
The <code>sample-portlet</code> uses the '''maven-compiler-plugin''' to compile the portlet code and the '''gwt-maven-plugin''' as well for GWT compilation.<br />
<br />
The generation of a war package can be triggered by typing :<br />
<br />
<pre><br />
mvn package <br />
</pre><br />
<br />
on your shell or running this build task within Eclipse.<br />
<br />
In both cases the war package will be generated under the ''target'' folder, e.g.:<br />
<br />
<pre><br />
target/sample-admin-portlet-1.0.0-SNAPSHOT.war<br />
</pre><br />
<br />
The generation of all project artifacts ( war, source jar, javadoc, SA) can be triggered by the build task :<br />
<br />
<pre><br />
mvn install <br />
</pre><br />
<br />
this will install as well all artifacts in your local repository, while the deployment of the artifacts to the remote gCube Maven repos can triggered by the build task:<br />
<br />
<pre><br />
mvn deploy <br />
</pre><br />
<br />
== Widget dependencies ==<br />
<br />
The system dependencies ( the one contained within the [[GCube_Portal_Installation#Portal_Bundle_Libraries|Portal bundle]] ) should not be included inside the portlet war. As said in this case the ''provided'' scope should be specified, e.g.<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portal</groupId><br />
<artifactId>custom-portal-handler</artifactId><br />
<version>1.1.0-SNAPSHOT</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.4.0,1.5.0]</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the widget is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>PortletAdmin</Class><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.portles.admin</groupId><br />
<artifactId>sample-admin-portlet</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/sample-admin-portlet-1.0.0-SNAPSHOT.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code> of the project;<br />
* package versions are aligned with the Maven <code>version</code> of the project;<br />
* package include <code>MavenCoordinates</code> that can be directly copied and pasted from the POM of the project;<br />
* packages do '''not''' specify dependencies;<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>sample-portlet</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>PortletAdmin</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/${build.finalName}.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
== Service Archive ==<br />
<br />
<code>sample-portlet</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the project.<br />
<br />
As shown for [[#Libraries|libraries]], the portlet uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources]] for details.</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Developing_gCube_Maven_Components&diff=29736Developing gCube Maven Components2017-10-30T13:02:53Z<p>Gabriele.giammatteo: /* Uberjar */</p>
<hr />
<div>[[Category: Developer's Guide]]<br />
__TOC__<br />
<br />
= Common =<br />
<br />
== Maven Coordinates ==<br />
Maven components' coordinates must follow these conventions:<br />
<br />
* their <code>groupId</code> must be of the following form:<br />
<br />
<pre>org.gcube.<class></pre><br />
<br />
: where ''class'' identifies a functionally correlated set of components and helps a range of users (developers, administrator) to discover components in various UIs/reports used within the system and in tools. Logically, the class corresponds to the gCube class of the component's, as specified in its gCube Software Profile. However, more specific, or else cross-class groups may be agreed upon among developers. In all cases, the group should align with official Maven [http://maven.apache.org/guides/mini/guide-naming-conventions.html guidelines]. By agreement, for example, components in gCube class <code>DataAccess</code> declare a <code>groupId</code> of <code>org.gcube.data.access</code> and those in gCube class <code>DataTransfer</code> declare a <code>groupId</code> of <code>org.gcube.data.transfer</code>. On the other hand, also by agreement, components in gCube class <code>InformationSystem</code> declare a <code>groupId</code> of <code>org.gcube.informationsystem</code>.<br />
<br />
* their <code>artifactId</code> must follow Maven conventions (e.g. ''avoid capital letters or punctuation other than hyphens'').<br />
<br />
== Parent ==<br />
All gCube components must inherit from '''maven-parent'''. This ensure and promotes compliance with system-wide requirements, from the enforcement of minimal Java and Maven versions to generation and packaging of Javadoc documentation and component sources.<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
<br />
The maven-parent defines:<br />
* several system-wide properties like <code>gcube.license</code>, <code>gcube.wikiRoot</code>, <code>distroDirectory</code><br />
* the default Java version (see more in [[Developing_gCube_Maven_Components#Java_Version|Java Version]])<br />
* the execution of some plugins that run for every component like: <code>maven-javadoc-plugin</code>, <code>maven-source-plugin</code><br />
* the configuration of the <code>maven-assembly-plugin</code> to generate some gCube-specific packages: <code>servicearchive</code>, <code>uberjar</code> and <code>source-package</code> (see more in [[Developing_gCube_Maven_Components#Artifact_Types|Artifact Types]])<br />
* the logic to enable the generation of the source package<br />
<br />
The '''maven-parent''' is at http://maven.research-infrastructures.eu/nexus/service/local/repositories/gcube-externals/content/org/gcube/tools/maven-parent/1.0.0/maven-parent-1.0.0.pom<br />
<br />
== The distro folder ==<br />
This folder contains files that will be distributed along with the code in the distribution packages. Further details are in the [[Maven distro directory layout]] page.<br />
<br />
<br />
== Dependencies ==<br />
Service and libraries that depend on other gCube components (Maven-based or Ant-based) can find them in the gCube Maven Repositories: <code>gcube-snapshots</code> <strike>and <code>gcube-releases</code></strike>. <br />
<br />
=== gCube Maven Repositories ===<br />
As a one-off task, developers must configure these repositories in the <code>~/.m2/settings.xml</code> file of their local Maven installation, as follows:<br />
<br />
<source lang="xml"><br />
<settings xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"<br />
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd"><br />
<br />
<servers><br />
<server><br />
<id>gcube-snapshots</id><br />
<username>gcube-user</username><br />
<password>maven</password><br />
</server><br />
</servers><br />
<br />
<br />
<profiles><br />
<profile><br />
<id>gcube</id><br />
<repositories><br />
<repository><br />
<id>gcube-snapshots</id><br />
<name>gCube Snapshots</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
</snapshots><br />
</repository><br />
<repository><br />
<id>gcube-externals</id><br />
<name>gCube Externals</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals</url><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
</repository><br />
</repositories><br />
<br />
<pluginRepositories><br />
<pluginRepository><br />
<id>gcube-snapshots</id><br />
<name>gCube Snapshots</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
</snapshots><br />
</pluginRepository><br />
<pluginRepository><br />
<id>gcube-externals</id><br />
<name>gCube Externals</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals</url><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
</pluginRepository><br />
</pluginRepositories><br />
<br />
</profile><br />
</profiles><br />
<br />
<activeProfiles><br />
<activeProfile>gcube</activeProfile><br />
</activeProfiles><br />
</settings><br />
</source><br />
<br />
<br />
'''Note:Developers must not configure these repositories in their POMs''', only in <code>settings.xml</code>. <br />
<br />
This is to:<br />
* keep their projects independent from future changes in repository locations; <br />
* avoid interference with ETICS builds, where access to these repositories may comprise correct system integration;<br />
<br />
With this settings.xml, developers will have access on their local development environment to development versions (gcube-snapshots) and externals (gcube-externals). This is the correct behaviour for most of use cases. If access to staging or releases repositories is necessary (e.g. patching a component already released in the branch where dependencies versions in the release are not in snapshots), developer should modify his settings.xml to point to gcube-staging(releases) instead of gcube-snapshots.<br />
<br />
=== Third-party Repositories ===<br />
Using the the <code>settings.xml</code> aforementioned, any dependency declared in pom.xml that can be found in one of gCube repositories or in Maven Central will be resolved at build time and deployment time without any further configuration. Frequently it may happens that a gCube component needs software stored on a public Maven repository not in on of those repositories. In this case, Maven provides an easy solution for declaring additional repositories for dependency resolution by adding a <repositories> section in the pom.xml of the project. E.g.:<br />
<br />
<source lang="xml" highlight="3,4"><br />
<repositories><br />
<repository><br />
<id>[repository-id]</id><br />
<url>[repository URL]</url><br />
</repository><br />
</repositories><br />
</source><br />
<br />
Though technically possible to add any Maven repository in this way, in order to preserve the quality of gCube software it is required to check trustworthiness of its dependencies disallowing usage of certain external software. To enforce such a control over public Maven repositories, '''gCube components are allowed to use external repository not directly but only through a proxy repository''' (a.k.a. shadow repository) created on CNR's Nexus Repository Portal (http://maven.research-infrastructures.eu/nexus/).<br />
<br />
List of all repositories in Nexus Portal is available at http://maven.research-infrastructures.eu/nexus/index.html#view-repositories<br />
<br />
'''Note''': if a component uses a public Maven repository directly, it will work for building but would fail in deployment phase. In fact, Software Gateway is only able to download software accessible from Nexus.<br />
<br />
==== Request new proxy repository in Nexus ====<br />
Proxy Repositories on Nexus are created by the Release Manager. To request a new proxy a new ticket must be created.<br />
<br />
Details of ticket are:<br />
# open a ticket in tracker "Support" in gCube ticketing system (at https://support.d4science.org/projects/gcube/)<br />
# ticket's '''description''' must report:<br />
#* URL of repository to add<br />
#* organization which hosts the repository<br />
#* which artifacts are needed from that repository<br />
#* which components needs those artifacts<br />
# '''Sprint''' field should be set to the gCube release in which the component is expected to be released <br />
# '''Category''' field must report the gCube subsystem that needs the external repository<br />
# '''Assignee''' is the Release Manager<br />
<br />
An example of ticket for requesting a new repository mirror is reported in the figure below.<br />
[[Image:New_maven_mirror_repo.png|600px|center|Ticket for requesting a new Mirror Repository]]<br />
<br />
Release Manager will process the ticket checking that no issues are foreseen from the usage of software stored in the repository. If request is accepted, a new proxy repository is created and the repository's URL will be communicated publicly on gCube's developers mailinglist (a.k.a. TCOM mailinglist).<br />
<br />
Knowing the mirror repository URL, developer can use it adding following lines in pom.xml. E.g.<br />
<br />
<source lang="xml" highlight="3,4"><br />
<repositories><br />
<repository><br />
<id>gcube-ext-osgeo</id><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/osgeo/</url><br />
</repository><br />
</repositories><br />
</source><br />
<br />
'''Note''': it is possible that software stored on proxy repositories is not included in search results on Nexus. This happens for those repositories that does not have indexes (or have indexes in a format that Nexus is not able to download). Anyway, once Nexus has downloaded artifacts from the original repository for the first time (e.g. because a compilation of a pom that uses the proxy repository), creates its own indexes and those artifacts will become available also for local search.<br />
<br />
=== Dependency Versions ===<br />
<br />
ETICS builds ensure correct integration of gCube components, regardless of the versions their dependencies. It also ensures that components are released with fixed versions for their dependencies, i.e. ensures build reproducibility.<br />
<br />
Accordingly, developers can be liberal in the choice of dependency versions. i.e. rely on version ranges to:<br />
<br />
* minimise the requirements for version management;<br />
* be readily informed of any mis-alignment with their dependencies;<br />
<br />
As we start adopting Maven, the lower-bound of version ranges will be a SNAPSHOT version. For example, a dependency on the <code>streams</code> library can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>streams</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
</dependency><br />
</source><br />
<br />
With this dependency, a component will be soon notified of a non-retro compatible change that, contrary to conventions, <code>streams</code> may have introduced in, say, version <code>1.3.0</code> (typically the day after at the latest). The notification will be delivered in the IDE, assuming the availability of Maven integration (e.g. <code>m2e</code> in Eclipse). <br />
The component is in fact free to open the range further if it wants to be aligned with its dependencies across major versions.<br />
<br />
=== Common Dependencies ===<br />
<br />
Many gCube components depend on <code>gcf</code>, which is now available as a Maven component since version <code>1.4</code>. At the time of writing, the component has not been released yet. A dependency on it is then specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
The scope <code>provided</code> indicates that these dependency will be satisfied in the development environment (the gHN) and should not be included in GAR and SA archives.<br />
<br />
Some components depend on the libraries available in all gHNs, though they do not depend on <code>gcf</code> itself. The subset of these libraries that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called <code>ghn-core-runtime</code>. A dependency on this Jar can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ghn-core-runtime</artifactId><br />
<version>1.0.0</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Note: a component that depends on <code>gcf</code> indirectly depends on <code>gun-core-runtime</code>.<br />
<br />
Another common dependency is on the gCube gRS2, an Ant component that the build infrastructure make available in Maven repositories. A dependency on this component can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.execution</groupId><br />
<artifactId>grs2library</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
</dependency><br />
</source><br />
<br />
== Java Version ==<br />
<br />
In Maven the Java version of the source code and of the generated classes can be controlled using the configuration of the <code>maven-compiler-plugin</code>. E.g.:<br />
<syntaxhighlight lang="xml" highlight="5,6"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-compiler-plugin</artifactId><br />
<configuration><br />
<source>1.7</source><br />
<target>1.8</target><br />
</configuration><br />
</plugin><br />
</syntaxhighlight><br />
<br />
<br />
In gCube, this mechanism is exploited to enforce the usage of the same Java version for all components. In particular, the maven-parent defines the properties <code>java_version</code>, <code>maven.compiler.source</code> and <code>maven.compiler.target</code> and use these properties in the configuration of the <code>maven-compiler-plugin</code>.<br />
<syntaxhighlight lang="xml" highlight="15,16"><br />
...<br />
<properties><br />
<java_version>1.8</java_version><br />
<maven.compiler.source>${java_version}</maven.compiler.source><br />
<maven.compiler.target>${java_version}</maven.compiler.target><br />
...<br />
</properties><br />
<br />
...<br />
<br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-compiler-plugin</artifactId><br />
<configuration><br />
<target>${maven.compiler.target}</target><br />
<source>${maven.compiler.source}</source><br />
</configuration><br />
</plugin><br />
...<br />
</syntaxhighlight><br />
<br />
'''All the gCube components inherit this default configuration from <code>maven-parent</code> and it is not recommended to override it in the component's pom.xml'''.<br />
<br />
If it is really needed to override these values at component's pom.xml level, instead of specify a new configuration for the maven-compiler-plugin, the value of the property should be re-defined:<br />
<br />
<syntaxhighlight lang="xml" highlight="3"><br />
...<br />
<properties><br />
<maven.compiler.source>1.7</maven.compiler.source><br />
...<br />
</properties><br />
<br />
</syntaxhighlight><br />
<br />
<br />
This configuration is flexible enough to allow to customize the Java compiler source and target values depending on different factors in the integration builds (e.g. during ETICS builds gCore components are built with Java 7, all the others with Java 8).<br />
<br />
<br />
<br />
== Artifact Types ==<br />
<br />
There are four gCube-specific artifact types that are generated during the build of the gCube components. Each of them is summarized in the following subsections.<br />
<br />
=== Source Package ===<br />
The source package is an artifact with classifier <code>src</code> that contains the source code of the component plus the files in the distro directory (see [[Maven distro directory layout]] for more).<br />
<br />
Its descriptor is at [http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/build-utils/shared-assemblies/src/main/resources/assemblies/gcube-source-package.xml link].<br />
<br />
It is automatically generated if the <code>generateDistribution</code> property is ''true''.<br />
<br />
=== Uberjar ===<br />
<br />
The Uberjar is an artifact with classifier <code>uberjar</code> that contains the component classes plus all the jar files of the dependencies. This is not mandatory for all components, but only for those that needs to be deployed along with their dependenices.<br />
<br />
Its descriptor is at [http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/build-utils/shared-assemblies/src/main/resources/assemblies/gcube-uberjar.xml link].<br />
<br />
The generation of this package is '''not enabled by default'''. To enable it, bind its configuration to a build phase in the component pom.xml:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<executions><br />
<execution><br />
<id>make-uberjar</id><br />
<phase>install</phase><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
=== Service Archive (SA) ===<br />
Whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary <code>tar.gz</code> artefact with classifier <code>servicearchive</code>. Note that the SA of Maven components does ''not'' need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);<br />
<br />
* for each deployable artefact they produce (primary or secondary), they must include the Maven coordinates of the arteact in the corresponding package section of the gCube Software Profile. Specifying package dependencies in the Software Profile, on the other hand, is no longer required;<br />
<br />
* whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary <code>tar.gz</code> artefact with classifier <code>servicearchive</code>. Note that the SA of Maven components does ''not'' need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);<br />
<br />
<br />
The '''maven-parent''' defines a default descriptor to build a servicearchive ([http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/build-utils/shared-assemblies/src/main/resources/assemblies/gcube-default-servicearchive.xml link]). Please not that, while this fits most of the single module projects that meet the requirements in [[Maven distro directory layout]], for multi-module projects and/or projects that need to include custom files a custom servicearchive might be needed. In the latter case, the default assembly configuration cannot be used and a custom maven-assembly-plugin configuration must be defined in the component's pom.xml.<br />
<br />
The generation of the default servicearchvie is disabled by default. To enable it, bind its configuration to a build phase in the component pom.xml:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<executions><br />
<execution><br />
<id>make-servicearchive</id><br />
<phase>install</phase><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
=== Javadoc ===<br />
It is the default artifact (with classifier <code>javadoc</code>) generated by the <code>maven-javadoc-plugin</code>. It is automatically generated for all components.<br />
<br />
= Libraries =<br />
The <code>streams</code> library is a Maven component in gCube class <code>DataAccess</code>. We will use it as example of a library component in Maven.<br />
<br />
=== Profile ===<br />
<br />
The gCube Software profile of the library is as follows:<br />
<br />
<source lang="xml" highlight="14-18"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>Embedded Domain-Specific Language for Stream Transformations</Description><br />
<Class>DataAccess</Class><br />
<Name>streams</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>streams</Name><br />
<Version>1.0.0-SNAPSHOT</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>streams</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<Files><br />
<File>streams-1.0.0-SNAPSHOT.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the profile includes a single package and this package corresponds to the main build artefact of the component; <br />
* the whole profile and the package have usual gCube coordinates;<br />
* the package name is aligned with the Maven <code>artifactId</code> of the component;<br />
* the package version is aligned with the Maven <code>version</code> of the component;<br />
* the package includes <code>MavenCoordinates</code> that can be directly copied and pasted from the POM;<br />
* the package does '''not''' specify dependencies;<br />
* the package points to the main artefact of the Maven build;<br />
<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>streams</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>DataAccess</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<Files><br />
<File>${build.finalName}.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
=== Service Archive ===<br />
<br />
<code>streams</code> is responsible for generating its own SA as part of its Maven build. This requires dedicated logic in the POM but has the advantage that:<br />
<br />
* SA validity can be verified locally;<br />
* no SA configurations are required in ETICS;<br />
* the required build logic can be easily reused across components;<br />
<br />
<code>streams</code> uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation in the profile and other static files. The approach is best illustrated with a reference to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/streams sources]] of the <code>streams</code> library.<br />
<br />
<br />
As we discuss [[#Creating the ETICS Configurations|below]], <code>streams</code> has the final requirement to produce an interpolated profile during ETICS builds, outside the context of the SA.<br />
<code>streams</code> meets this requirement with the [http://maven.apache.org/plugins/maven-resources-plugin/ Maven Resources Plugin]. Again, consult the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/streams sources]] of the <code>streams</code> library to reuse this approach.<br />
<br />
= Services =<br />
The <code>tree-manager</code> service is a Maven component in gCube class <code>DataAccess</code>. <br />
<br />
== Project Structure ==<br />
<br />
Differently from a library, the service is associated with a number of distinguished build artefacts:<br />
<br />
* the service implementation (a Jar);<br />
* the stubs automatically generated from the WSDL interfaces (a Jar);<br />
* the archive required for deployment in gCore (a Gar);<br />
* the archive required for registration in the system (a SA);<br />
<br />
Producing these artefacts requires sharing a number of static and dynamic files (from profiles to WSDLs to pre-processed WSDLs). The usual Maven approach of mapping the artefacts onto different Maven projects is then genuinely difficult. It is instead more practical to use a single multi-module Maven project. The <code>tree-manager</code> uses this module structure:<br />
<br />
<pre><br />
tree-manager (parent module)------- (produces) --- (POM only)<br />
|<br />
|----- tree-manager-service (service impl module)------- (produces) --- jar, gar, SA<br />
| |<br />
| | (depends on)<br />
| |<br />
| \/<br />
|----- tree-manager-stubs (service stubs module)------- (produces) --- jar<br />
</pre><br />
<br />
As customary, the parent does not produce any artefact (is a POM-only project). It declares these coordinates in its POM:<br />
<br />
<source lang="xml"><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
<packaging>pom</packaging><br />
</source><br />
<br />
The parent physically includes the sub-modules;<br />
<br />
<source lang="xml"><br />
<modules><br />
<module>tree-manager-stubs</module><br />
<module>tree-manager-service</module><br />
</modules><br />
</source> <br />
<br />
Finally, the parent inherits from '''maven-parent''':<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
The submodules inherit from the parent:<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>tree-manager</artifactId><br />
<groupId>org.gcube.data.access</groupId><br />
<version>1.0.0-SNAPSHOT</version><br />
<relativePath>..</relativePath><br />
</parent><br />
</source><br />
<br />
and share its <code>version</code> and <code>groupId</code>. Their coordinates are then minimal:<br />
<br />
<source lang="xml"><br />
<artifactId>tree-manager-service</artifactId><br />
</source><br />
<br />
<source lang="xml"><br />
<artifactId>tree-manager-stubs</artifactId><br />
</source><br />
<br />
The <code>tree-manager-service</code> module produces a Jar of its own implementation, as well the GAR and the SA of the whole services as secondary artefacts with classifiers of, respectively, <code>gar</code> and <code>service archive</code>.<br />
<br />
The <code>tree-manager-stubs</code> module produces a Jar of its own implementation.<br />
<br />
Finally, the parent module coordinates the build of its sub-modules and centralises the declaration of common dependencies and configuration. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] to reuse this approach.<br />
<br />
== Generating Stubs and Gars ==<br />
<br />
The <code>tree-manager-stubs</code> uses the '''gCube Service Plugin''' to generate its classes from WSDL interfaces. <br />
<br />
Similarly, the <code>tree-manager-service</code> uses the plugin to generate its secondary GAR artefact. <br />
<br />
The configuration of the plugin is conveniently shared in the parent module.<br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] of the <code>streams</code> for details.<br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the service is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>DataAccess</Class><br />
<Name>tree-manager-service</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Main><br />
<Name>tree-manager-service</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager-service</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<GARArchive>tree-manager-service-1.0.0-SNAPSHOT.gar</GARArchive><br />
<PortType><br />
<Name>gcube/data/tm/binder</Name><br />
</PortType><br />
<PortType><br />
<Name>gcube/data/tm/reader</Name><br />
</PortType><br />
<PortType><br />
<Name>gcube/data/tm/writer</Name><br />
</PortType><br />
</Main><br />
<Software><br />
<Name>tree-manager-stubs</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager-stubs</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<Type>library</Type><br />
<Files><br />
<File>tree-manager-stubs-1.0.0-SNAPSHOT.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the profile includes two packages and this package corresponds to the main build artefacts of the submodules; <br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code>s of the submodules;<br />
* package versions are aligned with the Maven <code>version</code>s of the submodules;<br />
* package include <code>MavenCoordinates</code>s that can be directly copied and pasted from the POM of the submodules;<br />
* packages do '''not''' specify dependencies;<br />
* packages points to the main artefact of the Maven build of the submodules;<br />
<br />
<strike>Notice that, unlike libraries, services cannot easily use Maven variables in their profiles (as these artefacts are used for different purposes by different modules). Accordingly, developers needs to maintain versions used in profiles across releases. In particular, they need to:<br />
<br />
* replace SNAPSHOT versions with release versions in release branches;<br />
* advance SNAPSHOT versions in HEAD;</strike><br />
<br />
=== Profile Interpolation ===<br />
The interpolation of profiles is required in order to keep information in profile in sync with <code>pom.xml</code>, in particular versions and filenames (that contain versions) that change at each release.<br />
<br />
In order to generate <code>profile.xml</code> during build process you need to:<br />
# add in the service's parent <code>pom.xml</code> the invocation to maven-resources-plugin<br />
# place a ''template'' <code>profile.xml</code> in the <code>${distroDirectory}</code><br />
<br />
Invocation to <code>maven-resources-plugin</code> looks like the following (to be placed in the <plugins> section inside the <build> section of the pom):<br />
<source lang="xml"><br />
<!-- interpolates profiles and copies from distribution location to configuration <br />
location, where it is need for embedding into stub artifact, gar generation, <br />
and service archive. --><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-resources-plugin</artifactId><br />
<version>2.6</version><br />
<executions><br />
<execution><br />
<id>copy-profile</id><br />
<goals><br />
<goal>copy-resources</goal><br />
</goals><br />
<phase>process-resources</phase><br />
<configuration><br />
<outputDirectory>${configDirectory}</outputDirectory><br />
<resources><br />
<resource><br />
<directory>${distroDirectory}</directory><br />
<includes><br />
<include>profile.xml</include><br />
</includes><br />
<filtering>true</filtering><br />
</resource><br />
</resources><br />
</configuration><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
The ''template'' <code>profile.xml</code> looks like the following:<br />
<source lang="xml" highlight="12,16,18,22,26,30"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>TestClass</Class><br />
<Name>TestService</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Main><br />
<Name>TestService</Name><br />
<Version>${project.version}</Version><br />
<MavenCoordinates><br />
<groupId>org.example</groupId><br />
<artifactId>test-service</artifactId><br />
<version>${project.version}</version><br />
</MavenCoordinates><br />
<GARArchive>>test-service-${project.version}.gar</GARArchive><br />
</Main><br />
<Software><br />
<Name>test-stubs</Name><br />
<Version>${project.version}</Version><br />
<MavenCoordinates><br />
<groupId>org.example</groupId><br />
<artifactId>test-stubs</artifactId><br />
<version>${project.version}</version><br />
</MavenCoordinates><br />
<Type>library</Type><br />
<Files><br />
<File>test-stubs-${project.version}.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
In the example, property <code>${project.version}</code> will be substituted with the actual service's version at build-time. It is also possible, but not required, to parametrize also other values of <code>profile.xml</code> such as maven's coordinates and package names. You can see a more complex <code>profile.xml</code> ''template'' in the <code>tree-manager</code> project: http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-access/tree-manager/distro/profile.xml<br />
<br />
Maven will copy the final <code>profile.xml</code> in <code>${configDirectory}</code> during the <code>process-resources</code> phase of the build. From there it will be copied in the service-archive package.<br />
<br />
== Service Archive ==<br />
<br />
<code>tree-manager-service</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the module and its parent.<br />
<br />
As shown for [[#Libraries|libraries]], the service uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] for details.<br />
<br />
= Portlets =<br />
<br />
The <code>sample-portlet</code> portlet is a Maven based sample portlet based on GWT. Refer to the [http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources] for details.<br />
<br />
== Project Structure ==<br />
<br />
As for a library project, the portlet is associated with 2 distinguished build artefacts:<br />
<br />
* the archive required for deployment in a Liferay Container (a WAR);<br />
* the archive required for registration in the system (a SA);<br />
<br />
In order to generate automatically the project structure and the base pom, we start by customizing the maven archetype liferay-portlet-archetype v. 6.0.6.<br />
<br />
The generation of the project structure and the pom can be performed either from Eclipse ( by using the m2eclipse plugin) or by command line using the Maven SDK.<br />
<br />
On Eclipse the following steps must be followed:<br />
<br />
* FIle -> New -> Project<br />
* From the wizard select Maven->Maven Project<br />
* From the Select Archetype step, Select 'All Catalogs' from the Catalogs combobox and type 'portlet' on the Filter field.<br />
* Select the '''liferay-portlet-archetype''' version 6.0.6 archetype ( if does not appear on the list just unselect 'show the last version of the Archetype only")<br />
* In the next step you have to provide Maven coordinates for your project<br />
* groupid : <your group id> e.g org.gcube.portlets.admin<br />
* artifactId : <your artifactId> e.g. sample-admin-portlet<br />
* version : <your project version> e.g. 1.0.0-SNAPSHOT ( use always SNAPSHOT versions for development versions of a project)<br />
* package: <your portlet code package> e.g. org.gcube.portlets.admin.sample<br />
<br />
while from shell the following mvn command should be issued :<br />
<br />
<pre><br />
mvn archetype:generate \<br />
-DarchetypeArtifactId=liferay-theme-archetype \<br />
-DarchetypeGroupId=com.liferay.maven.archetypes \<br />
-DarchetypeVersion=6.0.6 \<br />
-DartifactId=<your artifactId> \<br />
-DgroupId=<your group id> \<br />
-Dversion=1.0.0-SNAPSHOT<br />
</pre><br />
<br />
Both approaches generate a project structure similar to the picture below:<br />
<br />
[[image:sample-portlet-project.png|Sample portlet project structure]]<br />
<br />
In order to customize the build procedure to the gCube needs we should open the pom.xml and apply the following changes:<br />
<br />
Configure the inheritance from '''maven-parent''' :<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
Configure some project properties:<br />
<br />
<source lang="xml"><br />
<properties><br />
<liferay.version>6.0.6</liferay.version><br />
<liferay.auto.deploy.dir>${system.CATALINA_HOME}/../deploy</liferay.auto.deploy.dir><br />
<distroDirectory>${project.basedir}/distro</distroDirectory><br />
<configDirectory>${project.basedir}/config</configDirectory><br />
</properties><br />
</source><br />
<br />
Add the plugin for the generation of the SA, which has to be included under the build plugins section:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<version>2.2</version><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
In the case of a GWT portlet the [http://mojo.codehaus.org/gwt-maven-plugin gwt-maven-plugin] should also be configured ( within build plugins section as well), by also selecting the proper gwt version ( in our example 2.4)<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.codehaus.mojo</groupId><br />
<artifactId>gwt-maven-plugin</artifactId><br />
<version>${gwt.version}</version><br />
<executions><br />
<execution><br />
<goals><br />
<goal>compile</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
The proper GWT version should be added to the pom properties :<br />
<br />
<source lang="xml"><br />
<gwt.version>2.4.0</gwt.version><br />
</source><br />
<br />
and GWT dependencies should be also configured:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-user</artifactId><br />
<version>${gwt.version}</version><br />
</dependency><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-servlet</artifactId><br />
<version>${gwt.version}</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet/ project sources]] for further details on the project configuration.<br />
<br />
== Generating Artifacts ==<br />
<br />
The <code>sample-portlet</code> uses the '''maven-compiler-plugin''' to compile the portlet code and the '''gwt-maven-plugin''' as well for GWT compilation.<br />
<br />
The generation of a war package can be triggered by typing :<br />
<br />
<pre><br />
mvn package <br />
</pre><br />
<br />
on your shell or running this build task within Eclipse.<br />
<br />
In both cases the war package will be generated under the ''target'' folder, e.g.:<br />
<br />
<pre><br />
target/sample-admin-portlet-1.0.0-SNAPSHOT.war<br />
</pre><br />
<br />
The generation of all project artifacts ( war, source jar, javadoc, SA) can be triggered by the build task :<br />
<br />
<pre><br />
mvn install <br />
</pre><br />
<br />
this will install as well all artifacts in your local repository, while the deployment of the artifacts to the remote gCube Maven repos can triggered by the build task:<br />
<br />
<pre><br />
mvn deploy <br />
</pre><br />
<br />
== Portlet dependencies ==<br />
<br />
Specific portlet dependencies ( e.g. GWT, service stubs ) should be packaged within the portlet war. All dependencies specified within the pom file without the ''provided'' scope will be copied inside the WEB-INF/lib folder at packaging time.<br />
<br />
For this reason all portlet specific dependencies should not specified that scope e.g:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.externals</groupId><br />
<artifactId>gwt-ext-patched</artifactId><br />
<version>2.0.5</version><br />
</dependency><br />
</source><br />
<br />
On the other hand the system dependencies ( the one contained within the [[GCube_Portal_Installation#Portal_Bundle_Libraries|Portal bundle]] ) should not be included inside the portlet war. As said in this case the ''provided'' scope should be specified, e.g.<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portal</groupId><br />
<artifactId>custom-portal-handler</artifactId><br />
<version>1.1.0-SNAPSHOT</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.4.0,1.5.0]</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
<br />
A subset of these libraries ( contained in a GHN) that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called <code>ghn-core-runtime</code>. A dependency on this Jar can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ghn-core-runtime</artifactId><br />
<version>1.0.0</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Note: a component that depends on <code>gcf</code> indirectly depends on <code>gun-core-runtime</code>.<br />
<br />
== Include gCube Guided Tour using Maven ==<br />
<br />
This particular component is not yet ported to maven so in order to be included in a maven based project it should be declared as system dependency as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portletsuser</groupId><br />
<artifactId>gcube-guidedtour-widget</artifactId><br />
<version>1.0.0</version><br />
<scope>system</scope><br />
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/gcube-guidedtour-widget.jar</systemPath> <br />
</dependency><br />
</source><br />
<br />
together with the gCube Widget Library:<br />
<br />
<source lang="xml"> <br />
<dependency><br />
<groupId>org.gcube.portlets.user</groupId><br />
<artifactId>gcubewidgets</artifactId><br />
<version>1.2.0</version><br />
<scope>system</scope><br />
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/gCube-Widgets-Library.jar</systemPath><br />
</dependency><br />
</source><br />
<br />
both components can be downloaded from the distribution repository [[http://grids16.eng.it:8080/BuildReport/browse/Recent_Builds]]. <br />
<br />
In case the portlet depends on the ''gcf'' component, some dependency resolution changes are required to your portlet in order to properly compile ( because of libraries clashes)<br />
<br />
The ghn-core-runtime component ( gcf dependency) should be excluded:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.5.1-SNAPSHOT,2.0.0)</version><br />
<scope>provided</scope><br />
<exclusions><br />
<exclusion><br />
<artifactId>ghn-core-runtime</artifactId><br />
<groupId>org.gcube.distribution</groupId><br />
</exclusion><br />
</exclusions><br />
</dependency><br />
</source><br />
<br />
<br />
and some new dependencies should be added:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>xerces</groupId><br />
<artifactId>xercesImpl</artifactId><br />
<version>2.9.1</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ws-addressing</artifactId><br />
<version>1.0</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>axis-patched</artifactId><br />
<version>1.2RC</version><br />
<scope>provided</scope><br />
</dependency><br />
<br />
</source><br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the portlet is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>PortletAdmin</Class><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.portles.admin</groupId><br />
<artifactId>sample-admin-portlet</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/sample-admin-portlet-1.0.0-SNAPSHOT.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code> of the project;<br />
* package versions are aligned with the Maven <code>version</code> of the project;<br />
* package include <code>MavenCoordinates</code> that can be directly copied and pasted from the POM of the project;<br />
* packages do '''not''' specify dependencies;<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>sample-portlet</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>PortletAdmin</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/${build.finalName}.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
== Service Archive ==<br />
<br />
<code>sample-portlet</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the project.<br />
<br />
As shown for [[#Libraries|libraries]], the portlet uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources]] for details.<br />
<br />
== Use Case: Create a new Mavenized GWT Portlet ==<br />
<br />
In order to create a new Mavenized GWT Portlet this detailed guide can be also followed [https://gcube.wiki.gcube-system.org/gcube/index.php/Create_a_new_Mavenized_gCube_GWT_Portlet_Guide]<br />
<br />
= Widgets =<br />
<br />
Falls in this category of components mainly GWT widgets or so called GWT Libraries, which can be used to package classes for mutualization and/or modularization. A Widget is a java archive (JAR) containing both classes and java sources for later GWT compilation and a gwt.xml module descriptor. <br />
<br />
In this section we refer to a sample Widget with maven structure (https://svn.research-infrastructures.eu/d4science/gcube/trunk/portal/mvn-widget-sample/)<br />
<br />
== Project Structure ==<br />
<br />
As for a other project, the portlet is associated with 2 distinguished build artefacts:<br />
<br />
* the archive required for deployment (a JAR);<br />
* the archive required for registration in the system (a SA);<br />
<br />
In order to generate automatically the project structure and the base pom, we start by customizing the maven archetype liferay-portlet-archetype v. 6.0.6.<br />
<br />
The generation of the project structure and the pom can be performed either from Eclipse ( by using the m2eclipse plugin) or by command line using the Maven SDK.<br />
<br />
On Eclipse the following steps must be followed:<br />
<br />
* FIle -> New -> Project<br />
* From the wizard select Maven->Maven Project<br />
* From the Select Archetype step, Select 'All Catalogs' from the Catalogs combobox and type 'portlet' on the Filter field.<br />
* Select the '''maven-archetype-quickstart''' version 1.1.0 archetype ( if does not appear on the list just unselect 'show the last version of the Archetype only")<br />
* In the next step you have to provide Maven coordinates for your project<br />
* groupid : <your group id> e.g org.gcube.portlets.admin<br />
* artifactId : <your artifactId> e.g. sample-admin-portlet<br />
* version : <your project version> e.g. 1.0.0-SNAPSHOT ( use always SNAPSHOT versions for development versions of a project)<br />
* package: <your portlet code package> e.g. org.gcube.portlets.admin.sample<br />
<br />
while from shell the following mvn command should be issued :<br />
<br />
<pre><br />
mvn archetype:generate \<br />
-DarchetypeArtifactId=maven-archetype-quickstart \<br />
-DarchetypeGroupId=com.liferay.maven.archetypes \<br />
-DarchetypeVersion=1.1.0 \<br />
-DartifactId=<your artifactId> \<br />
-DgroupId=<your group id> \<br />
-Dversion=1.0.0-SNAPSHOT<br />
</pre><br />
<br />
Both approaches generate a project structure similar to the picture below:<br />
<br />
[[image:sample-widget-project.png|Sample widget project structure]]<br />
<br />
In order to customize the build procedure to the gCube needs we should open the pom.xml and apply the following changes:<br />
<br />
Configure the inheritance from '''maven-parent''' :<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
Configure some project properties:<br />
<br />
<source lang="xml"><br />
<properties><br />
<distroDirectory>${project.basedir}/distro</distroDirectory><br />
<configDirectory>${project.basedir}/config</configDirectory><br />
</properties><br />
</source><br />
<br />
Add the plugin for the generation of the SA, which has to be included under the build plugins section:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<version>2.2</version><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
In the case of a GWT widget the [http://mojo.codehaus.org/gwt-maven-plugin gwt-maven-plugin] should also be configured ( within build plugins section as well), by also selecting the proper gwt version ( in our example 2.4)<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.codehaus.mojo</groupId><br />
<artifactId>gwt-maven-plugin</artifactId><br />
<version>${gwt.version}</version><br />
<executions><br />
<execution><br />
<goals><br />
<goal>compile</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
The proper GWT version should be added to the pom properties :<br />
<br />
<source lang="xml"><br />
<gwt.version>2.4.0</gwt.version><br />
</source><br />
<br />
and GWT dependencies should be also configured:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-user</artifactId><br />
<version>${gwt.version}</version><br />
</dependency><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-servlet</artifactId><br />
<version>${gwt.version}</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/mvn-widget-sample/ project sources]] for further details on the project configuration.<br />
<br />
== Generating Artifacts ==<br />
<br />
The <code>sample-portlet</code> uses the '''maven-compiler-plugin''' to compile the portlet code and the '''gwt-maven-plugin''' as well for GWT compilation.<br />
<br />
The generation of a war package can be triggered by typing :<br />
<br />
<pre><br />
mvn package <br />
</pre><br />
<br />
on your shell or running this build task within Eclipse.<br />
<br />
In both cases the war package will be generated under the ''target'' folder, e.g.:<br />
<br />
<pre><br />
target/sample-admin-portlet-1.0.0-SNAPSHOT.war<br />
</pre><br />
<br />
The generation of all project artifacts ( war, source jar, javadoc, SA) can be triggered by the build task :<br />
<br />
<pre><br />
mvn install <br />
</pre><br />
<br />
this will install as well all artifacts in your local repository, while the deployment of the artifacts to the remote gCube Maven repos can triggered by the build task:<br />
<br />
<pre><br />
mvn deploy <br />
</pre><br />
<br />
== Widget dependencies ==<br />
<br />
The system dependencies ( the one contained within the [[GCube_Portal_Installation#Portal_Bundle_Libraries|Portal bundle]] ) should not be included inside the portlet war. As said in this case the ''provided'' scope should be specified, e.g.<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portal</groupId><br />
<artifactId>custom-portal-handler</artifactId><br />
<version>1.1.0-SNAPSHOT</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.4.0,1.5.0]</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the widget is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>PortletAdmin</Class><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.portles.admin</groupId><br />
<artifactId>sample-admin-portlet</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/sample-admin-portlet-1.0.0-SNAPSHOT.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code> of the project;<br />
* package versions are aligned with the Maven <code>version</code> of the project;<br />
* package include <code>MavenCoordinates</code> that can be directly copied and pasted from the POM of the project;<br />
* packages do '''not''' specify dependencies;<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>sample-portlet</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>PortletAdmin</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/${build.finalName}.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
== Service Archive ==<br />
<br />
<code>sample-portlet</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the project.<br />
<br />
As shown for [[#Libraries|libraries]], the portlet uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources]] for details.</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Maven_distro_directory_layout&diff=29735Maven distro directory layout2017-10-30T12:09:45Z<p>Gabriele.giammatteo: /* descriptor.xml */</p>
<hr />
<div>'''Important''': The guidelines provided in this page have been adopted starting from gCube 3.10.1 (March 2016).<br />
<br />
The ''distro'' directory of gCube Maven components contains a set of files and templates used to generate files that will be included in the distribution packages of the component.<br />
<br />
The following sections describes the mandatory and optional files expected to be in this directory. However, ''distro'' directory can also contains other component-specific files required to be distributed in the component packages.<br />
<br />
The layout, the files and the filtering mechanism presented in this section work only for Maven components. They will not work with the '''Ant''' components. However, since <u>also Ant components must provide the mandatory files</u> and since Ant does not provide any support for filtering of files, developers must take care of keep the files updated manually.<br />
<br />
= Mandatory Files =<br />
The following sections describe the files that must be provided by ALL components.<br />
<br />
== README ==<br />
Each gCube component must include a README file that must be included in distribution packages of the component. The file briefly describe the component and the gCube system and provide references to the source code, binaries, documentation, authors and maintainers and to the software license.<br />
<br />
<br />
In order to have as much as possible uniform README files across components and to reduce the effort to maintain the file updated, a standardized template is provided. The template makes use of placeholders (delimited by <code>${...}</code>) that are replaced at build-time with the value of Maven properties. This approach allows to keep the same README in the source code while distributing files containing always updated information.<br />
<br />
<br />
By convention, the README file should be located under the <code>distro/</code> folder. It should follow this template (only minor changes to fits the component needs are allowed):<br />
<br />
<source lang="text" highlight="23,24,30,31,48,55,56"><br />
The gCube System - ${name}<br />
--------------------------------------------------<br />
<br />
${description}<br />
<br />
<br />
${gcube.description}<br />
<br />
${gcube.funding}<br />
<br />
<br />
Version<br />
--------------------------------------------------<br />
<br />
${version} (${buildDate})<br />
<br />
Please see the file named "changelog.xml" in this directory for the release notes.<br />
<br />
<br />
Authors<br />
--------------------------------------------------<br />
<br />
* John Doe (john.doe-AT-acme.org), ACME Inc.<br />
* Second Author(email-AT-example.org), Organization, Country<br />
<br />
<br />
Maintainers<br />
-----------<br />
<br />
* John Doe (john.doe-AT-acme.org), ACME Inc.<br />
* Second Maintainer(email-AT-example.org), Organization, Country<br />
<br />
<br />
Download information<br />
--------------------------------------------------<br />
<br />
Source code is available from SVN: <br />
${scm.url}<br />
<br />
Binaries can be downloaded from the gCube website: <br />
${gcube.website}<br />
<br />
<br />
Installation<br />
--------------------------------------------------<br />
<br />
Installation documentation is available on-line in the gCube Wiki:<br />
${gcube.wikiRoot}/[admin guide page name here] <br />
<br />
<br />
Documentation <br />
--------------------------------------------------<br />
<br />
Documentation is available on-line in the gCube Wiki:<br />
${gcube.wikiRoot}/[user guide page name here]<br />
${gcube.wikiRoot}/[developer guide page name here]<br />
<br />
<br />
Support <br />
--------------------------------------------------<br />
<br />
Bugs and support requests can be reported in the gCube issue tracking tool:<br />
${gcube.issueTracking}<br />
<br />
<br />
Licensing<br />
--------------------------------------------------<br />
<br />
This software is licensed under the terms you may find in the file named "LICENSE" in this directory.<br />
<br />
</source><br />
<br />
<br />
Only the <span style="background-color: #ffc">highlighted lines</span> in the template must be customized and replaced with appropriate data for the component before commit it in SVN. In any case developers are free to define additional properties in their <code>pom.xml</code> and use them in the README.<br />
<br />
'''Important note''': the <code>@...@</code> sequence is used by Maven as delimiter for properties (alternative to <code>${...}</code>), therefore it is recommended to avoid to use the <code>@</code> character otherwise Maven will not be able to correctly parse the file. It is suggested to replace <code>@</code> with <code>-AT-</code> in the e-mail addresses.<br />
<br />
<br />
All the placeholders delimited by <code>${...}</code> must correspond to Maven properties. There are two source of properties:<br />
* the component pom.xml. It is possible to access custom-defined properties in the section <code><properties></code> of the pom as well as default properties defined by Maven (e.g. <code>${version}</code>, <code>${name}</code>);<br />
* the gCube <code>maven-parent</code> (available [http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals/org/gcube/tools/maven-parent/ here]);<br />
The properties are replaced during the build by the [https://maven.apache.org/shared/maven-filtering/ filtering mechanism] used by the plugins that create the distribution packages (i.e. the [http://maven.apache.org/plugins/maven-assembly-plugin/ assembly plugin]).<br />
<br />
These are the properties used in the README template provided above:<br />
{| class="wikitable"<br />
! property<br />
! definition<br />
! description<br />
|-<br />
| ${name}<br />
| pom.xml<br />
| The value of <code><name></code> tag in the pom<br />
|-<br />
| ${description}<br />
| pom.xml<br />
| The value of <code><description></code> tag in the pom<br />
|-<br />
| ${gcube.description}<br />
| maven-parent<br />
| A brief description of gCube framework<br />
|-<br />
| ${gcube.funding}<br />
| maven-parent<br />
| The declaration of EU projects that funded the gCube development<br />
|-<br />
| ${version}<br />
| pom.xml<br />
| The value of <code><version></code> tag in the pom<br />
|-<br />
| ${buildDate}<br />
| maven-parent<br />
| The date of the component build. It is in the YYYY-MM-dd format (format defined in maven-parent)<br />
|-<br />
| ${scm.url}<br />
| pom.xml<br />
| The location of component source code in SVN. It must be defined in the pom in <code><scm><url>[location]</url></scm></code><br />
|-<br />
| ${gcube.website}<br />
| maven-parent<br />
| The website of gCube system: http://gcube-system.org/<br />
|-<br />
| ${gcube.wikiRoot}<br />
| maven-parent<br />
| The home page of the gCube wiki: https://wiki.gcube-system.org/gcube/index.php/<br />
|-<br />
| ${gcube.issueTracking}<br />
| maven-parent<br />
| The home page of the gCube Issue Tracker: https://support.d4science.org/projects/gcube/<br />
|}<br />
<br />
== LICENSE ==<br />
This file (referenced by the README file) contains the text of the license under which the software is distributed. In order to avoid massive update of this file in all components if in the future, the gCube license will change, the entire text of this file is provided by a Maven property defined in the maven-parent: <code>${gcube.license}</code><br />
<br />
<source lang="text"><br />
${gcube.license}<br />
</source><br />
<br />
<br />
Currently the license under which gCube is distributed is the EUPL v1.1 and the <code>${gcube.license}</code> is replaced with [https://joinup.ec.europa.eu/sites/default/files/ckeditor_files/files/eupl%20v%20%201%201%20-%20EN.txt.txt this text].<br />
<br />
<br />
== changelog.xml ==<br />
<br />
This file (referenced by the README) contains the list of all changes, fixes and new features for each release of the component.An example of changelog.xml file is:<br />
<br />
<source lang="text"><br />
<ReleaseNotes><br />
<Changeset component="org.gcube.messaging.consumer.1-2-0" date="2010-07-16"><br />
<Change>Extended the Monitoring Generic Resource with the possibility to specify site managers for each site domain</Change><br />
<Change>Notification are now filtered over domain</Change><br />
<Change>Added daily notification summary</Change><br />
</Changeset><br />
<Changeset component="org.gcube.messaging.consumer-library.1-1-2" date="2010-07-16"><br />
<Change>Added queries for NodeAccounting portlet</Change><br />
</Changeset><br />
</ReleaseNotes><br />
</source><br />
<br />
This file contribute to the generation of the full gCube release changelog. In order to correctly generate the final changelog, the value of ''component'' tag must match exactly the name of the ETICS Configuration, therefore it is wrong to put here <code>${name}</code> (unless the name of the component matches the name of the ETIC configuration.<br />
<br />
Further information can be found in the [[Changelog Generation procedure]] page.<br />
<br />
<br />
== profile.xml ==<br />
This is a gCube-specific file used by gCube SoftwareGateway and Deployer services to correctly register the software in the infrastructure and deploy it on gCube nodes. It specifies the gCube coordinates (Class, Name and Version) and references to the packages of the component. Before the adoption of Maven, it was used also to specify dependencies of the software. <br />
<br />
Since the adoption of Maven, the profile.xml became much simple thanks to the use of Maven properties and resource filtering. More information on how to properly write a profile.xml is reported in these section: [[Developing_gCube_Maven_Components#Profile_2|general profiles]], for <br />
[[Developing_gCube_Maven_Components#Profile_3|portlets]] and for [[Developing_gCube_Maven_Components#Profile_4|widgets]].<br />
<br />
<br />
== descriptor.xml ==<br />
<br />
'''ATTENTION: Sine November 2017, it is possible to use a default descriptor in common to all gCube components. If used, the component does not need to include this file any-more. Please read more on this [[Developing_gCube_Maven_Components#Service_Archive_.28SA.29|here]].'''<br />
<br />
This file is used by the Maven [http://maven.apache.org/plugins/maven-assembly-plugin/ Assembly plugin] to build the '''servicearchive.tar.gz'' artifact. This package contains the binary files of the component plus all the files described in this page. <br />
<br />
This is a generic descriptor.xml file (it may be customized to include other files needed to be distributed for the component):<br />
<br />
<pre lang="xml"><br />
<assembly<br />
xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0"<br />
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"<br />
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0 http://maven.apache.org/xsd/assembly-1.1.0.xsd"><br />
<id>servicearchive</id><br />
<formats><br />
<format>tar.gz</format><br />
</formats><br />
<baseDirectory>/</baseDirectory><br />
<fileSets><br />
<fileSet><br />
<directory>${distroDirectory}</directory><br />
<outputDirectory>/</outputDirectory><br />
<useDefaultExcludes>true</useDefaultExcludes><br />
<includes><br />
<include>README</include><br />
<include>LICENSE</include><br />
<include>changelog.xml</include><br />
<include>profile.xml</include><br />
</includes><br />
<fileMode>755</fileMode><br />
<filtered>true</filtered><br />
</fileSet><br />
</fileSets><br />
<files><br />
<file><br />
<source>target/${build.finalName}.${project.packaging}</source><br />
<outputDirectory>/${artifactId}</outputDirectory><br />
</file><br />
<br />
</files><br />
</assembly><br />
</pre><br />
<br />
<br />
Plase note that the Assembly plugin must be explicitly activated in the component pom.xml:<br />
<pre><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</pre><br />
<br />
= Deprecated files =<br />
The following sections describe the files that were used in the past but that are now deprecated because the information contained is available elsewhere. In order to allow a smoother transition, all the release and distribution scripts continue to support these files (otherwise explicitly mentioned).<br />
<br />
== MAINTAINERS ==<br />
The README template also include a section for maintainers. It is recommended to include names and contacts of maintainers directly in the README and to remove the MAINTAINERS file.<br />
<br />
== INSTALL ==<br />
This file should briefly provide instructions to install the component, however in most cases this information is available in the gCube wiki (Admin Guide). To avoid replication of information, it is recommended to remove this file completely and to specify in the README (Installation section) the link to the on-line documentation.<br />
<br />
== svnpath.txt ==<br />
This file was used to keep track of the path of the source code in the SVN. However this information is available in the ${scm.url} Maven property. Maven components can remove this file.</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Developing_gCube_Maven_Components&diff=29734Developing gCube Maven Components2017-10-30T12:03:32Z<p>Gabriele.giammatteo: /* Artifact Types */</p>
<hr />
<div>[[Category: Developer's Guide]]<br />
__TOC__<br />
<br />
= Common =<br />
<br />
== Maven Coordinates ==<br />
Maven components' coordinates must follow these conventions:<br />
<br />
* their <code>groupId</code> must be of the following form:<br />
<br />
<pre>org.gcube.<class></pre><br />
<br />
: where ''class'' identifies a functionally correlated set of components and helps a range of users (developers, administrator) to discover components in various UIs/reports used within the system and in tools. Logically, the class corresponds to the gCube class of the component's, as specified in its gCube Software Profile. However, more specific, or else cross-class groups may be agreed upon among developers. In all cases, the group should align with official Maven [http://maven.apache.org/guides/mini/guide-naming-conventions.html guidelines]. By agreement, for example, components in gCube class <code>DataAccess</code> declare a <code>groupId</code> of <code>org.gcube.data.access</code> and those in gCube class <code>DataTransfer</code> declare a <code>groupId</code> of <code>org.gcube.data.transfer</code>. On the other hand, also by agreement, components in gCube class <code>InformationSystem</code> declare a <code>groupId</code> of <code>org.gcube.informationsystem</code>.<br />
<br />
* their <code>artifactId</code> must follow Maven conventions (e.g. ''avoid capital letters or punctuation other than hyphens'').<br />
<br />
== Parent ==<br />
All gCube components must inherit from '''maven-parent'''. This ensure and promotes compliance with system-wide requirements, from the enforcement of minimal Java and Maven versions to generation and packaging of Javadoc documentation and component sources.<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
<br />
The maven-parent defines:<br />
* several system-wide properties like <code>gcube.license</code>, <code>gcube.wikiRoot</code>, <code>distroDirectory</code><br />
* the default Java version (see more in [[Developing_gCube_Maven_Components#Java_Version|Java Version]])<br />
* the execution of some plugins that run for every component like: <code>maven-javadoc-plugin</code>, <code>maven-source-plugin</code><br />
* the configuration of the <code>maven-assembly-plugin</code> to generate some gCube-specific packages: <code>servicearchive</code>, <code>uberjar</code> and <code>source-package</code> (see more in [[Developing_gCube_Maven_Components#Artifact_Types|Artifact Types]])<br />
* the logic to enable the generation of the source package<br />
<br />
The '''maven-parent''' is at http://maven.research-infrastructures.eu/nexus/service/local/repositories/gcube-externals/content/org/gcube/tools/maven-parent/1.0.0/maven-parent-1.0.0.pom<br />
<br />
== The distro folder ==<br />
This folder contains files that will be distributed along with the code in the distribution packages. Further details are in the [[Maven distro directory layout]] page.<br />
<br />
<br />
== Dependencies ==<br />
Service and libraries that depend on other gCube components (Maven-based or Ant-based) can find them in the gCube Maven Repositories: <code>gcube-snapshots</code> <strike>and <code>gcube-releases</code></strike>. <br />
<br />
=== gCube Maven Repositories ===<br />
As a one-off task, developers must configure these repositories in the <code>~/.m2/settings.xml</code> file of their local Maven installation, as follows:<br />
<br />
<source lang="xml"><br />
<settings xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"<br />
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd"><br />
<br />
<servers><br />
<server><br />
<id>gcube-snapshots</id><br />
<username>gcube-user</username><br />
<password>maven</password><br />
</server><br />
</servers><br />
<br />
<br />
<profiles><br />
<profile><br />
<id>gcube</id><br />
<repositories><br />
<repository><br />
<id>gcube-snapshots</id><br />
<name>gCube Snapshots</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
</snapshots><br />
</repository><br />
<repository><br />
<id>gcube-externals</id><br />
<name>gCube Externals</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals</url><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
</repository><br />
</repositories><br />
<br />
<pluginRepositories><br />
<pluginRepository><br />
<id>gcube-snapshots</id><br />
<name>gCube Snapshots</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
</snapshots><br />
</pluginRepository><br />
<pluginRepository><br />
<id>gcube-externals</id><br />
<name>gCube Externals</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals</url><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
</pluginRepository><br />
</pluginRepositories><br />
<br />
</profile><br />
</profiles><br />
<br />
<activeProfiles><br />
<activeProfile>gcube</activeProfile><br />
</activeProfiles><br />
</settings><br />
</source><br />
<br />
<br />
'''Note:Developers must not configure these repositories in their POMs''', only in <code>settings.xml</code>. <br />
<br />
This is to:<br />
* keep their projects independent from future changes in repository locations; <br />
* avoid interference with ETICS builds, where access to these repositories may comprise correct system integration;<br />
<br />
With this settings.xml, developers will have access on their local development environment to development versions (gcube-snapshots) and externals (gcube-externals). This is the correct behaviour for most of use cases. If access to staging or releases repositories is necessary (e.g. patching a component already released in the branch where dependencies versions in the release are not in snapshots), developer should modify his settings.xml to point to gcube-staging(releases) instead of gcube-snapshots.<br />
<br />
=== Third-party Repositories ===<br />
Using the the <code>settings.xml</code> aforementioned, any dependency declared in pom.xml that can be found in one of gCube repositories or in Maven Central will be resolved at build time and deployment time without any further configuration. Frequently it may happens that a gCube component needs software stored on a public Maven repository not in on of those repositories. In this case, Maven provides an easy solution for declaring additional repositories for dependency resolution by adding a <repositories> section in the pom.xml of the project. E.g.:<br />
<br />
<source lang="xml" highlight="3,4"><br />
<repositories><br />
<repository><br />
<id>[repository-id]</id><br />
<url>[repository URL]</url><br />
</repository><br />
</repositories><br />
</source><br />
<br />
Though technically possible to add any Maven repository in this way, in order to preserve the quality of gCube software it is required to check trustworthiness of its dependencies disallowing usage of certain external software. To enforce such a control over public Maven repositories, '''gCube components are allowed to use external repository not directly but only through a proxy repository''' (a.k.a. shadow repository) created on CNR's Nexus Repository Portal (http://maven.research-infrastructures.eu/nexus/).<br />
<br />
List of all repositories in Nexus Portal is available at http://maven.research-infrastructures.eu/nexus/index.html#view-repositories<br />
<br />
'''Note''': if a component uses a public Maven repository directly, it will work for building but would fail in deployment phase. In fact, Software Gateway is only able to download software accessible from Nexus.<br />
<br />
==== Request new proxy repository in Nexus ====<br />
Proxy Repositories on Nexus are created by the Release Manager. To request a new proxy a new ticket must be created.<br />
<br />
Details of ticket are:<br />
# open a ticket in tracker "Support" in gCube ticketing system (at https://support.d4science.org/projects/gcube/)<br />
# ticket's '''description''' must report:<br />
#* URL of repository to add<br />
#* organization which hosts the repository<br />
#* which artifacts are needed from that repository<br />
#* which components needs those artifacts<br />
# '''Sprint''' field should be set to the gCube release in which the component is expected to be released <br />
# '''Category''' field must report the gCube subsystem that needs the external repository<br />
# '''Assignee''' is the Release Manager<br />
<br />
An example of ticket for requesting a new repository mirror is reported in the figure below.<br />
[[Image:New_maven_mirror_repo.png|600px|center|Ticket for requesting a new Mirror Repository]]<br />
<br />
Release Manager will process the ticket checking that no issues are foreseen from the usage of software stored in the repository. If request is accepted, a new proxy repository is created and the repository's URL will be communicated publicly on gCube's developers mailinglist (a.k.a. TCOM mailinglist).<br />
<br />
Knowing the mirror repository URL, developer can use it adding following lines in pom.xml. E.g.<br />
<br />
<source lang="xml" highlight="3,4"><br />
<repositories><br />
<repository><br />
<id>gcube-ext-osgeo</id><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/osgeo/</url><br />
</repository><br />
</repositories><br />
</source><br />
<br />
'''Note''': it is possible that software stored on proxy repositories is not included in search results on Nexus. This happens for those repositories that does not have indexes (or have indexes in a format that Nexus is not able to download). Anyway, once Nexus has downloaded artifacts from the original repository for the first time (e.g. because a compilation of a pom that uses the proxy repository), creates its own indexes and those artifacts will become available also for local search.<br />
<br />
=== Dependency Versions ===<br />
<br />
ETICS builds ensure correct integration of gCube components, regardless of the versions their dependencies. It also ensures that components are released with fixed versions for their dependencies, i.e. ensures build reproducibility.<br />
<br />
Accordingly, developers can be liberal in the choice of dependency versions. i.e. rely on version ranges to:<br />
<br />
* minimise the requirements for version management;<br />
* be readily informed of any mis-alignment with their dependencies;<br />
<br />
As we start adopting Maven, the lower-bound of version ranges will be a SNAPSHOT version. For example, a dependency on the <code>streams</code> library can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>streams</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
</dependency><br />
</source><br />
<br />
With this dependency, a component will be soon notified of a non-retro compatible change that, contrary to conventions, <code>streams</code> may have introduced in, say, version <code>1.3.0</code> (typically the day after at the latest). The notification will be delivered in the IDE, assuming the availability of Maven integration (e.g. <code>m2e</code> in Eclipse). <br />
The component is in fact free to open the range further if it wants to be aligned with its dependencies across major versions.<br />
<br />
=== Common Dependencies ===<br />
<br />
Many gCube components depend on <code>gcf</code>, which is now available as a Maven component since version <code>1.4</code>. At the time of writing, the component has not been released yet. A dependency on it is then specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
The scope <code>provided</code> indicates that these dependency will be satisfied in the development environment (the gHN) and should not be included in GAR and SA archives.<br />
<br />
Some components depend on the libraries available in all gHNs, though they do not depend on <code>gcf</code> itself. The subset of these libraries that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called <code>ghn-core-runtime</code>. A dependency on this Jar can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ghn-core-runtime</artifactId><br />
<version>1.0.0</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Note: a component that depends on <code>gcf</code> indirectly depends on <code>gun-core-runtime</code>.<br />
<br />
Another common dependency is on the gCube gRS2, an Ant component that the build infrastructure make available in Maven repositories. A dependency on this component can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.execution</groupId><br />
<artifactId>grs2library</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
</dependency><br />
</source><br />
<br />
== Java Version ==<br />
<br />
In Maven the Java version of the source code and of the generated classes can be controlled using the configuration of the <code>maven-compiler-plugin</code>. E.g.:<br />
<syntaxhighlight lang="xml" highlight="5,6"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-compiler-plugin</artifactId><br />
<configuration><br />
<source>1.7</source><br />
<target>1.8</target><br />
</configuration><br />
</plugin><br />
</syntaxhighlight><br />
<br />
<br />
In gCube, this mechanism is exploited to enforce the usage of the same Java version for all components. In particular, the maven-parent defines the properties <code>java_version</code>, <code>maven.compiler.source</code> and <code>maven.compiler.target</code> and use these properties in the configuration of the <code>maven-compiler-plugin</code>.<br />
<syntaxhighlight lang="xml" highlight="15,16"><br />
...<br />
<properties><br />
<java_version>1.8</java_version><br />
<maven.compiler.source>${java_version}</maven.compiler.source><br />
<maven.compiler.target>${java_version}</maven.compiler.target><br />
...<br />
</properties><br />
<br />
...<br />
<br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-compiler-plugin</artifactId><br />
<configuration><br />
<target>${maven.compiler.target}</target><br />
<source>${maven.compiler.source}</source><br />
</configuration><br />
</plugin><br />
...<br />
</syntaxhighlight><br />
<br />
'''All the gCube components inherit this default configuration from <code>maven-parent</code> and it is not recommended to override it in the component's pom.xml'''.<br />
<br />
If it is really needed to override these values at component's pom.xml level, instead of specify a new configuration for the maven-compiler-plugin, the value of the property should be re-defined:<br />
<br />
<syntaxhighlight lang="xml" highlight="3"><br />
...<br />
<properties><br />
<maven.compiler.source>1.7</maven.compiler.source><br />
...<br />
</properties><br />
<br />
</syntaxhighlight><br />
<br />
<br />
This configuration is flexible enough to allow to customize the Java compiler source and target values depending on different factors in the integration builds (e.g. during ETICS builds gCore components are built with Java 7, all the others with Java 8).<br />
<br />
<br />
<br />
== Artifact Types ==<br />
<br />
There are four gCube-specific artifact types that are generated during the build of the gCube components. Each of them is summarized in the following subsections.<br />
<br />
=== Source Package ===<br />
The source package is an artifact with classifier <code>src</code> that contains the source code of the component plus the files in the distro directory (see [[Maven distro directory layout]] for more).<br />
<br />
Its descriptor is at [http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/build-utils/shared-assemblies/src/main/resources/assemblies/gcube-source-package.xml link].<br />
<br />
It is automatically generated if the <code>generateDistribution</code> property is ''true''.<br />
<br />
=== Uberjar ===<br />
<br />
The Uberjar is an artifact with classifier <code>uberjar</code> that contains the component classes plus all the jar files of the dependencies. This is not mandatory for all components, but only for those that needs to be deployed along with their dependenices.<br />
<br />
Its descriptor is at [http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/build-utils/shared-assemblies/src/main/resources/assemblies/gcube-uberjar.xml link].<br />
<br />
The generation of this package is not enabled by default. To enable it, bind its configuration to a build phase in the component pom.xml:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<executions><br />
<execution><br />
<id>make-uberjar</id><br />
<phase>install</phase><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
=== Service Archive (SA) ===<br />
Whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary <code>tar.gz</code> artefact with classifier <code>servicearchive</code>. Note that the SA of Maven components does ''not'' need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);<br />
<br />
* for each deployable artefact they produce (primary or secondary), they must include the Maven coordinates of the arteact in the corresponding package section of the gCube Software Profile. Specifying package dependencies in the Software Profile, on the other hand, is no longer required;<br />
<br />
* whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary <code>tar.gz</code> artefact with classifier <code>servicearchive</code>. Note that the SA of Maven components does ''not'' need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);<br />
<br />
<br />
The '''maven-parent''' defines a default descriptor to build a servicearchive ([http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/build-utils/shared-assemblies/src/main/resources/assemblies/gcube-default-servicearchive.xml link]). Please not that, while this fits most of the single module projects that meet the requirements in [[Maven distro directory layout]], for multi-module projects and/or projects that need to include custom files a custom servicearchive might be needed. In the latter case, the default assembly configuration cannot be used and a custom maven-assembly-plugin configuration must be defined in the component's pom.xml.<br />
<br />
The generation of the default servicearchvie is disabled by default. To enable it, bind its configuration to a build phase in the component pom.xml:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<executions><br />
<execution><br />
<id>make-servicearchive</id><br />
<phase>install</phase><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
=== Javadoc ===<br />
It is the default artifact (with classifier <code>javadoc</code>) generated by the <code>maven-javadoc-plugin</code>. It is automatically generated for all components.<br />
<br />
= Libraries =<br />
The <code>streams</code> library is a Maven component in gCube class <code>DataAccess</code>. We will use it as example of a library component in Maven.<br />
<br />
=== Profile ===<br />
<br />
The gCube Software profile of the library is as follows:<br />
<br />
<source lang="xml" highlight="14-18"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>Embedded Domain-Specific Language for Stream Transformations</Description><br />
<Class>DataAccess</Class><br />
<Name>streams</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>streams</Name><br />
<Version>1.0.0-SNAPSHOT</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>streams</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<Files><br />
<File>streams-1.0.0-SNAPSHOT.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the profile includes a single package and this package corresponds to the main build artefact of the component; <br />
* the whole profile and the package have usual gCube coordinates;<br />
* the package name is aligned with the Maven <code>artifactId</code> of the component;<br />
* the package version is aligned with the Maven <code>version</code> of the component;<br />
* the package includes <code>MavenCoordinates</code> that can be directly copied and pasted from the POM;<br />
* the package does '''not''' specify dependencies;<br />
* the package points to the main artefact of the Maven build;<br />
<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>streams</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>DataAccess</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<Files><br />
<File>${build.finalName}.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
=== Service Archive ===<br />
<br />
<code>streams</code> is responsible for generating its own SA as part of its Maven build. This requires dedicated logic in the POM but has the advantage that:<br />
<br />
* SA validity can be verified locally;<br />
* no SA configurations are required in ETICS;<br />
* the required build logic can be easily reused across components;<br />
<br />
<code>streams</code> uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation in the profile and other static files. The approach is best illustrated with a reference to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/streams sources]] of the <code>streams</code> library.<br />
<br />
<br />
As we discuss [[#Creating the ETICS Configurations|below]], <code>streams</code> has the final requirement to produce an interpolated profile during ETICS builds, outside the context of the SA.<br />
<code>streams</code> meets this requirement with the [http://maven.apache.org/plugins/maven-resources-plugin/ Maven Resources Plugin]. Again, consult the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/streams sources]] of the <code>streams</code> library to reuse this approach.<br />
<br />
= Services =<br />
The <code>tree-manager</code> service is a Maven component in gCube class <code>DataAccess</code>. <br />
<br />
== Project Structure ==<br />
<br />
Differently from a library, the service is associated with a number of distinguished build artefacts:<br />
<br />
* the service implementation (a Jar);<br />
* the stubs automatically generated from the WSDL interfaces (a Jar);<br />
* the archive required for deployment in gCore (a Gar);<br />
* the archive required for registration in the system (a SA);<br />
<br />
Producing these artefacts requires sharing a number of static and dynamic files (from profiles to WSDLs to pre-processed WSDLs). The usual Maven approach of mapping the artefacts onto different Maven projects is then genuinely difficult. It is instead more practical to use a single multi-module Maven project. The <code>tree-manager</code> uses this module structure:<br />
<br />
<pre><br />
tree-manager (parent module)------- (produces) --- (POM only)<br />
|<br />
|----- tree-manager-service (service impl module)------- (produces) --- jar, gar, SA<br />
| |<br />
| | (depends on)<br />
| |<br />
| \/<br />
|----- tree-manager-stubs (service stubs module)------- (produces) --- jar<br />
</pre><br />
<br />
As customary, the parent does not produce any artefact (is a POM-only project). It declares these coordinates in its POM:<br />
<br />
<source lang="xml"><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
<packaging>pom</packaging><br />
</source><br />
<br />
The parent physically includes the sub-modules;<br />
<br />
<source lang="xml"><br />
<modules><br />
<module>tree-manager-stubs</module><br />
<module>tree-manager-service</module><br />
</modules><br />
</source> <br />
<br />
Finally, the parent inherits from '''maven-parent''':<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
The submodules inherit from the parent:<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>tree-manager</artifactId><br />
<groupId>org.gcube.data.access</groupId><br />
<version>1.0.0-SNAPSHOT</version><br />
<relativePath>..</relativePath><br />
</parent><br />
</source><br />
<br />
and share its <code>version</code> and <code>groupId</code>. Their coordinates are then minimal:<br />
<br />
<source lang="xml"><br />
<artifactId>tree-manager-service</artifactId><br />
</source><br />
<br />
<source lang="xml"><br />
<artifactId>tree-manager-stubs</artifactId><br />
</source><br />
<br />
The <code>tree-manager-service</code> module produces a Jar of its own implementation, as well the GAR and the SA of the whole services as secondary artefacts with classifiers of, respectively, <code>gar</code> and <code>service archive</code>.<br />
<br />
The <code>tree-manager-stubs</code> module produces a Jar of its own implementation.<br />
<br />
Finally, the parent module coordinates the build of its sub-modules and centralises the declaration of common dependencies and configuration. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] to reuse this approach.<br />
<br />
== Generating Stubs and Gars ==<br />
<br />
The <code>tree-manager-stubs</code> uses the '''gCube Service Plugin''' to generate its classes from WSDL interfaces. <br />
<br />
Similarly, the <code>tree-manager-service</code> uses the plugin to generate its secondary GAR artefact. <br />
<br />
The configuration of the plugin is conveniently shared in the parent module.<br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] of the <code>streams</code> for details.<br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the service is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>DataAccess</Class><br />
<Name>tree-manager-service</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Main><br />
<Name>tree-manager-service</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager-service</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<GARArchive>tree-manager-service-1.0.0-SNAPSHOT.gar</GARArchive><br />
<PortType><br />
<Name>gcube/data/tm/binder</Name><br />
</PortType><br />
<PortType><br />
<Name>gcube/data/tm/reader</Name><br />
</PortType><br />
<PortType><br />
<Name>gcube/data/tm/writer</Name><br />
</PortType><br />
</Main><br />
<Software><br />
<Name>tree-manager-stubs</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager-stubs</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<Type>library</Type><br />
<Files><br />
<File>tree-manager-stubs-1.0.0-SNAPSHOT.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the profile includes two packages and this package corresponds to the main build artefacts of the submodules; <br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code>s of the submodules;<br />
* package versions are aligned with the Maven <code>version</code>s of the submodules;<br />
* package include <code>MavenCoordinates</code>s that can be directly copied and pasted from the POM of the submodules;<br />
* packages do '''not''' specify dependencies;<br />
* packages points to the main artefact of the Maven build of the submodules;<br />
<br />
<strike>Notice that, unlike libraries, services cannot easily use Maven variables in their profiles (as these artefacts are used for different purposes by different modules). Accordingly, developers needs to maintain versions used in profiles across releases. In particular, they need to:<br />
<br />
* replace SNAPSHOT versions with release versions in release branches;<br />
* advance SNAPSHOT versions in HEAD;</strike><br />
<br />
=== Profile Interpolation ===<br />
The interpolation of profiles is required in order to keep information in profile in sync with <code>pom.xml</code>, in particular versions and filenames (that contain versions) that change at each release.<br />
<br />
In order to generate <code>profile.xml</code> during build process you need to:<br />
# add in the service's parent <code>pom.xml</code> the invocation to maven-resources-plugin<br />
# place a ''template'' <code>profile.xml</code> in the <code>${distroDirectory}</code><br />
<br />
Invocation to <code>maven-resources-plugin</code> looks like the following (to be placed in the <plugins> section inside the <build> section of the pom):<br />
<source lang="xml"><br />
<!-- interpolates profiles and copies from distribution location to configuration <br />
location, where it is need for embedding into stub artifact, gar generation, <br />
and service archive. --><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-resources-plugin</artifactId><br />
<version>2.6</version><br />
<executions><br />
<execution><br />
<id>copy-profile</id><br />
<goals><br />
<goal>copy-resources</goal><br />
</goals><br />
<phase>process-resources</phase><br />
<configuration><br />
<outputDirectory>${configDirectory}</outputDirectory><br />
<resources><br />
<resource><br />
<directory>${distroDirectory}</directory><br />
<includes><br />
<include>profile.xml</include><br />
</includes><br />
<filtering>true</filtering><br />
</resource><br />
</resources><br />
</configuration><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
The ''template'' <code>profile.xml</code> looks like the following:<br />
<source lang="xml" highlight="12,16,18,22,26,30"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>TestClass</Class><br />
<Name>TestService</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Main><br />
<Name>TestService</Name><br />
<Version>${project.version}</Version><br />
<MavenCoordinates><br />
<groupId>org.example</groupId><br />
<artifactId>test-service</artifactId><br />
<version>${project.version}</version><br />
</MavenCoordinates><br />
<GARArchive>>test-service-${project.version}.gar</GARArchive><br />
</Main><br />
<Software><br />
<Name>test-stubs</Name><br />
<Version>${project.version}</Version><br />
<MavenCoordinates><br />
<groupId>org.example</groupId><br />
<artifactId>test-stubs</artifactId><br />
<version>${project.version}</version><br />
</MavenCoordinates><br />
<Type>library</Type><br />
<Files><br />
<File>test-stubs-${project.version}.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
In the example, property <code>${project.version}</code> will be substituted with the actual service's version at build-time. It is also possible, but not required, to parametrize also other values of <code>profile.xml</code> such as maven's coordinates and package names. You can see a more complex <code>profile.xml</code> ''template'' in the <code>tree-manager</code> project: http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-access/tree-manager/distro/profile.xml<br />
<br />
Maven will copy the final <code>profile.xml</code> in <code>${configDirectory}</code> during the <code>process-resources</code> phase of the build. From there it will be copied in the service-archive package.<br />
<br />
== Service Archive ==<br />
<br />
<code>tree-manager-service</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the module and its parent.<br />
<br />
As shown for [[#Libraries|libraries]], the service uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] for details.<br />
<br />
= Portlets =<br />
<br />
The <code>sample-portlet</code> portlet is a Maven based sample portlet based on GWT. Refer to the [http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources] for details.<br />
<br />
== Project Structure ==<br />
<br />
As for a library project, the portlet is associated with 2 distinguished build artefacts:<br />
<br />
* the archive required for deployment in a Liferay Container (a WAR);<br />
* the archive required for registration in the system (a SA);<br />
<br />
In order to generate automatically the project structure and the base pom, we start by customizing the maven archetype liferay-portlet-archetype v. 6.0.6.<br />
<br />
The generation of the project structure and the pom can be performed either from Eclipse ( by using the m2eclipse plugin) or by command line using the Maven SDK.<br />
<br />
On Eclipse the following steps must be followed:<br />
<br />
* FIle -> New -> Project<br />
* From the wizard select Maven->Maven Project<br />
* From the Select Archetype step, Select 'All Catalogs' from the Catalogs combobox and type 'portlet' on the Filter field.<br />
* Select the '''liferay-portlet-archetype''' version 6.0.6 archetype ( if does not appear on the list just unselect 'show the last version of the Archetype only")<br />
* In the next step you have to provide Maven coordinates for your project<br />
* groupid : <your group id> e.g org.gcube.portlets.admin<br />
* artifactId : <your artifactId> e.g. sample-admin-portlet<br />
* version : <your project version> e.g. 1.0.0-SNAPSHOT ( use always SNAPSHOT versions for development versions of a project)<br />
* package: <your portlet code package> e.g. org.gcube.portlets.admin.sample<br />
<br />
while from shell the following mvn command should be issued :<br />
<br />
<pre><br />
mvn archetype:generate \<br />
-DarchetypeArtifactId=liferay-theme-archetype \<br />
-DarchetypeGroupId=com.liferay.maven.archetypes \<br />
-DarchetypeVersion=6.0.6 \<br />
-DartifactId=<your artifactId> \<br />
-DgroupId=<your group id> \<br />
-Dversion=1.0.0-SNAPSHOT<br />
</pre><br />
<br />
Both approaches generate a project structure similar to the picture below:<br />
<br />
[[image:sample-portlet-project.png|Sample portlet project structure]]<br />
<br />
In order to customize the build procedure to the gCube needs we should open the pom.xml and apply the following changes:<br />
<br />
Configure the inheritance from '''maven-parent''' :<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
Configure some project properties:<br />
<br />
<source lang="xml"><br />
<properties><br />
<liferay.version>6.0.6</liferay.version><br />
<liferay.auto.deploy.dir>${system.CATALINA_HOME}/../deploy</liferay.auto.deploy.dir><br />
<distroDirectory>${project.basedir}/distro</distroDirectory><br />
<configDirectory>${project.basedir}/config</configDirectory><br />
</properties><br />
</source><br />
<br />
Add the plugin for the generation of the SA, which has to be included under the build plugins section:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<version>2.2</version><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
In the case of a GWT portlet the [http://mojo.codehaus.org/gwt-maven-plugin gwt-maven-plugin] should also be configured ( within build plugins section as well), by also selecting the proper gwt version ( in our example 2.4)<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.codehaus.mojo</groupId><br />
<artifactId>gwt-maven-plugin</artifactId><br />
<version>${gwt.version}</version><br />
<executions><br />
<execution><br />
<goals><br />
<goal>compile</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
The proper GWT version should be added to the pom properties :<br />
<br />
<source lang="xml"><br />
<gwt.version>2.4.0</gwt.version><br />
</source><br />
<br />
and GWT dependencies should be also configured:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-user</artifactId><br />
<version>${gwt.version}</version><br />
</dependency><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-servlet</artifactId><br />
<version>${gwt.version}</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet/ project sources]] for further details on the project configuration.<br />
<br />
== Generating Artifacts ==<br />
<br />
The <code>sample-portlet</code> uses the '''maven-compiler-plugin''' to compile the portlet code and the '''gwt-maven-plugin''' as well for GWT compilation.<br />
<br />
The generation of a war package can be triggered by typing :<br />
<br />
<pre><br />
mvn package <br />
</pre><br />
<br />
on your shell or running this build task within Eclipse.<br />
<br />
In both cases the war package will be generated under the ''target'' folder, e.g.:<br />
<br />
<pre><br />
target/sample-admin-portlet-1.0.0-SNAPSHOT.war<br />
</pre><br />
<br />
The generation of all project artifacts ( war, source jar, javadoc, SA) can be triggered by the build task :<br />
<br />
<pre><br />
mvn install <br />
</pre><br />
<br />
this will install as well all artifacts in your local repository, while the deployment of the artifacts to the remote gCube Maven repos can triggered by the build task:<br />
<br />
<pre><br />
mvn deploy <br />
</pre><br />
<br />
== Portlet dependencies ==<br />
<br />
Specific portlet dependencies ( e.g. GWT, service stubs ) should be packaged within the portlet war. All dependencies specified within the pom file without the ''provided'' scope will be copied inside the WEB-INF/lib folder at packaging time.<br />
<br />
For this reason all portlet specific dependencies should not specified that scope e.g:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.externals</groupId><br />
<artifactId>gwt-ext-patched</artifactId><br />
<version>2.0.5</version><br />
</dependency><br />
</source><br />
<br />
On the other hand the system dependencies ( the one contained within the [[GCube_Portal_Installation#Portal_Bundle_Libraries|Portal bundle]] ) should not be included inside the portlet war. As said in this case the ''provided'' scope should be specified, e.g.<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portal</groupId><br />
<artifactId>custom-portal-handler</artifactId><br />
<version>1.1.0-SNAPSHOT</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.4.0,1.5.0]</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
<br />
A subset of these libraries ( contained in a GHN) that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called <code>ghn-core-runtime</code>. A dependency on this Jar can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ghn-core-runtime</artifactId><br />
<version>1.0.0</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Note: a component that depends on <code>gcf</code> indirectly depends on <code>gun-core-runtime</code>.<br />
<br />
== Include gCube Guided Tour using Maven ==<br />
<br />
This particular component is not yet ported to maven so in order to be included in a maven based project it should be declared as system dependency as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portletsuser</groupId><br />
<artifactId>gcube-guidedtour-widget</artifactId><br />
<version>1.0.0</version><br />
<scope>system</scope><br />
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/gcube-guidedtour-widget.jar</systemPath> <br />
</dependency><br />
</source><br />
<br />
together with the gCube Widget Library:<br />
<br />
<source lang="xml"> <br />
<dependency><br />
<groupId>org.gcube.portlets.user</groupId><br />
<artifactId>gcubewidgets</artifactId><br />
<version>1.2.0</version><br />
<scope>system</scope><br />
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/gCube-Widgets-Library.jar</systemPath><br />
</dependency><br />
</source><br />
<br />
both components can be downloaded from the distribution repository [[http://grids16.eng.it:8080/BuildReport/browse/Recent_Builds]]. <br />
<br />
In case the portlet depends on the ''gcf'' component, some dependency resolution changes are required to your portlet in order to properly compile ( because of libraries clashes)<br />
<br />
The ghn-core-runtime component ( gcf dependency) should be excluded:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.5.1-SNAPSHOT,2.0.0)</version><br />
<scope>provided</scope><br />
<exclusions><br />
<exclusion><br />
<artifactId>ghn-core-runtime</artifactId><br />
<groupId>org.gcube.distribution</groupId><br />
</exclusion><br />
</exclusions><br />
</dependency><br />
</source><br />
<br />
<br />
and some new dependencies should be added:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>xerces</groupId><br />
<artifactId>xercesImpl</artifactId><br />
<version>2.9.1</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ws-addressing</artifactId><br />
<version>1.0</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>axis-patched</artifactId><br />
<version>1.2RC</version><br />
<scope>provided</scope><br />
</dependency><br />
<br />
</source><br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the portlet is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>PortletAdmin</Class><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.portles.admin</groupId><br />
<artifactId>sample-admin-portlet</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/sample-admin-portlet-1.0.0-SNAPSHOT.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code> of the project;<br />
* package versions are aligned with the Maven <code>version</code> of the project;<br />
* package include <code>MavenCoordinates</code> that can be directly copied and pasted from the POM of the project;<br />
* packages do '''not''' specify dependencies;<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>sample-portlet</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>PortletAdmin</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/${build.finalName}.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
== Service Archive ==<br />
<br />
<code>sample-portlet</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the project.<br />
<br />
As shown for [[#Libraries|libraries]], the portlet uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources]] for details.<br />
<br />
== Use Case: Create a new Mavenized GWT Portlet ==<br />
<br />
In order to create a new Mavenized GWT Portlet this detailed guide can be also followed [https://gcube.wiki.gcube-system.org/gcube/index.php/Create_a_new_Mavenized_gCube_GWT_Portlet_Guide]<br />
<br />
= Widgets =<br />
<br />
Falls in this category of components mainly GWT widgets or so called GWT Libraries, which can be used to package classes for mutualization and/or modularization. A Widget is a java archive (JAR) containing both classes and java sources for later GWT compilation and a gwt.xml module descriptor. <br />
<br />
In this section we refer to a sample Widget with maven structure (https://svn.research-infrastructures.eu/d4science/gcube/trunk/portal/mvn-widget-sample/)<br />
<br />
== Project Structure ==<br />
<br />
As for a other project, the portlet is associated with 2 distinguished build artefacts:<br />
<br />
* the archive required for deployment (a JAR);<br />
* the archive required for registration in the system (a SA);<br />
<br />
In order to generate automatically the project structure and the base pom, we start by customizing the maven archetype liferay-portlet-archetype v. 6.0.6.<br />
<br />
The generation of the project structure and the pom can be performed either from Eclipse ( by using the m2eclipse plugin) or by command line using the Maven SDK.<br />
<br />
On Eclipse the following steps must be followed:<br />
<br />
* FIle -> New -> Project<br />
* From the wizard select Maven->Maven Project<br />
* From the Select Archetype step, Select 'All Catalogs' from the Catalogs combobox and type 'portlet' on the Filter field.<br />
* Select the '''maven-archetype-quickstart''' version 1.1.0 archetype ( if does not appear on the list just unselect 'show the last version of the Archetype only")<br />
* In the next step you have to provide Maven coordinates for your project<br />
* groupid : <your group id> e.g org.gcube.portlets.admin<br />
* artifactId : <your artifactId> e.g. sample-admin-portlet<br />
* version : <your project version> e.g. 1.0.0-SNAPSHOT ( use always SNAPSHOT versions for development versions of a project)<br />
* package: <your portlet code package> e.g. org.gcube.portlets.admin.sample<br />
<br />
while from shell the following mvn command should be issued :<br />
<br />
<pre><br />
mvn archetype:generate \<br />
-DarchetypeArtifactId=maven-archetype-quickstart \<br />
-DarchetypeGroupId=com.liferay.maven.archetypes \<br />
-DarchetypeVersion=1.1.0 \<br />
-DartifactId=<your artifactId> \<br />
-DgroupId=<your group id> \<br />
-Dversion=1.0.0-SNAPSHOT<br />
</pre><br />
<br />
Both approaches generate a project structure similar to the picture below:<br />
<br />
[[image:sample-widget-project.png|Sample widget project structure]]<br />
<br />
In order to customize the build procedure to the gCube needs we should open the pom.xml and apply the following changes:<br />
<br />
Configure the inheritance from '''maven-parent''' :<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
Configure some project properties:<br />
<br />
<source lang="xml"><br />
<properties><br />
<distroDirectory>${project.basedir}/distro</distroDirectory><br />
<configDirectory>${project.basedir}/config</configDirectory><br />
</properties><br />
</source><br />
<br />
Add the plugin for the generation of the SA, which has to be included under the build plugins section:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<version>2.2</version><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
In the case of a GWT widget the [http://mojo.codehaus.org/gwt-maven-plugin gwt-maven-plugin] should also be configured ( within build plugins section as well), by also selecting the proper gwt version ( in our example 2.4)<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.codehaus.mojo</groupId><br />
<artifactId>gwt-maven-plugin</artifactId><br />
<version>${gwt.version}</version><br />
<executions><br />
<execution><br />
<goals><br />
<goal>compile</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
The proper GWT version should be added to the pom properties :<br />
<br />
<source lang="xml"><br />
<gwt.version>2.4.0</gwt.version><br />
</source><br />
<br />
and GWT dependencies should be also configured:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-user</artifactId><br />
<version>${gwt.version}</version><br />
</dependency><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-servlet</artifactId><br />
<version>${gwt.version}</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/mvn-widget-sample/ project sources]] for further details on the project configuration.<br />
<br />
== Generating Artifacts ==<br />
<br />
The <code>sample-portlet</code> uses the '''maven-compiler-plugin''' to compile the portlet code and the '''gwt-maven-plugin''' as well for GWT compilation.<br />
<br />
The generation of a war package can be triggered by typing :<br />
<br />
<pre><br />
mvn package <br />
</pre><br />
<br />
on your shell or running this build task within Eclipse.<br />
<br />
In both cases the war package will be generated under the ''target'' folder, e.g.:<br />
<br />
<pre><br />
target/sample-admin-portlet-1.0.0-SNAPSHOT.war<br />
</pre><br />
<br />
The generation of all project artifacts ( war, source jar, javadoc, SA) can be triggered by the build task :<br />
<br />
<pre><br />
mvn install <br />
</pre><br />
<br />
this will install as well all artifacts in your local repository, while the deployment of the artifacts to the remote gCube Maven repos can triggered by the build task:<br />
<br />
<pre><br />
mvn deploy <br />
</pre><br />
<br />
== Widget dependencies ==<br />
<br />
The system dependencies ( the one contained within the [[GCube_Portal_Installation#Portal_Bundle_Libraries|Portal bundle]] ) should not be included inside the portlet war. As said in this case the ''provided'' scope should be specified, e.g.<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portal</groupId><br />
<artifactId>custom-portal-handler</artifactId><br />
<version>1.1.0-SNAPSHOT</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.4.0,1.5.0]</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the widget is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>PortletAdmin</Class><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.portles.admin</groupId><br />
<artifactId>sample-admin-portlet</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/sample-admin-portlet-1.0.0-SNAPSHOT.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code> of the project;<br />
* package versions are aligned with the Maven <code>version</code> of the project;<br />
* package include <code>MavenCoordinates</code> that can be directly copied and pasted from the POM of the project;<br />
* packages do '''not''' specify dependencies;<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>sample-portlet</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>PortletAdmin</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/${build.finalName}.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
== Service Archive ==<br />
<br />
<code>sample-portlet</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the project.<br />
<br />
As shown for [[#Libraries|libraries]], the portlet uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources]] for details.</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Developing_gCube_Maven_Components&diff=29733Developing gCube Maven Components2017-10-30T11:40:05Z<p>Gabriele.giammatteo: /* Common */</p>
<hr />
<div>[[Category: Developer's Guide]]<br />
__TOC__<br />
<br />
= Common =<br />
<br />
== Maven Coordinates ==<br />
Maven components' coordinates must follow these conventions:<br />
<br />
* their <code>groupId</code> must be of the following form:<br />
<br />
<pre>org.gcube.<class></pre><br />
<br />
: where ''class'' identifies a functionally correlated set of components and helps a range of users (developers, administrator) to discover components in various UIs/reports used within the system and in tools. Logically, the class corresponds to the gCube class of the component's, as specified in its gCube Software Profile. However, more specific, or else cross-class groups may be agreed upon among developers. In all cases, the group should align with official Maven [http://maven.apache.org/guides/mini/guide-naming-conventions.html guidelines]. By agreement, for example, components in gCube class <code>DataAccess</code> declare a <code>groupId</code> of <code>org.gcube.data.access</code> and those in gCube class <code>DataTransfer</code> declare a <code>groupId</code> of <code>org.gcube.data.transfer</code>. On the other hand, also by agreement, components in gCube class <code>InformationSystem</code> declare a <code>groupId</code> of <code>org.gcube.informationsystem</code>.<br />
<br />
* their <code>artifactId</code> must follow Maven conventions (e.g. ''avoid capital letters or punctuation other than hyphens'').<br />
<br />
== Parent ==<br />
All gCube components must inherit from '''maven-parent'''. This ensure and promotes compliance with system-wide requirements, from the enforcement of minimal Java and Maven versions to generation and packaging of Javadoc documentation and component sources.<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
<br />
The maven-parent defines:<br />
* several system-wide properties like <code>gcube.license</code>, <code>gcube.wikiRoot</code>, <code>distroDirectory</code><br />
* the default Java version (see more in [[Developing_gCube_Maven_Components#Java_Version|Java Version]])<br />
* the execution of some plugins that run for every component like: <code>maven-javadoc-plugin</code>, <code>maven-source-plugin</code><br />
* the configuration of the <code>maven-assembly-plugin</code> to generate some gCube-specific packages: <code>servicearchive</code>, <code>uberjar</code> and <code>source-package</code> (see more in [[Developing_gCube_Maven_Components#Artifact_Types|Artifact Types]])<br />
* the logic to enable the generation of the source package<br />
<br />
The '''maven-parent''' is at http://maven.research-infrastructures.eu/nexus/service/local/repositories/gcube-externals/content/org/gcube/tools/maven-parent/1.0.0/maven-parent-1.0.0.pom<br />
<br />
== The distro folder ==<br />
This folder contains files that will be distributed along with the code in the distribution packages. Further details are in the [[Maven distro directory layout]] page.<br />
<br />
<br />
== Dependencies ==<br />
Service and libraries that depend on other gCube components (Maven-based or Ant-based) can find them in the gCube Maven Repositories: <code>gcube-snapshots</code> <strike>and <code>gcube-releases</code></strike>. <br />
<br />
=== gCube Maven Repositories ===<br />
As a one-off task, developers must configure these repositories in the <code>~/.m2/settings.xml</code> file of their local Maven installation, as follows:<br />
<br />
<source lang="xml"><br />
<settings xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"<br />
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd"><br />
<br />
<servers><br />
<server><br />
<id>gcube-snapshots</id><br />
<username>gcube-user</username><br />
<password>maven</password><br />
</server><br />
</servers><br />
<br />
<br />
<profiles><br />
<profile><br />
<id>gcube</id><br />
<repositories><br />
<repository><br />
<id>gcube-snapshots</id><br />
<name>gCube Snapshots</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
</snapshots><br />
</repository><br />
<repository><br />
<id>gcube-externals</id><br />
<name>gCube Externals</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals</url><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
</repository><br />
</repositories><br />
<br />
<pluginRepositories><br />
<pluginRepository><br />
<id>gcube-snapshots</id><br />
<name>gCube Snapshots</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
</snapshots><br />
</pluginRepository><br />
<pluginRepository><br />
<id>gcube-externals</id><br />
<name>gCube Externals</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals</url><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
</pluginRepository><br />
</pluginRepositories><br />
<br />
</profile><br />
</profiles><br />
<br />
<activeProfiles><br />
<activeProfile>gcube</activeProfile><br />
</activeProfiles><br />
</settings><br />
</source><br />
<br />
<br />
'''Note:Developers must not configure these repositories in their POMs''', only in <code>settings.xml</code>. <br />
<br />
This is to:<br />
* keep their projects independent from future changes in repository locations; <br />
* avoid interference with ETICS builds, where access to these repositories may comprise correct system integration;<br />
<br />
With this settings.xml, developers will have access on their local development environment to development versions (gcube-snapshots) and externals (gcube-externals). This is the correct behaviour for most of use cases. If access to staging or releases repositories is necessary (e.g. patching a component already released in the branch where dependencies versions in the release are not in snapshots), developer should modify his settings.xml to point to gcube-staging(releases) instead of gcube-snapshots.<br />
<br />
=== Third-party Repositories ===<br />
Using the the <code>settings.xml</code> aforementioned, any dependency declared in pom.xml that can be found in one of gCube repositories or in Maven Central will be resolved at build time and deployment time without any further configuration. Frequently it may happens that a gCube component needs software stored on a public Maven repository not in on of those repositories. In this case, Maven provides an easy solution for declaring additional repositories for dependency resolution by adding a <repositories> section in the pom.xml of the project. E.g.:<br />
<br />
<source lang="xml" highlight="3,4"><br />
<repositories><br />
<repository><br />
<id>[repository-id]</id><br />
<url>[repository URL]</url><br />
</repository><br />
</repositories><br />
</source><br />
<br />
Though technically possible to add any Maven repository in this way, in order to preserve the quality of gCube software it is required to check trustworthiness of its dependencies disallowing usage of certain external software. To enforce such a control over public Maven repositories, '''gCube components are allowed to use external repository not directly but only through a proxy repository''' (a.k.a. shadow repository) created on CNR's Nexus Repository Portal (http://maven.research-infrastructures.eu/nexus/).<br />
<br />
List of all repositories in Nexus Portal is available at http://maven.research-infrastructures.eu/nexus/index.html#view-repositories<br />
<br />
'''Note''': if a component uses a public Maven repository directly, it will work for building but would fail in deployment phase. In fact, Software Gateway is only able to download software accessible from Nexus.<br />
<br />
==== Request new proxy repository in Nexus ====<br />
Proxy Repositories on Nexus are created by the Release Manager. To request a new proxy a new ticket must be created.<br />
<br />
Details of ticket are:<br />
# open a ticket in tracker "Support" in gCube ticketing system (at https://support.d4science.org/projects/gcube/)<br />
# ticket's '''description''' must report:<br />
#* URL of repository to add<br />
#* organization which hosts the repository<br />
#* which artifacts are needed from that repository<br />
#* which components needs those artifacts<br />
# '''Sprint''' field should be set to the gCube release in which the component is expected to be released <br />
# '''Category''' field must report the gCube subsystem that needs the external repository<br />
# '''Assignee''' is the Release Manager<br />
<br />
An example of ticket for requesting a new repository mirror is reported in the figure below.<br />
[[Image:New_maven_mirror_repo.png|600px|center|Ticket for requesting a new Mirror Repository]]<br />
<br />
Release Manager will process the ticket checking that no issues are foreseen from the usage of software stored in the repository. If request is accepted, a new proxy repository is created and the repository's URL will be communicated publicly on gCube's developers mailinglist (a.k.a. TCOM mailinglist).<br />
<br />
Knowing the mirror repository URL, developer can use it adding following lines in pom.xml. E.g.<br />
<br />
<source lang="xml" highlight="3,4"><br />
<repositories><br />
<repository><br />
<id>gcube-ext-osgeo</id><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/osgeo/</url><br />
</repository><br />
</repositories><br />
</source><br />
<br />
'''Note''': it is possible that software stored on proxy repositories is not included in search results on Nexus. This happens for those repositories that does not have indexes (or have indexes in a format that Nexus is not able to download). Anyway, once Nexus has downloaded artifacts from the original repository for the first time (e.g. because a compilation of a pom that uses the proxy repository), creates its own indexes and those artifacts will become available also for local search.<br />
<br />
=== Dependency Versions ===<br />
<br />
ETICS builds ensure correct integration of gCube components, regardless of the versions their dependencies. It also ensures that components are released with fixed versions for their dependencies, i.e. ensures build reproducibility.<br />
<br />
Accordingly, developers can be liberal in the choice of dependency versions. i.e. rely on version ranges to:<br />
<br />
* minimise the requirements for version management;<br />
* be readily informed of any mis-alignment with their dependencies;<br />
<br />
As we start adopting Maven, the lower-bound of version ranges will be a SNAPSHOT version. For example, a dependency on the <code>streams</code> library can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>streams</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
</dependency><br />
</source><br />
<br />
With this dependency, a component will be soon notified of a non-retro compatible change that, contrary to conventions, <code>streams</code> may have introduced in, say, version <code>1.3.0</code> (typically the day after at the latest). The notification will be delivered in the IDE, assuming the availability of Maven integration (e.g. <code>m2e</code> in Eclipse). <br />
The component is in fact free to open the range further if it wants to be aligned with its dependencies across major versions.<br />
<br />
=== Common Dependencies ===<br />
<br />
Many gCube components depend on <code>gcf</code>, which is now available as a Maven component since version <code>1.4</code>. At the time of writing, the component has not been released yet. A dependency on it is then specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
The scope <code>provided</code> indicates that these dependency will be satisfied in the development environment (the gHN) and should not be included in GAR and SA archives.<br />
<br />
Some components depend on the libraries available in all gHNs, though they do not depend on <code>gcf</code> itself. The subset of these libraries that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called <code>ghn-core-runtime</code>. A dependency on this Jar can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ghn-core-runtime</artifactId><br />
<version>1.0.0</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Note: a component that depends on <code>gcf</code> indirectly depends on <code>gun-core-runtime</code>.<br />
<br />
Another common dependency is on the gCube gRS2, an Ant component that the build infrastructure make available in Maven repositories. A dependency on this component can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.execution</groupId><br />
<artifactId>grs2library</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
</dependency><br />
</source><br />
<br />
== Java Version ==<br />
<br />
In Maven the Java version of the source code and of the generated classes can be controlled using the configuration of the <code>maven-compiler-plugin</code>. E.g.:<br />
<syntaxhighlight lang="xml" highlight="5,6"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-compiler-plugin</artifactId><br />
<configuration><br />
<source>1.7</source><br />
<target>1.8</target><br />
</configuration><br />
</plugin><br />
</syntaxhighlight><br />
<br />
<br />
In gCube, this mechanism is exploited to enforce the usage of the same Java version for all components. In particular, the maven-parent defines the properties <code>java_version</code>, <code>maven.compiler.source</code> and <code>maven.compiler.target</code> and use these properties in the configuration of the <code>maven-compiler-plugin</code>.<br />
<syntaxhighlight lang="xml" highlight="15,16"><br />
...<br />
<properties><br />
<java_version>1.8</java_version><br />
<maven.compiler.source>${java_version}</maven.compiler.source><br />
<maven.compiler.target>${java_version}</maven.compiler.target><br />
...<br />
</properties><br />
<br />
...<br />
<br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-compiler-plugin</artifactId><br />
<configuration><br />
<target>${maven.compiler.target}</target><br />
<source>${maven.compiler.source}</source><br />
</configuration><br />
</plugin><br />
...<br />
</syntaxhighlight><br />
<br />
'''All the gCube components inherit this default configuration from <code>maven-parent</code> and it is not recommended to override it in the component's pom.xml'''.<br />
<br />
If it is really needed to override these values at component's pom.xml level, instead of specify a new configuration for the maven-compiler-plugin, the value of the property should be re-defined:<br />
<br />
<syntaxhighlight lang="xml" highlight="3"><br />
...<br />
<properties><br />
<maven.compiler.source>1.7</maven.compiler.source><br />
...<br />
</properties><br />
<br />
</syntaxhighlight><br />
<br />
<br />
This configuration is flexible enough to allow to customize the Java compiler source and target values depending on different factors in the integration builds (e.g. during ETICS builds gCore components are built with Java 7, all the others with Java 8).<br />
<br />
<br />
<br />
== Artifact Types ==<br />
<br />
=== Service Archive (SA) ===<br />
Whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary <code>tar.gz</code> artefact with classifier <code>servicearchive</code>. Note that the SA of Maven components does ''not'' need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);<br />
<br />
* for each deployable artefact they produce (primary or secondary), they must include the Maven coordinates of the arteact in the corresponding package section of the gCube Software Profile. Specifying package dependencies in the Software Profile, on the other hand, is no longer required;<br />
<br />
* whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary <code>tar.gz</code> artefact with classifier <code>servicearchive</code>. Note that the SA of Maven components does ''not'' need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);<br />
<br />
= Libraries =<br />
The <code>streams</code> library is a Maven component in gCube class <code>DataAccess</code>. We will use it as example of a library component in Maven.<br />
<br />
=== Profile ===<br />
<br />
The gCube Software profile of the library is as follows:<br />
<br />
<source lang="xml" highlight="14-18"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>Embedded Domain-Specific Language for Stream Transformations</Description><br />
<Class>DataAccess</Class><br />
<Name>streams</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>streams</Name><br />
<Version>1.0.0-SNAPSHOT</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>streams</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<Files><br />
<File>streams-1.0.0-SNAPSHOT.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the profile includes a single package and this package corresponds to the main build artefact of the component; <br />
* the whole profile and the package have usual gCube coordinates;<br />
* the package name is aligned with the Maven <code>artifactId</code> of the component;<br />
* the package version is aligned with the Maven <code>version</code> of the component;<br />
* the package includes <code>MavenCoordinates</code> that can be directly copied and pasted from the POM;<br />
* the package does '''not''' specify dependencies;<br />
* the package points to the main artefact of the Maven build;<br />
<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>streams</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>DataAccess</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<Files><br />
<File>${build.finalName}.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
=== Service Archive ===<br />
<br />
<code>streams</code> is responsible for generating its own SA as part of its Maven build. This requires dedicated logic in the POM but has the advantage that:<br />
<br />
* SA validity can be verified locally;<br />
* no SA configurations are required in ETICS;<br />
* the required build logic can be easily reused across components;<br />
<br />
<code>streams</code> uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation in the profile and other static files. The approach is best illustrated with a reference to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/streams sources]] of the <code>streams</code> library.<br />
<br />
<br />
As we discuss [[#Creating the ETICS Configurations|below]], <code>streams</code> has the final requirement to produce an interpolated profile during ETICS builds, outside the context of the SA.<br />
<code>streams</code> meets this requirement with the [http://maven.apache.org/plugins/maven-resources-plugin/ Maven Resources Plugin]. Again, consult the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/streams sources]] of the <code>streams</code> library to reuse this approach.<br />
<br />
= Services =<br />
The <code>tree-manager</code> service is a Maven component in gCube class <code>DataAccess</code>. <br />
<br />
== Project Structure ==<br />
<br />
Differently from a library, the service is associated with a number of distinguished build artefacts:<br />
<br />
* the service implementation (a Jar);<br />
* the stubs automatically generated from the WSDL interfaces (a Jar);<br />
* the archive required for deployment in gCore (a Gar);<br />
* the archive required for registration in the system (a SA);<br />
<br />
Producing these artefacts requires sharing a number of static and dynamic files (from profiles to WSDLs to pre-processed WSDLs). The usual Maven approach of mapping the artefacts onto different Maven projects is then genuinely difficult. It is instead more practical to use a single multi-module Maven project. The <code>tree-manager</code> uses this module structure:<br />
<br />
<pre><br />
tree-manager (parent module)------- (produces) --- (POM only)<br />
|<br />
|----- tree-manager-service (service impl module)------- (produces) --- jar, gar, SA<br />
| |<br />
| | (depends on)<br />
| |<br />
| \/<br />
|----- tree-manager-stubs (service stubs module)------- (produces) --- jar<br />
</pre><br />
<br />
As customary, the parent does not produce any artefact (is a POM-only project). It declares these coordinates in its POM:<br />
<br />
<source lang="xml"><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
<packaging>pom</packaging><br />
</source><br />
<br />
The parent physically includes the sub-modules;<br />
<br />
<source lang="xml"><br />
<modules><br />
<module>tree-manager-stubs</module><br />
<module>tree-manager-service</module><br />
</modules><br />
</source> <br />
<br />
Finally, the parent inherits from '''maven-parent''':<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
The submodules inherit from the parent:<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>tree-manager</artifactId><br />
<groupId>org.gcube.data.access</groupId><br />
<version>1.0.0-SNAPSHOT</version><br />
<relativePath>..</relativePath><br />
</parent><br />
</source><br />
<br />
and share its <code>version</code> and <code>groupId</code>. Their coordinates are then minimal:<br />
<br />
<source lang="xml"><br />
<artifactId>tree-manager-service</artifactId><br />
</source><br />
<br />
<source lang="xml"><br />
<artifactId>tree-manager-stubs</artifactId><br />
</source><br />
<br />
The <code>tree-manager-service</code> module produces a Jar of its own implementation, as well the GAR and the SA of the whole services as secondary artefacts with classifiers of, respectively, <code>gar</code> and <code>service archive</code>.<br />
<br />
The <code>tree-manager-stubs</code> module produces a Jar of its own implementation.<br />
<br />
Finally, the parent module coordinates the build of its sub-modules and centralises the declaration of common dependencies and configuration. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] to reuse this approach.<br />
<br />
== Generating Stubs and Gars ==<br />
<br />
The <code>tree-manager-stubs</code> uses the '''gCube Service Plugin''' to generate its classes from WSDL interfaces. <br />
<br />
Similarly, the <code>tree-manager-service</code> uses the plugin to generate its secondary GAR artefact. <br />
<br />
The configuration of the plugin is conveniently shared in the parent module.<br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] of the <code>streams</code> for details.<br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the service is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>DataAccess</Class><br />
<Name>tree-manager-service</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Main><br />
<Name>tree-manager-service</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager-service</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<GARArchive>tree-manager-service-1.0.0-SNAPSHOT.gar</GARArchive><br />
<PortType><br />
<Name>gcube/data/tm/binder</Name><br />
</PortType><br />
<PortType><br />
<Name>gcube/data/tm/reader</Name><br />
</PortType><br />
<PortType><br />
<Name>gcube/data/tm/writer</Name><br />
</PortType><br />
</Main><br />
<Software><br />
<Name>tree-manager-stubs</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager-stubs</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<Type>library</Type><br />
<Files><br />
<File>tree-manager-stubs-1.0.0-SNAPSHOT.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the profile includes two packages and this package corresponds to the main build artefacts of the submodules; <br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code>s of the submodules;<br />
* package versions are aligned with the Maven <code>version</code>s of the submodules;<br />
* package include <code>MavenCoordinates</code>s that can be directly copied and pasted from the POM of the submodules;<br />
* packages do '''not''' specify dependencies;<br />
* packages points to the main artefact of the Maven build of the submodules;<br />
<br />
<strike>Notice that, unlike libraries, services cannot easily use Maven variables in their profiles (as these artefacts are used for different purposes by different modules). Accordingly, developers needs to maintain versions used in profiles across releases. In particular, they need to:<br />
<br />
* replace SNAPSHOT versions with release versions in release branches;<br />
* advance SNAPSHOT versions in HEAD;</strike><br />
<br />
=== Profile Interpolation ===<br />
The interpolation of profiles is required in order to keep information in profile in sync with <code>pom.xml</code>, in particular versions and filenames (that contain versions) that change at each release.<br />
<br />
In order to generate <code>profile.xml</code> during build process you need to:<br />
# add in the service's parent <code>pom.xml</code> the invocation to maven-resources-plugin<br />
# place a ''template'' <code>profile.xml</code> in the <code>${distroDirectory}</code><br />
<br />
Invocation to <code>maven-resources-plugin</code> looks like the following (to be placed in the <plugins> section inside the <build> section of the pom):<br />
<source lang="xml"><br />
<!-- interpolates profiles and copies from distribution location to configuration <br />
location, where it is need for embedding into stub artifact, gar generation, <br />
and service archive. --><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-resources-plugin</artifactId><br />
<version>2.6</version><br />
<executions><br />
<execution><br />
<id>copy-profile</id><br />
<goals><br />
<goal>copy-resources</goal><br />
</goals><br />
<phase>process-resources</phase><br />
<configuration><br />
<outputDirectory>${configDirectory}</outputDirectory><br />
<resources><br />
<resource><br />
<directory>${distroDirectory}</directory><br />
<includes><br />
<include>profile.xml</include><br />
</includes><br />
<filtering>true</filtering><br />
</resource><br />
</resources><br />
</configuration><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
The ''template'' <code>profile.xml</code> looks like the following:<br />
<source lang="xml" highlight="12,16,18,22,26,30"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>TestClass</Class><br />
<Name>TestService</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Main><br />
<Name>TestService</Name><br />
<Version>${project.version}</Version><br />
<MavenCoordinates><br />
<groupId>org.example</groupId><br />
<artifactId>test-service</artifactId><br />
<version>${project.version}</version><br />
</MavenCoordinates><br />
<GARArchive>>test-service-${project.version}.gar</GARArchive><br />
</Main><br />
<Software><br />
<Name>test-stubs</Name><br />
<Version>${project.version}</Version><br />
<MavenCoordinates><br />
<groupId>org.example</groupId><br />
<artifactId>test-stubs</artifactId><br />
<version>${project.version}</version><br />
</MavenCoordinates><br />
<Type>library</Type><br />
<Files><br />
<File>test-stubs-${project.version}.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
In the example, property <code>${project.version}</code> will be substituted with the actual service's version at build-time. It is also possible, but not required, to parametrize also other values of <code>profile.xml</code> such as maven's coordinates and package names. You can see a more complex <code>profile.xml</code> ''template'' in the <code>tree-manager</code> project: http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-access/tree-manager/distro/profile.xml<br />
<br />
Maven will copy the final <code>profile.xml</code> in <code>${configDirectory}</code> during the <code>process-resources</code> phase of the build. From there it will be copied in the service-archive package.<br />
<br />
== Service Archive ==<br />
<br />
<code>tree-manager-service</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the module and its parent.<br />
<br />
As shown for [[#Libraries|libraries]], the service uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] for details.<br />
<br />
= Portlets =<br />
<br />
The <code>sample-portlet</code> portlet is a Maven based sample portlet based on GWT. Refer to the [http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources] for details.<br />
<br />
== Project Structure ==<br />
<br />
As for a library project, the portlet is associated with 2 distinguished build artefacts:<br />
<br />
* the archive required for deployment in a Liferay Container (a WAR);<br />
* the archive required for registration in the system (a SA);<br />
<br />
In order to generate automatically the project structure and the base pom, we start by customizing the maven archetype liferay-portlet-archetype v. 6.0.6.<br />
<br />
The generation of the project structure and the pom can be performed either from Eclipse ( by using the m2eclipse plugin) or by command line using the Maven SDK.<br />
<br />
On Eclipse the following steps must be followed:<br />
<br />
* FIle -> New -> Project<br />
* From the wizard select Maven->Maven Project<br />
* From the Select Archetype step, Select 'All Catalogs' from the Catalogs combobox and type 'portlet' on the Filter field.<br />
* Select the '''liferay-portlet-archetype''' version 6.0.6 archetype ( if does not appear on the list just unselect 'show the last version of the Archetype only")<br />
* In the next step you have to provide Maven coordinates for your project<br />
* groupid : <your group id> e.g org.gcube.portlets.admin<br />
* artifactId : <your artifactId> e.g. sample-admin-portlet<br />
* version : <your project version> e.g. 1.0.0-SNAPSHOT ( use always SNAPSHOT versions for development versions of a project)<br />
* package: <your portlet code package> e.g. org.gcube.portlets.admin.sample<br />
<br />
while from shell the following mvn command should be issued :<br />
<br />
<pre><br />
mvn archetype:generate \<br />
-DarchetypeArtifactId=liferay-theme-archetype \<br />
-DarchetypeGroupId=com.liferay.maven.archetypes \<br />
-DarchetypeVersion=6.0.6 \<br />
-DartifactId=<your artifactId> \<br />
-DgroupId=<your group id> \<br />
-Dversion=1.0.0-SNAPSHOT<br />
</pre><br />
<br />
Both approaches generate a project structure similar to the picture below:<br />
<br />
[[image:sample-portlet-project.png|Sample portlet project structure]]<br />
<br />
In order to customize the build procedure to the gCube needs we should open the pom.xml and apply the following changes:<br />
<br />
Configure the inheritance from '''maven-parent''' :<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
Configure some project properties:<br />
<br />
<source lang="xml"><br />
<properties><br />
<liferay.version>6.0.6</liferay.version><br />
<liferay.auto.deploy.dir>${system.CATALINA_HOME}/../deploy</liferay.auto.deploy.dir><br />
<distroDirectory>${project.basedir}/distro</distroDirectory><br />
<configDirectory>${project.basedir}/config</configDirectory><br />
</properties><br />
</source><br />
<br />
Add the plugin for the generation of the SA, which has to be included under the build plugins section:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<version>2.2</version><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
In the case of a GWT portlet the [http://mojo.codehaus.org/gwt-maven-plugin gwt-maven-plugin] should also be configured ( within build plugins section as well), by also selecting the proper gwt version ( in our example 2.4)<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.codehaus.mojo</groupId><br />
<artifactId>gwt-maven-plugin</artifactId><br />
<version>${gwt.version}</version><br />
<executions><br />
<execution><br />
<goals><br />
<goal>compile</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
The proper GWT version should be added to the pom properties :<br />
<br />
<source lang="xml"><br />
<gwt.version>2.4.0</gwt.version><br />
</source><br />
<br />
and GWT dependencies should be also configured:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-user</artifactId><br />
<version>${gwt.version}</version><br />
</dependency><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-servlet</artifactId><br />
<version>${gwt.version}</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet/ project sources]] for further details on the project configuration.<br />
<br />
== Generating Artifacts ==<br />
<br />
The <code>sample-portlet</code> uses the '''maven-compiler-plugin''' to compile the portlet code and the '''gwt-maven-plugin''' as well for GWT compilation.<br />
<br />
The generation of a war package can be triggered by typing :<br />
<br />
<pre><br />
mvn package <br />
</pre><br />
<br />
on your shell or running this build task within Eclipse.<br />
<br />
In both cases the war package will be generated under the ''target'' folder, e.g.:<br />
<br />
<pre><br />
target/sample-admin-portlet-1.0.0-SNAPSHOT.war<br />
</pre><br />
<br />
The generation of all project artifacts ( war, source jar, javadoc, SA) can be triggered by the build task :<br />
<br />
<pre><br />
mvn install <br />
</pre><br />
<br />
this will install as well all artifacts in your local repository, while the deployment of the artifacts to the remote gCube Maven repos can triggered by the build task:<br />
<br />
<pre><br />
mvn deploy <br />
</pre><br />
<br />
== Portlet dependencies ==<br />
<br />
Specific portlet dependencies ( e.g. GWT, service stubs ) should be packaged within the portlet war. All dependencies specified within the pom file without the ''provided'' scope will be copied inside the WEB-INF/lib folder at packaging time.<br />
<br />
For this reason all portlet specific dependencies should not specified that scope e.g:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.externals</groupId><br />
<artifactId>gwt-ext-patched</artifactId><br />
<version>2.0.5</version><br />
</dependency><br />
</source><br />
<br />
On the other hand the system dependencies ( the one contained within the [[GCube_Portal_Installation#Portal_Bundle_Libraries|Portal bundle]] ) should not be included inside the portlet war. As said in this case the ''provided'' scope should be specified, e.g.<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portal</groupId><br />
<artifactId>custom-portal-handler</artifactId><br />
<version>1.1.0-SNAPSHOT</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.4.0,1.5.0]</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
<br />
A subset of these libraries ( contained in a GHN) that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called <code>ghn-core-runtime</code>. A dependency on this Jar can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ghn-core-runtime</artifactId><br />
<version>1.0.0</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Note: a component that depends on <code>gcf</code> indirectly depends on <code>gun-core-runtime</code>.<br />
<br />
== Include gCube Guided Tour using Maven ==<br />
<br />
This particular component is not yet ported to maven so in order to be included in a maven based project it should be declared as system dependency as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portletsuser</groupId><br />
<artifactId>gcube-guidedtour-widget</artifactId><br />
<version>1.0.0</version><br />
<scope>system</scope><br />
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/gcube-guidedtour-widget.jar</systemPath> <br />
</dependency><br />
</source><br />
<br />
together with the gCube Widget Library:<br />
<br />
<source lang="xml"> <br />
<dependency><br />
<groupId>org.gcube.portlets.user</groupId><br />
<artifactId>gcubewidgets</artifactId><br />
<version>1.2.0</version><br />
<scope>system</scope><br />
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/gCube-Widgets-Library.jar</systemPath><br />
</dependency><br />
</source><br />
<br />
both components can be downloaded from the distribution repository [[http://grids16.eng.it:8080/BuildReport/browse/Recent_Builds]]. <br />
<br />
In case the portlet depends on the ''gcf'' component, some dependency resolution changes are required to your portlet in order to properly compile ( because of libraries clashes)<br />
<br />
The ghn-core-runtime component ( gcf dependency) should be excluded:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.5.1-SNAPSHOT,2.0.0)</version><br />
<scope>provided</scope><br />
<exclusions><br />
<exclusion><br />
<artifactId>ghn-core-runtime</artifactId><br />
<groupId>org.gcube.distribution</groupId><br />
</exclusion><br />
</exclusions><br />
</dependency><br />
</source><br />
<br />
<br />
and some new dependencies should be added:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>xerces</groupId><br />
<artifactId>xercesImpl</artifactId><br />
<version>2.9.1</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ws-addressing</artifactId><br />
<version>1.0</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>axis-patched</artifactId><br />
<version>1.2RC</version><br />
<scope>provided</scope><br />
</dependency><br />
<br />
</source><br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the portlet is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>PortletAdmin</Class><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.portles.admin</groupId><br />
<artifactId>sample-admin-portlet</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/sample-admin-portlet-1.0.0-SNAPSHOT.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code> of the project;<br />
* package versions are aligned with the Maven <code>version</code> of the project;<br />
* package include <code>MavenCoordinates</code> that can be directly copied and pasted from the POM of the project;<br />
* packages do '''not''' specify dependencies;<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>sample-portlet</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>PortletAdmin</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/${build.finalName}.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
== Service Archive ==<br />
<br />
<code>sample-portlet</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the project.<br />
<br />
As shown for [[#Libraries|libraries]], the portlet uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources]] for details.<br />
<br />
== Use Case: Create a new Mavenized GWT Portlet ==<br />
<br />
In order to create a new Mavenized GWT Portlet this detailed guide can be also followed [https://gcube.wiki.gcube-system.org/gcube/index.php/Create_a_new_Mavenized_gCube_GWT_Portlet_Guide]<br />
<br />
= Widgets =<br />
<br />
Falls in this category of components mainly GWT widgets or so called GWT Libraries, which can be used to package classes for mutualization and/or modularization. A Widget is a java archive (JAR) containing both classes and java sources for later GWT compilation and a gwt.xml module descriptor. <br />
<br />
In this section we refer to a sample Widget with maven structure (https://svn.research-infrastructures.eu/d4science/gcube/trunk/portal/mvn-widget-sample/)<br />
<br />
== Project Structure ==<br />
<br />
As for a other project, the portlet is associated with 2 distinguished build artefacts:<br />
<br />
* the archive required for deployment (a JAR);<br />
* the archive required for registration in the system (a SA);<br />
<br />
In order to generate automatically the project structure and the base pom, we start by customizing the maven archetype liferay-portlet-archetype v. 6.0.6.<br />
<br />
The generation of the project structure and the pom can be performed either from Eclipse ( by using the m2eclipse plugin) or by command line using the Maven SDK.<br />
<br />
On Eclipse the following steps must be followed:<br />
<br />
* FIle -> New -> Project<br />
* From the wizard select Maven->Maven Project<br />
* From the Select Archetype step, Select 'All Catalogs' from the Catalogs combobox and type 'portlet' on the Filter field.<br />
* Select the '''maven-archetype-quickstart''' version 1.1.0 archetype ( if does not appear on the list just unselect 'show the last version of the Archetype only")<br />
* In the next step you have to provide Maven coordinates for your project<br />
* groupid : <your group id> e.g org.gcube.portlets.admin<br />
* artifactId : <your artifactId> e.g. sample-admin-portlet<br />
* version : <your project version> e.g. 1.0.0-SNAPSHOT ( use always SNAPSHOT versions for development versions of a project)<br />
* package: <your portlet code package> e.g. org.gcube.portlets.admin.sample<br />
<br />
while from shell the following mvn command should be issued :<br />
<br />
<pre><br />
mvn archetype:generate \<br />
-DarchetypeArtifactId=maven-archetype-quickstart \<br />
-DarchetypeGroupId=com.liferay.maven.archetypes \<br />
-DarchetypeVersion=1.1.0 \<br />
-DartifactId=<your artifactId> \<br />
-DgroupId=<your group id> \<br />
-Dversion=1.0.0-SNAPSHOT<br />
</pre><br />
<br />
Both approaches generate a project structure similar to the picture below:<br />
<br />
[[image:sample-widget-project.png|Sample widget project structure]]<br />
<br />
In order to customize the build procedure to the gCube needs we should open the pom.xml and apply the following changes:<br />
<br />
Configure the inheritance from '''maven-parent''' :<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
Configure some project properties:<br />
<br />
<source lang="xml"><br />
<properties><br />
<distroDirectory>${project.basedir}/distro</distroDirectory><br />
<configDirectory>${project.basedir}/config</configDirectory><br />
</properties><br />
</source><br />
<br />
Add the plugin for the generation of the SA, which has to be included under the build plugins section:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<version>2.2</version><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
In the case of a GWT widget the [http://mojo.codehaus.org/gwt-maven-plugin gwt-maven-plugin] should also be configured ( within build plugins section as well), by also selecting the proper gwt version ( in our example 2.4)<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.codehaus.mojo</groupId><br />
<artifactId>gwt-maven-plugin</artifactId><br />
<version>${gwt.version}</version><br />
<executions><br />
<execution><br />
<goals><br />
<goal>compile</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
The proper GWT version should be added to the pom properties :<br />
<br />
<source lang="xml"><br />
<gwt.version>2.4.0</gwt.version><br />
</source><br />
<br />
and GWT dependencies should be also configured:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-user</artifactId><br />
<version>${gwt.version}</version><br />
</dependency><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-servlet</artifactId><br />
<version>${gwt.version}</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/mvn-widget-sample/ project sources]] for further details on the project configuration.<br />
<br />
== Generating Artifacts ==<br />
<br />
The <code>sample-portlet</code> uses the '''maven-compiler-plugin''' to compile the portlet code and the '''gwt-maven-plugin''' as well for GWT compilation.<br />
<br />
The generation of a war package can be triggered by typing :<br />
<br />
<pre><br />
mvn package <br />
</pre><br />
<br />
on your shell or running this build task within Eclipse.<br />
<br />
In both cases the war package will be generated under the ''target'' folder, e.g.:<br />
<br />
<pre><br />
target/sample-admin-portlet-1.0.0-SNAPSHOT.war<br />
</pre><br />
<br />
The generation of all project artifacts ( war, source jar, javadoc, SA) can be triggered by the build task :<br />
<br />
<pre><br />
mvn install <br />
</pre><br />
<br />
this will install as well all artifacts in your local repository, while the deployment of the artifacts to the remote gCube Maven repos can triggered by the build task:<br />
<br />
<pre><br />
mvn deploy <br />
</pre><br />
<br />
== Widget dependencies ==<br />
<br />
The system dependencies ( the one contained within the [[GCube_Portal_Installation#Portal_Bundle_Libraries|Portal bundle]] ) should not be included inside the portlet war. As said in this case the ''provided'' scope should be specified, e.g.<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portal</groupId><br />
<artifactId>custom-portal-handler</artifactId><br />
<version>1.1.0-SNAPSHOT</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.4.0,1.5.0]</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the widget is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>PortletAdmin</Class><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.portles.admin</groupId><br />
<artifactId>sample-admin-portlet</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/sample-admin-portlet-1.0.0-SNAPSHOT.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code> of the project;<br />
* package versions are aligned with the Maven <code>version</code> of the project;<br />
* package include <code>MavenCoordinates</code> that can be directly copied and pasted from the POM of the project;<br />
* packages do '''not''' specify dependencies;<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>sample-portlet</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>PortletAdmin</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/${build.finalName}.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
== Service Archive ==<br />
<br />
<code>sample-portlet</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the project.<br />
<br />
As shown for [[#Libraries|libraries]], the portlet uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources]] for details.</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Developing_gCube_Maven_Components&diff=29732Developing gCube Maven Components2017-10-30T11:38:47Z<p>Gabriele.giammatteo: /* Parent */</p>
<hr />
<div>[[Category: Developer's Guide]]<br />
__TOC__<br />
<br />
= Common =<br />
== Maven Coordinates ==<br />
Maven components' coordinates must follow these conventions:<br />
<br />
* their <code>groupId</code> must be of the following form:<br />
<br />
<pre>org.gcube.<class></pre><br />
<br />
: where ''class'' identifies a functionally correlated set of components and helps a range of users (developers, administrator) to discover components in various UIs/reports used within the system and in tools. Logically, the class corresponds to the gCube class of the component's, as specified in its gCube Software Profile. However, more specific, or else cross-class groups may be agreed upon among developers. In all cases, the group should align with official Maven [http://maven.apache.org/guides/mini/guide-naming-conventions.html guidelines]. By agreement, for example, components in gCube class <code>DataAccess</code> declare a <code>groupId</code> of <code>org.gcube.data.access</code> and those in gCube class <code>DataTransfer</code> declare a <code>groupId</code> of <code>org.gcube.data.transfer</code>. On the other hand, also by agreement, components in gCube class <code>InformationSystem</code> declare a <code>groupId</code> of <code>org.gcube.informationsystem</code>.<br />
<br />
* their <code>artifactId</code> must follow Maven conventions (e.g. ''avoid capital letters or punctuation other than hyphens'').<br />
<br />
== Parent ==<br />
All gCube components must inherit from '''maven-parent'''. This ensure and promotes compliance with system-wide requirements, from the enforcement of minimal Java and Maven versions to generation and packaging of Javadoc documentation and component sources.<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
<br />
The maven-parent defines:<br />
* several system-wide properties like <code>gcube.license</code>, <code>gcube.wikiRoot</code>, <code>distroDirectory</code><br />
* the default Java version (see more in [[Developing_gCube_Maven_Components#Java_Version|Java Version]])<br />
* the execution of some plugins that run for every component like: <code>maven-javadoc-plugin</code>, <code>maven-source-plugin</code><br />
* the configuration of the <code>maven-assembly-plugin</code> to generate some gCube-specific packages: <code>servicearchive</code>, <code>uberjar</code> and <code>source-package</code> (see more in [[Developing_gCube_Maven_Components#Artifact_Types|Artifact Types]])<br />
* the logic to enable the generation of the source package<br />
<br />
The '''maven-parent''' is at http://maven.research-infrastructures.eu/nexus/service/local/repositories/gcube-externals/content/org/gcube/tools/maven-parent/1.0.0/maven-parent-1.0.0.pom<br />
<br />
== Artifact Types ==<br />
asd<br />
<br />
== The distro folder ==<br />
This folder contains files that will be distributed along with the code in the distribution packages. Further details are in the [[Maven distro directory layout]] page.<br />
<br />
== Service Archive (SA) ==<br />
Whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary <code>tar.gz</code> artefact with classifier <code>servicearchive</code>. Note that the SA of Maven components does ''not'' need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);<br />
<br />
* for each deployable artefact they produce (primary or secondary), they must include the Maven coordinates of the arteact in the corresponding package section of the gCube Software Profile. Specifying package dependencies in the Software Profile, on the other hand, is no longer required;<br />
<br />
* whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary <code>tar.gz</code> artefact with classifier <code>servicearchive</code>. Note that the SA of Maven components does ''not'' need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);<br />
<br />
== Dependencies ==<br />
Service and libraries that depend on other gCube components (Maven-based or Ant-based) can find them in the gCube Maven Repositories: <code>gcube-snapshots</code> <strike>and <code>gcube-releases</code></strike>. <br />
<br />
=== gCube Maven Repositories ===<br />
As a one-off task, developers must configure these repositories in the <code>~/.m2/settings.xml</code> file of their local Maven installation, as follows:<br />
<br />
<source lang="xml"><br />
<settings xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"<br />
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd"><br />
<br />
<servers><br />
<server><br />
<id>gcube-snapshots</id><br />
<username>gcube-user</username><br />
<password>maven</password><br />
</server><br />
</servers><br />
<br />
<br />
<profiles><br />
<profile><br />
<id>gcube</id><br />
<repositories><br />
<repository><br />
<id>gcube-snapshots</id><br />
<name>gCube Snapshots</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
</snapshots><br />
</repository><br />
<repository><br />
<id>gcube-externals</id><br />
<name>gCube Externals</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals</url><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
</repository><br />
</repositories><br />
<br />
<pluginRepositories><br />
<pluginRepository><br />
<id>gcube-snapshots</id><br />
<name>gCube Snapshots</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
</snapshots><br />
</pluginRepository><br />
<pluginRepository><br />
<id>gcube-externals</id><br />
<name>gCube Externals</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals</url><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
</pluginRepository><br />
</pluginRepositories><br />
<br />
</profile><br />
</profiles><br />
<br />
<activeProfiles><br />
<activeProfile>gcube</activeProfile><br />
</activeProfiles><br />
</settings><br />
</source><br />
<br />
<br />
'''Note:Developers must not configure these repositories in their POMs''', only in <code>settings.xml</code>. <br />
<br />
This is to:<br />
* keep their projects independent from future changes in repository locations; <br />
* avoid interference with ETICS builds, where access to these repositories may comprise correct system integration;<br />
<br />
With this settings.xml, developers will have access on their local development environment to development versions (gcube-snapshots) and externals (gcube-externals). This is the correct behaviour for most of use cases. If access to staging or releases repositories is necessary (e.g. patching a component already released in the branch where dependencies versions in the release are not in snapshots), developer should modify his settings.xml to point to gcube-staging(releases) instead of gcube-snapshots.<br />
<br />
=== Third-party Repositories ===<br />
Using the the <code>settings.xml</code> aforementioned, any dependency declared in pom.xml that can be found in one of gCube repositories or in Maven Central will be resolved at build time and deployment time without any further configuration. Frequently it may happens that a gCube component needs software stored on a public Maven repository not in on of those repositories. In this case, Maven provides an easy solution for declaring additional repositories for dependency resolution by adding a <repositories> section in the pom.xml of the project. E.g.:<br />
<br />
<source lang="xml" highlight="3,4"><br />
<repositories><br />
<repository><br />
<id>[repository-id]</id><br />
<url>[repository URL]</url><br />
</repository><br />
</repositories><br />
</source><br />
<br />
Though technically possible to add any Maven repository in this way, in order to preserve the quality of gCube software it is required to check trustworthiness of its dependencies disallowing usage of certain external software. To enforce such a control over public Maven repositories, '''gCube components are allowed to use external repository not directly but only through a proxy repository''' (a.k.a. shadow repository) created on CNR's Nexus Repository Portal (http://maven.research-infrastructures.eu/nexus/).<br />
<br />
List of all repositories in Nexus Portal is available at http://maven.research-infrastructures.eu/nexus/index.html#view-repositories<br />
<br />
'''Note''': if a component uses a public Maven repository directly, it will work for building but would fail in deployment phase. In fact, Software Gateway is only able to download software accessible from Nexus.<br />
<br />
==== Request new proxy repository in Nexus ====<br />
Proxy Repositories on Nexus are created by the Release Manager. To request a new proxy a new ticket must be created.<br />
<br />
Details of ticket are:<br />
# open a ticket in tracker "Support" in gCube ticketing system (at https://support.d4science.org/projects/gcube/)<br />
# ticket's '''description''' must report:<br />
#* URL of repository to add<br />
#* organization which hosts the repository<br />
#* which artifacts are needed from that repository<br />
#* which components needs those artifacts<br />
# '''Sprint''' field should be set to the gCube release in which the component is expected to be released <br />
# '''Category''' field must report the gCube subsystem that needs the external repository<br />
# '''Assignee''' is the Release Manager<br />
<br />
An example of ticket for requesting a new repository mirror is reported in the figure below.<br />
[[Image:New_maven_mirror_repo.png|600px|center|Ticket for requesting a new Mirror Repository]]<br />
<br />
Release Manager will process the ticket checking that no issues are foreseen from the usage of software stored in the repository. If request is accepted, a new proxy repository is created and the repository's URL will be communicated publicly on gCube's developers mailinglist (a.k.a. TCOM mailinglist).<br />
<br />
Knowing the mirror repository URL, developer can use it adding following lines in pom.xml. E.g.<br />
<br />
<source lang="xml" highlight="3,4"><br />
<repositories><br />
<repository><br />
<id>gcube-ext-osgeo</id><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/osgeo/</url><br />
</repository><br />
</repositories><br />
</source><br />
<br />
'''Note''': it is possible that software stored on proxy repositories is not included in search results on Nexus. This happens for those repositories that does not have indexes (or have indexes in a format that Nexus is not able to download). Anyway, once Nexus has downloaded artifacts from the original repository for the first time (e.g. because a compilation of a pom that uses the proxy repository), creates its own indexes and those artifacts will become available also for local search.<br />
<br />
=== Dependency Versions ===<br />
<br />
ETICS builds ensure correct integration of gCube components, regardless of the versions their dependencies. It also ensures that components are released with fixed versions for their dependencies, i.e. ensures build reproducibility.<br />
<br />
Accordingly, developers can be liberal in the choice of dependency versions. i.e. rely on version ranges to:<br />
<br />
* minimise the requirements for version management;<br />
* be readily informed of any mis-alignment with their dependencies;<br />
<br />
As we start adopting Maven, the lower-bound of version ranges will be a SNAPSHOT version. For example, a dependency on the <code>streams</code> library can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>streams</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
</dependency><br />
</source><br />
<br />
With this dependency, a component will be soon notified of a non-retro compatible change that, contrary to conventions, <code>streams</code> may have introduced in, say, version <code>1.3.0</code> (typically the day after at the latest). The notification will be delivered in the IDE, assuming the availability of Maven integration (e.g. <code>m2e</code> in Eclipse). <br />
The component is in fact free to open the range further if it wants to be aligned with its dependencies across major versions.<br />
<br />
=== Common Dependencies ===<br />
<br />
Many gCube components depend on <code>gcf</code>, which is now available as a Maven component since version <code>1.4</code>. At the time of writing, the component has not been released yet. A dependency on it is then specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
The scope <code>provided</code> indicates that these dependency will be satisfied in the development environment (the gHN) and should not be included in GAR and SA archives.<br />
<br />
Some components depend on the libraries available in all gHNs, though they do not depend on <code>gcf</code> itself. The subset of these libraries that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called <code>ghn-core-runtime</code>. A dependency on this Jar can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ghn-core-runtime</artifactId><br />
<version>1.0.0</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Note: a component that depends on <code>gcf</code> indirectly depends on <code>gun-core-runtime</code>.<br />
<br />
Another common dependency is on the gCube gRS2, an Ant component that the build infrastructure make available in Maven repositories. A dependency on this component can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.execution</groupId><br />
<artifactId>grs2library</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
</dependency><br />
</source><br />
<br />
== Java Version ==<br />
<br />
In Maven the Java version of the source code and of the generated classes can be controlled using the configuration of the <code>maven-compiler-plugin</code>. E.g.:<br />
<syntaxhighlight lang="xml" highlight="5,6"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-compiler-plugin</artifactId><br />
<configuration><br />
<source>1.7</source><br />
<target>1.8</target><br />
</configuration><br />
</plugin><br />
</syntaxhighlight><br />
<br />
<br />
In gCube, this mechanism is exploited to enforce the usage of the same Java version for all components. In particular, the maven-parent defines the properties <code>java_version</code>, <code>maven.compiler.source</code> and <code>maven.compiler.target</code> and use these properties in the configuration of the <code>maven-compiler-plugin</code>.<br />
<syntaxhighlight lang="xml" highlight="15,16"><br />
...<br />
<properties><br />
<java_version>1.8</java_version><br />
<maven.compiler.source>${java_version}</maven.compiler.source><br />
<maven.compiler.target>${java_version}</maven.compiler.target><br />
...<br />
</properties><br />
<br />
...<br />
<br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-compiler-plugin</artifactId><br />
<configuration><br />
<target>${maven.compiler.target}</target><br />
<source>${maven.compiler.source}</source><br />
</configuration><br />
</plugin><br />
...<br />
</syntaxhighlight><br />
<br />
'''All the gCube components inherit this default configuration from <code>maven-parent</code> and it is not recommended to override it in the component's pom.xml'''.<br />
<br />
If it is really needed to override these values at component's pom.xml level, instead of specify a new configuration for the maven-compiler-plugin, the value of the property should be re-defined:<br />
<br />
<syntaxhighlight lang="xml" highlight="3"><br />
...<br />
<properties><br />
<maven.compiler.source>1.7</maven.compiler.source><br />
...<br />
</properties><br />
<br />
</syntaxhighlight><br />
<br />
<br />
This configuration is flexible enough to allow to customize the Java compiler source and target values depending on different factors in the integration builds (e.g. during ETICS builds gCore components are built with Java 7, all the others with Java 8).<br />
<br />
= Libraries =<br />
The <code>streams</code> library is a Maven component in gCube class <code>DataAccess</code>. We will use it as example of a library component in Maven.<br />
<br />
=== Profile ===<br />
<br />
The gCube Software profile of the library is as follows:<br />
<br />
<source lang="xml" highlight="14-18"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>Embedded Domain-Specific Language for Stream Transformations</Description><br />
<Class>DataAccess</Class><br />
<Name>streams</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>streams</Name><br />
<Version>1.0.0-SNAPSHOT</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>streams</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<Files><br />
<File>streams-1.0.0-SNAPSHOT.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the profile includes a single package and this package corresponds to the main build artefact of the component; <br />
* the whole profile and the package have usual gCube coordinates;<br />
* the package name is aligned with the Maven <code>artifactId</code> of the component;<br />
* the package version is aligned with the Maven <code>version</code> of the component;<br />
* the package includes <code>MavenCoordinates</code> that can be directly copied and pasted from the POM;<br />
* the package does '''not''' specify dependencies;<br />
* the package points to the main artefact of the Maven build;<br />
<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>streams</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>DataAccess</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<Files><br />
<File>${build.finalName}.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
=== Service Archive ===<br />
<br />
<code>streams</code> is responsible for generating its own SA as part of its Maven build. This requires dedicated logic in the POM but has the advantage that:<br />
<br />
* SA validity can be verified locally;<br />
* no SA configurations are required in ETICS;<br />
* the required build logic can be easily reused across components;<br />
<br />
<code>streams</code> uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation in the profile and other static files. The approach is best illustrated with a reference to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/streams sources]] of the <code>streams</code> library.<br />
<br />
<br />
As we discuss [[#Creating the ETICS Configurations|below]], <code>streams</code> has the final requirement to produce an interpolated profile during ETICS builds, outside the context of the SA.<br />
<code>streams</code> meets this requirement with the [http://maven.apache.org/plugins/maven-resources-plugin/ Maven Resources Plugin]. Again, consult the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/streams sources]] of the <code>streams</code> library to reuse this approach.<br />
<br />
= Services =<br />
The <code>tree-manager</code> service is a Maven component in gCube class <code>DataAccess</code>. <br />
<br />
== Project Structure ==<br />
<br />
Differently from a library, the service is associated with a number of distinguished build artefacts:<br />
<br />
* the service implementation (a Jar);<br />
* the stubs automatically generated from the WSDL interfaces (a Jar);<br />
* the archive required for deployment in gCore (a Gar);<br />
* the archive required for registration in the system (a SA);<br />
<br />
Producing these artefacts requires sharing a number of static and dynamic files (from profiles to WSDLs to pre-processed WSDLs). The usual Maven approach of mapping the artefacts onto different Maven projects is then genuinely difficult. It is instead more practical to use a single multi-module Maven project. The <code>tree-manager</code> uses this module structure:<br />
<br />
<pre><br />
tree-manager (parent module)------- (produces) --- (POM only)<br />
|<br />
|----- tree-manager-service (service impl module)------- (produces) --- jar, gar, SA<br />
| |<br />
| | (depends on)<br />
| |<br />
| \/<br />
|----- tree-manager-stubs (service stubs module)------- (produces) --- jar<br />
</pre><br />
<br />
As customary, the parent does not produce any artefact (is a POM-only project). It declares these coordinates in its POM:<br />
<br />
<source lang="xml"><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
<packaging>pom</packaging><br />
</source><br />
<br />
The parent physically includes the sub-modules;<br />
<br />
<source lang="xml"><br />
<modules><br />
<module>tree-manager-stubs</module><br />
<module>tree-manager-service</module><br />
</modules><br />
</source> <br />
<br />
Finally, the parent inherits from '''maven-parent''':<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
The submodules inherit from the parent:<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>tree-manager</artifactId><br />
<groupId>org.gcube.data.access</groupId><br />
<version>1.0.0-SNAPSHOT</version><br />
<relativePath>..</relativePath><br />
</parent><br />
</source><br />
<br />
and share its <code>version</code> and <code>groupId</code>. Their coordinates are then minimal:<br />
<br />
<source lang="xml"><br />
<artifactId>tree-manager-service</artifactId><br />
</source><br />
<br />
<source lang="xml"><br />
<artifactId>tree-manager-stubs</artifactId><br />
</source><br />
<br />
The <code>tree-manager-service</code> module produces a Jar of its own implementation, as well the GAR and the SA of the whole services as secondary artefacts with classifiers of, respectively, <code>gar</code> and <code>service archive</code>.<br />
<br />
The <code>tree-manager-stubs</code> module produces a Jar of its own implementation.<br />
<br />
Finally, the parent module coordinates the build of its sub-modules and centralises the declaration of common dependencies and configuration. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] to reuse this approach.<br />
<br />
== Generating Stubs and Gars ==<br />
<br />
The <code>tree-manager-stubs</code> uses the '''gCube Service Plugin''' to generate its classes from WSDL interfaces. <br />
<br />
Similarly, the <code>tree-manager-service</code> uses the plugin to generate its secondary GAR artefact. <br />
<br />
The configuration of the plugin is conveniently shared in the parent module.<br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] of the <code>streams</code> for details.<br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the service is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>DataAccess</Class><br />
<Name>tree-manager-service</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Main><br />
<Name>tree-manager-service</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager-service</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<GARArchive>tree-manager-service-1.0.0-SNAPSHOT.gar</GARArchive><br />
<PortType><br />
<Name>gcube/data/tm/binder</Name><br />
</PortType><br />
<PortType><br />
<Name>gcube/data/tm/reader</Name><br />
</PortType><br />
<PortType><br />
<Name>gcube/data/tm/writer</Name><br />
</PortType><br />
</Main><br />
<Software><br />
<Name>tree-manager-stubs</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager-stubs</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<Type>library</Type><br />
<Files><br />
<File>tree-manager-stubs-1.0.0-SNAPSHOT.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the profile includes two packages and this package corresponds to the main build artefacts of the submodules; <br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code>s of the submodules;<br />
* package versions are aligned with the Maven <code>version</code>s of the submodules;<br />
* package include <code>MavenCoordinates</code>s that can be directly copied and pasted from the POM of the submodules;<br />
* packages do '''not''' specify dependencies;<br />
* packages points to the main artefact of the Maven build of the submodules;<br />
<br />
<strike>Notice that, unlike libraries, services cannot easily use Maven variables in their profiles (as these artefacts are used for different purposes by different modules). Accordingly, developers needs to maintain versions used in profiles across releases. In particular, they need to:<br />
<br />
* replace SNAPSHOT versions with release versions in release branches;<br />
* advance SNAPSHOT versions in HEAD;</strike><br />
<br />
=== Profile Interpolation ===<br />
The interpolation of profiles is required in order to keep information in profile in sync with <code>pom.xml</code>, in particular versions and filenames (that contain versions) that change at each release.<br />
<br />
In order to generate <code>profile.xml</code> during build process you need to:<br />
# add in the service's parent <code>pom.xml</code> the invocation to maven-resources-plugin<br />
# place a ''template'' <code>profile.xml</code> in the <code>${distroDirectory}</code><br />
<br />
Invocation to <code>maven-resources-plugin</code> looks like the following (to be placed in the <plugins> section inside the <build> section of the pom):<br />
<source lang="xml"><br />
<!-- interpolates profiles and copies from distribution location to configuration <br />
location, where it is need for embedding into stub artifact, gar generation, <br />
and service archive. --><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-resources-plugin</artifactId><br />
<version>2.6</version><br />
<executions><br />
<execution><br />
<id>copy-profile</id><br />
<goals><br />
<goal>copy-resources</goal><br />
</goals><br />
<phase>process-resources</phase><br />
<configuration><br />
<outputDirectory>${configDirectory}</outputDirectory><br />
<resources><br />
<resource><br />
<directory>${distroDirectory}</directory><br />
<includes><br />
<include>profile.xml</include><br />
</includes><br />
<filtering>true</filtering><br />
</resource><br />
</resources><br />
</configuration><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
The ''template'' <code>profile.xml</code> looks like the following:<br />
<source lang="xml" highlight="12,16,18,22,26,30"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>TestClass</Class><br />
<Name>TestService</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Main><br />
<Name>TestService</Name><br />
<Version>${project.version}</Version><br />
<MavenCoordinates><br />
<groupId>org.example</groupId><br />
<artifactId>test-service</artifactId><br />
<version>${project.version}</version><br />
</MavenCoordinates><br />
<GARArchive>>test-service-${project.version}.gar</GARArchive><br />
</Main><br />
<Software><br />
<Name>test-stubs</Name><br />
<Version>${project.version}</Version><br />
<MavenCoordinates><br />
<groupId>org.example</groupId><br />
<artifactId>test-stubs</artifactId><br />
<version>${project.version}</version><br />
</MavenCoordinates><br />
<Type>library</Type><br />
<Files><br />
<File>test-stubs-${project.version}.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
In the example, property <code>${project.version}</code> will be substituted with the actual service's version at build-time. It is also possible, but not required, to parametrize also other values of <code>profile.xml</code> such as maven's coordinates and package names. You can see a more complex <code>profile.xml</code> ''template'' in the <code>tree-manager</code> project: http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-access/tree-manager/distro/profile.xml<br />
<br />
Maven will copy the final <code>profile.xml</code> in <code>${configDirectory}</code> during the <code>process-resources</code> phase of the build. From there it will be copied in the service-archive package.<br />
<br />
== Service Archive ==<br />
<br />
<code>tree-manager-service</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the module and its parent.<br />
<br />
As shown for [[#Libraries|libraries]], the service uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] for details.<br />
<br />
= Portlets =<br />
<br />
The <code>sample-portlet</code> portlet is a Maven based sample portlet based on GWT. Refer to the [http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources] for details.<br />
<br />
== Project Structure ==<br />
<br />
As for a library project, the portlet is associated with 2 distinguished build artefacts:<br />
<br />
* the archive required for deployment in a Liferay Container (a WAR);<br />
* the archive required for registration in the system (a SA);<br />
<br />
In order to generate automatically the project structure and the base pom, we start by customizing the maven archetype liferay-portlet-archetype v. 6.0.6.<br />
<br />
The generation of the project structure and the pom can be performed either from Eclipse ( by using the m2eclipse plugin) or by command line using the Maven SDK.<br />
<br />
On Eclipse the following steps must be followed:<br />
<br />
* FIle -> New -> Project<br />
* From the wizard select Maven->Maven Project<br />
* From the Select Archetype step, Select 'All Catalogs' from the Catalogs combobox and type 'portlet' on the Filter field.<br />
* Select the '''liferay-portlet-archetype''' version 6.0.6 archetype ( if does not appear on the list just unselect 'show the last version of the Archetype only")<br />
* In the next step you have to provide Maven coordinates for your project<br />
* groupid : <your group id> e.g org.gcube.portlets.admin<br />
* artifactId : <your artifactId> e.g. sample-admin-portlet<br />
* version : <your project version> e.g. 1.0.0-SNAPSHOT ( use always SNAPSHOT versions for development versions of a project)<br />
* package: <your portlet code package> e.g. org.gcube.portlets.admin.sample<br />
<br />
while from shell the following mvn command should be issued :<br />
<br />
<pre><br />
mvn archetype:generate \<br />
-DarchetypeArtifactId=liferay-theme-archetype \<br />
-DarchetypeGroupId=com.liferay.maven.archetypes \<br />
-DarchetypeVersion=6.0.6 \<br />
-DartifactId=<your artifactId> \<br />
-DgroupId=<your group id> \<br />
-Dversion=1.0.0-SNAPSHOT<br />
</pre><br />
<br />
Both approaches generate a project structure similar to the picture below:<br />
<br />
[[image:sample-portlet-project.png|Sample portlet project structure]]<br />
<br />
In order to customize the build procedure to the gCube needs we should open the pom.xml and apply the following changes:<br />
<br />
Configure the inheritance from '''maven-parent''' :<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
Configure some project properties:<br />
<br />
<source lang="xml"><br />
<properties><br />
<liferay.version>6.0.6</liferay.version><br />
<liferay.auto.deploy.dir>${system.CATALINA_HOME}/../deploy</liferay.auto.deploy.dir><br />
<distroDirectory>${project.basedir}/distro</distroDirectory><br />
<configDirectory>${project.basedir}/config</configDirectory><br />
</properties><br />
</source><br />
<br />
Add the plugin for the generation of the SA, which has to be included under the build plugins section:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<version>2.2</version><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
In the case of a GWT portlet the [http://mojo.codehaus.org/gwt-maven-plugin gwt-maven-plugin] should also be configured ( within build plugins section as well), by also selecting the proper gwt version ( in our example 2.4)<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.codehaus.mojo</groupId><br />
<artifactId>gwt-maven-plugin</artifactId><br />
<version>${gwt.version}</version><br />
<executions><br />
<execution><br />
<goals><br />
<goal>compile</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
The proper GWT version should be added to the pom properties :<br />
<br />
<source lang="xml"><br />
<gwt.version>2.4.0</gwt.version><br />
</source><br />
<br />
and GWT dependencies should be also configured:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-user</artifactId><br />
<version>${gwt.version}</version><br />
</dependency><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-servlet</artifactId><br />
<version>${gwt.version}</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet/ project sources]] for further details on the project configuration.<br />
<br />
== Generating Artifacts ==<br />
<br />
The <code>sample-portlet</code> uses the '''maven-compiler-plugin''' to compile the portlet code and the '''gwt-maven-plugin''' as well for GWT compilation.<br />
<br />
The generation of a war package can be triggered by typing :<br />
<br />
<pre><br />
mvn package <br />
</pre><br />
<br />
on your shell or running this build task within Eclipse.<br />
<br />
In both cases the war package will be generated under the ''target'' folder, e.g.:<br />
<br />
<pre><br />
target/sample-admin-portlet-1.0.0-SNAPSHOT.war<br />
</pre><br />
<br />
The generation of all project artifacts ( war, source jar, javadoc, SA) can be triggered by the build task :<br />
<br />
<pre><br />
mvn install <br />
</pre><br />
<br />
this will install as well all artifacts in your local repository, while the deployment of the artifacts to the remote gCube Maven repos can triggered by the build task:<br />
<br />
<pre><br />
mvn deploy <br />
</pre><br />
<br />
== Portlet dependencies ==<br />
<br />
Specific portlet dependencies ( e.g. GWT, service stubs ) should be packaged within the portlet war. All dependencies specified within the pom file without the ''provided'' scope will be copied inside the WEB-INF/lib folder at packaging time.<br />
<br />
For this reason all portlet specific dependencies should not specified that scope e.g:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.externals</groupId><br />
<artifactId>gwt-ext-patched</artifactId><br />
<version>2.0.5</version><br />
</dependency><br />
</source><br />
<br />
On the other hand the system dependencies ( the one contained within the [[GCube_Portal_Installation#Portal_Bundle_Libraries|Portal bundle]] ) should not be included inside the portlet war. As said in this case the ''provided'' scope should be specified, e.g.<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portal</groupId><br />
<artifactId>custom-portal-handler</artifactId><br />
<version>1.1.0-SNAPSHOT</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.4.0,1.5.0]</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
<br />
A subset of these libraries ( contained in a GHN) that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called <code>ghn-core-runtime</code>. A dependency on this Jar can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ghn-core-runtime</artifactId><br />
<version>1.0.0</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Note: a component that depends on <code>gcf</code> indirectly depends on <code>gun-core-runtime</code>.<br />
<br />
== Include gCube Guided Tour using Maven ==<br />
<br />
This particular component is not yet ported to maven so in order to be included in a maven based project it should be declared as system dependency as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portletsuser</groupId><br />
<artifactId>gcube-guidedtour-widget</artifactId><br />
<version>1.0.0</version><br />
<scope>system</scope><br />
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/gcube-guidedtour-widget.jar</systemPath> <br />
</dependency><br />
</source><br />
<br />
together with the gCube Widget Library:<br />
<br />
<source lang="xml"> <br />
<dependency><br />
<groupId>org.gcube.portlets.user</groupId><br />
<artifactId>gcubewidgets</artifactId><br />
<version>1.2.0</version><br />
<scope>system</scope><br />
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/gCube-Widgets-Library.jar</systemPath><br />
</dependency><br />
</source><br />
<br />
both components can be downloaded from the distribution repository [[http://grids16.eng.it:8080/BuildReport/browse/Recent_Builds]]. <br />
<br />
In case the portlet depends on the ''gcf'' component, some dependency resolution changes are required to your portlet in order to properly compile ( because of libraries clashes)<br />
<br />
The ghn-core-runtime component ( gcf dependency) should be excluded:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.5.1-SNAPSHOT,2.0.0)</version><br />
<scope>provided</scope><br />
<exclusions><br />
<exclusion><br />
<artifactId>ghn-core-runtime</artifactId><br />
<groupId>org.gcube.distribution</groupId><br />
</exclusion><br />
</exclusions><br />
</dependency><br />
</source><br />
<br />
<br />
and some new dependencies should be added:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>xerces</groupId><br />
<artifactId>xercesImpl</artifactId><br />
<version>2.9.1</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ws-addressing</artifactId><br />
<version>1.0</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>axis-patched</artifactId><br />
<version>1.2RC</version><br />
<scope>provided</scope><br />
</dependency><br />
<br />
</source><br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the portlet is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>PortletAdmin</Class><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.portles.admin</groupId><br />
<artifactId>sample-admin-portlet</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/sample-admin-portlet-1.0.0-SNAPSHOT.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code> of the project;<br />
* package versions are aligned with the Maven <code>version</code> of the project;<br />
* package include <code>MavenCoordinates</code> that can be directly copied and pasted from the POM of the project;<br />
* packages do '''not''' specify dependencies;<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>sample-portlet</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>PortletAdmin</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/${build.finalName}.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
== Service Archive ==<br />
<br />
<code>sample-portlet</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the project.<br />
<br />
As shown for [[#Libraries|libraries]], the portlet uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources]] for details.<br />
<br />
== Use Case: Create a new Mavenized GWT Portlet ==<br />
<br />
In order to create a new Mavenized GWT Portlet this detailed guide can be also followed [https://gcube.wiki.gcube-system.org/gcube/index.php/Create_a_new_Mavenized_gCube_GWT_Portlet_Guide]<br />
<br />
= Widgets =<br />
<br />
Falls in this category of components mainly GWT widgets or so called GWT Libraries, which can be used to package classes for mutualization and/or modularization. A Widget is a java archive (JAR) containing both classes and java sources for later GWT compilation and a gwt.xml module descriptor. <br />
<br />
In this section we refer to a sample Widget with maven structure (https://svn.research-infrastructures.eu/d4science/gcube/trunk/portal/mvn-widget-sample/)<br />
<br />
== Project Structure ==<br />
<br />
As for a other project, the portlet is associated with 2 distinguished build artefacts:<br />
<br />
* the archive required for deployment (a JAR);<br />
* the archive required for registration in the system (a SA);<br />
<br />
In order to generate automatically the project structure and the base pom, we start by customizing the maven archetype liferay-portlet-archetype v. 6.0.6.<br />
<br />
The generation of the project structure and the pom can be performed either from Eclipse ( by using the m2eclipse plugin) or by command line using the Maven SDK.<br />
<br />
On Eclipse the following steps must be followed:<br />
<br />
* FIle -> New -> Project<br />
* From the wizard select Maven->Maven Project<br />
* From the Select Archetype step, Select 'All Catalogs' from the Catalogs combobox and type 'portlet' on the Filter field.<br />
* Select the '''maven-archetype-quickstart''' version 1.1.0 archetype ( if does not appear on the list just unselect 'show the last version of the Archetype only")<br />
* In the next step you have to provide Maven coordinates for your project<br />
* groupid : <your group id> e.g org.gcube.portlets.admin<br />
* artifactId : <your artifactId> e.g. sample-admin-portlet<br />
* version : <your project version> e.g. 1.0.0-SNAPSHOT ( use always SNAPSHOT versions for development versions of a project)<br />
* package: <your portlet code package> e.g. org.gcube.portlets.admin.sample<br />
<br />
while from shell the following mvn command should be issued :<br />
<br />
<pre><br />
mvn archetype:generate \<br />
-DarchetypeArtifactId=maven-archetype-quickstart \<br />
-DarchetypeGroupId=com.liferay.maven.archetypes \<br />
-DarchetypeVersion=1.1.0 \<br />
-DartifactId=<your artifactId> \<br />
-DgroupId=<your group id> \<br />
-Dversion=1.0.0-SNAPSHOT<br />
</pre><br />
<br />
Both approaches generate a project structure similar to the picture below:<br />
<br />
[[image:sample-widget-project.png|Sample widget project structure]]<br />
<br />
In order to customize the build procedure to the gCube needs we should open the pom.xml and apply the following changes:<br />
<br />
Configure the inheritance from '''maven-parent''' :<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
Configure some project properties:<br />
<br />
<source lang="xml"><br />
<properties><br />
<distroDirectory>${project.basedir}/distro</distroDirectory><br />
<configDirectory>${project.basedir}/config</configDirectory><br />
</properties><br />
</source><br />
<br />
Add the plugin for the generation of the SA, which has to be included under the build plugins section:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<version>2.2</version><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
In the case of a GWT widget the [http://mojo.codehaus.org/gwt-maven-plugin gwt-maven-plugin] should also be configured ( within build plugins section as well), by also selecting the proper gwt version ( in our example 2.4)<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.codehaus.mojo</groupId><br />
<artifactId>gwt-maven-plugin</artifactId><br />
<version>${gwt.version}</version><br />
<executions><br />
<execution><br />
<goals><br />
<goal>compile</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
The proper GWT version should be added to the pom properties :<br />
<br />
<source lang="xml"><br />
<gwt.version>2.4.0</gwt.version><br />
</source><br />
<br />
and GWT dependencies should be also configured:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-user</artifactId><br />
<version>${gwt.version}</version><br />
</dependency><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-servlet</artifactId><br />
<version>${gwt.version}</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/mvn-widget-sample/ project sources]] for further details on the project configuration.<br />
<br />
== Generating Artifacts ==<br />
<br />
The <code>sample-portlet</code> uses the '''maven-compiler-plugin''' to compile the portlet code and the '''gwt-maven-plugin''' as well for GWT compilation.<br />
<br />
The generation of a war package can be triggered by typing :<br />
<br />
<pre><br />
mvn package <br />
</pre><br />
<br />
on your shell or running this build task within Eclipse.<br />
<br />
In both cases the war package will be generated under the ''target'' folder, e.g.:<br />
<br />
<pre><br />
target/sample-admin-portlet-1.0.0-SNAPSHOT.war<br />
</pre><br />
<br />
The generation of all project artifacts ( war, source jar, javadoc, SA) can be triggered by the build task :<br />
<br />
<pre><br />
mvn install <br />
</pre><br />
<br />
this will install as well all artifacts in your local repository, while the deployment of the artifacts to the remote gCube Maven repos can triggered by the build task:<br />
<br />
<pre><br />
mvn deploy <br />
</pre><br />
<br />
== Widget dependencies ==<br />
<br />
The system dependencies ( the one contained within the [[GCube_Portal_Installation#Portal_Bundle_Libraries|Portal bundle]] ) should not be included inside the portlet war. As said in this case the ''provided'' scope should be specified, e.g.<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portal</groupId><br />
<artifactId>custom-portal-handler</artifactId><br />
<version>1.1.0-SNAPSHOT</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.4.0,1.5.0]</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the widget is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>PortletAdmin</Class><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.portles.admin</groupId><br />
<artifactId>sample-admin-portlet</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/sample-admin-portlet-1.0.0-SNAPSHOT.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code> of the project;<br />
* package versions are aligned with the Maven <code>version</code> of the project;<br />
* package include <code>MavenCoordinates</code> that can be directly copied and pasted from the POM of the project;<br />
* packages do '''not''' specify dependencies;<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>sample-portlet</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>PortletAdmin</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/${build.finalName}.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
== Service Archive ==<br />
<br />
<code>sample-portlet</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the project.<br />
<br />
As shown for [[#Libraries|libraries]], the portlet uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources]] for details.</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Developing_gCube_Maven_Components&diff=29731Developing gCube Maven Components2017-10-30T11:38:08Z<p>Gabriele.giammatteo: /* Parent */</p>
<hr />
<div>[[Category: Developer's Guide]]<br />
__TOC__<br />
<br />
= Common =<br />
== Maven Coordinates ==<br />
Maven components' coordinates must follow these conventions:<br />
<br />
* their <code>groupId</code> must be of the following form:<br />
<br />
<pre>org.gcube.<class></pre><br />
<br />
: where ''class'' identifies a functionally correlated set of components and helps a range of users (developers, administrator) to discover components in various UIs/reports used within the system and in tools. Logically, the class corresponds to the gCube class of the component's, as specified in its gCube Software Profile. However, more specific, or else cross-class groups may be agreed upon among developers. In all cases, the group should align with official Maven [http://maven.apache.org/guides/mini/guide-naming-conventions.html guidelines]. By agreement, for example, components in gCube class <code>DataAccess</code> declare a <code>groupId</code> of <code>org.gcube.data.access</code> and those in gCube class <code>DataTransfer</code> declare a <code>groupId</code> of <code>org.gcube.data.transfer</code>. On the other hand, also by agreement, components in gCube class <code>InformationSystem</code> declare a <code>groupId</code> of <code>org.gcube.informationsystem</code>.<br />
<br />
* their <code>artifactId</code> must follow Maven conventions (e.g. ''avoid capital letters or punctuation other than hyphens'').<br />
<br />
== Parent ==<br />
All gCube components must inherit from '''maven-parent'''. This ensure and promotes compliance with system-wide requirements, from the enforcement of minimal Java and Maven versions to generation and packaging of Javadoc documentation and component sources.<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
<br />
The maven-parent defines:<br />
* several system-wide properties like <code>gcube.license</code>, <code>gcube.wikiRoot</code>, <code>distroDirectory</code><br />
* the default Java version (see more in [[Developing_gCube_Maven_Components#Java_Version|Java Version]])<br />
* the execution of some plugins that run for every component like: <code>maven-javadoc-plugin</code>, <code>maven-source-plugin</code><br />
* the configuration of the <code>maven-assembly-plugin</code> to generate some gCube-specific packages: <code>servicearchive</code>, <code>uberjar</code> and <code>source-package</code> (see more in [[Developing_gCube_Maven_Components#Artifact_Types|Artifact Types]])<br />
* the logic to enable the generation of the source package<br />
<br />
The '''maven-parent''' is at http://maven.research-infrastructures.eu/nexus/service/local/repositories/gcube-externals/content/org/gcube/tools/maven-parent/1.0.0/maven-parent-1.0.0.pom<br />
<br />
== The distro folder ==<br />
This folder contains files that will be distributed along with the code in the distribution packages. Further details are in the [[Maven distro directory layout]] page.<br />
<br />
== Service Archive (SA) ==<br />
Whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary <code>tar.gz</code> artefact with classifier <code>servicearchive</code>. Note that the SA of Maven components does ''not'' need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);<br />
<br />
* for each deployable artefact they produce (primary or secondary), they must include the Maven coordinates of the arteact in the corresponding package section of the gCube Software Profile. Specifying package dependencies in the Software Profile, on the other hand, is no longer required;<br />
<br />
* whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary <code>tar.gz</code> artefact with classifier <code>servicearchive</code>. Note that the SA of Maven components does ''not'' need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);<br />
<br />
== Dependencies ==<br />
Service and libraries that depend on other gCube components (Maven-based or Ant-based) can find them in the gCube Maven Repositories: <code>gcube-snapshots</code> <strike>and <code>gcube-releases</code></strike>. <br />
<br />
=== gCube Maven Repositories ===<br />
As a one-off task, developers must configure these repositories in the <code>~/.m2/settings.xml</code> file of their local Maven installation, as follows:<br />
<br />
<source lang="xml"><br />
<settings xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"<br />
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd"><br />
<br />
<servers><br />
<server><br />
<id>gcube-snapshots</id><br />
<username>gcube-user</username><br />
<password>maven</password><br />
</server><br />
</servers><br />
<br />
<br />
<profiles><br />
<profile><br />
<id>gcube</id><br />
<repositories><br />
<repository><br />
<id>gcube-snapshots</id><br />
<name>gCube Snapshots</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
</snapshots><br />
</repository><br />
<repository><br />
<id>gcube-externals</id><br />
<name>gCube Externals</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals</url><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
</repository><br />
</repositories><br />
<br />
<pluginRepositories><br />
<pluginRepository><br />
<id>gcube-snapshots</id><br />
<name>gCube Snapshots</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
</snapshots><br />
</pluginRepository><br />
<pluginRepository><br />
<id>gcube-externals</id><br />
<name>gCube Externals</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals</url><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
</pluginRepository><br />
</pluginRepositories><br />
<br />
</profile><br />
</profiles><br />
<br />
<activeProfiles><br />
<activeProfile>gcube</activeProfile><br />
</activeProfiles><br />
</settings><br />
</source><br />
<br />
<br />
'''Note:Developers must not configure these repositories in their POMs''', only in <code>settings.xml</code>. <br />
<br />
This is to:<br />
* keep their projects independent from future changes in repository locations; <br />
* avoid interference with ETICS builds, where access to these repositories may comprise correct system integration;<br />
<br />
With this settings.xml, developers will have access on their local development environment to development versions (gcube-snapshots) and externals (gcube-externals). This is the correct behaviour for most of use cases. If access to staging or releases repositories is necessary (e.g. patching a component already released in the branch where dependencies versions in the release are not in snapshots), developer should modify his settings.xml to point to gcube-staging(releases) instead of gcube-snapshots.<br />
<br />
=== Third-party Repositories ===<br />
Using the the <code>settings.xml</code> aforementioned, any dependency declared in pom.xml that can be found in one of gCube repositories or in Maven Central will be resolved at build time and deployment time without any further configuration. Frequently it may happens that a gCube component needs software stored on a public Maven repository not in on of those repositories. In this case, Maven provides an easy solution for declaring additional repositories for dependency resolution by adding a <repositories> section in the pom.xml of the project. E.g.:<br />
<br />
<source lang="xml" highlight="3,4"><br />
<repositories><br />
<repository><br />
<id>[repository-id]</id><br />
<url>[repository URL]</url><br />
</repository><br />
</repositories><br />
</source><br />
<br />
Though technically possible to add any Maven repository in this way, in order to preserve the quality of gCube software it is required to check trustworthiness of its dependencies disallowing usage of certain external software. To enforce such a control over public Maven repositories, '''gCube components are allowed to use external repository not directly but only through a proxy repository''' (a.k.a. shadow repository) created on CNR's Nexus Repository Portal (http://maven.research-infrastructures.eu/nexus/).<br />
<br />
List of all repositories in Nexus Portal is available at http://maven.research-infrastructures.eu/nexus/index.html#view-repositories<br />
<br />
'''Note''': if a component uses a public Maven repository directly, it will work for building but would fail in deployment phase. In fact, Software Gateway is only able to download software accessible from Nexus.<br />
<br />
==== Request new proxy repository in Nexus ====<br />
Proxy Repositories on Nexus are created by the Release Manager. To request a new proxy a new ticket must be created.<br />
<br />
Details of ticket are:<br />
# open a ticket in tracker "Support" in gCube ticketing system (at https://support.d4science.org/projects/gcube/)<br />
# ticket's '''description''' must report:<br />
#* URL of repository to add<br />
#* organization which hosts the repository<br />
#* which artifacts are needed from that repository<br />
#* which components needs those artifacts<br />
# '''Sprint''' field should be set to the gCube release in which the component is expected to be released <br />
# '''Category''' field must report the gCube subsystem that needs the external repository<br />
# '''Assignee''' is the Release Manager<br />
<br />
An example of ticket for requesting a new repository mirror is reported in the figure below.<br />
[[Image:New_maven_mirror_repo.png|600px|center|Ticket for requesting a new Mirror Repository]]<br />
<br />
Release Manager will process the ticket checking that no issues are foreseen from the usage of software stored in the repository. If request is accepted, a new proxy repository is created and the repository's URL will be communicated publicly on gCube's developers mailinglist (a.k.a. TCOM mailinglist).<br />
<br />
Knowing the mirror repository URL, developer can use it adding following lines in pom.xml. E.g.<br />
<br />
<source lang="xml" highlight="3,4"><br />
<repositories><br />
<repository><br />
<id>gcube-ext-osgeo</id><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/osgeo/</url><br />
</repository><br />
</repositories><br />
</source><br />
<br />
'''Note''': it is possible that software stored on proxy repositories is not included in search results on Nexus. This happens for those repositories that does not have indexes (or have indexes in a format that Nexus is not able to download). Anyway, once Nexus has downloaded artifacts from the original repository for the first time (e.g. because a compilation of a pom that uses the proxy repository), creates its own indexes and those artifacts will become available also for local search.<br />
<br />
=== Dependency Versions ===<br />
<br />
ETICS builds ensure correct integration of gCube components, regardless of the versions their dependencies. It also ensures that components are released with fixed versions for their dependencies, i.e. ensures build reproducibility.<br />
<br />
Accordingly, developers can be liberal in the choice of dependency versions. i.e. rely on version ranges to:<br />
<br />
* minimise the requirements for version management;<br />
* be readily informed of any mis-alignment with their dependencies;<br />
<br />
As we start adopting Maven, the lower-bound of version ranges will be a SNAPSHOT version. For example, a dependency on the <code>streams</code> library can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>streams</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
</dependency><br />
</source><br />
<br />
With this dependency, a component will be soon notified of a non-retro compatible change that, contrary to conventions, <code>streams</code> may have introduced in, say, version <code>1.3.0</code> (typically the day after at the latest). The notification will be delivered in the IDE, assuming the availability of Maven integration (e.g. <code>m2e</code> in Eclipse). <br />
The component is in fact free to open the range further if it wants to be aligned with its dependencies across major versions.<br />
<br />
=== Common Dependencies ===<br />
<br />
Many gCube components depend on <code>gcf</code>, which is now available as a Maven component since version <code>1.4</code>. At the time of writing, the component has not been released yet. A dependency on it is then specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
The scope <code>provided</code> indicates that these dependency will be satisfied in the development environment (the gHN) and should not be included in GAR and SA archives.<br />
<br />
Some components depend on the libraries available in all gHNs, though they do not depend on <code>gcf</code> itself. The subset of these libraries that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called <code>ghn-core-runtime</code>. A dependency on this Jar can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ghn-core-runtime</artifactId><br />
<version>1.0.0</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Note: a component that depends on <code>gcf</code> indirectly depends on <code>gun-core-runtime</code>.<br />
<br />
Another common dependency is on the gCube gRS2, an Ant component that the build infrastructure make available in Maven repositories. A dependency on this component can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.execution</groupId><br />
<artifactId>grs2library</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
</dependency><br />
</source><br />
<br />
== Java Version ==<br />
<br />
In Maven the Java version of the source code and of the generated classes can be controlled using the configuration of the <code>maven-compiler-plugin</code>. E.g.:<br />
<syntaxhighlight lang="xml" highlight="5,6"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-compiler-plugin</artifactId><br />
<configuration><br />
<source>1.7</source><br />
<target>1.8</target><br />
</configuration><br />
</plugin><br />
</syntaxhighlight><br />
<br />
<br />
In gCube, this mechanism is exploited to enforce the usage of the same Java version for all components. In particular, the maven-parent defines the properties <code>java_version</code>, <code>maven.compiler.source</code> and <code>maven.compiler.target</code> and use these properties in the configuration of the <code>maven-compiler-plugin</code>.<br />
<syntaxhighlight lang="xml" highlight="15,16"><br />
...<br />
<properties><br />
<java_version>1.8</java_version><br />
<maven.compiler.source>${java_version}</maven.compiler.source><br />
<maven.compiler.target>${java_version}</maven.compiler.target><br />
...<br />
</properties><br />
<br />
...<br />
<br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-compiler-plugin</artifactId><br />
<configuration><br />
<target>${maven.compiler.target}</target><br />
<source>${maven.compiler.source}</source><br />
</configuration><br />
</plugin><br />
...<br />
</syntaxhighlight><br />
<br />
'''All the gCube components inherit this default configuration from <code>maven-parent</code> and it is not recommended to override it in the component's pom.xml'''.<br />
<br />
If it is really needed to override these values at component's pom.xml level, instead of specify a new configuration for the maven-compiler-plugin, the value of the property should be re-defined:<br />
<br />
<syntaxhighlight lang="xml" highlight="3"><br />
...<br />
<properties><br />
<maven.compiler.source>1.7</maven.compiler.source><br />
...<br />
</properties><br />
<br />
</syntaxhighlight><br />
<br />
<br />
This configuration is flexible enough to allow to customize the Java compiler source and target values depending on different factors in the integration builds (e.g. during ETICS builds gCore components are built with Java 7, all the others with Java 8).<br />
<br />
= Libraries =<br />
The <code>streams</code> library is a Maven component in gCube class <code>DataAccess</code>. We will use it as example of a library component in Maven.<br />
<br />
=== Profile ===<br />
<br />
The gCube Software profile of the library is as follows:<br />
<br />
<source lang="xml" highlight="14-18"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>Embedded Domain-Specific Language for Stream Transformations</Description><br />
<Class>DataAccess</Class><br />
<Name>streams</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>streams</Name><br />
<Version>1.0.0-SNAPSHOT</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>streams</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<Files><br />
<File>streams-1.0.0-SNAPSHOT.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the profile includes a single package and this package corresponds to the main build artefact of the component; <br />
* the whole profile and the package have usual gCube coordinates;<br />
* the package name is aligned with the Maven <code>artifactId</code> of the component;<br />
* the package version is aligned with the Maven <code>version</code> of the component;<br />
* the package includes <code>MavenCoordinates</code> that can be directly copied and pasted from the POM;<br />
* the package does '''not''' specify dependencies;<br />
* the package points to the main artefact of the Maven build;<br />
<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>streams</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>DataAccess</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<Files><br />
<File>${build.finalName}.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
=== Service Archive ===<br />
<br />
<code>streams</code> is responsible for generating its own SA as part of its Maven build. This requires dedicated logic in the POM but has the advantage that:<br />
<br />
* SA validity can be verified locally;<br />
* no SA configurations are required in ETICS;<br />
* the required build logic can be easily reused across components;<br />
<br />
<code>streams</code> uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation in the profile and other static files. The approach is best illustrated with a reference to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/streams sources]] of the <code>streams</code> library.<br />
<br />
<br />
As we discuss [[#Creating the ETICS Configurations|below]], <code>streams</code> has the final requirement to produce an interpolated profile during ETICS builds, outside the context of the SA.<br />
<code>streams</code> meets this requirement with the [http://maven.apache.org/plugins/maven-resources-plugin/ Maven Resources Plugin]. Again, consult the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/streams sources]] of the <code>streams</code> library to reuse this approach.<br />
<br />
= Services =<br />
The <code>tree-manager</code> service is a Maven component in gCube class <code>DataAccess</code>. <br />
<br />
== Project Structure ==<br />
<br />
Differently from a library, the service is associated with a number of distinguished build artefacts:<br />
<br />
* the service implementation (a Jar);<br />
* the stubs automatically generated from the WSDL interfaces (a Jar);<br />
* the archive required for deployment in gCore (a Gar);<br />
* the archive required for registration in the system (a SA);<br />
<br />
Producing these artefacts requires sharing a number of static and dynamic files (from profiles to WSDLs to pre-processed WSDLs). The usual Maven approach of mapping the artefacts onto different Maven projects is then genuinely difficult. It is instead more practical to use a single multi-module Maven project. The <code>tree-manager</code> uses this module structure:<br />
<br />
<pre><br />
tree-manager (parent module)------- (produces) --- (POM only)<br />
|<br />
|----- tree-manager-service (service impl module)------- (produces) --- jar, gar, SA<br />
| |<br />
| | (depends on)<br />
| |<br />
| \/<br />
|----- tree-manager-stubs (service stubs module)------- (produces) --- jar<br />
</pre><br />
<br />
As customary, the parent does not produce any artefact (is a POM-only project). It declares these coordinates in its POM:<br />
<br />
<source lang="xml"><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
<packaging>pom</packaging><br />
</source><br />
<br />
The parent physically includes the sub-modules;<br />
<br />
<source lang="xml"><br />
<modules><br />
<module>tree-manager-stubs</module><br />
<module>tree-manager-service</module><br />
</modules><br />
</source> <br />
<br />
Finally, the parent inherits from '''maven-parent''':<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
The submodules inherit from the parent:<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>tree-manager</artifactId><br />
<groupId>org.gcube.data.access</groupId><br />
<version>1.0.0-SNAPSHOT</version><br />
<relativePath>..</relativePath><br />
</parent><br />
</source><br />
<br />
and share its <code>version</code> and <code>groupId</code>. Their coordinates are then minimal:<br />
<br />
<source lang="xml"><br />
<artifactId>tree-manager-service</artifactId><br />
</source><br />
<br />
<source lang="xml"><br />
<artifactId>tree-manager-stubs</artifactId><br />
</source><br />
<br />
The <code>tree-manager-service</code> module produces a Jar of its own implementation, as well the GAR and the SA of the whole services as secondary artefacts with classifiers of, respectively, <code>gar</code> and <code>service archive</code>.<br />
<br />
The <code>tree-manager-stubs</code> module produces a Jar of its own implementation.<br />
<br />
Finally, the parent module coordinates the build of its sub-modules and centralises the declaration of common dependencies and configuration. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] to reuse this approach.<br />
<br />
== Generating Stubs and Gars ==<br />
<br />
The <code>tree-manager-stubs</code> uses the '''gCube Service Plugin''' to generate its classes from WSDL interfaces. <br />
<br />
Similarly, the <code>tree-manager-service</code> uses the plugin to generate its secondary GAR artefact. <br />
<br />
The configuration of the plugin is conveniently shared in the parent module.<br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] of the <code>streams</code> for details.<br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the service is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>DataAccess</Class><br />
<Name>tree-manager-service</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Main><br />
<Name>tree-manager-service</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager-service</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<GARArchive>tree-manager-service-1.0.0-SNAPSHOT.gar</GARArchive><br />
<PortType><br />
<Name>gcube/data/tm/binder</Name><br />
</PortType><br />
<PortType><br />
<Name>gcube/data/tm/reader</Name><br />
</PortType><br />
<PortType><br />
<Name>gcube/data/tm/writer</Name><br />
</PortType><br />
</Main><br />
<Software><br />
<Name>tree-manager-stubs</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager-stubs</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<Type>library</Type><br />
<Files><br />
<File>tree-manager-stubs-1.0.0-SNAPSHOT.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the profile includes two packages and this package corresponds to the main build artefacts of the submodules; <br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code>s of the submodules;<br />
* package versions are aligned with the Maven <code>version</code>s of the submodules;<br />
* package include <code>MavenCoordinates</code>s that can be directly copied and pasted from the POM of the submodules;<br />
* packages do '''not''' specify dependencies;<br />
* packages points to the main artefact of the Maven build of the submodules;<br />
<br />
<strike>Notice that, unlike libraries, services cannot easily use Maven variables in their profiles (as these artefacts are used for different purposes by different modules). Accordingly, developers needs to maintain versions used in profiles across releases. In particular, they need to:<br />
<br />
* replace SNAPSHOT versions with release versions in release branches;<br />
* advance SNAPSHOT versions in HEAD;</strike><br />
<br />
=== Profile Interpolation ===<br />
The interpolation of profiles is required in order to keep information in profile in sync with <code>pom.xml</code>, in particular versions and filenames (that contain versions) that change at each release.<br />
<br />
In order to generate <code>profile.xml</code> during build process you need to:<br />
# add in the service's parent <code>pom.xml</code> the invocation to maven-resources-plugin<br />
# place a ''template'' <code>profile.xml</code> in the <code>${distroDirectory}</code><br />
<br />
Invocation to <code>maven-resources-plugin</code> looks like the following (to be placed in the <plugins> section inside the <build> section of the pom):<br />
<source lang="xml"><br />
<!-- interpolates profiles and copies from distribution location to configuration <br />
location, where it is need for embedding into stub artifact, gar generation, <br />
and service archive. --><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-resources-plugin</artifactId><br />
<version>2.6</version><br />
<executions><br />
<execution><br />
<id>copy-profile</id><br />
<goals><br />
<goal>copy-resources</goal><br />
</goals><br />
<phase>process-resources</phase><br />
<configuration><br />
<outputDirectory>${configDirectory}</outputDirectory><br />
<resources><br />
<resource><br />
<directory>${distroDirectory}</directory><br />
<includes><br />
<include>profile.xml</include><br />
</includes><br />
<filtering>true</filtering><br />
</resource><br />
</resources><br />
</configuration><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
The ''template'' <code>profile.xml</code> looks like the following:<br />
<source lang="xml" highlight="12,16,18,22,26,30"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>TestClass</Class><br />
<Name>TestService</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Main><br />
<Name>TestService</Name><br />
<Version>${project.version}</Version><br />
<MavenCoordinates><br />
<groupId>org.example</groupId><br />
<artifactId>test-service</artifactId><br />
<version>${project.version}</version><br />
</MavenCoordinates><br />
<GARArchive>>test-service-${project.version}.gar</GARArchive><br />
</Main><br />
<Software><br />
<Name>test-stubs</Name><br />
<Version>${project.version}</Version><br />
<MavenCoordinates><br />
<groupId>org.example</groupId><br />
<artifactId>test-stubs</artifactId><br />
<version>${project.version}</version><br />
</MavenCoordinates><br />
<Type>library</Type><br />
<Files><br />
<File>test-stubs-${project.version}.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
In the example, property <code>${project.version}</code> will be substituted with the actual service's version at build-time. It is also possible, but not required, to parametrize also other values of <code>profile.xml</code> such as maven's coordinates and package names. You can see a more complex <code>profile.xml</code> ''template'' in the <code>tree-manager</code> project: http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-access/tree-manager/distro/profile.xml<br />
<br />
Maven will copy the final <code>profile.xml</code> in <code>${configDirectory}</code> during the <code>process-resources</code> phase of the build. From there it will be copied in the service-archive package.<br />
<br />
== Service Archive ==<br />
<br />
<code>tree-manager-service</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the module and its parent.<br />
<br />
As shown for [[#Libraries|libraries]], the service uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] for details.<br />
<br />
= Portlets =<br />
<br />
The <code>sample-portlet</code> portlet is a Maven based sample portlet based on GWT. Refer to the [http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources] for details.<br />
<br />
== Project Structure ==<br />
<br />
As for a library project, the portlet is associated with 2 distinguished build artefacts:<br />
<br />
* the archive required for deployment in a Liferay Container (a WAR);<br />
* the archive required for registration in the system (a SA);<br />
<br />
In order to generate automatically the project structure and the base pom, we start by customizing the maven archetype liferay-portlet-archetype v. 6.0.6.<br />
<br />
The generation of the project structure and the pom can be performed either from Eclipse ( by using the m2eclipse plugin) or by command line using the Maven SDK.<br />
<br />
On Eclipse the following steps must be followed:<br />
<br />
* FIle -> New -> Project<br />
* From the wizard select Maven->Maven Project<br />
* From the Select Archetype step, Select 'All Catalogs' from the Catalogs combobox and type 'portlet' on the Filter field.<br />
* Select the '''liferay-portlet-archetype''' version 6.0.6 archetype ( if does not appear on the list just unselect 'show the last version of the Archetype only")<br />
* In the next step you have to provide Maven coordinates for your project<br />
* groupid : <your group id> e.g org.gcube.portlets.admin<br />
* artifactId : <your artifactId> e.g. sample-admin-portlet<br />
* version : <your project version> e.g. 1.0.0-SNAPSHOT ( use always SNAPSHOT versions for development versions of a project)<br />
* package: <your portlet code package> e.g. org.gcube.portlets.admin.sample<br />
<br />
while from shell the following mvn command should be issued :<br />
<br />
<pre><br />
mvn archetype:generate \<br />
-DarchetypeArtifactId=liferay-theme-archetype \<br />
-DarchetypeGroupId=com.liferay.maven.archetypes \<br />
-DarchetypeVersion=6.0.6 \<br />
-DartifactId=<your artifactId> \<br />
-DgroupId=<your group id> \<br />
-Dversion=1.0.0-SNAPSHOT<br />
</pre><br />
<br />
Both approaches generate a project structure similar to the picture below:<br />
<br />
[[image:sample-portlet-project.png|Sample portlet project structure]]<br />
<br />
In order to customize the build procedure to the gCube needs we should open the pom.xml and apply the following changes:<br />
<br />
Configure the inheritance from '''maven-parent''' :<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
Configure some project properties:<br />
<br />
<source lang="xml"><br />
<properties><br />
<liferay.version>6.0.6</liferay.version><br />
<liferay.auto.deploy.dir>${system.CATALINA_HOME}/../deploy</liferay.auto.deploy.dir><br />
<distroDirectory>${project.basedir}/distro</distroDirectory><br />
<configDirectory>${project.basedir}/config</configDirectory><br />
</properties><br />
</source><br />
<br />
Add the plugin for the generation of the SA, which has to be included under the build plugins section:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<version>2.2</version><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
In the case of a GWT portlet the [http://mojo.codehaus.org/gwt-maven-plugin gwt-maven-plugin] should also be configured ( within build plugins section as well), by also selecting the proper gwt version ( in our example 2.4)<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.codehaus.mojo</groupId><br />
<artifactId>gwt-maven-plugin</artifactId><br />
<version>${gwt.version}</version><br />
<executions><br />
<execution><br />
<goals><br />
<goal>compile</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
The proper GWT version should be added to the pom properties :<br />
<br />
<source lang="xml"><br />
<gwt.version>2.4.0</gwt.version><br />
</source><br />
<br />
and GWT dependencies should be also configured:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-user</artifactId><br />
<version>${gwt.version}</version><br />
</dependency><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-servlet</artifactId><br />
<version>${gwt.version}</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet/ project sources]] for further details on the project configuration.<br />
<br />
== Generating Artifacts ==<br />
<br />
The <code>sample-portlet</code> uses the '''maven-compiler-plugin''' to compile the portlet code and the '''gwt-maven-plugin''' as well for GWT compilation.<br />
<br />
The generation of a war package can be triggered by typing :<br />
<br />
<pre><br />
mvn package <br />
</pre><br />
<br />
on your shell or running this build task within Eclipse.<br />
<br />
In both cases the war package will be generated under the ''target'' folder, e.g.:<br />
<br />
<pre><br />
target/sample-admin-portlet-1.0.0-SNAPSHOT.war<br />
</pre><br />
<br />
The generation of all project artifacts ( war, source jar, javadoc, SA) can be triggered by the build task :<br />
<br />
<pre><br />
mvn install <br />
</pre><br />
<br />
this will install as well all artifacts in your local repository, while the deployment of the artifacts to the remote gCube Maven repos can triggered by the build task:<br />
<br />
<pre><br />
mvn deploy <br />
</pre><br />
<br />
== Portlet dependencies ==<br />
<br />
Specific portlet dependencies ( e.g. GWT, service stubs ) should be packaged within the portlet war. All dependencies specified within the pom file without the ''provided'' scope will be copied inside the WEB-INF/lib folder at packaging time.<br />
<br />
For this reason all portlet specific dependencies should not specified that scope e.g:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.externals</groupId><br />
<artifactId>gwt-ext-patched</artifactId><br />
<version>2.0.5</version><br />
</dependency><br />
</source><br />
<br />
On the other hand the system dependencies ( the one contained within the [[GCube_Portal_Installation#Portal_Bundle_Libraries|Portal bundle]] ) should not be included inside the portlet war. As said in this case the ''provided'' scope should be specified, e.g.<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portal</groupId><br />
<artifactId>custom-portal-handler</artifactId><br />
<version>1.1.0-SNAPSHOT</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.4.0,1.5.0]</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
<br />
A subset of these libraries ( contained in a GHN) that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called <code>ghn-core-runtime</code>. A dependency on this Jar can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ghn-core-runtime</artifactId><br />
<version>1.0.0</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Note: a component that depends on <code>gcf</code> indirectly depends on <code>gun-core-runtime</code>.<br />
<br />
== Include gCube Guided Tour using Maven ==<br />
<br />
This particular component is not yet ported to maven so in order to be included in a maven based project it should be declared as system dependency as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portletsuser</groupId><br />
<artifactId>gcube-guidedtour-widget</artifactId><br />
<version>1.0.0</version><br />
<scope>system</scope><br />
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/gcube-guidedtour-widget.jar</systemPath> <br />
</dependency><br />
</source><br />
<br />
together with the gCube Widget Library:<br />
<br />
<source lang="xml"> <br />
<dependency><br />
<groupId>org.gcube.portlets.user</groupId><br />
<artifactId>gcubewidgets</artifactId><br />
<version>1.2.0</version><br />
<scope>system</scope><br />
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/gCube-Widgets-Library.jar</systemPath><br />
</dependency><br />
</source><br />
<br />
both components can be downloaded from the distribution repository [[http://grids16.eng.it:8080/BuildReport/browse/Recent_Builds]]. <br />
<br />
In case the portlet depends on the ''gcf'' component, some dependency resolution changes are required to your portlet in order to properly compile ( because of libraries clashes)<br />
<br />
The ghn-core-runtime component ( gcf dependency) should be excluded:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.5.1-SNAPSHOT,2.0.0)</version><br />
<scope>provided</scope><br />
<exclusions><br />
<exclusion><br />
<artifactId>ghn-core-runtime</artifactId><br />
<groupId>org.gcube.distribution</groupId><br />
</exclusion><br />
</exclusions><br />
</dependency><br />
</source><br />
<br />
<br />
and some new dependencies should be added:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>xerces</groupId><br />
<artifactId>xercesImpl</artifactId><br />
<version>2.9.1</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ws-addressing</artifactId><br />
<version>1.0</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>axis-patched</artifactId><br />
<version>1.2RC</version><br />
<scope>provided</scope><br />
</dependency><br />
<br />
</source><br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the portlet is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>PortletAdmin</Class><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.portles.admin</groupId><br />
<artifactId>sample-admin-portlet</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/sample-admin-portlet-1.0.0-SNAPSHOT.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code> of the project;<br />
* package versions are aligned with the Maven <code>version</code> of the project;<br />
* package include <code>MavenCoordinates</code> that can be directly copied and pasted from the POM of the project;<br />
* packages do '''not''' specify dependencies;<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>sample-portlet</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>PortletAdmin</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/${build.finalName}.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
== Service Archive ==<br />
<br />
<code>sample-portlet</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the project.<br />
<br />
As shown for [[#Libraries|libraries]], the portlet uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources]] for details.<br />
<br />
== Use Case: Create a new Mavenized GWT Portlet ==<br />
<br />
In order to create a new Mavenized GWT Portlet this detailed guide can be also followed [https://gcube.wiki.gcube-system.org/gcube/index.php/Create_a_new_Mavenized_gCube_GWT_Portlet_Guide]<br />
<br />
= Widgets =<br />
<br />
Falls in this category of components mainly GWT widgets or so called GWT Libraries, which can be used to package classes for mutualization and/or modularization. A Widget is a java archive (JAR) containing both classes and java sources for later GWT compilation and a gwt.xml module descriptor. <br />
<br />
In this section we refer to a sample Widget with maven structure (https://svn.research-infrastructures.eu/d4science/gcube/trunk/portal/mvn-widget-sample/)<br />
<br />
== Project Structure ==<br />
<br />
As for a other project, the portlet is associated with 2 distinguished build artefacts:<br />
<br />
* the archive required for deployment (a JAR);<br />
* the archive required for registration in the system (a SA);<br />
<br />
In order to generate automatically the project structure and the base pom, we start by customizing the maven archetype liferay-portlet-archetype v. 6.0.6.<br />
<br />
The generation of the project structure and the pom can be performed either from Eclipse ( by using the m2eclipse plugin) or by command line using the Maven SDK.<br />
<br />
On Eclipse the following steps must be followed:<br />
<br />
* FIle -> New -> Project<br />
* From the wizard select Maven->Maven Project<br />
* From the Select Archetype step, Select 'All Catalogs' from the Catalogs combobox and type 'portlet' on the Filter field.<br />
* Select the '''maven-archetype-quickstart''' version 1.1.0 archetype ( if does not appear on the list just unselect 'show the last version of the Archetype only")<br />
* In the next step you have to provide Maven coordinates for your project<br />
* groupid : <your group id> e.g org.gcube.portlets.admin<br />
* artifactId : <your artifactId> e.g. sample-admin-portlet<br />
* version : <your project version> e.g. 1.0.0-SNAPSHOT ( use always SNAPSHOT versions for development versions of a project)<br />
* package: <your portlet code package> e.g. org.gcube.portlets.admin.sample<br />
<br />
while from shell the following mvn command should be issued :<br />
<br />
<pre><br />
mvn archetype:generate \<br />
-DarchetypeArtifactId=maven-archetype-quickstart \<br />
-DarchetypeGroupId=com.liferay.maven.archetypes \<br />
-DarchetypeVersion=1.1.0 \<br />
-DartifactId=<your artifactId> \<br />
-DgroupId=<your group id> \<br />
-Dversion=1.0.0-SNAPSHOT<br />
</pre><br />
<br />
Both approaches generate a project structure similar to the picture below:<br />
<br />
[[image:sample-widget-project.png|Sample widget project structure]]<br />
<br />
In order to customize the build procedure to the gCube needs we should open the pom.xml and apply the following changes:<br />
<br />
Configure the inheritance from '''maven-parent''' :<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
Configure some project properties:<br />
<br />
<source lang="xml"><br />
<properties><br />
<distroDirectory>${project.basedir}/distro</distroDirectory><br />
<configDirectory>${project.basedir}/config</configDirectory><br />
</properties><br />
</source><br />
<br />
Add the plugin for the generation of the SA, which has to be included under the build plugins section:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<version>2.2</version><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
In the case of a GWT widget the [http://mojo.codehaus.org/gwt-maven-plugin gwt-maven-plugin] should also be configured ( within build plugins section as well), by also selecting the proper gwt version ( in our example 2.4)<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.codehaus.mojo</groupId><br />
<artifactId>gwt-maven-plugin</artifactId><br />
<version>${gwt.version}</version><br />
<executions><br />
<execution><br />
<goals><br />
<goal>compile</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
The proper GWT version should be added to the pom properties :<br />
<br />
<source lang="xml"><br />
<gwt.version>2.4.0</gwt.version><br />
</source><br />
<br />
and GWT dependencies should be also configured:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-user</artifactId><br />
<version>${gwt.version}</version><br />
</dependency><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-servlet</artifactId><br />
<version>${gwt.version}</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/mvn-widget-sample/ project sources]] for further details on the project configuration.<br />
<br />
== Generating Artifacts ==<br />
<br />
The <code>sample-portlet</code> uses the '''maven-compiler-plugin''' to compile the portlet code and the '''gwt-maven-plugin''' as well for GWT compilation.<br />
<br />
The generation of a war package can be triggered by typing :<br />
<br />
<pre><br />
mvn package <br />
</pre><br />
<br />
on your shell or running this build task within Eclipse.<br />
<br />
In both cases the war package will be generated under the ''target'' folder, e.g.:<br />
<br />
<pre><br />
target/sample-admin-portlet-1.0.0-SNAPSHOT.war<br />
</pre><br />
<br />
The generation of all project artifacts ( war, source jar, javadoc, SA) can be triggered by the build task :<br />
<br />
<pre><br />
mvn install <br />
</pre><br />
<br />
this will install as well all artifacts in your local repository, while the deployment of the artifacts to the remote gCube Maven repos can triggered by the build task:<br />
<br />
<pre><br />
mvn deploy <br />
</pre><br />
<br />
== Widget dependencies ==<br />
<br />
The system dependencies ( the one contained within the [[GCube_Portal_Installation#Portal_Bundle_Libraries|Portal bundle]] ) should not be included inside the portlet war. As said in this case the ''provided'' scope should be specified, e.g.<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portal</groupId><br />
<artifactId>custom-portal-handler</artifactId><br />
<version>1.1.0-SNAPSHOT</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.4.0,1.5.0]</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the widget is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>PortletAdmin</Class><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.portles.admin</groupId><br />
<artifactId>sample-admin-portlet</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/sample-admin-portlet-1.0.0-SNAPSHOT.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code> of the project;<br />
* package versions are aligned with the Maven <code>version</code> of the project;<br />
* package include <code>MavenCoordinates</code> that can be directly copied and pasted from the POM of the project;<br />
* packages do '''not''' specify dependencies;<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>sample-portlet</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>PortletAdmin</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/${build.finalName}.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
== Service Archive ==<br />
<br />
<code>sample-portlet</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the project.<br />
<br />
As shown for [[#Libraries|libraries]], the portlet uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources]] for details.</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Developing_gCube_Maven_Components&diff=29730Developing gCube Maven Components2017-10-30T11:32:00Z<p>Gabriele.giammatteo: /* Parent */</p>
<hr />
<div>[[Category: Developer's Guide]]<br />
__TOC__<br />
<br />
= Common =<br />
== Maven Coordinates ==<br />
Maven components' coordinates must follow these conventions:<br />
<br />
* their <code>groupId</code> must be of the following form:<br />
<br />
<pre>org.gcube.<class></pre><br />
<br />
: where ''class'' identifies a functionally correlated set of components and helps a range of users (developers, administrator) to discover components in various UIs/reports used within the system and in tools. Logically, the class corresponds to the gCube class of the component's, as specified in its gCube Software Profile. However, more specific, or else cross-class groups may be agreed upon among developers. In all cases, the group should align with official Maven [http://maven.apache.org/guides/mini/guide-naming-conventions.html guidelines]. By agreement, for example, components in gCube class <code>DataAccess</code> declare a <code>groupId</code> of <code>org.gcube.data.access</code> and those in gCube class <code>DataTransfer</code> declare a <code>groupId</code> of <code>org.gcube.data.transfer</code>. On the other hand, also by agreement, components in gCube class <code>InformationSystem</code> declare a <code>groupId</code> of <code>org.gcube.informationsystem</code>.<br />
<br />
* their <code>artifactId</code> must follow Maven conventions (e.g. ''avoid capital letters or punctuation other than hyphens'').<br />
<br />
== Parent ==<br />
All gCube components must inherit from '''maven-parent'''. This ensure and promotes compliance with system-wide requirements, from the enforcement of minimal Java and Maven versions to generation and packaging of Javadoc documentation and component sources.<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
<br />
The maven-parent defines:<br />
* several system-wide properties like <code>gcube.license</code>, <code>gcube.wikiRoot</code>, <code>distroDirectory</code><br />
* the default Java version (see more in [[Developing_gCube_Maven_Components#Java_Version|Java Version]])<br />
* the execution of some plugins that run for every component like: <code>maven-javadoc-plugin</code>, <code>maven-source-plugin</code><br />
* the configuration of the <code>maven-assembly-plugin</code> to generate some gCube-specific packages: <code>servicearchive</code>, <code>uberjar</code> and <code>source-package</code> (see more in [[Developing_gCube_Maven_Components#Java_Version|Artifact Types]])<br />
* the logic to enable the generation of the source package<br />
<br />
The '''maven-parent''' is at http://maven.research-infrastructures.eu/nexus/service/local/repositories/gcube-externals/content/org/gcube/tools/maven-parent/1.0.0/maven-parent-1.0.0.pom<br />
<br />
== The distro folder ==<br />
This folder contains files that will be distributed along with the code in the distribution packages. Further details are in the [[Maven distro directory layout]] page.<br />
<br />
== Service Archive (SA) ==<br />
Whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary <code>tar.gz</code> artefact with classifier <code>servicearchive</code>. Note that the SA of Maven components does ''not'' need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);<br />
<br />
* for each deployable artefact they produce (primary or secondary), they must include the Maven coordinates of the arteact in the corresponding package section of the gCube Software Profile. Specifying package dependencies in the Software Profile, on the other hand, is no longer required;<br />
<br />
* whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary <code>tar.gz</code> artefact with classifier <code>servicearchive</code>. Note that the SA of Maven components does ''not'' need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);<br />
<br />
== Dependencies ==<br />
Service and libraries that depend on other gCube components (Maven-based or Ant-based) can find them in the gCube Maven Repositories: <code>gcube-snapshots</code> <strike>and <code>gcube-releases</code></strike>. <br />
<br />
=== gCube Maven Repositories ===<br />
As a one-off task, developers must configure these repositories in the <code>~/.m2/settings.xml</code> file of their local Maven installation, as follows:<br />
<br />
<source lang="xml"><br />
<settings xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"<br />
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd"><br />
<br />
<servers><br />
<server><br />
<id>gcube-snapshots</id><br />
<username>gcube-user</username><br />
<password>maven</password><br />
</server><br />
</servers><br />
<br />
<br />
<profiles><br />
<profile><br />
<id>gcube</id><br />
<repositories><br />
<repository><br />
<id>gcube-snapshots</id><br />
<name>gCube Snapshots</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
</snapshots><br />
</repository><br />
<repository><br />
<id>gcube-externals</id><br />
<name>gCube Externals</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals</url><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
</repository><br />
</repositories><br />
<br />
<pluginRepositories><br />
<pluginRepository><br />
<id>gcube-snapshots</id><br />
<name>gCube Snapshots</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
</snapshots><br />
</pluginRepository><br />
<pluginRepository><br />
<id>gcube-externals</id><br />
<name>gCube Externals</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals</url><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
</pluginRepository><br />
</pluginRepositories><br />
<br />
</profile><br />
</profiles><br />
<br />
<activeProfiles><br />
<activeProfile>gcube</activeProfile><br />
</activeProfiles><br />
</settings><br />
</source><br />
<br />
<br />
'''Note:Developers must not configure these repositories in their POMs''', only in <code>settings.xml</code>. <br />
<br />
This is to:<br />
* keep their projects independent from future changes in repository locations; <br />
* avoid interference with ETICS builds, where access to these repositories may comprise correct system integration;<br />
<br />
With this settings.xml, developers will have access on their local development environment to development versions (gcube-snapshots) and externals (gcube-externals). This is the correct behaviour for most of use cases. If access to staging or releases repositories is necessary (e.g. patching a component already released in the branch where dependencies versions in the release are not in snapshots), developer should modify his settings.xml to point to gcube-staging(releases) instead of gcube-snapshots.<br />
<br />
=== Third-party Repositories ===<br />
Using the the <code>settings.xml</code> aforementioned, any dependency declared in pom.xml that can be found in one of gCube repositories or in Maven Central will be resolved at build time and deployment time without any further configuration. Frequently it may happens that a gCube component needs software stored on a public Maven repository not in on of those repositories. In this case, Maven provides an easy solution for declaring additional repositories for dependency resolution by adding a <repositories> section in the pom.xml of the project. E.g.:<br />
<br />
<source lang="xml" highlight="3,4"><br />
<repositories><br />
<repository><br />
<id>[repository-id]</id><br />
<url>[repository URL]</url><br />
</repository><br />
</repositories><br />
</source><br />
<br />
Though technically possible to add any Maven repository in this way, in order to preserve the quality of gCube software it is required to check trustworthiness of its dependencies disallowing usage of certain external software. To enforce such a control over public Maven repositories, '''gCube components are allowed to use external repository not directly but only through a proxy repository''' (a.k.a. shadow repository) created on CNR's Nexus Repository Portal (http://maven.research-infrastructures.eu/nexus/).<br />
<br />
List of all repositories in Nexus Portal is available at http://maven.research-infrastructures.eu/nexus/index.html#view-repositories<br />
<br />
'''Note''': if a component uses a public Maven repository directly, it will work for building but would fail in deployment phase. In fact, Software Gateway is only able to download software accessible from Nexus.<br />
<br />
==== Request new proxy repository in Nexus ====<br />
Proxy Repositories on Nexus are created by the Release Manager. To request a new proxy a new ticket must be created.<br />
<br />
Details of ticket are:<br />
# open a ticket in tracker "Support" in gCube ticketing system (at https://support.d4science.org/projects/gcube/)<br />
# ticket's '''description''' must report:<br />
#* URL of repository to add<br />
#* organization which hosts the repository<br />
#* which artifacts are needed from that repository<br />
#* which components needs those artifacts<br />
# '''Sprint''' field should be set to the gCube release in which the component is expected to be released <br />
# '''Category''' field must report the gCube subsystem that needs the external repository<br />
# '''Assignee''' is the Release Manager<br />
<br />
An example of ticket for requesting a new repository mirror is reported in the figure below.<br />
[[Image:New_maven_mirror_repo.png|600px|center|Ticket for requesting a new Mirror Repository]]<br />
<br />
Release Manager will process the ticket checking that no issues are foreseen from the usage of software stored in the repository. If request is accepted, a new proxy repository is created and the repository's URL will be communicated publicly on gCube's developers mailinglist (a.k.a. TCOM mailinglist).<br />
<br />
Knowing the mirror repository URL, developer can use it adding following lines in pom.xml. E.g.<br />
<br />
<source lang="xml" highlight="3,4"><br />
<repositories><br />
<repository><br />
<id>gcube-ext-osgeo</id><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/osgeo/</url><br />
</repository><br />
</repositories><br />
</source><br />
<br />
'''Note''': it is possible that software stored on proxy repositories is not included in search results on Nexus. This happens for those repositories that does not have indexes (or have indexes in a format that Nexus is not able to download). Anyway, once Nexus has downloaded artifacts from the original repository for the first time (e.g. because a compilation of a pom that uses the proxy repository), creates its own indexes and those artifacts will become available also for local search.<br />
<br />
=== Dependency Versions ===<br />
<br />
ETICS builds ensure correct integration of gCube components, regardless of the versions their dependencies. It also ensures that components are released with fixed versions for their dependencies, i.e. ensures build reproducibility.<br />
<br />
Accordingly, developers can be liberal in the choice of dependency versions. i.e. rely on version ranges to:<br />
<br />
* minimise the requirements for version management;<br />
* be readily informed of any mis-alignment with their dependencies;<br />
<br />
As we start adopting Maven, the lower-bound of version ranges will be a SNAPSHOT version. For example, a dependency on the <code>streams</code> library can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>streams</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
</dependency><br />
</source><br />
<br />
With this dependency, a component will be soon notified of a non-retro compatible change that, contrary to conventions, <code>streams</code> may have introduced in, say, version <code>1.3.0</code> (typically the day after at the latest). The notification will be delivered in the IDE, assuming the availability of Maven integration (e.g. <code>m2e</code> in Eclipse). <br />
The component is in fact free to open the range further if it wants to be aligned with its dependencies across major versions.<br />
<br />
=== Common Dependencies ===<br />
<br />
Many gCube components depend on <code>gcf</code>, which is now available as a Maven component since version <code>1.4</code>. At the time of writing, the component has not been released yet. A dependency on it is then specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
The scope <code>provided</code> indicates that these dependency will be satisfied in the development environment (the gHN) and should not be included in GAR and SA archives.<br />
<br />
Some components depend on the libraries available in all gHNs, though they do not depend on <code>gcf</code> itself. The subset of these libraries that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called <code>ghn-core-runtime</code>. A dependency on this Jar can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ghn-core-runtime</artifactId><br />
<version>1.0.0</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Note: a component that depends on <code>gcf</code> indirectly depends on <code>gun-core-runtime</code>.<br />
<br />
Another common dependency is on the gCube gRS2, an Ant component that the build infrastructure make available in Maven repositories. A dependency on this component can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.execution</groupId><br />
<artifactId>grs2library</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
</dependency><br />
</source><br />
<br />
== Java Version ==<br />
<br />
In Maven the Java version of the source code and of the generated classes can be controlled using the configuration of the <code>maven-compiler-plugin</code>. E.g.:<br />
<syntaxhighlight lang="xml" highlight="5,6"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-compiler-plugin</artifactId><br />
<configuration><br />
<source>1.7</source><br />
<target>1.8</target><br />
</configuration><br />
</plugin><br />
</syntaxhighlight><br />
<br />
<br />
In gCube, this mechanism is exploited to enforce the usage of the same Java version for all components. In particular, the maven-parent defines the properties <code>java_version</code>, <code>maven.compiler.source</code> and <code>maven.compiler.target</code> and use these properties in the configuration of the <code>maven-compiler-plugin</code>.<br />
<syntaxhighlight lang="xml" highlight="15,16"><br />
...<br />
<properties><br />
<java_version>1.8</java_version><br />
<maven.compiler.source>${java_version}</maven.compiler.source><br />
<maven.compiler.target>${java_version}</maven.compiler.target><br />
...<br />
</properties><br />
<br />
...<br />
<br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-compiler-plugin</artifactId><br />
<configuration><br />
<target>${maven.compiler.target}</target><br />
<source>${maven.compiler.source}</source><br />
</configuration><br />
</plugin><br />
...<br />
</syntaxhighlight><br />
<br />
'''All the gCube components inherit this default configuration from <code>maven-parent</code> and it is not recommended to override it in the component's pom.xml'''.<br />
<br />
If it is really needed to override these values at component's pom.xml level, instead of specify a new configuration for the maven-compiler-plugin, the value of the property should be re-defined:<br />
<br />
<syntaxhighlight lang="xml" highlight="3"><br />
...<br />
<properties><br />
<maven.compiler.source>1.7</maven.compiler.source><br />
...<br />
</properties><br />
<br />
</syntaxhighlight><br />
<br />
<br />
This configuration is flexible enough to allow to customize the Java compiler source and target values depending on different factors in the integration builds (e.g. during ETICS builds gCore components are built with Java 7, all the others with Java 8).<br />
<br />
= Libraries =<br />
The <code>streams</code> library is a Maven component in gCube class <code>DataAccess</code>. We will use it as example of a library component in Maven.<br />
<br />
=== Profile ===<br />
<br />
The gCube Software profile of the library is as follows:<br />
<br />
<source lang="xml" highlight="14-18"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>Embedded Domain-Specific Language for Stream Transformations</Description><br />
<Class>DataAccess</Class><br />
<Name>streams</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>streams</Name><br />
<Version>1.0.0-SNAPSHOT</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>streams</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<Files><br />
<File>streams-1.0.0-SNAPSHOT.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the profile includes a single package and this package corresponds to the main build artefact of the component; <br />
* the whole profile and the package have usual gCube coordinates;<br />
* the package name is aligned with the Maven <code>artifactId</code> of the component;<br />
* the package version is aligned with the Maven <code>version</code> of the component;<br />
* the package includes <code>MavenCoordinates</code> that can be directly copied and pasted from the POM;<br />
* the package does '''not''' specify dependencies;<br />
* the package points to the main artefact of the Maven build;<br />
<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>streams</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>DataAccess</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<Files><br />
<File>${build.finalName}.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
=== Service Archive ===<br />
<br />
<code>streams</code> is responsible for generating its own SA as part of its Maven build. This requires dedicated logic in the POM but has the advantage that:<br />
<br />
* SA validity can be verified locally;<br />
* no SA configurations are required in ETICS;<br />
* the required build logic can be easily reused across components;<br />
<br />
<code>streams</code> uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation in the profile and other static files. The approach is best illustrated with a reference to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/streams sources]] of the <code>streams</code> library.<br />
<br />
<br />
As we discuss [[#Creating the ETICS Configurations|below]], <code>streams</code> has the final requirement to produce an interpolated profile during ETICS builds, outside the context of the SA.<br />
<code>streams</code> meets this requirement with the [http://maven.apache.org/plugins/maven-resources-plugin/ Maven Resources Plugin]. Again, consult the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/streams sources]] of the <code>streams</code> library to reuse this approach.<br />
<br />
= Services =<br />
The <code>tree-manager</code> service is a Maven component in gCube class <code>DataAccess</code>. <br />
<br />
== Project Structure ==<br />
<br />
Differently from a library, the service is associated with a number of distinguished build artefacts:<br />
<br />
* the service implementation (a Jar);<br />
* the stubs automatically generated from the WSDL interfaces (a Jar);<br />
* the archive required for deployment in gCore (a Gar);<br />
* the archive required for registration in the system (a SA);<br />
<br />
Producing these artefacts requires sharing a number of static and dynamic files (from profiles to WSDLs to pre-processed WSDLs). The usual Maven approach of mapping the artefacts onto different Maven projects is then genuinely difficult. It is instead more practical to use a single multi-module Maven project. The <code>tree-manager</code> uses this module structure:<br />
<br />
<pre><br />
tree-manager (parent module)------- (produces) --- (POM only)<br />
|<br />
|----- tree-manager-service (service impl module)------- (produces) --- jar, gar, SA<br />
| |<br />
| | (depends on)<br />
| |<br />
| \/<br />
|----- tree-manager-stubs (service stubs module)------- (produces) --- jar<br />
</pre><br />
<br />
As customary, the parent does not produce any artefact (is a POM-only project). It declares these coordinates in its POM:<br />
<br />
<source lang="xml"><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
<packaging>pom</packaging><br />
</source><br />
<br />
The parent physically includes the sub-modules;<br />
<br />
<source lang="xml"><br />
<modules><br />
<module>tree-manager-stubs</module><br />
<module>tree-manager-service</module><br />
</modules><br />
</source> <br />
<br />
Finally, the parent inherits from '''maven-parent''':<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
The submodules inherit from the parent:<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>tree-manager</artifactId><br />
<groupId>org.gcube.data.access</groupId><br />
<version>1.0.0-SNAPSHOT</version><br />
<relativePath>..</relativePath><br />
</parent><br />
</source><br />
<br />
and share its <code>version</code> and <code>groupId</code>. Their coordinates are then minimal:<br />
<br />
<source lang="xml"><br />
<artifactId>tree-manager-service</artifactId><br />
</source><br />
<br />
<source lang="xml"><br />
<artifactId>tree-manager-stubs</artifactId><br />
</source><br />
<br />
The <code>tree-manager-service</code> module produces a Jar of its own implementation, as well the GAR and the SA of the whole services as secondary artefacts with classifiers of, respectively, <code>gar</code> and <code>service archive</code>.<br />
<br />
The <code>tree-manager-stubs</code> module produces a Jar of its own implementation.<br />
<br />
Finally, the parent module coordinates the build of its sub-modules and centralises the declaration of common dependencies and configuration. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] to reuse this approach.<br />
<br />
== Generating Stubs and Gars ==<br />
<br />
The <code>tree-manager-stubs</code> uses the '''gCube Service Plugin''' to generate its classes from WSDL interfaces. <br />
<br />
Similarly, the <code>tree-manager-service</code> uses the plugin to generate its secondary GAR artefact. <br />
<br />
The configuration of the plugin is conveniently shared in the parent module.<br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] of the <code>streams</code> for details.<br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the service is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>DataAccess</Class><br />
<Name>tree-manager-service</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Main><br />
<Name>tree-manager-service</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager-service</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<GARArchive>tree-manager-service-1.0.0-SNAPSHOT.gar</GARArchive><br />
<PortType><br />
<Name>gcube/data/tm/binder</Name><br />
</PortType><br />
<PortType><br />
<Name>gcube/data/tm/reader</Name><br />
</PortType><br />
<PortType><br />
<Name>gcube/data/tm/writer</Name><br />
</PortType><br />
</Main><br />
<Software><br />
<Name>tree-manager-stubs</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager-stubs</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<Type>library</Type><br />
<Files><br />
<File>tree-manager-stubs-1.0.0-SNAPSHOT.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the profile includes two packages and this package corresponds to the main build artefacts of the submodules; <br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code>s of the submodules;<br />
* package versions are aligned with the Maven <code>version</code>s of the submodules;<br />
* package include <code>MavenCoordinates</code>s that can be directly copied and pasted from the POM of the submodules;<br />
* packages do '''not''' specify dependencies;<br />
* packages points to the main artefact of the Maven build of the submodules;<br />
<br />
<strike>Notice that, unlike libraries, services cannot easily use Maven variables in their profiles (as these artefacts are used for different purposes by different modules). Accordingly, developers needs to maintain versions used in profiles across releases. In particular, they need to:<br />
<br />
* replace SNAPSHOT versions with release versions in release branches;<br />
* advance SNAPSHOT versions in HEAD;</strike><br />
<br />
=== Profile Interpolation ===<br />
The interpolation of profiles is required in order to keep information in profile in sync with <code>pom.xml</code>, in particular versions and filenames (that contain versions) that change at each release.<br />
<br />
In order to generate <code>profile.xml</code> during build process you need to:<br />
# add in the service's parent <code>pom.xml</code> the invocation to maven-resources-plugin<br />
# place a ''template'' <code>profile.xml</code> in the <code>${distroDirectory}</code><br />
<br />
Invocation to <code>maven-resources-plugin</code> looks like the following (to be placed in the <plugins> section inside the <build> section of the pom):<br />
<source lang="xml"><br />
<!-- interpolates profiles and copies from distribution location to configuration <br />
location, where it is need for embedding into stub artifact, gar generation, <br />
and service archive. --><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-resources-plugin</artifactId><br />
<version>2.6</version><br />
<executions><br />
<execution><br />
<id>copy-profile</id><br />
<goals><br />
<goal>copy-resources</goal><br />
</goals><br />
<phase>process-resources</phase><br />
<configuration><br />
<outputDirectory>${configDirectory}</outputDirectory><br />
<resources><br />
<resource><br />
<directory>${distroDirectory}</directory><br />
<includes><br />
<include>profile.xml</include><br />
</includes><br />
<filtering>true</filtering><br />
</resource><br />
</resources><br />
</configuration><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
The ''template'' <code>profile.xml</code> looks like the following:<br />
<source lang="xml" highlight="12,16,18,22,26,30"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>TestClass</Class><br />
<Name>TestService</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Main><br />
<Name>TestService</Name><br />
<Version>${project.version}</Version><br />
<MavenCoordinates><br />
<groupId>org.example</groupId><br />
<artifactId>test-service</artifactId><br />
<version>${project.version}</version><br />
</MavenCoordinates><br />
<GARArchive>>test-service-${project.version}.gar</GARArchive><br />
</Main><br />
<Software><br />
<Name>test-stubs</Name><br />
<Version>${project.version}</Version><br />
<MavenCoordinates><br />
<groupId>org.example</groupId><br />
<artifactId>test-stubs</artifactId><br />
<version>${project.version}</version><br />
</MavenCoordinates><br />
<Type>library</Type><br />
<Files><br />
<File>test-stubs-${project.version}.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
In the example, property <code>${project.version}</code> will be substituted with the actual service's version at build-time. It is also possible, but not required, to parametrize also other values of <code>profile.xml</code> such as maven's coordinates and package names. You can see a more complex <code>profile.xml</code> ''template'' in the <code>tree-manager</code> project: http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-access/tree-manager/distro/profile.xml<br />
<br />
Maven will copy the final <code>profile.xml</code> in <code>${configDirectory}</code> during the <code>process-resources</code> phase of the build. From there it will be copied in the service-archive package.<br />
<br />
== Service Archive ==<br />
<br />
<code>tree-manager-service</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the module and its parent.<br />
<br />
As shown for [[#Libraries|libraries]], the service uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] for details.<br />
<br />
= Portlets =<br />
<br />
The <code>sample-portlet</code> portlet is a Maven based sample portlet based on GWT. Refer to the [http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources] for details.<br />
<br />
== Project Structure ==<br />
<br />
As for a library project, the portlet is associated with 2 distinguished build artefacts:<br />
<br />
* the archive required for deployment in a Liferay Container (a WAR);<br />
* the archive required for registration in the system (a SA);<br />
<br />
In order to generate automatically the project structure and the base pom, we start by customizing the maven archetype liferay-portlet-archetype v. 6.0.6.<br />
<br />
The generation of the project structure and the pom can be performed either from Eclipse ( by using the m2eclipse plugin) or by command line using the Maven SDK.<br />
<br />
On Eclipse the following steps must be followed:<br />
<br />
* FIle -> New -> Project<br />
* From the wizard select Maven->Maven Project<br />
* From the Select Archetype step, Select 'All Catalogs' from the Catalogs combobox and type 'portlet' on the Filter field.<br />
* Select the '''liferay-portlet-archetype''' version 6.0.6 archetype ( if does not appear on the list just unselect 'show the last version of the Archetype only")<br />
* In the next step you have to provide Maven coordinates for your project<br />
* groupid : <your group id> e.g org.gcube.portlets.admin<br />
* artifactId : <your artifactId> e.g. sample-admin-portlet<br />
* version : <your project version> e.g. 1.0.0-SNAPSHOT ( use always SNAPSHOT versions for development versions of a project)<br />
* package: <your portlet code package> e.g. org.gcube.portlets.admin.sample<br />
<br />
while from shell the following mvn command should be issued :<br />
<br />
<pre><br />
mvn archetype:generate \<br />
-DarchetypeArtifactId=liferay-theme-archetype \<br />
-DarchetypeGroupId=com.liferay.maven.archetypes \<br />
-DarchetypeVersion=6.0.6 \<br />
-DartifactId=<your artifactId> \<br />
-DgroupId=<your group id> \<br />
-Dversion=1.0.0-SNAPSHOT<br />
</pre><br />
<br />
Both approaches generate a project structure similar to the picture below:<br />
<br />
[[image:sample-portlet-project.png|Sample portlet project structure]]<br />
<br />
In order to customize the build procedure to the gCube needs we should open the pom.xml and apply the following changes:<br />
<br />
Configure the inheritance from '''maven-parent''' :<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
Configure some project properties:<br />
<br />
<source lang="xml"><br />
<properties><br />
<liferay.version>6.0.6</liferay.version><br />
<liferay.auto.deploy.dir>${system.CATALINA_HOME}/../deploy</liferay.auto.deploy.dir><br />
<distroDirectory>${project.basedir}/distro</distroDirectory><br />
<configDirectory>${project.basedir}/config</configDirectory><br />
</properties><br />
</source><br />
<br />
Add the plugin for the generation of the SA, which has to be included under the build plugins section:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<version>2.2</version><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
In the case of a GWT portlet the [http://mojo.codehaus.org/gwt-maven-plugin gwt-maven-plugin] should also be configured ( within build plugins section as well), by also selecting the proper gwt version ( in our example 2.4)<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.codehaus.mojo</groupId><br />
<artifactId>gwt-maven-plugin</artifactId><br />
<version>${gwt.version}</version><br />
<executions><br />
<execution><br />
<goals><br />
<goal>compile</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
The proper GWT version should be added to the pom properties :<br />
<br />
<source lang="xml"><br />
<gwt.version>2.4.0</gwt.version><br />
</source><br />
<br />
and GWT dependencies should be also configured:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-user</artifactId><br />
<version>${gwt.version}</version><br />
</dependency><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-servlet</artifactId><br />
<version>${gwt.version}</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet/ project sources]] for further details on the project configuration.<br />
<br />
== Generating Artifacts ==<br />
<br />
The <code>sample-portlet</code> uses the '''maven-compiler-plugin''' to compile the portlet code and the '''gwt-maven-plugin''' as well for GWT compilation.<br />
<br />
The generation of a war package can be triggered by typing :<br />
<br />
<pre><br />
mvn package <br />
</pre><br />
<br />
on your shell or running this build task within Eclipse.<br />
<br />
In both cases the war package will be generated under the ''target'' folder, e.g.:<br />
<br />
<pre><br />
target/sample-admin-portlet-1.0.0-SNAPSHOT.war<br />
</pre><br />
<br />
The generation of all project artifacts ( war, source jar, javadoc, SA) can be triggered by the build task :<br />
<br />
<pre><br />
mvn install <br />
</pre><br />
<br />
this will install as well all artifacts in your local repository, while the deployment of the artifacts to the remote gCube Maven repos can triggered by the build task:<br />
<br />
<pre><br />
mvn deploy <br />
</pre><br />
<br />
== Portlet dependencies ==<br />
<br />
Specific portlet dependencies ( e.g. GWT, service stubs ) should be packaged within the portlet war. All dependencies specified within the pom file without the ''provided'' scope will be copied inside the WEB-INF/lib folder at packaging time.<br />
<br />
For this reason all portlet specific dependencies should not specified that scope e.g:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.externals</groupId><br />
<artifactId>gwt-ext-patched</artifactId><br />
<version>2.0.5</version><br />
</dependency><br />
</source><br />
<br />
On the other hand the system dependencies ( the one contained within the [[GCube_Portal_Installation#Portal_Bundle_Libraries|Portal bundle]] ) should not be included inside the portlet war. As said in this case the ''provided'' scope should be specified, e.g.<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portal</groupId><br />
<artifactId>custom-portal-handler</artifactId><br />
<version>1.1.0-SNAPSHOT</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.4.0,1.5.0]</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
<br />
A subset of these libraries ( contained in a GHN) that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called <code>ghn-core-runtime</code>. A dependency on this Jar can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ghn-core-runtime</artifactId><br />
<version>1.0.0</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Note: a component that depends on <code>gcf</code> indirectly depends on <code>gun-core-runtime</code>.<br />
<br />
== Include gCube Guided Tour using Maven ==<br />
<br />
This particular component is not yet ported to maven so in order to be included in a maven based project it should be declared as system dependency as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portletsuser</groupId><br />
<artifactId>gcube-guidedtour-widget</artifactId><br />
<version>1.0.0</version><br />
<scope>system</scope><br />
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/gcube-guidedtour-widget.jar</systemPath> <br />
</dependency><br />
</source><br />
<br />
together with the gCube Widget Library:<br />
<br />
<source lang="xml"> <br />
<dependency><br />
<groupId>org.gcube.portlets.user</groupId><br />
<artifactId>gcubewidgets</artifactId><br />
<version>1.2.0</version><br />
<scope>system</scope><br />
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/gCube-Widgets-Library.jar</systemPath><br />
</dependency><br />
</source><br />
<br />
both components can be downloaded from the distribution repository [[http://grids16.eng.it:8080/BuildReport/browse/Recent_Builds]]. <br />
<br />
In case the portlet depends on the ''gcf'' component, some dependency resolution changes are required to your portlet in order to properly compile ( because of libraries clashes)<br />
<br />
The ghn-core-runtime component ( gcf dependency) should be excluded:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.5.1-SNAPSHOT,2.0.0)</version><br />
<scope>provided</scope><br />
<exclusions><br />
<exclusion><br />
<artifactId>ghn-core-runtime</artifactId><br />
<groupId>org.gcube.distribution</groupId><br />
</exclusion><br />
</exclusions><br />
</dependency><br />
</source><br />
<br />
<br />
and some new dependencies should be added:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>xerces</groupId><br />
<artifactId>xercesImpl</artifactId><br />
<version>2.9.1</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ws-addressing</artifactId><br />
<version>1.0</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>axis-patched</artifactId><br />
<version>1.2RC</version><br />
<scope>provided</scope><br />
</dependency><br />
<br />
</source><br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the portlet is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>PortletAdmin</Class><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.portles.admin</groupId><br />
<artifactId>sample-admin-portlet</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/sample-admin-portlet-1.0.0-SNAPSHOT.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code> of the project;<br />
* package versions are aligned with the Maven <code>version</code> of the project;<br />
* package include <code>MavenCoordinates</code> that can be directly copied and pasted from the POM of the project;<br />
* packages do '''not''' specify dependencies;<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>sample-portlet</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>PortletAdmin</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/${build.finalName}.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
== Service Archive ==<br />
<br />
<code>sample-portlet</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the project.<br />
<br />
As shown for [[#Libraries|libraries]], the portlet uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources]] for details.<br />
<br />
== Use Case: Create a new Mavenized GWT Portlet ==<br />
<br />
In order to create a new Mavenized GWT Portlet this detailed guide can be also followed [https://gcube.wiki.gcube-system.org/gcube/index.php/Create_a_new_Mavenized_gCube_GWT_Portlet_Guide]<br />
<br />
= Widgets =<br />
<br />
Falls in this category of components mainly GWT widgets or so called GWT Libraries, which can be used to package classes for mutualization and/or modularization. A Widget is a java archive (JAR) containing both classes and java sources for later GWT compilation and a gwt.xml module descriptor. <br />
<br />
In this section we refer to a sample Widget with maven structure (https://svn.research-infrastructures.eu/d4science/gcube/trunk/portal/mvn-widget-sample/)<br />
<br />
== Project Structure ==<br />
<br />
As for a other project, the portlet is associated with 2 distinguished build artefacts:<br />
<br />
* the archive required for deployment (a JAR);<br />
* the archive required for registration in the system (a SA);<br />
<br />
In order to generate automatically the project structure and the base pom, we start by customizing the maven archetype liferay-portlet-archetype v. 6.0.6.<br />
<br />
The generation of the project structure and the pom can be performed either from Eclipse ( by using the m2eclipse plugin) or by command line using the Maven SDK.<br />
<br />
On Eclipse the following steps must be followed:<br />
<br />
* FIle -> New -> Project<br />
* From the wizard select Maven->Maven Project<br />
* From the Select Archetype step, Select 'All Catalogs' from the Catalogs combobox and type 'portlet' on the Filter field.<br />
* Select the '''maven-archetype-quickstart''' version 1.1.0 archetype ( if does not appear on the list just unselect 'show the last version of the Archetype only")<br />
* In the next step you have to provide Maven coordinates for your project<br />
* groupid : <your group id> e.g org.gcube.portlets.admin<br />
* artifactId : <your artifactId> e.g. sample-admin-portlet<br />
* version : <your project version> e.g. 1.0.0-SNAPSHOT ( use always SNAPSHOT versions for development versions of a project)<br />
* package: <your portlet code package> e.g. org.gcube.portlets.admin.sample<br />
<br />
while from shell the following mvn command should be issued :<br />
<br />
<pre><br />
mvn archetype:generate \<br />
-DarchetypeArtifactId=maven-archetype-quickstart \<br />
-DarchetypeGroupId=com.liferay.maven.archetypes \<br />
-DarchetypeVersion=1.1.0 \<br />
-DartifactId=<your artifactId> \<br />
-DgroupId=<your group id> \<br />
-Dversion=1.0.0-SNAPSHOT<br />
</pre><br />
<br />
Both approaches generate a project structure similar to the picture below:<br />
<br />
[[image:sample-widget-project.png|Sample widget project structure]]<br />
<br />
In order to customize the build procedure to the gCube needs we should open the pom.xml and apply the following changes:<br />
<br />
Configure the inheritance from '''maven-parent''' :<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
Configure some project properties:<br />
<br />
<source lang="xml"><br />
<properties><br />
<distroDirectory>${project.basedir}/distro</distroDirectory><br />
<configDirectory>${project.basedir}/config</configDirectory><br />
</properties><br />
</source><br />
<br />
Add the plugin for the generation of the SA, which has to be included under the build plugins section:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<version>2.2</version><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
In the case of a GWT widget the [http://mojo.codehaus.org/gwt-maven-plugin gwt-maven-plugin] should also be configured ( within build plugins section as well), by also selecting the proper gwt version ( in our example 2.4)<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.codehaus.mojo</groupId><br />
<artifactId>gwt-maven-plugin</artifactId><br />
<version>${gwt.version}</version><br />
<executions><br />
<execution><br />
<goals><br />
<goal>compile</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
The proper GWT version should be added to the pom properties :<br />
<br />
<source lang="xml"><br />
<gwt.version>2.4.0</gwt.version><br />
</source><br />
<br />
and GWT dependencies should be also configured:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-user</artifactId><br />
<version>${gwt.version}</version><br />
</dependency><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-servlet</artifactId><br />
<version>${gwt.version}</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/mvn-widget-sample/ project sources]] for further details on the project configuration.<br />
<br />
== Generating Artifacts ==<br />
<br />
The <code>sample-portlet</code> uses the '''maven-compiler-plugin''' to compile the portlet code and the '''gwt-maven-plugin''' as well for GWT compilation.<br />
<br />
The generation of a war package can be triggered by typing :<br />
<br />
<pre><br />
mvn package <br />
</pre><br />
<br />
on your shell or running this build task within Eclipse.<br />
<br />
In both cases the war package will be generated under the ''target'' folder, e.g.:<br />
<br />
<pre><br />
target/sample-admin-portlet-1.0.0-SNAPSHOT.war<br />
</pre><br />
<br />
The generation of all project artifacts ( war, source jar, javadoc, SA) can be triggered by the build task :<br />
<br />
<pre><br />
mvn install <br />
</pre><br />
<br />
this will install as well all artifacts in your local repository, while the deployment of the artifacts to the remote gCube Maven repos can triggered by the build task:<br />
<br />
<pre><br />
mvn deploy <br />
</pre><br />
<br />
== Widget dependencies ==<br />
<br />
The system dependencies ( the one contained within the [[GCube_Portal_Installation#Portal_Bundle_Libraries|Portal bundle]] ) should not be included inside the portlet war. As said in this case the ''provided'' scope should be specified, e.g.<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portal</groupId><br />
<artifactId>custom-portal-handler</artifactId><br />
<version>1.1.0-SNAPSHOT</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.4.0,1.5.0]</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the widget is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>PortletAdmin</Class><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.portles.admin</groupId><br />
<artifactId>sample-admin-portlet</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/sample-admin-portlet-1.0.0-SNAPSHOT.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code> of the project;<br />
* package versions are aligned with the Maven <code>version</code> of the project;<br />
* package include <code>MavenCoordinates</code> that can be directly copied and pasted from the POM of the project;<br />
* packages do '''not''' specify dependencies;<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>sample-portlet</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>PortletAdmin</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/${build.finalName}.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
== Service Archive ==<br />
<br />
<code>sample-portlet</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the project.<br />
<br />
As shown for [[#Libraries|libraries]], the portlet uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources]] for details.</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Integration_of_Maven_Components&diff=29207Integration of Maven Components2017-05-30T10:22:59Z<p>Gabriele.giammatteo: /* gCube Maven Repositories */</p>
<hr />
<div>[[Category: Developer's Guide]]<br />
= Introduction =<br />
This document describes the procedures and tools used for the integration and release of Maven-based gCube components. After an initial overview of the overall solution, the document narrow its focus to the integration of Maven components in ETICS. <br />
<br />
This document is also an operational guide for [[Role Developer|developers]] with all information needed to properly configure a Maven-based gCube component in ETICS to successfully integrate with other components in gCube. It provides also very quick-to-read checklist with the minimal set of things to know and to do in order to let things work in ETICS.<br />
<br />
== Glossary ==<br />
* '''Maven component''': a gCube component which uses Maven ad building tool<br />
* '''Legacy component''': a gCube component (most probably developed before iMarine project) which uses the classical building system based on ant<br />
* '''Mavenizer''': a tool developed within gCube project able to create a pom.xml file<br />
<br />
= gCube Maven Repositories =<br />
'''gCube Releases'''<br />
* Contains artifacts officially released.<br />
* URL: http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-releases/<br />
* access: only release manager<br />
* only release version, no re-deploy<br />
* updated by etics as last step of the release process<br />
<br />
'''gCube Staging'''<br />
* Contains artifacts produced during the release cycle.<br />
* URL: http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-staging/<br />
* access: release manager, developers<br />
* only release versions, redeploy allowed<br />
* updated by:<br />
** etics nightly builds and on-demand builds for SmartGears release<br />
<br />
'''gCube Staging gCore'''<br />
* Contains artifacts produced during the release cycle.<br />
* URL: http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-staging-gcore/<br />
* access: release manager, developers<br />
* only release versions, redeploy allowed<br />
* updated by:<br />
** etics nightly builds and on-demand builds for gCore release<br />
<br />
'''gCube Staging All'''<br />
* Contains artifacts produced during the release cycle.<br />
* URL: http://maven.research-infrastructures.eu/nexus/content/groups/gcube-staging-all/<br />
* access: release manager, developers<br />
* only release versions, redeploy allowed<br />
* virtual repository that merges gCube Staging and gCube Staging gCore<br />
<br />
'''gCube Snapshots'''<br />
* Contains snapshot versions of of gCube components. It is used mainly during the development phase to make public and available for integration not yet released software<br />
* URL: http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots/<br />
* access: release manager, developers<br />
* only snapshot versions, redeploy allowed<br />
* updated by:<br />
** developers from their local environment<br />
** etics on-demand builds for HEAD configurations built within gcube.HEAD<br />
** etics daily builds of gcube.HEAD<br />
<br />
'''gCube Externals'''<br />
* Contains patched versions of third party components. Those components cannot be considered part of gCube system<br />
* URL: http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals/<br />
* access: release manager<br />
<br />
= Overall strategy =<br />
Figure below catches the main components, and their interactions, that are involved in the gCube' software integration, release and distribution. <br />
<br />
[[Image:Maven.png|center|Overall Strategy]]<br />
<br />
The central element of the gCube release and distribution strategy is the '''gCube Releases Repository'''. It is a classical Maven repository where all released versions of all gCube components are stored.The gCube Releases Repository is the main distribution site for gCube Software:<br />
* Developers configure their development machine to use gCube Releases to resolve and fetch dependencies (a) for gCube components their are developing.<br />
* the Software Gateway service uses gCube Releases as back-end (among, potentially, other third party Maven repositories (c) ) making software stored there available for dynamic deployment within the D4Science Infrastructure.<br />
<br />
<br />
Another central role is played by ETICS: the integration system used by gCube. It is responsible for the publication (d) in gCube Releases of artifacts that comes out from the integration of gCube components. In particular, actions performed by ETICS are:<br />
* checkout (e), compilation and testing of gCube Maven components from the gCube' source code repository<br />
* validation and certification of artifacts. This ensure that components published in gCube Releases satisfy the quality level defined in the gCube project<br />
* ''mavenization'' of legacy components. This make also gCube components that builds with ant available for dependency resolution in Maven<br />
* publication of artifacts in gCube Releases<br />
<br />
<br />
An identical approach is used also for integration of development version of components. Main differences in this case are:<br />
* gCube Snapshots is used instead of gCube Releases<br />
* also Developers can commit a snapshot version of their component directly from their development machine<br />
* development versions are never deployed in the D4Science production infrastructure (but they are used in the testing infrastructure)<br />
<br />
= Bringing Maven to ETICS: the maven-builder =<br />
ETICS builds take place on a remote machine on a "empty" environment. In order to properly build Maven components it is needed Maven to be installed and configured on the building machine. All those tasks are performed by an ETICS component created for this purpose called '''maven-builder'''.<br />
<br />
Briefly, this is what maven-builder provides:<br />
* downloads and install Maven tool on the building machine<br />
* downloads and install the jdk and Ant tool on the building machine<br />
* provides the customized maven' settings file ''gcube-default-settings.xml'' which contains credentials to access gCube Snapshot repository<br />
* overrides the default Maven's local repository location (which is not writable by ETICS)<br />
* defines env variables for gCube repositories<br />
* replaces original ''mvn'' command with the Maven Proxy in the PATH (see section [[#Maven Proxy|Maven Proxy]])<br />
* provides mavenizer tool and put it in the PATH (see section [[#Mavenizer|Mavenizer]])<br />
<br />
<br />
In detail, maven-builder exports following environment variables:<br />
<pre><br />
MAVEN_GCUBE_RELEASES: http://maven.research-infrastructures.eu:8081/nexus/content/repositories/gcube-releases/<br />
MAVEN_GCUBE_SNAPSHOTS: http://maven.research-infrastructures.eu:8081/nexus/content/repositories/gcube-snapshots/<br />
MAVEN_GCUBE_STAGING: http://maven.research-infrastructures.eu:8081/nexus/content/repositories/gcube-staging/<br />
MAVEN_LOCAL_REPOSITORY: ${moduleDir}/localrepository<br />
MAVEN_SETTINGS: ${moduleDir}/etc/gcube-default-settings.xml<br />
PATH: ${moduleDir}/bin:$PATH<br />
</pre><br />
<br />
<br />
'''maven-builder is a pre-requisite to build any gCube component in ETICS. All maven-based components must have a direct or indirect dependency on maven-builder'''.<br />
<br />
To simplify dependency specification in your component, for all basic components on which almost all other gCube components depends, a dependency on maven-builder has been set. This allows, for most of components, to avoid a direct dependency on maven-builder.<br />
[[Image:maven-builder-deps.png|200px|center|Dependency graph of maven-builder]]<br />
If your components does not functionally depends on any of ''gcf'', ''legacy-ws-core'', 'maven-parent-*'', you will have to explicitly set a dependency on maven-builder.<br />
<br />
<br />
== Maven Proxy ==<br />
When running Maven in ETICS' integration builds, a set of properties and plugins have to be properly configured in order to let Maven behave as expected. To achieve this result transparently for Developers, it has been decided to have a ''proxy'' script for Maven. It is called mvn (like the original Maven command) and the maven-builder replace the original mvn with the proxy script in the PATH. The final result is that when mvn is called from inside a build command of an ETICS configuration, the mvn proxy is invoked instead.<br />
<br />
<br />
Actions performed by mvn proxy are, in order:<br />
# sets settings.xml file and path for maven local repository<br />
# checks that pom and ETICS versions are coherent<br />
# runs '''versions plugin''' replacing dependencies' versions in pom.xml files to force Maven to use dependencies stored in the local repository instead of download version required from remote repositories.<br />
# set -Dmaven.deploy.skip property that avoid publication of artifacts on gCube Releases if not the last build<br />
# call original mvn<br />
<br />
<br />
Step 3) is required because we want to ensure a real integration between components: if a component '''A''' depends on component '''B''', we want that '''B''' is the one compiled in the context of the same build and not another one compiled previously and uploaded on gCube repository. Please note that this does not apply for externals: Maven Central repository is enabled.<br />
<br />
== Mavenizer ==<br />
<br />
README: As of September 2015 (last update of this wiki), all gCube components are using Maven. The Mavenizer is still supported in ETICS but not longer used.<br />
<br />
''Mavenizer'' is a tool created with the intent of making gCube legacy components compatible with new maven components. It is able to generate maven artifacts starting from outcomes of compilation. It is meant to be used by ETICS during its builds to ''mavenize'' legacy components with a twofold benefit:<br />
* being able to have those components in gCube Maven Repositories<br />
* make legacy components available for Maven components in the local Maven repository for compile-time dependency resolution purposes<br />
<br />
It provides two commands that must be added in build commands of each legacy configuration: [[#mavenize|mavenize]] and [[#mavenize_sa|mavenize_sa]].<br />
<br />
=== mavenize ===<br />
This command take as input an artifact that can be .jar, .gar, .war, writes a pom.xml on-the-fly for that artifact and deploys it in the gCube's Maven Repository.<br />
<br />
''mavenize'' command determines (if not provided from command line - see below) Maven coordinates groupId:artifactId:version for the artifact it is going to create from the '''profile.xml''' file of the component. The algorithm is:<br />
* <code>groupId</code> is equal to org.gcube.<ServiceClass>, where <ServiceClass> is the string found in the profile with "-" and "." replaced by "" and all in lowercase<br />
* <code>artifactId</code> is equal to the <PackageName> value found in the profile.xml for the <Software> section corresponding to the artifact that is going to be mavenized;<br />
* <code>version</code> is equal to the <version> value found in the profile.xml for the <Software> section corresponding to the artifact that is going to be mavenized;<br />
<br />
The snippet below shows the input parameters accepted by ''mavenize'' script.<br />
<pre><br />
gabriele@laptop-trasferte:~/tmp$ mavenize --help<br />
USAGE: /home/gabriele/projects/ENG.iMarine/workspace/maven-builder/bin/mavenize [flags]<br />
flags:<br />
-f,--file: artifact file to mavenize. If not specified, a search will be done in the etics module home trying to find a suitable artifact (default: '')<br />
-p,--profile: profile.xml file from where take information about the artifact (default: '')<br />
-g,--groupid: profile.xml file from where take information about the artifact (default: '')<br />
-a,--artifactid: profile.xml file from where take information about the artifact (default: '')<br />
-v,--version: profile.xml file from where take information about the artifact (default: '')<br />
-n,--[no]dryrun: no act. Perform just a simulation (default: false)<br />
-l,--loglevel: log level. Accepted values are 0, 1, 2 (default: 1)<br />
-h,--[no]help: show this help (default: false)<br />
</pre><br />
<br />
'''All parameters are optional'''. If a parameter is not provided, ''mavenize'' assumes that files are in the standard location for gCube components: <br />
* if <code>--file</code> is not provided, by default a jar/gar/war will be searched in the <code>${prefix}</code> directory. If multiple artifacts are found in <code>${prefix}</code> the command fails<br />
* if <code>--profile</code> is not specified, by default the file in ${moduleDir}/etc/profile.xml will be used. If it is not present, the command fails<br />
* if <code>--groupid</code>, <code>--artifactid</code> and/or <code>--version</code> are specified, their values will override values found in the profile.xml<br />
<br />
'''To trigger execution of mavenize on a given legacy component, it is enough to have an invocation to ''mavenize'' in the component's ETICS build configuration in target ''prepublish'' or ''postpublish''''' (as shown in figure below).<br />
<br />
[[Image:Mavenizer build configuration.png|500px|center|Mavenizer, example of build configuration]]<br />
<br />
'''Note''': call to ''mavenize'' should be already present in all legacy components since it has been added automatically by ETICS administrators.<br />
<br />
=== mavenize_sa ===<br />
This command is specific to execute specific actions on legacy '''servicearchive''' components. In particular it:<br />
* updates the profile.xml contained in the servicearchive adding, for each package, the maven coordinates of that package following the same convention described in [[#mavenize|mavenize]]<br />
* for each <code>doc</code> component found in the servicearchive, creates a jar artifacts and deploy it on the gCube Maven repositories with the same coordinates of the package to which it refers with the classifier <code>javadoc</code><br />
* transform the servicearchive package into a maven artifact that has the same coordinates of the first package found in the profile.xml and deploy it on gCube Maven repositories with the classifier <code>servicearchive</code><br />
<br />
The ''mavenize_sa'' accept as input parameter the servicearchive artifact to process. If not provided, it will search for a .tar.gz file into the standard location where ETICS creates packages. <br />
<pre><br />
gabriele@laptop-trasferte:~/tmp$ mavenize_sa --help<br />
USAGE: /home/gabriele/projects/ENG.iMarine/workspace/maven-builder/bin/mavenize_sa [flags]<br />
flags:<br />
-s,--sa: servicearchive file containing the profile to update (default: '')<br />
-n,--[no]dryrun: no act. Perform a simulation (default: false)<br />
-l,--loglevel: log level. Accepted values are 0, 1, 2 (default: 1)<br />
-h,--[no]help: show this help (default: false)<br />
</pre><br />
<br />
'''To trigger execution of mavenize_sa on a given legacy servicearchive, it is enough to have an invocation to ''mavenize_sa'' in the component's ETICS build configuration in target ''prepublish''<br />
<br />
'''Note''': a call to ''mavenize_sa'' should be already present in all legacy servicearchive since it has been added automatically by ETICS administrators.<br />
<br />
= Further Reading =<br />
* the presentation '''Bringing Maven to gCube''' taken at 1st TCOM event: http://bscw.research-infrastructures.eu/bscw/bscw.cgi/d244165/WP7_Bringing_Maven_to_gCube.pdf<br />
* the document '''Bringing Maven to gCube''' that explains in detail the usage of Maven in gCube, not only in ETICS but also in the development phase: http://bscw.research-infrastructures.eu/bscw/bscw.cgi/d244327/Project%20Apash.pdf</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Developing_gCube_Maven_Components&diff=29034Developing gCube Maven Components2017-05-04T10:58:56Z<p>Gabriele.giammatteo: /* Java Version */</p>
<hr />
<div>[[Category: Developer's Guide]]<br />
__TOC__<br />
<br />
= Common =<br />
== Maven Coordinates ==<br />
Maven components' coordinates must follow these conventions:<br />
<br />
* their <code>groupId</code> must be of the following form:<br />
<br />
<pre>org.gcube.<class></pre><br />
<br />
: where ''class'' identifies a functionally correlated set of components and helps a range of users (developers, administrator) to discover components in various UIs/reports used within the system and in tools. Logically, the class corresponds to the gCube class of the component's, as specified in its gCube Software Profile. However, more specific, or else cross-class groups may be agreed upon among developers. In all cases, the group should align with official Maven [http://maven.apache.org/guides/mini/guide-naming-conventions.html guidelines]. By agreement, for example, components in gCube class <code>DataAccess</code> declare a <code>groupId</code> of <code>org.gcube.data.access</code> and those in gCube class <code>DataTransfer</code> declare a <code>groupId</code> of <code>org.gcube.data.transfer</code>. On the other hand, also by agreement, components in gCube class <code>InformationSystem</code> declare a <code>groupId</code> of <code>org.gcube.informationsystem</code>.<br />
<br />
* their <code>artifactId</code> must follow Maven conventions (e.g. ''avoid capital letters or punctuation other than hyphens'').<br />
<br />
== Parent ==<br />
All gCube components must inherit from '''maven-parent'''. This ensure and promotes compliance with project-wide requirements, from the enforcement of minimal Java and Maven versions to generation and packaging of Javadoc documentation and component sources.<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
<br />
== The distro folder ==<br />
This folder contains files that will be distributed along with the code in the distribution packages. Further details are in the [[Maven distro directory layout]] page.<br />
<br />
== Service Archive (SA) ==<br />
Whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary <code>tar.gz</code> artefact with classifier <code>servicearchive</code>. Note that the SA of Maven components does ''not'' need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);<br />
<br />
* for each deployable artefact they produce (primary or secondary), they must include the Maven coordinates of the arteact in the corresponding package section of the gCube Software Profile. Specifying package dependencies in the Software Profile, on the other hand, is no longer required;<br />
<br />
* whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary <code>tar.gz</code> artefact with classifier <code>servicearchive</code>. Note that the SA of Maven components does ''not'' need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);<br />
<br />
== Dependencies ==<br />
Service and libraries that depend on other gCube components (Maven-based or Ant-based) can find them in the gCube Maven Repositories: <code>gcube-snapshots</code> <strike>and <code>gcube-releases</code></strike>. <br />
<br />
=== gCube Maven Repositories ===<br />
As a one-off task, developers must configure these repositories in the <code>~/.m2/settings.xml</code> file of their local Maven installation, as follows:<br />
<br />
<source lang="xml"><br />
<settings xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"<br />
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd"><br />
<br />
<servers><br />
<server><br />
<id>gcube-snapshots</id><br />
<username>gcube-user</username><br />
<password>maven</password><br />
</server><br />
</servers><br />
<br />
<br />
<profiles><br />
<profile><br />
<id>gcube</id><br />
<repositories><br />
<repository><br />
<id>gcube-snapshots</id><br />
<name>gCube Snapshots</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
</snapshots><br />
</repository><br />
<repository><br />
<id>gcube-externals</id><br />
<name>gCube Externals</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals</url><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
</repository><br />
</repositories><br />
<br />
<pluginRepositories><br />
<pluginRepository><br />
<id>gcube-snapshots</id><br />
<name>gCube Snapshots</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
</snapshots><br />
</pluginRepository><br />
<pluginRepository><br />
<id>gcube-externals</id><br />
<name>gCube Externals</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals</url><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
</pluginRepository><br />
</pluginRepositories><br />
<br />
</profile><br />
</profiles><br />
<br />
<activeProfiles><br />
<activeProfile>gcube</activeProfile><br />
</activeProfiles><br />
</settings><br />
</source><br />
<br />
<br />
'''Note:Developers must not configure these repositories in their POMs''', only in <code>settings.xml</code>. <br />
<br />
This is to:<br />
* keep their projects independent from future changes in repository locations; <br />
* avoid interference with ETICS builds, where access to these repositories may comprise correct system integration;<br />
<br />
With this settings.xml, developers will have access on their local development environment to development versions (gcube-snapshots) and externals (gcube-externals). This is the correct behaviour for most of use cases. If access to staging or releases repositories is necessary (e.g. patching a component already released in the branch where dependencies versions in the release are not in snapshots), developer should modify his settings.xml to point to gcube-staging(releases) instead of gcube-snapshots.<br />
<br />
=== Third-party Repositories ===<br />
Using the the <code>settings.xml</code> aforementioned, any dependency declared in pom.xml that can be found in one of gCube repositories or in Maven Central will be resolved at build time and deployment time without any further configuration. Frequently it may happens that a gCube component needs software stored on a public Maven repository not in on of those repositories. In this case, Maven provides an easy solution for declaring additional repositories for dependency resolution by adding a <repositories> section in the pom.xml of the project. E.g.:<br />
<br />
<source lang="xml" highlight="3,4"><br />
<repositories><br />
<repository><br />
<id>[repository-id]</id><br />
<url>[repository URL]</url><br />
</repository><br />
</repositories><br />
</source><br />
<br />
Though technically possible to add any Maven repository in this way, in order to preserve the quality of gCube software it is required to check trustworthiness of its dependencies disallowing usage of certain external software. To enforce such a control over public Maven repositories, '''gCube components are allowed to use external repository not directly but only through a proxy repository''' (a.k.a. shadow repository) created on CNR's Nexus Repository Portal (http://maven.research-infrastructures.eu/nexus/).<br />
<br />
List of all repositories in Nexus Portal is available at http://maven.research-infrastructures.eu/nexus/index.html#view-repositories<br />
<br />
'''Note''': if a component uses a public Maven repository directly, it will work for building but would fail in deployment phase. In fact, Software Gateway is only able to download software accessible from Nexus.<br />
<br />
==== Request new proxy repository in Nexus ====<br />
Proxy Repositories on Nexus are created by the Release Manager. To request a new proxy a new ticket must be created.<br />
<br />
Details of ticket are:<br />
# open a ticket in tracker "Support" in gCube ticketing system (at https://support.d4science.org/projects/gcube/)<br />
# ticket's '''description''' must report:<br />
#* URL of repository to add<br />
#* organization which hosts the repository<br />
#* which artifacts are needed from that repository<br />
#* which components needs those artifacts<br />
# '''Sprint''' field should be set to the gCube release in which the component is expected to be released <br />
# '''Category''' field must report the gCube subsystem that needs the external repository<br />
# '''Assignee''' is the Release Manager<br />
<br />
An example of ticket for requesting a new repository mirror is reported in the figure below.<br />
[[Image:New_maven_mirror_repo.png|600px|center|Ticket for requesting a new Mirror Repository]]<br />
<br />
Release Manager will process the ticket checking that no issues are foreseen from the usage of software stored in the repository. If request is accepted, a new proxy repository is created and the repository's URL will be communicated publicly on gCube's developers mailinglist (a.k.a. TCOM mailinglist).<br />
<br />
Knowing the mirror repository URL, developer can use it adding following lines in pom.xml. E.g.<br />
<br />
<source lang="xml" highlight="3,4"><br />
<repositories><br />
<repository><br />
<id>gcube-ext-osgeo</id><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/osgeo/</url><br />
</repository><br />
</repositories><br />
</source><br />
<br />
'''Note''': it is possible that software stored on proxy repositories is not included in search results on Nexus. This happens for those repositories that does not have indexes (or have indexes in a format that Nexus is not able to download). Anyway, once Nexus has downloaded artifacts from the original repository for the first time (e.g. because a compilation of a pom that uses the proxy repository), creates its own indexes and those artifacts will become available also for local search.<br />
<br />
=== Dependency Versions ===<br />
<br />
ETICS builds ensure correct integration of gCube components, regardless of the versions their dependencies. It also ensures that components are released with fixed versions for their dependencies, i.e. ensures build reproducibility.<br />
<br />
Accordingly, developers can be liberal in the choice of dependency versions. i.e. rely on version ranges to:<br />
<br />
* minimise the requirements for version management;<br />
* be readily informed of any mis-alignment with their dependencies;<br />
<br />
As we start adopting Maven, the lower-bound of version ranges will be a SNAPSHOT version. For example, a dependency on the <code>streams</code> library can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>streams</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
</dependency><br />
</source><br />
<br />
With this dependency, a component will be soon notified of a non-retro compatible change that, contrary to conventions, <code>streams</code> may have introduced in, say, version <code>1.3.0</code> (typically the day after at the latest). The notification will be delivered in the IDE, assuming the availability of Maven integration (e.g. <code>m2e</code> in Eclipse). <br />
The component is in fact free to open the range further if it wants to be aligned with its dependencies across major versions.<br />
<br />
=== Common Dependencies ===<br />
<br />
Many gCube components depend on <code>gcf</code>, which is now available as a Maven component since version <code>1.4</code>. At the time of writing, the component has not been released yet. A dependency on it is then specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
The scope <code>provided</code> indicates that these dependency will be satisfied in the development environment (the gHN) and should not be included in GAR and SA archives.<br />
<br />
Some components depend on the libraries available in all gHNs, though they do not depend on <code>gcf</code> itself. The subset of these libraries that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called <code>ghn-core-runtime</code>. A dependency on this Jar can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ghn-core-runtime</artifactId><br />
<version>1.0.0</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Note: a component that depends on <code>gcf</code> indirectly depends on <code>gun-core-runtime</code>.<br />
<br />
Another common dependency is on the gCube gRS2, an Ant component that the build infrastructure make available in Maven repositories. A dependency on this component can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.execution</groupId><br />
<artifactId>grs2library</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
</dependency><br />
</source><br />
<br />
== Java Version ==<br />
<br />
In Maven the Java version of the source code and of the generated classes can be controlled using the configuration of the <code>maven-compiler-plugin</code>. E.g.:<br />
<syntaxhighlight lang="xml" highlight="5,6"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-compiler-plugin</artifactId><br />
<configuration><br />
<source>1.7</source><br />
<target>1.8</target><br />
</configuration><br />
</plugin><br />
</syntaxhighlight><br />
<br />
<br />
In gCube, this mechanism is exploited to enforce the usage of the same Java version for all components. In particular, the maven-parent defines the properties <code>java_version</code>, <code>maven.compiler.source</code> and <code>maven.compiler.target</code> and use these properties in the configuration of the <code>maven-compiler-plugin</code>.<br />
<syntaxhighlight lang="xml" highlight="15,16"><br />
...<br />
<properties><br />
<java_version>1.8</java_version><br />
<maven.compiler.source>${java_version}</maven.compiler.source><br />
<maven.compiler.target>${java_version}</maven.compiler.target><br />
...<br />
</properties><br />
<br />
...<br />
<br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-compiler-plugin</artifactId><br />
<configuration><br />
<target>${maven.compiler.target}</target><br />
<source>${maven.compiler.source}</source><br />
</configuration><br />
</plugin><br />
...<br />
</syntaxhighlight><br />
<br />
'''All the gCube components inherit this default configuration from <code>maven-parent</code> and it is not recommended to override it in the component's pom.xml'''.<br />
<br />
If it is really needed to override these values at component's pom.xml level, instead of specify a new configuration for the maven-compiler-plugin, the value of the property should be re-defined:<br />
<br />
<syntaxhighlight lang="xml" highlight="3"><br />
...<br />
<properties><br />
<maven.compiler.source>1.7</maven.compiler.source><br />
...<br />
</properties><br />
<br />
</syntaxhighlight><br />
<br />
<br />
This configuration is flexible enough to allow to customize the Java compiler source and target values depending on different factors in the integration builds (e.g. during ETICS builds gCore components are built with Java 7, all the others with Java 8).<br />
<br />
= Libraries =<br />
The <code>streams</code> library is a Maven component in gCube class <code>DataAccess</code>. We will use it as example of a library component in Maven.<br />
<br />
=== Profile ===<br />
<br />
The gCube Software profile of the library is as follows:<br />
<br />
<source lang="xml" highlight="14-18"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>Embedded Domain-Specific Language for Stream Transformations</Description><br />
<Class>DataAccess</Class><br />
<Name>streams</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>streams</Name><br />
<Version>1.0.0-SNAPSHOT</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>streams</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<Files><br />
<File>streams-1.0.0-SNAPSHOT.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the profile includes a single package and this package corresponds to the main build artefact of the component; <br />
* the whole profile and the package have usual gCube coordinates;<br />
* the package name is aligned with the Maven <code>artifactId</code> of the component;<br />
* the package version is aligned with the Maven <code>version</code> of the component;<br />
* the package includes <code>MavenCoordinates</code> that can be directly copied and pasted from the POM;<br />
* the package does '''not''' specify dependencies;<br />
* the package points to the main artefact of the Maven build;<br />
<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>streams</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>DataAccess</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<Files><br />
<File>${build.finalName}.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
=== Service Archive ===<br />
<br />
<code>streams</code> is responsible for generating its own SA as part of its Maven build. This requires dedicated logic in the POM but has the advantage that:<br />
<br />
* SA validity can be verified locally;<br />
* no SA configurations are required in ETICS;<br />
* the required build logic can be easily reused across components;<br />
<br />
<code>streams</code> uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation in the profile and other static files. The approach is best illustrated with a reference to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/streams sources]] of the <code>streams</code> library.<br />
<br />
<br />
As we discuss [[#Creating the ETICS Configurations|below]], <code>streams</code> has the final requirement to produce an interpolated profile during ETICS builds, outside the context of the SA.<br />
<code>streams</code> meets this requirement with the [http://maven.apache.org/plugins/maven-resources-plugin/ Maven Resources Plugin]. Again, consult the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/streams sources]] of the <code>streams</code> library to reuse this approach.<br />
<br />
= Services =<br />
The <code>tree-manager</code> service is a Maven component in gCube class <code>DataAccess</code>. <br />
<br />
== Project Structure ==<br />
<br />
Differently from a library, the service is associated with a number of distinguished build artefacts:<br />
<br />
* the service implementation (a Jar);<br />
* the stubs automatically generated from the WSDL interfaces (a Jar);<br />
* the archive required for deployment in gCore (a Gar);<br />
* the archive required for registration in the system (a SA);<br />
<br />
Producing these artefacts requires sharing a number of static and dynamic files (from profiles to WSDLs to pre-processed WSDLs). The usual Maven approach of mapping the artefacts onto different Maven projects is then genuinely difficult. It is instead more practical to use a single multi-module Maven project. The <code>tree-manager</code> uses this module structure:<br />
<br />
<pre><br />
tree-manager (parent module)------- (produces) --- (POM only)<br />
|<br />
|----- tree-manager-service (service impl module)------- (produces) --- jar, gar, SA<br />
| |<br />
| | (depends on)<br />
| |<br />
| \/<br />
|----- tree-manager-stubs (service stubs module)------- (produces) --- jar<br />
</pre><br />
<br />
As customary, the parent does not produce any artefact (is a POM-only project). It declares these coordinates in its POM:<br />
<br />
<source lang="xml"><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
<packaging>pom</packaging><br />
</source><br />
<br />
The parent physically includes the sub-modules;<br />
<br />
<source lang="xml"><br />
<modules><br />
<module>tree-manager-stubs</module><br />
<module>tree-manager-service</module><br />
</modules><br />
</source> <br />
<br />
Finally, the parent inherits from '''maven-parent''':<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
The submodules inherit from the parent:<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>tree-manager</artifactId><br />
<groupId>org.gcube.data.access</groupId><br />
<version>1.0.0-SNAPSHOT</version><br />
<relativePath>..</relativePath><br />
</parent><br />
</source><br />
<br />
and share its <code>version</code> and <code>groupId</code>. Their coordinates are then minimal:<br />
<br />
<source lang="xml"><br />
<artifactId>tree-manager-service</artifactId><br />
</source><br />
<br />
<source lang="xml"><br />
<artifactId>tree-manager-stubs</artifactId><br />
</source><br />
<br />
The <code>tree-manager-service</code> module produces a Jar of its own implementation, as well the GAR and the SA of the whole services as secondary artefacts with classifiers of, respectively, <code>gar</code> and <code>service archive</code>.<br />
<br />
The <code>tree-manager-stubs</code> module produces a Jar of its own implementation.<br />
<br />
Finally, the parent module coordinates the build of its sub-modules and centralises the declaration of common dependencies and configuration. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] to reuse this approach.<br />
<br />
== Generating Stubs and Gars ==<br />
<br />
The <code>tree-manager-stubs</code> uses the '''gCube Service Plugin''' to generate its classes from WSDL interfaces. <br />
<br />
Similarly, the <code>tree-manager-service</code> uses the plugin to generate its secondary GAR artefact. <br />
<br />
The configuration of the plugin is conveniently shared in the parent module.<br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] of the <code>streams</code> for details.<br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the service is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>DataAccess</Class><br />
<Name>tree-manager-service</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Main><br />
<Name>tree-manager-service</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager-service</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<GARArchive>tree-manager-service-1.0.0-SNAPSHOT.gar</GARArchive><br />
<PortType><br />
<Name>gcube/data/tm/binder</Name><br />
</PortType><br />
<PortType><br />
<Name>gcube/data/tm/reader</Name><br />
</PortType><br />
<PortType><br />
<Name>gcube/data/tm/writer</Name><br />
</PortType><br />
</Main><br />
<Software><br />
<Name>tree-manager-stubs</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager-stubs</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<Type>library</Type><br />
<Files><br />
<File>tree-manager-stubs-1.0.0-SNAPSHOT.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the profile includes two packages and this package corresponds to the main build artefacts of the submodules; <br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code>s of the submodules;<br />
* package versions are aligned with the Maven <code>version</code>s of the submodules;<br />
* package include <code>MavenCoordinates</code>s that can be directly copied and pasted from the POM of the submodules;<br />
* packages do '''not''' specify dependencies;<br />
* packages points to the main artefact of the Maven build of the submodules;<br />
<br />
<strike>Notice that, unlike libraries, services cannot easily use Maven variables in their profiles (as these artefacts are used for different purposes by different modules). Accordingly, developers needs to maintain versions used in profiles across releases. In particular, they need to:<br />
<br />
* replace SNAPSHOT versions with release versions in release branches;<br />
* advance SNAPSHOT versions in HEAD;</strike><br />
<br />
=== Profile Interpolation ===<br />
The interpolation of profiles is required in order to keep information in profile in sync with <code>pom.xml</code>, in particular versions and filenames (that contain versions) that change at each release.<br />
<br />
In order to generate <code>profile.xml</code> during build process you need to:<br />
# add in the service's parent <code>pom.xml</code> the invocation to maven-resources-plugin<br />
# place a ''template'' <code>profile.xml</code> in the <code>${distroDirectory}</code><br />
<br />
Invocation to <code>maven-resources-plugin</code> looks like the following (to be placed in the <plugins> section inside the <build> section of the pom):<br />
<source lang="xml"><br />
<!-- interpolates profiles and copies from distribution location to configuration <br />
location, where it is need for embedding into stub artifact, gar generation, <br />
and service archive. --><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-resources-plugin</artifactId><br />
<version>2.6</version><br />
<executions><br />
<execution><br />
<id>copy-profile</id><br />
<goals><br />
<goal>copy-resources</goal><br />
</goals><br />
<phase>process-resources</phase><br />
<configuration><br />
<outputDirectory>${configDirectory}</outputDirectory><br />
<resources><br />
<resource><br />
<directory>${distroDirectory}</directory><br />
<includes><br />
<include>profile.xml</include><br />
</includes><br />
<filtering>true</filtering><br />
</resource><br />
</resources><br />
</configuration><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
The ''template'' <code>profile.xml</code> looks like the following:<br />
<source lang="xml" highlight="12,16,18,22,26,30"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>TestClass</Class><br />
<Name>TestService</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Main><br />
<Name>TestService</Name><br />
<Version>${project.version}</Version><br />
<MavenCoordinates><br />
<groupId>org.example</groupId><br />
<artifactId>test-service</artifactId><br />
<version>${project.version}</version><br />
</MavenCoordinates><br />
<GARArchive>>test-service-${project.version}.gar</GARArchive><br />
</Main><br />
<Software><br />
<Name>test-stubs</Name><br />
<Version>${project.version}</Version><br />
<MavenCoordinates><br />
<groupId>org.example</groupId><br />
<artifactId>test-stubs</artifactId><br />
<version>${project.version}</version><br />
</MavenCoordinates><br />
<Type>library</Type><br />
<Files><br />
<File>test-stubs-${project.version}.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
In the example, property <code>${project.version}</code> will be substituted with the actual service's version at build-time. It is also possible, but not required, to parametrize also other values of <code>profile.xml</code> such as maven's coordinates and package names. You can see a more complex <code>profile.xml</code> ''template'' in the <code>tree-manager</code> project: http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-access/tree-manager/distro/profile.xml<br />
<br />
Maven will copy the final <code>profile.xml</code> in <code>${configDirectory}</code> during the <code>process-resources</code> phase of the build. From there it will be copied in the service-archive package.<br />
<br />
== Service Archive ==<br />
<br />
<code>tree-manager-service</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the module and its parent.<br />
<br />
As shown for [[#Libraries|libraries]], the service uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] for details.<br />
<br />
= Portlets =<br />
<br />
The <code>sample-portlet</code> portlet is a Maven based sample portlet based on GWT. Refer to the [http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources] for details.<br />
<br />
== Project Structure ==<br />
<br />
As for a library project, the portlet is associated with 2 distinguished build artefacts:<br />
<br />
* the archive required for deployment in a Liferay Container (a WAR);<br />
* the archive required for registration in the system (a SA);<br />
<br />
In order to generate automatically the project structure and the base pom, we start by customizing the maven archetype liferay-portlet-archetype v. 6.0.6.<br />
<br />
The generation of the project structure and the pom can be performed either from Eclipse ( by using the m2eclipse plugin) or by command line using the Maven SDK.<br />
<br />
On Eclipse the following steps must be followed:<br />
<br />
* FIle -> New -> Project<br />
* From the wizard select Maven->Maven Project<br />
* From the Select Archetype step, Select 'All Catalogs' from the Catalogs combobox and type 'portlet' on the Filter field.<br />
* Select the '''liferay-portlet-archetype''' version 6.0.6 archetype ( if does not appear on the list just unselect 'show the last version of the Archetype only")<br />
* In the next step you have to provide Maven coordinates for your project<br />
* groupid : <your group id> e.g org.gcube.portlets.admin<br />
* artifactId : <your artifactId> e.g. sample-admin-portlet<br />
* version : <your project version> e.g. 1.0.0-SNAPSHOT ( use always SNAPSHOT versions for development versions of a project)<br />
* package: <your portlet code package> e.g. org.gcube.portlets.admin.sample<br />
<br />
while from shell the following mvn command should be issued :<br />
<br />
<pre><br />
mvn archetype:generate \<br />
-DarchetypeArtifactId=liferay-theme-archetype \<br />
-DarchetypeGroupId=com.liferay.maven.archetypes \<br />
-DarchetypeVersion=6.0.6 \<br />
-DartifactId=<your artifactId> \<br />
-DgroupId=<your group id> \<br />
-Dversion=1.0.0-SNAPSHOT<br />
</pre><br />
<br />
Both approaches generate a project structure similar to the picture below:<br />
<br />
[[image:sample-portlet-project.png|Sample portlet project structure]]<br />
<br />
In order to customize the build procedure to the gCube needs we should open the pom.xml and apply the following changes:<br />
<br />
Configure the inheritance from '''maven-parent''' :<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
Configure some project properties:<br />
<br />
<source lang="xml"><br />
<properties><br />
<liferay.version>6.0.6</liferay.version><br />
<liferay.auto.deploy.dir>${system.CATALINA_HOME}/../deploy</liferay.auto.deploy.dir><br />
<distroDirectory>${project.basedir}/distro</distroDirectory><br />
<configDirectory>${project.basedir}/config</configDirectory><br />
</properties><br />
</source><br />
<br />
Add the plugin for the generation of the SA, which has to be included under the build plugins section:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<version>2.2</version><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
In the case of a GWT portlet the [http://mojo.codehaus.org/gwt-maven-plugin gwt-maven-plugin] should also be configured ( within build plugins section as well), by also selecting the proper gwt version ( in our example 2.4)<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.codehaus.mojo</groupId><br />
<artifactId>gwt-maven-plugin</artifactId><br />
<version>${gwt.version}</version><br />
<executions><br />
<execution><br />
<goals><br />
<goal>compile</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
The proper GWT version should be added to the pom properties :<br />
<br />
<source lang="xml"><br />
<gwt.version>2.4.0</gwt.version><br />
</source><br />
<br />
and GWT dependencies should be also configured:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-user</artifactId><br />
<version>${gwt.version}</version><br />
</dependency><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-servlet</artifactId><br />
<version>${gwt.version}</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet/ project sources]] for further details on the project configuration.<br />
<br />
== Generating Artifacts ==<br />
<br />
The <code>sample-portlet</code> uses the '''maven-compiler-plugin''' to compile the portlet code and the '''gwt-maven-plugin''' as well for GWT compilation.<br />
<br />
The generation of a war package can be triggered by typing :<br />
<br />
<pre><br />
mvn package <br />
</pre><br />
<br />
on your shell or running this build task within Eclipse.<br />
<br />
In both cases the war package will be generated under the ''target'' folder, e.g.:<br />
<br />
<pre><br />
target/sample-admin-portlet-1.0.0-SNAPSHOT.war<br />
</pre><br />
<br />
The generation of all project artifacts ( war, source jar, javadoc, SA) can be triggered by the build task :<br />
<br />
<pre><br />
mvn install <br />
</pre><br />
<br />
this will install as well all artifacts in your local repository, while the deployment of the artifacts to the remote gCube Maven repos can triggered by the build task:<br />
<br />
<pre><br />
mvn deploy <br />
</pre><br />
<br />
== Portlet dependencies ==<br />
<br />
Specific portlet dependencies ( e.g. GWT, service stubs ) should be packaged within the portlet war. All dependencies specified within the pom file without the ''provided'' scope will be copied inside the WEB-INF/lib folder at packaging time.<br />
<br />
For this reason all portlet specific dependencies should not specified that scope e.g:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.externals</groupId><br />
<artifactId>gwt-ext-patched</artifactId><br />
<version>2.0.5</version><br />
</dependency><br />
</source><br />
<br />
On the other hand the system dependencies ( the one contained within the [[GCube_Portal_Installation#Portal_Bundle_Libraries|Portal bundle]] ) should not be included inside the portlet war. As said in this case the ''provided'' scope should be specified, e.g.<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portal</groupId><br />
<artifactId>custom-portal-handler</artifactId><br />
<version>1.1.0-SNAPSHOT</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.4.0,1.5.0]</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
<br />
A subset of these libraries ( contained in a GHN) that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called <code>ghn-core-runtime</code>. A dependency on this Jar can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ghn-core-runtime</artifactId><br />
<version>1.0.0</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Note: a component that depends on <code>gcf</code> indirectly depends on <code>gun-core-runtime</code>.<br />
<br />
== Include gCube Guided Tour using Maven ==<br />
<br />
This particular component is not yet ported to maven so in order to be included in a maven based project it should be declared as system dependency as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portletsuser</groupId><br />
<artifactId>gcube-guidedtour-widget</artifactId><br />
<version>1.0.0</version><br />
<scope>system</scope><br />
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/gcube-guidedtour-widget.jar</systemPath> <br />
</dependency><br />
</source><br />
<br />
together with the gCube Widget Library:<br />
<br />
<source lang="xml"> <br />
<dependency><br />
<groupId>org.gcube.portlets.user</groupId><br />
<artifactId>gcubewidgets</artifactId><br />
<version>1.2.0</version><br />
<scope>system</scope><br />
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/gCube-Widgets-Library.jar</systemPath><br />
</dependency><br />
</source><br />
<br />
both components can be downloaded from the distribution repository [[http://grids16.eng.it:8080/BuildReport/browse/Recent_Builds]]. <br />
<br />
In case the portlet depends on the ''gcf'' component, some dependency resolution changes are required to your portlet in order to properly compile ( because of libraries clashes)<br />
<br />
The ghn-core-runtime component ( gcf dependency) should be excluded:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.5.1-SNAPSHOT,2.0.0)</version><br />
<scope>provided</scope><br />
<exclusions><br />
<exclusion><br />
<artifactId>ghn-core-runtime</artifactId><br />
<groupId>org.gcube.distribution</groupId><br />
</exclusion><br />
</exclusions><br />
</dependency><br />
</source><br />
<br />
<br />
and some new dependencies should be added:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>xerces</groupId><br />
<artifactId>xercesImpl</artifactId><br />
<version>2.9.1</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ws-addressing</artifactId><br />
<version>1.0</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>axis-patched</artifactId><br />
<version>1.2RC</version><br />
<scope>provided</scope><br />
</dependency><br />
<br />
</source><br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the portlet is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>PortletAdmin</Class><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.portles.admin</groupId><br />
<artifactId>sample-admin-portlet</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/sample-admin-portlet-1.0.0-SNAPSHOT.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code> of the project;<br />
* package versions are aligned with the Maven <code>version</code> of the project;<br />
* package include <code>MavenCoordinates</code> that can be directly copied and pasted from the POM of the project;<br />
* packages do '''not''' specify dependencies;<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>sample-portlet</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>PortletAdmin</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/${build.finalName}.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
== Service Archive ==<br />
<br />
<code>sample-portlet</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the project.<br />
<br />
As shown for [[#Libraries|libraries]], the portlet uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources]] for details.<br />
<br />
== Use Case: Create a new Mavenized GWT Portlet ==<br />
<br />
In order to create a new Mavenized GWT Portlet this detailed guide can be also followed [https://gcube.wiki.gcube-system.org/gcube/index.php/Create_a_new_Mavenized_gCube_GWT_Portlet_Guide]<br />
<br />
= Widgets =<br />
<br />
Falls in this category of components mainly GWT widgets or so called GWT Libraries, which can be used to package classes for mutualization and/or modularization. A Widget is a java archive (JAR) containing both classes and java sources for later GWT compilation and a gwt.xml module descriptor. <br />
<br />
In this section we refer to a sample Widget with maven structure (https://svn.research-infrastructures.eu/d4science/gcube/trunk/portal/mvn-widget-sample/)<br />
<br />
== Project Structure ==<br />
<br />
As for a other project, the portlet is associated with 2 distinguished build artefacts:<br />
<br />
* the archive required for deployment (a JAR);<br />
* the archive required for registration in the system (a SA);<br />
<br />
In order to generate automatically the project structure and the base pom, we start by customizing the maven archetype liferay-portlet-archetype v. 6.0.6.<br />
<br />
The generation of the project structure and the pom can be performed either from Eclipse ( by using the m2eclipse plugin) or by command line using the Maven SDK.<br />
<br />
On Eclipse the following steps must be followed:<br />
<br />
* FIle -> New -> Project<br />
* From the wizard select Maven->Maven Project<br />
* From the Select Archetype step, Select 'All Catalogs' from the Catalogs combobox and type 'portlet' on the Filter field.<br />
* Select the '''maven-archetype-quickstart''' version 1.1.0 archetype ( if does not appear on the list just unselect 'show the last version of the Archetype only")<br />
* In the next step you have to provide Maven coordinates for your project<br />
* groupid : <your group id> e.g org.gcube.portlets.admin<br />
* artifactId : <your artifactId> e.g. sample-admin-portlet<br />
* version : <your project version> e.g. 1.0.0-SNAPSHOT ( use always SNAPSHOT versions for development versions of a project)<br />
* package: <your portlet code package> e.g. org.gcube.portlets.admin.sample<br />
<br />
while from shell the following mvn command should be issued :<br />
<br />
<pre><br />
mvn archetype:generate \<br />
-DarchetypeArtifactId=maven-archetype-quickstart \<br />
-DarchetypeGroupId=com.liferay.maven.archetypes \<br />
-DarchetypeVersion=1.1.0 \<br />
-DartifactId=<your artifactId> \<br />
-DgroupId=<your group id> \<br />
-Dversion=1.0.0-SNAPSHOT<br />
</pre><br />
<br />
Both approaches generate a project structure similar to the picture below:<br />
<br />
[[image:sample-widget-project.png|Sample widget project structure]]<br />
<br />
In order to customize the build procedure to the gCube needs we should open the pom.xml and apply the following changes:<br />
<br />
Configure the inheritance from '''maven-parent''' :<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
Configure some project properties:<br />
<br />
<source lang="xml"><br />
<properties><br />
<distroDirectory>${project.basedir}/distro</distroDirectory><br />
<configDirectory>${project.basedir}/config</configDirectory><br />
</properties><br />
</source><br />
<br />
Add the plugin for the generation of the SA, which has to be included under the build plugins section:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<version>2.2</version><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
In the case of a GWT widget the [http://mojo.codehaus.org/gwt-maven-plugin gwt-maven-plugin] should also be configured ( within build plugins section as well), by also selecting the proper gwt version ( in our example 2.4)<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.codehaus.mojo</groupId><br />
<artifactId>gwt-maven-plugin</artifactId><br />
<version>${gwt.version}</version><br />
<executions><br />
<execution><br />
<goals><br />
<goal>compile</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
The proper GWT version should be added to the pom properties :<br />
<br />
<source lang="xml"><br />
<gwt.version>2.4.0</gwt.version><br />
</source><br />
<br />
and GWT dependencies should be also configured:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-user</artifactId><br />
<version>${gwt.version}</version><br />
</dependency><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-servlet</artifactId><br />
<version>${gwt.version}</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/mvn-widget-sample/ project sources]] for further details on the project configuration.<br />
<br />
== Generating Artifacts ==<br />
<br />
The <code>sample-portlet</code> uses the '''maven-compiler-plugin''' to compile the portlet code and the '''gwt-maven-plugin''' as well for GWT compilation.<br />
<br />
The generation of a war package can be triggered by typing :<br />
<br />
<pre><br />
mvn package <br />
</pre><br />
<br />
on your shell or running this build task within Eclipse.<br />
<br />
In both cases the war package will be generated under the ''target'' folder, e.g.:<br />
<br />
<pre><br />
target/sample-admin-portlet-1.0.0-SNAPSHOT.war<br />
</pre><br />
<br />
The generation of all project artifacts ( war, source jar, javadoc, SA) can be triggered by the build task :<br />
<br />
<pre><br />
mvn install <br />
</pre><br />
<br />
this will install as well all artifacts in your local repository, while the deployment of the artifacts to the remote gCube Maven repos can triggered by the build task:<br />
<br />
<pre><br />
mvn deploy <br />
</pre><br />
<br />
== Widget dependencies ==<br />
<br />
The system dependencies ( the one contained within the [[GCube_Portal_Installation#Portal_Bundle_Libraries|Portal bundle]] ) should not be included inside the portlet war. As said in this case the ''provided'' scope should be specified, e.g.<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portal</groupId><br />
<artifactId>custom-portal-handler</artifactId><br />
<version>1.1.0-SNAPSHOT</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.4.0,1.5.0]</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the widget is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>PortletAdmin</Class><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.portles.admin</groupId><br />
<artifactId>sample-admin-portlet</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/sample-admin-portlet-1.0.0-SNAPSHOT.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code> of the project;<br />
* package versions are aligned with the Maven <code>version</code> of the project;<br />
* package include <code>MavenCoordinates</code> that can be directly copied and pasted from the POM of the project;<br />
* packages do '''not''' specify dependencies;<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>sample-portlet</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>PortletAdmin</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/${build.finalName}.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
== Service Archive ==<br />
<br />
<code>sample-portlet</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the project.<br />
<br />
As shown for [[#Libraries|libraries]], the portlet uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources]] for details.</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Developing_gCube_Maven_Components&diff=29033Developing gCube Maven Components2017-05-04T10:50:50Z<p>Gabriele.giammatteo: /* Java Version */</p>
<hr />
<div>[[Category: Developer's Guide]]<br />
__TOC__<br />
<br />
= Common =<br />
== Maven Coordinates ==<br />
Maven components' coordinates must follow these conventions:<br />
<br />
* their <code>groupId</code> must be of the following form:<br />
<br />
<pre>org.gcube.<class></pre><br />
<br />
: where ''class'' identifies a functionally correlated set of components and helps a range of users (developers, administrator) to discover components in various UIs/reports used within the system and in tools. Logically, the class corresponds to the gCube class of the component's, as specified in its gCube Software Profile. However, more specific, or else cross-class groups may be agreed upon among developers. In all cases, the group should align with official Maven [http://maven.apache.org/guides/mini/guide-naming-conventions.html guidelines]. By agreement, for example, components in gCube class <code>DataAccess</code> declare a <code>groupId</code> of <code>org.gcube.data.access</code> and those in gCube class <code>DataTransfer</code> declare a <code>groupId</code> of <code>org.gcube.data.transfer</code>. On the other hand, also by agreement, components in gCube class <code>InformationSystem</code> declare a <code>groupId</code> of <code>org.gcube.informationsystem</code>.<br />
<br />
* their <code>artifactId</code> must follow Maven conventions (e.g. ''avoid capital letters or punctuation other than hyphens'').<br />
<br />
== Parent ==<br />
All gCube components must inherit from '''maven-parent'''. This ensure and promotes compliance with project-wide requirements, from the enforcement of minimal Java and Maven versions to generation and packaging of Javadoc documentation and component sources.<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
<br />
== The distro folder ==<br />
This folder contains files that will be distributed along with the code in the distribution packages. Further details are in the [[Maven distro directory layout]] page.<br />
<br />
== Service Archive (SA) ==<br />
Whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary <code>tar.gz</code> artefact with classifier <code>servicearchive</code>. Note that the SA of Maven components does ''not'' need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);<br />
<br />
* for each deployable artefact they produce (primary or secondary), they must include the Maven coordinates of the arteact in the corresponding package section of the gCube Software Profile. Specifying package dependencies in the Software Profile, on the other hand, is no longer required;<br />
<br />
* whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary <code>tar.gz</code> artefact with classifier <code>servicearchive</code>. Note that the SA of Maven components does ''not'' need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);<br />
<br />
== Dependencies ==<br />
Service and libraries that depend on other gCube components (Maven-based or Ant-based) can find them in the gCube Maven Repositories: <code>gcube-snapshots</code> <strike>and <code>gcube-releases</code></strike>. <br />
<br />
=== gCube Maven Repositories ===<br />
As a one-off task, developers must configure these repositories in the <code>~/.m2/settings.xml</code> file of their local Maven installation, as follows:<br />
<br />
<source lang="xml"><br />
<settings xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"<br />
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd"><br />
<br />
<servers><br />
<server><br />
<id>gcube-snapshots</id><br />
<username>gcube-user</username><br />
<password>maven</password><br />
</server><br />
</servers><br />
<br />
<br />
<profiles><br />
<profile><br />
<id>gcube</id><br />
<repositories><br />
<repository><br />
<id>gcube-snapshots</id><br />
<name>gCube Snapshots</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
</snapshots><br />
</repository><br />
<repository><br />
<id>gcube-externals</id><br />
<name>gCube Externals</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals</url><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
</repository><br />
</repositories><br />
<br />
<pluginRepositories><br />
<pluginRepository><br />
<id>gcube-snapshots</id><br />
<name>gCube Snapshots</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
</snapshots><br />
</pluginRepository><br />
<pluginRepository><br />
<id>gcube-externals</id><br />
<name>gCube Externals</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals</url><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
</pluginRepository><br />
</pluginRepositories><br />
<br />
</profile><br />
</profiles><br />
<br />
<activeProfiles><br />
<activeProfile>gcube</activeProfile><br />
</activeProfiles><br />
</settings><br />
</source><br />
<br />
<br />
'''Note:Developers must not configure these repositories in their POMs''', only in <code>settings.xml</code>. <br />
<br />
This is to:<br />
* keep their projects independent from future changes in repository locations; <br />
* avoid interference with ETICS builds, where access to these repositories may comprise correct system integration;<br />
<br />
With this settings.xml, developers will have access on their local development environment to development versions (gcube-snapshots) and externals (gcube-externals). This is the correct behaviour for most of use cases. If access to staging or releases repositories is necessary (e.g. patching a component already released in the branch where dependencies versions in the release are not in snapshots), developer should modify his settings.xml to point to gcube-staging(releases) instead of gcube-snapshots.<br />
<br />
=== Third-party Repositories ===<br />
Using the the <code>settings.xml</code> aforementioned, any dependency declared in pom.xml that can be found in one of gCube repositories or in Maven Central will be resolved at build time and deployment time without any further configuration. Frequently it may happens that a gCube component needs software stored on a public Maven repository not in on of those repositories. In this case, Maven provides an easy solution for declaring additional repositories for dependency resolution by adding a <repositories> section in the pom.xml of the project. E.g.:<br />
<br />
<source lang="xml" highlight="3,4"><br />
<repositories><br />
<repository><br />
<id>[repository-id]</id><br />
<url>[repository URL]</url><br />
</repository><br />
</repositories><br />
</source><br />
<br />
Though technically possible to add any Maven repository in this way, in order to preserve the quality of gCube software it is required to check trustworthiness of its dependencies disallowing usage of certain external software. To enforce such a control over public Maven repositories, '''gCube components are allowed to use external repository not directly but only through a proxy repository''' (a.k.a. shadow repository) created on CNR's Nexus Repository Portal (http://maven.research-infrastructures.eu/nexus/).<br />
<br />
List of all repositories in Nexus Portal is available at http://maven.research-infrastructures.eu/nexus/index.html#view-repositories<br />
<br />
'''Note''': if a component uses a public Maven repository directly, it will work for building but would fail in deployment phase. In fact, Software Gateway is only able to download software accessible from Nexus.<br />
<br />
==== Request new proxy repository in Nexus ====<br />
Proxy Repositories on Nexus are created by the Release Manager. To request a new proxy a new ticket must be created.<br />
<br />
Details of ticket are:<br />
# open a ticket in tracker "Support" in gCube ticketing system (at https://support.d4science.org/projects/gcube/)<br />
# ticket's '''description''' must report:<br />
#* URL of repository to add<br />
#* organization which hosts the repository<br />
#* which artifacts are needed from that repository<br />
#* which components needs those artifacts<br />
# '''Sprint''' field should be set to the gCube release in which the component is expected to be released <br />
# '''Category''' field must report the gCube subsystem that needs the external repository<br />
# '''Assignee''' is the Release Manager<br />
<br />
An example of ticket for requesting a new repository mirror is reported in the figure below.<br />
[[Image:New_maven_mirror_repo.png|600px|center|Ticket for requesting a new Mirror Repository]]<br />
<br />
Release Manager will process the ticket checking that no issues are foreseen from the usage of software stored in the repository. If request is accepted, a new proxy repository is created and the repository's URL will be communicated publicly on gCube's developers mailinglist (a.k.a. TCOM mailinglist).<br />
<br />
Knowing the mirror repository URL, developer can use it adding following lines in pom.xml. E.g.<br />
<br />
<source lang="xml" highlight="3,4"><br />
<repositories><br />
<repository><br />
<id>gcube-ext-osgeo</id><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/osgeo/</url><br />
</repository><br />
</repositories><br />
</source><br />
<br />
'''Note''': it is possible that software stored on proxy repositories is not included in search results on Nexus. This happens for those repositories that does not have indexes (or have indexes in a format that Nexus is not able to download). Anyway, once Nexus has downloaded artifacts from the original repository for the first time (e.g. because a compilation of a pom that uses the proxy repository), creates its own indexes and those artifacts will become available also for local search.<br />
<br />
=== Dependency Versions ===<br />
<br />
ETICS builds ensure correct integration of gCube components, regardless of the versions their dependencies. It also ensures that components are released with fixed versions for their dependencies, i.e. ensures build reproducibility.<br />
<br />
Accordingly, developers can be liberal in the choice of dependency versions. i.e. rely on version ranges to:<br />
<br />
* minimise the requirements for version management;<br />
* be readily informed of any mis-alignment with their dependencies;<br />
<br />
As we start adopting Maven, the lower-bound of version ranges will be a SNAPSHOT version. For example, a dependency on the <code>streams</code> library can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>streams</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
</dependency><br />
</source><br />
<br />
With this dependency, a component will be soon notified of a non-retro compatible change that, contrary to conventions, <code>streams</code> may have introduced in, say, version <code>1.3.0</code> (typically the day after at the latest). The notification will be delivered in the IDE, assuming the availability of Maven integration (e.g. <code>m2e</code> in Eclipse). <br />
The component is in fact free to open the range further if it wants to be aligned with its dependencies across major versions.<br />
<br />
=== Common Dependencies ===<br />
<br />
Many gCube components depend on <code>gcf</code>, which is now available as a Maven component since version <code>1.4</code>. At the time of writing, the component has not been released yet. A dependency on it is then specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
The scope <code>provided</code> indicates that these dependency will be satisfied in the development environment (the gHN) and should not be included in GAR and SA archives.<br />
<br />
Some components depend on the libraries available in all gHNs, though they do not depend on <code>gcf</code> itself. The subset of these libraries that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called <code>ghn-core-runtime</code>. A dependency on this Jar can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ghn-core-runtime</artifactId><br />
<version>1.0.0</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Note: a component that depends on <code>gcf</code> indirectly depends on <code>gun-core-runtime</code>.<br />
<br />
Another common dependency is on the gCube gRS2, an Ant component that the build infrastructure make available in Maven repositories. A dependency on this component can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.execution</groupId><br />
<artifactId>grs2library</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
</dependency><br />
</source><br />
<br />
== Java Version ==<br />
<br />
In Maven the Java version of the source code and of the generated classes can be controlled using the configuration of the <code>maven-compiler-plugin</code>. E.g.:<br />
<syntaxhighlight lang="xml" highlight="5,6"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-compiler-plugin</artifactId><br />
<configuration><br />
<source>1.7</source><br />
<target>1.8</target><br />
</configuration><br />
</plugin><br />
</syntaxhighlight><br />
<br />
<br />
In gCube, this mechanism is exploited to enforce the usage of the same Java version for all components. In particular, the maven-parent defines the properties <code>java_version</code>, <code>maven.compiler.source</code> and <code>maven.compiler.target</code> and use these properties in the configuration of the <code>maven-compiler-plugin</code>.<br />
<syntaxhighlight lang="xml" highlight="15,16"><br />
...<br />
<properties><br />
<java_version>1.8</java_version><br />
<maven.compiler.source>${java_version}</maven.compiler.source><br />
<maven.compiler.target>${java_version}</maven.compiler.target><br />
...<br />
</properties><br />
<br />
...<br />
<br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-compiler-plugin</artifactId><br />
<configuration><br />
<target>${maven.compiler.target}</target><br />
<source>${maven.compiler.source}</source><br />
</configuration><br />
</plugin><br />
...<br />
</syntaxhighlight><br />
<br />
'''All the gCube components inherit this default configuration from <code>maven-parent</code> and it is not recommended to override it in the component's pom.xml''' excepting for very few exceptions.<br />
<br />
This configuration allows to customize the Java compiler source and target values depending on different factors in the integration builds (e.g. during ETICS builds gCore components are built with Java 7, all the others with Java 8).<br />
<br />
Current value for the <code>java_version</code> property is 1.8.<br />
<br />
= Libraries =<br />
The <code>streams</code> library is a Maven component in gCube class <code>DataAccess</code>. We will use it as example of a library component in Maven.<br />
<br />
=== Profile ===<br />
<br />
The gCube Software profile of the library is as follows:<br />
<br />
<source lang="xml" highlight="14-18"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>Embedded Domain-Specific Language for Stream Transformations</Description><br />
<Class>DataAccess</Class><br />
<Name>streams</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>streams</Name><br />
<Version>1.0.0-SNAPSHOT</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>streams</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<Files><br />
<File>streams-1.0.0-SNAPSHOT.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the profile includes a single package and this package corresponds to the main build artefact of the component; <br />
* the whole profile and the package have usual gCube coordinates;<br />
* the package name is aligned with the Maven <code>artifactId</code> of the component;<br />
* the package version is aligned with the Maven <code>version</code> of the component;<br />
* the package includes <code>MavenCoordinates</code> that can be directly copied and pasted from the POM;<br />
* the package does '''not''' specify dependencies;<br />
* the package points to the main artefact of the Maven build;<br />
<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>streams</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>DataAccess</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<Files><br />
<File>${build.finalName}.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
=== Service Archive ===<br />
<br />
<code>streams</code> is responsible for generating its own SA as part of its Maven build. This requires dedicated logic in the POM but has the advantage that:<br />
<br />
* SA validity can be verified locally;<br />
* no SA configurations are required in ETICS;<br />
* the required build logic can be easily reused across components;<br />
<br />
<code>streams</code> uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation in the profile and other static files. The approach is best illustrated with a reference to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/streams sources]] of the <code>streams</code> library.<br />
<br />
<br />
As we discuss [[#Creating the ETICS Configurations|below]], <code>streams</code> has the final requirement to produce an interpolated profile during ETICS builds, outside the context of the SA.<br />
<code>streams</code> meets this requirement with the [http://maven.apache.org/plugins/maven-resources-plugin/ Maven Resources Plugin]. Again, consult the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/streams sources]] of the <code>streams</code> library to reuse this approach.<br />
<br />
= Services =<br />
The <code>tree-manager</code> service is a Maven component in gCube class <code>DataAccess</code>. <br />
<br />
== Project Structure ==<br />
<br />
Differently from a library, the service is associated with a number of distinguished build artefacts:<br />
<br />
* the service implementation (a Jar);<br />
* the stubs automatically generated from the WSDL interfaces (a Jar);<br />
* the archive required for deployment in gCore (a Gar);<br />
* the archive required for registration in the system (a SA);<br />
<br />
Producing these artefacts requires sharing a number of static and dynamic files (from profiles to WSDLs to pre-processed WSDLs). The usual Maven approach of mapping the artefacts onto different Maven projects is then genuinely difficult. It is instead more practical to use a single multi-module Maven project. The <code>tree-manager</code> uses this module structure:<br />
<br />
<pre><br />
tree-manager (parent module)------- (produces) --- (POM only)<br />
|<br />
|----- tree-manager-service (service impl module)------- (produces) --- jar, gar, SA<br />
| |<br />
| | (depends on)<br />
| |<br />
| \/<br />
|----- tree-manager-stubs (service stubs module)------- (produces) --- jar<br />
</pre><br />
<br />
As customary, the parent does not produce any artefact (is a POM-only project). It declares these coordinates in its POM:<br />
<br />
<source lang="xml"><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
<packaging>pom</packaging><br />
</source><br />
<br />
The parent physically includes the sub-modules;<br />
<br />
<source lang="xml"><br />
<modules><br />
<module>tree-manager-stubs</module><br />
<module>tree-manager-service</module><br />
</modules><br />
</source> <br />
<br />
Finally, the parent inherits from '''maven-parent''':<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
The submodules inherit from the parent:<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>tree-manager</artifactId><br />
<groupId>org.gcube.data.access</groupId><br />
<version>1.0.0-SNAPSHOT</version><br />
<relativePath>..</relativePath><br />
</parent><br />
</source><br />
<br />
and share its <code>version</code> and <code>groupId</code>. Their coordinates are then minimal:<br />
<br />
<source lang="xml"><br />
<artifactId>tree-manager-service</artifactId><br />
</source><br />
<br />
<source lang="xml"><br />
<artifactId>tree-manager-stubs</artifactId><br />
</source><br />
<br />
The <code>tree-manager-service</code> module produces a Jar of its own implementation, as well the GAR and the SA of the whole services as secondary artefacts with classifiers of, respectively, <code>gar</code> and <code>service archive</code>.<br />
<br />
The <code>tree-manager-stubs</code> module produces a Jar of its own implementation.<br />
<br />
Finally, the parent module coordinates the build of its sub-modules and centralises the declaration of common dependencies and configuration. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] to reuse this approach.<br />
<br />
== Generating Stubs and Gars ==<br />
<br />
The <code>tree-manager-stubs</code> uses the '''gCube Service Plugin''' to generate its classes from WSDL interfaces. <br />
<br />
Similarly, the <code>tree-manager-service</code> uses the plugin to generate its secondary GAR artefact. <br />
<br />
The configuration of the plugin is conveniently shared in the parent module.<br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] of the <code>streams</code> for details.<br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the service is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>DataAccess</Class><br />
<Name>tree-manager-service</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Main><br />
<Name>tree-manager-service</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager-service</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<GARArchive>tree-manager-service-1.0.0-SNAPSHOT.gar</GARArchive><br />
<PortType><br />
<Name>gcube/data/tm/binder</Name><br />
</PortType><br />
<PortType><br />
<Name>gcube/data/tm/reader</Name><br />
</PortType><br />
<PortType><br />
<Name>gcube/data/tm/writer</Name><br />
</PortType><br />
</Main><br />
<Software><br />
<Name>tree-manager-stubs</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager-stubs</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<Type>library</Type><br />
<Files><br />
<File>tree-manager-stubs-1.0.0-SNAPSHOT.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the profile includes two packages and this package corresponds to the main build artefacts of the submodules; <br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code>s of the submodules;<br />
* package versions are aligned with the Maven <code>version</code>s of the submodules;<br />
* package include <code>MavenCoordinates</code>s that can be directly copied and pasted from the POM of the submodules;<br />
* packages do '''not''' specify dependencies;<br />
* packages points to the main artefact of the Maven build of the submodules;<br />
<br />
<strike>Notice that, unlike libraries, services cannot easily use Maven variables in their profiles (as these artefacts are used for different purposes by different modules). Accordingly, developers needs to maintain versions used in profiles across releases. In particular, they need to:<br />
<br />
* replace SNAPSHOT versions with release versions in release branches;<br />
* advance SNAPSHOT versions in HEAD;</strike><br />
<br />
=== Profile Interpolation ===<br />
The interpolation of profiles is required in order to keep information in profile in sync with <code>pom.xml</code>, in particular versions and filenames (that contain versions) that change at each release.<br />
<br />
In order to generate <code>profile.xml</code> during build process you need to:<br />
# add in the service's parent <code>pom.xml</code> the invocation to maven-resources-plugin<br />
# place a ''template'' <code>profile.xml</code> in the <code>${distroDirectory}</code><br />
<br />
Invocation to <code>maven-resources-plugin</code> looks like the following (to be placed in the <plugins> section inside the <build> section of the pom):<br />
<source lang="xml"><br />
<!-- interpolates profiles and copies from distribution location to configuration <br />
location, where it is need for embedding into stub artifact, gar generation, <br />
and service archive. --><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-resources-plugin</artifactId><br />
<version>2.6</version><br />
<executions><br />
<execution><br />
<id>copy-profile</id><br />
<goals><br />
<goal>copy-resources</goal><br />
</goals><br />
<phase>process-resources</phase><br />
<configuration><br />
<outputDirectory>${configDirectory}</outputDirectory><br />
<resources><br />
<resource><br />
<directory>${distroDirectory}</directory><br />
<includes><br />
<include>profile.xml</include><br />
</includes><br />
<filtering>true</filtering><br />
</resource><br />
</resources><br />
</configuration><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
The ''template'' <code>profile.xml</code> looks like the following:<br />
<source lang="xml" highlight="12,16,18,22,26,30"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>TestClass</Class><br />
<Name>TestService</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Main><br />
<Name>TestService</Name><br />
<Version>${project.version}</Version><br />
<MavenCoordinates><br />
<groupId>org.example</groupId><br />
<artifactId>test-service</artifactId><br />
<version>${project.version}</version><br />
</MavenCoordinates><br />
<GARArchive>>test-service-${project.version}.gar</GARArchive><br />
</Main><br />
<Software><br />
<Name>test-stubs</Name><br />
<Version>${project.version}</Version><br />
<MavenCoordinates><br />
<groupId>org.example</groupId><br />
<artifactId>test-stubs</artifactId><br />
<version>${project.version}</version><br />
</MavenCoordinates><br />
<Type>library</Type><br />
<Files><br />
<File>test-stubs-${project.version}.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
In the example, property <code>${project.version}</code> will be substituted with the actual service's version at build-time. It is also possible, but not required, to parametrize also other values of <code>profile.xml</code> such as maven's coordinates and package names. You can see a more complex <code>profile.xml</code> ''template'' in the <code>tree-manager</code> project: http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-access/tree-manager/distro/profile.xml<br />
<br />
Maven will copy the final <code>profile.xml</code> in <code>${configDirectory}</code> during the <code>process-resources</code> phase of the build. From there it will be copied in the service-archive package.<br />
<br />
== Service Archive ==<br />
<br />
<code>tree-manager-service</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the module and its parent.<br />
<br />
As shown for [[#Libraries|libraries]], the service uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] for details.<br />
<br />
= Portlets =<br />
<br />
The <code>sample-portlet</code> portlet is a Maven based sample portlet based on GWT. Refer to the [http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources] for details.<br />
<br />
== Project Structure ==<br />
<br />
As for a library project, the portlet is associated with 2 distinguished build artefacts:<br />
<br />
* the archive required for deployment in a Liferay Container (a WAR);<br />
* the archive required for registration in the system (a SA);<br />
<br />
In order to generate automatically the project structure and the base pom, we start by customizing the maven archetype liferay-portlet-archetype v. 6.0.6.<br />
<br />
The generation of the project structure and the pom can be performed either from Eclipse ( by using the m2eclipse plugin) or by command line using the Maven SDK.<br />
<br />
On Eclipse the following steps must be followed:<br />
<br />
* FIle -> New -> Project<br />
* From the wizard select Maven->Maven Project<br />
* From the Select Archetype step, Select 'All Catalogs' from the Catalogs combobox and type 'portlet' on the Filter field.<br />
* Select the '''liferay-portlet-archetype''' version 6.0.6 archetype ( if does not appear on the list just unselect 'show the last version of the Archetype only")<br />
* In the next step you have to provide Maven coordinates for your project<br />
* groupid : <your group id> e.g org.gcube.portlets.admin<br />
* artifactId : <your artifactId> e.g. sample-admin-portlet<br />
* version : <your project version> e.g. 1.0.0-SNAPSHOT ( use always SNAPSHOT versions for development versions of a project)<br />
* package: <your portlet code package> e.g. org.gcube.portlets.admin.sample<br />
<br />
while from shell the following mvn command should be issued :<br />
<br />
<pre><br />
mvn archetype:generate \<br />
-DarchetypeArtifactId=liferay-theme-archetype \<br />
-DarchetypeGroupId=com.liferay.maven.archetypes \<br />
-DarchetypeVersion=6.0.6 \<br />
-DartifactId=<your artifactId> \<br />
-DgroupId=<your group id> \<br />
-Dversion=1.0.0-SNAPSHOT<br />
</pre><br />
<br />
Both approaches generate a project structure similar to the picture below:<br />
<br />
[[image:sample-portlet-project.png|Sample portlet project structure]]<br />
<br />
In order to customize the build procedure to the gCube needs we should open the pom.xml and apply the following changes:<br />
<br />
Configure the inheritance from '''maven-parent''' :<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
Configure some project properties:<br />
<br />
<source lang="xml"><br />
<properties><br />
<liferay.version>6.0.6</liferay.version><br />
<liferay.auto.deploy.dir>${system.CATALINA_HOME}/../deploy</liferay.auto.deploy.dir><br />
<distroDirectory>${project.basedir}/distro</distroDirectory><br />
<configDirectory>${project.basedir}/config</configDirectory><br />
</properties><br />
</source><br />
<br />
Add the plugin for the generation of the SA, which has to be included under the build plugins section:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<version>2.2</version><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
In the case of a GWT portlet the [http://mojo.codehaus.org/gwt-maven-plugin gwt-maven-plugin] should also be configured ( within build plugins section as well), by also selecting the proper gwt version ( in our example 2.4)<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.codehaus.mojo</groupId><br />
<artifactId>gwt-maven-plugin</artifactId><br />
<version>${gwt.version}</version><br />
<executions><br />
<execution><br />
<goals><br />
<goal>compile</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
The proper GWT version should be added to the pom properties :<br />
<br />
<source lang="xml"><br />
<gwt.version>2.4.0</gwt.version><br />
</source><br />
<br />
and GWT dependencies should be also configured:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-user</artifactId><br />
<version>${gwt.version}</version><br />
</dependency><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-servlet</artifactId><br />
<version>${gwt.version}</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet/ project sources]] for further details on the project configuration.<br />
<br />
== Generating Artifacts ==<br />
<br />
The <code>sample-portlet</code> uses the '''maven-compiler-plugin''' to compile the portlet code and the '''gwt-maven-plugin''' as well for GWT compilation.<br />
<br />
The generation of a war package can be triggered by typing :<br />
<br />
<pre><br />
mvn package <br />
</pre><br />
<br />
on your shell or running this build task within Eclipse.<br />
<br />
In both cases the war package will be generated under the ''target'' folder, e.g.:<br />
<br />
<pre><br />
target/sample-admin-portlet-1.0.0-SNAPSHOT.war<br />
</pre><br />
<br />
The generation of all project artifacts ( war, source jar, javadoc, SA) can be triggered by the build task :<br />
<br />
<pre><br />
mvn install <br />
</pre><br />
<br />
this will install as well all artifacts in your local repository, while the deployment of the artifacts to the remote gCube Maven repos can triggered by the build task:<br />
<br />
<pre><br />
mvn deploy <br />
</pre><br />
<br />
== Portlet dependencies ==<br />
<br />
Specific portlet dependencies ( e.g. GWT, service stubs ) should be packaged within the portlet war. All dependencies specified within the pom file without the ''provided'' scope will be copied inside the WEB-INF/lib folder at packaging time.<br />
<br />
For this reason all portlet specific dependencies should not specified that scope e.g:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.externals</groupId><br />
<artifactId>gwt-ext-patched</artifactId><br />
<version>2.0.5</version><br />
</dependency><br />
</source><br />
<br />
On the other hand the system dependencies ( the one contained within the [[GCube_Portal_Installation#Portal_Bundle_Libraries|Portal bundle]] ) should not be included inside the portlet war. As said in this case the ''provided'' scope should be specified, e.g.<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portal</groupId><br />
<artifactId>custom-portal-handler</artifactId><br />
<version>1.1.0-SNAPSHOT</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.4.0,1.5.0]</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
<br />
A subset of these libraries ( contained in a GHN) that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called <code>ghn-core-runtime</code>. A dependency on this Jar can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ghn-core-runtime</artifactId><br />
<version>1.0.0</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Note: a component that depends on <code>gcf</code> indirectly depends on <code>gun-core-runtime</code>.<br />
<br />
== Include gCube Guided Tour using Maven ==<br />
<br />
This particular component is not yet ported to maven so in order to be included in a maven based project it should be declared as system dependency as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portletsuser</groupId><br />
<artifactId>gcube-guidedtour-widget</artifactId><br />
<version>1.0.0</version><br />
<scope>system</scope><br />
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/gcube-guidedtour-widget.jar</systemPath> <br />
</dependency><br />
</source><br />
<br />
together with the gCube Widget Library:<br />
<br />
<source lang="xml"> <br />
<dependency><br />
<groupId>org.gcube.portlets.user</groupId><br />
<artifactId>gcubewidgets</artifactId><br />
<version>1.2.0</version><br />
<scope>system</scope><br />
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/gCube-Widgets-Library.jar</systemPath><br />
</dependency><br />
</source><br />
<br />
both components can be downloaded from the distribution repository [[http://grids16.eng.it:8080/BuildReport/browse/Recent_Builds]]. <br />
<br />
In case the portlet depends on the ''gcf'' component, some dependency resolution changes are required to your portlet in order to properly compile ( because of libraries clashes)<br />
<br />
The ghn-core-runtime component ( gcf dependency) should be excluded:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.5.1-SNAPSHOT,2.0.0)</version><br />
<scope>provided</scope><br />
<exclusions><br />
<exclusion><br />
<artifactId>ghn-core-runtime</artifactId><br />
<groupId>org.gcube.distribution</groupId><br />
</exclusion><br />
</exclusions><br />
</dependency><br />
</source><br />
<br />
<br />
and some new dependencies should be added:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>xerces</groupId><br />
<artifactId>xercesImpl</artifactId><br />
<version>2.9.1</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ws-addressing</artifactId><br />
<version>1.0</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>axis-patched</artifactId><br />
<version>1.2RC</version><br />
<scope>provided</scope><br />
</dependency><br />
<br />
</source><br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the portlet is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>PortletAdmin</Class><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.portles.admin</groupId><br />
<artifactId>sample-admin-portlet</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/sample-admin-portlet-1.0.0-SNAPSHOT.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code> of the project;<br />
* package versions are aligned with the Maven <code>version</code> of the project;<br />
* package include <code>MavenCoordinates</code> that can be directly copied and pasted from the POM of the project;<br />
* packages do '''not''' specify dependencies;<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>sample-portlet</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>PortletAdmin</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/${build.finalName}.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
== Service Archive ==<br />
<br />
<code>sample-portlet</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the project.<br />
<br />
As shown for [[#Libraries|libraries]], the portlet uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources]] for details.<br />
<br />
== Use Case: Create a new Mavenized GWT Portlet ==<br />
<br />
In order to create a new Mavenized GWT Portlet this detailed guide can be also followed [https://gcube.wiki.gcube-system.org/gcube/index.php/Create_a_new_Mavenized_gCube_GWT_Portlet_Guide]<br />
<br />
= Widgets =<br />
<br />
Falls in this category of components mainly GWT widgets or so called GWT Libraries, which can be used to package classes for mutualization and/or modularization. A Widget is a java archive (JAR) containing both classes and java sources for later GWT compilation and a gwt.xml module descriptor. <br />
<br />
In this section we refer to a sample Widget with maven structure (https://svn.research-infrastructures.eu/d4science/gcube/trunk/portal/mvn-widget-sample/)<br />
<br />
== Project Structure ==<br />
<br />
As for a other project, the portlet is associated with 2 distinguished build artefacts:<br />
<br />
* the archive required for deployment (a JAR);<br />
* the archive required for registration in the system (a SA);<br />
<br />
In order to generate automatically the project structure and the base pom, we start by customizing the maven archetype liferay-portlet-archetype v. 6.0.6.<br />
<br />
The generation of the project structure and the pom can be performed either from Eclipse ( by using the m2eclipse plugin) or by command line using the Maven SDK.<br />
<br />
On Eclipse the following steps must be followed:<br />
<br />
* FIle -> New -> Project<br />
* From the wizard select Maven->Maven Project<br />
* From the Select Archetype step, Select 'All Catalogs' from the Catalogs combobox and type 'portlet' on the Filter field.<br />
* Select the '''maven-archetype-quickstart''' version 1.1.0 archetype ( if does not appear on the list just unselect 'show the last version of the Archetype only")<br />
* In the next step you have to provide Maven coordinates for your project<br />
* groupid : <your group id> e.g org.gcube.portlets.admin<br />
* artifactId : <your artifactId> e.g. sample-admin-portlet<br />
* version : <your project version> e.g. 1.0.0-SNAPSHOT ( use always SNAPSHOT versions for development versions of a project)<br />
* package: <your portlet code package> e.g. org.gcube.portlets.admin.sample<br />
<br />
while from shell the following mvn command should be issued :<br />
<br />
<pre><br />
mvn archetype:generate \<br />
-DarchetypeArtifactId=maven-archetype-quickstart \<br />
-DarchetypeGroupId=com.liferay.maven.archetypes \<br />
-DarchetypeVersion=1.1.0 \<br />
-DartifactId=<your artifactId> \<br />
-DgroupId=<your group id> \<br />
-Dversion=1.0.0-SNAPSHOT<br />
</pre><br />
<br />
Both approaches generate a project structure similar to the picture below:<br />
<br />
[[image:sample-widget-project.png|Sample widget project structure]]<br />
<br />
In order to customize the build procedure to the gCube needs we should open the pom.xml and apply the following changes:<br />
<br />
Configure the inheritance from '''maven-parent''' :<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
Configure some project properties:<br />
<br />
<source lang="xml"><br />
<properties><br />
<distroDirectory>${project.basedir}/distro</distroDirectory><br />
<configDirectory>${project.basedir}/config</configDirectory><br />
</properties><br />
</source><br />
<br />
Add the plugin for the generation of the SA, which has to be included under the build plugins section:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<version>2.2</version><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
In the case of a GWT widget the [http://mojo.codehaus.org/gwt-maven-plugin gwt-maven-plugin] should also be configured ( within build plugins section as well), by also selecting the proper gwt version ( in our example 2.4)<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.codehaus.mojo</groupId><br />
<artifactId>gwt-maven-plugin</artifactId><br />
<version>${gwt.version}</version><br />
<executions><br />
<execution><br />
<goals><br />
<goal>compile</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
The proper GWT version should be added to the pom properties :<br />
<br />
<source lang="xml"><br />
<gwt.version>2.4.0</gwt.version><br />
</source><br />
<br />
and GWT dependencies should be also configured:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-user</artifactId><br />
<version>${gwt.version}</version><br />
</dependency><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-servlet</artifactId><br />
<version>${gwt.version}</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/mvn-widget-sample/ project sources]] for further details on the project configuration.<br />
<br />
== Generating Artifacts ==<br />
<br />
The <code>sample-portlet</code> uses the '''maven-compiler-plugin''' to compile the portlet code and the '''gwt-maven-plugin''' as well for GWT compilation.<br />
<br />
The generation of a war package can be triggered by typing :<br />
<br />
<pre><br />
mvn package <br />
</pre><br />
<br />
on your shell or running this build task within Eclipse.<br />
<br />
In both cases the war package will be generated under the ''target'' folder, e.g.:<br />
<br />
<pre><br />
target/sample-admin-portlet-1.0.0-SNAPSHOT.war<br />
</pre><br />
<br />
The generation of all project artifacts ( war, source jar, javadoc, SA) can be triggered by the build task :<br />
<br />
<pre><br />
mvn install <br />
</pre><br />
<br />
this will install as well all artifacts in your local repository, while the deployment of the artifacts to the remote gCube Maven repos can triggered by the build task:<br />
<br />
<pre><br />
mvn deploy <br />
</pre><br />
<br />
== Widget dependencies ==<br />
<br />
The system dependencies ( the one contained within the [[GCube_Portal_Installation#Portal_Bundle_Libraries|Portal bundle]] ) should not be included inside the portlet war. As said in this case the ''provided'' scope should be specified, e.g.<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portal</groupId><br />
<artifactId>custom-portal-handler</artifactId><br />
<version>1.1.0-SNAPSHOT</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.4.0,1.5.0]</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the widget is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>PortletAdmin</Class><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.portles.admin</groupId><br />
<artifactId>sample-admin-portlet</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/sample-admin-portlet-1.0.0-SNAPSHOT.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code> of the project;<br />
* package versions are aligned with the Maven <code>version</code> of the project;<br />
* package include <code>MavenCoordinates</code> that can be directly copied and pasted from the POM of the project;<br />
* packages do '''not''' specify dependencies;<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>sample-portlet</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>PortletAdmin</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/${build.finalName}.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
== Service Archive ==<br />
<br />
<code>sample-portlet</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the project.<br />
<br />
As shown for [[#Libraries|libraries]], the portlet uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources]] for details.</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Developing_gCube_Maven_Components&diff=29032Developing gCube Maven Components2017-05-04T09:16:30Z<p>Gabriele.giammatteo: /* Java Version */</p>
<hr />
<div>[[Category: Developer's Guide]]<br />
__TOC__<br />
<br />
= Common =<br />
== Maven Coordinates ==<br />
Maven components' coordinates must follow these conventions:<br />
<br />
* their <code>groupId</code> must be of the following form:<br />
<br />
<pre>org.gcube.<class></pre><br />
<br />
: where ''class'' identifies a functionally correlated set of components and helps a range of users (developers, administrator) to discover components in various UIs/reports used within the system and in tools. Logically, the class corresponds to the gCube class of the component's, as specified in its gCube Software Profile. However, more specific, or else cross-class groups may be agreed upon among developers. In all cases, the group should align with official Maven [http://maven.apache.org/guides/mini/guide-naming-conventions.html guidelines]. By agreement, for example, components in gCube class <code>DataAccess</code> declare a <code>groupId</code> of <code>org.gcube.data.access</code> and those in gCube class <code>DataTransfer</code> declare a <code>groupId</code> of <code>org.gcube.data.transfer</code>. On the other hand, also by agreement, components in gCube class <code>InformationSystem</code> declare a <code>groupId</code> of <code>org.gcube.informationsystem</code>.<br />
<br />
* their <code>artifactId</code> must follow Maven conventions (e.g. ''avoid capital letters or punctuation other than hyphens'').<br />
<br />
== Parent ==<br />
All gCube components must inherit from '''maven-parent'''. This ensure and promotes compliance with project-wide requirements, from the enforcement of minimal Java and Maven versions to generation and packaging of Javadoc documentation and component sources.<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
<br />
== The distro folder ==<br />
This folder contains files that will be distributed along with the code in the distribution packages. Further details are in the [[Maven distro directory layout]] page.<br />
<br />
== Service Archive (SA) ==<br />
Whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary <code>tar.gz</code> artefact with classifier <code>servicearchive</code>. Note that the SA of Maven components does ''not'' need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);<br />
<br />
* for each deployable artefact they produce (primary or secondary), they must include the Maven coordinates of the arteact in the corresponding package section of the gCube Software Profile. Specifying package dependencies in the Software Profile, on the other hand, is no longer required;<br />
<br />
* whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary <code>tar.gz</code> artefact with classifier <code>servicearchive</code>. Note that the SA of Maven components does ''not'' need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);<br />
<br />
== Dependencies ==<br />
Service and libraries that depend on other gCube components (Maven-based or Ant-based) can find them in the gCube Maven Repositories: <code>gcube-snapshots</code> <strike>and <code>gcube-releases</code></strike>. <br />
<br />
=== gCube Maven Repositories ===<br />
As a one-off task, developers must configure these repositories in the <code>~/.m2/settings.xml</code> file of their local Maven installation, as follows:<br />
<br />
<source lang="xml"><br />
<settings xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"<br />
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd"><br />
<br />
<servers><br />
<server><br />
<id>gcube-snapshots</id><br />
<username>gcube-user</username><br />
<password>maven</password><br />
</server><br />
</servers><br />
<br />
<br />
<profiles><br />
<profile><br />
<id>gcube</id><br />
<repositories><br />
<repository><br />
<id>gcube-snapshots</id><br />
<name>gCube Snapshots</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
</snapshots><br />
</repository><br />
<repository><br />
<id>gcube-externals</id><br />
<name>gCube Externals</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals</url><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
</repository><br />
</repositories><br />
<br />
<pluginRepositories><br />
<pluginRepository><br />
<id>gcube-snapshots</id><br />
<name>gCube Snapshots</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
</snapshots><br />
</pluginRepository><br />
<pluginRepository><br />
<id>gcube-externals</id><br />
<name>gCube Externals</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals</url><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
</pluginRepository><br />
</pluginRepositories><br />
<br />
</profile><br />
</profiles><br />
<br />
<activeProfiles><br />
<activeProfile>gcube</activeProfile><br />
</activeProfiles><br />
</settings><br />
</source><br />
<br />
<br />
'''Note:Developers must not configure these repositories in their POMs''', only in <code>settings.xml</code>. <br />
<br />
This is to:<br />
* keep their projects independent from future changes in repository locations; <br />
* avoid interference with ETICS builds, where access to these repositories may comprise correct system integration;<br />
<br />
With this settings.xml, developers will have access on their local development environment to development versions (gcube-snapshots) and externals (gcube-externals). This is the correct behaviour for most of use cases. If access to staging or releases repositories is necessary (e.g. patching a component already released in the branch where dependencies versions in the release are not in snapshots), developer should modify his settings.xml to point to gcube-staging(releases) instead of gcube-snapshots.<br />
<br />
=== Third-party Repositories ===<br />
Using the the <code>settings.xml</code> aforementioned, any dependency declared in pom.xml that can be found in one of gCube repositories or in Maven Central will be resolved at build time and deployment time without any further configuration. Frequently it may happens that a gCube component needs software stored on a public Maven repository not in on of those repositories. In this case, Maven provides an easy solution for declaring additional repositories for dependency resolution by adding a <repositories> section in the pom.xml of the project. E.g.:<br />
<br />
<source lang="xml" highlight="3,4"><br />
<repositories><br />
<repository><br />
<id>[repository-id]</id><br />
<url>[repository URL]</url><br />
</repository><br />
</repositories><br />
</source><br />
<br />
Though technically possible to add any Maven repository in this way, in order to preserve the quality of gCube software it is required to check trustworthiness of its dependencies disallowing usage of certain external software. To enforce such a control over public Maven repositories, '''gCube components are allowed to use external repository not directly but only through a proxy repository''' (a.k.a. shadow repository) created on CNR's Nexus Repository Portal (http://maven.research-infrastructures.eu/nexus/).<br />
<br />
List of all repositories in Nexus Portal is available at http://maven.research-infrastructures.eu/nexus/index.html#view-repositories<br />
<br />
'''Note''': if a component uses a public Maven repository directly, it will work for building but would fail in deployment phase. In fact, Software Gateway is only able to download software accessible from Nexus.<br />
<br />
==== Request new proxy repository in Nexus ====<br />
Proxy Repositories on Nexus are created by the Release Manager. To request a new proxy a new ticket must be created.<br />
<br />
Details of ticket are:<br />
# open a ticket in tracker "Support" in gCube ticketing system (at https://support.d4science.org/projects/gcube/)<br />
# ticket's '''description''' must report:<br />
#* URL of repository to add<br />
#* organization which hosts the repository<br />
#* which artifacts are needed from that repository<br />
#* which components needs those artifacts<br />
# '''Sprint''' field should be set to the gCube release in which the component is expected to be released <br />
# '''Category''' field must report the gCube subsystem that needs the external repository<br />
# '''Assignee''' is the Release Manager<br />
<br />
An example of ticket for requesting a new repository mirror is reported in the figure below.<br />
[[Image:New_maven_mirror_repo.png|600px|center|Ticket for requesting a new Mirror Repository]]<br />
<br />
Release Manager will process the ticket checking that no issues are foreseen from the usage of software stored in the repository. If request is accepted, a new proxy repository is created and the repository's URL will be communicated publicly on gCube's developers mailinglist (a.k.a. TCOM mailinglist).<br />
<br />
Knowing the mirror repository URL, developer can use it adding following lines in pom.xml. E.g.<br />
<br />
<source lang="xml" highlight="3,4"><br />
<repositories><br />
<repository><br />
<id>gcube-ext-osgeo</id><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/osgeo/</url><br />
</repository><br />
</repositories><br />
</source><br />
<br />
'''Note''': it is possible that software stored on proxy repositories is not included in search results on Nexus. This happens for those repositories that does not have indexes (or have indexes in a format that Nexus is not able to download). Anyway, once Nexus has downloaded artifacts from the original repository for the first time (e.g. because a compilation of a pom that uses the proxy repository), creates its own indexes and those artifacts will become available also for local search.<br />
<br />
=== Dependency Versions ===<br />
<br />
ETICS builds ensure correct integration of gCube components, regardless of the versions their dependencies. It also ensures that components are released with fixed versions for their dependencies, i.e. ensures build reproducibility.<br />
<br />
Accordingly, developers can be liberal in the choice of dependency versions. i.e. rely on version ranges to:<br />
<br />
* minimise the requirements for version management;<br />
* be readily informed of any mis-alignment with their dependencies;<br />
<br />
As we start adopting Maven, the lower-bound of version ranges will be a SNAPSHOT version. For example, a dependency on the <code>streams</code> library can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>streams</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
</dependency><br />
</source><br />
<br />
With this dependency, a component will be soon notified of a non-retro compatible change that, contrary to conventions, <code>streams</code> may have introduced in, say, version <code>1.3.0</code> (typically the day after at the latest). The notification will be delivered in the IDE, assuming the availability of Maven integration (e.g. <code>m2e</code> in Eclipse). <br />
The component is in fact free to open the range further if it wants to be aligned with its dependencies across major versions.<br />
<br />
=== Common Dependencies ===<br />
<br />
Many gCube components depend on <code>gcf</code>, which is now available as a Maven component since version <code>1.4</code>. At the time of writing, the component has not been released yet. A dependency on it is then specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
The scope <code>provided</code> indicates that these dependency will be satisfied in the development environment (the gHN) and should not be included in GAR and SA archives.<br />
<br />
Some components depend on the libraries available in all gHNs, though they do not depend on <code>gcf</code> itself. The subset of these libraries that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called <code>ghn-core-runtime</code>. A dependency on this Jar can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ghn-core-runtime</artifactId><br />
<version>1.0.0</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Note: a component that depends on <code>gcf</code> indirectly depends on <code>gun-core-runtime</code>.<br />
<br />
Another common dependency is on the gCube gRS2, an Ant component that the build infrastructure make available in Maven repositories. A dependency on this component can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.execution</groupId><br />
<artifactId>grs2library</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
</dependency><br />
</source><br />
<br />
== Java Version ==<br />
<br />
In Maven the Java version of the source code and of the generated classes can be controlled using the configuration of the <code>maven-compiler-plugin</code><br />
<syntaxhighlight lang="xml" highlight="5,6"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-compiler-plugin</artifactId><br />
<configuration><br />
<source>1.7</source><br />
<target>1.8</target><br />
</configuration><br />
</plugin><br />
</syntaxhighlight><br />
<br />
<br />
In gCube, this mechanism is exploited to enforce the usage of the same Java version for all components. In particular, the maven-parent defines the property <code>java_version</code> and use this property in the configuration of the <code>maven-compiler-plugin</code>. '''All the gCube components inherit this configuration from maven-parent and it is not recommended to override it'''.<br />
<br />
<syntaxhighlight lang="xml" highlight="13,14"><br />
...<br />
<properties><br />
<java_version>1.8</java_version><br />
...<br />
</properties><br />
<br />
...<br />
<br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-compiler-plugin</artifactId><br />
<configuration><br />
<source>${java_version}</source><br />
<target>${java_version}</target><br />
</configuration><br />
</plugin><br />
...<br />
</syntaxhighlight><br />
<br />
<br />
Using the property instead of specify the version in the configuration, we are more flexible because the property can be overridden at various levels (settings, profile, pom, commandline) and allow to customize the java version to use depending on different factors (e.g. during ETICS builds gCore components are built with Java 7, all the others with Java 8).<br />
<br />
Current value for the <code>java_version</code> property is 1.8.<br />
<br />
= Libraries =<br />
The <code>streams</code> library is a Maven component in gCube class <code>DataAccess</code>. We will use it as example of a library component in Maven.<br />
<br />
=== Profile ===<br />
<br />
The gCube Software profile of the library is as follows:<br />
<br />
<source lang="xml" highlight="14-18"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>Embedded Domain-Specific Language for Stream Transformations</Description><br />
<Class>DataAccess</Class><br />
<Name>streams</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>streams</Name><br />
<Version>1.0.0-SNAPSHOT</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>streams</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<Files><br />
<File>streams-1.0.0-SNAPSHOT.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the profile includes a single package and this package corresponds to the main build artefact of the component; <br />
* the whole profile and the package have usual gCube coordinates;<br />
* the package name is aligned with the Maven <code>artifactId</code> of the component;<br />
* the package version is aligned with the Maven <code>version</code> of the component;<br />
* the package includes <code>MavenCoordinates</code> that can be directly copied and pasted from the POM;<br />
* the package does '''not''' specify dependencies;<br />
* the package points to the main artefact of the Maven build;<br />
<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>streams</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>DataAccess</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<Files><br />
<File>${build.finalName}.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
=== Service Archive ===<br />
<br />
<code>streams</code> is responsible for generating its own SA as part of its Maven build. This requires dedicated logic in the POM but has the advantage that:<br />
<br />
* SA validity can be verified locally;<br />
* no SA configurations are required in ETICS;<br />
* the required build logic can be easily reused across components;<br />
<br />
<code>streams</code> uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation in the profile and other static files. The approach is best illustrated with a reference to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/streams sources]] of the <code>streams</code> library.<br />
<br />
<br />
As we discuss [[#Creating the ETICS Configurations|below]], <code>streams</code> has the final requirement to produce an interpolated profile during ETICS builds, outside the context of the SA.<br />
<code>streams</code> meets this requirement with the [http://maven.apache.org/plugins/maven-resources-plugin/ Maven Resources Plugin]. Again, consult the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/streams sources]] of the <code>streams</code> library to reuse this approach.<br />
<br />
= Services =<br />
The <code>tree-manager</code> service is a Maven component in gCube class <code>DataAccess</code>. <br />
<br />
== Project Structure ==<br />
<br />
Differently from a library, the service is associated with a number of distinguished build artefacts:<br />
<br />
* the service implementation (a Jar);<br />
* the stubs automatically generated from the WSDL interfaces (a Jar);<br />
* the archive required for deployment in gCore (a Gar);<br />
* the archive required for registration in the system (a SA);<br />
<br />
Producing these artefacts requires sharing a number of static and dynamic files (from profiles to WSDLs to pre-processed WSDLs). The usual Maven approach of mapping the artefacts onto different Maven projects is then genuinely difficult. It is instead more practical to use a single multi-module Maven project. The <code>tree-manager</code> uses this module structure:<br />
<br />
<pre><br />
tree-manager (parent module)------- (produces) --- (POM only)<br />
|<br />
|----- tree-manager-service (service impl module)------- (produces) --- jar, gar, SA<br />
| |<br />
| | (depends on)<br />
| |<br />
| \/<br />
|----- tree-manager-stubs (service stubs module)------- (produces) --- jar<br />
</pre><br />
<br />
As customary, the parent does not produce any artefact (is a POM-only project). It declares these coordinates in its POM:<br />
<br />
<source lang="xml"><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
<packaging>pom</packaging><br />
</source><br />
<br />
The parent physically includes the sub-modules;<br />
<br />
<source lang="xml"><br />
<modules><br />
<module>tree-manager-stubs</module><br />
<module>tree-manager-service</module><br />
</modules><br />
</source> <br />
<br />
Finally, the parent inherits from '''maven-parent''':<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
The submodules inherit from the parent:<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>tree-manager</artifactId><br />
<groupId>org.gcube.data.access</groupId><br />
<version>1.0.0-SNAPSHOT</version><br />
<relativePath>..</relativePath><br />
</parent><br />
</source><br />
<br />
and share its <code>version</code> and <code>groupId</code>. Their coordinates are then minimal:<br />
<br />
<source lang="xml"><br />
<artifactId>tree-manager-service</artifactId><br />
</source><br />
<br />
<source lang="xml"><br />
<artifactId>tree-manager-stubs</artifactId><br />
</source><br />
<br />
The <code>tree-manager-service</code> module produces a Jar of its own implementation, as well the GAR and the SA of the whole services as secondary artefacts with classifiers of, respectively, <code>gar</code> and <code>service archive</code>.<br />
<br />
The <code>tree-manager-stubs</code> module produces a Jar of its own implementation.<br />
<br />
Finally, the parent module coordinates the build of its sub-modules and centralises the declaration of common dependencies and configuration. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] to reuse this approach.<br />
<br />
== Generating Stubs and Gars ==<br />
<br />
The <code>tree-manager-stubs</code> uses the '''gCube Service Plugin''' to generate its classes from WSDL interfaces. <br />
<br />
Similarly, the <code>tree-manager-service</code> uses the plugin to generate its secondary GAR artefact. <br />
<br />
The configuration of the plugin is conveniently shared in the parent module.<br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] of the <code>streams</code> for details.<br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the service is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>DataAccess</Class><br />
<Name>tree-manager-service</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Main><br />
<Name>tree-manager-service</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager-service</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<GARArchive>tree-manager-service-1.0.0-SNAPSHOT.gar</GARArchive><br />
<PortType><br />
<Name>gcube/data/tm/binder</Name><br />
</PortType><br />
<PortType><br />
<Name>gcube/data/tm/reader</Name><br />
</PortType><br />
<PortType><br />
<Name>gcube/data/tm/writer</Name><br />
</PortType><br />
</Main><br />
<Software><br />
<Name>tree-manager-stubs</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager-stubs</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<Type>library</Type><br />
<Files><br />
<File>tree-manager-stubs-1.0.0-SNAPSHOT.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the profile includes two packages and this package corresponds to the main build artefacts of the submodules; <br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code>s of the submodules;<br />
* package versions are aligned with the Maven <code>version</code>s of the submodules;<br />
* package include <code>MavenCoordinates</code>s that can be directly copied and pasted from the POM of the submodules;<br />
* packages do '''not''' specify dependencies;<br />
* packages points to the main artefact of the Maven build of the submodules;<br />
<br />
<strike>Notice that, unlike libraries, services cannot easily use Maven variables in their profiles (as these artefacts are used for different purposes by different modules). Accordingly, developers needs to maintain versions used in profiles across releases. In particular, they need to:<br />
<br />
* replace SNAPSHOT versions with release versions in release branches;<br />
* advance SNAPSHOT versions in HEAD;</strike><br />
<br />
=== Profile Interpolation ===<br />
The interpolation of profiles is required in order to keep information in profile in sync with <code>pom.xml</code>, in particular versions and filenames (that contain versions) that change at each release.<br />
<br />
In order to generate <code>profile.xml</code> during build process you need to:<br />
# add in the service's parent <code>pom.xml</code> the invocation to maven-resources-plugin<br />
# place a ''template'' <code>profile.xml</code> in the <code>${distroDirectory}</code><br />
<br />
Invocation to <code>maven-resources-plugin</code> looks like the following (to be placed in the <plugins> section inside the <build> section of the pom):<br />
<source lang="xml"><br />
<!-- interpolates profiles and copies from distribution location to configuration <br />
location, where it is need for embedding into stub artifact, gar generation, <br />
and service archive. --><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-resources-plugin</artifactId><br />
<version>2.6</version><br />
<executions><br />
<execution><br />
<id>copy-profile</id><br />
<goals><br />
<goal>copy-resources</goal><br />
</goals><br />
<phase>process-resources</phase><br />
<configuration><br />
<outputDirectory>${configDirectory}</outputDirectory><br />
<resources><br />
<resource><br />
<directory>${distroDirectory}</directory><br />
<includes><br />
<include>profile.xml</include><br />
</includes><br />
<filtering>true</filtering><br />
</resource><br />
</resources><br />
</configuration><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
The ''template'' <code>profile.xml</code> looks like the following:<br />
<source lang="xml" highlight="12,16,18,22,26,30"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>TestClass</Class><br />
<Name>TestService</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Main><br />
<Name>TestService</Name><br />
<Version>${project.version}</Version><br />
<MavenCoordinates><br />
<groupId>org.example</groupId><br />
<artifactId>test-service</artifactId><br />
<version>${project.version}</version><br />
</MavenCoordinates><br />
<GARArchive>>test-service-${project.version}.gar</GARArchive><br />
</Main><br />
<Software><br />
<Name>test-stubs</Name><br />
<Version>${project.version}</Version><br />
<MavenCoordinates><br />
<groupId>org.example</groupId><br />
<artifactId>test-stubs</artifactId><br />
<version>${project.version}</version><br />
</MavenCoordinates><br />
<Type>library</Type><br />
<Files><br />
<File>test-stubs-${project.version}.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
In the example, property <code>${project.version}</code> will be substituted with the actual service's version at build-time. It is also possible, but not required, to parametrize also other values of <code>profile.xml</code> such as maven's coordinates and package names. You can see a more complex <code>profile.xml</code> ''template'' in the <code>tree-manager</code> project: http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-access/tree-manager/distro/profile.xml<br />
<br />
Maven will copy the final <code>profile.xml</code> in <code>${configDirectory}</code> during the <code>process-resources</code> phase of the build. From there it will be copied in the service-archive package.<br />
<br />
== Service Archive ==<br />
<br />
<code>tree-manager-service</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the module and its parent.<br />
<br />
As shown for [[#Libraries|libraries]], the service uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] for details.<br />
<br />
= Portlets =<br />
<br />
The <code>sample-portlet</code> portlet is a Maven based sample portlet based on GWT. Refer to the [http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources] for details.<br />
<br />
== Project Structure ==<br />
<br />
As for a library project, the portlet is associated with 2 distinguished build artefacts:<br />
<br />
* the archive required for deployment in a Liferay Container (a WAR);<br />
* the archive required for registration in the system (a SA);<br />
<br />
In order to generate automatically the project structure and the base pom, we start by customizing the maven archetype liferay-portlet-archetype v. 6.0.6.<br />
<br />
The generation of the project structure and the pom can be performed either from Eclipse ( by using the m2eclipse plugin) or by command line using the Maven SDK.<br />
<br />
On Eclipse the following steps must be followed:<br />
<br />
* FIle -> New -> Project<br />
* From the wizard select Maven->Maven Project<br />
* From the Select Archetype step, Select 'All Catalogs' from the Catalogs combobox and type 'portlet' on the Filter field.<br />
* Select the '''liferay-portlet-archetype''' version 6.0.6 archetype ( if does not appear on the list just unselect 'show the last version of the Archetype only")<br />
* In the next step you have to provide Maven coordinates for your project<br />
* groupid : <your group id> e.g org.gcube.portlets.admin<br />
* artifactId : <your artifactId> e.g. sample-admin-portlet<br />
* version : <your project version> e.g. 1.0.0-SNAPSHOT ( use always SNAPSHOT versions for development versions of a project)<br />
* package: <your portlet code package> e.g. org.gcube.portlets.admin.sample<br />
<br />
while from shell the following mvn command should be issued :<br />
<br />
<pre><br />
mvn archetype:generate \<br />
-DarchetypeArtifactId=liferay-theme-archetype \<br />
-DarchetypeGroupId=com.liferay.maven.archetypes \<br />
-DarchetypeVersion=6.0.6 \<br />
-DartifactId=<your artifactId> \<br />
-DgroupId=<your group id> \<br />
-Dversion=1.0.0-SNAPSHOT<br />
</pre><br />
<br />
Both approaches generate a project structure similar to the picture below:<br />
<br />
[[image:sample-portlet-project.png|Sample portlet project structure]]<br />
<br />
In order to customize the build procedure to the gCube needs we should open the pom.xml and apply the following changes:<br />
<br />
Configure the inheritance from '''maven-parent''' :<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
Configure some project properties:<br />
<br />
<source lang="xml"><br />
<properties><br />
<liferay.version>6.0.6</liferay.version><br />
<liferay.auto.deploy.dir>${system.CATALINA_HOME}/../deploy</liferay.auto.deploy.dir><br />
<distroDirectory>${project.basedir}/distro</distroDirectory><br />
<configDirectory>${project.basedir}/config</configDirectory><br />
</properties><br />
</source><br />
<br />
Add the plugin for the generation of the SA, which has to be included under the build plugins section:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<version>2.2</version><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
In the case of a GWT portlet the [http://mojo.codehaus.org/gwt-maven-plugin gwt-maven-plugin] should also be configured ( within build plugins section as well), by also selecting the proper gwt version ( in our example 2.4)<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.codehaus.mojo</groupId><br />
<artifactId>gwt-maven-plugin</artifactId><br />
<version>${gwt.version}</version><br />
<executions><br />
<execution><br />
<goals><br />
<goal>compile</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
The proper GWT version should be added to the pom properties :<br />
<br />
<source lang="xml"><br />
<gwt.version>2.4.0</gwt.version><br />
</source><br />
<br />
and GWT dependencies should be also configured:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-user</artifactId><br />
<version>${gwt.version}</version><br />
</dependency><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-servlet</artifactId><br />
<version>${gwt.version}</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet/ project sources]] for further details on the project configuration.<br />
<br />
== Generating Artifacts ==<br />
<br />
The <code>sample-portlet</code> uses the '''maven-compiler-plugin''' to compile the portlet code and the '''gwt-maven-plugin''' as well for GWT compilation.<br />
<br />
The generation of a war package can be triggered by typing :<br />
<br />
<pre><br />
mvn package <br />
</pre><br />
<br />
on your shell or running this build task within Eclipse.<br />
<br />
In both cases the war package will be generated under the ''target'' folder, e.g.:<br />
<br />
<pre><br />
target/sample-admin-portlet-1.0.0-SNAPSHOT.war<br />
</pre><br />
<br />
The generation of all project artifacts ( war, source jar, javadoc, SA) can be triggered by the build task :<br />
<br />
<pre><br />
mvn install <br />
</pre><br />
<br />
this will install as well all artifacts in your local repository, while the deployment of the artifacts to the remote gCube Maven repos can triggered by the build task:<br />
<br />
<pre><br />
mvn deploy <br />
</pre><br />
<br />
== Portlet dependencies ==<br />
<br />
Specific portlet dependencies ( e.g. GWT, service stubs ) should be packaged within the portlet war. All dependencies specified within the pom file without the ''provided'' scope will be copied inside the WEB-INF/lib folder at packaging time.<br />
<br />
For this reason all portlet specific dependencies should not specified that scope e.g:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.externals</groupId><br />
<artifactId>gwt-ext-patched</artifactId><br />
<version>2.0.5</version><br />
</dependency><br />
</source><br />
<br />
On the other hand the system dependencies ( the one contained within the [[GCube_Portal_Installation#Portal_Bundle_Libraries|Portal bundle]] ) should not be included inside the portlet war. As said in this case the ''provided'' scope should be specified, e.g.<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portal</groupId><br />
<artifactId>custom-portal-handler</artifactId><br />
<version>1.1.0-SNAPSHOT</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.4.0,1.5.0]</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
<br />
A subset of these libraries ( contained in a GHN) that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called <code>ghn-core-runtime</code>. A dependency on this Jar can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ghn-core-runtime</artifactId><br />
<version>1.0.0</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Note: a component that depends on <code>gcf</code> indirectly depends on <code>gun-core-runtime</code>.<br />
<br />
== Include gCube Guided Tour using Maven ==<br />
<br />
This particular component is not yet ported to maven so in order to be included in a maven based project it should be declared as system dependency as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portletsuser</groupId><br />
<artifactId>gcube-guidedtour-widget</artifactId><br />
<version>1.0.0</version><br />
<scope>system</scope><br />
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/gcube-guidedtour-widget.jar</systemPath> <br />
</dependency><br />
</source><br />
<br />
together with the gCube Widget Library:<br />
<br />
<source lang="xml"> <br />
<dependency><br />
<groupId>org.gcube.portlets.user</groupId><br />
<artifactId>gcubewidgets</artifactId><br />
<version>1.2.0</version><br />
<scope>system</scope><br />
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/gCube-Widgets-Library.jar</systemPath><br />
</dependency><br />
</source><br />
<br />
both components can be downloaded from the distribution repository [[http://grids16.eng.it:8080/BuildReport/browse/Recent_Builds]]. <br />
<br />
In case the portlet depends on the ''gcf'' component, some dependency resolution changes are required to your portlet in order to properly compile ( because of libraries clashes)<br />
<br />
The ghn-core-runtime component ( gcf dependency) should be excluded:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.5.1-SNAPSHOT,2.0.0)</version><br />
<scope>provided</scope><br />
<exclusions><br />
<exclusion><br />
<artifactId>ghn-core-runtime</artifactId><br />
<groupId>org.gcube.distribution</groupId><br />
</exclusion><br />
</exclusions><br />
</dependency><br />
</source><br />
<br />
<br />
and some new dependencies should be added:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>xerces</groupId><br />
<artifactId>xercesImpl</artifactId><br />
<version>2.9.1</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ws-addressing</artifactId><br />
<version>1.0</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>axis-patched</artifactId><br />
<version>1.2RC</version><br />
<scope>provided</scope><br />
</dependency><br />
<br />
</source><br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the portlet is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>PortletAdmin</Class><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.portles.admin</groupId><br />
<artifactId>sample-admin-portlet</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/sample-admin-portlet-1.0.0-SNAPSHOT.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code> of the project;<br />
* package versions are aligned with the Maven <code>version</code> of the project;<br />
* package include <code>MavenCoordinates</code> that can be directly copied and pasted from the POM of the project;<br />
* packages do '''not''' specify dependencies;<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>sample-portlet</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>PortletAdmin</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/${build.finalName}.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
== Service Archive ==<br />
<br />
<code>sample-portlet</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the project.<br />
<br />
As shown for [[#Libraries|libraries]], the portlet uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources]] for details.<br />
<br />
== Use Case: Create a new Mavenized GWT Portlet ==<br />
<br />
In order to create a new Mavenized GWT Portlet this detailed guide can be also followed [https://gcube.wiki.gcube-system.org/gcube/index.php/Create_a_new_Mavenized_gCube_GWT_Portlet_Guide]<br />
<br />
= Widgets =<br />
<br />
Falls in this category of components mainly GWT widgets or so called GWT Libraries, which can be used to package classes for mutualization and/or modularization. A Widget is a java archive (JAR) containing both classes and java sources for later GWT compilation and a gwt.xml module descriptor. <br />
<br />
In this section we refer to a sample Widget with maven structure (https://svn.research-infrastructures.eu/d4science/gcube/trunk/portal/mvn-widget-sample/)<br />
<br />
== Project Structure ==<br />
<br />
As for a other project, the portlet is associated with 2 distinguished build artefacts:<br />
<br />
* the archive required for deployment (a JAR);<br />
* the archive required for registration in the system (a SA);<br />
<br />
In order to generate automatically the project structure and the base pom, we start by customizing the maven archetype liferay-portlet-archetype v. 6.0.6.<br />
<br />
The generation of the project structure and the pom can be performed either from Eclipse ( by using the m2eclipse plugin) or by command line using the Maven SDK.<br />
<br />
On Eclipse the following steps must be followed:<br />
<br />
* FIle -> New -> Project<br />
* From the wizard select Maven->Maven Project<br />
* From the Select Archetype step, Select 'All Catalogs' from the Catalogs combobox and type 'portlet' on the Filter field.<br />
* Select the '''maven-archetype-quickstart''' version 1.1.0 archetype ( if does not appear on the list just unselect 'show the last version of the Archetype only")<br />
* In the next step you have to provide Maven coordinates for your project<br />
* groupid : <your group id> e.g org.gcube.portlets.admin<br />
* artifactId : <your artifactId> e.g. sample-admin-portlet<br />
* version : <your project version> e.g. 1.0.0-SNAPSHOT ( use always SNAPSHOT versions for development versions of a project)<br />
* package: <your portlet code package> e.g. org.gcube.portlets.admin.sample<br />
<br />
while from shell the following mvn command should be issued :<br />
<br />
<pre><br />
mvn archetype:generate \<br />
-DarchetypeArtifactId=maven-archetype-quickstart \<br />
-DarchetypeGroupId=com.liferay.maven.archetypes \<br />
-DarchetypeVersion=1.1.0 \<br />
-DartifactId=<your artifactId> \<br />
-DgroupId=<your group id> \<br />
-Dversion=1.0.0-SNAPSHOT<br />
</pre><br />
<br />
Both approaches generate a project structure similar to the picture below:<br />
<br />
[[image:sample-widget-project.png|Sample widget project structure]]<br />
<br />
In order to customize the build procedure to the gCube needs we should open the pom.xml and apply the following changes:<br />
<br />
Configure the inheritance from '''maven-parent''' :<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
Configure some project properties:<br />
<br />
<source lang="xml"><br />
<properties><br />
<distroDirectory>${project.basedir}/distro</distroDirectory><br />
<configDirectory>${project.basedir}/config</configDirectory><br />
</properties><br />
</source><br />
<br />
Add the plugin for the generation of the SA, which has to be included under the build plugins section:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<version>2.2</version><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
In the case of a GWT widget the [http://mojo.codehaus.org/gwt-maven-plugin gwt-maven-plugin] should also be configured ( within build plugins section as well), by also selecting the proper gwt version ( in our example 2.4)<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.codehaus.mojo</groupId><br />
<artifactId>gwt-maven-plugin</artifactId><br />
<version>${gwt.version}</version><br />
<executions><br />
<execution><br />
<goals><br />
<goal>compile</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
The proper GWT version should be added to the pom properties :<br />
<br />
<source lang="xml"><br />
<gwt.version>2.4.0</gwt.version><br />
</source><br />
<br />
and GWT dependencies should be also configured:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-user</artifactId><br />
<version>${gwt.version}</version><br />
</dependency><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-servlet</artifactId><br />
<version>${gwt.version}</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/mvn-widget-sample/ project sources]] for further details on the project configuration.<br />
<br />
== Generating Artifacts ==<br />
<br />
The <code>sample-portlet</code> uses the '''maven-compiler-plugin''' to compile the portlet code and the '''gwt-maven-plugin''' as well for GWT compilation.<br />
<br />
The generation of a war package can be triggered by typing :<br />
<br />
<pre><br />
mvn package <br />
</pre><br />
<br />
on your shell or running this build task within Eclipse.<br />
<br />
In both cases the war package will be generated under the ''target'' folder, e.g.:<br />
<br />
<pre><br />
target/sample-admin-portlet-1.0.0-SNAPSHOT.war<br />
</pre><br />
<br />
The generation of all project artifacts ( war, source jar, javadoc, SA) can be triggered by the build task :<br />
<br />
<pre><br />
mvn install <br />
</pre><br />
<br />
this will install as well all artifacts in your local repository, while the deployment of the artifacts to the remote gCube Maven repos can triggered by the build task:<br />
<br />
<pre><br />
mvn deploy <br />
</pre><br />
<br />
== Widget dependencies ==<br />
<br />
The system dependencies ( the one contained within the [[GCube_Portal_Installation#Portal_Bundle_Libraries|Portal bundle]] ) should not be included inside the portlet war. As said in this case the ''provided'' scope should be specified, e.g.<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portal</groupId><br />
<artifactId>custom-portal-handler</artifactId><br />
<version>1.1.0-SNAPSHOT</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.4.0,1.5.0]</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the widget is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>PortletAdmin</Class><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.portles.admin</groupId><br />
<artifactId>sample-admin-portlet</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/sample-admin-portlet-1.0.0-SNAPSHOT.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code> of the project;<br />
* package versions are aligned with the Maven <code>version</code> of the project;<br />
* package include <code>MavenCoordinates</code> that can be directly copied and pasted from the POM of the project;<br />
* packages do '''not''' specify dependencies;<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>sample-portlet</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>PortletAdmin</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/${build.finalName}.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
== Service Archive ==<br />
<br />
<code>sample-portlet</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the project.<br />
<br />
As shown for [[#Libraries|libraries]], the portlet uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources]] for details.</div>Gabriele.giammatteohttps://gcube.wiki.gcube-system.org/index.php?title=Developing_gCube_Maven_Components&diff=29031Developing gCube Maven Components2017-05-04T07:10:06Z<p>Gabriele.giammatteo: /* Java Version */</p>
<hr />
<div>[[Category: Developer's Guide]]<br />
__TOC__<br />
<br />
= Common =<br />
== Maven Coordinates ==<br />
Maven components' coordinates must follow these conventions:<br />
<br />
* their <code>groupId</code> must be of the following form:<br />
<br />
<pre>org.gcube.<class></pre><br />
<br />
: where ''class'' identifies a functionally correlated set of components and helps a range of users (developers, administrator) to discover components in various UIs/reports used within the system and in tools. Logically, the class corresponds to the gCube class of the component's, as specified in its gCube Software Profile. However, more specific, or else cross-class groups may be agreed upon among developers. In all cases, the group should align with official Maven [http://maven.apache.org/guides/mini/guide-naming-conventions.html guidelines]. By agreement, for example, components in gCube class <code>DataAccess</code> declare a <code>groupId</code> of <code>org.gcube.data.access</code> and those in gCube class <code>DataTransfer</code> declare a <code>groupId</code> of <code>org.gcube.data.transfer</code>. On the other hand, also by agreement, components in gCube class <code>InformationSystem</code> declare a <code>groupId</code> of <code>org.gcube.informationsystem</code>.<br />
<br />
* their <code>artifactId</code> must follow Maven conventions (e.g. ''avoid capital letters or punctuation other than hyphens'').<br />
<br />
== Parent ==<br />
All gCube components must inherit from '''maven-parent'''. This ensure and promotes compliance with project-wide requirements, from the enforcement of minimal Java and Maven versions to generation and packaging of Javadoc documentation and component sources.<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
<br />
== The distro folder ==<br />
This folder contains files that will be distributed along with the code in the distribution packages. Further details are in the [[Maven distro directory layout]] page.<br />
<br />
== Service Archive (SA) ==<br />
Whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary <code>tar.gz</code> artefact with classifier <code>servicearchive</code>. Note that the SA of Maven components does ''not'' need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);<br />
<br />
* for each deployable artefact they produce (primary or secondary), they must include the Maven coordinates of the arteact in the corresponding package section of the gCube Software Profile. Specifying package dependencies in the Software Profile, on the other hand, is no longer required;<br />
<br />
* whenever they are associated with a gCube Software Archive (SA), they must produce it as a secondary <code>tar.gz</code> artefact with classifier <code>servicearchive</code>. Note that the SA of Maven components does ''not'' need to include Javadoc documentation (as this documentation is produced and published directly as a Maven artefact);<br />
<br />
== Dependencies ==<br />
Service and libraries that depend on other gCube components (Maven-based or Ant-based) can find them in the gCube Maven Repositories: <code>gcube-snapshots</code> <strike>and <code>gcube-releases</code></strike>. <br />
<br />
=== gCube Maven Repositories ===<br />
As a one-off task, developers must configure these repositories in the <code>~/.m2/settings.xml</code> file of their local Maven installation, as follows:<br />
<br />
<source lang="xml"><br />
<settings xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"<br />
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd"><br />
<br />
<servers><br />
<server><br />
<id>gcube-snapshots</id><br />
<username>gcube-user</username><br />
<password>maven</password><br />
</server><br />
</servers><br />
<br />
<br />
<profiles><br />
<profile><br />
<id>gcube</id><br />
<repositories><br />
<repository><br />
<id>gcube-snapshots</id><br />
<name>gCube Snapshots</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
</snapshots><br />
</repository><br />
<repository><br />
<id>gcube-externals</id><br />
<name>gCube Externals</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals</url><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
</repository><br />
</repositories><br />
<br />
<pluginRepositories><br />
<pluginRepository><br />
<id>gcube-snapshots</id><br />
<name>gCube Snapshots</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-snapshots</url><br />
<releases><br />
<enabled>false</enabled><br />
</releases><br />
<snapshots><br />
<enabled>true</enabled><br />
</snapshots><br />
</pluginRepository><br />
<pluginRepository><br />
<id>gcube-externals</id><br />
<name>gCube Externals</name><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-externals</url><br />
<snapshots><br />
<enabled>false</enabled><br />
</snapshots><br />
<releases><br />
<enabled>true</enabled><br />
</releases><br />
</pluginRepository><br />
</pluginRepositories><br />
<br />
</profile><br />
</profiles><br />
<br />
<activeProfiles><br />
<activeProfile>gcube</activeProfile><br />
</activeProfiles><br />
</settings><br />
</source><br />
<br />
<br />
'''Note:Developers must not configure these repositories in their POMs''', only in <code>settings.xml</code>. <br />
<br />
This is to:<br />
* keep their projects independent from future changes in repository locations; <br />
* avoid interference with ETICS builds, where access to these repositories may comprise correct system integration;<br />
<br />
With this settings.xml, developers will have access on their local development environment to development versions (gcube-snapshots) and externals (gcube-externals). This is the correct behaviour for most of use cases. If access to staging or releases repositories is necessary (e.g. patching a component already released in the branch where dependencies versions in the release are not in snapshots), developer should modify his settings.xml to point to gcube-staging(releases) instead of gcube-snapshots.<br />
<br />
=== Third-party Repositories ===<br />
Using the the <code>settings.xml</code> aforementioned, any dependency declared in pom.xml that can be found in one of gCube repositories or in Maven Central will be resolved at build time and deployment time without any further configuration. Frequently it may happens that a gCube component needs software stored on a public Maven repository not in on of those repositories. In this case, Maven provides an easy solution for declaring additional repositories for dependency resolution by adding a <repositories> section in the pom.xml of the project. E.g.:<br />
<br />
<source lang="xml" highlight="3,4"><br />
<repositories><br />
<repository><br />
<id>[repository-id]</id><br />
<url>[repository URL]</url><br />
</repository><br />
</repositories><br />
</source><br />
<br />
Though technically possible to add any Maven repository in this way, in order to preserve the quality of gCube software it is required to check trustworthiness of its dependencies disallowing usage of certain external software. To enforce such a control over public Maven repositories, '''gCube components are allowed to use external repository not directly but only through a proxy repository''' (a.k.a. shadow repository) created on CNR's Nexus Repository Portal (http://maven.research-infrastructures.eu/nexus/).<br />
<br />
List of all repositories in Nexus Portal is available at http://maven.research-infrastructures.eu/nexus/index.html#view-repositories<br />
<br />
'''Note''': if a component uses a public Maven repository directly, it will work for building but would fail in deployment phase. In fact, Software Gateway is only able to download software accessible from Nexus.<br />
<br />
==== Request new proxy repository in Nexus ====<br />
Proxy Repositories on Nexus are created by the Release Manager. To request a new proxy a new ticket must be created.<br />
<br />
Details of ticket are:<br />
# open a ticket in tracker "Support" in gCube ticketing system (at https://support.d4science.org/projects/gcube/)<br />
# ticket's '''description''' must report:<br />
#* URL of repository to add<br />
#* organization which hosts the repository<br />
#* which artifacts are needed from that repository<br />
#* which components needs those artifacts<br />
# '''Sprint''' field should be set to the gCube release in which the component is expected to be released <br />
# '''Category''' field must report the gCube subsystem that needs the external repository<br />
# '''Assignee''' is the Release Manager<br />
<br />
An example of ticket for requesting a new repository mirror is reported in the figure below.<br />
[[Image:New_maven_mirror_repo.png|600px|center|Ticket for requesting a new Mirror Repository]]<br />
<br />
Release Manager will process the ticket checking that no issues are foreseen from the usage of software stored in the repository. If request is accepted, a new proxy repository is created and the repository's URL will be communicated publicly on gCube's developers mailinglist (a.k.a. TCOM mailinglist).<br />
<br />
Knowing the mirror repository URL, developer can use it adding following lines in pom.xml. E.g.<br />
<br />
<source lang="xml" highlight="3,4"><br />
<repositories><br />
<repository><br />
<id>gcube-ext-osgeo</id><br />
<url>http://maven.research-infrastructures.eu/nexus/content/repositories/osgeo/</url><br />
</repository><br />
</repositories><br />
</source><br />
<br />
'''Note''': it is possible that software stored on proxy repositories is not included in search results on Nexus. This happens for those repositories that does not have indexes (or have indexes in a format that Nexus is not able to download). Anyway, once Nexus has downloaded artifacts from the original repository for the first time (e.g. because a compilation of a pom that uses the proxy repository), creates its own indexes and those artifacts will become available also for local search.<br />
<br />
=== Dependency Versions ===<br />
<br />
ETICS builds ensure correct integration of gCube components, regardless of the versions their dependencies. It also ensures that components are released with fixed versions for their dependencies, i.e. ensures build reproducibility.<br />
<br />
Accordingly, developers can be liberal in the choice of dependency versions. i.e. rely on version ranges to:<br />
<br />
* minimise the requirements for version management;<br />
* be readily informed of any mis-alignment with their dependencies;<br />
<br />
As we start adopting Maven, the lower-bound of version ranges will be a SNAPSHOT version. For example, a dependency on the <code>streams</code> library can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>streams</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
</dependency><br />
</source><br />
<br />
With this dependency, a component will be soon notified of a non-retro compatible change that, contrary to conventions, <code>streams</code> may have introduced in, say, version <code>1.3.0</code> (typically the day after at the latest). The notification will be delivered in the IDE, assuming the availability of Maven integration (e.g. <code>m2e</code> in Eclipse). <br />
The component is in fact free to open the range further if it wants to be aligned with its dependencies across major versions.<br />
<br />
=== Common Dependencies ===<br />
<br />
Many gCube components depend on <code>gcf</code>, which is now available as a Maven component since version <code>1.4</code>. At the time of writing, the component has not been released yet. A dependency on it is then specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
The scope <code>provided</code> indicates that these dependency will be satisfied in the development environment (the gHN) and should not be included in GAR and SA archives.<br />
<br />
Some components depend on the libraries available in all gHNs, though they do not depend on <code>gcf</code> itself. The subset of these libraries that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called <code>ghn-core-runtime</code>. A dependency on this Jar can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ghn-core-runtime</artifactId><br />
<version>1.0.0</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Note: a component that depends on <code>gcf</code> indirectly depends on <code>gun-core-runtime</code>.<br />
<br />
Another common dependency is on the gCube gRS2, an Ant component that the build infrastructure make available in Maven repositories. A dependency on this component can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.execution</groupId><br />
<artifactId>grs2library</artifactId><br />
<version>[1.0.0-SNAPSHOT,2.0.0-SNAPSHOT)</version><br />
</dependency><br />
</source><br />
<br />
== Java Version ==<br />
<br />
In Maven the Java version of the source code and of the generated classes can be controlled using the configuration of the <code>maven-compiler-plugin</code><br />
<syntaxhighlight lang="xml" highlight="5,6"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-compiler-plugin</artifactId><br />
<configuration><br />
<source>1.7</source><br />
<target>1.8</target><br />
</configuration><br />
</plugin><br />
</syntaxhighlight><br />
<br />
<br />
In gCube, this mechanism is exploited to enforce the usage of the same Java version for all components. In particular, the maven-parent defines the property <code>java_version</code> and use this property in the configuration of the <code>maven-compiler-plugin</code>. '''All the gCube components inherit this configuration and it is not recommended to override it'''.<br />
<br />
<syntaxhighlight lang="xml" highlight="13,14"><br />
...<br />
<properties><br />
<java_version>1.8</java_version><br />
...<br />
</properties><br />
<br />
...<br />
<br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-compiler-plugin</artifactId><br />
<configuration><br />
<source>${java_version}</source><br />
<target>${java_version}</target><br />
</configuration><br />
</plugin><br />
...<br />
</syntaxhighlight><br />
<br />
<br />
Using the property instead of specify the version in the configuration, we are more flexible because the property can be overridden at various levels (settings, profile, pom, commandline) and allow to customize the java version to use depending on different factors (e.g. during ETICS builds gCore components are built with Java 7, all the others with Java 8).<br />
<br />
Current value for the <code>java_version</code> property is 1.8.<br />
<br />
= Libraries =<br />
The <code>streams</code> library is a Maven component in gCube class <code>DataAccess</code>. We will use it as example of a library component in Maven.<br />
<br />
=== Profile ===<br />
<br />
The gCube Software profile of the library is as follows:<br />
<br />
<source lang="xml" highlight="14-18"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>Embedded Domain-Specific Language for Stream Transformations</Description><br />
<Class>DataAccess</Class><br />
<Name>streams</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>streams</Name><br />
<Version>1.0.0-SNAPSHOT</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>streams</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<Files><br />
<File>streams-1.0.0-SNAPSHOT.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the profile includes a single package and this package corresponds to the main build artefact of the component; <br />
* the whole profile and the package have usual gCube coordinates;<br />
* the package name is aligned with the Maven <code>artifactId</code> of the component;<br />
* the package version is aligned with the Maven <code>version</code> of the component;<br />
* the package includes <code>MavenCoordinates</code> that can be directly copied and pasted from the POM;<br />
* the package does '''not''' specify dependencies;<br />
* the package points to the main artefact of the Maven build;<br />
<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>streams</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>DataAccess</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<Files><br />
<File>${build.finalName}.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
=== Service Archive ===<br />
<br />
<code>streams</code> is responsible for generating its own SA as part of its Maven build. This requires dedicated logic in the POM but has the advantage that:<br />
<br />
* SA validity can be verified locally;<br />
* no SA configurations are required in ETICS;<br />
* the required build logic can be easily reused across components;<br />
<br />
<code>streams</code> uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation in the profile and other static files. The approach is best illustrated with a reference to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/streams sources]] of the <code>streams</code> library.<br />
<br />
<br />
As we discuss [[#Creating the ETICS Configurations|below]], <code>streams</code> has the final requirement to produce an interpolated profile during ETICS builds, outside the context of the SA.<br />
<code>streams</code> meets this requirement with the [http://maven.apache.org/plugins/maven-resources-plugin/ Maven Resources Plugin]. Again, consult the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/streams sources]] of the <code>streams</code> library to reuse this approach.<br />
<br />
= Services =<br />
The <code>tree-manager</code> service is a Maven component in gCube class <code>DataAccess</code>. <br />
<br />
== Project Structure ==<br />
<br />
Differently from a library, the service is associated with a number of distinguished build artefacts:<br />
<br />
* the service implementation (a Jar);<br />
* the stubs automatically generated from the WSDL interfaces (a Jar);<br />
* the archive required for deployment in gCore (a Gar);<br />
* the archive required for registration in the system (a SA);<br />
<br />
Producing these artefacts requires sharing a number of static and dynamic files (from profiles to WSDLs to pre-processed WSDLs). The usual Maven approach of mapping the artefacts onto different Maven projects is then genuinely difficult. It is instead more practical to use a single multi-module Maven project. The <code>tree-manager</code> uses this module structure:<br />
<br />
<pre><br />
tree-manager (parent module)------- (produces) --- (POM only)<br />
|<br />
|----- tree-manager-service (service impl module)------- (produces) --- jar, gar, SA<br />
| |<br />
| | (depends on)<br />
| |<br />
| \/<br />
|----- tree-manager-stubs (service stubs module)------- (produces) --- jar<br />
</pre><br />
<br />
As customary, the parent does not produce any artefact (is a POM-only project). It declares these coordinates in its POM:<br />
<br />
<source lang="xml"><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
<packaging>pom</packaging><br />
</source><br />
<br />
The parent physically includes the sub-modules;<br />
<br />
<source lang="xml"><br />
<modules><br />
<module>tree-manager-stubs</module><br />
<module>tree-manager-service</module><br />
</modules><br />
</source> <br />
<br />
Finally, the parent inherits from '''maven-parent''':<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
The submodules inherit from the parent:<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>tree-manager</artifactId><br />
<groupId>org.gcube.data.access</groupId><br />
<version>1.0.0-SNAPSHOT</version><br />
<relativePath>..</relativePath><br />
</parent><br />
</source><br />
<br />
and share its <code>version</code> and <code>groupId</code>. Their coordinates are then minimal:<br />
<br />
<source lang="xml"><br />
<artifactId>tree-manager-service</artifactId><br />
</source><br />
<br />
<source lang="xml"><br />
<artifactId>tree-manager-stubs</artifactId><br />
</source><br />
<br />
The <code>tree-manager-service</code> module produces a Jar of its own implementation, as well the GAR and the SA of the whole services as secondary artefacts with classifiers of, respectively, <code>gar</code> and <code>service archive</code>.<br />
<br />
The <code>tree-manager-stubs</code> module produces a Jar of its own implementation.<br />
<br />
Finally, the parent module coordinates the build of its sub-modules and centralises the declaration of common dependencies and configuration. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] to reuse this approach.<br />
<br />
== Generating Stubs and Gars ==<br />
<br />
The <code>tree-manager-stubs</code> uses the '''gCube Service Plugin''' to generate its classes from WSDL interfaces. <br />
<br />
Similarly, the <code>tree-manager-service</code> uses the plugin to generate its secondary GAR artefact. <br />
<br />
The configuration of the plugin is conveniently shared in the parent module.<br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] of the <code>streams</code> for details.<br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the service is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>DataAccess</Class><br />
<Name>tree-manager-service</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Main><br />
<Name>tree-manager-service</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager-service</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<GARArchive>tree-manager-service-1.0.0-SNAPSHOT.gar</GARArchive><br />
<PortType><br />
<Name>gcube/data/tm/binder</Name><br />
</PortType><br />
<PortType><br />
<Name>gcube/data/tm/reader</Name><br />
</PortType><br />
<PortType><br />
<Name>gcube/data/tm/writer</Name><br />
</PortType><br />
</Main><br />
<Software><br />
<Name>tree-manager-stubs</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.data.access</groupId><br />
<artifactId>tree-manager-stubs</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<Type>library</Type><br />
<Files><br />
<File>tree-manager-stubs-1.0.0-SNAPSHOT.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the profile includes two packages and this package corresponds to the main build artefacts of the submodules; <br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code>s of the submodules;<br />
* package versions are aligned with the Maven <code>version</code>s of the submodules;<br />
* package include <code>MavenCoordinates</code>s that can be directly copied and pasted from the POM of the submodules;<br />
* packages do '''not''' specify dependencies;<br />
* packages points to the main artefact of the Maven build of the submodules;<br />
<br />
<strike>Notice that, unlike libraries, services cannot easily use Maven variables in their profiles (as these artefacts are used for different purposes by different modules). Accordingly, developers needs to maintain versions used in profiles across releases. In particular, they need to:<br />
<br />
* replace SNAPSHOT versions with release versions in release branches;<br />
* advance SNAPSHOT versions in HEAD;</strike><br />
<br />
=== Profile Interpolation ===<br />
The interpolation of profiles is required in order to keep information in profile in sync with <code>pom.xml</code>, in particular versions and filenames (that contain versions) that change at each release.<br />
<br />
In order to generate <code>profile.xml</code> during build process you need to:<br />
# add in the service's parent <code>pom.xml</code> the invocation to maven-resources-plugin<br />
# place a ''template'' <code>profile.xml</code> in the <code>${distroDirectory}</code><br />
<br />
Invocation to <code>maven-resources-plugin</code> looks like the following (to be placed in the <plugins> section inside the <build> section of the pom):<br />
<source lang="xml"><br />
<!-- interpolates profiles and copies from distribution location to configuration <br />
location, where it is need for embedding into stub artifact, gar generation, <br />
and service archive. --><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-resources-plugin</artifactId><br />
<version>2.6</version><br />
<executions><br />
<execution><br />
<id>copy-profile</id><br />
<goals><br />
<goal>copy-resources</goal><br />
</goals><br />
<phase>process-resources</phase><br />
<configuration><br />
<outputDirectory>${configDirectory}</outputDirectory><br />
<resources><br />
<resource><br />
<directory>${distroDirectory}</directory><br />
<includes><br />
<include>profile.xml</include><br />
</includes><br />
<filtering>true</filtering><br />
</resource><br />
</resources><br />
</configuration><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
The ''template'' <code>profile.xml</code> looks like the following:<br />
<source lang="xml" highlight="12,16,18,22,26,30"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>TestClass</Class><br />
<Name>TestService</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Main><br />
<Name>TestService</Name><br />
<Version>${project.version}</Version><br />
<MavenCoordinates><br />
<groupId>org.example</groupId><br />
<artifactId>test-service</artifactId><br />
<version>${project.version}</version><br />
</MavenCoordinates><br />
<GARArchive>>test-service-${project.version}.gar</GARArchive><br />
</Main><br />
<Software><br />
<Name>test-stubs</Name><br />
<Version>${project.version}</Version><br />
<MavenCoordinates><br />
<groupId>org.example</groupId><br />
<artifactId>test-stubs</artifactId><br />
<version>${project.version}</version><br />
</MavenCoordinates><br />
<Type>library</Type><br />
<Files><br />
<File>test-stubs-${project.version}.jar</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
In the example, property <code>${project.version}</code> will be substituted with the actual service's version at build-time. It is also possible, but not required, to parametrize also other values of <code>profile.xml</code> such as maven's coordinates and package names. You can see a more complex <code>profile.xml</code> ''template'' in the <code>tree-manager</code> project: http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-access/tree-manager/distro/profile.xml<br />
<br />
Maven will copy the final <code>profile.xml</code> in <code>${configDirectory}</code> during the <code>process-resources</code> phase of the build. From there it will be copied in the service-archive package.<br />
<br />
== Service Archive ==<br />
<br />
<code>tree-manager-service</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the module and its parent.<br />
<br />
As shown for [[#Libraries|libraries]], the service uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/data-access/tree-manager project sources]] for details.<br />
<br />
= Portlets =<br />
<br />
The <code>sample-portlet</code> portlet is a Maven based sample portlet based on GWT. Refer to the [http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources] for details.<br />
<br />
== Project Structure ==<br />
<br />
As for a library project, the portlet is associated with 2 distinguished build artefacts:<br />
<br />
* the archive required for deployment in a Liferay Container (a WAR);<br />
* the archive required for registration in the system (a SA);<br />
<br />
In order to generate automatically the project structure and the base pom, we start by customizing the maven archetype liferay-portlet-archetype v. 6.0.6.<br />
<br />
The generation of the project structure and the pom can be performed either from Eclipse ( by using the m2eclipse plugin) or by command line using the Maven SDK.<br />
<br />
On Eclipse the following steps must be followed:<br />
<br />
* FIle -> New -> Project<br />
* From the wizard select Maven->Maven Project<br />
* From the Select Archetype step, Select 'All Catalogs' from the Catalogs combobox and type 'portlet' on the Filter field.<br />
* Select the '''liferay-portlet-archetype''' version 6.0.6 archetype ( if does not appear on the list just unselect 'show the last version of the Archetype only")<br />
* In the next step you have to provide Maven coordinates for your project<br />
* groupid : <your group id> e.g org.gcube.portlets.admin<br />
* artifactId : <your artifactId> e.g. sample-admin-portlet<br />
* version : <your project version> e.g. 1.0.0-SNAPSHOT ( use always SNAPSHOT versions for development versions of a project)<br />
* package: <your portlet code package> e.g. org.gcube.portlets.admin.sample<br />
<br />
while from shell the following mvn command should be issued :<br />
<br />
<pre><br />
mvn archetype:generate \<br />
-DarchetypeArtifactId=liferay-theme-archetype \<br />
-DarchetypeGroupId=com.liferay.maven.archetypes \<br />
-DarchetypeVersion=6.0.6 \<br />
-DartifactId=<your artifactId> \<br />
-DgroupId=<your group id> \<br />
-Dversion=1.0.0-SNAPSHOT<br />
</pre><br />
<br />
Both approaches generate a project structure similar to the picture below:<br />
<br />
[[image:sample-portlet-project.png|Sample portlet project structure]]<br />
<br />
In order to customize the build procedure to the gCube needs we should open the pom.xml and apply the following changes:<br />
<br />
Configure the inheritance from '''maven-parent''' :<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
Configure some project properties:<br />
<br />
<source lang="xml"><br />
<properties><br />
<liferay.version>6.0.6</liferay.version><br />
<liferay.auto.deploy.dir>${system.CATALINA_HOME}/../deploy</liferay.auto.deploy.dir><br />
<distroDirectory>${project.basedir}/distro</distroDirectory><br />
<configDirectory>${project.basedir}/config</configDirectory><br />
</properties><br />
</source><br />
<br />
Add the plugin for the generation of the SA, which has to be included under the build plugins section:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<version>2.2</version><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
In the case of a GWT portlet the [http://mojo.codehaus.org/gwt-maven-plugin gwt-maven-plugin] should also be configured ( within build plugins section as well), by also selecting the proper gwt version ( in our example 2.4)<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.codehaus.mojo</groupId><br />
<artifactId>gwt-maven-plugin</artifactId><br />
<version>${gwt.version}</version><br />
<executions><br />
<execution><br />
<goals><br />
<goal>compile</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
The proper GWT version should be added to the pom properties :<br />
<br />
<source lang="xml"><br />
<gwt.version>2.4.0</gwt.version><br />
</source><br />
<br />
and GWT dependencies should be also configured:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-user</artifactId><br />
<version>${gwt.version}</version><br />
</dependency><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-servlet</artifactId><br />
<version>${gwt.version}</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet/ project sources]] for further details on the project configuration.<br />
<br />
== Generating Artifacts ==<br />
<br />
The <code>sample-portlet</code> uses the '''maven-compiler-plugin''' to compile the portlet code and the '''gwt-maven-plugin''' as well for GWT compilation.<br />
<br />
The generation of a war package can be triggered by typing :<br />
<br />
<pre><br />
mvn package <br />
</pre><br />
<br />
on your shell or running this build task within Eclipse.<br />
<br />
In both cases the war package will be generated under the ''target'' folder, e.g.:<br />
<br />
<pre><br />
target/sample-admin-portlet-1.0.0-SNAPSHOT.war<br />
</pre><br />
<br />
The generation of all project artifacts ( war, source jar, javadoc, SA) can be triggered by the build task :<br />
<br />
<pre><br />
mvn install <br />
</pre><br />
<br />
this will install as well all artifacts in your local repository, while the deployment of the artifacts to the remote gCube Maven repos can triggered by the build task:<br />
<br />
<pre><br />
mvn deploy <br />
</pre><br />
<br />
== Portlet dependencies ==<br />
<br />
Specific portlet dependencies ( e.g. GWT, service stubs ) should be packaged within the portlet war. All dependencies specified within the pom file without the ''provided'' scope will be copied inside the WEB-INF/lib folder at packaging time.<br />
<br />
For this reason all portlet specific dependencies should not specified that scope e.g:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.externals</groupId><br />
<artifactId>gwt-ext-patched</artifactId><br />
<version>2.0.5</version><br />
</dependency><br />
</source><br />
<br />
On the other hand the system dependencies ( the one contained within the [[GCube_Portal_Installation#Portal_Bundle_Libraries|Portal bundle]] ) should not be included inside the portlet war. As said in this case the ''provided'' scope should be specified, e.g.<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portal</groupId><br />
<artifactId>custom-portal-handler</artifactId><br />
<version>1.1.0-SNAPSHOT</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.4.0,1.5.0]</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
<br />
A subset of these libraries ( contained in a GHN) that are required for compilation and integration testing have been coalesced in a single Jar, distributed as a Maven component called <code>ghn-core-runtime</code>. A dependency on this Jar can be specified as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ghn-core-runtime</artifactId><br />
<version>1.0.0</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Note: a component that depends on <code>gcf</code> indirectly depends on <code>gun-core-runtime</code>.<br />
<br />
== Include gCube Guided Tour using Maven ==<br />
<br />
This particular component is not yet ported to maven so in order to be included in a maven based project it should be declared as system dependency as follows:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portletsuser</groupId><br />
<artifactId>gcube-guidedtour-widget</artifactId><br />
<version>1.0.0</version><br />
<scope>system</scope><br />
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/gcube-guidedtour-widget.jar</systemPath> <br />
</dependency><br />
</source><br />
<br />
together with the gCube Widget Library:<br />
<br />
<source lang="xml"> <br />
<dependency><br />
<groupId>org.gcube.portlets.user</groupId><br />
<artifactId>gcubewidgets</artifactId><br />
<version>1.2.0</version><br />
<scope>system</scope><br />
<systemPath>${basedir}/src/main/webapp/WEB-INF/lib/gCube-Widgets-Library.jar</systemPath><br />
</dependency><br />
</source><br />
<br />
both components can be downloaded from the distribution repository [[http://grids16.eng.it:8080/BuildReport/browse/Recent_Builds]]. <br />
<br />
In case the portlet depends on the ''gcf'' component, some dependency resolution changes are required to your portlet in order to properly compile ( because of libraries clashes)<br />
<br />
The ghn-core-runtime component ( gcf dependency) should be excluded:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.5.1-SNAPSHOT,2.0.0)</version><br />
<scope>provided</scope><br />
<exclusions><br />
<exclusion><br />
<artifactId>ghn-core-runtime</artifactId><br />
<groupId>org.gcube.distribution</groupId><br />
</exclusion><br />
</exclusions><br />
</dependency><br />
</source><br />
<br />
<br />
and some new dependencies should be added:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>xerces</groupId><br />
<artifactId>xercesImpl</artifactId><br />
<version>2.9.1</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>ws-addressing</artifactId><br />
<version>1.0</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.distribution</groupId><br />
<artifactId>axis-patched</artifactId><br />
<version>1.2RC</version><br />
<scope>provided</scope><br />
</dependency><br />
<br />
</source><br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the portlet is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>PortletAdmin</Class><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.portles.admin</groupId><br />
<artifactId>sample-admin-portlet</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/sample-admin-portlet-1.0.0-SNAPSHOT.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code> of the project;<br />
* package versions are aligned with the Maven <code>version</code> of the project;<br />
* package include <code>MavenCoordinates</code> that can be directly copied and pasted from the POM of the project;<br />
* packages do '''not''' specify dependencies;<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>sample-portlet</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>PortletAdmin</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/${build.finalName}.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
== Service Archive ==<br />
<br />
<code>sample-portlet</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the project.<br />
<br />
As shown for [[#Libraries|libraries]], the portlet uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources]] for details.<br />
<br />
== Use Case: Create a new Mavenized GWT Portlet ==<br />
<br />
In order to create a new Mavenized GWT Portlet this detailed guide can be also followed [https://gcube.wiki.gcube-system.org/gcube/index.php/Create_a_new_Mavenized_gCube_GWT_Portlet_Guide]<br />
<br />
= Widgets =<br />
<br />
Falls in this category of components mainly GWT widgets or so called GWT Libraries, which can be used to package classes for mutualization and/or modularization. A Widget is a java archive (JAR) containing both classes and java sources for later GWT compilation and a gwt.xml module descriptor. <br />
<br />
In this section we refer to a sample Widget with maven structure (https://svn.research-infrastructures.eu/d4science/gcube/trunk/portal/mvn-widget-sample/)<br />
<br />
== Project Structure ==<br />
<br />
As for a other project, the portlet is associated with 2 distinguished build artefacts:<br />
<br />
* the archive required for deployment (a JAR);<br />
* the archive required for registration in the system (a SA);<br />
<br />
In order to generate automatically the project structure and the base pom, we start by customizing the maven archetype liferay-portlet-archetype v. 6.0.6.<br />
<br />
The generation of the project structure and the pom can be performed either from Eclipse ( by using the m2eclipse plugin) or by command line using the Maven SDK.<br />
<br />
On Eclipse the following steps must be followed:<br />
<br />
* FIle -> New -> Project<br />
* From the wizard select Maven->Maven Project<br />
* From the Select Archetype step, Select 'All Catalogs' from the Catalogs combobox and type 'portlet' on the Filter field.<br />
* Select the '''maven-archetype-quickstart''' version 1.1.0 archetype ( if does not appear on the list just unselect 'show the last version of the Archetype only")<br />
* In the next step you have to provide Maven coordinates for your project<br />
* groupid : <your group id> e.g org.gcube.portlets.admin<br />
* artifactId : <your artifactId> e.g. sample-admin-portlet<br />
* version : <your project version> e.g. 1.0.0-SNAPSHOT ( use always SNAPSHOT versions for development versions of a project)<br />
* package: <your portlet code package> e.g. org.gcube.portlets.admin.sample<br />
<br />
while from shell the following mvn command should be issued :<br />
<br />
<pre><br />
mvn archetype:generate \<br />
-DarchetypeArtifactId=maven-archetype-quickstart \<br />
-DarchetypeGroupId=com.liferay.maven.archetypes \<br />
-DarchetypeVersion=1.1.0 \<br />
-DartifactId=<your artifactId> \<br />
-DgroupId=<your group id> \<br />
-Dversion=1.0.0-SNAPSHOT<br />
</pre><br />
<br />
Both approaches generate a project structure similar to the picture below:<br />
<br />
[[image:sample-widget-project.png|Sample widget project structure]]<br />
<br />
In order to customize the build procedure to the gCube needs we should open the pom.xml and apply the following changes:<br />
<br />
Configure the inheritance from '''maven-parent''' :<br />
<br />
<source lang="xml"><br />
<parent><br />
<artifactId>maven-parent</artifactId><br />
<groupId>org.gcube.tools</groupId><br />
<version>1.0.0</version><br />
<relativePath /><br />
</parent><br />
</source><br />
<br />
Configure some project properties:<br />
<br />
<source lang="xml"><br />
<properties><br />
<distroDirectory>${project.basedir}/distro</distroDirectory><br />
<configDirectory>${project.basedir}/config</configDirectory><br />
</properties><br />
</source><br />
<br />
Add the plugin for the generation of the SA, which has to be included under the build plugins section:<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.apache.maven.plugins</groupId><br />
<artifactId>maven-assembly-plugin</artifactId><br />
<version>2.2</version><br />
<configuration><br />
<descriptors><br />
<descriptor>${distroDirectory}/descriptor.xml</descriptor><br />
</descriptors><br />
</configuration><br />
<executions><br />
<execution><br />
<id>servicearchive</id><br />
<phase>install</phase><br />
<goals><br />
<goal>single</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
In the case of a GWT widget the [http://mojo.codehaus.org/gwt-maven-plugin gwt-maven-plugin] should also be configured ( within build plugins section as well), by also selecting the proper gwt version ( in our example 2.4)<br />
<br />
<source lang="xml"><br />
<plugin><br />
<groupId>org.codehaus.mojo</groupId><br />
<artifactId>gwt-maven-plugin</artifactId><br />
<version>${gwt.version}</version><br />
<executions><br />
<execution><br />
<goals><br />
<goal>compile</goal><br />
</goals><br />
</execution><br />
</executions><br />
</plugin><br />
</source><br />
<br />
<br />
The proper GWT version should be added to the pom properties :<br />
<br />
<source lang="xml"><br />
<gwt.version>2.4.0</gwt.version><br />
</source><br />
<br />
and GWT dependencies should be also configured:<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-user</artifactId><br />
<version>${gwt.version}</version><br />
</dependency><br />
<dependency><br />
<groupId>com.google.gwt</groupId><br />
<artifactId>gwt-servlet</artifactId><br />
<version>${gwt.version}</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/mvn-widget-sample/ project sources]] for further details on the project configuration.<br />
<br />
== Generating Artifacts ==<br />
<br />
The <code>sample-portlet</code> uses the '''maven-compiler-plugin''' to compile the portlet code and the '''gwt-maven-plugin''' as well for GWT compilation.<br />
<br />
The generation of a war package can be triggered by typing :<br />
<br />
<pre><br />
mvn package <br />
</pre><br />
<br />
on your shell or running this build task within Eclipse.<br />
<br />
In both cases the war package will be generated under the ''target'' folder, e.g.:<br />
<br />
<pre><br />
target/sample-admin-portlet-1.0.0-SNAPSHOT.war<br />
</pre><br />
<br />
The generation of all project artifacts ( war, source jar, javadoc, SA) can be triggered by the build task :<br />
<br />
<pre><br />
mvn install <br />
</pre><br />
<br />
this will install as well all artifacts in your local repository, while the deployment of the artifacts to the remote gCube Maven repos can triggered by the build task:<br />
<br />
<pre><br />
mvn deploy <br />
</pre><br />
<br />
== Widget dependencies ==<br />
<br />
The system dependencies ( the one contained within the [[GCube_Portal_Installation#Portal_Bundle_Libraries|Portal bundle]] ) should not be included inside the portlet war. As said in this case the ''provided'' scope should be specified, e.g.<br />
<br />
<source lang="xml"><br />
<dependency><br />
<groupId>org.gcube.portal</groupId><br />
<artifactId>custom-portal-handler</artifactId><br />
<version>1.1.0-SNAPSHOT</version><br />
<scope>provided</scope><br />
</dependency><br />
<dependency><br />
<groupId>org.gcube.core</groupId><br />
<artifactId>gcf</artifactId><br />
<version>[1.4.0,1.5.0]</version><br />
<scope>provided</scope><br />
</dependency><br />
</source><br />
<br />
== Profile ==<br />
<br />
The gCube Software profile (<code>profile.xml</code>) of a development version of the widget is as follows:<br />
<br />
<source lang="xml" highlight="13-17,32-36"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource><br />
<ID></ID><br />
<Type>Service</Type><br />
<Profile><br />
<Class>PortletAdmin</Class><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>sample-admin-portlet</Name><br />
<Version>1.0.0</Version><br />
<MavenCoordinates><br />
<groupId>org.gcube.portles.admin</groupId><br />
<artifactId>sample-admin-portlet</artifactId><br />
<version>1.0.0-SNAPSHOT</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/sample-admin-portlet-1.0.0-SNAPSHOT.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Notice that:<br />
<br />
* the whole profile and the packages have usual gCube coordinates;<br />
* package names are aligned with the Maven <code>artifactId</code> of the project;<br />
* package versions are aligned with the Maven <code>version</code> of the project;<br />
* package include <code>MavenCoordinates</code> that can be directly copied and pasted from the POM of the project;<br />
* packages do '''not''' specify dependencies;<br />
<br />
The alignment between profile and POM can be exploited to simplify the management of the profile across different component versions.<br />
In particular, <code>sample-portlet</code> uses POM variables top define its profile:<br />
<br />
<source lang="xml"><br />
<?xml version="1.0" encoding="UTF-8"?><br />
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><br />
<ID /><br />
<Type>Service</Type><br />
<Profile><br />
<Description>${description}</Description><br />
<Class>PortletAdmin</Class><br />
<Name>${artifactId}</Name><br />
<Version>1.0.0</Version><br />
<Packages><br />
<Software><br />
<Name>${artifactId}</Name><br />
<Version>${version}</Version><br />
<MavenCoordinates><br />
<groupId>${groupId}</groupId><br />
<artifactId>${artifactId}</artifactId><br />
<version>${version}</version><br />
</MavenCoordinates><br />
<InstallScripts><br />
<File>installPortlet.sh</File><br />
</InstallScripts><br />
<Files><br />
<File>target/${build.finalName}.war</File><br />
</Files><br />
</Software><br />
</Packages><br />
</Profile><br />
</Resource><br />
</source><br />
<br />
Note that the variables must be resolved (i.e. the profile is ''interpolated'') before the profile can be used within the system. In particular, the main destination of the profile is the gCube Software Archive (SA) which packages the component for registration within the system.<br />
<br />
== Service Archive ==<br />
<br />
<code>sample-portlet</code> generates the service SA as part of its Maven build. This requires dedicated logic in the POM of the project.<br />
<br />
As shown for [[#Libraries|libraries]], the portlet uses the [http://maven.apache.org/plugins/maven-assembly-plugin Maven Assembly Plugin] to generate its own SA, with an approach that makes optimal use of Maven variable interpolation in static files such as <code>README</code>, <code>svnpath.txt</code>, <code>MAINTAINERS</code>, <code>changelog.xml</code>, etc. In particular, the <code>Assembly</code> plugin takes care of variable interpolation for these files. Refer to the [[http://svn.d4science.research-infrastructures.eu/gcube/trunk/portal/maven-sample-portlet project sources]] for details.</div>Gabriele.giammatteo