Difference between revisions of "Service Archive Specification"

From Gcube Wiki
Jump to: navigation, search
(Prepare the archive)
(Example)
 
(34 intermediate revisions by 4 users not shown)
Line 1: Line 1:
==Introduction==
+
==Short background==
  
A 'Service' in DILIGENT is a software system that delivers functionalities, it is composed by a set of related 'Packages'.
+
A 'Service' in D4Science is a software system that delivers functionalities and it is composed by a set of related 'Packages'.
  
In the DILIGENT context, a 'Package' is a 'piece of software' that can be deployed in a DHN. Packages are single files, compliant with the Package Model, that contain the files to be installed, along with rules describing software/packages dependencies, deployment instructions, etc.
+
In the D4Science context, a 'Package' is a 'piece of software' that can be deployed in a GHN. Packages are single tarballs, compliant with the Package Model, that contain the files to be installed, along with rules describing software/packages dependencies, deployment instructions, etc.
  
Each service is described by a 'Profile' document, the 'Service Profile'.  
+
Each service is described by a 'Profile' document, named 'Service Profile'.  
  
 
For each Service Profile a corresponding 'Service Archive' should be delivered.
 
For each Service Profile a corresponding 'Service Archive' should be delivered.
 
  
 
==Structure==
 
==Structure==
 
A Service Archive is a single TAR GZ file, which contains all the files declared on the Service Profile and has the following structure:
 
A Service Archive is a single TAR GZ file, which contains all the files declared on the Service Profile and has the following structure:
 
<pre>
 
<pre>
/-ServiceProfile_<ServiceName>_<ServiceClass>.xml
+
 
 +
|-profile.xml
 
|
 
|
/-<PackageName>
 
|    |
 
|    |-<LibraryFile/GARArchive>
 
|    |-<additional files>
 
 
|
 
|
/-<PackageName>
+
|--<PackageName>(Main)
|   |
+
| |--<GARArchive>
|    |-<LibraryFile/GARArchive>
+
| |--svnpath.txt
|   |-<additional files>
+
|  |--<additional files>
 +
|  |--doc
 +
|    \--api
 +
|        |- index.html
 +
|        \- others javadoc files
 
|
 
|
|-<PackageName>
 
|    |
 
|    |-<LibraryFile/GARArchive>
 
|    |-<additional files>
 
 
|
 
|
 +
|--<PackageName>
 +
|  |--<LibraryFile>
 +
|  |--svnpath.txt
 +
|  |--<additional files>
 +
|  |--doc
 +
|    \--api
 +
|        |-- index.html
 +
|        \-- others javadoc files
 +
|
 +
|
 +
|....
 +
|
 +
|
 +
\--<PackageName>
 +
  |--<LibraryFile>
 +
  |--svnpath.txt
 +
  |--<additional files>
 +
  |--doc
 +
      \--api
 +
        |-- index.html
 +
        \-- others javadoc files
 +
 +
 +
 
/.....
 
/.....
 
</pre>
 
</pre>
Line 40: Line 60:
 
and any other additional file declared in the service profile (e.g. installation or reboot scripts)
 
and any other additional file declared in the service profile (e.g. installation or reboot scripts)
  
 +
In Main packages the 'doc/api' folder with javadoc is mandatory, optional for the others packages.
 +
 +
The 'svnpath.txt' file is mandatory for each packages and contains the link to the svn source code.
  
 
==Creating Service Archives==
 
==Creating Service Archives==
Line 56: Line 79:
 
Create a new component in ETICS for each service profile you already defined in order to create the related Service Archive as artefact. Let see now how you should do this in ETICS.
 
Create a new component in ETICS for each service profile you already defined in order to create the related Service Archive as artefact. Let see now how you should do this in ETICS.
  
# name the component as <ServiceName>.SERVICEARCHIVE
+
# name the component as <ServiceName>-servicearchive
# declare a static dependency against each ETICS component that produces a package to be included in the Service Archive. In this way, the packages forming the whole service are available and they are built immediately before they are packaged. Link this configuration with the corresponding root configuration, es. HEAD, 0_2_0.
+
# declare a dynamic dependency against each ETICS component that produces a package to be included in the Service Archive. In this way, the packages forming the whole service are available and they are built immediately before they are packaged. Link this configuration with the corresponding parent configuration.
 
# use the CVS Commands to download additional files related to the whole service, but not included in the packages (like the Service Profile)
 
# use the CVS Commands to download additional files related to the whole service, but not included in the packages (like the Service Profile)
 
# use the Build Commands available in ETICS in the following way:
 
# use the Build Commands available in ETICS in the following way:
Line 65: Line 88:
  
 
INSTALL:
 
INSTALL:
The install command has to be used to move the GAR/JAR files and other additional files from their
+
The install command has to be used to move the GAR/JAR files and other additional files from their ${moduleDir} directory to the appropriate <PackageName> directory previously created.
${moduleDir}directory to the appropriate <PackageName> directory previously created.
+
Copy the javadoc to the appropriate <PackageName> directory.
 +
Furthermore echo command with vcsroot link have to be redirected to a file 'svnpath.txt' in the appropriate <PackageName> directory.
  
 
PACKAGING:
 
PACKAGING:
Line 73: Line 97:
 
</pre>
 
</pre>
  
To better understand the final result of the process above, you can have a look to the *.SERVICEARCHIVE components already created in ETICS.
+
To better understand the final result of the process above, you can have a look to the *-servicearchive components already created in ETICS.
  
==Creating Software Archives==
+
==Example==
 
+
The following Profile describes a service and its stubs as two packages
In some cases, it could be useful to define a group of packages not directly bound to a specific service. This is typically the case of:
+
* [[Profile_Specification#ExternalSoftware | external software]]
+
* [[Profile_Specification#External_Library | external libraries]]
+
* [[Profile_Specification#Library | common libraries]] developed as part of the gCube framework not logically related to a single service
+
A Software Archive is a [[Service_Archive_Specification|Service Archive]] file that includes only packages of the above types, without any WSRF service defined inside. Differently from the others, it is not possible to generate a Running Instance profile from this Archives at deployment time. They have the only role to group packages.
+
 
+
==== Prepare the packages ====
+
----
+
The way in which each package of the Service Archive has to be prepared depends on its type:
+
:* if the package is a gCube library it has to be built and packaged as any other package following the ETICS guidelines
+
:* if the package is an external software or library the appropriate tarball (.tar.gz) has to be prepared and made available on an HTTP location (typically on the ETICS repository). Then, an ETICS subsystem under the "ext" project that refers this tarball file has to be created. The Collective Layer behaves with these two kind of package as follows:
+
::&rarr; external library
+
::: all the JAR archives included in the tarball are copied in the $GLOBUS_LOCATION/lib folder and made available to the services hosted in that container
+
::&rarr; external software
+
::: the tarball is uncompressed and the [[Profile_Specification#InstallScripts|install scripts]] inside are executed, while the [[Profile_Specification#RebootScripts|reboot scripts]] and [[Profile_Specification#UninstallScripts|uninstall scripts]] scripts are scheduled; these scripts have to be declared in the package definition with their relative path inside the tarball.
+
 
+
==== Prepare the archive ====
+
----
+
In order to compile such kind of Service profile, the following steps have to be followed:
+
* declare the service class as ''SoftwareArchive''
+
* define the list of one or more packages as described [[Profile_Specification#PackagesList_section|here]] (no WSRFService and/or Portlet package is accepted in the SoftwareArchive class)
+
* define on ETICS a subsystem (named with _SERVICEARCHIVE extension) that creates the related Service Archive tarball (.tar.gz) to upload on the Package Repository service as result of its build activities (see [[Service_Archive_Specification| here]])
+
 
+
Example:
+
 
+
The following profile defines a Service Archive profile including an external software (eXist) available on a HTTP location:
+
 
<pre>
 
<pre>
<DILIGENTResource>
+
<?xml version="1.0" encoding="UTF-8"?>
     <UniqueID>gfhwofi-423432-fdsff2-324432</UniqueID>
+
<Resource>
     <ResourceType>Service</ResourceType>
+
     <ID>gfgdf80-131b-11db-90fgf98-dgfgfgf66ed3f</ID>
    <AuthorizationPolicies />
+
     <Type>Service</Type>
 
     <Profile>
 
     <Profile>
         <Class>SoftwareArchive</Class>
+
         <Class>InformationSystem</Class>
         <Name>eXist</Name>
+
         <Name>IS-Registry</Name>
         <DescriptiveParameters>
+
         <Version>1.0.0</Version>
             <DescParameter />
+
        <Packages>
        </DescriptiveParameters>
+
             <Main>
        <QoS />
+
<Name>IS-Registry-service</Name>
        <DLDependencies />
+
<Version>1.0</Version>
        <SpecificData />
+
<Mandatory level="VO"/>
        <PackagesList>
+
<Shareable level="VO"/>
          <Package>
+
<GHNRequirements>
                <PackageName>eXist</PackageName>
+
<Requirement category="Site" requirement="string" value="java1.5" operator="ge"/>
                <PackageType>ExternalSoftware</PackageType>
+
</GHNRequirements>
                <Version>1.1</Version>
+
<Dependencies>
                <DLMandatory value="false" />
+
<Dependency>
                <DHNMandatory value="false" />
+
<Service>
                <VOMandatory value="false" />
+
  <Class>InformationSystem</Class>
                <DLSharable value="true" />
+
  <Name>IS-Registry</Name>
                <DisposeInterfaceSupport value="false" />
+
  <Version>1.0.0</Version>
                <MultiVersionSupport value="false" />
+
</Service>
                <VOSharingSupport value="false" />
+
<Package>IS-Registry-stubs</Package>
                <ManifestFile />
+
<Version>1.0.0</Version>
                <InstallScripts>
+
<Scope level="GHN"/>
                    <File>scripts/install1.1.sh</File>
+
<Optional>false</Optional>
                </InstallScripts>
+
</Dependency>
                <UninstallScripts />
+
</Dependencies>
                <RebootScripts>
+
<GARArchive>org.gcube.informationsystem.registry.gar</GARArchive>
                    <File>scripts/reboot.sh</File>
+
            <PortType>
                </RebootScripts>
+
                <Name>gcube/informationsystem/registry/registryFactory</Name>
                <Dependencies />
+
                    <Security/>
                <DHNRequirements>
+
<WSDL/>
                    <Req category="RunTimeEnv" operator="eq" requirement="Variable" value="java1.5" />
+
            </PortType>
                    <Req category="OperatingSystem" operator="eq" requirement="Name" value="Linux" />
+
            <PortType>
                </DHNRequirements>
+
                <Name>gcube/informationsystem/registry/registry</Name>
                <ConfigurationFiles />
+
                    <Security/>
                <ExternalSoftware>
+
<WSDL/>
                    <DownloadFrom>http://grids17.eng.it/engrepository/exist/1.1.1/noarch/eXist-1.1.1-newcore-build4311.tar.gz</DownloadFrom>
+
            </PortType>
                    <Description>eXist XML database</Description>
+
            </Main>
                </ExternalSoftware>
+
            <Software>
                <OtherFiles />
+
            <Name>IS-Registry-stubs</Name>
                <OtherProperties />
+
<Version>1.0</Version>
            </Package>
+
<MultiVersion value="true"/>
         </PackagesList>
+
<Mandatory level="GHN"/>
 +
<Shareable level="VO"/>
 +
<GHNRequirements>
 +
<Requirement category="Site" operator="ge" requirement="string" value="java1.5"/>
 +
</GHNRequirements>
 +
<SpecificData/>
 +
<Type>library</Type>
 +
<Files>
 +
<File>org.gcube.informationsystem.registry.stubs.jar</File>
 +
</Files>
 +
  </Software>          
 +
         </Packages>
 
     </Profile>
 
     </Profile>
</DILIGENTResource>
+
</Resource>
 
</pre>
 
</pre>
  
The following tarball file has to be produced as artifact by the *_SERVICEARCHIVE ETICS subsystem defined for such a purpose:
+
This service has to produce as ServiceArchive a tar.gz file with the following structure:
 +
 
 
<pre>
 
<pre>
|-/eXist
+
|--profile.xml
  |-ServiceProfile_eXist_SoftwareArchive.xml
+
|
  |-/scripts
+
|
      |- install1.1.sh
+
|--IS-Registry-service
      |- reboot.sh
+
| |--org.gcube.informationsystem.registry.gar
 +
|--svnpath.txt
 +
|  \--doc
 +
|    \--api
 +
|        |--index.html
 +
|        \--others javadoc files
 +
|
 +
|
 +
\--IS-Registry-stubs
 +
  |--org.gcube.informationsystem.registry.stubs.jar
 +
  \--svnpath.txt
 +
 
 +
 
 
</pre>
 
</pre>
  
  
A [[Profile_Specification#How_to_specify_dependencies|dependency]] against any package declared in a SoftwareArchive class has to be specified as usual.
+
For this service archive the ETICS configuration is
  
--[[User:Henri|Henri]] 17:40, 27 March 2007 (EEST)
+
<pre>
 +
INIT:
 +
mkdir -p ${prefix};
 +
mkdir -p ${prefix}/IS-Registry-service;
 +
mkdir -p ${prefix}/IS-Registry-stubs;
  
--Updated by [[User:Manuelesimi|Manuele Simi]] 18:43, 20 July 2007 (EET)
+
 
 +
INSTALL:
 +
cp ${org.gcube.information-system.registry.moduleDir}/etc/profile.xml ${prefix};
 +
cp ${org.gcube.information-system.registry-stubs.moduleDir}/lib/org.gcube.informationsystem.registry.stubs.jar ${prefix}/IS-Registry-stubs;
 +
cp ${org.gcube.information-system.registry.moduleDir}/lib/*.gar ${prefix}/IS-Registry-service;
 +
cp -r ${org.gcube.information-system.registry.moduleDir}/doc ${prefix}/IS-Registry-service;
 +
echo ${org.gcube.information-system.registry.vcsroot}/${org.gcube.information-system.registry.tag} > ${prefix}/IS-Registry-service/svnpath.txt;
 +
echo ${org.gcube.information-system.registry-stubs.vcsroot}/${org.gcube.information-system.registry-stubs.tag} > ${prefix}/IS-Registry-stubs/svnpath.txt;
 +
 
 +
 
 +
</pre>

Latest revision as of 12:34, 29 August 2008

Short background

A 'Service' in D4Science is a software system that delivers functionalities and it is composed by a set of related 'Packages'.

In the D4Science context, a 'Package' is a 'piece of software' that can be deployed in a GHN. Packages are single tarballs, compliant with the Package Model, that contain the files to be installed, along with rules describing software/packages dependencies, deployment instructions, etc.

Each service is described by a 'Profile' document, named 'Service Profile'.

For each Service Profile a corresponding 'Service Archive' should be delivered.

Structure

A Service Archive is a single TAR GZ file, which contains all the files declared on the Service Profile and has the following structure:


|-profile.xml
|
|
|--<PackageName>(Main)
|  |--<GARArchive>
|  |--svnpath.txt
|  |--<additional files>
|  |--doc
|     \--api
|        |- index.html
|        \- others javadoc files
|
|
|--<PackageName>
|  |--<LibraryFile>
|  |--svnpath.txt
|  |--<additional files>
|  |--doc
|     \--api
|        |-- index.html
|        \-- others javadoc files
|
|
|....
|
|
\--<PackageName>
   |--<LibraryFile>
   |--svnpath.txt
   |--<additional files>
   |--doc
      \--api
         |-- index.html
         \-- others javadoc files



/.....

where:

  • at the root level, the archive contains the Service Profile of the service itself. All the file names included in the archive have to be reported with a relative path starting from their <PackageName> folder (not included) *as defined on the Service Profile*.
  • for every Package declared inside the Service Profile, the archive contains a directory (called with the same name of the PackageName field of the profile) including the corresponding jar/gar file:
    • for a WSRF service the name reported into the GARArchive field;
    • for a library/stubs the name reported into the LibraryFile field.

and any other additional file declared in the service profile (e.g. installation or reboot scripts)

In Main packages the 'doc/api' folder with javadoc is mandatory, optional for the others packages.

The 'svnpath.txt' file is mandatory for each packages and contains the link to the svn source code.

Creating Service Archives

Manually

An easy way to create a Service Archive is to do it manually.

  1. Create a folder structure as described in previous section according to the corresponding Service Profile.
  2. Create a TAR GZ file of the created folder structure.


Using ETICS to automatically create Service Archives

Create a new component in ETICS for each service profile you already defined in order to create the related Service Archive as artefact. Let see now how you should do this in ETICS.

  1. name the component as <ServiceName>-servicearchive
  2. declare a dynamic dependency against each ETICS component that produces a package to be included in the Service Archive. In this way, the packages forming the whole service are available and they are built immediately before they are packaged. Link this configuration with the corresponding parent configuration.
  3. use the CVS Commands to download additional files related to the whole service, but not included in the packages (like the Service Profile)
  4. use the Build Commands available in ETICS in the following way:
INIT:
The init command has to be used to create the ${prefix} directory and all the <PackageName> subdirs

INSTALL:
The install command has to be used to move the GAR/JAR files and other additional files from their ${moduleDir} directory to the appropriate <PackageName> directory previously created.
Copy the javadoc to the appropriate <PackageName> directory.
Furthermore echo command with vcsroot link have to be redirected to a file 'svnpath.txt' in the appropriate <PackageName> directory.

PACKAGING:
Leave this command blank. At the end of the build process the default ETICS package command will create a
tar.gz (the ServiceArchive!) of the ${prefix} folder

To better understand the final result of the process above, you can have a look to the *-servicearchive components already created in ETICS.

Example

The following Profile describes a service and its stubs as two packages

<?xml version="1.0" encoding="UTF-8"?>
<Resource>
    <ID>gfgdf80-131b-11db-90fgf98-dgfgfgf66ed3f</ID>
    <Type>Service</Type>
    <Profile>
        <Class>InformationSystem</Class>
        <Name>IS-Registry</Name>
        <Version>1.0.0</Version>
        <Packages>
            <Main>
		<Name>IS-Registry-service</Name>
		<Version>1.0</Version>
		<Mandatory level="VO"/>
		<Shareable level="VO"/>
		<GHNRequirements>
			<Requirement category="Site" requirement="string" value="java1.5" operator="ge"/>
		</GHNRequirements>
		<Dependencies>
			<Dependency>
				<Service>
				   <Class>InformationSystem</Class>
		 		   <Name>IS-Registry</Name>
		 		   <Version>1.0.0</Version>
				</Service>
				<Package>IS-Registry-stubs</Package>
				<Version>1.0.0</Version>
				<Scope level="GHN"/>
				<Optional>false</Optional>
			</Dependency>
		</Dependencies>
		<GARArchive>org.gcube.informationsystem.registry.gar</GARArchive>
            	<PortType>
                	<Name>gcube/informationsystem/registry/registryFactory</Name>
                    	<Security/>			
			<WSDL/>
            	</PortType>
            	<PortType>
                	<Name>gcube/informationsystem/registry/registry</Name>
                    	<Security/>			
			<WSDL/>
            	</PortType>
            </Main>
            <Software>
            	<Name>IS-Registry-stubs</Name>
		<Version>1.0</Version>
		<MultiVersion value="true"/>
		<Mandatory level="GHN"/>
		<Shareable level="VO"/>
		<GHNRequirements>
			<Requirement category="Site" operator="ge" requirement="string" value="java1.5"/>
		</GHNRequirements>
		<SpecificData/>
		<Type>library</Type>
		<Files>
			<File>org.gcube.informationsystem.registry.stubs.jar</File>
		</Files>
	   </Software>           
        </Packages>
    </Profile>
</Resource>

This service has to produce as ServiceArchive a tar.gz file with the following structure:

|--profile.xml
|
|
|--IS-Registry-service
|  |--org.gcube.informationsystem.registry.gar
|  |--svnpath.txt
|  \--doc
|     \--api
|        |--index.html
|        \--others javadoc files
|
|
\--IS-Registry-stubs
   |--org.gcube.informationsystem.registry.stubs.jar
   \--svnpath.txt



For this service archive the ETICS configuration is

INIT: 
mkdir -p ${prefix};
mkdir -p ${prefix}/IS-Registry-service;
mkdir -p ${prefix}/IS-Registry-stubs;


INSTALL:
cp ${org.gcube.information-system.registry.moduleDir}/etc/profile.xml ${prefix};
cp ${org.gcube.information-system.registry-stubs.moduleDir}/lib/org.gcube.informationsystem.registry.stubs.jar ${prefix}/IS-Registry-stubs;
cp ${org.gcube.information-system.registry.moduleDir}/lib/*.gar ${prefix}/IS-Registry-service;
cp -r ${org.gcube.information-system.registry.moduleDir}/doc ${prefix}/IS-Registry-service;
echo ${org.gcube.information-system.registry.vcsroot}/${org.gcube.information-system.registry.tag} > ${prefix}/IS-Registry-service/svnpath.txt;
echo ${org.gcube.information-system.registry-stubs.vcsroot}/${org.gcube.information-system.registry-stubs.tag} > ${prefix}/IS-Registry-stubs/svnpath.txt;