Core Services Installation

Keeper

The Keeper service groups a set of software components that take care of the creation and maintenance of dynamic DLs by instantiating the appropriate resources and authorizing users to get access to those resources. In order to perform these activities, the Keeper service focusses on the entire lifecycle of software packages management ranging from the specifications and conventions they must follow to their physical deployment and relocation. The Keeper service is a logical service composed by:

• Package Repository Service (WSRFService) – The Package Repository validates, stores, and manages DILIGENT packages. It checks packages dependencies and giving both access to packages relations and access to the stored packages supports the dynamic packages deployment. This repository accepts registration requests coming from the Service Archive Registration Protlet that is part of the VDL service, whilst accepts access requests from the DL service and the HNM service.
• Grid Access Service (WSRFService) – The Grid Access Service (GAS) provides Grid storage capabilities to all services running in a DHN, allowing services to store on the Grid local files. GAS works at Running Instance level, so each service RI can store its state, or any other data that should survive besides the DHN life cicle. It manages storing, listing, and getting of its own Grid files.
• DL Management Service (WSRFService) – This component coordinates the packages deployment process when a new DL is instantiated and during its lifetime. The operational context that transforms a set of distributed deployed DILIGENT resources into a single application is managed by this component by constructing a DL Map, i.e. a map containing the DL resources locations, their configuration, and the rules to access them.
• Hosting Node Manager Service (WSRFService) – The Hosting Node Manager (HNM) manages the Diligent Hosting Node (DHN) by providing the context to deploy the DILIGENT packages accordingly to the DL Management instructions. In particular, when the HNM is deployed, it controls the software dependencies by using the Package Repository, and then it automatically downloads and deploys all the DHN mandatory packages. It also deploys by default the shared Node Access Library (NAL) that exposes a uniform API allowing to query the current node configuration on the node and giving access to GAS. Moreover, it creates and publishes into the DIS the profiles of all Running Instances deployed in the Java WS Core. Clearly, the HNM must be pre-deployed on each DHN of the DILIGENTinfrastructure.
• Node Access Library (Library) – The Node Access Library (NAL) provides functionalities to access the local DHN configuration and allows RI to store on the Grid its own data.
• Profile Manager Library (Library) – The Profile Manager library is an utility package that lets DILIGENT services manage easily DILIGENT XML Resource profiles.
• Keeper Common Library (Library) – This library is composed by a set of WSDL definitions, that all the Keeper Services include in their WSDL. At compilation time (using the WSDL2Java tool), these definitions are evaluated and generate a set of Java classes that can be referred inside service code.

Pre-installation setup

All the Keeper Libraries and stubs are deployed togheter with the DHN installation, that contains also the Hosting Node Manager (HNM) Service. The Package Repository DHN requires a node with Apache Server installed and therefore the port Apache listens for connections has not to be behind a firewall.

DL Management

The DLManagement Service is a VO specific service that has to be deployed for each VO ( in the gCube Advanced release will be introduced the Dynamic-VO deployment and therefore also the DLManagement will be automatically deployed for each VO). As any of the Collective layer services managing the root VO, the DLManagement as to be deployed manually and properly configured.

The DLManagement ServiceArchive 0_3_0 can be downloaded from the Eng repository . These are the installation steps to follow:

• Unpack the ServiceArchive tar.gz file;
• type globus-deploy-gar org_diligentproject_keeperservice_dlmanagement.gar to deploy the DLManagement Service on the local container;
• copy ServiceProfile_Keeper_DLManagement.xml under the $GLOBUS_LOCATION/etc/org_diligentproject_keeperservice_hnm/ServicesProfiles/ folder in order to publish the DL Management Service on the Running Instance of the DHN and in order to enable the service to accept requests from its clients.

DONE!

Security Settings

The DLManagement Service, in case of Secure infrastructure, work using Service Credentials for they operations. The Manually deployment of the services implies also the manual setting of Credential Renewal Task that will let the services work with fresh credentials. This can be done using the Credential Renewal Service, interacting with it using the CredentialRenewal API and Stubs, contained inside the Credentials Renewal ServiceArchive 0_3_0 that can be downloaded from the Eng repository .

Setting the DLManagement Credential Renewal Task:

java org.diligentproject.dvos.credentialRenewal.ui.CredentialRenewalUI -addTask -accountID b8cf9578-1bcf-4471-b982-356781cc7df5 \
 -host grids06.eng.it -port 8086 -voName diligent -groupName /diligent/devsec -serviceClass Keeper -serviceName DLManagement -proxy \
<proxyFile> -delegationID diligentproject/keeperservice/dlmanagement/DLManagementFactoryService \
-delegationServiceURL http://<host>:<port>/wsrf/services/diligentproject/dvos/delegation/DelegationService \
-period 24 -roles Credentials-Manager

Post-installation configuration

DLManagement Service installation depends on the type of VO the DLManagement has to manage:

• "Root" VO;
• Sub-VO;

In case of root VO the DLManagement Service needs static information about the root VO name and VOMap file. These parameters has to be configured through the Service JNDI file:

• voName : the root VO Name
• voMapPath : the VOMap path relative to the DLMangement container folder.

The JNDI file contains further parameter to be configured (both for VO - subVO deployment):

• useBMM : if true the DLManagement will use the BMM Service to retrieve the best deployment Schema during automatic deployment
• startSweeper : if true starts the DHNSweeper, that removes from the DIS RI and related DHN Profiles not more reachable.
• sweeperDelay : the sweeper delay in ms.

Installation troubleshooting

Things that can go wrong. Error messages that my appear. Workarounds to common problems

Package Repository

The Package Repository should not be directly used by other services besides Service Archive registration Portlet for storing issues, DLManagement Service to query for dependencies and HNM to get stored packages. The VDL, DL Mng., and HNM act as PR clients, so PR stubs should be on that clients installations.

Access to this component has to be done, as for any other WSRF service, by creating the appropriate portType connected to the EPR of an active instance of the service using the stubs classes distributed with the component.

Package Repository service is distributed with a PR client that can be used to check all PR functionalities as well as a system to initialize a DILIGENT infrastructure from scratch.

Pre-installation setup

The PR Service is an Infrastructure specific service (there is the need of an instance of it for each Infrastructure).

The service installation requires:

• a node from where it is possible to access to the Grid storage;
• a DHN installed and configured; and
• a set of environment variables to be defined.

Setting up the Access to the Grid Storage

Access to a gLIte SE (e.g. DPM) and a gLite LFC is mandatory to run the Package Repository.

From the machine where PR will be deployed execute as root following operations:

OS and gLite Installation

1:. Install SLC3

2:. Install APT:

wget ftp://ftp.scientificlinux.org/linux/scientific/30x/i386/SL/RPMS/apt-xxx.i386.rpm 
rpm -ivh apt-XXX.i386.rpm

3:. Check/add APT repositories to the configuration files:

/etc/apt/sources.list.d/glite.list: rpm http://glitesoft.cern.ch/EGEE/gLite/APT/R3.0/ rhel30 externals Release3.0 updates
/etc/apt/sources.list.d/lcg-ca.list: rpm http://grid-deployment.web.cern.ch/grid-deployment/gis apt/LCG_CA/en/i386 lcg

4:. Install the following APT packages:

apt-get install lcg_util
apt-get install LFC-client LFC-interfaces
apt-get install glite-security-voms-clients

Security Configuration

1:. Install CA certificate:

apt-get install lcg-CA
rpm -ivh ca_ENG-xxx.rpm
rpm -ivh ca_UMIT-xxx.rpm

2:. Install VOMS server certificate:

/etc/grid-security/vomsdir/grids03.eng.it.pem

3:. Install host certificates:

/etc/grid-security/hostcert.pem
/etc/grid-security/hostkey.pem

check files permissions (chmod 644 and 400)

Install a DHN

see here .

Main installation procedure

The PR ServiceArchive 0_3_0 can be downloaded from the Eng repository . These are the installation steps to follow:

1:. From the PR (user) account unpack the ServiceArchive tar.gz file;

2:. Type

globus-deploy-gar org_diligentproject_keeperservice_packagerepository.gar 

to deploy the PR Service on the local container;

3:. Copy

ServiceProfile_PackageRepository_Keeper.xml 

under the

$GLOBUS_LOCATION/etc/org_diligentproject_keeperservice_hnm/ServicesProfiles/

folder in order to publish the PR Service on the Running Instance of the DHN and in order to enable the service to accept requests from its clients.

4:. Configure environment

* update $GLOBUS_LOCATION/etc/org_diligentproject_keeperservice_packagerepository/PackageRepository/PR.properties

* update start-PR.sh script variables 

* change $HTTP_APACHE_DIR directory rights to allow write

Source start-PR.sh and ${GLOBUS_LOCATION}/etc/globus-devel-env.sh

5:. Create vomses file

~/.glite/vomses: "diligent" "grids03.eng.it" "15001" "/O=Grid/OU=GlobusTest/OU=simpleCA-gauss.eng.it/CN=grids03.eng.it" "diligent"

copy user certificates to user account

~/.globus

6:. Copy all libs into Java WS-Core lib ($GLOBUS_LOCATION/lib) if not already installed

activation.jar (*)
commons-compress-20061201.jar 
jaxb1-impl.jar (*)
jaxb-api.jar (*)
jaxb-impl.jar (*)
jaxb-xjc.jar  (*)
jsr173_api.jar (*)
org_diligentproject_common_profile.jar (*)
org.diligentproject.keeperservice.hnm.impl.nal.jar (*)
org_diligentproject_keeperservice_hnm_stubs.jar (*)
yu_1.2_forDiligent.jar

(*) already installed with DHN.

Post-installation configuration

1:. Manual proxy generation

voms-proxy-init -voms diligent

2:. Start Apache

 
/etc/init.d/httpd start

3:. Start container

globus-start-container -nosec -debug >container.log 2>error.log &

Testing and verifying the installation

Check Developers manual client section....

Installation troubleshooting

Things that can go wrong. Error messages that may appear. Workarounds to common problems

Broker & Matchmaker (BMM)

The BMM Service is composed by the following components:

• The BMM Connector (Java Library) and the BMM API (Java Library) allow clients (e.g. the DL Management service) to send a matching request, and notify them with the response.
• The DIS Connector (Java Library) is in charge of keeping up-to-date tracks of the DHN profiles received from the DIS and to query the DIS in order to gather information the service or the algorithm needs for their computations.
• The BMM Service (WSRF Service) provides the core functionalities of the BMM component. By invoking the DIS Connector it queries the DIS in order to gather information about packages, then it forwards the BMM Connector request to the BMM Algorithm and, when the response is ready, it returns back the result.
• The BMM Utils (Java Library) is a library shared by other components. It defines exceptions and provides the validator used to parse the request and the response, as well as other helper classes.
• The BMM Algorithm (Java Library) calculates, by running a customized version of first-fit algorithm, the associations among packages and DHNs.

Pre-installation setup

The BMM Service is VO specific service (there is the need of an instance of it for each VO) and so the service installation requires a node (DHN) for each VO. The BMM API and the BMM Connector libraries should be deployed on the client side. The other components on the server side.

Main installation procedure

The BMM ServiceArchive 0_3_0 can be downloaded from the Eng repository . These are the installation steps to follow:

• Unpack the ServiceArchive tar.gz file;
• type globus-deploy-gar bmm.matchMaker-service.gar to deploy the BMM Service on the local container;
• copy ServiceProfile_broker_BMM.xml under the $GLOBUS_LOCATION/etc/org_diligentproject_keeperservice_hnm/ServicesProfiles/ folder in order to publish the BMM Service on the Running Instance of the DHN and in order to enable the service to accept requests from its clients.

DONE!

Post-installation configuration

None.

Testing and verifying the installation

Provide instructions that will assist the administrator in verifying that the service has been installed and is running appropriately. Troubleshooting of the installation together with error messages and common compensation actions should be provided in detail in chapter 4.

Installation troubleshooting

Things that can go wrong. Error messages that may appear. Workarounds to common problems

DILIGENT Information Service (DIS)

The following components compose the DILIGENT Information Service:

• DIS-IP (Library) – The DIS-IP is responsible for registering/unregistering a group of resource properties as Aggregator Source to one or more DIS-ICs. It also allows to register/unregister groups of Topics in the DIS-Broker.
• DIS-HLSClient (Library) – The DIS-HLSClient is a library used by DILIGENT services to access the information maintained by the DILIGENT Information Service. Using a DIS-HLSClient it is possible to query a DIS-IC to discover Profiles or WS-Resource properties.
• DIS-IC (WSRFService) – This service is the Information Collector (IC) of all the data published in the DIS. It is implemented as Aggregator Sink that collects RPs from the registered (via DIS-IP) Aggregator Sources.
• DIS-BDIIClient (WSRFService) – This service is in charge of harvesting resource information from the BDII Server1 it has been configured to interact with. The gathered information is manipulated in order to make it compliant with the schema adopted in DILIGENT. Then such information is published as WS-Resources via the DIS-IP and as a DILIGENT Resource of type gLiteResource using the mechanism offered by the DIS-Registry Service.
• DIS-Registry (WSRFService) – This service provides registration and un-registration facilities for the DILIGENT resources profiles.
• DILIGENTProvider (Library) – This operation provider adds resource properties to the group of properties registered by a service in the DIS-IC. This additional information allows enlarging the spectrum of functionalities offered to identify the source that publishes the data and to perform fine-grained queries.
• DIS-Broker (WSRFService) – This service provides registration/unregistration of Topics (events to be notified on) for DILIGENT Services. This allows clients to subscribe to/unsubscribe from topics without having to know the physical locations of the services that expose them.

Pre-installation setup

The DIS core Services (DIS-Registry, DIS-Broker and DIS-IC), are VO specific services ( there is the need of an instance of those services for each VO). Starting from the root VO ( the "diligent" root VO), but at the time being also all the sub-VO (i.e. ARTE) need a manual installation. The root DIS is the first DILIGENT services that has to be installed in the infrastructure: it will contain all DHN Profiles and RI Profiles of Services running on the root VO and DHN profiles of Sub-VO node ( that can be assigned to sub-VOs by VO Managers).

The Installation of the root DIS requires at least 3 nodes:

• the DIS-Registry DHN

• the DIS-Broker DHN

• the DIS-IC DHN

The DIS-BDIIClient is a VO specific Services and is no needed at root VO level. In order to speed up the performance and exploits the distributed nature of the GT4 Aggregator Framework, the best DIS Services deployment strategy would be:

• Deploy the DIS-Broker and the DIS-Registry on the same DHN

• Deploy the DIS-IC on a separate DHN.

The following installation documentation assumes that this is the target deployment schema.

Main installation procedure

The DIS Installation needs a manual change on the DHN behaviour. The HNM in general is configured to publish the RI profiles of the codeployed services and the related DHN profile on the root VO. In case the HNM is codeployed toghether with DIS Services (so it has to register Profiles using the codeployed instance of the DIS-Registry) the DHN has to be configured to act as a "root" DHN. The related HNMService inthis context will create all the DIS Running Instance Profiles (togheter with the DHN profile), but it will be DIS-Registry itself that will register these profiles.

DHN root Installation

The "root" DHN has to be installed following the Admin guide. Once the installation has been done, the only change to standard DHN installation is on the HNM Service JNDI file:

• The "rootDHN" parameter has to be set to true ( the DIS DHN is also of type Static)

DIS-IC Installation

TBD

DIS-Broker Installation

The DISBroker ServiceArchive 0_3_0 can be downloaded from the Eng repository . These are the installation steps to follow:

• Unpack the ServiceArchive tar.gz file;
• type globus-deploy-gar org_diligentproject_informationservice_disbroker.gar to deploy the DIS-Broker Service on the local container;

DONE!

DIS-Registry Installation

The DISRegistry ServiceArchive 0_3_0 can be downloaded from the Eng repository . These are the installation steps to follow:

• Unpack the ServiceArchive tar.gz file;
• type globus-deploy-gar org_diligentproject_informationservice_disregistry.gar to deploy the DIS-Registry Service on the local container;

DONE!

Security Setting

All DIS services can be configured to run in a secure/unsecure context. In case the VO to deploy has to run in a secure way the stardard installation will provide server-config.wsdd files that already contain security-descriptor for DIS services. In case the VO has to be deployed without security just:

• enter the specific container folder of DIS services (i.e for DIS-Registry : $GLOBUS_LOCATION/etc/org_diligentproject_informationservice_disregistry )
• copy the content of deploy-server.wsdd_NOSEC file inside server-config.wsdd file
• this will remove the link to the service security-descriptor and has to be done for all DIS services.

The DIS-Registy and DIS-Broker Service, in case of Secure infrastructure, work using Service Credentials for they operations. The Manually deployment of the services implies also the manual setting of Credential Renewal Task that will let the services work with fresh credentials. This can be done using the Credential Renewal Service, interacting with it using the CredentialRenewal API and Stubs, contained inside the Credentials Renewal ServiceArchive 0_3_0 that can be downloaded from the Eng repository .

Setting DIS-Registry and DIS-Broker Credential Renewal Task:

java org.diligentproject.dvos.credentialRenewal.ui.CredentialRenewalUI -addTask -accountID <ID> \
 -host <CredentialHost> -port <CredentialPort> -voName <VO> -groupName <VOMS group> -serviceClass <serviceClass> -serviceName <ServiceName> -proxy \
<proxyFil> -delegationID <WSRFEntryPoint> 
-delegationServiceURL http://<host>:<port>/wsrf/services/diligentproject/dvos/delegation/DelegationService \
-period 24 -roles Basic

IT'S IMPORTANT TO CREATE FIRST THE DIS-BROKER TASK, THEN THE DIS-REGISTRY TASK ( in order to have the DIS-Broker ready first)

i.e TO create a DIS-Registry Task:

java org.diligentproject.dvos.credentialRenewal.ui.CredentialRenewalUI -addTask -accountID b8cf9578-1bcf-4471-b982-356781cc7df5 \
 -host grids06.eng.it -port 8086 -voName diligent -groupName /diligent/devsec -serviceClass InformationSystem -serviceName DIS-Registry -proxy \
/home/andrea/proxy -delegationID diligentproject/informationservice/disregistry/DISRegistryFactoryService \
-delegationServiceURL http://dlib19.isti.cnr.it:8084/wsrf/services/diligentproject/dvos/delegation/DelegationService \
-period 24 -roles Basic

and a DIS-Broker Task:

java org.diligentproject.dvos.credentialRenewal.ui.CredentialRenewalUI -addTask -accountID b8cf9578-1bcf-4471-b982-356781cc7df5 \
 -host grids06.eng.it -port 8086 -voName diligent -groupName /diligent/devsec -serviceClass InformationSystem -serviceName DIS-Broker -proxy \
/home/andrea/proxy -delegationID diligentproject/informationservice/disbroker/DISBrokerService \
-delegationServiceURL http://dlib19.isti.cnr.it:8084/wsrf/services/diligentproject/dvos/delegation/DelegationService \
-period 24 -roles Basic

Post-installation configuration

After the deployment the VOMap for the VO the DIS installed refers to has to be properly configured. So in case of /diligent root VO just change the file VOMap_diligent.xml located into the $GLOBUS_LOCATION/etc/org_diligentproject_keeperservice_hnm/VOMaps modifyng service endpoint according to your DIS installation. In case of sub-VO DIS installation just creates a VOMap_<yourSubVo>.xml file containing as above the endpoint to your DIS installation.

Testing and verifying the installation

Provide instructions that will assist the administrator in verifying that the service has been installed and is running appropriately. Troubleshooting of the installation together with error messages and common compensation actions should be provided in detail in chapter 4.

Installation troubleshooting

Things that can go wrong. Error messages that my appear. Workarounds to common problems

VDL Generator

Pre-installation setup

Actions to be performed before initiating the installation of this service.

Main installation procedure

Describe in full detail all required steps for installing/deploying all components of the service. Group the steps in subparagraphs providing a meaningful header. This section should contain instructions for at least the following sub-services:

• Package Repository
• DL Management and
• Hosting Node Management

Post-installation configuration

Configuration files that have to be edited after the installation. Scripts that have to be run that take care of post installation activities.

Testing and verifying the installation

Provide instructions that will assist the administrator in verifying that the service has been installed and is running appropriately. Troubleshooting of the installation together with error messages and common compensation actions should be provided in detail in chapter 4.

Installation troubleshooting

Things that can go wrong. Error messages that my appear. Workarounds to common problems

Dynamic Virtual Organization Support (DVOS)

The components of the Dynamic Virtual Organization Support are:

• DVOS Common library - A package containing common classes used in DVOS components (part of the Diligent Hosting Node, see here for installation instructions)
• Authentication-API library - A library providing DILIGENT services with some utility method useful to manage authentication tokens (part of the Diligent Hosting Node, see here for installation instructions)
• Delegation Service - A service allowing clients to delegate proxy credentials to DILIGENT services running on a DHN (part of the Diligent Hosting Node, see here for installation instructions)
• Credentials Renewal Service - This service is in charge of periodically refresh credentials of other DILIGENT services
• Authorization service - A service allowing management of DILIGENT authorization elements, for a detailed description of DILIGENT authorization model see here

Pre-installation setup

The DILIGENT security model is based on some existing security components. Following services must be installed (or already present in the infrastructure) to enable security funcionalities.

• MyProxy repository - This repository is required to host temporary credentials of users liable for DILIGENT RIs. These credentials are then delegated to services. To install and configure a MyProxy repository service you can follow these steps.
• MyProxy OnlineCA - This service is required to create temporary credentials for users and services authenticated with the DILIGENT Simple Certification Authority. To install and configure a MyProxy online CA service you can follow these steps.
• VOMS - This service is part of the gLite grid middleware and can be installed following steps of this document.
• A valid proxy certificate of the VOMS VO Administrator (containing the VO-Admin role at the root level)
• VOMSServlet - the servlet required by DILIGENT services to interoperate with the VOMS administration web interface. For detailed information about how to install such a component you can refer here.

Main installation procedure

Credentials Renewal Installation

An installer is avalible to automatically configure the infrastructure, deploy the Credentials Renewal service and configure it.

The Credentials Renewal ServiceArchive 0_3_0 can be downloaded from the Eng repository . These are the installation steps to follow:

• Unpack the ServiceArchive tar.gz file;
• Enter in the install directory with the command cd install
• Set following parameters accordingly to your infrastructure in the install.properties file:

Parameter Name	Description	Example
voAdminProxy	The location of the proxy certificate of a VOMS VO Administrator (containing the VO-Admin role at the root level)	/root/AdminProxy
myProxyRepositoryHost	The host name where the myProxy Repository is running	grids02.eng.it
myProxyOnlineCAHost	The host name where the myProxy Online CA is running	grids04.eng.it
voName	The name of the VOMS VO used by this service ( as contained in the local vomses file )	diligent
groupName	The group where this Credentials Renewal service will operate	/diligent
servletHost	The host name where the VOMS Servlet is running	grids15.eng.it
servletPort	The port number of the container hosting the VOMS Servlet	8094

• Launch the installation script using the command ant -file install.xml. This procedure will automatically:
  • Configure the VOMS selected to enable operations of the CredentialsRenewal service
  • Deploy the service in the local DHN
  • Configure the jndi-config file of the CredentialsRenewal service with parameters set in the previous installation step

DONE!

Authorization Service Installation

TO BE ADDED

Post-installation configuration

Credentials Renewal Post-installation Configuration

None

Authorization Service Post-installation Configuration

TO BE ADDED

Testing and verifying the installation

Provide instructions that will assist the administrator in verifying that the service has been installed and is running appropriately. Troubleshooting of the installation together with error messages and common compensation actions should be provided in detail in chapter 4.

Installation troubleshooting

Things that can go wrong. Error messages that my appear. Workarounds to common problems

Portals

Pre-installation setup

Actions to be performed before initiating the installation of this service.

Main installation procedure

Describe in full detail all required steps for installing/deploying all components of the service. Group the steps in subparagraphs providing a meaningful header. This section should contain instructions for at least the following sub-services:

• Package Repository
• DL Management and
• Hosting Node Management

Post-installation configuration

Configuration files that have to be edited after the installation. Scripts that have to be run that take care of post installation activities.

Testing and verifying the installation

Provide instructions that will assist the administrator in verifying that the service has been installed and is running appropriately. Troubleshooting of the installation together with error messages and common compensation actions should be provided in detail in chapter 4.

Installation troubleshooting

Things that can go wrong. Error messages that my appear. Workarounds to common problems