Difference between revisions of "SmartGears Docker image"

From Gcube Wiki
Jump to: navigation, search
(Trobleshooting)
 
(7 intermediate revisions by the same user not shown)
Line 10: Line 10:
  
 
== How to build ==
 
== How to build ==
In order to build a SmartGears Docker image the user should either run the smartgears-base-image Ansible role or customize the Dockerfile contained in said role. The Ansible role is available at CNR's gitorious ansible-playbooks repository, [https://gitorious.research-infrastructures.eu/infrastructure-management/ansible-playbooks/trees/master/library/roles/smartgears/smartgears-base-image here]. In order to execute such role the user needs to run a playbook that is available [https://gitorious.research-infrastructures.eu/infrastructure-management/ansible-playbooks/blobs/master/d4science-ghn-cluster/smartgears-base-image.yml here] or write his own.  
+
In order to build a SmartGears Docker image the user should either run the smartgears-base-image Ansible role or customize the Dockerfile distributed with said role. The Ansible role is available at CNR's gitorious ansible-playbooks repository, [https://gitorious.research-infrastructures.eu/infrastructure-management/ansible-playbooks/trees/master/library/roles/smartgears/smartgears-base-image here]. In order to execute such role the user needs to run a playbook that is available [https://gitorious.research-infrastructures.eu/infrastructure-management/ansible-playbooks/blobs/master/d4science-ghn-cluster/smartgears-base-image.yml here] or write his own.  
  
 
Let's take a closer look at the playbook:
 
Let's take a closer look at the playbook:
<source lang="xml">
+
<source lang="yaml">
 
- hosts: dockerbuilder # this is the target host with a running docker instance, this should be replaced with a resolvable hostname such as "localhost"
 
- hosts: dockerbuilder # this is the target host with a running docker instance, this should be replaced with a resolvable hostname such as "localhost"
 
   remote_user: root
 
   remote_user: root
Line 24: Line 24:
  
  
Let's assume that we want to build and run a container image locally. After cloning the ansible-playbooks git repo, we access the ansible-playbooks dir and run the ansible-playbook command:
+
Let's assume that we want to build and run a container image locally. After cloning the ansible-playbooks git repo, we access the ansible-playbooks directory and run the following ansible-playbook command:
  
<source lang="xml">
+
<source lang="bash">
 
ansible-playbook d4science-ghn-cluster/smartgears-base-image.yml -i dockerbuilder, -e orchestrator_ssh_key:"xxx" --skip-tags=docker
 
ansible-playbook d4science-ghn-cluster/smartgears-base-image.yml -i dockerbuilder, -e orchestrator_ssh_key:"xxx" --skip-tags=docker
 
</source>
 
</source>
 
  
 
* ''"smartgears-base-image.yml"'' is a playbook that summons the smartgears-base-image role
 
* ''"smartgears-base-image.yml"'' is a playbook that summons the smartgears-base-image role
* ''"-i"'' tells ansible-playbook the inventory file to use, we're passing the "dockerbuilder" target host directly as example
+
* ''"-i"'' tells ansible-playbook the inventory file to use, we're passing the "dockerbuilder" target host directly as an example
 
* ''"-e"'' is used to override variables from the command line, we're using it to provision a public ssh key that will be used to connect via ssh to a running instance on this image  
 
* ''"-e"'' is used to override variables from the command line, we're using it to provision a public ssh key that will be used to connect via ssh to a running instance on this image  
 
* ''"-skip-tags=docker"'' tells Ansible to ignore the "docker" role dependency declared in the [https://gitorious.research-infrastructures.eu/infrastructure-management/ansible-playbooks/blobs/master/library/roles/smartgears/smartgears-base-image/meta/main.yml meta file], this is to avoid running the "docker" role every time we're creating a new image.
 
* ''"-skip-tags=docker"'' tells Ansible to ignore the "docker" role dependency declared in the [https://gitorious.research-infrastructures.eu/infrastructure-management/ansible-playbooks/blobs/master/library/roles/smartgears/smartgears-base-image/meta/main.yml meta file], this is to avoid running the "docker" role every time we're creating a new image.
Line 39: Line 38:
 
This will result in the creation of a SmartGears Docker image. By running "docker images" we should see the image locally available:
 
This will result in the creation of a SmartGears Docker image. By running "docker images" we should see the image locally available:
  
<source lang="xml">
+
<source lang="bash">
 
[root@dockerbuilder]# docker images  
 
[root@dockerbuilder]# docker images  
 
REPOSITORY                        TAG                  IMAGE ID            CREATED            SIZE
 
REPOSITORY                        TAG                  IMAGE ID            CREATED            SIZE
Line 50: Line 49:
  
 
== Build an image with a bundled app ==
 
== Build an image with a bundled app ==
In order to deploy any web application to a SmartGears container two methods are availble. Either the user employs an ssh session to connect to a running instance or he defines a Dockerfile that builds upon the smartgears-base-image by making the required changes - such as depolying a war file, creating a user, manipulating config files, etc. An exemplifying Ansible role here that adds the Home Library war file to the image is available [https://gitorious.research-infrastructures.eu/infrastructure-management/ansible-playbooks/trees/master/library/roles/smartgears/smartgears-container-home-library here]. Let's take a look at the included Dockerfile:
+
In order to deploy any web application to a SmartGears container two methods are availble. Either the user employs an ssh session to connect to a running instance or he defines a Dockerfile that builds upon the smartgears-base-image by making the required changes - such as depolying a war file, creating a user, manipulating config files, etc. An exemplifying Ansible role that adds the Home Library war file to the image is available [https://gitorious.research-infrastructures.eu/infrastructure-management/ansible-playbooks/trees/master/library/roles/smartgears/smartgears-container-home-library here]. Let's take a look at the included Dockerfile:
  
<source lang="xml">
+
<source lang="yaml">
 
FROM smartgears_base_image:"{{ smartgears_distribution_version }}"
 
FROM smartgears_base_image:"{{ smartgears_distribution_version }}"
 
ADD "{{ home_library_artifact_url }}" "{{ catalina_home }}/webapps/"
 
ADD "{{ home_library_artifact_url }}" "{{ catalina_home }}/webapps/"
Line 58: Line 57:
  
  
The ''FROM'' directive tells docker which is the source image, while the ''ADD'' directive tells which file to add to the image and where to place it on the container filesystem. Building this Dockerfile will effectively pack the Home Library war file with the SmartGears base container. Please note that every Ansible variable will need to be replaced manually if the user whishes to manually build the Dockerfile. Assuming that we'll build this new Docker image on the same target host as before, we will run Ansible like this:
+
The ''FROM'' directive tells docker which is the name and tag of the source image, while the ''ADD'' directive tells which file to add to the image and where to place it on the container filesystem. Building this Dockerfile will effectively pack the Home Library war file with the SmartGears base container. Please note that every Ansible variable will need to be replaced manually if the user whishes to manually build the Dockerfile.  
<source lang="xml">
+
 
 +
Assuming that we'll build this new Docker image on the same target host as before, we will run Ansible like this:
 +
<source lang="bash">
 
ansible-playbook d4science-ghn-cluster/smartgears-container-home-library.yml -i dockerbuilder,
 
ansible-playbook d4science-ghn-cluster/smartgears-container-home-library.yml -i dockerbuilder,
 
</source>
 
</source>
Line 66: Line 67:
 
Once the building run is finished, the user will be able to check the freshly built image on the target docker host:
 
Once the building run is finished, the user will be able to check the freshly built image on the target docker host:
  
<source lang="xml">
+
<source lang="bash">
 
[root@dockerbuilder]# docker images  
 
[root@dockerbuilder]# docker images  
 
REPOSITORY                        TAG                  IMAGE ID            CREATED            SIZE
 
REPOSITORY                        TAG                  IMAGE ID            CREATED            SIZE
Line 73: Line 74:
 
docker.io/ubuntu                  14.04                67759a80360c        4 days ago          221.4 MB
 
docker.io/ubuntu                  14.04                67759a80360c        4 days ago          221.4 MB
 
</source>
 
</source>
 
  
 
== How to run ==
 
== How to run ==
There are two ways to run a SmartGears Docker image, either by running it with ''"docker run"'' or by executing the ansible-container-bootstrap Ansible role available [https://gitorious.research-infrastructures.eu/infrastructure-management/ansible-playbooks/trees/master/library/roles/smartgears/smartgears-container-bootstrap here].  
+
There are two ways to run a SmartGears Docker image, either by running it with ''"docker run"'' or by executing the ''"ansible-container-bootstrap"'' Ansible role available [https://gitorious.research-infrastructures.eu/infrastructure-management/ansible-playbooks/trees/master/library/roles/smartgears/smartgears-container-bootstrap here].  
The ''smartgears-container-bootstrap'' Ansible role is especially useful when combined with other roles that actually build and configure the container as an all-in-one, build-and-run approach - for example see here (3-how to build).
+
The ''"smartgears-container-bootstrap"'' Ansible role is especially useful when combined with other roles that actually build and configure the container as an all-in-one, build-and-run approach - see the playbook example in [[#How to build]].
  
 
However, if the user whishes to manually execute a SmartGears Docker image, a new container can be instantiated by using Docker directy like this:
 
However, if the user whishes to manually execute a SmartGears Docker image, a new container can be instantiated by using Docker directy like this:
  
<source lang="xml">
+
<source lang="bash">
 
docker run -t -e CONTAINER_TOKENS=5token1,token2,token3 -e CONTAINER_MODE=online -e CONTAINER_HOSTNAME=myhostname -e CONTAINER_PORT=8080 -e CONTAINER_INFRASTRUCTURE=d4science smartgears-home-library:1.10.2-4.9.0-160502
 
docker run -t -e CONTAINER_TOKENS=5token1,token2,token3 -e CONTAINER_MODE=online -e CONTAINER_HOSTNAME=myhostname -e CONTAINER_PORT=8080 -e CONTAINER_INFRASTRUCTURE=d4science smartgears-home-library:1.10.2-4.9.0-160502
 
</source>
 
</source>
Line 88: Line 88:
 
Unless run with the -d (detach) flag, the expected result should be:
 
Unless run with the -d (detach) flag, the expected result should be:
  
<source lang="xml">
+
<source lang="bash">
 
Container configuration done
 
Container configuration done
 
Starting Tomcat7
 
Starting Tomcat7
Line 111: Line 111:
  
 
== Trobleshooting ==
 
== Trobleshooting ==
In order to troubleshoot a deployed SmrtGears instance, here's a quick rundown of commands to execute against a running container:
+
In order to troubleshoot a deployed SmartGears container instance, here's a quick rundown of commands to execute against a running container:
  
 
* check the standard out/err:
 
* check the standard out/err:
<source lang="xml">
+
<source lang="bash">
 
docker logs --follow --details --timestamps container_name
 
docker logs --follow --details --timestamps container_name
 
</source>
 
</source>
  
 
* check Tomcat logs:
 
* check Tomcat logs:
<source lang="xml">
+
<source lang="bash">
 
docker exec -it  container_name cat /var/log/tomcat7/catalina.out
 
docker exec -it  container_name cat /var/log/tomcat7/catalina.out
 
</source>
 
</source>
  
 
* check SmartGears logs:
 
* check SmartGears logs:
<source lang="xml">
+
<source lang="bash">
 
docker exec -it  container_name cat /var/log/tomcat7/ghn.log
 
docker exec -it  container_name cat /var/log/tomcat7/ghn.log
 
</source>
 
</source>
  
 
* attach to the running container instance and execute a bash shell:
 
* attach to the running container instance and execute a bash shell:
<source lang="xml">
+
<source lang="bash">
 
docker exec -it  container_name  /bin/bash
 
docker exec -it  container_name  /bin/bash
 
</source>
 
</source>
  
* Typical errors include:
 
  
<source lang="xml">
+
''Typical errors'' include:
 +
 
 +
<source lang="bash">
 
ERROR ContainerManager: error contacting authorization for token xxx  
 
ERROR ContainerManager: error contacting authorization for token xxx  
 
java.lang.RuntimeException: token required for this method
 
java.lang.RuntimeException: token required for this method
 
</source>
 
</source>
  
Either the SmartGears instance can't contact the authorization endpoint or the token provided is invalid. Double check the provided tokens and that the correct authorization endpoint is configured (endpoints are contained in the common-authorization-xxx.jar withinh the SmartGears distribution) and available.
+
* Either the SmartGears instance can't contact the authorization endpoint or the token provided is invalid. Double check the provided tokens and that the correct authorization endpoint is configured (endpoints are contained in the common-authorization-xxx.jar within the SmartGears distribution) and available.
  
<source lang="xml">
+
<source lang="bash">
 
ERROR ContainerManager: no valid starting token are specified, moving the container to failed
 
ERROR ContainerManager: no valid starting token are specified, moving the container to failed
 
</source>
 
</source>
  
Either the SmartGears instance can't contact the authorization endpoint or the token provided is invalid. Double check the provided tokens and that the correct authorization endpoint is configured (endpoints are contained in the common-authorization-xxx.jar withinh the SmartGears distribution) and available.
+
* Either the SmartGears instance can't contact the authorization endpoint or the token provided is invalid. Double check the provided tokens and that the correct authorization endpoint is configured (endpoints are contained in the common-authorization-xxx.jar within the SmartGears distribution) and available.

Latest revision as of 18:25, 27 December 2017

What is a SmartGears Docker image?

"smartgears-base-image" is a Docker based container image, built upon Ubuntu 14.04 LTS, that incorporates a SmartGears node instance with the required dependencies. Such dependencies consist in either Oracle JDK 8 or OpenJDK 8 and Tomcat 7. Furthermore an SSH server (sshd) is configured and started at the instantiation of the container to allow remote connection and provisioning.

While this approach (using ssh) is generally deemed unnecessary since a user can attach to a running container from the host it's running on, this allows the user to remotely connect and run orchestration/configuration management software such as Ansible to feed further web applications and/or configurations to the SmartGears instance.

Advantages

Docker is the de-facto industry standard in containerization, both when used directly or when leveraged by tools such as Google's Kubernetes. Bringing Docker support to SmartGears is a way to make it more appealing and easier to adopt by parties interested in running SmartGears nodes by using containerization and container-based technologies.

A Docker container can be easily built from a Dockerfile or pulled from a remote repository and, since Docker is crossplatform, it can be executed on any hosting OS, making it easier to deploy, for example, both a development instance locally on the developer's OSX/Linux/Windows workstation and on an on-premises production server or a cloud service such as Google Cloud Platform or Amazon Elastic Container Service. This makes adopting and distributing SmarGears both easier and faster regardless of the OS, virtualization infrastructure or cloud service employed.

How to build

In order to build a SmartGears Docker image the user should either run the smartgears-base-image Ansible role or customize the Dockerfile distributed with said role. The Ansible role is available at CNR's gitorious ansible-playbooks repository, here. In order to execute such role the user needs to run a playbook that is available here or write his own.

Let's take a closer look at the playbook:

- hosts: dockerbuilder 	# this is the target host with a running docker instance, this should be replaced with a resolvable hostname such as "localhost"
  remote_user: root
  vars_files: 		# this are "global" Ansible variables, can be overridden for eg. from the command line 
    - ../library/vars/isti-global.yml
  roles:
    - { role: ../library/roles/smartgears/smartgears-base-image/ }		# this is the role required to build the Smartgears docker image
    - { role: ../library/roles/smartgears/smartgears-container-bootstrap }	# this is another role that actually runs a Smartgears container


Let's assume that we want to build and run a container image locally. After cloning the ansible-playbooks git repo, we access the ansible-playbooks directory and run the following ansible-playbook command:

ansible-playbook d4science-ghn-cluster/smartgears-base-image.yml -i dockerbuilder, -e orchestrator_ssh_key:"xxx" --skip-tags=docker
  • "smartgears-base-image.yml" is a playbook that summons the smartgears-base-image role
  • "-i" tells ansible-playbook the inventory file to use, we're passing the "dockerbuilder" target host directly as an example
  • "-e" is used to override variables from the command line, we're using it to provision a public ssh key that will be used to connect via ssh to a running instance on this image
  • "-skip-tags=docker" tells Ansible to ignore the "docker" role dependency declared in the meta file, this is to avoid running the "docker" role every time we're creating a new image.


This will result in the creation of a SmartGears Docker image. By running "docker images" we should see the image locally available:

[root@dockerbuilder]# docker images 
REPOSITORY                        TAG                  IMAGE ID            CREATED             SIZE
smartgears-base-image-java8-jdk   2.1.3-4.7.0-154641   4028fde4cfbf        13 minutes ago      832.2 MB
docker.io/ubuntu                  14.04                67759a80360c        4 days ago          221.4 MB


Another approach is to use the Dockerfile released within the Ansible role and replace the Ansible variables and conditionals as the user sees fit for his purpose.

Build an image with a bundled app

In order to deploy any web application to a SmartGears container two methods are availble. Either the user employs an ssh session to connect to a running instance or he defines a Dockerfile that builds upon the smartgears-base-image by making the required changes - such as depolying a war file, creating a user, manipulating config files, etc. An exemplifying Ansible role that adds the Home Library war file to the image is available here. Let's take a look at the included Dockerfile:

FROM smartgears_base_image:"{{ smartgears_distribution_version }}"
ADD "{{ home_library_artifact_url }}" "{{ catalina_home }}/webapps/"


The FROM directive tells docker which is the name and tag of the source image, while the ADD directive tells which file to add to the image and where to place it on the container filesystem. Building this Dockerfile will effectively pack the Home Library war file with the SmartGears base container. Please note that every Ansible variable will need to be replaced manually if the user whishes to manually build the Dockerfile.

Assuming that we'll build this new Docker image on the same target host as before, we will run Ansible like this:

ansible-playbook d4science-ghn-cluster/smartgears-container-home-library.yml -i dockerbuilder,


Once the building run is finished, the user will be able to check the freshly built image on the target docker host:

[root@dockerbuilder]# docker images 
REPOSITORY                        TAG                   IMAGE ID            CREATED             SIZE
smartgears-home-library           1.10.2-4.9.0-160502   c8c4ae763012        24 minutes ago      890.2 MB
smartgears-base-image-java8-jdk   2.1.3-4.7.0-154641    4028fde4cfbf        5 hours ago         832.2 MB
docker.io/ubuntu                  14.04                 67759a80360c        4 days ago          221.4 MB

How to run

There are two ways to run a SmartGears Docker image, either by running it with "docker run" or by executing the "ansible-container-bootstrap" Ansible role available here. The "smartgears-container-bootstrap" Ansible role is especially useful when combined with other roles that actually build and configure the container as an all-in-one, build-and-run approach - see the playbook example in #How to build.

However, if the user whishes to manually execute a SmartGears Docker image, a new container can be instantiated by using Docker directy like this:

docker run -t -e CONTAINER_TOKENS=5token1,token2,token3 -e CONTAINER_MODE=online -e CONTAINER_HOSTNAME=myhostname -e CONTAINER_PORT=8080 -e CONTAINER_INFRASTRUCTURE=d4science smartgears-home-library:1.10.2-4.9.0-160502


Unless run with the -d (detach) flag, the expected result should be:

Container configuration done
Starting Tomcat7
 * Starting Tomcat servlet engine tomcat7                                                                                                      [ OK ] 
Starting ssh server in foreground


When deployed, a SmartGears container image executes a shell script that configures the SmartGears instance that resides inside the image itself. The bash shell script can be inspected here. When running a SmartGears container image the script needs several parameters. For more info on the SmartGears node configuration parameters and requirements please refer to the SmartGears wiki page SmartGears_Web_Hosting_Node_(wHN)_Installation.

Here's a break down of each parameter that is being passed to the smartgears_run.sh shell script:

  • CONTAINER_TOKENS - all the tokens required to join a specific infrastructure/VO/VRE, comma separated (see token generation). This parameter is mandatory.
  • CONTAINER_MODE - either online or offline
  • CONTAINER_HOSTNAME - the hostname that the SmartGears instance instance will use. This needs to be the hostname used to generate the token(s).
  • CONTAINER_PORT - the port that gets bound by the SmartGears instance
  • CONTAINER_INFRASTRUCTURE - the infrastructure that the SmartGears instance will join
  • PATCH_COMMON_SCOPES - boolean value, required for devel purposes when running the node out of the distributed infrastructure scopes that are present in the common-scope-maps.jar. Defaults to "0".
  • PATCH_COMMON_AUTHORIZATION - boolean value, required for devel purposes when running the node out of the distributed infastructure authorization endpoints that are present in the common-authorization.jar. Defaults to "0".


Trobleshooting

In order to troubleshoot a deployed SmartGears container instance, here's a quick rundown of commands to execute against a running container:

  • check the standard out/err:
docker logs --follow --details --timestamps container_name
  • check Tomcat logs:
docker exec -it  container_name cat /var/log/tomcat7/catalina.out
  • check SmartGears logs:
docker exec -it  container_name cat /var/log/tomcat7/ghn.log
  • attach to the running container instance and execute a bash shell:
docker exec -it  container_name  /bin/bash


Typical errors include:

ERROR ContainerManager: error contacting authorization for token xxx 
java.lang.RuntimeException: token required for this method
  • Either the SmartGears instance can't contact the authorization endpoint or the token provided is invalid. Double check the provided tokens and that the correct authorization endpoint is configured (endpoints are contained in the common-authorization-xxx.jar within the SmartGears distribution) and available.
ERROR ContainerManager: no valid starting token are specified, moving the container to failed
  • Either the SmartGears instance can't contact the authorization endpoint or the token provided is invalid. Double check the provided tokens and that the correct authorization endpoint is configured (endpoints are contained in the common-authorization-xxx.jar within the SmartGears distribution) and available.