Difference between revisions of "How to use the Data Miner Pool Manager"
| Line 2: | Line 2: | ||
=Data Miner Pool Manager= | =Data Miner Pool Manager= | ||
| − | + | DMPM is a REST service able to rationalize and automatize the current process for publishing SAI algorithms on DataMiner nodes and keep DataMiner cluster updated. | |
| + | |||
| + | The service is now able to: | ||
| + | - accept an algorithm descriptor, including its dependencies (either os, r and custom packages) | ||
| + | - query the IS for dataminers in the current scope | ||
| + | - generate (via templating) ansible playbook, inventory and roles for relevant stuff (algorithm installer, algorithms, dependencies) | ||
| + | - execute ansible playbook on a remote machine hosted at eng (no dataminer there; no need for that at this stage) | ||
| + | - roles/templates for the installation of the algorithm installer and installation of os packages are working fine | ||
| + | |||
| + | Next steps are (list is not exhaustive): | ||
| + | - accept as input the url of an algorithm package (including jar, and metadata) | ||
| + | - complete the role for algorithm installation (i.e. the one wrapping 'addAlgorithm.sh') | ||
| + | - execute the playbook on a machine running a dataminer | ||
| + | - return execution outcome to the caller | ||
| + | |||
| + | |||
| + | Completed features: | ||
| + | - accept as input the url of an algorithm package (including jar, and metadata) | ||
| + | - ansible role for algorithm installation (i.e. the one wrapping 'addAlgorithm.sh') | ||
| + | - support for custom ansible roles (e.g. for CRAN & custom packages) | ||
| + | |||
| + | Ongoing features: | ||
| + | - 'smartgear-isation' of the service is ongoing | ||
| + | |||
| + | One-week from now: | ||
| + | - smartgear-isation of the service | ||
| + | - return (asynchronously) execution outcome to the caller | ||
| + | |||
==API== | ==API== | ||
| + | Currently the service exposes the following REST methods: | ||
| + | |||
| + | |||
| + | -- "addAlgorithmToVRE" returning immediatly the ID of the log | ||
| + | |||
| + | skipjava | ||
| + | ... | ||
| + | |||
| + | |||
| + | -- "getLogById" returning asynchronously the detail of the execution | ||
==Usage== | ==Usage== | ||
==Testing== | ==Testing== | ||
| − | + | DMPM is a SmartGear compliant service. | |
| − | + | An instance has already been deployed and configured at Development level. | |
| − | - | + | |
| + | http://node2-d-d4s.d4science.org:8080 | ||
| + | |||
| + | In order to allow Ansible to access the DataMiner host it is necessary that the SSH key of the host where the Service is deployed is correctly configured at DataMiner host level. | ||
Revision as of 15:04, 3 April 2017
Data Miner Pool Manager
DMPM is a REST service able to rationalize and automatize the current process for publishing SAI algorithms on DataMiner nodes and keep DataMiner cluster updated.
The service is now able to: - accept an algorithm descriptor, including its dependencies (either os, r and custom packages) - query the IS for dataminers in the current scope - generate (via templating) ansible playbook, inventory and roles for relevant stuff (algorithm installer, algorithms, dependencies) - execute ansible playbook on a remote machine hosted at eng (no dataminer there; no need for that at this stage) - roles/templates for the installation of the algorithm installer and installation of os packages are working fine
Next steps are (list is not exhaustive): - accept as input the url of an algorithm package (including jar, and metadata) - complete the role for algorithm installation (i.e. the one wrapping 'addAlgorithm.sh') - execute the playbook on a machine running a dataminer - return execution outcome to the caller
Completed features:
- accept as input the url of an algorithm package (including jar, and metadata)
- ansible role for algorithm installation (i.e. the one wrapping 'addAlgorithm.sh')
- support for custom ansible roles (e.g. for CRAN & custom packages)
Ongoing features: - 'smartgear-isation' of the service is ongoing
One-week from now: - smartgear-isation of the service - return (asynchronously) execution outcome to the caller
API
Currently the service exposes the following REST methods:
-- "addAlgorithmToVRE" returning immediatly the ID of the log
skipjava ...
-- "getLogById" returning asynchronously the detail of the execution
Usage
Testing
DMPM is a SmartGear compliant service. An instance has already been deployed and configured at Development level.
http://node2-d-d4s.d4science.org:8080
In order to allow Ansible to access the DataMiner host it is necessary that the SSH key of the host where the Service is deployed is correctly configured at DataMiner host level.
