Difference between revisions of "Statistical Algorithms Importer: Python Project FAQ"

From Gcube Wiki
Jump to: navigation, search
(→‎Matplotlib)
(→‎How to use the enumerated parameters)
 
(28 intermediate revisions by 2 users not shown)
Line 3: Line 3:
 
|}
 
|}
  
−
F.A.Q. of Statistical Algorithms Importer (SAI), here are common mistakes we have found in Python Project.
+
F.A.Q. of [[Statistical_Algorithms_Importer|Statistical Algorithms Importer (SAI)]], here are common mistakes we have found in Python Project.
  
 +
== How to use the enumerated parameters ==
 +
:For example we consider this parameter:
 +
[[Image:StatisticalAlgorithmsImporter_InputBasicParamEnumertated0.png|thumb|center|700px|Enumerated Param, SAI]]
 +
 +
:As you can see, it is a Enumerated parameter and it is specified using a pipe-separated list of strings. The parameter will be transformed into a controlled vocabulary and will be displayed via a drop-down list in DataMiner.
 +
[[Image:StatisticalAlgorithmsImporter_InputBasicParamEnumertated1.png|thumb|center|700px|Enumerated Param in DataMiner, SAI]]
 +
 +
== How to use the input and output file parameters ==
 +
:For example we consider Phonebook algorithm:
 +
[[Image:StatisticalAlgorithmsImporter_Phonebook0.png|thumb|center|700px|Phonebook, SAI]]
 +
 +
[[Image:StatisticalAlgorithmsImporter_Phonebook1.png|thumb|center|700px|Phonebook, SAI]]
 +
 +
[[Image:StatisticalAlgorithmsImporter_Phonebook2.png|thumb|center|700px|Phonebook, SAI]]
 +
 +
 +
:DataMiner show the Phonebook algorithm in this way:
 +
[[Image:StatisticalAlgorithmsImporter_Phonebook3.png|thumb|center|700px|Phonebook in DataMiner interface, SAI]]
 +
 +
 +
:Python script code in sample:
 +
<source lang='python'>
 +
#
 +
# author Giancarlo Panichi
 +
#
 +
# Phonebook
 +
#
 +
import sys
 +
import string
 +
 +
def print_numbers(numbers):
 +
    print "Telephone Numbers:"
 +
    for x in numbers.keys():
 +
        value="Name: "+x+" Number: "+numbers[x]
 +
        print value,
 +
       
 +
def lookup_number(numbers,name):
 +
    if numbers.has_key(name):
 +
        return "For "+name+" the number is "+numbers[name]
 +
    else:
 +
        return "For "+name+" was not found"
 +
 +
def load_numbers(numbers,filename):
 +
    with open(filename,"r") as f:
 +
        for in_line in f:           
 +
            [name,number] = string.split(in_line,",")
 +
            numbers[name] = number
 +
   
 +
def save_result(result,filename):
 +
    with open(filename,"w") as f:
 +
        f.write(result)
 +
       
 +
phonebookFile=sys.argv[1]
 +
name=sys.argv[2]
 +
phone_list = {}
 +
 +
load_numbers(phone_list,phonebookFile)
 +
print_numbers(phone_list)
 +
number=lookup_number(phone_list,name)
 +
print number
 +
save_result(number,"result.txt")
 +
 +
</source>
 +
 +
 +
 +
:phonebook.csv:
 +
<pre style="display:block;font-family:monospace;white-space:pre;margin:1em 0;">
 +
Giancarlo,0123456789
 +
Gianpaolo,9876541234
 +
Paolo,1231410414
 +
</pre>
 +
 +
 +
:phonebook result with Giancarlo:
 +
<pre style="display:block;font-family:monospace;white-space:pre;margin:1em 0;">
 +
For Giancarlo the number is 0123456789
 +
</pre>
 +
 +
 +
 +
:Phonebook code:
 +
[[File:Phonebook.zip|Phonebook.zip]]
  
 
== Matplotlib ==
 
== Matplotlib ==
Line 10: Line 93:
  
 
<source lang='python'>
 
<source lang='python'>
−
 
+
#
 +
# author Giancarlo Panichi
 +
#
 +
# Write file image.png
 +
#
 
import matplotlib as mpl
 
import matplotlib as mpl
 
mpl.use('Agg')
 
mpl.use('Agg')
 
import matplotlib.pyplot as plt
 
import matplotlib.pyplot as plt
 
import numpy as np
 
import numpy as np
−
 
 
....
 
....
−
 
−
 
 
plt.imshow(image)
 
plt.imshow(image)
 
plt.savefig('image.png')
 
plt.savefig('image.png')
 
</source>
 
</source>
 +
 +
== Urllib ==
 +
On DataMiner services is installed Urllib3, see:
 +
 +
* http://urllib3.readthedocs.io/en/latest/user-guide.html
 +
 +
<source lang='python'>
 +
#
 +
# author Giancarlo Panichi
 +
#
 +
# Request Url
 +
#
 +
import urllib3
 +
http = urllib3.PoolManager()
 +
r = http.request('GET', 'http://httpbin.org/robots.txt')
 +
 +
print(r.status)
 +
print(r.data)
 +
 +
 +
</source>
 +
 +
 +
== StorageHub ==
 +
:[[StorageHub REST API|StorageHub]] is the service for accessing to the user's D4Science Workspace. Below we show the StorageHubFacilityPython algorithm, it exhibits the interactions with StorageHub through its Rest API:
 +
[[Image:StorageHubFacilityPython0.png|thumb|center|750px|StorageHub Facility Python, SAI]]
 +
 +
:Indicates the I/O parameters:
 +
[[Image:StorageHubFacilityPython1.png|thumb|center|750px|StorageHub Facility Python I/O parameters, SAI]]
 +
 +
:Indicates the python version:
 +
[[Image:StorageHubFacilityPython2.png|thumb|center|750px|StorageHub Facility Python version, SAI]]
 +
 +
:Indicates the main code python:
 +
[[Image:StorageHubFacilityPython3.png|thumb|center|750px|StorageHub Facility Python Code, SAI]]
 +
 +
 +
:DataMiner result:
 +
[[Image:StorageHubFacilityPython4.png|thumb|center|750px|StorageHub Facility Python in DataMiner, SAI]]
 +
 +
:This algorithm shows 5 types of interactions with StorageHub:
 +
:* Get Root Info
 +
:* Get Item Info (requires an itemId as argument1)
 +
:* Get Root Children
 +
:* Get Item Children (requires an itemId as argument1)
 +
:* Item Download (requires an itemId as argument1)
 +
 +
:Python source code of StorageHubFacilityPython:
 +
:[[File:StorageHubFacilityPython.zip|StorageHubFacilityPython.zip]]
 +
 +
:View the demo in DataMiner:
 +
:[https://services.d4science.org/group/rprototypinglab/data-miner?OperatorId=org.gcube.dataanalysis.wps.statisticalmanager.synchserver.mappedclasses.transducerers.STORAGEHUBFACILITYPYTHON StorageHub Facility Python]
 +
  
 
[[Category:Statistical Algorithms Importer]]
 
[[Category:Statistical Algorithms Importer]]

Latest revision as of 11:47, 12 February 2021

F.A.Q. of Statistical Algorithms Importer (SAI), here are common mistakes we have found in Python Project.

How to use the enumerated parameters

For example we consider this parameter:
Enumerated Param, SAI
As you can see, it is a Enumerated parameter and it is specified using a pipe-separated list of strings. The parameter will be transformed into a controlled vocabulary and will be displayed via a drop-down list in DataMiner.
Enumerated Param in DataMiner, SAI

How to use the input and output file parameters

For example we consider Phonebook algorithm:
Phonebook, SAI
Phonebook, SAI
Phonebook, SAI


DataMiner show the Phonebook algorithm in this way:
Phonebook in DataMiner interface, SAI


Python script code in sample:
#
# author Giancarlo Panichi
#
# Phonebook
# 
import sys
import string
 
def print_numbers(numbers):
    print "Telephone Numbers:"
    for x in numbers.keys():
        value="Name: "+x+" Number: "+numbers[x]
        print value,
 
def lookup_number(numbers,name):
    if numbers.has_key(name):
        return "For "+name+" the number is "+numbers[name]
    else:
        return "For "+name+" was not found"
 
def load_numbers(numbers,filename):
    with open(filename,"r") as f:
        for in_line in f:            
            [name,number] = string.split(in_line,",")
            numbers[name] = number
 
def save_result(result,filename):
    with open(filename,"w") as f:
        f.write(result)
 
phonebookFile=sys.argv[1] 
name=sys.argv[2]
phone_list = {}
 
load_numbers(phone_list,phonebookFile)
print_numbers(phone_list)
number=lookup_number(phone_list,name)
print number
save_result(number,"result.txt")


phonebook.csv:
Giancarlo,0123456789
Gianpaolo,9876541234
Paolo,1231410414


phonebook result with Giancarlo:
For Giancarlo the number is 0123456789


Phonebook code:

File:Phonebook.zip

Matplotlib

Matplotlib require the $DISPLAY environment variable which means a running X server. DataMiner service do not allow a running X server session. So your code must be modified: 

#
# author Giancarlo Panichi
#
# Write file image.png
# 
import matplotlib as mpl
mpl.use('Agg')
import matplotlib.pyplot as plt
import numpy as np
....
plt.imshow(image)
plt.savefig('image.png')

Urllib

On DataMiner services is installed Urllib3, see: 

#
# author Giancarlo Panichi
#
# Request Url
# 
import urllib3
http = urllib3.PoolManager()
r = http.request('GET', 'http://httpbin.org/robots.txt')
 
print(r.status)
print(r.data)


StorageHub

StorageHub is the service for accessing to the user's D4Science Workspace. Below we show the StorageHubFacilityPython algorithm, it exhibits the interactions with StorageHub through its Rest API:
StorageHub Facility Python, SAI
Indicates the I/O parameters:
StorageHub Facility Python I/O parameters, SAI
Indicates the python version:
StorageHub Facility Python version, SAI
Indicates the main code python:
StorageHub Facility Python Code, SAI


DataMiner result:
StorageHub Facility Python in DataMiner, SAI
This algorithm shows 5 types of interactions with StorageHub:
  • Get Root Info
  • Get Item Info (requires an itemId as argument1)
  • Get Root Children
  • Get Item Children (requires an itemId as argument1)
  • Item Download (requires an itemId as argument1)
Python source code of StorageHubFacilityPython:
File:StorageHubFacilityPython.zip
View the demo in DataMiner:
StorageHub Facility Python
Retrieved from "https://gcube.wiki.gcube-system.org/index.php?title=Statistical_Algorithms_Importer:_Python_Project_FAQ&oldid=33839"