Difference between revisions of "Statistical Algorithms Importer: Python Project FAQ"
From Gcube Wiki
(Created page with "{| align="right" ||__TOC__ |} F.A.Q. of Statistical Algorithms Importer (SAI), here are common mistakes we have found in Python Project. == Matplotlib == Matplotlib require...") |
(→How to use the enumerated parameters) |
||
(29 intermediate revisions by 2 users not shown) | |||
Line 3: | Line 3: | ||
|} | |} | ||
− | F.A.Q. of Statistical Algorithms Importer (SAI), here are common mistakes we have found in Python Project. | + | F.A.Q. of [[Statistical_Algorithms_Importer|Statistical Algorithms Importer (SAI)]], here are common mistakes we have found in Python Project. |
+ | == How to use the enumerated parameters == | ||
+ | :For example we consider this parameter: | ||
+ | [[Image:StatisticalAlgorithmsImporter_InputBasicParamEnumertated0.png|thumb|center|700px|Enumerated Param, SAI]] | ||
+ | |||
+ | :As you can see, it is a Enumerated parameter and it is specified using a pipe-separated list of strings. The parameter will be transformed into a controlled vocabulary and will be displayed via a drop-down list in DataMiner. | ||
+ | [[Image:StatisticalAlgorithmsImporter_InputBasicParamEnumertated1.png|thumb|center|700px|Enumerated Param in DataMiner, SAI]] | ||
+ | |||
+ | == How to use the input and output file parameters == | ||
+ | :For example we consider Phonebook algorithm: | ||
+ | [[Image:StatisticalAlgorithmsImporter_Phonebook0.png|thumb|center|700px|Phonebook, SAI]] | ||
+ | |||
+ | [[Image:StatisticalAlgorithmsImporter_Phonebook1.png|thumb|center|700px|Phonebook, SAI]] | ||
+ | |||
+ | [[Image:StatisticalAlgorithmsImporter_Phonebook2.png|thumb|center|700px|Phonebook, SAI]] | ||
+ | |||
+ | |||
+ | :DataMiner show the Phonebook algorithm in this way: | ||
+ | [[Image:StatisticalAlgorithmsImporter_Phonebook3.png|thumb|center|700px|Phonebook in DataMiner interface, SAI]] | ||
+ | |||
+ | |||
+ | :Python script code in sample: | ||
+ | <source lang='python'> | ||
+ | # | ||
+ | # author Giancarlo Panichi | ||
+ | # | ||
+ | # Phonebook | ||
+ | # | ||
+ | import sys | ||
+ | import string | ||
+ | |||
+ | def print_numbers(numbers): | ||
+ | print "Telephone Numbers:" | ||
+ | for x in numbers.keys(): | ||
+ | value="Name: "+x+" Number: "+numbers[x] | ||
+ | print value, | ||
+ | |||
+ | def lookup_number(numbers,name): | ||
+ | if numbers.has_key(name): | ||
+ | return "For "+name+" the number is "+numbers[name] | ||
+ | else: | ||
+ | return "For "+name+" was not found" | ||
+ | |||
+ | def load_numbers(numbers,filename): | ||
+ | with open(filename,"r") as f: | ||
+ | for in_line in f: | ||
+ | [name,number] = string.split(in_line,",") | ||
+ | numbers[name] = number | ||
+ | |||
+ | def save_result(result,filename): | ||
+ | with open(filename,"w") as f: | ||
+ | f.write(result) | ||
+ | |||
+ | phonebookFile=sys.argv[1] | ||
+ | name=sys.argv[2] | ||
+ | phone_list = {} | ||
+ | |||
+ | load_numbers(phone_list,phonebookFile) | ||
+ | print_numbers(phone_list) | ||
+ | number=lookup_number(phone_list,name) | ||
+ | print number | ||
+ | save_result(number,"result.txt") | ||
+ | |||
+ | </source> | ||
+ | |||
+ | |||
+ | |||
+ | :phonebook.csv: | ||
+ | <pre style="display:block;font-family:monospace;white-space:pre;margin:1em 0;"> | ||
+ | Giancarlo,0123456789 | ||
+ | Gianpaolo,9876541234 | ||
+ | Paolo,1231410414 | ||
+ | </pre> | ||
+ | |||
+ | |||
+ | :phonebook result with Giancarlo: | ||
+ | <pre style="display:block;font-family:monospace;white-space:pre;margin:1em 0;"> | ||
+ | For Giancarlo the number is 0123456789 | ||
+ | </pre> | ||
+ | |||
+ | |||
+ | |||
+ | :Phonebook code: | ||
+ | [[File:Phonebook.zip|Phonebook.zip]] | ||
== Matplotlib == | == Matplotlib == | ||
Matplotlib require the $DISPLAY environment variable which means a running X server. DataMiner service do not allow a running X server session. So your code must be modified: | Matplotlib require the $DISPLAY environment variable which means a running X server. DataMiner service do not allow a running X server session. So your code must be modified: | ||
− | + | <source lang='python'> | |
+ | # | ||
+ | # author Giancarlo Panichi | ||
+ | # | ||
+ | # Write file image.png | ||
+ | # | ||
import matplotlib as mpl | import matplotlib as mpl | ||
mpl.use('Agg') | mpl.use('Agg') | ||
import matplotlib.pyplot as plt | import matplotlib.pyplot as plt | ||
import numpy as np | import numpy as np | ||
− | |||
.... | .... | ||
+ | plt.imshow(image) | ||
+ | plt.savefig('image.png') | ||
+ | </source> | ||
+ | |||
+ | == Urllib == | ||
+ | On DataMiner services is installed Urllib3, see: | ||
+ | |||
+ | * http://urllib3.readthedocs.io/en/latest/user-guide.html | ||
+ | |||
+ | <source lang='python'> | ||
+ | # | ||
+ | # author Giancarlo Panichi | ||
+ | # | ||
+ | # Request Url | ||
+ | # | ||
+ | import urllib3 | ||
+ | http = urllib3.PoolManager() | ||
+ | r = http.request('GET', 'http://httpbin.org/robots.txt') | ||
+ | |||
+ | print(r.status) | ||
+ | print(r.data) | ||
+ | |||
+ | |||
+ | </source> | ||
+ | |||
+ | |||
+ | == StorageHub == | ||
+ | :[[StorageHub REST API|StorageHub]] is the service for accessing to the user's D4Science Workspace. Below we show the StorageHubFacilityPython algorithm, it exhibits the interactions with StorageHub through its Rest API: | ||
+ | [[Image:StorageHubFacilityPython0.png|thumb|center|750px|StorageHub Facility Python, SAI]] | ||
+ | |||
+ | :Indicates the I/O parameters: | ||
+ | [[Image:StorageHubFacilityPython1.png|thumb|center|750px|StorageHub Facility Python I/O parameters, SAI]] | ||
+ | |||
+ | :Indicates the python version: | ||
+ | [[Image:StorageHubFacilityPython2.png|thumb|center|750px|StorageHub Facility Python version, SAI]] | ||
+ | |||
+ | :Indicates the main code python: | ||
+ | [[Image:StorageHubFacilityPython3.png|thumb|center|750px|StorageHub Facility Python Code, SAI]] | ||
+ | |||
+ | |||
+ | :DataMiner result: | ||
+ | [[Image:StorageHubFacilityPython4.png|thumb|center|750px|StorageHub Facility Python in DataMiner, SAI]] | ||
+ | |||
+ | :This algorithm shows 5 types of interactions with StorageHub: | ||
+ | :* Get Root Info | ||
+ | :* Get Item Info (requires an itemId as argument1) | ||
+ | :* Get Root Children | ||
+ | :* Get Item Children (requires an itemId as argument1) | ||
+ | :* Item Download (requires an itemId as argument1) | ||
+ | :Python source code of StorageHubFacilityPython: | ||
+ | :[[File:StorageHubFacilityPython.zip|StorageHubFacilityPython.zip]] | ||
− | + | :View the demo in DataMiner: | |
− | + | :[https://services.d4science.org/group/rprototypinglab/data-miner?OperatorId=org.gcube.dataanalysis.wps.statisticalmanager.synchserver.mappedclasses.transducerers.STORAGEHUBFACILITYPYTHON StorageHub Facility Python] | |
[[Category:Statistical Algorithms Importer]] | [[Category:Statistical Algorithms Importer]] |
Latest revision as of 11:47, 12 February 2021
F.A.Q. of Statistical Algorithms Importer (SAI), here are common mistakes we have found in Python Project.
How to use the enumerated parameters
- For example we consider this parameter:
- As you can see, it is a Enumerated parameter and it is specified using a pipe-separated list of strings. The parameter will be transformed into a controlled vocabulary and will be displayed via a drop-down list in DataMiner.
How to use the input and output file parameters
- For example we consider Phonebook algorithm:
- DataMiner show the Phonebook algorithm in this way:
- Python script code in sample:
# # author Giancarlo Panichi # # Phonebook # import sys import string def print_numbers(numbers): print "Telephone Numbers:" for x in numbers.keys(): value="Name: "+x+" Number: "+numbers[x] print value, def lookup_number(numbers,name): if numbers.has_key(name): return "For "+name+" the number is "+numbers[name] else: return "For "+name+" was not found" def load_numbers(numbers,filename): with open(filename,"r") as f: for in_line in f: [name,number] = string.split(in_line,",") numbers[name] = number def save_result(result,filename): with open(filename,"w") as f: f.write(result) phonebookFile=sys.argv[1] name=sys.argv[2] phone_list = {} load_numbers(phone_list,phonebookFile) print_numbers(phone_list) number=lookup_number(phone_list,name) print number save_result(number,"result.txt")
- phonebook.csv:
Giancarlo,0123456789 Gianpaolo,9876541234 Paolo,1231410414
- phonebook result with Giancarlo:
For Giancarlo the number is 0123456789
- Phonebook code:
Matplotlib
Matplotlib require the $DISPLAY environment variable which means a running X server. DataMiner service do not allow a running X server session. So your code must be modified:
# # author Giancarlo Panichi # # Write file image.png # import matplotlib as mpl mpl.use('Agg') import matplotlib.pyplot as plt import numpy as np .... plt.imshow(image) plt.savefig('image.png')
Urllib
On DataMiner services is installed Urllib3, see:
# # author Giancarlo Panichi # # Request Url # import urllib3 http = urllib3.PoolManager() r = http.request('GET', 'http://httpbin.org/robots.txt') print(r.status) print(r.data)
StorageHub
- StorageHub is the service for accessing to the user's D4Science Workspace. Below we show the StorageHubFacilityPython algorithm, it exhibits the interactions with StorageHub through its Rest API:
- Indicates the I/O parameters:
- Indicates the python version:
- Indicates the main code python:
- DataMiner result:
- This algorithm shows 5 types of interactions with StorageHub:
- Get Root Info
- Get Item Info (requires an itemId as argument1)
- Get Root Children
- Get Item Children (requires an itemId as argument1)
- Item Download (requires an itemId as argument1)
- Python source code of StorageHubFacilityPython:
- File:StorageHubFacilityPython.zip
- View the demo in DataMiner:
- StorageHub Facility Python