Difference between revisions of "Statistical Algorithms Importer: Python Project FAQ"
From Gcube Wiki
(→How to use the input file parameters) |
(→How to use the enumerated parameters) |
||
(21 intermediate revisions by 2 users not shown) | |||
Line 3: | Line 3: | ||
|} | |} | ||
− | F.A.Q. of Statistical Algorithms Importer (SAI), here are common mistakes we have found in Python Project. | + | F.A.Q. of [[Statistical_Algorithms_Importer|Statistical Algorithms Importer (SAI)]], here are common mistakes we have found in Python Project. |
+ | == How to use the enumerated parameters == | ||
+ | :For example we consider this parameter: | ||
+ | [[Image:StatisticalAlgorithmsImporter_InputBasicParamEnumertated0.png|thumb|center|700px|Enumerated Param, SAI]] | ||
− | + | :As you can see, it is a Enumerated parameter and it is specified using a pipe-separated list of strings. The parameter will be transformed into a controlled vocabulary and will be displayed via a drop-down list in DataMiner. | |
− | + | [[Image:StatisticalAlgorithmsImporter_InputBasicParamEnumertated1.png|thumb|center|700px|Enumerated Param in DataMiner, SAI]] | |
− | + | == How to use the input and output file parameters == | |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | == How to use the input file parameters == | + | |
:For example we consider Phonebook algorithm: | :For example we consider Phonebook algorithm: | ||
[[Image:StatisticalAlgorithmsImporter_Phonebook0.png|thumb|center|700px|Phonebook, SAI]] | [[Image:StatisticalAlgorithmsImporter_Phonebook0.png|thumb|center|700px|Phonebook, SAI]] | ||
Line 102: | Line 89: | ||
[[File:Phonebook.zip|Phonebook.zip]] | [[File:Phonebook.zip|Phonebook.zip]] | ||
+ | == Matplotlib == | ||
+ | Matplotlib require the $DISPLAY environment variable which means a running X server. DataMiner service do not allow a running X server session. So your code must be modified: | ||
+ | |||
+ | <source lang='python'> | ||
+ | # | ||
+ | # author Giancarlo Panichi | ||
+ | # | ||
+ | # Write file image.png | ||
+ | # | ||
+ | import matplotlib as mpl | ||
+ | mpl.use('Agg') | ||
+ | import matplotlib.pyplot as plt | ||
+ | import numpy as np | ||
+ | .... | ||
+ | plt.imshow(image) | ||
+ | plt.savefig('image.png') | ||
+ | </source> | ||
+ | |||
+ | == Urllib == | ||
+ | On DataMiner services is installed Urllib3, see: | ||
+ | |||
+ | * http://urllib3.readthedocs.io/en/latest/user-guide.html | ||
+ | |||
+ | <source lang='python'> | ||
+ | # | ||
+ | # author Giancarlo Panichi | ||
+ | # | ||
+ | # Request Url | ||
+ | # | ||
+ | import urllib3 | ||
+ | http = urllib3.PoolManager() | ||
+ | r = http.request('GET', 'http://httpbin.org/robots.txt') | ||
+ | |||
+ | print(r.status) | ||
+ | print(r.data) | ||
+ | |||
+ | |||
+ | </source> | ||
+ | |||
+ | |||
+ | == StorageHub == | ||
+ | :[[StorageHub REST API|StorageHub]] is the service for accessing to the user's D4Science Workspace. Below we show the StorageHubFacilityPython algorithm, it exhibits the interactions with StorageHub through its Rest API: | ||
+ | [[Image:StorageHubFacilityPython0.png|thumb|center|750px|StorageHub Facility Python, SAI]] | ||
+ | |||
+ | :Indicates the I/O parameters: | ||
+ | [[Image:StorageHubFacilityPython1.png|thumb|center|750px|StorageHub Facility Python I/O parameters, SAI]] | ||
+ | |||
+ | :Indicates the python version: | ||
+ | [[Image:StorageHubFacilityPython2.png|thumb|center|750px|StorageHub Facility Python version, SAI]] | ||
+ | |||
+ | :Indicates the main code python: | ||
+ | [[Image:StorageHubFacilityPython3.png|thumb|center|750px|StorageHub Facility Python Code, SAI]] | ||
+ | |||
+ | |||
+ | :DataMiner result: | ||
+ | [[Image:StorageHubFacilityPython4.png|thumb|center|750px|StorageHub Facility Python in DataMiner, SAI]] | ||
+ | |||
+ | :This algorithm shows 5 types of interactions with StorageHub: | ||
+ | :* Get Root Info | ||
+ | :* Get Item Info (requires an itemId as argument1) | ||
+ | :* Get Root Children | ||
+ | :* Get Item Children (requires an itemId as argument1) | ||
+ | :* Item Download (requires an itemId as argument1) | ||
+ | |||
+ | :Python source code of StorageHubFacilityPython: | ||
+ | :[[File:StorageHubFacilityPython.zip|StorageHubFacilityPython.zip]] | ||
+ | |||
+ | :View the demo in DataMiner: | ||
+ | :[https://services.d4science.org/group/rprototypinglab/data-miner?OperatorId=org.gcube.dataanalysis.wps.statisticalmanager.synchserver.mappedclasses.transducerers.STORAGEHUBFACILITYPYTHON StorageHub Facility Python] | ||
[[Category:Statistical Algorithms Importer]] | [[Category:Statistical Algorithms Importer]] |
Latest revision as of 11:47, 12 February 2021
F.A.Q. of Statistical Algorithms Importer (SAI), here are common mistakes we have found in Python Project.
How to use the enumerated parameters
- For example we consider this parameter:
- As you can see, it is a Enumerated parameter and it is specified using a pipe-separated list of strings. The parameter will be transformed into a controlled vocabulary and will be displayed via a drop-down list in DataMiner.
How to use the input and output file parameters
- For example we consider Phonebook algorithm:
- DataMiner show the Phonebook algorithm in this way:
- Python script code in sample:
# # author Giancarlo Panichi # # Phonebook # import sys import string def print_numbers(numbers): print "Telephone Numbers:" for x in numbers.keys(): value="Name: "+x+" Number: "+numbers[x] print value, def lookup_number(numbers,name): if numbers.has_key(name): return "For "+name+" the number is "+numbers[name] else: return "For "+name+" was not found" def load_numbers(numbers,filename): with open(filename,"r") as f: for in_line in f: [name,number] = string.split(in_line,",") numbers[name] = number def save_result(result,filename): with open(filename,"w") as f: f.write(result) phonebookFile=sys.argv[1] name=sys.argv[2] phone_list = {} load_numbers(phone_list,phonebookFile) print_numbers(phone_list) number=lookup_number(phone_list,name) print number save_result(number,"result.txt")
- phonebook.csv:
Giancarlo,0123456789 Gianpaolo,9876541234 Paolo,1231410414
- phonebook result with Giancarlo:
For Giancarlo the number is 0123456789
- Phonebook code:
Matplotlib
Matplotlib require the $DISPLAY environment variable which means a running X server. DataMiner service do not allow a running X server session. So your code must be modified:
# # author Giancarlo Panichi # # Write file image.png # import matplotlib as mpl mpl.use('Agg') import matplotlib.pyplot as plt import numpy as np .... plt.imshow(image) plt.savefig('image.png')
Urllib
On DataMiner services is installed Urllib3, see:
# # author Giancarlo Panichi # # Request Url # import urllib3 http = urllib3.PoolManager() r = http.request('GET', 'http://httpbin.org/robots.txt') print(r.status) print(r.data)
StorageHub
- StorageHub is the service for accessing to the user's D4Science Workspace. Below we show the StorageHubFacilityPython algorithm, it exhibits the interactions with StorageHub through its Rest API:
- Indicates the I/O parameters:
- Indicates the python version:
- Indicates the main code python:
- DataMiner result:
- This algorithm shows 5 types of interactions with StorageHub:
- Get Root Info
- Get Item Info (requires an itemId as argument1)
- Get Root Children
- Get Item Children (requires an itemId as argument1)
- Item Download (requires an itemId as argument1)
- Python source code of StorageHubFacilityPython:
- File:StorageHubFacilityPython.zip
- View the demo in DataMiner:
- StorageHub Facility Python