Difference between revisions of "DataMiner Algorithms"
(Created page with "{| align="right" ||__TOC__ |} The complete list of algorithms supported by the DataMiner service is reported below. {|border="1" cellpadding="5" cel...") |
|||
| (4 intermediate revisions by one other user not shown) | |||
| Line 3: | Line 3: | ||
|} | |} | ||
| − | The complete list of algorithms | + | The complete list of currently active algorithms freely published by the [[DataMiner_Manager | DataMiner]] service is reported below. |
| − | + | <!-- ******************* --> | |
| + | <!-- AQUAMAPS ALGORITHMS --> | ||
| + | <!-- ******************* --> | ||
| − | + | = Aquamaps-related Algorithms = | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == ABSENCE_CELLS_FROM_AQUAMAPS == | |
| − | + | An algorithm producing cells and features (HCAF) for a species containing absence points taken by an Aquamaps Distribution | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ==AQUAMAPS_NATIVE== | |
| − | + | Algorithm for Native Range by Aquamaps on a single node | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == AQUAMAPS_NATIVE_2050 == | |
| − | + | Algorithm for Native Range in 2050 by Aquamaps on a single node | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ==AQUAMAPS_NATIVE_NEURALNETWORK== | |
| − | + | Aquamaps Native Algorithm calculated by a Neural Network. A distribution algorithm that relies on Neural Networks and AquaMaps data for native distributions to generate a table containing species distribution probabilities on half-degree cells. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == AQUAMAPS_SUITABLE == | |
| − | + | Algorithm by Aquamaps on a single node | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == AQUAMAPS_SUITABLE_2050 == | |
| − | + | Algorithm for Suitable Range in 2050 by Aquamaps on a single node | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == AQUAMAPS_SUITABLE_NEURALNETWORK == | |
| − | + | Aquamaps Algorithm for Suitable Environment calculated by Neural Network. A distribution algorithm that relies on Neural Networks and AquaMaps data for suitable distributions to generate a table containing species distribution probabilities on half-degree cells. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == AQUAMAPSNN == | |
| − | + | The AquaMaps model trained using a Feed Forward Neural Network. This is a method to train a generic Feed Forward Artifical Neural Network to be used by the AquaMaps Neural Network algorithm. Produces a trained neural network in the form of a compiled file which can be used later. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == BIOCLIMATE_HCAF == | |
| − | + | A transducer algorithm that generates an Half-degree Cells Authority File (HCAF) dataset for a certain time frame, with environmental parameters used by the AquaMaps approach. Evaluates the climatic changes impact on the variation of the ocean features contained in HCAF tables | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == BIOCLIMATE_HSPEC == | |
| − | + | A transducer algorithm that generates a table containing an estimate of species distributions per half-degree cell (HSPEC) in time. Evaluates the climatic changes impact on species presence. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == BIOCLIMATE_HSPEN == | |
| − | + | A transducer algorithm that generates a table containing species envelops (HSPEN) in time, i.e. models capturing species tolerance with respect to environmental parameters, used by the AquaMaps approach. Evaluates the climatic changes impact on the variation of the salinity values in several ranges of a set of species envelopes | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == HCAF_FILTER == | |
| − | + | An algorithm producing a HCAF table on a selected Bounding Box (default identifies Indonesia) | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == HCAF_INTERPOLATION== | |
| − | + | Evaluates the climatic changes impact on species presence | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == HSPEN == | |
| − | + | The AquMaps HSPEN algorithm. A modeling algorithm that generates a table containing species envelops (HSPEN), i.e. models capturing species tolerance with respect to environmental parameters, to be used by the AquaMaps approach. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == PRESENCE_CELLS_GENERATION == | |
| − | + | An algorithm producing cells and features (HCAF) for a species containing presence points | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | <!-- *************************** --> | |
| − | + | <!-- DATA CLIUSTERING ALGORITHMS --> | |
| − | + | <!-- *************************** --> | |
| − | + | ||
| − | + | ||
| − | + | = Data clustering algorithms = | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == DBSCAN == | |
| − | + | A clustering algorithm for real valued vectors that relies on the density-based spatial clustering of applications with noise (DBSCAN) algorithm. A maximum of 4000 points is allowed. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == KMEANS == | |
| − | + | A clustering algorithm for real valued vectors that relies on the k-means algorithm, i.e. a method aiming to partition n observations into k clusters in which each observation belongs to the cluster with the nearest mean, serving as a prototype of the cluster. A Maximum of 4000 points is allowed. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == LOF == | |
| − | + | Local Outlier Factor (LOF). A clustering algorithm for real valued vectors that relies on Local Outlier Factor algorithm, i.e. an algorithm for finding anomalous data points by measuring the local deviation of a given data point with respect to its neighbours. A Maximum of 4000 points is allowed. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == XMEANS == | |
| − | + | A clustering algorithm for occurrence points that relies on the X-Means algorithm, i.e. an extended version of the K-Means algorithm improved by an Improve-Structure part. A Maximum of 4000 points is allowed. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | <!-- *************************** --> | |
| − | + | <!-- GEOSPATIAL ALGORITHMS --> | |
| − | + | <!-- *************************** --> | |
| − | + | ||
| − | + | ||
| − | + | = Geospatial Data Analytics = | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == ESRI_GRID_EXTRACTION == | |
| − | + | An algorithm to extract values associated to an environmental feature repository (e.g. NETCDF, ASC, GeoTiff files etc. ). A grid of points at a certain resolution is specified by the user and values are associated to the points from the environmental repository. It accepts as one geospatial repository ID (via their UUIDs in the infrastructure spatial data repository - recoverable through the Geoexplorer portlet) or a direct link to a file and the specification about time and space. The algorithm produces one ESRI GRID ASCII file containing the values associated to the selected bounding box. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == MAPS_COMPARISON == | |
| − | + | An algorithm for comparing two OGC/NetCDF maps in seamless way to the user. The algorithm assesses the similarities between two geospatial maps by comparing them in a point-to-point fashion. It accepts as input the two geospatial maps (via their UUIDs in the infrastructure spatial data repository - recoverable through the Geoexplorer portlet) and some parameters affecting the comparison such as the z-index, the time index, the comparison threshold. Note: in the case of WFS layers it makes comparisons on the last feature column. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == TIMEEXTRACTION == | |
| − | + | An algorithm to extract a time series of values associated to a geospatial features repository (e.g. NETCDF, ASC, GeoTiff files etc. ). The algorithm analyses the time series and automatically searches for hidden periodicities. It produces one chart of the time series, one table containing the time series values and possibly the spectrogram. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == TIMEEXTRACTION_TABLE == | |
| − | + | An algorithm to extract a time series of values associated to a table containing geospatial information. The algorithm analyses the time series and automatically searches for hidden periodicities. It produces one chart of the time series, one table containing the time series values and possibly the spectrogram. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == XYEXTRACTOR == | |
| − | + | An algorithm to extract values associated to an environmental feature repository (e.g. NETCDF, ASC, GeoTiff files etc. ). A grid of points at a certain resolution is specified by the user and values are associated to the points from the environmental repository. It accepts as one geospatial repository ID (via their UUIDs in the infrastructure spatial data repository - recoverable through the Geoexplorer portlet) or a direct link to a file and the specification about time and space. The algorithm produces one table containing the values associated to the selected bounding box. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == XYEXTRACTOR_TABLE == | |
| − | + | An algorithm to extract values associated to a table containing geospatial features (e.g. Vessel Routes, Species distribution maps etc. ). A grid of points at a certain resolution is specified by the user and values are associated to the points from the environmental repository. It accepts as one geospatial table and the specification about time and space. The algorithm produces one table containing the values associated to the selected bounding box. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == ZEXTRACTION == | |
| − | + | An algorithm to extract the Z values from a geospatial features repository (e.g. NETCDF, ASC, GeoTiff files etc. ). The algorithm analyses the repository and automatically extracts the Z values according to the resolution wanted by the user. It produces one chart of the Z values and one table containing the values. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == ZEXTRACTION_TABLE == | |
| − | + | An algorithm to extract a time series of values associated to a table containing geospatial information. The algorithm analyses the time series and automatically searches for hidden periodicities. It produces one chart of the time series, one table containing the time series values and possibly the spectrogram. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | <!-- ************************************* --> | |
| − | + | <!-- GEOSPATIAL DATA PUBLISHING ALGORITHMS --> | |
| − | + | <!-- ************************************* --> | |
| − | + | ||
| − | + | ||
| − | + | = Geospatial data publishing = | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == RASTER_DATA_PUBLISHER == | |
| − | + | This algorithm publishes a raster file as a maps or datasets in the e-Infrastructure. NetCDF-CF files are encouraged, as WMS and WCS maps will be produced using this format. For other types of files (GeoTiffs, ASC etc.) only the raw datasets will be published. The resulting map or dataset will be accessible via the VRE GeoExplorer by the VRE participants. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == SHAPEFILE_PUBLISHER == | |
| − | + | An algorithm to publish shapefiles under WMS and WFS standards in the e-Infrastructure. The produced WMS, WFS links are reported as output of this process. The map will be available in the VRE for consultation. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | <!-- *********************** --> | |
| − | + | <!-- OBIS RELATED ALGORITHMS --> | |
| − | + | <!-- *********************** --> | |
| − | + | ||
| − | + | ||
| − | + | = OBIS-related Algorithms = | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == ABSENCE_GENERATION_FROM_OBIS == | |
| − | + | An algorithm to estimate absence records from survey data in OBIS. Based on the work in Coro, G., Magliozzi, C., Berghe, E. V., Bailly, N., Ellenbroek, A., & Pagano, P. (2016). Estimating absence locations of marine species from data of scientific surveys in OBIS. Ecological Modelling, 323, 61-76. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == MOST_OBSERVED_SPECIES == | |
| − | + | An algorithm producing a bar chart for the most observed species in a certain years range (with respect to the OBIS database) | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == MOST_OBSERVED_TAXA == | |
| − | + | An algorithm producing a bar chart for the most observed taxa in a certain years range (with respect to the OBIS database) | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == SPECIES_OBSERVATION_LME_AREA_PER_YEAR== | |
| − | + | Algorithm returning most observed species in a specific years range (data collected from OBIS database). | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == SPECIES_OBSERVATION_MEOW_AREA_PER_YEAR== | |
| − | + | Algorithm returning most observed species in a specific years range (data collected from OBIS database). | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == SPECIES_OBSERVATIONS_PER_AREA == | |
| − | + | An algorithm producing a bar chart for the distribution of a species along a certain type of marine area (e.g. LME or MEOW) | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == SPECIES_OBSERVATIONS_TREND_PER_YEAR == | |
| − | + | An algorithm producing the trend of the observations for a certain species in a certain years range. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == TAXONOMY_OBSERVATIONS_TREND_PER_YEAR == | |
| − | + | Algorithm returning most observations taxonomy trend in a specific years range (with respect to the OBIS database) | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | <!-- ******************* --> | |
| − | + | <!-- Occurrence Data ALGORITHMS --> | |
| − | + | <!-- ******************* --> | |
| − | + | ||
| − | + | ||
| − | + | = Occurrence Data Management Algorithms = | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == OCCURRENCES_DUPLICATES_DELETER == | |
| − | + | A transducer algorithm that produces a duplicate free table of species occurrence points where duplicates have been identified via user defined comparison thresholds. Works with up to 100 000 points | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == OCCURRENCE_ENRICHMENT == | |
| − | + | An algorithm performing occurrences enrichment. Takes as input one table containing occurrence points for a set of species and a list of environmental layer, taken either from the e-infrastructure GeoNetwork (through the GeoExplorer application) or from direct HTTP links. Produces one table reporting the set of environmental values associated to the occurrence points. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == OCCURRENCES_MERGER == | |
| − | + | A transducer algorithm that produces a duplicate-free table resulting from the union of two occurrence points tables where points equivalence is identified via user defined comparison thresholds. Works with up to 10000 points per table. Between two Ocurrence Sets, enrichs the Left Set with the elements of the Right Set that are not in the Left Set. Updates the elements of the Left Set with more recent elements in the Right Set. If one element in the Left Set corresponds to several recent elements in the Right Set, these will be all substituted to the element of the Left Set. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == OCCURRENCES_INTERSECTOR == | |
| − | + | A transducer algorithm that produces a table of species occurrence points that are contained in both the two starting tables where points equivalence is identified via user defined comparison thresholds. Works with up to 10000 points per table. Between two occurrence sets, it keeps the elements of the Right Set that are similar to elements in the Left Set. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == OCCURRENCES_SUBTRACTION == | |
| − | + | A transducer algorithm that produces a table resulting from the difference between two occurrence points tables where points equivalence is identified via user defined comparison thresholds. Works with up to 10000 points per table. Between two Ocurrence Sets, keeps the elements of the Left Set that are not similar to any element in the Right Set. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == OCCURRENCES_MARINE_TERRESTRIAL == | |
| − | + | A transducer algorithm that produces a table containing occurrence points by filtering them by type of area, i.e. by recognising whether they are marine or terrestrial. Works with up to 10000 points per table. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | <!-- ******************* --> | |
| − | + | <!-- RDB ALGORITHMS --> | |
| − | + | <!-- ******************* --> | |
| − | + | ||
| − | + | ||
| − | + | = RBDMS Inspection Algorithms = | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == LISTDBINFO == | |
| − | + | Algorithm that allows to view information about one chosen resource of Database Type in the Infrastructure | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == LISTDBNAMES == | |
| − | + | Algorithm that allows to view the available database resources names in the Infrastructure | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == LISTDBSCHEMA == | |
| − | + | Algorithm that allows to view the schema names of a chosen database for which the type is Postgres | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == LISTTABLES == | |
| − | + | Algorithm that allows to view the table names of a chosen database | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | <!-- *************************** --> | |
| − | + | <!-- TO BE CLASSIFIED ALGORITHMS --> | |
| − | + | <!-- *************************** --> | |
| − | + | ||
| − | + | ||
| − | + | = Other Algorithms = | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == BIONYM == | |
| − | + | An algorithm implementing BiOnym, a flexible workflow approach to taxon name matching. The workflow allows to activate several taxa names matching algorithms and to get the list of possible transcriptions for a list of input raw species names with possible authorship indication. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == BIONYM_BIODIV == | |
| − | + | An algorithm implementing BiOnym oriented to Biodiversity Taxa Names Matching with a predefined and optimized workflow. This version applies in sequence the following Matchers: GSay (thr:0.6, maxRes:10), FuzzyMatcher (thr:0.6, maxRes:10), Levenshtein (thr:0.4, maxRes:10), Trigram (thr:0.4, maxRes:10). BiOnym is a flexible workflow approach to taxon name matching. The workflow allows to activate several taxa names matching algorithms and to get the list of possible transcriptions for a list of input raw species names with possible authorship indication. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == BIONYM_LOCAL == | |
| − | + | A fast version of the algorithm implementing BiOnym, a flexible workflow approach to taxon name matching. The workflow allows to activate several taxa names matching algorithms and to get the list of possible transcriptions for a list of input raw species names with possible authorship indication. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == CMSY_2 == | |
| − | + | The CMSY method for data-limited stock assessment. Described in Froese, R., Demirel, N., Coro, G., Kleisner, K. M., Winker, H. (2016). Estimating fisheries reference points from catch and resilience. Fish and Fisheries. Paper link: http://onlinelibrary.wiley.com/doi/10.1111/faf.12190/ Full Instructions and code: https://github.com/SISTA16/cmsy | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == CSQUARE_COLUMN_CREATOR == | |
| − | + | An algorithm that adds a column containing the CSquare codes associated to longitude and latitude columns. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == DISCREPANCY_ANALYSIS == | |
| − | + | An evaluator algorithm that compares two tables containing real valued vectors. It drives the comparison by relying on a geographical distance threshold and a threshold for K-Statistic. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == ECOPATH_WITH_ECOSIM == | |
| − | + | Ecopath with Ecosim (EwE) is a free ecological/ecosystem modeling software suite. This algorithm implementation expects a model and a configuration file as inputs; the result of the analysis is returned as a zip archive. References: Christensen, V., & Walters, C. J. (2004). Ecopath with Ecosim: methods, capabilities and limitations. Ecological modelling, 172(2), 109-139. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == ESTIMATE_FISHING_ACTIVITY == | |
| − | + | An algorithm that estimates activity hours (fishing or other) from vessels trajectories, adds bathymetry information to the table and classifies (point-by-point) fishing activity of the involved vessels according to two algorithms: one based on speed (activity_class_speed output column) and the other based on speed and bathymetry (activity_class_speed_bath output column). The algorithm produces new columns containing this information. This algorithm is based on the paper 'Deriving Fishing Monthly Effort and Caught Species' (Coro et al. 2013, in proc. of OCEANS - Bergen, 2013 MTS/IEEE). Example of input table (NAFO anonymised data): http://goo.gl/3auJkM | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == ESTIMATE_MONTHLY_FISHING_EFFORT == | |
| − | + | An algorithm that estimates fishing exploitation at 0.5 degrees resolution from activity-classified vessels trajectories. Produces a table with csquare codes, latitudes, longitudes and resolution and associated overall fishing hours in the time frame of the vessels activity. Requires each activity point to be classified as Fishing or other. This algorithm is based on the paper 'Deriving Fishing Monthly Effort and Caught Species' (Coro et al. 2013, in proc. of OCEANS - Bergen, 2013 MTS/IEEE). Example of input table (NAFO anonymised data): http://goo.gl/3auJkM | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == FAOMSY == | |
| − | + | An algorithm to be used by Fisheries managers for stock assessment. Estimates the Maximum Sustainable Yield (MSY) of a stock, based on a catch trend. The algorithm has been developed by the Resource Use and Conservation Division of the FAO Fisheries and Aquaculture Department (contact: Yimin Ye, yimin.ye@fao.org). It is applicable to a CSV file containing metadata and catch statistics for a set of marine species and produces MSY estimates for each species. The CSV must follow a FAO-defined format (e.g. http://goo.gl/g6YtVx). The output is made up of two (optional) files: one for sucessfully processed species and another one for species that could not be processed because data were not sufficient to estimate MSY. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == FAO_OCEAN_AREA_COLUMN_CREATOR == | |
| − | + | An algorithm that adds a column containing the FAO Ocean Area codes associated to longitude and latitude columns. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == FAO_OCEAN_AREA_COLUMN_CREATOR_FROM_QUADRANT == | |
| − | + | An algorithm that adds a column containing the FAO Ocean Area codes associated to longitude, latitude and quadrant columns. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == FEED_FORWARD_A_N_N_DISTRIBUTION == | |
| − | + | A Bayesian method using a Feed Forward Neural Network to simulate a function from the features space (R^n) to R. A modeling algorithm that relies on Neural Networks to simulate a real valued function. It accepts as input a table containing the training dataset and some parameters affecting the algorithm behaviour such as the number of neurons, the learning threshold and the maximum number of iterations. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == FEED_FORWARD_ANN == | |
| − | + | A method to train a generic Feed Forward Artifical Neural Network in order to simulate a function from the features space (R^n) to R. Uses the Back-propagation method. Produces a trained neural network in the form of a compiled file which can be used in the FEED FORWARD NEURAL NETWORK DISTRIBUTION algorithm. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == FEED_FORWARD_NEURAL_NETWORK_REGRESSOR == | |
| − | + | The algorithm simulates a real-valued vector function using a trained Feed Forward Artificial Neural Network and returns a table containing the function actual inputs and the predicted outputs | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == FEED_FORWARD_NEURAL_NETWORK_TRAINER == | |
| − | + | The algorithm trains a Feed Forward Artificial Neural Network using an online Back-Propagation procedure and returns the training error and a binary file containing the trained network | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == FIGIS_SDMX_DATA_CONVERTER == | |
| − | + | This tool allows to convert easily a SDMX dataset into CSV, by calling the rsdmx package for R | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == FIGIS_SPATIAL_REALLOCATION_GENERIC == | |
| − | + | The Spatial Reallocation algorithm allows to estimate statistics for other areas from those where they were reported. The algorithm is based on spatial disaggregation techniques and provides at now an area-weighted reallocation. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == FIGIS_SPATIAL_REALLOCATION_SIMPLIFIED == | |
| − | + | The Spatial Reallocaton algorithm allows to estimate statistics for other areas from those where they were reported. The algorithm is based on spatial disaggregation technics and provides at now an area-weighted reallocation. This simplified algorithm is specifically targeting users from the FAO Fisheries and Aquaculture department, aims to facilitate its execution by doing abstraction of the intersections to provide. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == FIGIS_SPATIAL_REALLOCATION_SIMPLIFIED_TABLE == | |
| − | + | The Spatial Reallocaton algorithm allows to estimate statistics for other areas from those where they were reported. The algorithm is based on spatial disaggregation technics and provides at now an area-weighted reallocation. This simplified algorithm is specifically targeting users from the FAO Fisheries and Aquaculture department, aims to facilitate its execution by doing abstraction of the intersections to provide. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == GENERIC_CHARTS == | |
| − | + | An algorithm producing generic charts of attributes vs. quantities. Charts are displayed per quantity column. Histograms, Scattering and Radar charts are produced for the top ten quantities. A gaussian distribution reports overall statistics for the quantities. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == GEO_CHART == | |
| − | + | An algorithm producing a charts that displays quantities as colors of countries. The color indicates the sum of the values recorded in a country. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == GRID_CWP_TO_COORDINATES == | |
| − | + | An algorithm that adds longitude, latitude and resolution columns analysing a column containing FAO Ocean Area codes (CWP format). | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == HRS == | |
| − | + | An evaluator algorithm that calculates the Habitat Representativeness Score, i.e. an indicator of the assessment of whether a specific survey coverage or another environmental features dataset, contains data that are representative of all available habitat variable combinations in an area. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == LWR == | |
| − | + | An algorithm to estimate Length-Weight relationship parameters for marine species, using Bayesian methods. Runs an R procedure. Based on the Cube-law theory. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == MAX_ENT_NICHE_MODELLING == | |
| − | + | A Maximum-Entropy model for species habitat modeling, based on the implementation by Shapire et al. v 3.3.3k, Princeton University, http://www.cs.princeton.edu/schapire/maxent/. In this adaptation for the D4Science infrastructure, the software accepts a table produced by the Species Product Discovery service and a set of environmental layers in various formats (NetCDF, WFS, WCS, ASC, GeoTiff) via direct links or GeoExplorer UUIDs. The user can also establish the bounding box and the spatial resolution (in decimal deg.) of the training and the projection. The application will adapt the layers to that resolution if this is higher than the native one.The output contains: a thumbnail map of the projected model, the ROC curve, the Omission/Commission chart, a table containing the raw assigned values, a threshold to transform the table into a 0-1 probability distribution, a report of the importance of the used layers in the model, ASCII representations of the input layers to check their alignment.Other processes can be later applied to the raw values to produce a GIS map (e.g. the Statistical Manager Points-to-Map process) and results can be shared. Demo video: http://goo.gl/TYYnTO and instructions http://wiki.i-marine.eu/index.php/MaxEnt | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == POINTS_TO_MAP == | |
| − | + | A transducer algorithm to produce a GIS map of points from a set of points with x,y coordinates indications. A maximum of 259000 is allowed | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == POLYGONS_TO_MAP == | |
| − | + | A transducer algorithm to produce a GIS map of filled polygons associated to x,y coordinates and a certain resolution. A maximum of 259000 is allowed | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == QUALITY_ANALYSIS == | |
| − | + | An evaluator algorithm that assesses the effectiveness of a distribution model by computing the Receiver Operating Characteristics (ROC), the Area Under Curve (AUC) and the Accuracy of a model | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == SEADATANET_INTERPOLATOR == | |
| − | + | A connector for the SeaDataNet infrastructure. This algorithms invokes the Data-Interpolating Variational Analysis (DIVA) SeaDataNet service to interpolate spatial data. The model uses GEBCO bathymetry data and requires an estimate of the maximum spatial span of the correlation between points and the signal-to-noise ratio, among the other parameters. It can interpolate up to 10,000 points randomly taken from the input table. As output, it produces a NetCDF file with a uniform grid of values. This powerful interpolation model is described in Troupin et al. 2012, 'Generation of analysis and consistent error fields using the Data Interpolating Variational Analysis (Diva)', Ocean Modelling, 52-53, 90-101. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == SGVM_INTERPOLATION == | |
| − | + | An interpolation method relying on the implementation by the Study Group on VMS (SGVMS). The method uses two interpolation approached to simulate vessels points at a certain temporal resolution. The input is a file in TACSAT format uploaded on the Statistical Manager. The output is another TACSAT file containing interpolated points.The underlying R code has been extracted from the SGVM VMSTools framework. This algorithm comes after a feasibility study (http://goo.gl/risQre) which clarifies the features an e-Infrastructure adds to the original scripts. Limitation: the input will be processed up to 10000 vessels trajectory points. Credits: Hintzen, N. T., Bastardie, F., Beare, D., Piet, G. J., Ulrich, C., Deporte, N., Egekvist, J., et al. 2012. VMStools: Open-source software for the processing, analysis and visualisation of fisheries logbook and VMS data. Fisheries Research, 115-116: 31-43. Hintzen, N. T., Piet, G. J., and Brunel, T. 2010. Improved estimation of trawling tracks using cubic Hermite spline interpolation of position registration data. Fisheries Research, 101: 108-115. VMStools, available as an add-on package for R. Documentation available at https://code.google.com/p/vmstools/. Build versions of VMStools for Window, Mac, Linux available at https://docs.google.com/. Authors: Niels T. Hintzen, Doug Beare | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == SPECIES_MAP_FROM_CSQUARES == | |
| − | + | A transducer algorithm to produce a GIS map from a probability distribution associated to a set of csquare codes. A maximum of 259000 is allowed | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == SPECIES_MAP_FROM_POINTS == | |
| − | + | A transducer algorithm to produce a GIS map from a probability distribution made of x,y coordinates and a certain resolution. A maximum of 259000 is allowed | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == SUPPORT_VECTOR_MACHINE_REGRESSOR== | |
| − | + | A simple algorithm for regression using and already trained Support Vector Machine | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == SUPPORT_VECTOR_MACHINE_TRAINER == | |
| − | + | A simple algorithm to train a Support Vector Machine | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == TIME_GEO_CHART == | |
| − | + | An algorithm producing an animated gif displaying quantities as colors in time. The color indicates the sum of the values recorded in a country. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == TIME_SERIES_ANALYSIS == | |
| − | + | An algorithms applying signal processing to a non uniform time series. A maximum of 10000 distinct points in time is allowed to be processed. The process uniformly samples the series, then extracts hidden periodicities and signal properties. The sampling period is the shortest time difference between two points. Finally, by using Caterpillar-SSA the algorithm forecasts the Time Series. The output shows the detected periodicity, the forecasted signal and the spectrogram. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ==TIME_SERIES_CHARTS== | |
| − | + | An algorithm producing time series charts of attributes vs. quantities. Charts are displayed per quantity column and superposing quantities are summed. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == WEB_APP_PUBLISHER == | |
| − | + | This algorithm publishes a zip file containing a Web site, based on html and javascript in the e-Infrastructure. It generates a public URL to the application that can be shared. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
Latest revision as of 10:55, 30 November 2017
The complete list of currently active algorithms freely published by the DataMiner service is reported below.
ABSENCE_CELLS_FROM_AQUAMAPS
An algorithm producing cells and features (HCAF) for a species containing absence points taken by an Aquamaps Distribution
AQUAMAPS_NATIVE
Algorithm for Native Range by Aquamaps on a single node
AQUAMAPS_NATIVE_2050
Algorithm for Native Range in 2050 by Aquamaps on a single node
AQUAMAPS_NATIVE_NEURALNETWORK
Aquamaps Native Algorithm calculated by a Neural Network. A distribution algorithm that relies on Neural Networks and AquaMaps data for native distributions to generate a table containing species distribution probabilities on half-degree cells.
AQUAMAPS_SUITABLE
Algorithm by Aquamaps on a single node
AQUAMAPS_SUITABLE_2050
Algorithm for Suitable Range in 2050 by Aquamaps on a single node
AQUAMAPS_SUITABLE_NEURALNETWORK
Aquamaps Algorithm for Suitable Environment calculated by Neural Network. A distribution algorithm that relies on Neural Networks and AquaMaps data for suitable distributions to generate a table containing species distribution probabilities on half-degree cells.
AQUAMAPSNN
The AquaMaps model trained using a Feed Forward Neural Network. This is a method to train a generic Feed Forward Artifical Neural Network to be used by the AquaMaps Neural Network algorithm. Produces a trained neural network in the form of a compiled file which can be used later.
BIOCLIMATE_HCAF
A transducer algorithm that generates an Half-degree Cells Authority File (HCAF) dataset for a certain time frame, with environmental parameters used by the AquaMaps approach. Evaluates the climatic changes impact on the variation of the ocean features contained in HCAF tables
BIOCLIMATE_HSPEC
A transducer algorithm that generates a table containing an estimate of species distributions per half-degree cell (HSPEC) in time. Evaluates the climatic changes impact on species presence.
BIOCLIMATE_HSPEN
A transducer algorithm that generates a table containing species envelops (HSPEN) in time, i.e. models capturing species tolerance with respect to environmental parameters, used by the AquaMaps approach. Evaluates the climatic changes impact on the variation of the salinity values in several ranges of a set of species envelopes
HCAF_FILTER
An algorithm producing a HCAF table on a selected Bounding Box (default identifies Indonesia)
HCAF_INTERPOLATION
Evaluates the climatic changes impact on species presence
HSPEN
The AquMaps HSPEN algorithm. A modeling algorithm that generates a table containing species envelops (HSPEN), i.e. models capturing species tolerance with respect to environmental parameters, to be used by the AquaMaps approach.
PRESENCE_CELLS_GENERATION
An algorithm producing cells and features (HCAF) for a species containing presence points
Data clustering algorithms
DBSCAN
A clustering algorithm for real valued vectors that relies on the density-based spatial clustering of applications with noise (DBSCAN) algorithm. A maximum of 4000 points is allowed.
KMEANS
A clustering algorithm for real valued vectors that relies on the k-means algorithm, i.e. a method aiming to partition n observations into k clusters in which each observation belongs to the cluster with the nearest mean, serving as a prototype of the cluster. A Maximum of 4000 points is allowed.
LOF
Local Outlier Factor (LOF). A clustering algorithm for real valued vectors that relies on Local Outlier Factor algorithm, i.e. an algorithm for finding anomalous data points by measuring the local deviation of a given data point with respect to its neighbours. A Maximum of 4000 points is allowed.
XMEANS
A clustering algorithm for occurrence points that relies on the X-Means algorithm, i.e. an extended version of the K-Means algorithm improved by an Improve-Structure part. A Maximum of 4000 points is allowed.
Geospatial Data Analytics
ESRI_GRID_EXTRACTION
An algorithm to extract values associated to an environmental feature repository (e.g. NETCDF, ASC, GeoTiff files etc. ). A grid of points at a certain resolution is specified by the user and values are associated to the points from the environmental repository. It accepts as one geospatial repository ID (via their UUIDs in the infrastructure spatial data repository - recoverable through the Geoexplorer portlet) or a direct link to a file and the specification about time and space. The algorithm produces one ESRI GRID ASCII file containing the values associated to the selected bounding box.
MAPS_COMPARISON
An algorithm for comparing two OGC/NetCDF maps in seamless way to the user. The algorithm assesses the similarities between two geospatial maps by comparing them in a point-to-point fashion. It accepts as input the two geospatial maps (via their UUIDs in the infrastructure spatial data repository - recoverable through the Geoexplorer portlet) and some parameters affecting the comparison such as the z-index, the time index, the comparison threshold. Note: in the case of WFS layers it makes comparisons on the last feature column.
TIMEEXTRACTION
An algorithm to extract a time series of values associated to a geospatial features repository (e.g. NETCDF, ASC, GeoTiff files etc. ). The algorithm analyses the time series and automatically searches for hidden periodicities. It produces one chart of the time series, one table containing the time series values and possibly the spectrogram.
TIMEEXTRACTION_TABLE
An algorithm to extract a time series of values associated to a table containing geospatial information. The algorithm analyses the time series and automatically searches for hidden periodicities. It produces one chart of the time series, one table containing the time series values and possibly the spectrogram.
XYEXTRACTOR
An algorithm to extract values associated to an environmental feature repository (e.g. NETCDF, ASC, GeoTiff files etc. ). A grid of points at a certain resolution is specified by the user and values are associated to the points from the environmental repository. It accepts as one geospatial repository ID (via their UUIDs in the infrastructure spatial data repository - recoverable through the Geoexplorer portlet) or a direct link to a file and the specification about time and space. The algorithm produces one table containing the values associated to the selected bounding box.
XYEXTRACTOR_TABLE
An algorithm to extract values associated to a table containing geospatial features (e.g. Vessel Routes, Species distribution maps etc. ). A grid of points at a certain resolution is specified by the user and values are associated to the points from the environmental repository. It accepts as one geospatial table and the specification about time and space. The algorithm produces one table containing the values associated to the selected bounding box.
ZEXTRACTION
An algorithm to extract the Z values from a geospatial features repository (e.g. NETCDF, ASC, GeoTiff files etc. ). The algorithm analyses the repository and automatically extracts the Z values according to the resolution wanted by the user. It produces one chart of the Z values and one table containing the values.
ZEXTRACTION_TABLE
An algorithm to extract a time series of values associated to a table containing geospatial information. The algorithm analyses the time series and automatically searches for hidden periodicities. It produces one chart of the time series, one table containing the time series values and possibly the spectrogram.
Geospatial data publishing
RASTER_DATA_PUBLISHER
This algorithm publishes a raster file as a maps or datasets in the e-Infrastructure. NetCDF-CF files are encouraged, as WMS and WCS maps will be produced using this format. For other types of files (GeoTiffs, ASC etc.) only the raw datasets will be published. The resulting map or dataset will be accessible via the VRE GeoExplorer by the VRE participants.
SHAPEFILE_PUBLISHER
An algorithm to publish shapefiles under WMS and WFS standards in the e-Infrastructure. The produced WMS, WFS links are reported as output of this process. The map will be available in the VRE for consultation.
ABSENCE_GENERATION_FROM_OBIS
An algorithm to estimate absence records from survey data in OBIS. Based on the work in Coro, G., Magliozzi, C., Berghe, E. V., Bailly, N., Ellenbroek, A., & Pagano, P. (2016). Estimating absence locations of marine species from data of scientific surveys in OBIS. Ecological Modelling, 323, 61-76.
MOST_OBSERVED_SPECIES
An algorithm producing a bar chart for the most observed species in a certain years range (with respect to the OBIS database)
MOST_OBSERVED_TAXA
An algorithm producing a bar chart for the most observed taxa in a certain years range (with respect to the OBIS database)
SPECIES_OBSERVATION_LME_AREA_PER_YEAR
Algorithm returning most observed species in a specific years range (data collected from OBIS database).
SPECIES_OBSERVATION_MEOW_AREA_PER_YEAR
Algorithm returning most observed species in a specific years range (data collected from OBIS database).
SPECIES_OBSERVATIONS_PER_AREA
An algorithm producing a bar chart for the distribution of a species along a certain type of marine area (e.g. LME or MEOW)
SPECIES_OBSERVATIONS_TREND_PER_YEAR
An algorithm producing the trend of the observations for a certain species in a certain years range.
TAXONOMY_OBSERVATIONS_TREND_PER_YEAR
Algorithm returning most observations taxonomy trend in a specific years range (with respect to the OBIS database)
Occurrence Data Management Algorithms
OCCURRENCES_DUPLICATES_DELETER
A transducer algorithm that produces a duplicate free table of species occurrence points where duplicates have been identified via user defined comparison thresholds. Works with up to 100 000 points
OCCURRENCE_ENRICHMENT
An algorithm performing occurrences enrichment. Takes as input one table containing occurrence points for a set of species and a list of environmental layer, taken either from the e-infrastructure GeoNetwork (through the GeoExplorer application) or from direct HTTP links. Produces one table reporting the set of environmental values associated to the occurrence points.
OCCURRENCES_MERGER
A transducer algorithm that produces a duplicate-free table resulting from the union of two occurrence points tables where points equivalence is identified via user defined comparison thresholds. Works with up to 10000 points per table. Between two Ocurrence Sets, enrichs the Left Set with the elements of the Right Set that are not in the Left Set. Updates the elements of the Left Set with more recent elements in the Right Set. If one element in the Left Set corresponds to several recent elements in the Right Set, these will be all substituted to the element of the Left Set.
OCCURRENCES_INTERSECTOR
A transducer algorithm that produces a table of species occurrence points that are contained in both the two starting tables where points equivalence is identified via user defined comparison thresholds. Works with up to 10000 points per table. Between two occurrence sets, it keeps the elements of the Right Set that are similar to elements in the Left Set.
OCCURRENCES_SUBTRACTION
A transducer algorithm that produces a table resulting from the difference between two occurrence points tables where points equivalence is identified via user defined comparison thresholds. Works with up to 10000 points per table. Between two Ocurrence Sets, keeps the elements of the Left Set that are not similar to any element in the Right Set.
OCCURRENCES_MARINE_TERRESTRIAL
A transducer algorithm that produces a table containing occurrence points by filtering them by type of area, i.e. by recognising whether they are marine or terrestrial. Works with up to 10000 points per table.
RBDMS Inspection Algorithms
LISTDBINFO
Algorithm that allows to view information about one chosen resource of Database Type in the Infrastructure
LISTDBNAMES
Algorithm that allows to view the available database resources names in the Infrastructure
LISTDBSCHEMA
Algorithm that allows to view the schema names of a chosen database for which the type is Postgres
LISTTABLES
Algorithm that allows to view the table names of a chosen database
Other Algorithms
BIONYM
An algorithm implementing BiOnym, a flexible workflow approach to taxon name matching. The workflow allows to activate several taxa names matching algorithms and to get the list of possible transcriptions for a list of input raw species names with possible authorship indication.
BIONYM_BIODIV
An algorithm implementing BiOnym oriented to Biodiversity Taxa Names Matching with a predefined and optimized workflow. This version applies in sequence the following Matchers: GSay (thr:0.6, maxRes:10), FuzzyMatcher (thr:0.6, maxRes:10), Levenshtein (thr:0.4, maxRes:10), Trigram (thr:0.4, maxRes:10). BiOnym is a flexible workflow approach to taxon name matching. The workflow allows to activate several taxa names matching algorithms and to get the list of possible transcriptions for a list of input raw species names with possible authorship indication.
BIONYM_LOCAL
A fast version of the algorithm implementing BiOnym, a flexible workflow approach to taxon name matching. The workflow allows to activate several taxa names matching algorithms and to get the list of possible transcriptions for a list of input raw species names with possible authorship indication.
CMSY_2
The CMSY method for data-limited stock assessment. Described in Froese, R., Demirel, N., Coro, G., Kleisner, K. M., Winker, H. (2016). Estimating fisheries reference points from catch and resilience. Fish and Fisheries. Paper link: http://onlinelibrary.wiley.com/doi/10.1111/faf.12190/ Full Instructions and code: https://github.com/SISTA16/cmsy
CSQUARE_COLUMN_CREATOR
An algorithm that adds a column containing the CSquare codes associated to longitude and latitude columns.
DISCREPANCY_ANALYSIS
An evaluator algorithm that compares two tables containing real valued vectors. It drives the comparison by relying on a geographical distance threshold and a threshold for K-Statistic.
ECOPATH_WITH_ECOSIM
Ecopath with Ecosim (EwE) is a free ecological/ecosystem modeling software suite. This algorithm implementation expects a model and a configuration file as inputs; the result of the analysis is returned as a zip archive. References: Christensen, V., & Walters, C. J. (2004). Ecopath with Ecosim: methods, capabilities and limitations. Ecological modelling, 172(2), 109-139.
ESTIMATE_FISHING_ACTIVITY
An algorithm that estimates activity hours (fishing or other) from vessels trajectories, adds bathymetry information to the table and classifies (point-by-point) fishing activity of the involved vessels according to two algorithms: one based on speed (activity_class_speed output column) and the other based on speed and bathymetry (activity_class_speed_bath output column). The algorithm produces new columns containing this information. This algorithm is based on the paper 'Deriving Fishing Monthly Effort and Caught Species' (Coro et al. 2013, in proc. of OCEANS - Bergen, 2013 MTS/IEEE). Example of input table (NAFO anonymised data): http://goo.gl/3auJkM
ESTIMATE_MONTHLY_FISHING_EFFORT
An algorithm that estimates fishing exploitation at 0.5 degrees resolution from activity-classified vessels trajectories. Produces a table with csquare codes, latitudes, longitudes and resolution and associated overall fishing hours in the time frame of the vessels activity. Requires each activity point to be classified as Fishing or other. This algorithm is based on the paper 'Deriving Fishing Monthly Effort and Caught Species' (Coro et al. 2013, in proc. of OCEANS - Bergen, 2013 MTS/IEEE). Example of input table (NAFO anonymised data): http://goo.gl/3auJkM
FAOMSY
An algorithm to be used by Fisheries managers for stock assessment. Estimates the Maximum Sustainable Yield (MSY) of a stock, based on a catch trend. The algorithm has been developed by the Resource Use and Conservation Division of the FAO Fisheries and Aquaculture Department (contact: Yimin Ye, yimin.ye@fao.org). It is applicable to a CSV file containing metadata and catch statistics for a set of marine species and produces MSY estimates for each species. The CSV must follow a FAO-defined format (e.g. http://goo.gl/g6YtVx). The output is made up of two (optional) files: one for sucessfully processed species and another one for species that could not be processed because data were not sufficient to estimate MSY.
FAO_OCEAN_AREA_COLUMN_CREATOR
An algorithm that adds a column containing the FAO Ocean Area codes associated to longitude and latitude columns.
FAO_OCEAN_AREA_COLUMN_CREATOR_FROM_QUADRANT
An algorithm that adds a column containing the FAO Ocean Area codes associated to longitude, latitude and quadrant columns.
FEED_FORWARD_A_N_N_DISTRIBUTION
A Bayesian method using a Feed Forward Neural Network to simulate a function from the features space (R^n) to R. A modeling algorithm that relies on Neural Networks to simulate a real valued function. It accepts as input a table containing the training dataset and some parameters affecting the algorithm behaviour such as the number of neurons, the learning threshold and the maximum number of iterations.
FEED_FORWARD_ANN
A method to train a generic Feed Forward Artifical Neural Network in order to simulate a function from the features space (R^n) to R. Uses the Back-propagation method. Produces a trained neural network in the form of a compiled file which can be used in the FEED FORWARD NEURAL NETWORK DISTRIBUTION algorithm.
FEED_FORWARD_NEURAL_NETWORK_REGRESSOR
The algorithm simulates a real-valued vector function using a trained Feed Forward Artificial Neural Network and returns a table containing the function actual inputs and the predicted outputs
FEED_FORWARD_NEURAL_NETWORK_TRAINER
The algorithm trains a Feed Forward Artificial Neural Network using an online Back-Propagation procedure and returns the training error and a binary file containing the trained network
FIGIS_SDMX_DATA_CONVERTER
This tool allows to convert easily a SDMX dataset into CSV, by calling the rsdmx package for R
FIGIS_SPATIAL_REALLOCATION_GENERIC
The Spatial Reallocation algorithm allows to estimate statistics for other areas from those where they were reported. The algorithm is based on spatial disaggregation techniques and provides at now an area-weighted reallocation.
FIGIS_SPATIAL_REALLOCATION_SIMPLIFIED
The Spatial Reallocaton algorithm allows to estimate statistics for other areas from those where they were reported. The algorithm is based on spatial disaggregation technics and provides at now an area-weighted reallocation. This simplified algorithm is specifically targeting users from the FAO Fisheries and Aquaculture department, aims to facilitate its execution by doing abstraction of the intersections to provide.
FIGIS_SPATIAL_REALLOCATION_SIMPLIFIED_TABLE
The Spatial Reallocaton algorithm allows to estimate statistics for other areas from those where they were reported. The algorithm is based on spatial disaggregation technics and provides at now an area-weighted reallocation. This simplified algorithm is specifically targeting users from the FAO Fisheries and Aquaculture department, aims to facilitate its execution by doing abstraction of the intersections to provide.
GENERIC_CHARTS
An algorithm producing generic charts of attributes vs. quantities. Charts are displayed per quantity column. Histograms, Scattering and Radar charts are produced for the top ten quantities. A gaussian distribution reports overall statistics for the quantities.
GEO_CHART
An algorithm producing a charts that displays quantities as colors of countries. The color indicates the sum of the values recorded in a country.
GRID_CWP_TO_COORDINATES
An algorithm that adds longitude, latitude and resolution columns analysing a column containing FAO Ocean Area codes (CWP format).
HRS
An evaluator algorithm that calculates the Habitat Representativeness Score, i.e. an indicator of the assessment of whether a specific survey coverage or another environmental features dataset, contains data that are representative of all available habitat variable combinations in an area.
LWR
An algorithm to estimate Length-Weight relationship parameters for marine species, using Bayesian methods. Runs an R procedure. Based on the Cube-law theory.
MAX_ENT_NICHE_MODELLING
A Maximum-Entropy model for species habitat modeling, based on the implementation by Shapire et al. v 3.3.3k, Princeton University, http://www.cs.princeton.edu/schapire/maxent/. In this adaptation for the D4Science infrastructure, the software accepts a table produced by the Species Product Discovery service and a set of environmental layers in various formats (NetCDF, WFS, WCS, ASC, GeoTiff) via direct links or GeoExplorer UUIDs. The user can also establish the bounding box and the spatial resolution (in decimal deg.) of the training and the projection. The application will adapt the layers to that resolution if this is higher than the native one.The output contains: a thumbnail map of the projected model, the ROC curve, the Omission/Commission chart, a table containing the raw assigned values, a threshold to transform the table into a 0-1 probability distribution, a report of the importance of the used layers in the model, ASCII representations of the input layers to check their alignment.Other processes can be later applied to the raw values to produce a GIS map (e.g. the Statistical Manager Points-to-Map process) and results can be shared. Demo video: http://goo.gl/TYYnTO and instructions http://wiki.i-marine.eu/index.php/MaxEnt
POINTS_TO_MAP
A transducer algorithm to produce a GIS map of points from a set of points with x,y coordinates indications. A maximum of 259000 is allowed
POLYGONS_TO_MAP
A transducer algorithm to produce a GIS map of filled polygons associated to x,y coordinates and a certain resolution. A maximum of 259000 is allowed
QUALITY_ANALYSIS
An evaluator algorithm that assesses the effectiveness of a distribution model by computing the Receiver Operating Characteristics (ROC), the Area Under Curve (AUC) and the Accuracy of a model
SEADATANET_INTERPOLATOR
A connector for the SeaDataNet infrastructure. This algorithms invokes the Data-Interpolating Variational Analysis (DIVA) SeaDataNet service to interpolate spatial data. The model uses GEBCO bathymetry data and requires an estimate of the maximum spatial span of the correlation between points and the signal-to-noise ratio, among the other parameters. It can interpolate up to 10,000 points randomly taken from the input table. As output, it produces a NetCDF file with a uniform grid of values. This powerful interpolation model is described in Troupin et al. 2012, 'Generation of analysis and consistent error fields using the Data Interpolating Variational Analysis (Diva)', Ocean Modelling, 52-53, 90-101.
SGVM_INTERPOLATION
An interpolation method relying on the implementation by the Study Group on VMS (SGVMS). The method uses two interpolation approached to simulate vessels points at a certain temporal resolution. The input is a file in TACSAT format uploaded on the Statistical Manager. The output is another TACSAT file containing interpolated points.The underlying R code has been extracted from the SGVM VMSTools framework. This algorithm comes after a feasibility study (http://goo.gl/risQre) which clarifies the features an e-Infrastructure adds to the original scripts. Limitation: the input will be processed up to 10000 vessels trajectory points. Credits: Hintzen, N. T., Bastardie, F., Beare, D., Piet, G. J., Ulrich, C., Deporte, N., Egekvist, J., et al. 2012. VMStools: Open-source software for the processing, analysis and visualisation of fisheries logbook and VMS data. Fisheries Research, 115-116: 31-43. Hintzen, N. T., Piet, G. J., and Brunel, T. 2010. Improved estimation of trawling tracks using cubic Hermite spline interpolation of position registration data. Fisheries Research, 101: 108-115. VMStools, available as an add-on package for R. Documentation available at https://code.google.com/p/vmstools/. Build versions of VMStools for Window, Mac, Linux available at https://docs.google.com/. Authors: Niels T. Hintzen, Doug Beare
SPECIES_MAP_FROM_CSQUARES
A transducer algorithm to produce a GIS map from a probability distribution associated to a set of csquare codes. A maximum of 259000 is allowed
SPECIES_MAP_FROM_POINTS
A transducer algorithm to produce a GIS map from a probability distribution made of x,y coordinates and a certain resolution. A maximum of 259000 is allowed
SUPPORT_VECTOR_MACHINE_REGRESSOR
A simple algorithm for regression using and already trained Support Vector Machine
SUPPORT_VECTOR_MACHINE_TRAINER
A simple algorithm to train a Support Vector Machine
TIME_GEO_CHART
An algorithm producing an animated gif displaying quantities as colors in time. The color indicates the sum of the values recorded in a country.
TIME_SERIES_ANALYSIS
An algorithms applying signal processing to a non uniform time series. A maximum of 10000 distinct points in time is allowed to be processed. The process uniformly samples the series, then extracts hidden periodicities and signal properties. The sampling period is the shortest time difference between two points. Finally, by using Caterpillar-SSA the algorithm forecasts the Time Series. The output shows the detected periodicity, the forecasted signal and the spectrogram.
TIME_SERIES_CHARTS
An algorithm producing time series charts of attributes vs. quantities. Charts are displayed per quantity column and superposing quantities are summed.
WEB_APP_PUBLISHER
This algorithm publishes a zip file containing a Web site, based on html and javascript in the e-Infrastructure. It generates a public URL to the application that can be shared.
