Difference between revisions of "Creating Indices at the VO Level"
(→Creating an Index for an OAI-DC collection) |
(→FtsRowset_Transformer) |
||
Line 26: | Line 26: | ||
[[File:FtsRowset_Transformer.xml]] | [[File:FtsRowset_Transformer.xml]] | ||
− | |||
− | |||
=== Index Types === | === Index Types === |
Latest revision as of 17:04, 9 May 2014
Contents
Indexing Procedure
The Indexing procedure refers to the creation of indices for the collections imported in a Virtual Organization. It consists of three steps:
- Creation of the Rowset XSLT generic resources, that transform collection data into data that can be fed to an Index.
- Creation of the Index type generic resources, that define the Index configuration.
- Definition of an IRBootstrapper job that will perform the steps required to create the Indices.
In the first two steps we create generic resources for the Rowset XSLTs and Index Types through the Resource Management portlet . You can find detailed descriptions for the Rowset data (the output of the Rowset XSLT transformation) in the following section:
You can find detailed descriptions for the Index Type definition here:
For the third step, a definition of an IRBootstrapper job is required. You can find the details for defining such a job in the IR Bootstrapper section. To complete the Index creation, the administrator must go to the IRBootstrapper and run the job. The two examples that follow will clarify the three steps.
Creating an Index for an OAI-DC collection
DataTransformation Programs
FtsRowset_Transformer
The following transformation program is called for fulltext rowset creation. Transformation unit with id="6" takes multiple XSLTs and applies final XSLT at the end.
File:FtsRowset Transformer.xml
Index Types
In this section we present the required IndexType for (FullText) Index.
FullTextIndexType
In order to extract the fields from the OAI-DC payload and build the FullText Index the following FullTextIndexType is required:
<Name>IndexType_ft_oai_dc_1.0</Name> <SecondaryType>FullTextIndexType</SecondaryType> <Description>Definition of the fulltext index type for the 'oai dc' schema</Description> <Body> <index-type name="default"> <field-list sort-xnear-stop-word-threshold="2E8"> <field name="contributor"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="coverage"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="creator"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="date"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="description"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="format"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="identifier"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="language"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="publisher"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="relation"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="rights"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="source"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="subject"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="title"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="type"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="ObjectID"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="gDocCollectionID"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="gDocCollectionLang"> <index>yes</index> <store>yes</store> <return>yes</return> <tokenize>yes</tokenize> <sort>no</sort> <boost>1.0</boost> </field> <field name="S"> <index>no</index> <store>yes</store> <return>yes</return> <tokenize>no</tokenize> <sort>no</sort> <boost>1.0</boost> </field> </field-list> </index-type> </Body>
Bootstrapper Configuration
The IRBootstrapper portlet requires a Generic Resource to be available on the IS with name: IRBootstrapperConfiguration and secondary type: IRBootstrapperConfig. For more information please refer to this section IRBoostrapperConfiguration Generic Resource
An example of the configuration is the following:
File:Bootstrapper Configuration.xml
Metadata Broker XSLT
- BrokerXSLT_oai_dc_anylanguage_to_ftRowset_anylanguage
The following XSLT transforms data elements with oai-dc schema to fulltext rowsets:
File:BrokerXSLT oai dc anylanguage to ftRowset anylanguage.xml