Bureaucrats, editor, reviewer, Administrators
13,554
edits
Changes
Jump to navigation
Jump to search
no edit summary
This article is part of the [[XQuery|XQuery Portal]].
It summarizes the full-text and language-specific features of BaseX.
Full-text retrieval is an essential query feature for working with XML documents, and BaseX was the first query processor that fully supported the [http://www.w3.org/TR/xpath-full-text-10/ W3C XQuery Full Text 1.0] Recommendation. This page lists some singularities and extensions of the BaseX implementation.
the compiler a second chance and try different rewritings of the same query.
==Full-Text Features== ===Options===
The available full-text index can handle various combinations of the match options defined in the XQuery Full Text Recommendation. By default, most options are disabled. The GUI dialogs for creating new databases or displaying the database properties contain a tab for choosing between all available options. On the command-line, the <code>SET</code> command can be used to activate full-text indexing or creating a full-text index for existing databases:
* '''Stopword List''': a stop word list can be defined to reduce the number of indexed tokens (<code>SET STOPWORDS [filename]</code>).
===Languages===
The chosen language determines how the input text will be tokenized and stemmed. The basic code base and <code>jar</code> file of BaseX comes with built-in support for English and German. More languages are supported if the following libraries are found in the classpath:
</pre>
===Scoring===
The XQuery Full Text Recommendation allows for the usage of scoring models
Terms will be ranked higher if they are found in short text nodes.
===Thesaurus===
BaseX supports full-text queries using thesauri, but it does not provide a default thesaurus. This is why query such as
The format of the thesaurus files must be the same as the format of the thesauri provided by the [http://dev.w3.org/2007/xpath-full-text-10-test-suite XQuery and XPath Full Text 1.0 Test Suite]. It is an XML with structure defined by an [http://dev.w3.org/cvsweb/~checkout~/2007/xpath-full-text-10-test-suite/TestSuiteStagingArea/TestSources/thesaurus.xsd?rev=1.3;content-type=application%2Fxml XSD Schema].
===Fuzzy Querying===
In addition to the official recommendation, BaseX supports fuzzy querying.
The XQFT grammar was enhanced by the FTMatchOption <code>using fuzzy </code>
Some additional [[Full-Text Module|Full-Text Functions]] have been added to BaseX to extend the official language recommendation with useful features, such as explicitly requesting the score value of an item, marking the hits of a full-text request, or directly accessing the full-text index with the default index options.
==Collations==
Another XQuery feature related to full-text operations are '''Collations'''. By default, XQuery uses the {{Code|http://www.w3.org/2003/05/xpath-functions/collation/codepoint}} collation, which compares strings by the Unicode codepoint order. The syntax of alternative collation URIs is implementation-defined; in BaseX, the following syntax is supported (semicolons can be replaced with ampersands):
<nowiki>http://basex.org/collation?lang=...;strength=...;decomposition=...</nowiki>
Semicolons can be replaced with ampersands, and all arguments are optional:
{| class="wikitable sortable"
|-
! Argument
! Description
|-
| {{Code|lang}}
| A language code, followed by an optional language variant. Examples: {{Code|de}}, {{Code|en-US}}.
|-
| {{Code|strength}}
| Level of difference considered significant in comparisons. Four strengths are supported: {{Code|primary}}, {{Code|secondary}}, {{Code|tertiary}}, and {{Code|identical}}. For example, in German, "Ä" and "A" are considered primary differences, "Ä" and "ä" are secondary, "ä" and "A[http://www.fileformat.info/info/unicode/char/308/index.htm &#x308;]" are tertiary, and "A" and "A" are identical.
|-
| {{Code|decomposition}}
| Defines how composed characters are handled. Three decompositions are supported: {{Code|none}}, {{Code|standard}}, and {{Code|full}}. More information is found on the [http://docs.oracle.com/javase/7/docs/api/java/text/Collator.html JavaDoc Collator]] page.
|}
=Changelog=
; Version 7.7:
* Added: [[#Collations|Collations]] support.
; Version 7.3:
