Changes

* '''Language''': [[#Languages|see below]] for more details (<code>SET LANGUAGE EN</code>).* '''Stemming''': tokens are stemmed with the Porter Stemmer before being indexed (<code>SET STEMMING true</code>) .* '''Case Sensitive''': tokens are indexed in case-sensitive mode (<code>SET CASESENS true</code>) .* '''Diacritics''': diacritics are indexed as well (<code>SET DIACRITICS true</code>) .* '''Stopword List''': a stop word list can be defined to reduce the number of indexed tokens (<code>SET STOPWORDS [filename]</code>).* {{Mark|Removed in Version 7.2.2}}: '''TF/IDF Scoring''': TF/IDF-based scoring values are calculated and stored in the index (<code>SET SCORING 0/1/2</code>). This feature was removed in favor of the internal scoring model; [[#Scoring|see below]] for more details.* {{Mark|Removed in Version 7.2.2}}: '''Support Wildcards''': a trie-based index can be applied to support wildcard searches (<code>SET WILDCARDS true</code>). This option is was discarded in {{Version|7.2.2}}, as the index then now supports both wildcard and fuzzy queries.

BaseX offers an efficient internal scoring model which can be easily extended todifferent application scenarios. Additionally, BaseX allows to store scoringvalues within the full-text index structure (demanding additional time andmemory). Three scoring types are currently available, which can be adjustedwith the <code>SCORING</code> property (Default: <code>SET SCORING 0</code>):

*<code>0{{Mark|Updated in Version 7.2.2:<}}TF/code> This algorithm yields IDF scoring was discarded in favor of the best internal scoring model, which proved to yield better results for general-purpose use XML documents in most of the cases. It calculates the scoring value out The score of a full-text result is calculated by taking the length number of a term found terms and its their frequency in a single text nodeinto account. This algorithm is also applied if no index exists, or Terms will be ranked higher if the index cannot be applied they are found in a query.*<code>1:</code> Standard TF/IDF algorithm, which treats ''document nodes'' as document units.*<code>2:</code> Each ''text node'' is treated as a document unit in the TF/IDF algorithm. This variant is an alternative for type <code>1</code>, if the database contains large, few XML filesshort texts.

==Querying Using Thesaurus==