Bureaucrats, editor, reviewer, Administrators
13,554
edits
Changes
Jump to navigation
Jump to search
m
{{Announce|Updated with Version 10}}: Additionally disable hostname verification.
|}
===TEXTPARSER===
{| width='100%'
|- valign="top"
| width='120' | '''Signature'''
|{{Code|TEXTPARSER [options]}}
|- valign="top"
| '''Default'''
|''empty''
|- valign="top"
| '''Summary'''
|Specifies the way how TEXT data will be parsed. Keys and values are delimited with <code>=</code>, and multiple options are delimited with <code>,</code>. The available options are listed in the [[Parsers]] article.
|- valign="top"
| '''Examples'''
|<code>lines=true</code> creates a single element for each line of text.
{{Announce|Updated with Version 10:}} Renamed (before: {{Code|CHOP}}), new default: {{Code|false}}.
{{Announce|Updated with Version 10:}} Renamed from <code>CATFILE</code>.
Text replacement - "syntaxhighlight" to "pre"
Values of options are either ''strings'', ''numbers'' or ''booleans''. Options are ''static'' and not bound to a single operation (for example, the next command). Various ways exist to access and change options:
* The current value of an option can be requested with the {{Command|SHOW OPTIONS}} command. Local options can be changed via {{Command|SET}} (all global options, except for {{Option|DEBUG}}, can only be changed at startup time). If an option is of type ''boolean'', and if no value is specified, its current value will be is inverted.
* The {{Code|.basex}} [[Configuration#Configuration Files|configuration file]] is parsed by every new local BaseX instance. It contains all global options. Local options can be specified at the end of the file after the {{Code|Local Options}} comment:
<syntaxhighlight pre lang="perl">
# General Options
DEBUG = false
# Local Options
CATALOG = etc/w3-catalog.xml
</syntaxhighlightpre>
* Initial values for global options can also be specified via system properties, which can be passed on with the [https://docs.oracle.com/en/java/javase/11/tools/java.html -D flag] on the command line, or with [https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/System.html#setProperty(java.lang.String,java.lang.String) System.setProperty()] before creating a BaseX instance. The specified keys need to be prefixed with {{Code|org.basex.}}. An example:
<syntaxhighlight pre lang="perl">
java -Dorg.basex.CATALOG=etc/w3-catalog.xml -cp basex.jar org.basex.BaseX -c"SHOW OPTIONS catalog"
CATALOG: etc/w3-catalog.xml
</syntaxhighlightpre>
* If the Mac OS X packaged application is used, global options can be set within the Info.plist file within the Contents folder of the application package. For example:
<syntaxhighlight pre lang="xml">
<key>JVMOptions</key>
<array>
<string>-Dorg.basex.CATALOG=etc/w3-catalog.xml</string>
</array>
</syntaxhighlightpre>
* In a [[Web Application]], the default can be adjusted in the {{Code|web.xml}} file as follows:
<syntaxhighlight pre lang="xml">
<context-param>
<param-name>org.basex.catalog</param-name>
<param-value>etc/w3-catalog.xml</param-value>
</context-param>
</syntaxhighlightpre>
* In XQuery, local options can be set via option declarations and [[XQuery Extensions#Pragmas|pragmas]].
If options are changed by operations in the [[GUI]], the underlying commands will be listed are output in the [[GUI#Visualizations|Info View]].<br/>
=Global Options=
|- valign="top"
| '''Summary'''
|Sends internal debug info information to STDERR. This option can be turned on enabled to get additional information for development and debugging purposes. It can also be triggered on the [[Command-Line Options#BaseX Standalone|command line]] via <code>-d</code>. Note that the amount of output can be overwhelming, so it should not be used productively.
|}
|- valign="top"
| '''Summary'''
|Points to the directory in which all where [[Logging|log files]] are stored. Relative paths will be are resolved against the {{Option|DBPATH}} directory.
|}
|- valign="top"
| '''Summary'''
|Specifies the interface language. Currently, seven languages are available: 'English', 'German', 'French', 'Dutch', 'Italian', 'Japanese', and 'Vietnamese'.
|}
| '''Summary'''
|Defines the locking strategy:
* By default, non-fair locking is used. Read transactions will be are favored, and transactions that no not access no databases can be evaluated even if the limit of for parallel transactions (specified via {{Option|PARALLEL}}) has been is reached. This prevents update operations from blocking all other requests. For example, the DBA can further still be used to see which jobs are running, even if the queue is full.* If When fair locking is enabled, read and write transactions will be are treated equally (first in, first out). This avoids starvation of update operations, updates and it should be used if the when prompt evaluation of update operations updates is critical.
|}
|- valign="top"
| '''Summary'''
|Specifies how many seconds the results of queries, which have been queued by the [[Job Module|asynchronously executedjobs]], will be are cached in main memory.
|}
|- valign="top"
| '''Summary'''
|This is the port the database server will be is listening to. This option can also be changed when running the server on [[Command-Line Options#BaseX Server|command line]] via <code>-p</code>.
|}
| '''Summary'''
|Represents a username, which is used for accessing the server or an HTTP service:
* The default value will be is overwritten if a client specifies its own credentials.
* If the default value is empty, login will only be possible if the client specifies credentials.
* The option can also be changed on [[Command-Line Options#BaseX Client|command line]] via <code>-U</code>.
| '''Summary'''
|Represents a password, which is used for accessing the server:
* The default value will be is overwritten if a client specifies its own credentials.
* If the default value is empty, authentication will only be possible if the client supplies credentials.
* The option can also be changed on [[Command-Line Options#BaseX Client|command line]] via <code>-P</code>.
|- valign="top"
| '''Summary'''
|Specifies the default authentication method, which will be used by the [[Web Application|HTTP server]] for negotiating credentials. Allowed values are {{Code|Basic}}, {{Code|Digest}}, and {{Code|Custom}}:
* If basic access is chosen, the client can still request digest authentication.
* This is different for digest access, which cannot be overwritten.
|- valign="top"
| '''Summary'''
|This is the host name or IP address the server is bound to. If the option is set to an empty string (which is the default), the server will be open to accepts requests from all clients.
|}
|- valign="top"
| '''Summary'''
|This is the host name of a proxy server. If the value is an empty string, it will be is ignored.
|}
|- valign="top"
| '''Summary'''
|This is the port number of a proxy server. If the value is set to {{Code|0}}, it will be is ignored.
|}
|- valign="top"
| '''Summary'''
|This is a list of hosts that should be directly accessed. If the value is an empty string, it will be is ignored.
|}
===IGNORECERT===
{| width='100%'
|- valign="top"
| '''Summary'''
|Specifies the maximum time a transaction triggered by a client may take. If an An operation is interrupted when it takes longer than the specified number of seconds, it will be aborted. Active Running update operations will are not be affected by this timeout, as this would corrupt the integrity of the database. The timeout is deactivated disabled if the timeout is set to {{Code|0}}. It is ignored for operations with [[User Management|admin permissions]].
|}
|- valign="top"
| '''Summary'''
|Specifies the maximum time a client will be is remembered by the server. If there has been no interaction with a client for a longer time than specified by this timeout, it will be disconnected. Running operations will not be affected by this option. The keepalive check is deactivated disabled if the value is set to {{Code|0}}.
|}
|- valign="top"
| '''Summary'''
|If BaseX is running as a [[Web Application]], trace output (generated via {{Code|fn:trace}}, {{Function|Profiling|prof:dump}} and similar functions) is written to the [[Logging|database logs]]. If When this option is disabled, trace output will be is redirected to standard error, as it is known from the standalone version of BaseX.
|}
==HTTP Services==
Most HTTP options are defined in the {{Code|jetty.xml}} and {{Code|web.xml}} configuration files in the <code>[https://github.com/BaseXdb/basex/tree/master/basex-api/src/main/webapp/WEB-INF webapp/WEB-INF]</code> directory. Some additional BaseX-specific options exist that will be are set before the web server is started:
===WEBPATH===
|Points to the directory in which all the [[Web Application]] contents are stored, including XQuery, Script, [[RESTXQ]] and configuration files:
* The option is ignored if BaseX is deployed as [[Web Application#Servlet_Container|web servlet]].
* It cannot be assigned via the {{Code|web.xml}} file, as it will be is evaluated before the configuration files are parsed.
|}
|Jetty provides a [https://www.eclipse.org/jetty/documentation/current/gzip-filter.html Gzip handler] for dynamically uncompressing requests and compressing responses. This feature can be enabled if Jetty is started via the [[Web Application|BaseX HTTP Server]]:
* The option can also be enabled on [[Command-Line Options#HTTP Server|command line]] via <code>-g</code>.
* It cannot be assigned via the {{Code|web.xml}} file, as it will be the option is evaluated before the configuration files are parsed.
* In addition to the [https://github.com/eclipse/jetty.project/blob/7cc552013eb4d05cb603ba0bc85d176c97957cd4/jetty-server/src/main/java/org/eclipse/jetty/server/handler/gzip/GzipHandler.java#L187-L211 Jetty defaults] (GET requests, exclusion of binaries, MSIE 6.0, etc.), POST and PUT requests are supported.
|}
|- valign="top"
| '''Summary'''
|Points to the directory which contains the [[RESTXQ]] modules of a web application. Relative paths will be are resolved against the {{Option|WEBPATH}} directory.
|}
|- valign="top"
| '''Summary'''
|Timeout after which the RESTXQ directory will be is parsed for changes:* If {{Code|0}} is specified, the directory will be is parsed every time a RESTXQ function is called.* A positive value defines the idle time , in seconds , after which parsing will be is enforced. The default value is {{Code|3}}: Changes in the RESTXQ directory will be is detected after 3 seconds without RESTXQ function calls.
* Monitoring is completely disabled if a negative value is specified.
|- valign="top"
| '''Summary'''
|Points to the directory which contains XQuery files and command scripts, which can be evaluated via the [[REST#GET Requests|REST run operation]]. Relative paths will be are resolved against the {{Option|WEBPATH}} directory.
|}
|- valign="top"
| '''Summary'''
|By default, if BaseX is run as [[Web Application]], the database server instance will be is started in addition, which can then be addressed by [[Clients]] via the database port (see {{Option|PORT}}). If the option is set to {{Code|true}}, no database server will be is launched.
|}
|- valign="top"
| '''Summary'''
|This If Jetty is started from BaseX, this is the port on which the [[Startup#BaseX HTTP Server|HTTP Server]] can be locally closed:
* The listener for stopping the web server will only be started if the specified value is greater than {{Code|0}}.
* The option is ignored if BaseX is used as a [[Web Application]] or started via [[Web Application#Maven|Maven]].
|- valign="top"
| '''Summary'''
|If this option is turned on, new databases will be created arecreated in main memory:* Most queries will be are evaluated faster in main-memory mode, but all data is lost if the BaseX instance in which the database was created is shut down.
* It is not possible to store binary resources in a main-memory database.
* A main-memory database will have no disk representation. However, it is possible to export the database via the {{Command|EXPORT}} command, and create a new database from the exported file in a second step.
* This option will not be available for {{Function|Database|db:create}}, because the database would not be accessible anymore after database creation, i. ee., outside the query scope.
|}
|- valign="top"
| '''Summary'''
|If this option is set to {{Code|true}}, the file name of parsed archives will be is included in the document paths.
|}
|- valign="top"
| '''Summary'''
|Skips corrupt (i.e., not well-formed) files while creating a database or adding new documents. If this option is activated, document updates are slowed down, as all files will be are parsed twice. Next, main -memory consumption will be is higher , as parsed files will be are cached in main memory.
|}
|- valign="top"
| '''Summary'''
|If this option is enabled, all resources that are filtered out by the {{Option|CREATEFILTER}} option while being added to a database will be are stored as [[Binary Data|raw files]] instead (i.e., in their binary representation).
|}
|- valign="top"
| '''Summary'''
|Defines a [[Parsers|parser]] for importing new files to the database. Available parsers are {{Code|XML}}, {{Code|JSON}}, {{Code|CSV}}, {{Code|TEXT}}, {{Code|HTML}}, and {{Code|RAW}}. HTML input will be is parsed as XML documents if [[Parsers#HTML_ParserHTMLParser|Tagsoup]] is not found in the classpathclass path.
|}
|- valign="top"
| '''Summary'''
|Specifies the way how CSV data will be is parsed. Keys and values are delimited with <code>=</code>, and multiple options are delimited with <code>,</code>. The available options (except for the additional <code>encoding</code> option) are described in the [[CSV Module#Options|CSV Module]].
|- valign="top"
| '''Examples'''
|- valign="top"
| '''Summary'''
|Specifies the way how JSON data will be is parsed. Keys and values are delimited with <code>=</code>, and multiple options are delimited with <code>,</code>. The available options (except for the additional <code>encoding</code> option) are described in the [[JSON Module#Options|JSON Module]].
|- valign="top"
| '''Examples'''
|- valign="top"
| '''Summary'''
|Specifies the way how HTML data will be is parsed. Keys and values are delimited with <code>=</code>, and multiple options are delimited with <code>,</code>. The available options are described in the [[Parsers#Options|Parsers]] article.
|- valign="top"
| '''Examples'''
|
* <code>encoding=Shift-JIS,nons=true</code> parses the input as SihftShift-JIS and suppresses namespaces.
* <code>lexical=true</code> preserves comments.
|}
===STRIPWS===
{| width='100%'
| '''Summary'''
|Many XML documents include whitespaces that have been added to improve readability. This option controls the [https://www.w3.org/TR/REC-xml/#sec-white-space white-space processing mode] of the XML parser:
* If the option is set to {{Code|true}}, leading and trailing whitespaces from text nodes will be are stripped, and empty text nodes will be are discarded.
* The flag should be disabled if a document contains [[Full-Text#Mixed Content|mixed content]].
* The flag can also be turned on via the [[Command-Line Options#BaseX Standalone|command line]] and <code>-w</code>.
* If the option is enabled, whitespaces of an element and its descendants can locally be preserved with the <code>xml:space="preserve"</code> attribute:
<syntaxhighlight pre lang="xml">
<xml>
<title>
<text xml:space="preserve">To <b>be</b>, or not to <b>be</b>, that is the question.</text>
</xml>
</syntaxhighlightpre>
If whitespaces are stripped, <code>indent=yes</code> can be assigned to the {{Option|SERIALIZER}} option to get properly indented XML output.<br/>
See [[BaseX_10#Whitespaces|BaseX 10: Whitespaces]] for more information on whitespace handling.
===CATALOG===
{| width='100%'
==Indexing==
The following options control the creation of index structures. The current values will be are considered if a new database is created. See [[Indexes]] for more details.
===TEXTINDEX===
|- valign="top"
| '''Summary'''
|Defines name patterns for the parent elements of texts that are indexed. By default, all text nodes will be are indexed.<br/>Name patterns are separated by commas. See [[Indexes#Selective Indexing|Selective Indexing]] for more details.
|}
|- valign="top"
| '''Summary'''
|Defines name patterns for the attributes to be indexed. By default, all attribute nodes will be are indexed.<br/>Name patterns are separated by commas. See [[Indexes#Selective Indexing|Selective Indexing]] for more details.
|}
|- valign="top"
| '''Summary'''
|Defines name patterns for the attributes to be indexed. By default, tokens in all attribute nodes will be are indexed.<br/>Name patterns are separated by commas. See [[Indexes#Selective Indexing|Selective Indexing]] for more details.
|}
|- valign="top"
| '''Summary'''
|Defines name patterns for the parent elements of texts that are indexed. By default, all text nodes will be are indexed.<br/>Name patterns are separated by commas. See [[Indexes#Selective Indexing|Selective Indexing]] for more details.
|}
|- valign="top"
| '''Summary'''
|Specifies the maximum length for strings to be stored in [[Indexes|index structures]]. The value of this option will be is assigned once to a new database, and can only be changed by creating a new database or doing a [[Commands#OPTIMIZE|full optimization]].
|}
|- valign="top"
| '''Summary'''
|Specifies the maximum number of distinct values (categories) that will be are stored together with the element/attribute names or unique paths in the [[Index#Name Index|Name Index]] or [[Index#Path Index|Path Index]]. The value of this option will be is assigned once to a new database, and cannot be changed after that.
|}
|- valign="top"
| '''Summary'''
|If turned on, incremental indexing will be is enabled:* The current value of this option will be is assigned to new databases. It can be changed for existing databases by running {{Command|OPTIMIZE}} with the {{Code|ALL}} keyword or {{Function|Database|db:optimize}} and {{Code|true()}} as second argument.* After each update, the value indexes will be are refreshed as well. Incremental updates are currently not available for the full-text index and database statistics.
* Find more details in the article on [[Index#Updates|Index Structures]].
|}
|- valign="top"
| '''Summary'''
|If turned on, auto optimization will be is applied to new databases:* With each update, outdated indexes and database statistics will be are recreated.
* As a result, the index structures will always be up-to-date.
* However, updates can take much longer, so this option should only be activated for medium-sized databases.
* The value of this option will be is assigned once to a new database. It can be reassigned by running {{Command|OPTIMIZE}} or {{Function|Database|db:optimize}}.
|}
* By default, if the value is set to {{Code|0}}, some heuristics are applied, based on the current memory consumption. Usually, this works fine.
* If explicit garbage collection is disabled when running Java (e.g. via the JVM option {{Code|-XX:+DisableExplicitGC}}), you may need to choose a custom split size.
* You can e. gg. start with {{Code|1000000}} (one million) index operations and adjust this value in the next steps.* The larger the assigned value is, the less fewer splits will take place, and the more main -memory will be requiredconsumption increases.
|}
|- valign="top"
| '''Summary'''
|If {{Code|true}}, all tokens will be are stemmed during full-text indexing, using a language-specific stemmer implementation. By default, tokens will not be stemmed. See [[Indexes#Full-Text Index|Full-Text Index]] for more details.
|}
|- valign="top"
| '''Summary'''
|If {{Code|true}}, the case of tokens will be is preserved during full-text indexing. By default, case will be is ignored (all tokens will be are indexed in lower case). See [[Indexes#Full-Text Index|Full-Text Index]] for more details.
|}
|- valign="top"
| '''Summary'''
|If set to {{Code|true}}, diacritics will be are preserved during full-text indexing. By default, diacritics will be are removed. See [[Indexes#Full-Text Index|Full-Text Index]] for more details.
|}
|- valign="top"
| '''Summary'''
|The specified language will influence the way how texts will be are tokenized and stemmed. It can be the name of a language or a language code. See [[Indexes#Full-Text Index|Full-Text Index]] for more details.
|}
|- valign="top"
| '''Summary'''
|If a text file with stop words is specified, frequently used terms contained in that file will be are ignored when a full-text index is created. A stopword stop word list may decrease the size of the full -text index and speed up your queries. See [[Indexes#Full-Text Index|Full-Text Index]] for more details.
|}
|- valign="top"
| '''Summary'''
|Allows queries to both contain updating and non-updating expressions. All updating constraints will be are turned off, and nodes to be returned will be are copied before they are modified by an updating expression. By default, in compliance with the XQuery Update Facility, this option is set to {{Code|false}}. See [[XQuery Update#Returning Results|Returning Results]] for more details.
|}
===BINDINGS===
{{Announce|Updated with Version 11}}: The Clark notation was replaced with the [[XQuery 3.0#Expanded QNames|Expanded QNames]] notation.
{| width='100%'
* Commas in values must be duplicated.
* Variables may optionally be introduced with a leading dollar sign.
* If a variable uses a namespace different to from the default namespace, it can be specified with the [http://www.jclark.com/xml/xmlns.htm Clark Notation] or [https://wwwXQuery 3.w3.org/TR/xquery-30/0#id-basics Expanded QName NotationQNames|Expanded QNames]]notation.
This option can also be used on [[Command-Line Options#BaseX Standalone|command line]] with the flag <code>-b</code>.
|- valign="top"
* <code>{URI}a=x</code> binds the value {{Code|x}} to the variable $a with the namespace {{Code|URI}}.
* In the following [[Commands#Command_Scripts| Command Script]], the value {{Code|hello world!}} is bound to the variable {{Code|$GREETING}}:
<syntaxhighlight pre lang="'xquery"'>
SET BINDINGS GREETING="hello world!"
XQUERY declare variable $GREETING external; $GREETING
</syntaxhighlightpre>
|}
|- valign="top"
| '''Summary'''
|When creating new nodes in XQuery via [https://www.w3.org/TR/xquery-31/#id-constructors Node Constructors], copies of all enclosed nodes will be are created, and the copied nodes get new node identities. As a result, the following query yields <code>false</code>:<syntaxhighlight pre lang="'xquery"'>
let $a := <a/>
let $b := <b>{ $a }</b>
return $b/a is $a
</syntaxhighlightpre>
This step can be expensive and memory consuming. If the option is disabled, child nodes will only be linked to the new parent nodes, and the upper query returns <code>true</code>.<br/>The option should be used carefully as it changes the semantics of XQuery. It should preferably be used in [[XQuery Extensions#Database Pragmas|Pragmas]].
|}
| '''Summary'''
|By default, resources specified via {{Code|fn:doc}} and {{Code|fn:collection}} are looked up both in the database and in the file system. If you always use {{Function|Database|db:get}} to access databases, it is recommendable to disable this option:
* No locks will be are created for the two functions (see [[Transaction Management#Limitations|limitations of database locking]] for more details).* Access to local and external resources will be is faster, as the database lookup will be is skipped.
|}
|- valign="top"
| '''Summary'''
|If this option is turned on, paths specified in the {{Code|fn:doc}} and {{Code|fn:collection}} functions will first be resolved against a database that has been opened in the global context outside the query (e.g. by the {{Command|OPEN}} command). If the path does not match any existing resources, it will be is resolved as described in the article on [[Databases#Access Resources|accessing database resources]].
|}
|- valign="top"
| '''Summary'''
|By activating enabling this option, database instances will be are created with if the XQuery functions {{Code|fn:doc}} and {{Code|fn:collection}}are called.
|}
|- valign="top"
| '''Summary'''
|By default, characters from external sources that are invalid in XML will trigger an error. If the option is set to <code>false</code>, these characters will be are replaced with the Unicode replacement character <code>FFFD</code> (�). The option affects [[Java Bindings]] and string conversion and input functions such as {{Function|Archive|archive:create}}, {{Function|Archive|archive:extract-text}}, and {{Function|Archive|archive:update}}.
|}
|- valign="top"
| '''Summary'''
|Defines the way how values that result from Java code invocation will be are converted to XQuery items. Allowed values: {{Code|none}}, {{Code|all}}, {{Code|some}}, {{Code|instance}}, and {{Code|void}}. See [[Java_Bindings#Conversion to XQuery|Java Bindings]] for further details.
|}
|- valign="top"
| '''Summary'''
|Specifies if a query will be is executed or parsed only. This option can also be changed on [[Command-Line Options#BaseX Standalone|command line]] via <code>-R</code>.
|}
|- valign="top"
| '''Summary'''
|Specifies how often a query will be is evaluated. The result is serialized only once, and the measured times are averages of all runs. This option can also be changed on [[Command-Line Options#BaseX Standalone|command line]] via <code>-r</code>.
|}
|- valign="top"
| '''Summary'''
|Results of XQuery expressions will be are serialized if this option is turned on. For debugging purposes and performance measurements, this option can be set to {{Code|false}}. It can also be turned off on [[Command-Line Options#BaseX Standalone|command line]] via <code>-z</code>.
|}
|- valign="top"
| '''Summary'''
|Flushes database buffers to disk after each update. If this option is set to {{Code|false}}, bulk operations (multiple single updates) will be are evaluated faster. As a drawback, the chance of data loss increases if the database is not explicitly flushed via the {{Command|FLUSH}} command.
|}
|Propagates updates on main-memory instances of files that have been retrieved via {{Code|fn:doc}} and {{Code|fn:collection}} back to disk:
* This option can also be activated on [[Command-Line Options#BaseX Standalone|command line]] via <code>-u</code>.
* Please take in mind that no backup will be is created from your original files.
* The serialization options can be controlled via the {{Option|EXPORTER}} option.
|}
=Changelog=
;Version 11.0
* Removed: <code>TEXTPARSER</code>
* Updated: {{Option|BINDINGS}}: The Clark notation was replaced with the [[XQuery 3.0#Expanded QNames|Expanded QNames]] notation.
;Version 10.0
;Version 7.8
* Added: {{Option|CSVPARSER}}, {{Option|JSONPARSER}}, {{Option|TEXTPARSER}}, {{Option|HTMLPARSER}}, {{Option|INLINELIMIT}}, {{Option|TAILCALLS}}, {{Option|DEFAULTDB}}, {{Option|RUNQUERY}}
* Updated: {{Option|WRITEBACK}} only applies to main-memory document instances.
* Updated: {{Option|DEBUG}} option can be changed at runtime by users with admin permissions.
