Changes

This article is part of the [[Getting StartedAdvanced User's Guide]] Section.It presents the available parsers that can be used to import various data sources in BaseX databases.Please visit the [[Serialization]] article if you want to know how to export data.

The [[{{Function|Database Module#db:add|db:add]] }} and [[{{Function|Database Module#db:replace|db:replace]] put}} functions can also be used to add new XML documents to the database. The following example query uses the internal XML parser and adds all files to the database <code>DB</code> that are found in the directory <code>2Bimported</code>:

<syntaxhighlight pre lang="'xquery"'>

</syntaxhighlightpre>

If [http://vrici.lojban.org/~cowan/tagsoup/ TagSoup] is found in the [[Startup#Distributions|classpath]], HTML can be imported in BaseX without any problems. TagSoup ensures that only input is converted to well-formed HTML arrives at the XML parser (correct opening and closing tags, etc.)documents.

If TagSoup is not available on a system, the default XML parser will be used. (Only) , and the import will only succeed if the input is already well-formed XML, (which is usually the import will succeedcase for XHTML documents).

====Downloads====

====Maven====

An easy way to add TagSoup to your project is to follow this stepsas follows:

<syntaxhighlight pre lang="xml">

</syntaxhighlightpre>

4. copy that in your own maven project’s Insert the XML fragment into the <code>pom.xml<dependencies></code> file into the element of your project’s <code><dependencies>pom.xml</code> elementfile.

====Debian==== With Debian, TagSoup will automatically be automatically detected and included after it has been installed via:

TagSoup offers a variety of options to customize the HTML conversion. For the complete list, please visit the [http://vrici.lojban.org/~cowan/XML/tagsoup/#program TagSoup] website. BaseX supportsmost of these options , with a few exceptions:

* '''files''': not supported as the input documents are piped directly to the XML parser.

===GUI===

Go to Menu ''Database'' → ''New'' and select "HTML" in the input format combo box.There's an info in the "Parsing" tab about whether TagSoup is available or not. Thesame applies to the "Resources" tab in the "Database Properties" dialog.

These two dialogs come with an input field 'Parameters' where TagSoup optionscan be entered. Keys and values are separated by the equal sign, multiple options are separated by commas, e.g.: <code>method=xml,nons=true</code>.

===Command Line===

Turn on the HTML Parser before parsing documents, and set a file filterand optional parsing options:

===XQuery=== The [[HTML Module]] provides a function for converting HTML to XML documents.

Documents The [[HTML Module]] provides XQuery functions for converting HTML to XML. In addition, all functions that accept [[Options#Parsing|Parsing Options]] can also be converted by specifying the parser and additional optionsused as function argumentswell to convert HTML:

<syntaxhighlight pre lang="'xquery"'>fetch:xmldoc("index.html", map {

'htmlparser': map { 'htmlmethod': false()'xml', 'nodefaultsnons': true() }

</syntaxhighlightpre>

With BaseX can also import , JSON documentscan be imported. The resulting format is JSON parsing options are described in the documentation for of the XQuery [[JSON Module]]:.

Go to Menu ''Database'' → ''New'' and select "JSON" in the input format combo box.You can set the following options for parsing JSON documents in the "Parsing" tab: * '''Encoding''': Choose the appropriate encoding of the JSON file.* '''JsonML''': Activate this option if the incoming file is a JsonML file.

The [[JSON Module]] provides functions for converting JSON objects to XML documents. In addition, all functions that accept [[Options#Parsing|Parsing Options]] can be used as well to convert JSON. The following query converts a JSON string to an XML document, using the conversion format {{Code|[[JSON Module#Basic|basic]]}}: <pre lang='xquery'>db:create('json-db', '{ "key": "value" }', 'example.json', map { 'parser': 'json', 'jsonparser': map { 'format': 'basic' }})</pre>

BaseX can be used to import CSV documents. Different alternatives how to proceed The CSV parsing options are shown described in the following:documentation of the [[CSV Module]].

You can set the following options for parsing CSV documents in the "Parsing" tab: * '''Encoding''': Choose the appropriate encoding of the CSV file.* '''Separator''': Choose the column separator of the CSV file. Possible: <code>comma</code>, <code>semicolon</code>, <code>tab</code> or <code>space</code> or an arbitrary character.* '''Header''': Activate this option if the incoming CSV files have a header line.

Turn on the Enable CSV Parser Parsing before parsing documents, and set some optional, parser-specific options and a file filter. Unicode code points can be specified as separators; {{Code|32}} is the code point for spaces:

SET {{Option|CSVPARSER}} encoding=utf-8, linesheader=true, header=false, separator=spacesemicolon

The [[CSV Module]] provides a function for converting CSV to XML documents.

Documents In addition, all functions that accept [[Options#Parsing|Parsing Options]] can also be converted by specifying used as well to convert CSV. In the parser in an XQuery function.The following example query adds , all CSV files that from a directory are located in thedirectory {{Code|2Bimported}} added to the a database {{Code|DB}} and interprets, with the first lines line of each file interpreted as column headersheader:

<syntaxhighlight pre lang="'xquery"'>

return db:add("DBcsv-db", $file, "", map {

</syntaxhighlightpre>

=Text ParserChangelog=

Plain text can be imported as well: ==GUI== Go to Menu ''Database'' → ''New'' and select "TEXT" in the input format combobox.You can set the following option for parsing text documents in the "Parsing" tab: * '''Encoding''': Choose the appropriate encoding of the text file;Version 11.0* '''Lines'''Removed: Activate this option to create a <code>&lt;line&gt;...&lt;/line&gt;</code> element for each line of the input text file. ==Command Line== Turn on the CSV Text Parser before parsing documents and set some optional, parser-specific options and a file filter:  SET {{Option|PARSER}} text SET {{Option|TEXTPARSER}} lines=yes SET {{Option|CREATEFILTER}} * ==XQuery== Similar to the other formats, the text parser can also be specified via XQuery: <syntaxhighlight lang="xquery">for $file in file:list("2Bimported", true(), "*.txt")return db:add("DB", $file, "", map { 'parser': 'text' })</syntaxhighlight> =Changelog=