Changes

This article is part of the [[XQuery|XQuery Portal]].It summarizes provides a summary of the new most important features of the [httphttps://www.w3.org/TR/xquery-31/ XQuery 3.1] Working Draftthat are already supported by BaseXRecommendation.

==Maps==

<pre classlang="brush:'xquery"'>

The function corresponding to the map has the signature {{Code|function($key as xs:anyAtomicType) as item()*}}. The expression {{Code|$map($key)}} returns the associated value; the function call {{Code|map:get($map, $key)}} is equivalent. For example, if {{Code|$books-by-isbn}} is a map whose keys are ISBNs and whose associated values are {{Code|book}} elements, then the expression {{Code|$books-by-isbn("0470192747")}} returns the {{Code|book}} element with the given ISBN. The fact that a map is a function item allows it to be passed as an argument to higher[[Higher-order functions Order Functions]] that expect a function item as one of their arguments. As an example, the following query uses the higher-order function {{Code|fn:map($f, $seq)}} to extract all bound values from a map:

<pre classlang="brush:'xquery"'>

Like all other values, maps are immutable. For example, the <code>[[{{Function|Map Module#map:remove|map:remove]]</code> }} function creates a new map by removing an entry from an existing map, but the existing map is not changed by the operation. Like sequences, maps have no identity. It is meaningful to compare the contents of two maps, but there is no way of asking whether they are "the same map": two maps with the same content are indistinguishable.

==Arrays==

An ''array'' is a function that associates a set of positions, represented as positive integer keys, with values. The first position in an array is associated with the integer {{Code|1}}. The values of an array are called its members. In the type hierarchy, array has a distinct type, which is derived from function. In BaseX, arrays (as well as sequences) are based on an efficient [https://en.wikipedia.org/wiki/Finger_tree Finger Tree] implementation.

<pre classlang="brush:'xquery"'>[], (: empty array :)[ (1, 2) ], (: array with single member :)[ 1 to 2, 3 ] (: array with two members; same as: [ (1, 2), 3 ] :)

<pre classlang="brush:'xquery"'>array { }, (: empty array; same as: array { () } :) array { (1, 2) }, (: array with three two members; same as: array { 1, 2 } :)array { 1 to 2, 3 } (: array with three members; same as: array { 1, 2, 3 } :)

<pre classlang="brush:'xquery"'>

Like all other values, arrays are immutable. For example, the <code>[[{{Function|Array Module#array:reverse|array:reverse]]</code> }} function creates a new array containing a re-ordering of the members of an existing array, but the existing array is not changed by the operation. Like sequences, arrays have no identity. It is meaningful to compare the contents of two arrays, but there is no way of asking whether they are "the same array": two arrays with the same content are indistinguishable.

==Lookup OperatorAtomization==

If an array is ''atomized'', all of its members will be atomized. As a result, an atomized item may now result in more than one item. Some examples: <pre lang='xquery'>fn:data([1 to 2]) (: returns the sequence 1, 2 :)[ 'a', 'b', 'c' ] = 'b' (: returns true :)<a>{ [ 1, 2 ] }</a> (: returns <a>1 2</a> :)array { 1 to 2 } + 3 (: error: the left operand returns two items :)</pre> Atomization also applies to function arguments. The following query returns 5, because the array will be atomized to a sequence of 5 integers: <pre lang='xquery'>let $f := function($x as xs:integer*) { count($x) }return $f([1 to 5])</pre> However, the next query returns 1, because the array is already of the general type {{Code|item()}}, and no atomization will take place: <pre lang='xquery'>let $f := function($x as item()*) { count($x) }return $f([1 to 5])</pre> Arrays can be compared with the {{Code|fn:deep-equal}} function. The [[Array Module]] describes the available set of array functions. =Lookup Operator= The lookup operator provides some syntactic sugar to access values of maps or array members at a specified position. It is introduced by the question mark ({{Code|?}}) and followed by a specifier. The specifier can be:

<pre classlang="brush:'xquery"'>

$map ?* (: 1. returns all values ; same as: map:keys($map) ! $map(.) :), $map ?R (: 2. returns the value associated with the key 'R'; equivalent to same as: $map('R') :), $map ?('G','B') (: 43. returns the values associated with the key 'G' and 'B' :)

$array ?* (: 1. returns all values ; same as: (1 to array:size($array)) ! $array(.) :), $array ?1 (: 32. returns the first value; equivalent to same as: $array(21) :), $array ?(2 to 3) (: 43. returns the value second and third values; same as: (1 to 2 and 3 ) ! $array(.) :)

<pre classlang="brush:'xquery"'>for let $map in maps := (

return $mapmaps[?name = 'Hildur'] ?city</pre> =Arrow Operator= The arrow operator <code>=></code> provides a convenient alternative syntax for passing on functions to a value. The expression that precedes the operator will be supplied as first argument of the function that follows the arrow. If <code>$v</code> is a value and <code>f()</code> is a function, then <code>$v => f()</code> is equivalent to <code>f($v)</code>, and <code>$v => f($j)</code> is equivalent to <code>f($v, $j)</code>: <pre lang='xquery'>(: Returns 3 :)count(('A', 'B', 'C')),('A', 'B', 'C') => count(),('A', 'B', 'C') => (function( $sequence) { count( $sequence)})(), (: Returns W-E-L-C-O-M-E :)string-join(tokenize(upper-case('w e l c o m e')), '-'),'w e l c o m e' => upper-case() => tokenize() => string-join('-'), (: Returns xfmdpnf :)codepoints-to-string( for $i in string-to-codepoints('welcome') return $i + 1),(for $i in 'welcome' => string-to-codepoints() return $i + 1) => codepoints-to-string()

If =String Constructor= The string constructor has been inspired by [https://en.wikipedia.org/wiki/Here_document here document] literals of the Unix shell and script languages. It allows you to generate strings that contain various characters that would otherwise be interpreted as XQuery delimiters. The string constructors syntax uses two backticks and a square bracket for opening and closing a string: <pre lang='xquery'>(: Returns "This is a 'new' & 'flexible' syntax." :)``["This is a 'new' & 'flexible' syntax."]``</pre> XQuery expressions can be embedded via backticks and a curly bracket. The evaluated results will be separated with spaces, and all strings will eventually be concatenated: <pre lang='xquery'>(: Returns »Count 1 2 3, and I will be there.« :)let $c := 1 to 3return ``[»Count `{ $c }`, and I will be there.«]``</pre> =Serialization= Two [[Serialization]] methods have been added to the [https://www.w3.org/TR/xslt-xquery-serialization-31 Serialization spec]: ==Adaptive Serialization== The {{Code|adaptive}} serialization provides an array intuitive textual representation for all XDM types, including maps and arrays, functions, attributes, and namespaces. All items will be separated by the value of the {{Code|item-separator}} parameter, which by default is a newline character. It is utilized by the functions {{Function|Profiling|prof:dump}} and <code>[https://www.w3.org/TR/xpath-functions-31/#func-trace fn:trace]</code>. Example: <pre lang='xquery'>declare option output:method 'adaptive';<element id='id0'/>/@id,xs:token("abc"),map { 'key'atomized: 'value'},true#0</pre> Result: <pre lang="xml">id="id0"xs:token("abc"),map { "key": "value"}fn:true#0</pre> ==JSON Serialization== The new {{Code|json}} serialization output method can be used to serialize XQuery maps, arrays, all atomic values and empty sequences as JSON. The {{Code|json}} output method has been introduced in BaseX before it was added to the official specification. It complies with the standard serialization rules and, at the same time, preserves the existing semantics: * If an XML node of type {{Code|element(json)}} is found, it will be serialized following the serialization rules of its members the [[JSON Module]].* Any other node or atomic value, map, array, or empty sequence will be serialized according to the [https://www.w3.org/TR/xslt-xquery-serialization-31/#json-output rules in the specification]. The following two queries will both return the JSON snippet <code>{ "key": "value" }</code>: <pre lang='xquery'>declare option output:method 'json';map { "key": "value" }</pre> <pre lang='xquery'>declare option output:method 'json';<json type='object'> <key>value</key></json></pre> =Functions= The following functions have been added in the [https://www.w3.org/TR/xpath-functions-31/ XQuery 3.1 Functions and Operators] Specification: ==Map Functions== <code>map:merge</code>, <code>map:size</code>, <code>map:keys</code>, <code>map:contains</code>, <code>map:get</code>, <code>map:entry</code>, <code>map:put</code>, <code>map:remove</code>, <code>map:for-each</code> Please check out the [[Map Module]] for more details. ==Array Functions== <code>array:size</code>, <code>array:append</code>, <code>array:subarray</code>, <code>array:remove</code>, <code>array:insert-before</code>, <code>array:head</code>, <code>array:tail</code>, <code>array:reverse</code>, <code>array:join</code>, <code>array:flatten</code>, <code>array:for-each</code>, <code>array:filter</code>, <code>array:fold-left</code>, <code>array:fold-right</code>, <code>array:for-each-pair</code> Please check out the [[Array Module]] for more details. ==JSON Functions== With XQuery 3.1, native support for JSON objects was added. Strings and resources can be atomizedparsed to XQuery items and, as [[#JSON Serialization|shown above]], serialized back to their original form. As  ===fn:parse-json=== <pre>fn:parse-json( $json as xs:string? $options as map(*) := ()) as item()?</pre> Parses the supplied string as JSON text and returns its item representation. The result may be a map, an array, a string, a double, a resultboolean, or an atomized empty sequence. The allowed options can be looked up in the [https://www.w3.org/TR/xpath-functions-31/#func-parse-json specification]. <pre lang='xquery'>parse-json('{ "name": "john" }') (: yields { "name": "json" } :),parse-json('[ 1, 2, 4, 8, 16]') (: yields [ 1, 2, 4, 8, 16 ] :)</pre> ===fn:json-doc=== <pre>fn:json-doc( $href as xs:string? $options as map(*) := ()) as item()?</pre> Retrieves the text from the specified URI, parses the supplied string as JSON text and returns its item may now representation (see {{Function||fn:parse-json}} for more details). <pre lang='xquery'>json-doc("http://ip.jsontest.com/")('ip') (: returns your IP address :)</pre> ===fn:json-to-xml=== <pre>fn:json-to-xml( $json as xs:string? $options as map(*) := ()) as document-node()?</pre> Converts a JSON string to an XML node representation. The allowed options can be looked up in the [https://www.w3.org/TR/xpath-functions-31/#func-json-to-xm specification]. <pre lang='xquery'>json-to-xml('{ "message": "world" }') (: result :<map xmlns="http://www.w3.org/2005/xpath-functions"> <string key="message">world</string></map> :)</pre> ===fn:xml-to-json=== <pre>fn:xml-to-json( $node as xs:string? $options as map(*) := ()) as xs:string?</pre> Converts an XML node, whose format conforms to the results created by {{Function||fn:json-to-xml}}, to a JSON string representation. The allowed options can be looked up in the [https://www.w3.org/TR/xpath-functions-31/#func-xml-to-json specification]. <pre lang='xquery'>(: returns "JSON" :)xml-to-json(<string xmlns="http://www.w3.org/2005/xpath-functions">JSON</string>)</pre> ==fn:sort== <pre>fn:sort( $input as item()*, $collation as xs:string? := fn:default-collation(), $key as function(item()) as xs:anyAtomicType* := fn:data#1) as item()*</pre> Returns a new sequence with sorted {{Code|$input}} items, using an optional {{Code|$collation}}. If a {{Code|$key}} function is supplied, it will be applied on all items. The items of the resulting values will be sorted using the semantics of the {{Code|lt}} expression. <pre lang='xquery'>sort(reverse(1 to 3)) (: yields 1, 2, 3 :),reverse(sort(1 to 3)) (: returns the sorted order in more than descending order :),sort((3,-2,1), (), abs#1) (: yields 1, -2, 3 :),sort((1,2,3), (), function($x) { -$x }) (: yields 3, 2, 1 :),sort((1,'a')) (: yields an error, as strings and integers cannot be compared :)</pre> ==fn:contains-token== <pre>fn:contains-token( $value as xs:string*, $token as xs:string, $collation as xs:string? := fn:default-collation()) as xs:boolean</pre> The supplied strings will be tokenized at whitespace boundaries. The function returns {{Code|true}} if one of the strings equals the supplied token, possibly under the rules of a supplied collation: <pre lang='xquery'>contains-token(('a', 'b c', 'd'), 'c') (: yields true :)<xml class='one two'/>/contains-token(@class, 'one ') (: yields true :)</pre> ==fn:parse-ietf-date== <pre>fn:parse-ietf-date( $value as xs:string?) as xs:dateTime?</pre> Parses a string in the IETF format (which is widely used on the Internet) and returns a {{Code|xs:dateTime}} item: <pre lang='xquery'>fn:parse-ietf-date('28-Feb-1984 07:07:07')" (: yields 1984-02-28T07:07:07Z :),fn:parse-ietf-date('Wed, 01 Jun 2001 23:45:54 +02:00')" (: yields 2001-06-01T23:45:54+02:00 :)</pre> ==fn:apply== <pre>fn:apply( $function as function(*), $arguments as array(*)) as item()*</pre> The supplied {{Code|$function}} is invoked with the specified {{Code|$arguments}}. The arity of the function must be the same as the size of the array. Example: <pre lang='xquery'>fn:apply(concat#5, array { 1 to 5 }) (: 12345 :)fn:apply(function($a) { sum($a) }, [ 1 to 5 ]) (: 15 :)fn:apply(count#1, [ 1,2 ]) (: error. the array has two members :)</pre> ==fn:random-number-generator== <pre>fn:random-number-generator( $seed as xs:anyAtomicType? := ()) as map(xs:string, item())</pre> Creates a random number generator, using an optional seed. The returned map contains three entries: * {{Code|number}} is a random double between 0 and 1* {{Code|next}} is a function that returns another random number generator* {{Code|permute}} is a function that returns a random permutation of its argument The returned random generator is ''deterministic'': If the function is called twice with the same arguments and in the same execution scope, it will always return the same result. Some examples Example: <pre lang='xquery'>let $rng := fn:random-number-generator()let $number := $rng('number') (: returns a random number :)let $next-rng := $rng('next')() (: returns a new generator :)let $next-number := $next-rng('number') (: returns another random number :)let $permutation := $rng('permute')(1 to 5) (: returns a random permutation of (1,2,3,4,5) :)return ($number, $next-number, $permutation)</pre> ==fn:format-number== The function has been extended to support scientific notation:

<pre classlang="brush:'xquery"'>fn:dataformat-number([1 to 2]) (: returns the sequence 1, 2 :)[ 'a'1984.42, 'b', 'c' ] = 'b00.0e0' ) (: returns true :)<a>{ [ 1, 2 ] }</a> (: returns <a>1 2</a> :)array { 1 to 2 } + 3 (: error: the left operand returns two items yields 19.8e2 :)

Atomization also applies to function arguments. The following query returns 5==fn:tokenize== If no separator is specified as second argument, because the array a string will be atomized to a sequence of 5 integerstokenized at whitespace boundaries:

<pre classlang="brush:'xquery"'>let $f fn:= functiontokenize($x as xs:integer*" a b c d") { count ($x) }return $f([1 to 5]: yields "a", "b", "c", "d" :)

However, the next query returns 1, because the array is already of the general type {{Code|item()}}, and no atomization will take place==fn:trace== The second argument can now be omitted:

<pre classlang='xquery'>fn:trace(<xml/>, "Node: "brush)/node() (:xqueryyields the debugging output "Node: <xml/>" :),let $f fn:= function($x as itemtrace(<xml/>)*) { count/node($x) }return $f ([1 to 5]: returns the debugging output "<xml/>" :)

Arrays can be compared with the {{Code|==fn:deepstring-equal}} function. join== The [[Array Module]] describes type of the available set of array functions.first argument is now <code>xs:anyAtomicType*</code>, and all items will be implicitly cast to strings:

<pre lang==Functions=='xquery'>fn:string-join(1 to 3) (: yields the string "123" :)</pre>

The following functions of the [http==fn://www.w3.org/TR/xpathdefault-functions-31/ XQuery 3.1 Functions and Operators] Working Draft have already been implemented:language==

* [[Map Module|Map functions]]: <code>map:merge</code>, <code>map:size</code>, <code>map:keys</code>, <code>map:contains</code>, <code>map:get</code>, <code>map:entry</code>, <code>map:put</code>, <code>map:remove</code>, <code>map:Returns the default language used for-each-entry</code>* [[Array Moduleformatting numbers and dates. BaseX always returns {{Code|Array functions]]: <code>array:size</code>, <code>array:append</code>, <code>array:subarray</code>, <code>array:remove</code>, <code>array:insert-before</code>, <code>array:head</code>, <code>array:tail</code>, <code>array:reverse</code>, <code>array:join</code>, <code>array:for-each-member</code>, <code>array:filter</code>, <code>array:for-each-pair</code>* <code>fn:contains-token($input as xs:string*, $token as string, $collation as xs:string) as xs:boolean</code>: Checks if the input strings contain a given tokenen}}.

* The three functions <code>fn:tokenize($string as xstransform</code>, <code>fn:string) as xsload-xquery-module</code> and <code>fn:string*collation-key</code>: Splits may be added in a string at whitespace boundariesfuture version of BaseX as their implementation might require the use of additional external libraries.

==Binary Data==

Items of type <code>xs:hexBinary</code> and <code>xs:base64Binary</code> can now be compared against each other. The following queries all yieldl yield {{Code|true}}:

<pre classlang="brush:'xquery"'>

=Collations=Pending Features XQuery 3.1 provides a default collation, which allows for a case-insensitive comparison of ASCII characters (<code>A-Z</code> =<code>a-z</code>). This query returns <code>true</code>: <pre lang='xquery'>declare default collation 'http://www.w3.org/2005/xpath-functions/collation/html-ascii-case-insensitive';'HTML' = 'html'</pre> If the [http://site.icu-project.org/download ICU Library] is downloaded and added to the classpath, the full [https://www.w3.org/TR/xpath-functions-31/#uca-collations Unicode Collation Algorithm] features become available in BaseX: <pre lang='xquery'>(: returns 0 (both strings are compared as equal) :)compare('a-b', 'ab', 'http://www.w3.org/2013/collation/UCA?alternate=shifted')</pre> =Enclosed Expressions=

The features of XQuery 3.1 ''Enclosed expression'' is the syntactical term for the expressions that are still subject to changespecified inside a function body, try/catch clauses, but we are planning to add node constructors and some other expressions. In the following enhancements in near futureexample expressions, its the empty sequence:

* Arrow operator (<code>pre lang='xquery'></code>declare function local:x() { () };try { (): applies a function to an item, using the item as the first argument to the function. The expression <code>$i=>$f} catch * { ()</code> is equivalent to <code>$f},element x { ($i)</code>}, and <code>$i=>$ftext { ($j)}</codepre>  With XQuery 3.1, the expression can be omitted. The following query is equivalent to the upper one: <codepre lang='xquery'>$fdeclare function local:x($i, $j)</code>.{ };try { } catch * Support for JSON: <code>fn:json-doc</code>{ }, <code>fn:parse-json</code>* New functions: <code>array:fold-left</code>, <code>array:fold-right</code>, <code>fn:collation-key</code>, <code>fn:load-module</code>, <code>fn:parse-ietf-date</code>, <code>fn:transform</code>, <code>map:for-each-entry</code>element x { }text { }* Support for scientific notation: <code>format-number</codepre>