Bureaucrats, editor, reviewer, Administrators
13,554
edits
Changes
Jump to navigation
Jump to search
m
|-
|{{Code|BXFE0002}}
|The specified encoding is not supported, or unknown.
[[Category:XQuery]]
Text replacement - "<syntaxhighlight lang="xquery">" to "<pre lang='xquery'>"
This [[Module Library|XQuery Module]] provides simple functions to fetch the content of resources identified by URIs. Resources can be stored locally or remotely and e.g. use the {{Code|file://}} or {{Code|http://}} scheme. If more control over HTTP requests is required, the [[HTTP Client Module]] can be used. With the [[HTML Module]], retrieved HTML documents can be converted to XML.
=Conventions=
All functions and errors in this module are assigned to the <code><nowiki>http://basex.org/modules/fetch</nowiki></code> namespace, which is statically bound to the {{Code|fetch}} prefix.<br/>All errors are assigned URI arguments can point be URLs or point to local files. Relative file paths will be resolved against the <code><nowiki>http://basex.org/errors</nowiki></code> namespace''current working directory'' (for more details, which is statically bound to have a look at the {{Code[[File Module#File Paths|bxerr}} prefixFile Module]]).
=Functions=
==fetch:binary==
{| width='100%'
|-valign="top"| width='120' | '''SignaturesSignature'''|{{Func|<pre>fetch:binary|( $uri href as xs:string|) as xs:base64Binary}}<br/pre>|-valign="top"
| '''Summary'''
|Fetches the resource referred to by the given URI {{Code|href}} string and returns it as [[Streaming Lazy Module|streamablelazy]] {{Code|xs:base64Binary}}item.|-valign="top"
| '''Errors'''
|{{Error|BXFE0001open|XQuery Errors#Functions Errors}} the URI could not be resolved, or the resource could not be retrieved.|-valign="top"
| '''Examples'''
|
* <code><nowiki>fetch:binary("http://images.trulia.com/blogimg/c/5/f/4/679932_1298401950553_o.jpg")</nowiki></code> returns the addressed image.
* <code><nowiki>streamlazy:materializecache(fetch:binary("http://en.wikipedia.org"))</nowiki></code> returns a materialized representation of enforces the streamable resultfetch operation (otherwise, it will be delayed until requested first).
|}
==fetch:text==
{| width='100%'
|-valign="top"| width='120' | '''SignaturesSignature'''|{{Func|<pre>fetch:text|( $uri href as xs:string|xs:string}}<br/>{{Func|fetch:text|, $uri encoding as xs:string := (), $encoding fallback as xs:string|boolean? := false()) as xs:string}}<br/pre>|-valign="top"
| '''Summary'''
|Fetches the resource referred to by the given URI {{Code|href}} string and returns it as [[Streaming Lazy Module|streamablelazy]] {{Code|xs:string}}item:* The UTF-8 default encoding can be overwritten with the optional {{Code|$encoding}} argument.* By default, invalid characters will be rejected. If {{Code|$fallback}} is set to true, these characters will be replaced with the Unicode replacement character <code>FFFD</code> (�).|-valign="top"
| '''Errors'''
|{{Error|BXFE0001open|XQuery Errors#Functions Errors}} the URI could not be resolved, or the resource could not be retrieved. Invalid XML characters will be ignored if the <code>[[Options#CHECKSTRINGS|CHECKSTRINGS]]</code> option is turned off.<br/>{{Error|BXFE0002encoding|XQuery Errors#Functions Errors}} the specified encoding is not supported, or unknown.|-valign="top"
| '''Examples'''
|
* <code><nowiki>fetch:text("http://en.wikipedia.org")</nowiki></code> returns a string representation of the English Wikipedia main HTML page.
* <code><nowiki>streamfetch:text("http://www.bbc.com","US-ASCII",true())</nowiki></code> returns the BBC homepage in US-ASCII with all non-US-ASCII characters replaced with �.* <code><nowiki>lazy:materializecache(fetch:text("http://en.wikipedia.org"))</nowiki></code> returns a materialized representation of enforces the streamable resultfetch operation (otherwise, it will be delayed until requested first).
|}
==fetch:xmldoc==
{| width='100%'
|-valign="top"| width='120' | '''SignaturesSignature'''|{{Func|fetch:xml|$uri as xs:string|document-node()}}<br/pre>{{Func|fetch:xml|doc( $uri href as xs:string, $options as map(*)|? := map { }) as document-node()}}</pre>|-valign="top"
| '''Summary'''
|Fetches the resource referred to by the given {{Code|$urihref}} string and returns it as an XML document.<br/>In contrast to <code>fn:doc</code>, each function call returns a different document node. As a consequence, document instances created by this function will not be kept in memory until the end of query evaluation.<br/>The {{Code|$options}} argument can be used to change the parsing behavior. Allowed options are all [[Options#Parsing|parsing]] and [[Options#XML Parsing|XML parsing]] options in lower case.<br/>The function differs from {{Code|fn:doc}} in various aspects:* It is ''nondeterministic'', i.e., a new document node will be created by each call of this function.* A document created by this function will be garbage-collected as soon as it is not referenced anymore.* URIs will not be resolved against existing databases. As a result, it will not trigger any locks (see [[Transaction Management#Limitations|limitations of database locking]] for more details).|-valign="top"
| '''Errors'''
|{{Error|BXFE0001open|XQuery Errors#Functions Errors}} the URI could not be resolved, or the resource could not be retrieved.|-valign="top"
| '''Examples'''
|The following expression returns * Retrieve an XML representation of the English Wikipedia main HTML page and chops all with whitespace nodesstripped:<pre classlang="brush:'xquery"'>fetch:xmldoc("http://en.wikipedia.org", map { 'chopstripws': true() })</pre>* Return a web page as XML, preserve namespaces:<pre lang='xquery'>fetch:doc( 'http://basex.org/', map { 'parser': 'html', 'htmlparser': map { 'nons': false() } })
</pre>
|}
==fetch:binary-doc==
{| width='100%'
|- valign="top"
| width='120' | '''Signature'''
|<pre>fetch:binary-doc(
$input as xs:anyAtomicType,
$options as map(*)? := map { }
) as document-node()</pre>
|- valign="top"
| '''Summary'''
|Converts the specified {{Code|$input}} ({{Code|xs:base64Binary}}, {{Code|xs:hexBinary}}) to XML and returns it as a document node.<br/>In contrast to {{Code|fn:parse-xml}}, which expects a string, the input can be arbitrarily encoded. The encoding will be derived from the XML declaration or (in case of UTF-16 or UTF-32) from the first bytes of the input.<br/>The {{Code|$options}} argument can be used to change the parsing behavior. Allowed options are all [[Options#Parsing|parsing]] and [[Options#XML Parsing|XML parsing]] options in lower case.
|- valign="top"
| '''Examples'''
|
* Retrieves file input as binary data and parses it as XML:
<pre lang='xquery'>
fetch:binary-doc(file:read-binary('doc.xml'))
</pre>
* Encodes a string as CP1252 and parses it as XML. The input and the string {{Code|touché}} will be correctly decoded because of the XML declaration:
<pre lang='xquery'>
fetch:binary-doc(convert:string-to-base64(
"<?xml version='1.0' encoding='CP1252'?><xml>touché</xml>",
"CP1252"
))
</pre>
* Encodes a string as UTF-16 and parses it as XML. The document will be correctly decoded, as the first bytes of the data indicate that the input must be UTF-16:
<pre lang='xquery'>
fetch:binary-doc(convert:string-to-base64("<xml/>", "UTF16"))
</pre>
|- valign="top"
| '''Errors'''
|{{Error|open|#Errors}} the input could not be parsed.
|}
==fetch:content-type==
{| width='100%'
|-valign="top"| width='120' | '''SignaturesSignature'''|{{Func|<pre>fetch:content-type|( $uri href as xs:string|) as xs:string}}<br/pre>|-valign="top"
| '''Summary'''
|Returns the content-type (also called mime-type) of the resource specified by {{Code|$urihref}}string:
* If a remote resource is addressed, the request header will be evaluated.
* If the addressed resource is locally stored, the content-type will be guessed based on the file extension.
|-valign="top"
| '''Errors'''
|{{Error|BXFE0001open|XQuery Errors#Functions Errors}} the URI could not be resolved, or the resource could not be retrieved.|-valign="top"
| '''Examples'''
|
! width="110"|Code
|Description
|-valign="top"|{{Code|BXFE0001encoding}}|The specified encoding is not supported, or unknown.|- valign="top"|{{Code|open}}
|The URI could not be resolved, or the resource could not be retrieved.
|}
=Changelog=
;Version 10.0
* Updated: {{Function||fetch:doc}} renamed (before: {{Code|fetch:xml}}).
* Updated: {{Function||fetch:binary-doc}} renamed (before: {{Code|fetch:xml-binary}}).
;Version 9.0
* Added: {{Code|fetch:xml-binary}}
* Updated: error codes updated; errors now use the module namespace
;Version 8.5
* Updated: {{Function||fetch:text}}: <code>$fallback</code> argument added.
;Version 8.0
* Added: [[#fetch:xml{{Code|fetch:xml]]}}
The module was introduced with Version 7.6.
