Changes

This [[Module Library|XQuery Module]] provides functions for evaluating XQuery expressions in separate threadsregistering new query jobs and orchestrating existing jobs. Query execution Jobs can both be parallelized queries, commands, operations performed by a database client, and postponed to be executed asynchronouslyHTTP requests.

All functions in this module are assigned to the <code><nowiki>http://basex.org/modules/asyncjob</nowiki></code> namespace, which is statically bound to the {{Code|asyncjob}} prefix. Errors will be bound to the same prefix.

=Parallelized ExecutionServices=

Parallel query execution is recommendable if you have various calls that require a lot of time, but cannot A job can be sped up registered as ''service'' by rewriting supplying the code. This is e. g. the case if external URLs are called. If you are parallelizing local data reads (such as accessing a database), your single-threaded query will usually be faster, because parallelized access {{Code|service}} option to disk data will often lead to randomized access patterns, which can hardly be optimized by your HD or SSD.{{Function||job:eval}}:

<pre lang==async'xquery'>(: register job as service; will be run every day at 1 am :)job:eval('db:drop("tmp")', (), map { 'id':'cleanup', 'start':'01:00:00', 'interval':'P1D', 'service':fork-join==true() }),

{| width='100%'|-| width='120' | '''Signatures'''(: list registered services :)|{{Func|asyncjob:fork-join|$functions as function(*)*|itemservices()*}}|-|'''Summary''',|This function executes the supplied (non: result: <job base-updating) functions in paralleluri="...|-| '''Examples'''|* The following function sleeps in parallel; it will be finished in 1 second if your system has at least 2 cores" id="cleanup" interval="P1D" start="01:<pre class='brush00:xquery'00">asyncdb:fork-joindrop("tmp")</job> :) for $i in 1 to 2 return function(: unregister job :) job:remove('cleanup', map { prof'service':sleeptrue(1000) })

'''Some more notes:''' * All job services will be scheduled for evaluation when the BaseX server or BaseX HTTP server is started.* If a job service is outdated (e.g. because a supplied end time has been exceeded), it will be removed from the jobs file at startup time.* The job definitions are stored in a {{Code|jobs.xml}} file in the database directory. It can also be edited manually. =Asynchronous ExecutionExecuting Jobs=

Asynchronous query execution is recommendable if There are cases in which a client does not, or cannot, wait until a request is fully processed. This is e. g. the case with web browsersThe client may be a browser, which will usually cancel a sends an HTTP request after a specific timeout. In such cases, you can use asynchronous execution to trigger another the server-side process, which will to start the another time-consuming processquery job. The functions in this section allow you to register new query jobs and access existing ones. Jobs can be executed immediately (i.e., and fetch the result later on as soon as it a [[Transaction Management#Concurrency Control|free slot is available]]) or scheduled for repeated execution. Each registered job gets a job ID, and the ID can be used to retrieve a query result, stop a job, or wait for its termination.

==asyncjob:eval==

|-valign="top"| width='120' | '''SignaturesSignature'''|{{Func|async:eval|$query as xs:string|xs:string}}<br /pre>{{Func|asyncjob:eval|( $query as xs:stringanyAtomicType, $bindings as map(*)|xs? :string= map { }}<br />{{Func|async:eval|$query as xs:string, $bindings options as map(*), $options as ? := map(xs:string, xs:string{ })|as xs:string}}<br /pre>|-valign="top"

|Prepares Schedules the evaluation a new query job for the supplied {{Code|$query}} (of type {{Code|xs:string for asynchronous execution }}, or of type {{Code|xs:anyURI}} if points to a resource), and returns a query idjob ID. The query job will be queued as described in the article on [[Transaction Management]]until a free slot is available, and the query result will can be cached in main-memory until it is fetched via [[#async:result|async:result]], or until {{Option|ASYNCTIMEOUT}} is exceeded. Queries may can be updating.<br/>Variables , and variables and the context items value can be declared via {{Code|$bindings}} (see [[{{Function|XQuery Module#xquery:eval|xquery:eval]] }} for more details). The following {{Code|$options}} parameter contains evaluation optionscan be supplied:* {{Code|cache}}: indicates if the query result will be cached or ignored (default: {{Code|false}}):** The result will be cached in main-memory until it is fetched via {{Function||job:result}}, or until {{Option|CACHETIMEOUT}} is exceeded.** If the query raises an error, it will be cached and returned instead.* {{Code|start}}: a dayTimeDuration, time, dateTime or integer can be specified to delay the execution of the query:** If a dayTimeDuration is specified, the query will be queued after the specified duration has passed. Examples of valid values are: <code>P1D</code> (1 day), <code>PT5M</code> (5 minutes), <code>PT0.1S</code> (100 ms). An error will be raised if a negative value is specified.** If a dateTime is specified, the query will be executed at this date. Examples for valid values are: <code>true2018-12-31T23:59:59</code>(New Year's Eve 2018, close to midnight). An error will be raised if the specified time lies in the past.** If a time is specified, the query result will not be cachedexecuted at this time of the day. Examples of valid times are: <code>02:00:00</code> (2am local time), <code>12:00:00Z</code> (noon, UTC). If the time lies in the past, the query will be executed the next day.** An integer will be interpreted as minutes. If the specified number is greater than the elapsed minutes of the current hour, the query id will immediately be discarded after executed one hour later.* {{Code|interval}}: a dayTimeDuration string can be specified to execute the query periodically. An error is raised if the specified interval is less than one second (<code>PT1S</code>). If the next scheduled call is due, and if a query executionwith the same ID is still running, tooit will be skipped.* {{Code|end}}: scheduling can be stopped after a given time or duration. The string format is the same as for {{Code|start}}. An error is raised if the resulting end time is smaller than the start time.* {{Code|base-uri}}: set sets the [https://www.w3.org/TR/xquery-31/#dt-static-base-uri base-uri property] for the query. This URI will be used when resolving relative URIs by functions , such as with {{Code|fn:doc}} (default.* {{Code|id}}: sets a custom job ID. The ID must not start with the standard <code>job</code> prefix, and it can only be assigned if no job with the same name exists.* {{Code|service}}: additionally registers the job as [[#Services|service]]. Registered services must have no variable bindings.* {{Code|log}}: ''empty writes the specified string'')to the [[Logging|database logs]]. Two log entries are stored, one at the beginning and another one after the execution of the job.|-valign="top"

|{{Error|overflow|#Errors}} Too Query execution is rejected because too many queries or query results jobs are queuedor being executed. To fix this, {{Option|CACHETIMEOUT}} can be decreased if the query results should be retrieveddefault setting is too restrictive.<br/>{{Error|range|#Errors}} A specified time or duration is out of range.<br/>{{Error|id|#Errors}} The specified ID is invalid or has already been assigned.<br/>{{Error|options|#Errors}} The specified options are conflicting.|-valign="top"

* Cache query result. The returned ID can be used to pick up the result with {{CodeFunction|async|job:result}}:<pre lang='xquery'>job:eval("1+3", ()}} returns a query id, e.g. map {{Code|Query-abc'cache': true() }}. The result can )</pre>* A happy birthday mail will be retrieved via a second query in sent at the same BaseX contextgiven date:<pre lang='xquery'>job: {{Code|asynceval("import module namespace mail='mail'; mail:resultsend('Happy birthday!')"Query, (), map { 'start': '2018-abc"09-01T06:00:00' })}}<br /pre>* The following [[RESTXQ]] function functions can be called to execute a query at 2 am every day. An ID will return be returned by the id of the query threadfirst function, which evaluates can be used to stop the query that has been specified in scheduler via the body of a POST requestsecond function:<pre classlang='brush:xquery'>declare %rest:POST("{$query}") %rest:path('/evalstart-scheduling') function local:evalstart($query) { asyncjob:eval($query, (), map { 'start': '02:00:00', 'interval': 'P1D' })};declare %rest:path('/stop-scheduling/{$id}') function local:stop($id) { job:remove($id)

==asyncjob:result==

|-valign="top"| width='120' | '''SignaturesSignature'''|{{Func|async<pre>job:result|( $id as xs:string|, $options as map(*)? := map { }) as item()*}}</pre>|-valign="top"

|Returns the cached result of an asynchronously executed query a job with the specified query job {{Code|$id}}:* Results can only be retrieved once. After retrieval, the cached result will be discarded.* If the query original job has raised an error, the cached error will be raised instead.|-* The cached result or error will be dropped after it has been retrieved.| '''Errors'''* If the result has not been cached or if it has been dropped, an empty sequence is returned.|The following {{ErrorCode|is-running|#Errors$options}} the query is still running.<br/>can be supplied:* {{ErrorCode|unknown|#Errorskeep}} : Keep the supplied query id is unknown: The query cached result may already have been retrieved, or query execution may have been stoppederror after retrieval.<br/> |-valign="top"

* The following [[RESTXQ]] function will either return the result of a previously started query job or raise an error:<pre classlang='brush:xquery'>

asyncjob:result($id)

* The following query demonstrates how the results of an asynchronously executed query can be returned in a single within the same query. Please remember that (see below why you should avoid this is not the common way how these functions are used pattern in practice):<pre classlang='brush:xquery'>let $query := asyncjob:eval('(1 to 10000000)[. = 1]', map { }, map { 'cache': true() })

hofjob:until( function($result) { async:finishedwait($query) }, function($curr) { prof:sleep(10) }, () ), asyncjob:result($query)

==asyncjob:finishedremove==

|-valign="top"| width='120' | '''SignaturesSignature'''|{{Func|async<pre>job:finished|remove( $id as xs:string|xs, $options as map(*)? :boolean}= map { }) as empty-sequence()</pre>|-valign="top"

|Indicates if Triggers the evaluation cancelation of a query job with the specified query {{Code|$id}} has finished, cancels a scheduled job or removes a cached result. Unknown IDs are ignored. All jobs are gracefully stopped; it is up to the process to decide when it is safe to shut down. The following {{Code|$options}} can be supplied:* {{Code|service}}: additionally removes the job from the [[#Services|job services]] list.|- valign="top"| '''Examples'''|* <code>job:list()[. != job:current()] ! job:remove(.)</code> stops and discards all jobs except for the current one. If * <code>falsejob:remove(job:current())</code> is returned, interrupts the query is still runningcurrent job. An error |} ==job:wait== {| width='100%'|- valign="top"| width='120' | '''Signature'''|<pre>job:wait( $id as xs:string) as empty-sequence()</pre>|- valign="top"| '''Summary'''|Waits for the completion of a job with the specified {{Code|$id}}:* The function will be raised terminate immediately if the query result was job ID is unknown. This is the case if a future job has not cached been queued yet, or if the ID has already been retrieveddiscarded after job evaluation.* If the function is called with the ID of a queued job, or repeatedly executed job, it may stall and never terminate.|-valign="top"

|{{Error|unknownself|#Errors}} the supplied query id The current job is unknown: The query result may already have been retrieved, or query execution may have been stoppedaddressed.<br/>

=Listing Jobs=async ==job:stopcurrent==

|-valign="top"| width='120' | '''SignaturesSignature'''|{{Func|async<pre>job:stop|$id current() as xs:string|empty-sequence()}}</pre>|-valign="top"

|Cancels Returns the execution ID of a query with the specified query current job.|} ==job:list== {{Code|$id}}width='100%'|- valign="top"| width='120' | '''Signature'''|<pre>job:list() as xs:string*</pre>|- valign="top"| '''Summary'''|Returns the IDs of all jobs that are currently registered. The list includes scheduled, queued, running, stopped, or drops the query result if it has already been executedand finished jobs with cached results.|-valign="top"| '''ErrorsExamples'''|<code>job:list()</code> returns the same job ID as {{ErrorFunction|unknown|#Errorsjob:current}} the supplied query id if no other job is unknown: The query result may already have been retrieved, or query execution may have been stoppedregistered.<br/>

==asyncjob:idslist-details==

|-valign="top"| width='120' | '''SignaturesSignature'''|{{Func|async<pre>job:ids||list-details( $id as xs:string := ()) as element(job)*}}</pre>|-valign="top"

|Returns the ids of information on all queries jobs that are either being executed asynchronouslycurrently registered, or on a job with the specified {{Code|$id}} (or an empty sequence if this job is not found). The list includes scheduled, queued, running jobs, and cached jobs. A string representation of the job, or that its URI, will be returned as a value. The returned elements have been executed and additional attributes:* <code>id</code>: job ID* <code>type</code>: type of the results job (command, query, REST, RESTXQ, etc.)* <code>state</code>: current state of which have been the job: <code>scheduled</code>, <code>queued</code>, <code>running</code>, <code>cached</code>* <code>user</code>: user who started the job* <code>duration</code>: evaluation time (included if a job is running or if the result was cached.)* <code>start</code>: next start of job (included if a job will be executed repeatedly)* <code>time</code>: time when job was registered|-valign="top"

|<code>job:list-details()</code> returns information on the currently running job and possibly others:<pre lang="xml"><job id="job1" type="XQuery" state="running" user="admin" duration="PT0.001S"> XQUERY job:list-details()</job></pre>|} ==job:bindings== {| width='100%'|- valign="top"| width='120' | '''Signature'''|<pre>job:bindings( $id as xs:string) as map(* )</pre>|- valign="top"| '''Summary'''|Returns the variable bindings of an existing job with the specified {{Code|$id}}. If no variables have been bound to this job, an empty map is returned.|} ==job:finished== {| width='100%'|- valign="top"| width='120' | '''Signature'''|<codepre>asyncjob:idsfinished( $id as xs:string) ! asyncas xs:stopboolean</pre>|- valign="top"| '''Summary'''|Indicates if the evaluation of an already running job with the specified {{Code|$id}} has finished. As the IDs of finished jobs will usually be discarded, unless caching is enabled, the function will also return <code>true</code> for unknown jobs.* <code>false</code> indicates that the job ID is scheduled, queued, or currently running.* <code>true</code> will be returned if the job has either finished, or if the ID is unknown (because the IDs of all finished jobs will not be cached).|} ==job:services== {| width='100%'|- valign="top"| width='120' | '''Signature'''|<pre>job:services()as element(job)*</codepre> stops and invalidates |- valign="top"| '''Summary'''|Returns a list of all asynchronous queries and resultsjobs that have been persistently registered as [[#Services|Services]].|- valign="top"| '''Errors'''|{{Error|services|#Errors}} Registered services cannot be parsed.<br/>

|-valign="top"|{{Code|unexpectedoptions}}| An unexpected error occurred while running a query or function in a separate threadThe specified options are conflicting.|-valign="top"|{{Code|unknownid}}| The supplied query id specified ID is unknown invalid or not available anymore.|-|{{Code|is-running}}| A query is still runninghas already been assigned.|-valign="top"