Changes

This [[Module Library|XQuery Module]] provides functions for organizing running registering new query jobs and orchestrating existing jobs. Jobs can be queries, commands , operations performed by a database client, and queries (…more to come)HTTP requests.

All functions in this module are assigned to the <code><nowiki>http://basex.org/modules/jobsjob</nowiki></code> namespace, which is statically bound to the {{Code|jobsjob}} prefix. Errors will be bound to the same prefix.

=FunctionsServices=

A job can be registered as ''service'' by supplying the {{Code|service}} option to {{Function||job:eval}}: <pre lang='xquery'>(: register job as service; will be run every day at 1 am :)job:eval('db:drop("tmp")', (), map { 'id':'cleanup', 'start':'01:00:00', 'interval':'P1D', 'service': true() }), (: list registered services :)job:services(),(: result: <job base-uri="..." id="cleanup" interval="P1D" start="01:00:00">db:drop("tmp")</job> :) (: unregister job :)job:remove('cleanup', map { 'service': true() })</pre> '''Some more notes:''' * All job services will be scheduled for evaluation when the BaseX server or BaseX HTTP server is started.* If a job service is outdated (e.g. because a supplied end time has been exceeded), it will be removed from the jobs file at startup time.* The job definitions are stored in a {{Code|jobs.xml}} file in the database directory. It can also be edited manually. =Executing Jobs= There are cases in which a client does not, or cannot, wait until a request is fully processed. The client may be a browser, which sends an HTTP request to the server to start another time-consuming query job. The functions in this section allow you to register new query jobsand access existing ones. Jobs can be executed immediately (i.e., as soon as a [[Transaction Management#Concurrency Control|free slot is available]]) or scheduled for repeated execution. Each registered job gets a job ID, and the ID can be used to retrieve a query result, stop a job, or wait for its termination. ==job:currenteval==

|-valign="top"| width='120' | '''SignaturesSignature'''|<pre>job:eval( $query as xs:anyAtomicType, $bindings as map(*)? := map {}, $options as map(*)? := map {Func|jobs:current||}) as xs:string}}</pre>|-valign="top"

|Schedules the evaluation a new query job for the supplied {{Code|$query}} (of type {{Code|xs:string}}, or of type {{Code|xs:anyURI}} if points to a resource), and returns a job ID. The job will be queued until a free slot is available, and the query result can be cached. Queries can be updating, and variables and the context value can be declared via {{Code|$bindings}} (see {{Function|XQuery|xquery:eval}} for more details). The following {{Code|$options}} can be supplied:* {{Code|cache}}: indicates if the query result will be cached or ignored (default: {{Code|false}}):** The result will be cached in main-memory until it is fetched via {{Function||job:result}}, or until {{Option|CACHETIMEOUT}} is exceeded.** If the query raises an error, it will be cached and returned instead.* {{Code|start}}: a dayTimeDuration, time, dateTime or integer can be specified to delay the execution of the query:** If a dayTimeDuration is specified, the query will be queued after the specified duration has passed. Examples of valid values are: <code>P1D</code> (1 day), <code>PT5M</code> (5 minutes), <code>PT0.1S</code> (100 ms). An error will be raised if a negative value is specified.** If a dateTime is specified, the query will be executed at this date. Examples for valid values are: <code>2018-12-31T23:59:59</code> (New Year's Eve 2018, close to midnight). An error will be raised if the specified time lies in the past.** If a time is specified, the query will be executed at this time of the day. Examples of valid times are: <code>02:00:00</code> (2am local time), <code>12:00:00Z</code> (noon, UTC). If the time lies in the past, the query will be executed the next day.** An integer will be interpreted as minutes. If the specified number is greater than the elapsed minutes of the current hour, the query will be executed one hour later.* {{Code|interval}}: a dayTimeDuration string can be specified to execute the query periodically. An error is raised if the specified interval is less than one second (<code>PT1S</code>). If the next scheduled call is due, and if a query with the same ID is still running, it will be skipped.* {{Code|end}}: scheduling can be stopped after a given time or duration. The string format is the same as for {{Code|start}}. An error is raised if the resulting end time is smaller than the start time.* {{Code|base-uri}}: sets the [https://www.w3.org/TR/xquery-31/#dt-static-base-uri base-uri property] for the query. This URI will be used when resolving relative URIs, such as with {{Code|fn:doc}}.* {{Code|id}}: sets a custom job ID. The ID must not start with the standard <code>job</code> prefix, and it can only be assigned if no job with the same name exists.* {{Code|service}}: additionally registers the job as [[#Services|service]]. Registered services must have no variable bindings.* {{Code|log}}: writes the specified string to the [[Logging|database logs]]. Two log entries are stored, one at the beginning and another one after the execution of the job.|- valign="top"| '''Errors'''|{{Error|overflow|#Errors}} Query execution is rejected because too many jobs are queued or being executed. {{Option|CACHETIMEOUT}} can be decreased if the default setting is too restrictive.<br/>{{Error|range|#Errors}} A specified time or duration is out of range.<br/>{{Error|id|#Errors}} The specified ID is invalid or has already been assigned.<br/>{{Error|options|#Errors}} The specified options are conflicting.|- valign="top"| '''Examples'''|* Cache query result. The returned ID can be used to pick up the result with {{Function||job:result}}:<pre lang='xquery'>job:eval("1+3", (), map { 'cache': true() })</pre>* A happy birthday mail will be sent at the given date:<pre lang='xquery'>job:eval("import module namespace mail='mail'; mail:send('Happy birthday!')", (), map { 'start': '2018-09-01T06:00:00' })}}</pre>* The following [[RESTXQ]] functions can be called to execute a query at 2 am every day. An ID will be returned by the first function, which can be used to stop the scheduler via the second function:<pre lang='xquery'>declare %rest:POST("{$query}") %rest:path('/start-scheduling') function local:start($query) { job:eval($query, (), map { 'start': '02:00:00', 'interval': 'P1D' })};declare %rest:path('/stop-scheduling/{$id}') function local:stop($id) { job:remove($id)};</pre>* Query execution is scheduled for every second, and for 10 seconds in total. As the query itself will take 1.5 seconds, it will only be executed every second time:<pre lang='xquery'>job:eval("prof:sleep(1500)", (), map { 'interval': 'PT1S', 'end': 'PT10S' })</pre>* The query in the specified file will be evaluated once:<pre lang='xquery'>job:eval(xs:anyURI('cleanup.xq'))</pre>* The following expression, if stored in a file, will be evaluated every 5 seconds:<pre lang='xquery'>job:eval( static-base-uri(), map { }, map { 'start': 'PT5S' })</pre>|} ==job:result== {| width='100%'|- valign="top"| width='120' | '''Signature'''|<pre>job:result( $id as xs:string, $options as map(*)? := map { }) as item()*</pre>|- valign="top"| '''Summary'''|Returns the cached result of a job with the specified job {{Code|$id }}:* If the original job has raised an error, the cached error will be raised instead.* The cached result or error will be dropped after it has been retrieved.* If the result has not been cached or if it has been dropped, an empty sequence is returned.The following {{Code|$options}} can be supplied:* {{Code|keep}}: Keep the cached result or error after retrieval. |- valign="top"| '''Examples'''|* The following [[RESTXQ]] function will either return the result of a previously started job or raise an error:<pre lang='xquery'>declare %rest:path('/result/{$id}') function local:result($id) { job:result($id)};</pre>* The following query demonstrates how the results of an executed query can be returned within the current same query (see below why you should avoid this pattern in practice):<pre lang='xquery'>let $query := job:eval('(1 to 10000000)[. = 1]', map { }, map { 'cache': true() })return ( job:wait($query ), job:result($query))</pre>Queries of this kind can cause deadlocks! If the original query and the new query perform updates on the same database, the second query will only be run after the first one has been executed, and the first query will wait for the second query forever. You should resort to {{Function|XQuery|xquery:fork-join}} if you want to have full control on parallel query execution.

==jobsjob:listremove==

|-valign="top"| width='120' | '''SignaturesSignature'''|{{Func|jobs<pre>job:list||remove( $id as xs:string, $options as map(*)? := map { }}) as empty-sequence()</pre>|-valign="top"

|Returns Triggers the ids cancelation of all a job with the specified {{Code|$id}}, cancels a scheduled job or removes a cached result. Unknown IDs are ignored. All jobs that are currently queued or executedgracefully stopped; it is up to the process to decide when it is safe to shut down. The following {{Code|$options}} can be supplied:* {{Code|service}}: additionally removes the job from the [[#Services|job services]] list.|-valign="top"

* <code>jobsjob:list()[. != job:current()] ! job:remove(.)</code> returns stops and discards all jobs except for the same job id as [[#jobs:current|jobs:current]] if no other job is runningone.* <code>jobsjob:listremove() ! jobsjob:stopcurrent(.))</code> stops and invalidates all asynchronous queries and resultsinterrupts the current job.

==jobsjob:finishedwait==

|-valign="top"| width='120' | '''SignaturesSignature'''|{{Func|jobs<pre>job:finished|wait( $id as xs:string|xs:boolean}}) as empty-sequence()</pre>|-valign="top"

|Indicates if Waits for the evaluation completion of a job with the specified job {{Code|$id}} has finished:* <code>false</code> indicates that The function will terminate immediately if the job ID is unknown. This is the case if a future job with this id is currently runninghas not been queued yet, or if the ID has already been discarded after job evaluation.* <code>true</code> indicates that If the function is called with the ID of a queued job has either finished, or that the repeatedly executed job, it may stall and never terminate.|- valign="top"| '''Errors'''|{{Error|self|#Errors}} The current job id is unknownaddressed.<br/>

=Listing Jobs=jobs ==job:stopcurrent==

|-valign="top"| width='120' | '''SignaturesSignature'''|{{Func|jobs<pre>job:stop|$id current() as xs:string|empty-sequence()}}</pre>|-valign="top"

|Cancels Returns the execution ID of a the current job with the specified {{Code|$id}}, or drops the cached result of a query. Unknown ids are ignored.

=Asynchronous Execution=job:list==

Asynchronous query execution is recommendable if a client does not, or cannot, wait until a request is fully processed. This is e{| width='100%'|- valign="top"| width='120' | '''Signature'''|<pre>job:list() as xs:string*</pre>|- valign="top"| '''Summary'''|Returns the IDs of all jobs that are currently registered. g. the case with web browsersThe list includes scheduled, which will usually cancel a request after a specific timeout. In such casesqueued, you can use asynchronous execution to trigger another server-side processrunning, which will start the time-consuming processstopped, and fetch finished jobs with cached results.|- valign="top"| '''Examples'''| <code>job:list()</code> returns the result later on same job ID as soon as it {{Function||job:current}} if no other job is availableregistered.|}

==jobsjob:evallist-details==

|-valign="top"| width='120' | '''SignaturesSignature'''|{{Func|jobs:eval|$query as xs:string|xs:string}}<br /pre>{{Func|jobsjob:eval|list-details( $query id as xs:string, $bindings as map := (*)|xs:string}}<br />{{Func|jobs:eval|$query as xs:string, $bindings as map(*), $options as mapelement(xs:string, xs:stringjob)|xs:string}}*<br /pre>|-valign="top"

|Prepares Returns information on all jobs that are currently registered, or on a job with the supplied specified {{Code|$queryid}} string for asynchronous execution and returns a query id(or an empty sequence if this job is not found). The query will be list includes scheduled, queued as described in the article on [[Transaction Management]], running jobs, and the result will be cached in main-memory until it is fetched via [[#jobs:result|jobs:result]]. A string representation of the job, or until {{Option|ASYNCTIMEOUT}} is exceeded. Queries may its URI, will be updatingreturned as a value.The returned elements have additional attributes:* <code>id<br/code>Variables and context items can be declared via {{Code|$bindings}} (see [[XQuery Module#xquery:eval|xquery:eval]] for more details). The {{Code|$options}} parameter contains evaluation options:job ID* {{Code|cache}}: indicates if the query result will be cached or ignored (default: <code>truetype</code>). If : type of the job (command, query result will not be cached, the query id will immediately be discarded after query executionREST, RESTXQ, tooetc.)* {{Code|base-uri}}<code>state</code>: set [httpscurrent state of the job:<code>scheduled</code>, <code>queued</www.w3.orgcode>, <code>running</TRcode>, <code>cached</xquery-31code>* <code>user</#dt-static-base-uri base-uri property] for code>: user who started the query. This URI will be used when resolving relative URIs by functions such as {{Code|fnjob* <code>duration</code>:doc}} evaluation time (defaultincluded if a job is running or if the result was cached)* <code>start</code>: ''empty string''next start of job (included if a job will be executed repeatedly).|-| '''Errors'''|{{Error|overflow|#Errors}} Too many queries or query results are queued. To fix this, the query results should be retrieved.* <code>time<br/code>: time when job was registered|-valign="top"

|* {{Code|jobs<code>job:eval("1+3")}} returns a query id, e.g. {{Code|Querylist-abc}}. The result can be retrieved via a second query in the same BaseX context: {{Code|jobs:resultdetails("Query-abc")}}<br /code>* The following [[RESTXQ]] function will return the id of the query thread, which evaluates the query that has been specified in returns information on the body of a POST requestcurrently running job and possibly others:<pre classlang='brush:xquery'"xml">declare %rest:POST(<job id="job1" type="XQuery" state="running" user="admin"{$query}duration=") %rest:path('/eval') function local:eval($query) {PT0.001S"> jobsXQUERY job:evallist-details($query)};</job>

==jobsjob:resultbindings==

|-valign="top"| width='120' | '''SignaturesSignature'''|{{Func|jobs<pre>job:result|bindings( $id as xs:string|item) as map(*)*}}</pre>|-valign="top"

|Returns the cached result variable bindings of a query an existing job with the specified job {{Code|$id}}. If no variables have been bound to this job, an empty map is returned.|} ==job:finished== {| width='100%'* Results can only |- valign="top"| width='120' | '''Signature'''|<pre>job:finished( $id as xs:string) as xs:boolean</pre>|- valign="top"| '''Summary'''|Indicates if the evaluation of an already running job with the specified {{Code|$id}} has finished. As the IDs of finished jobs will usually be retrieved once. After retrievaldiscarded, unless caching is enabled, the cached result function will be droppedalso return <code>true</code> for unknown jobs.* <code>false</code> indicates that the job ID is scheduled, queued, or currently running.* If <code>true</code> will be returned if the query raised an errorjob has either finished, or if the cached error ID is unknown (because the IDs of all finished jobs will not be raised insteadcached).|} ==job:services== {| width='100%'|- valign="top"| width='120' | '''Signature'''|<pre>job:services() as element(job)*</pre>|- valign="top"| '''Summary'''|Returns a list of all jobs that have been persistently registered as [[#Services|Services]].|-valign="top"

|{{Error|runningservices|#Errors}} the query is still runningRegistered services cannot be parsed.<br/>{{Error|unknown|#Errors}} the supplied query id is unknown: The query result may already have been retrieved, or query execution may have been stopped.<br/>|-| '''Examples'''|* The following [[RESTXQ]] function will either return the result of a previously started query or an error:<pre class='brush:xquery'>declare %rest:path('/result/{$id}') function local:result($id) { jobs:result($id)};</pre>* The following query demonstrates how the results of an asynchronously executed query can be returned in a single query. Please remember that this is not the common way how these functions are used in practice:<pre class='brush:xquery'>let $query := jobs:eval('(1 to 10000000)[. = 1]')return ( hof:until( function($result) { jobs:finished($query) }, function($curr) { prof:sleep(10) }, () ), jobs:result($query))</pre>

|-valign="top"|{{Code|unknownoptions}}| The supplied query specified options are conflicting.|- valign="top"|{{Code|id }}| The specified ID is unknown invalid or has already been assigned.|- valign="top"|{{Code|overflow}}| Too many queries or query results are queued.|- valign="top"|{{Code|range}}| A specified time or not available anymoreduration is out of range.|-valign="top"

|-valign="top"|{{Code|overflowself}}| Too many queries The current job cannot be addressed.|- valign="top"|{{Code|service}}| Registered services cannot be parsed, added or query results are queuedremoved.