Bureaucrats, editor, reviewer, Administrators
13,554
edits
Changes
Jump to navigation
Jump to search
m
Text replacement - "syntaxhighlight" to "pre"
= BaseX as Twitter Storage=
For retrieving the Twitter stream we connect with the Streaming API to the endpoint of Twitter and receive a never -ending tweet stream. As Twitter delivers the tweets as [https://www.json.org/ JSON] objects , the objects has to bedata is converted into XML fragments. For this purpose , the parse function of the [[JSON Module|XQuery JSON Module]] is used. In the examples section both versions are shown ([[#Example Tweet (JSON)|tweet as JSON]] and [[#Example Tweet (XML)|tweet as XML]]). For storing the tweets including the meta-data, we use the standard ''insert'' function of [[Updates|XQuery Update]].
=Twitter’s Streaming Data=
==Example Tweet (JSON)==
<syntaxhighlight pre lang="json">
{
"contributors": null,
"coordinates": null
}
</syntaxhighlightpre>
==Example Tweet (XML)==
<syntaxhighlight pre lang="xml">
<json booleans="retweeted possibly__sensitive truncated favorited possibly__sensitive__editable default__profile profile__background__tile protected contributors__enabled verified default__profile__image geo__enabled profile__use__background__image show__all__inline__media is__translator"
numbers="id retweet__count statuses__count favourites__count followers__count friends__count listed__count"
<coordinates/>
</json>
</syntaxhighlightpre>
= BaseX Performance =
The test show the time BaseX needs to insert large amounts of real tweets into a database. We can derive that BaseX scales very well and can keep up
with the incoming amount of tweets in the stream. Some lower values can occur, cause the size of the tweets differ according to the meta-data contained in the tweet object.<br />
Note: The {{Option|AUTOFLUSH}} option is set to <code>FALSE</code>.
