1. Scripting
Documents
with XQuery
Vyacheslav
Zholudev, Scripting Documents with XQuery –
Michael
Kohlhase
Virtual Documents in TNTBase
TNTBase
Recap
Virtual
Documents
Vyacheslav Zholudev, Michael Kohlhase
VDoc Specifications
VDoc FS Entity
Querying VDocs Jacobs University Bremen
VDoc Editing
Conclusion KWARC – Knowledge Adaptation and Reasoning for Content
Demo
DFKI, Bremen
Use Cases
Future work
September 16, 2010
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 1

2. TNTBase – Versioned Storage for XML
Scripting
Documents
with XQuery
Vyacheslav
Zholudev,
Michael
Kohlhase
TNTBase
Recap Why Versioning ?
Virtual
Documents Why XML?
VDoc Specifications
VDoc FS Entity
Querying VDocs
VDoc Editing
Conclusion
Demo
Use Cases
Future work
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 2

3. Favourite example
Scripting
Documents SVN Book sources:
with XQuery
Vyacheslav
http://svnbook.googlecode.com/svn/
Zholudev,
Michael
Kohlhase
TNTBase
Recap
Virtual
Documents
VDoc Specifications
VDoc FS Entity
Querying VDocs
VDoc Editing
Conclusion
Demo
Use Cases
Future work
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 3

4. Our Goal: Support your Workflows
Scripting
Documents
with XQuery
Vyacheslav
Zholudev,
Michael
Kohlhase
Provide a basis for XML-based (e.g. DocBook-based)
TNTBase management systems
Recap
Virtual XML-fragment access techniques
Documents
VDoc Specifications Make possible workflows and globally distributed
VDoc FS Entity
Querying VDocs project teams as we know them from Open Source
VDoc Editing
Conclusion community
Demo
Use Cases
Future work
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 4

5. What is TNTBase?
Scripting
Documents
with XQuery
Vyacheslav
Zholudev,
Michael
Kohlhase
TNTBase Client-server architecture
Recap
Virtual
TNTBase Core = Subversion + Berkeley DB XML
Documents = xSVN
VDoc Specifications
VDoc FS Entity
Querying VDocs
Web application based on xSVN
VDoc Editing
Conclusion
Demo
Use Cases
Future work
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 5

6. What do we inherit from SVN?
Scripting
Documents
with XQuery
Vyacheslav
Zholudev,
Michael Communication
Kohlhase
Any SVN client (Command-line, SVNKit Java Library,
TNTBase Tortoise SVN)
Recap
Virtual
Sophisticated versioning
Documents History support
VDoc Specifications
VDoc FS Entity Collaborative editing
Querying VDocs
VDoc Editing Branching, tagging
Conclusion
Storing non-XML files
Demo
Use Cases Facilitates migrating from SVN-based systems/applications
Future work
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 6

7. What do we get from DB XML?
Scripting
Documents
with XQuery
Vyacheslav
Zholudev,
Michael Communication
Kohlhase
Via RESTful interface build on top of DB XML storage
TNTBase
Recap
Sophisticated XML database
Virtual XQuery/XQuery Update
Documents Indexing
VDoc Specifications
VDoc FS Entity XQuery external functions
Querying VDocs
VDoc Editing ACID Transactions
Conclusion
Data integrity
Demo
Use Cases Perfect basis for managing huge collections of XML data
Future work
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 7

8. What is on top of xSVN?
Scripting
Documents Communication: RESTful interface
with XQuery
TNTBase XQuery functions
Vyacheslav
Zholudev, addressing path-based collections with revision axis
Michael
Kohlhase repository information
... more
TNTBase
Recap XML-diff
Virtual
Documents
Your own validation and presentational workflows
VDoc Specifications
VDoc FS Entity
managed by SVN properties
Querying VDocs
VDoc Editing
Plugin architecture for validation, presentation, interface
Conclusion
extraction and XML-diff
Demo
Number of built-in plugins, e.g.
Use Cases
Extracting RDF from XML files
Future work
Content MathML → Presentation MathML
Summary
Number of XML differs
and ...
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 8

9. Scripting
Documents
with XQuery
Vyacheslav
Zholudev,
Michael
Kohlhase
TNTBase
Recap
Virtual
Documents
VDoc Specifications
Virtual Documents
VDoc FS Entity
Querying VDocs
VDoc Editing
Conclusion
Demo
Use Cases
Future work
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 9

10. Motivation
Scripting
Documents
with XQuery
Vyacheslav
Zholudev,
Michael
Kohlhase
TNTBase
Recap No in-document macros in XML
Virtual No analogues of views in XML databases
Documents
VDoc Specifications
VDoc FS Entity
“Views” should be first-class citizens in XML databases
Querying VDocs
VDoc Editing
Conclusion
Demo
Use Cases
Future work
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 10

11. Superficial Glance
Scripting
Documents
with XQuery
Vyacheslav Virtual Documents (VDocs) are first class citizens in
Zholudev,
Michael TNTBase
Kohlhase
“XML-database” views that are:
TNTBase
Recap Queryable
Virtual Parametrizable
Documents
VDoc Specifications
Editable
VDoc FS Entity Materializable
Querying VDocs
VDoc Editing Comprise our repository’s document format ⇒ can
Conclusion
reuse our workflows
Demo
Use Cases
OR
Future work Like JSP is for HTML/Java, VDocs are for XML/XQuery
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 11

12. Running example
Scripting
Documents
with XQuery
Vyacheslav
Zholudev,
Michael
Kohlhase Have a collection of mathematical documents in
OMDoc
TNTBase
Recap
Want to get a “view” (VDoc) on all exercises together
Virtual
Documents with authors → two simple XQueries
VDoc Specifications
VDoc FS Entity VDoc should comprise the same XML format ⇒ Need
Querying VDocs
VDoc Editing XML wrappers
Conclusion
Demo Add new documents → VDoc is automatically updated
Use Cases
Future work
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 12

13. The result should be like ...
Scripting
Documents
with XQuery <omdoc xmlns:dc="http://purl.org/dc/elements/1.1/">
Vyacheslav < dc:title >Exercises for Computer Science lectures</dc:title>
Zholudev, <dc:creator>Vyacheslav Zholudev</dc:creator>
Michael <section>
Kohlhase
< dc:title >Acknowledgements</dc:title>
TNTBase
<omtext>
Recap The following individuals have contributed material to this document:
<author>Vyacheslav Zholudev</author>
Virtual
Documents <author>Michael Kohlhase</author>
VDoc Specifications ...
VDoc FS Entity
Querying VDocs
</omtext>
VDoc Editing </section>
Conclusion <section>
Demo < dc:title >Exercises</ dc:title >
Use Cases
<exercise title="Graph properties">...</exercise>
...
Future work </section>
Summary </omdoc>
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 13

14. VDocs Workflow
Scripting
Documents
with XQuery
Vyacheslav
Zholudev,
Michael
Kohlhase
TNTBase
Recap
Virtual
Documents
VDoc Specifications
VDoc FS Entity
Querying VDocs
VDoc Editing
Conclusion
Demo
Use Cases
Future work
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 14

15. VDoc Specifications
Scripting
Documents <tnt:virtualdocument xmlns:tnt=" http: // tntbase.mathweb.org/ns">
with XQuery <tnt:skeleton xml:id="exercises">
Vyacheslav
<omdoc xmlns:dc="http://purl.org/dc/elements/1.1/">
Zholudev, < dc:title >Exercises for Computer Science lectures</dc:title>
Michael <dc:creator>Vyacheslav Zholudev</dc:creator>
Kohlhase
... <!−− XQueries go in between −−>
</omdoc>
TNTBase
Recap </ tnt:skeleton >
Virtual
Documents <tnt:query name="exercises.xq"> for $t in $topics return
VDoc Specifications tnt:collection (concat(’ /exercises/ ’ , $t , ’ /∗. omdoc’))//
VDoc FS Entity
exercise[ position () le $max]
Querying VDocs
VDoc Editing </tnt:query>
Conclusion
Demo <tnt:params>
<tnt:param name="max">
Use Cases
<tnt:value>10</tnt:value>
Future work </tnt:param>
Summary ....
</tnt:params>
</tnt:virtualdocument>
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 15

16. VDoc Specifications
Scripting
Documents
with XQuery
Vyacheslav
Zholudev,
Michael
Kohlhase
Skeleton
TNTBase
Recap
XQueries (optional)
Virtual Parameters (optional)
Documents
VDoc Specifications
VDoc FS Entity There is a schema for VDoc specifications
Querying VDocs
VDoc Editing https://svn.tntbase.org/repos/tntbase/trunk/
Conclusion
DbXmlAccessLib/resources/tnt-vd-spec.rnc
Demo
Use Cases
Future work
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 16

17. Skeletons
Scripting
Documents
with XQuery
<tnt:skeleton xml:id="exercises">
Vyacheslav <omdoc xmlns:dc="http://purl.org/dc/elements/1.1/">
Zholudev,
Michael
< dc:title >Exercises for Computer Science lectures</dc:title>
Kohlhase <dc:creator>Vyacheslav Zholudev</dc:creator>
<section>
TNTBase < dc:title >Acknowledgements</dc:title>
Recap <omtext>
Virtual <tnt:xqinclude query="tnt:collection(’/exercises//*.omdoc’)//dc:creator/*">
Documents < tnt:return >
VDoc Specifications
VDoc FS Entity
<author><tnt:result /><author>
Querying VDocs </ tnt:return >
VDoc Editing
</ tnt:xqinclude >
Conclusion
</omtext>
Demo </section>
Use Cases ...
Future work </omdoc>
</ tnt:skeleton >
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 17

18. Skeletons
Scripting
Documents
with XQuery
Vyacheslav <tnt:skeleton xml:id="exercises">
Zholudev,
Michael <omdoc xmlns:dc="http://purl.org/dc/elements/1.1/">
Kohlhase < dc:title >Exercises for Computer Science lectures</dc:title>
<dc:creator>Vyacheslav Zholudev</dc:creator>
TNTBase ...
Recap
<section>
Virtual < dc:title >Exercises</ dc:title >
Documents
VDoc Specifications
<tnt:xqinclude>
VDoc FS Entity <tnt:query name="exercises.xq"/>
Querying VDocs
< tnt:return ><tnt:result/></ tnt:return >
VDoc Editing
Conclusion
</ tnt:xqinclude >
</section>
Demo
</omdoc>
Use Cases </ tnt:skeleton >
Future work
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 18

19. Queries
Scripting
Documents
with XQuery
Vyacheslav
Zholudev, <tnt:query name="exercises.xq">
Michael for $t in $topics return
Kohlhase tnt:collection (concat(’ /exercises/ ’ , $t , ’ /∗. omdoc’))//
exercise[ position () le $max]
TNTBase
Recap
</tnt:query>
Virtual
Documents Queries can be defined as
VDoc Specifications
VDoc FS Entity
Querying VDocs
an attribute in a skeleton
VDoc Editing
Conclusion
a child element of tnt:xqinclude
Demo a reference to a query element
Use Cases
a reference to a repository file
Future work
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 19

20. Parameters
Scripting
Documents
with XQuery
Vyacheslav
Zholudev, tnt:collection(concat(’/exercises/’, $t,
Michael
Kohlhase ’/*.omdoc’))//exercise[position() le $max]
TNTBase <tnt:params>
Recap <tnt:param name="max">
Virtual <tnt:value>10</tnt:value>
Documents </tnt:param>
VDoc Specifications
VDoc FS Entity
<tnt:param name="topics">
Querying VDocs <tnt:value>search</tnt:value>
VDoc Editing
<tnt:value>graphs</tnt:value>
Conclusion
</tnt:param>
Demo </tnt:params>
Use Cases
Future work
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 20

21. Other Features of VDoc Specs
Scripting
Documents
with XQuery
Vyacheslav
Zholudev,
Michael
Kohlhase
VDoc Specs can inherit skeletons from other VDoc
TNTBase
Recap Specs
Virtual
Documents
XQueries can reside in a separate repository file
VDoc Specifications
VDoc FS Entity
You can override XQueries and parameters
Querying VDocs
VDoc Editing VDoc Specs can be “abstract”
Conclusion
Demo
Use Cases
Future work
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 21

22. VDoc File System entity
Scripting
Documents
with XQuery
Vyacheslav
Zholudev,
Michael
Kohlhase
TNTBase
Recap
Virtual
Documents
VDoc Specifications
VDoc FS Entity
Querying VDocs
VDoc Editing
Conclusion
Demo
Use Cases
Future work
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 22

23. VDoc File System entity and Materialization
Scripting
Documents Q: How do I create a VDoc?
with XQuery
A: Associate a file system entity with a specification
Vyacheslav
Zholudev,
Michael
Kohlhase Q: Can I influence VDoc Spec parameters?
A: Yes!
TNTBase
Recap
Virtual
Documents
Q: Are VDocs versioned?
VDoc Specifications
VDoc FS Entity
A: No, but VDoc Specs are!
Querying VDocs
VDoc Editing
Conclusion
Q: How do I get the content?
Demo
A: Use XQuery or RESTful interface
Use Cases
tnt:vdoc(’/path/to/exercises.vdoc’)
Future work
Summary
Q: What to do if I want to version the content?
A: Materialize them!
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 23

24. Querying VDocs
Scripting
Documents
with XQuery
Vyacheslav
Zholudev,
Michael
Kohlhase
TNTBase
It’s easy:
Recap
tnt:vdoc($path as xs:string) as
Virtual
Documents document-node()
VDoc Specifications
VDoc FS Entity
Querying VDocs
Can use VDocs in XQueries of other VDocs
VDoc Editing
Conclusion
Demo
Use Cases
Future work
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 24

25. VDoc editing
Scripting
Documents
with XQuery
Vyacheslav
Zholudev,
Michael
Kohlhase
TNTBase
Recap
Virtual
Documents
VDoc Specifications
VDoc FS Entity
Querying VDocs
VDoc Editing
Conclusion
Demo
Use Cases
Future work
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 25

26. VDoc Editing
Scripting
Documents
with XQuery
Vyacheslav
Zholudev,
Michael
Kohlhase
VDocs can be edited and “committed” back
TNTBase
Recap Changes are propagated back to the source documents
Virtual
Documents
History is preserved
VDoc Specifications
VDoc FS Entity Abstraction from a notion of a file
Querying VDocs
VDoc Editing
Conclusion
“Committing” via RESTful interface
Demo
Use Cases
Future work
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 26

27. Prerequisites for Editing
Scripting
Documents
with XQuery
Vyacheslav
Zholudev,
Michael
Kohlhase
Only XQuery result nodes can be edited
TNTBase
Recap Only XML elements that came from physical files are
Virtual
Documents
editable
VDoc Specifications
VDoc FS Entity
The XML element can be present only once in a VDoc
Querying VDocs
VDoc Editing Only HEAD revision results are editable
Conclusion
Demo
Use Cases
Future work
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 27

28. Why not a single XQuery?
Scripting
Documents
with XQuery
Vyacheslav
Zholudev,
Michael
Kohlhase
Think of JSP
Modularization
TNTBase
Recap
Distribution of responsibilities
Virtual
Documents Inheriting of VDoc Specs
VDoc Specifications
VDoc FS Entity
Querying VDocs
Overriding XQueries/parameters
VDoc Editing
Conclusion One VDoc Spec for multiple VDocs that differ in
Demo parameters
Use Cases
Future work
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 28

29. Scripting
Documents
with XQuery
Vyacheslav
Zholudev,
Michael
Kohlhase
TNTBase
Recap
Virtual
There is nothing as practical
Documents
VDoc Specifications
VDoc FS Entity
as a good demo
Querying VDocs
VDoc Editing
Conclusion
Demo
Use Cases
Future work
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 29

30. Scripting
Documents
with XQuery
Vyacheslav
Zholudev,
Michael
Kohlhase
TNTBase
Recap
Virtual
Documents
VDoc Specifications
Real-world Use Cases
VDoc FS Entity
Querying VDocs
VDoc Editing
Conclusion
Demo
Use Cases
Future work
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 30

31. Exam generation
Scripting
Documents
with XQuery
Vyacheslav
Zholudev,
Michael
Kohlhase Corpus of more than 1000 exercises for students in the
OMDoc format
TNTBase
Recap
Automated exam generation by means of VDocs
Virtual
Documents Different parameters for different type of exams
VDoc Specifications
VDoc FS Entity
Querying VDocs
Editing facilities allow to easily fix mistakes in
VDoc Editing
Conclusion
underlying documents
Demo
Write a VDoc Spec once and reuse it over the years!
Use Cases
Future work
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 31

32. Multiple Versions of Documents
Scripting
Documents
with XQuery Example:
Vyacheslav
Zholudev, XML 1.0/1.1
Michael
Kohlhase XQuery 1.0/1.1
TNTBase MathML1.0/1.0.1/2.0/2.0(2e)/3.0
Recap
Virtual
...
Documents
VDoc Specifications Experimenting with XML specifications
VDoc FS Entity
Querying VDocs in XMLSpec format
VDoc Editing
Conclusion
What formal parts have been changed in XML 1.1?
Demo
Use Cases
Provide a Diff VDoc that summarizes formal changes
Future work Fixing bugs without navigating to the source documents
Summary
Get the up-to-date content via VDoc “file” in a repository
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 32

33. Ontology refactoring
Scripting
Documents
with XQuery
Vyacheslav Preview refactored ontologies before saving them!
Zholudev,
Michael renaming entities
Kohlhase
factoring out or merging modules
TNTBase
Recap rewriting axioms
Virtual
Documents lowering expressivity
VDoc Specifications
VDoc FS Entity
Querying VDocs
stripping axiom annotations
VDoc Editing
Conclusion
Single VDoc Spec is applicable for different ontologies for
Demo the same type of a refactoring!
Use Cases Demo video:
Future work http://tntbase.org/wiki/screencasts
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 33

34. Future work
Scripting
Documents
with XQuery
Vyacheslav
Zholudev,
Michael
Kohlhase
TNTBase
Editing of a VDoc presentation (like XHTML)
Recap
More checks for validity of a VDoc Spec
Virtual
Documents
VDoc Specifications
Make XQuery errors more informative
VDoc FS Entity
Querying VDocs Web interface for managing VDocs
VDoc Editing
Conclusion
Demo
Use Cases
Future work
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 34

35. Summary
Scripting
Documents
with XQuery
Vyacheslav
Zholudev,
Michael
Kohlhase 1 A
Analogue of JSP/PHP or TEX/LTEX in the XML world
TNTBase
Recap
2 Enabling technology
Virtual
3 VDocs are not tied to TNTBase
Documents
VDoc Specifications
VDoc FS Entity
4 Write once - use for different needs
Querying VDocs
VDoc Editing
5 Many applications like generation of news or guided
Conclusion
tours
Demo
Use Cases
Future work
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 35

36. Thanks for your attention
Scripting
Documents
with XQuery
Vyacheslav
Zholudev,
Michael
Kohlhase
Questions???
TNTBase
Recap
Virtual
Documents
VDoc Specifications
VDoc FS Entity
Querying VDocs
VDoc Editing
Conclusion
Demo
Project page: http://tntbase.org
Use Cases
E-mail: v.zholudev@jacobs-university.de
Future work
Summary
September 16, 2010
Vyacheslav Zholudev, Michael Kohlhase Scripting Documents with XQuery 36