Lotus' DXL development team is working to bring DXL to full fidelity and usefulness. Come see our progress to date, based on priorities established in consultation with our partners. You'll learn how we've automated our testing process to guarantee that we find even tiny discrepancies when doing a DXL "round-trip." You'll see how you can use DXL to develop tools to enhance the productivity of your developers, QE, administrators and end users. We'll demo high-value applications that take advantage of DXL's improved capabilities, such as automating internationalization of design elements, scanning for design errors, making action bars colors and fonts consistent, and automating generation of rich text.
2. Financial Disclaimer
The information on the new product is intended to outline our general product direction and it should
not be relied on in making a purchasing decision. The information on the new product is for
informational purposes only and may not be incorporated into any contract. The information on the
new product is not a commitment, promise, or legal obligation to deliver any material, code or
functionality. The development, release, and timing of any features or functionality described for our
products remains at our sole discretion
2
3. Agenda
● Progress in 2009 and future plans
● New automated testing tool for DXL round-trip fidelity
● Demos
3
4. Highlights of Progress in 2009
● Overall mission: Round-trip fidelity of design elements
● High-priority issues were addressed in 8.51
● Set priorities and made plans for remaining work
● Improved richtext fidelity
● Designed a spiffy new testing tool
● Began the drive towards full DXL fidelity
DXL
4
5. Progress in 8.51
● Agent improvements, including full support for search expressions
● Testing of low-level <note> format – verified as loss-less *
▬ Low-level format -- ForceNoteFormat=True and RichTextOption=RICHTEXTOPTION_RAW
▬ * One fix in 8.52 for $Formula item on views and folders
● 34 SPRs plus additional issues fixed
● Fixed reproducible hangs and crashes
● Fixed 19 different errors (Exporter/Importer logs) from round-tripping ND
templates
● Crowded Wisdom survey completed
5
6. Crowded Wisdom Survey
Priority Issue Weight
1 Don't crash 179
2 Full implementation for view/folder design elements 148
3 Full implementation of Agents 134
4 Full implementation of Script Libraries 129
5 Full implementation of Forms/Pages/Subforms 128
6 Rich text: File attachment issues 121
7 Make XPages and other text-based file elements optionally 102
include text rather than Base64
8 Make all non-text based file elements give original binary file 100
contents, rather than binary with CD record headers in it
9 Full implementation of Web Service Consumers 93
10 Other rich text issues besides layout regions and attachments 90
11 Full implementation of Web Service Providers 86
12 Full implementation of Outlines 79
13 Full implementation of Images 71
14 Full implementation of Shared Fields 69
... 35 DXL exporter output must be valid to DTD and Schema 1
6
7. Progress for Future Releases
● Specs for XPages and other file-based design elements
● DXL Fidelity Verification Tool (DFVT)
● <compositedata> - eliminates dropped richtext CD records
● Fidelity retention processes
● Nearing full round-trip fidelity of views and folders, and all they contain
● 82 SPRs fixed (mostly for view/folder fidelity)
The information on the new product is intended to
outline our general product direction and it should
not be relied on in making a purchasing decision.
The information on the new product is for
informational purposes only and may not be
incorporated into any contract. The information
on the new product is not a commitment, promise,
or legal obligation to deliver any material, code or
functionality. The development, release, and
timing of any features or functionality described
for our products remains at our sole discretion.
7
8. XPages Specification - Highlights
● Applies to File Resources, Stylesheets, Image Resources, Themes,
XPages, Custom Controls, Wiring Properties, Composite Applications,
Components – all file-based design elements
● Current support (File, Stylesheet, Image) always emits Base64
● New Exporter property FileFormat (RAW, ADAPTIVE, TEXT)
▬ ADAPTIVE (default) = raw if might not contain text, else text
● Text formatting issues
▬ Character encoding issues – identifying character set, translations
▬ Byte order marks
▬ Line terminators
● Text data is in CDATA section(s), so we can retain it verbatim
(especially important for XML files)
▬ If data itself contains CDATA termination sequence ]]> then data is split into multiple CDATA
sections
8
9. DXL Fidelity Verification Tool (DFVT)
● New comprehensive testing tool
● It's an IBM Lotus® Domino® app!
● 10,000+ lines of LotusScript® code
● Allows us to verify full fidelity
● More in just a few moments
9
10. DXL <compositedata> Element
● Problem: Exporter doesn't recognize some richtext CD records, so their
data is not represented in DXL, and is lost in a round-trip
● Ultimate solution: All CD records/fields/bits are represented in DXL
● Upcoming solution: <compositedata> - save/restore CD record bits
▬ Holds raw canonical data for the CD record, encoded in Base64
▬ Has contextual attributes that allow Importer to place record correctly in CD stream – proximity to
BEGIN and HOTSPOTBEGIN records, previous and next record signatures
▬ Allows DXL to round-trip data that previously was lost, e.g. Layout regions
▬ Not a complete solution – no help for records that are already handled, but only partially
richtext
improved
10
11. Fidelity Retention Processes
● Must actively update DXL to remain at full fidelity as ND changes
● We've implemented some internal processes to help us determine when
DXL changes are needed
● We regularly round-trip all ND templates to detect regressions
immediately
● We are considering additional ways to improve on this
11
12. View and Folder Full Fidelity
● We are close to having DXL export and import of views and folders with
no data loss
● Q: Is this bit-for-bit binary compatibility? A: No – but don't fret...
▬ It is binary compatibility whenever we can
▬ We are documenting and will share all “acceptable differences” allowed by DFVT
▬ The goal is functional equivalence!
● We've worked through lots of issues
▬ View and column properties are very complex
▬ Legacy design elements may differ from what current Designer produces
▬ Lots of common constructs needed work as well
Upcoming
views folders Release
12
13. Dependencies to Achieve View/Folder
Fidelity
● References to Shared Columns
● Action Bars, Actions
● References to Shared Actions
● Named element references (images, etc.)
● Formulas
● LotusScript code
● JavaScript code
● Borders and Backgrounds
● Colors
● Fonts
● and more!
13
14. Future Plans – 2010 and Beyond
● Design element fidelity, continued
▬ Agents, Script Libraries, Shared Columns, Shared Actions should be relatively easy
▬ Forms, Pages, Subforms
▬ will benefit from common constructs already done
▬ richtext will be a large effort – may defer parts of it until rest of design element work is done
▬ XPages, and all other file-based design elements
▬ Straightforward, since they are just a file and some metadata
● Designer source control integration
● Other possibilities
▬ e.g. merging template customizations into an updated template
The information on the new product is intended to
outline our general product direction and it should
not be relied on in making a purchasing decision.
The information on the new product is for
informational purposes only and may not be
agents forms XPages incorporated into any contract. The information
... on the new product is not a commitment, promise,
or legal obligation to deliver any material, code or
functionality. The development, release, and
timing of any features or functionality described
for our products remains at our sole discretion.
14
15. DFVT Mission
● Provide automated DXL fidelity testing of ND apps at a binary level
● Easy to use
● Highly configurable
● Transparent
● Future possibility: Share with Design Partners
15
16. DFVT Summary
● Does DXL high-level round-trip to get “before” and “after” design
elements
● Exports “before” and “after” design elements using low-level DXL
● Parses low-level DXL and generates detailed dump (kind of like
Notespeek)
● Factors out acceptable differences (there are lots!)
▬ applies configurable rules
▬ Success = no differences
● Guiding principle: Current Designer behavior is the “Gold”
16
17. DFVT Block Diagram
High- Round-
Input DXL Export level DXL Import
Db DXL
trip Db
Raw Raw
“Before” “After”
DXL Export DXL DXL DXL Export
Parse and Parse and
Apply Rules Apply Rules
Diff Tool
“Before” “After”
Output Output
Diffs
Goal is no diffs!
17
19. DFVT Acceptable Differences
● Lots of legitimate reasons for diffs, for example
▬ Timestamps
▬ “Don't care” fill bytes to align records on 2-byte boundaries
▬ Note references within the database – UNIDs differ
▬ Font tables (same fonts, different order)
▬ Formulas (local variable timestamps, stale column names)
▬ Sometimes: data that is grayed out or hidden in the Designer UI
▬ New Designer options added after design element originally saved, using previously unused fields
(e.g. original contains 0, round-trip contains new default value 4)
● DFVT “factors out” acceptable diffs, leaves remaining diffs in output
This was This was
created on created on
01/20/2010 xx/xx/xxxx
19
21. Demo 1 – Form Letters
● Issue: Need to generate form letters with rich content
● Solution
▬ Template document contains the fixed content with placeholders for variable content
▬ A set of documents contain the variable content
▬ Use DXL and DOM to do a mail merge
Dear ____,
21
22. Template document
Demo 1 Block Diagram Dear <name>,
We hear you are crazy.
Appt Record DXL Please come to our <clinic>
Name: Rex Stinson Exporter clinic to have your head
Clinic: Maple Road examined on <apptdate> at
Date: 30 Jan 2010 <appttime>.
Time: 4:00 PM Signed,
DOM Parser <signature>
Dear Rex Stinson,
MailMerge We hear you are crazy. Please
process come to our Maple Road clinic to
Name: Wallace Bean have your head examined on 30
Signature: Jan 2010 at 4:00 PM.
DOM Parser Signed,
Wallace Bean, Director
DXL Wallace Bean, Director
Exporter DXL
Importer
22
24. Demo 2 – Scanning for Design Errors
● Issue: Template databases that don't fully adhere to conventions
▬ Hard to manually find all the places that need addressing
● Solution:
▬ Export to DXL
▬ Scan the DXL for any problems
▬ Generate report
.ntf Generate Report
24
25. Demo 2 Block Diagram
Scan for missing field help:
Server: Casbah
Filepath: dentabc.nsf
abc.nsf
Go
DXL
Exporter
SAX Parser Missing field help:
Form A:
field Age
field Zip
25
27. Demo 3 – Template Internationalization
● Issue: Template databases need to be localized to other languages
▬ Time consuming and error-prone to find all occurrences of strings needing translation
▬ Changes need to be made in Designer, but translator might not be familiar with it
● Solution
▬ Use DXL and XSLT style sheet to pull out all strings needing translation to a simple file
▬ Translator makes changes to the simple file
▬ Use DXL and XML processors to make changes to the corresponding design elements
English Español
27
28. XSLT
Style
Demo 3 Block Diagram Sheet
XML
English DXL Export XSLT for
translation
Db views
Translated
DXL XML
DOM Tree DOM Tree
DOM Parse DOM Parse
Serialize
Spanish
Translated DXL Import Db
DXL
28
30. Demo 4 – Action Bar UI Consistency
● Issue: Different colors and fonts used in application's action bars,
therefore application lacks a consistent “look and feel”
● Solution
▬ Identify a target design element to use as a model
▬ Export design elements of that type to DXL
▬ Process the DXL to generate a ToDo list of changes needed to enforce consistency
▬ Automatically apply the ToDo list
30
31. Demo 4 Block Diagram
Action Bar Scan:
Server: Casbah NotesNoteCollection 2
Filepath: dentabc.nsf DXL
Model design element: Exporter
3
View
1 DOM Parser
By Title 4
By Date Action Bar ToDo List:
My Docs Form WtItem
abc.nsf - change background to 133, 240, 240
- display button border always
Go 7
5 Fix automatically
6
DOM Parser
DXL
Importer
31
33. Summary
● Soon, you will be able to use DXL confidently for any processing of
views and folders
● We've shown you the tool that we're using to guarantee DXL fidelity
● We will be driving the remaining design elements to full fidelity
views folders
The information on the new product is intended to
outline our general product direction and it should
not be relied on in making a purchasing decision.
The information on the new product is for
informational purposes only and may not be
incorporated into any contract. The information
agents forms XPages ... on the new product is not a commitment, promise,
or legal obligation to deliver any material, code or
functionality. The development, release, and
timing of any features or functionality described
for our products remains at our sole discretion.
33
34. Q&A
● Also, come see us in the Meet the Developers lab – Dolphin Asia 1 – 2
● Download examples from openntf.org
● http://www-10.lotus.com/ldd/ddwiki.nsf/dx/ls-design-programming.htm
Manipulating Design Elements with LotusScript (or JavaTM)
Andre Guirard Dick Annicchiarico
34