1.
Scaling Xtext
Lieven Lemiengre

2.
Sigasi
● IDE for Hardware Description Languages
○ VHDL, (System)Verilog
● Using Xtext for 4 years
● Large user base
○ (commercial, free, students)

3.
Our company goal
● Assist hardware designer
● High quality interactive front-end compiler
○ Instant error reporting
■ parsing, semantic, linting, style checking
○ Visualisations, design exploration, ...
○ Integrate with ecosystem
■ other compilers, simulators, synthesizers

4.
The challenge
● Large projects
○ > 250 KLOC is not uncommon
○ design + external libraries
○ big files
■ some libraries are distributed as 1 file
■ generated hardware cores
●

5.
Adopting Xtext
● Started with the early Xtext 2.0 snapshots
● Initial performance analysis
○ Clean build performance of a big project (330k LOC)
■ > 20 minutes
■ > 2 GB
○ Editing big files (> 1 MB)
■ unusable

6.
Adopting Xtext
● Started with the early Xtext 2.0 snapshots
● Initial performance analysis
○ Clean build performance of a big project (330k LOC)
■ > 20 minutes → < 1 min
■ > 2 GB → < 1 GB memory
○ Editing big files (> 1 MB)
■ unusable → usable with reduced editor

7.
● Xtext framework improvements
● Measure → analyze → improve or cheat
○ faster build
○ reduce memory usage
○ UI responsiveness
Improving performance

8.
Overview
● Analysing build performance
○ Analyze the build
■ Macro build measurements
■ Key performance points
○ Reduce workload
○ Parallelize the build
● Analyzing UI issues
○ Monitoring the UI thread
○ Saveguards

9.
Analyzing builds: builder overview
Global
indexing
Linking
Validation
Custom
Validation
Global
index
Eclipse
resources
warnings
errors
resource
descriptions
Builder
Participants
resource
changes
?

10.
Analyzing builds: metrics
● For each build
○ # of files being build
○ timing: Global index, Linking, Validation, Individual
builder participants
● Instrument by overriding
ClusteringBuilderState & XtextBuilder
● Example: Building 134 resources, timing: {
global index=1806,
linking=378,
validation=823,
totalLinkingAndValidation=1364
}

11.
Analyzing builds: resource loads
● Observation:
○ Most time spent in resource loads
○ Certain files are loaded multiple times?!
Global
indexing
Linking Validation
Builder
Participants
resources
LOAD
POTENTIAL
RELOAD
POTENTIAL
RELOAD

12.
Analyzing builds: Memory footprint
Global
index
Resource
Set
● Global index
○ always loaded →limit size
○ use hashing: name + hash of contents
● Reduce EMF size
derived model -> Xcore model
http://www.sigasi.com/content/view-complexity-your-xtext-ecore-model

13.
Optimize loading
● What is resource load?
○ Parse
○ build EMF model
○ build Node model
● Parallelise
○ parse multiple files simultaneously
○ ~3 time faster loads on 4 core machine
○ EMF synchronization issues
● Cache
○ serialize EMF and Node model in a cache
○ 3-4 time faster loads

14.
Linking
Global
indexing
Linking Validation
Builder
Participants
● Language specific
○ VHDL vs Verilog
● Avoiding linking
○ expressions that we couldn’t handle (yet)
○ library files, only when used in user-code
● Many iterations
○ lazy linking vs eager linking
○ From 40% of build time to 20%

15.
Linking
Pseudo-algorithm
● In your IScopePovider
○ Find a eContainer that can be linked eagerly
○ Link everything
■ unresolved links become UnlinkedDeclaration
● In your DefaultLinkingService
○ getLinkedObjects(...)
■ already resolved? → skip it
● Disable notifications

16.
Validation
Global
indexing
Linking Validation
Builder
Participants
● Combine validations to avoid model
traversals
● Local analysis, do global validations moved
into builder participant
● Avoid validation
○ disabled validations
○ libraries: errors & warnings are suppressed anyway
● Monitor

17.
Track performance
● Nightly build
● log build times

18.
UI responsiveness
● Measuring: detect a blocked UI thread
○ initially Svelto https://github.com/dragos/svelto
○ now our own method & logging
○ Eclipse Mars
● Improvements
○ UI is for drawing only!
○ Make sure everything is cancellable
● Safeguards
○ certain services should never be executed on the UI
thread => check & log

19.
Lightweight Editor (fallback)
● Syntax-highlighting + markers
● For files > 1 MB
● Based on ContentTypes extension point

20.
Two ContentTypes (based on file size)
<extension point="org.eclipse.core.contenttype.contentTypes">
<content-type ...
describer="com.sigasi...FullVhdlContentDescriber"
name="VHDL editor"
<describer class="...FullVhdlContentDescriber" />
</content-type>
<content-type ...
describer="com.sigasi....LightweightVhdlContentDescriber"
name="Lightweight VHDL editor"
<describer class="...LightweightVhdlContentDescriber" />
</content-type>
</extension>

21.
Future work
● Continuous process
● Cache global index info per resource?
● Linking without node model?
● Avoid project locking build (Xtext 2.8?)

22.
Come talk to us about...
● Documentation generation
● Fancy linking algorithms / type systems
● Graphical views
● Cross-language support
● Testing Xtext-plugins
● Lexical macros
● Manage large amount of validations
● ...

23.
+1 0 -1
Sign in: www.eclipsecon.org
Evaluate the sessions