SlideShare ist ein Scribd-Unternehmen logo
1 von 23
Downloaden Sie, um offline zu lesen
Scaling Xtext
Lieven Lemiengre
Sigasi
● IDE for Hardware Description Languages
○ VHDL, (System)Verilog
● Using Xtext for 4 years
● Large user base
○ (commercial, free, students)
Our company goal
● Assist hardware designer
● High quality interactive front-end compiler
○ Instant error reporting
■ parsing, semantic, linting, style checking
○ Visualisations, design exploration, ...
○ Integrate with ecosystem
■ other compilers, simulators, synthesizers
The challenge
● Large projects
○ > 250 KLOC is not uncommon
○ design + external libraries
○ big files
■ some libraries are distributed as 1 file
■ generated hardware cores
●
Adopting Xtext
● Started with the early Xtext 2.0 snapshots
● Initial performance analysis
○ Clean build performance of a big project (330k LOC)
■ > 20 minutes
■ > 2 GB
○ Editing big files (> 1 MB)
■ unusable
Adopting Xtext
● Started with the early Xtext 2.0 snapshots
● Initial performance analysis
○ Clean build performance of a big project (330k LOC)
■ > 20 minutes → < 1 min
■ > 2 GB → < 1 GB memory
○ Editing big files (> 1 MB)
■ unusable → usable with reduced editor
● Xtext framework improvements
● Measure → analyze → improve or cheat
○ faster build
○ reduce memory usage
○ UI responsiveness
Improving performance
Overview
● Analysing build performance
○ Analyze the build
■ Macro build measurements
■ Key performance points
○ Reduce workload
○ Parallelize the build
● Analyzing UI issues
○ Monitoring the UI thread
○ Saveguards
Analyzing builds: builder overview
Global
indexing
Linking
Validation
Custom
Validation
Global
index
Eclipse
resources
warnings
errors
resource
descriptions
Builder
Participants
resource
changes
?
Analyzing builds: metrics
● For each build
○ # of files being build
○ timing: Global index, Linking, Validation, Individual
builder participants
● Instrument by overriding
ClusteringBuilderState & XtextBuilder
● Example: Building 134 resources, timing: {
global index=1806,
linking=378,
validation=823,
totalLinkingAndValidation=1364
}
Analyzing builds: resource loads
● Observation:
○ Most time spent in resource loads
○ Certain files are loaded multiple times?!
Global
indexing
Linking Validation
Builder
Participants
resources
LOAD
POTENTIAL
RELOAD
POTENTIAL
RELOAD
Analyzing builds: Memory footprint
Global
index
Resource
Set
● Global index
○ always loaded →limit size
○ use hashing: name + hash of contents
● Reduce EMF size
derived model -> Xcore model
http://www.sigasi.com/content/view-complexity-your-xtext-ecore-model
Optimize loading
● What is resource load?
○ Parse
○ build EMF model
○ build Node model
● Parallelise
○ parse multiple files simultaneously
○ ~3 time faster loads on 4 core machine
○ EMF synchronization issues
● Cache
○ serialize EMF and Node model in a cache
○ 3-4 time faster loads
Linking
Global
indexing
Linking Validation
Builder
Participants
● Language specific
○ VHDL vs Verilog
● Avoiding linking
○ expressions that we couldn’t handle (yet)
○ library files, only when used in user-code
● Many iterations
○ lazy linking vs eager linking
○ From 40% of build time to 20%
Linking
Pseudo-algorithm
● In your IScopePovider
○ Find a eContainer that can be linked eagerly
○ Link everything
■ unresolved links become UnlinkedDeclaration
● In your DefaultLinkingService
○ getLinkedObjects(...)
■ already resolved? → skip it
● Disable notifications
Validation
Global
indexing
Linking Validation
Builder
Participants
● Combine validations to avoid model
traversals
● Local analysis, do global validations moved
into builder participant
● Avoid validation
○ disabled validations
○ libraries: errors & warnings are suppressed anyway
● Monitor
Track performance
● Nightly build
● log build times
UI responsiveness
● Measuring: detect a blocked UI thread
○ initially Svelto https://github.com/dragos/svelto
○ now our own method & logging
○ Eclipse Mars
● Improvements
○ UI is for drawing only!
○ Make sure everything is cancellable
● Safeguards
○ certain services should never be executed on the UI
thread => check & log
Lightweight Editor (fallback)
● Syntax-highlighting + markers
● For files > 1 MB
● Based on ContentTypes extension point
Two ContentTypes (based on file size)
<extension point="org.eclipse.core.contenttype.contentTypes">
<content-type ...
describer="com.sigasi...FullVhdlContentDescriber"
name="VHDL editor"
<describer class="...FullVhdlContentDescriber" />
</content-type>
<content-type ...
describer="com.sigasi....LightweightVhdlContentDescriber"
name="Lightweight VHDL editor"
<describer class="...LightweightVhdlContentDescriber" />
</content-type>
</extension>
Future work
● Continuous process
● Cache global index info per resource?
● Linking without node model?
● Avoid project locking build (Xtext 2.8?)
Come talk to us about...
● Documentation generation
● Fancy linking algorithms / type systems
● Graphical views
● Cross-language support
● Testing Xtext-plugins
● Lexical macros
● Manage large amount of validations
● ...
+1 0 -1
Sign in: www.eclipsecon.org
Evaluate the sessions

Weitere ähnliche Inhalte

Was ist angesagt?

Microservices Architecture and Containers.
Microservices Architecture and Containers.Microservices Architecture and Containers.
Microservices Architecture and Containers.imjacobclark
 
Atmosphere packages and the chuck norris effect
Atmosphere packages and the chuck norris effectAtmosphere packages and the chuck norris effect
Atmosphere packages and the chuck norris effectFabian Kromer
 
Web App Prototypes with Google App Engine
Web App Prototypes with Google App EngineWeb App Prototypes with Google App Engine
Web App Prototypes with Google App EngineVlad Filippov
 
Perl Tools for Productivity
Perl Tools for ProductivityPerl Tools for Productivity
Perl Tools for ProductivityTudor Constantin
 
Azure functions: from a function to a whole application in 60 minutes
Azure functions: from a function to a whole application in 60 minutesAzure functions: from a function to a whole application in 60 minutes
Azure functions: from a function to a whole application in 60 minutesAlessandro Melchiori
 
Continuous Deployment of Front-end JavaScript with StriderCD, Github and Sauc...
Continuous Deployment of Front-end JavaScript with StriderCD, Github and Sauc...Continuous Deployment of Front-end JavaScript with StriderCD, Github and Sauc...
Continuous Deployment of Front-end JavaScript with StriderCD, Github and Sauc...niallo
 
Getting Started with Java
Getting Started with JavaGetting Started with Java
Getting Started with JavaMichael Redlich
 
JS Fest 2019: Comparing Node.js processes and threads for clustering HTTP, TC...
JS Fest 2019: Comparing Node.js processes and threads for clustering HTTP, TC...JS Fest 2019: Comparing Node.js processes and threads for clustering HTTP, TC...
JS Fest 2019: Comparing Node.js processes and threads for clustering HTTP, TC...Vitalii Kukhar
 
JS Fest 2019/Autumn. Виталий Кухар. Сравнение кластеризации HTTP, TCP и UDP н...
JS Fest 2019/Autumn. Виталий Кухар. Сравнение кластеризации HTTP, TCP и UDP н...JS Fest 2019/Autumn. Виталий Кухар. Сравнение кластеризации HTTP, TCP и UDP н...
JS Fest 2019/Autumn. Виталий Кухар. Сравнение кластеризации HTTP, TCP и UDP н...JSFestUA
 
Implementation of the new REST API for Open Source LBS-platform Geo2Tag
Implementation of the new REST API for Open Source LBS-platform Geo2TagImplementation of the new REST API for Open Source LBS-platform Geo2Tag
Implementation of the new REST API for Open Source LBS-platform Geo2TagOSLL
 
The Professional Programmer
The Professional ProgrammerThe Professional Programmer
The Professional ProgrammerDave Cross
 
Building RESTtful services in MEAN
Building RESTtful services in MEANBuilding RESTtful services in MEAN
Building RESTtful services in MEANMadhukara Phatak
 
Eclipse demo camp 2016 Toronto
Eclipse demo camp 2016 TorontoEclipse demo camp 2016 Toronto
Eclipse demo camp 2016 TorontoGorkem Ercan
 
Synapse india reviews sharing chapter 23 – asp.net
Synapse india reviews sharing  chapter 23 – asp.netSynapse india reviews sharing  chapter 23 – asp.net
Synapse india reviews sharing chapter 23 – asp.netSynapseindiaComplaints
 
OSMC 2018 | Stream connector: Easily sending events and/or metrics from the C...
OSMC 2018 | Stream connector: Easily sending events and/or metrics from the C...OSMC 2018 | Stream connector: Easily sending events and/or metrics from the C...
OSMC 2018 | Stream connector: Easily sending events and/or metrics from the C...NETWAYS
 

Was ist angesagt? (19)

Microservices Architecture and Containers.
Microservices Architecture and Containers.Microservices Architecture and Containers.
Microservices Architecture and Containers.
 
Atmosphere packages and the chuck norris effect
Atmosphere packages and the chuck norris effectAtmosphere packages and the chuck norris effect
Atmosphere packages and the chuck norris effect
 
Web App Prototypes with Google App Engine
Web App Prototypes with Google App EngineWeb App Prototypes with Google App Engine
Web App Prototypes with Google App Engine
 
Perl Tools for Productivity
Perl Tools for ProductivityPerl Tools for Productivity
Perl Tools for Productivity
 
Azure functions: from a function to a whole application in 60 minutes
Azure functions: from a function to a whole application in 60 minutesAzure functions: from a function to a whole application in 60 minutes
Azure functions: from a function to a whole application in 60 minutes
 
Hello android
Hello androidHello android
Hello android
 
Continuous Deployment of Front-end JavaScript with StriderCD, Github and Sauc...
Continuous Deployment of Front-end JavaScript with StriderCD, Github and Sauc...Continuous Deployment of Front-end JavaScript with StriderCD, Github and Sauc...
Continuous Deployment of Front-end JavaScript with StriderCD, Github and Sauc...
 
Getting Started with Java
Getting Started with JavaGetting Started with Java
Getting Started with Java
 
JS Fest 2019: Comparing Node.js processes and threads for clustering HTTP, TC...
JS Fest 2019: Comparing Node.js processes and threads for clustering HTTP, TC...JS Fest 2019: Comparing Node.js processes and threads for clustering HTTP, TC...
JS Fest 2019: Comparing Node.js processes and threads for clustering HTTP, TC...
 
JS Fest 2019/Autumn. Виталий Кухар. Сравнение кластеризации HTTP, TCP и UDP н...
JS Fest 2019/Autumn. Виталий Кухар. Сравнение кластеризации HTTP, TCP и UDP н...JS Fest 2019/Autumn. Виталий Кухар. Сравнение кластеризации HTTP, TCP и UDP н...
JS Fest 2019/Autumn. Виталий Кухар. Сравнение кластеризации HTTP, TCP и UDP н...
 
Implementation of the new REST API for Open Source LBS-platform Geo2Tag
Implementation of the new REST API for Open Source LBS-platform Geo2TagImplementation of the new REST API for Open Source LBS-platform Geo2Tag
Implementation of the new REST API for Open Source LBS-platform Geo2Tag
 
The Professional Programmer
The Professional ProgrammerThe Professional Programmer
The Professional Programmer
 
Docker
DockerDocker
Docker
 
Linq To XML Overview
Linq To XML OverviewLinq To XML Overview
Linq To XML Overview
 
Web Assembly
Web AssemblyWeb Assembly
Web Assembly
 
Building RESTtful services in MEAN
Building RESTtful services in MEANBuilding RESTtful services in MEAN
Building RESTtful services in MEAN
 
Eclipse demo camp 2016 Toronto
Eclipse demo camp 2016 TorontoEclipse demo camp 2016 Toronto
Eclipse demo camp 2016 Toronto
 
Synapse india reviews sharing chapter 23 – asp.net
Synapse india reviews sharing  chapter 23 – asp.netSynapse india reviews sharing  chapter 23 – asp.net
Synapse india reviews sharing chapter 23 – asp.net
 
OSMC 2018 | Stream connector: Easily sending events and/or metrics from the C...
OSMC 2018 | Stream connector: Easily sending events and/or metrics from the C...OSMC 2018 | Stream connector: Easily sending events and/or metrics from the C...
OSMC 2018 | Stream connector: Easily sending events and/or metrics from the C...
 

Andere mochten auch

Pengembalian keputusan dalam organisasi
Pengembalian keputusan dalam organisasiPengembalian keputusan dalam organisasi
Pengembalian keputusan dalam organisasiulungfurtuna
 
Ordin 1802 abrogare 3055
Ordin 1802 abrogare 3055Ordin 1802 abrogare 3055
Ordin 1802 abrogare 3055CityConta.ro
 
Faiz Aldalbhi CV English dated 17 Nov 15
Faiz Aldalbhi CV English dated 17 Nov 15Faiz Aldalbhi CV English dated 17 Nov 15
Faiz Aldalbhi CV English dated 17 Nov 15Faiz Aldalbhi
 
Faiz Aldalbhi CV English dated 17 Nov 15
Faiz Aldalbhi CV English dated 17 Nov 15Faiz Aldalbhi CV English dated 17 Nov 15
Faiz Aldalbhi CV English dated 17 Nov 15Faiz Aldalbhi
 
Allied Presentation PD Jan 2015
Allied Presentation  PD Jan 2015Allied Presentation  PD Jan 2015
Allied Presentation PD Jan 2015Paul W Davies
 
презентация звезды 2015
презентация звезды 2015презентация звезды 2015
презентация звезды 2015лена резник
 
Digital signal processing_by_j.s.chitode
Digital signal processing_by_j.s.chitodeDigital signal processing_by_j.s.chitode
Digital signal processing_by_j.s.chitodeDevi Devika
 
Create Awareness Through HSE Posters
Create Awareness Through HSE PostersCreate Awareness Through HSE Posters
Create Awareness Through HSE PostersSachin Sarkhot
 
Happy birthday colorful cake
Happy birthday colorful cakeHappy birthday colorful cake
Happy birthday colorful cakekayum1
 
Effect of planting containers, cutting types and IBA on transplants productio...
Effect of planting containers, cutting types and IBA on transplants productio...Effect of planting containers, cutting types and IBA on transplants productio...
Effect of planting containers, cutting types and IBA on transplants productio...Ahmedabd Eleslamboly Eleslamboly
 
Session 10 – advanced goal &amp; instanter
Session 10 – advanced goal &amp; instanterSession 10 – advanced goal &amp; instanter
Session 10 – advanced goal &amp; instanterTrí Bằng
 
Session 00 – Course overview Maya Dynamics
Session 00 – Course overview Maya DynamicsSession 00 – Course overview Maya Dynamics
Session 00 – Course overview Maya DynamicsTrí Bằng
 

Andere mochten auch (20)

Vipra nov 15
Vipra nov 15Vipra nov 15
Vipra nov 15
 
Pengembalian keputusan dalam organisasi
Pengembalian keputusan dalam organisasiPengembalian keputusan dalam organisasi
Pengembalian keputusan dalam organisasi
 
Places to visit in andaman
Places to visit in andamanPlaces to visit in andaman
Places to visit in andaman
 
Research Paper
Research PaperResearch Paper
Research Paper
 
Ordin 1802 abrogare 3055
Ordin 1802 abrogare 3055Ordin 1802 abrogare 3055
Ordin 1802 abrogare 3055
 
Faiz Aldalbhi CV English dated 17 Nov 15
Faiz Aldalbhi CV English dated 17 Nov 15Faiz Aldalbhi CV English dated 17 Nov 15
Faiz Aldalbhi CV English dated 17 Nov 15
 
How To Celebrate Labor Day
How To Celebrate Labor DayHow To Celebrate Labor Day
How To Celebrate Labor Day
 
Faiz Aldalbhi CV English dated 17 Nov 15
Faiz Aldalbhi CV English dated 17 Nov 15Faiz Aldalbhi CV English dated 17 Nov 15
Faiz Aldalbhi CV English dated 17 Nov 15
 
Allied Presentation PD Jan 2015
Allied Presentation  PD Jan 2015Allied Presentation  PD Jan 2015
Allied Presentation PD Jan 2015
 
презентация звезды 2015
презентация звезды 2015презентация звезды 2015
презентация звезды 2015
 
Digital signal processing_by_j.s.chitode
Digital signal processing_by_j.s.chitodeDigital signal processing_by_j.s.chitode
Digital signal processing_by_j.s.chitode
 
створення візитки
створення візиткистворення візитки
створення візитки
 
Create Awareness Through HSE Posters
Create Awareness Through HSE PostersCreate Awareness Through HSE Posters
Create Awareness Through HSE Posters
 
Happy birthday colorful cake
Happy birthday colorful cakeHappy birthday colorful cake
Happy birthday colorful cake
 
SachinKumar_Imp
SachinKumar_ImpSachinKumar_Imp
SachinKumar_Imp
 
Effect of planting containers, cutting types and IBA on transplants productio...
Effect of planting containers, cutting types and IBA on transplants productio...Effect of planting containers, cutting types and IBA on transplants productio...
Effect of planting containers, cutting types and IBA on transplants productio...
 
Vipra jan 2016
Vipra jan 2016Vipra jan 2016
Vipra jan 2016
 
Session 10 – advanced goal &amp; instanter
Session 10 – advanced goal &amp; instanterSession 10 – advanced goal &amp; instanter
Session 10 – advanced goal &amp; instanter
 
Bhtkh july15
Bhtkh july15Bhtkh july15
Bhtkh july15
 
Session 00 – Course overview Maya Dynamics
Session 00 – Course overview Maya DynamicsSession 00 – Course overview Maya Dynamics
Session 00 – Course overview Maya Dynamics
 

Ähnlich wie Scaling Xtext for Large Hardware Design Projects

Devoxx : being productive with JHipster
Devoxx : being productive with JHipsterDevoxx : being productive with JHipster
Devoxx : being productive with JHipsterJulien Dubois
 
Dart the better Javascript 2015
Dart the better Javascript 2015Dart the better Javascript 2015
Dart the better Javascript 2015Jorg Janke
 
Future of Microservices - Jakub Hadvig
Future of Microservices - Jakub HadvigFuture of Microservices - Jakub Hadvig
Future of Microservices - Jakub HadvigWEBtlak
 
Node.js Web Apps @ ebay scale
Node.js Web Apps @ ebay scaleNode.js Web Apps @ ebay scale
Node.js Web Apps @ ebay scaleDmytro Semenov
 
Plc2 2015 your own ide
Plc2 2015 your own idePlc2 2015 your own ide
Plc2 2015 your own ideSigasi
 
Who needs containers in a serverless world
Who needs containers in a serverless worldWho needs containers in a serverless world
Who needs containers in a serverless worldMatthias Luebken
 
Electron JS | Build cross-platform desktop applications with web technologies
Electron JS | Build cross-platform desktop applications with web technologiesElectron JS | Build cross-platform desktop applications with web technologies
Electron JS | Build cross-platform desktop applications with web technologiesBethmi Gunasekara
 
Instant developer onboarding with self contained repositories
Instant developer onboarding with self contained repositoriesInstant developer onboarding with self contained repositories
Instant developer onboarding with self contained repositoriesYshay Yaacobi
 
Design Summit - Technology Vision - Oleg Barenboim and Jason Frey
Design Summit - Technology Vision - Oleg Barenboim and Jason FreyDesign Summit - Technology Vision - Oleg Barenboim and Jason Frey
Design Summit - Technology Vision - Oleg Barenboim and Jason FreyManageIQ
 
Netflix Container Scheduling and Execution - QCon New York 2016
Netflix Container Scheduling and Execution - QCon New York 2016Netflix Container Scheduling and Execution - QCon New York 2016
Netflix Container Scheduling and Execution - QCon New York 2016aspyker
 
Scheduling a fuller house - Talk at QCon NY 2016
Scheduling a fuller house - Talk at QCon NY 2016Scheduling a fuller house - Talk at QCon NY 2016
Scheduling a fuller house - Talk at QCon NY 2016Sharma Podila
 
Continuous Delivery: 5 years later (Incontro DevOps 2018)
Continuous Delivery: 5 years later (Incontro DevOps 2018)Continuous Delivery: 5 years later (Incontro DevOps 2018)
Continuous Delivery: 5 years later (Incontro DevOps 2018)Giovanni Toraldo
 
Architektura html, css i javascript - Jan Kraus
Architektura html, css i javascript - Jan KrausArchitektura html, css i javascript - Jan Kraus
Architektura html, css i javascript - Jan KrausWomen in Technology Poland
 
GeoServer Developers Workshop
GeoServer Developers WorkshopGeoServer Developers Workshop
GeoServer Developers WorkshopJody Garnett
 
[scala.by] Launching new application fast
[scala.by] Launching new application fast[scala.by] Launching new application fast
[scala.by] Launching new application fastDenis Karpenko
 
Front-End Developer's Career Roadmap
Front-End Developer's Career RoadmapFront-End Developer's Career Roadmap
Front-End Developer's Career RoadmapWebStackAcademy
 

Ähnlich wie Scaling Xtext for Large Hardware Design Projects (20)

Devoxx : being productive with JHipster
Devoxx : being productive with JHipsterDevoxx : being productive with JHipster
Devoxx : being productive with JHipster
 
Dart the better Javascript 2015
Dart the better Javascript 2015Dart the better Javascript 2015
Dart the better Javascript 2015
 
Future of Microservices - Jakub Hadvig
Future of Microservices - Jakub HadvigFuture of Microservices - Jakub Hadvig
Future of Microservices - Jakub Hadvig
 
Paris.py
Paris.pyParis.py
Paris.py
 
Code-Hub
Code-HubCode-Hub
Code-Hub
 
Node.js Web Apps @ ebay scale
Node.js Web Apps @ ebay scaleNode.js Web Apps @ ebay scale
Node.js Web Apps @ ebay scale
 
Plc2 2015 your own ide
Plc2 2015 your own idePlc2 2015 your own ide
Plc2 2015 your own ide
 
Who needs containers in a serverless world
Who needs containers in a serverless worldWho needs containers in a serverless world
Who needs containers in a serverless world
 
Electron JS | Build cross-platform desktop applications with web technologies
Electron JS | Build cross-platform desktop applications with web technologiesElectron JS | Build cross-platform desktop applications with web technologies
Electron JS | Build cross-platform desktop applications with web technologies
 
Instant developer onboarding with self contained repositories
Instant developer onboarding with self contained repositoriesInstant developer onboarding with self contained repositories
Instant developer onboarding with self contained repositories
 
Design Summit - Technology Vision - Oleg Barenboim and Jason Frey
Design Summit - Technology Vision - Oleg Barenboim and Jason FreyDesign Summit - Technology Vision - Oleg Barenboim and Jason Frey
Design Summit - Technology Vision - Oleg Barenboim and Jason Frey
 
Netflix Container Scheduling and Execution - QCon New York 2016
Netflix Container Scheduling and Execution - QCon New York 2016Netflix Container Scheduling and Execution - QCon New York 2016
Netflix Container Scheduling and Execution - QCon New York 2016
 
Scheduling a fuller house - Talk at QCon NY 2016
Scheduling a fuller house - Talk at QCon NY 2016Scheduling a fuller house - Talk at QCon NY 2016
Scheduling a fuller house - Talk at QCon NY 2016
 
Continuous Delivery: 5 years later (Incontro DevOps 2018)
Continuous Delivery: 5 years later (Incontro DevOps 2018)Continuous Delivery: 5 years later (Incontro DevOps 2018)
Continuous Delivery: 5 years later (Incontro DevOps 2018)
 
Architektura html, css i javascript - Jan Kraus
Architektura html, css i javascript - Jan KrausArchitektura html, css i javascript - Jan Kraus
Architektura html, css i javascript - Jan Kraus
 
GeoServer Developers Workshop
GeoServer Developers WorkshopGeoServer Developers Workshop
GeoServer Developers Workshop
 
[scala.by] Launching new application fast
[scala.by] Launching new application fast[scala.by] Launching new application fast
[scala.by] Launching new application fast
 
Software maintenance PyConUK 2016
Software maintenance PyConUK 2016Software maintenance PyConUK 2016
Software maintenance PyConUK 2016
 
Front-End Developer's Career Roadmap
Front-End Developer's Career RoadmapFront-End Developer's Career Roadmap
Front-End Developer's Career Roadmap
 
Embedded Linux - Building toolchain
Embedded Linux - Building toolchainEmbedded Linux - Building toolchain
Embedded Linux - Building toolchain
 

Kürzlich hochgeladen

Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfNeo4j
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA
 
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesMuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesManik S Magar
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integrationmarketing932765
 
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxGenerative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxfnnc6jmgwh
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPathCommunity
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfAarwolf Industries LLC
 
Français Patch Tuesday - Avril
Français Patch Tuesday - AvrilFrançais Patch Tuesday - Avril
Français Patch Tuesday - AvrilIvanti
 
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...panagenda
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityIES VE
 
Kuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorialKuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorialJoão Esperancinha
 
WomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneWomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneUiPathCommunity
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observabilityitnewsafrica
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFMichael Gough
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesThousandEyes
 
Infrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsInfrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsYoss Cohen
 
Accelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with PlatformlessAccelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with PlatformlessWSO2
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch TuesdayIvanti
 
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Mark Simos
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfpanagenda
 

Kürzlich hochgeladen (20)

Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdf
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024
 
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesMuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
 
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxGenerative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to Hero
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdf
 
Français Patch Tuesday - Avril
Français Patch Tuesday - AvrilFrançais Patch Tuesday - Avril
Français Patch Tuesday - Avril
 
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a reality
 
Kuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorialKuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorial
 
WomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneWomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyone
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDF
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
 
Infrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsInfrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platforms
 
Accelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with PlatformlessAccelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with Platformless
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch Tuesday
 
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
 

Scaling Xtext for Large Hardware Design Projects

  • 2. Sigasi ● IDE for Hardware Description Languages ○ VHDL, (System)Verilog ● Using Xtext for 4 years ● Large user base ○ (commercial, free, students)
  • 3. Our company goal ● Assist hardware designer ● High quality interactive front-end compiler ○ Instant error reporting ■ parsing, semantic, linting, style checking ○ Visualisations, design exploration, ... ○ Integrate with ecosystem ■ other compilers, simulators, synthesizers
  • 4. The challenge ● Large projects ○ > 250 KLOC is not uncommon ○ design + external libraries ○ big files ■ some libraries are distributed as 1 file ■ generated hardware cores ●
  • 5. Adopting Xtext ● Started with the early Xtext 2.0 snapshots ● Initial performance analysis ○ Clean build performance of a big project (330k LOC) ■ > 20 minutes ■ > 2 GB ○ Editing big files (> 1 MB) ■ unusable
  • 6. Adopting Xtext ● Started with the early Xtext 2.0 snapshots ● Initial performance analysis ○ Clean build performance of a big project (330k LOC) ■ > 20 minutes → < 1 min ■ > 2 GB → < 1 GB memory ○ Editing big files (> 1 MB) ■ unusable → usable with reduced editor
  • 7. ● Xtext framework improvements ● Measure → analyze → improve or cheat ○ faster build ○ reduce memory usage ○ UI responsiveness Improving performance
  • 8. Overview ● Analysing build performance ○ Analyze the build ■ Macro build measurements ■ Key performance points ○ Reduce workload ○ Parallelize the build ● Analyzing UI issues ○ Monitoring the UI thread ○ Saveguards
  • 9. Analyzing builds: builder overview Global indexing Linking Validation Custom Validation Global index Eclipse resources warnings errors resource descriptions Builder Participants resource changes ?
  • 10. Analyzing builds: metrics ● For each build ○ # of files being build ○ timing: Global index, Linking, Validation, Individual builder participants ● Instrument by overriding ClusteringBuilderState & XtextBuilder ● Example: Building 134 resources, timing: { global index=1806, linking=378, validation=823, totalLinkingAndValidation=1364 }
  • 11. Analyzing builds: resource loads ● Observation: ○ Most time spent in resource loads ○ Certain files are loaded multiple times?! Global indexing Linking Validation Builder Participants resources LOAD POTENTIAL RELOAD POTENTIAL RELOAD
  • 12. Analyzing builds: Memory footprint Global index Resource Set ● Global index ○ always loaded →limit size ○ use hashing: name + hash of contents ● Reduce EMF size derived model -> Xcore model http://www.sigasi.com/content/view-complexity-your-xtext-ecore-model
  • 13. Optimize loading ● What is resource load? ○ Parse ○ build EMF model ○ build Node model ● Parallelise ○ parse multiple files simultaneously ○ ~3 time faster loads on 4 core machine ○ EMF synchronization issues ● Cache ○ serialize EMF and Node model in a cache ○ 3-4 time faster loads
  • 14. Linking Global indexing Linking Validation Builder Participants ● Language specific ○ VHDL vs Verilog ● Avoiding linking ○ expressions that we couldn’t handle (yet) ○ library files, only when used in user-code ● Many iterations ○ lazy linking vs eager linking ○ From 40% of build time to 20%
  • 15. Linking Pseudo-algorithm ● In your IScopePovider ○ Find a eContainer that can be linked eagerly ○ Link everything ■ unresolved links become UnlinkedDeclaration ● In your DefaultLinkingService ○ getLinkedObjects(...) ■ already resolved? → skip it ● Disable notifications
  • 16. Validation Global indexing Linking Validation Builder Participants ● Combine validations to avoid model traversals ● Local analysis, do global validations moved into builder participant ● Avoid validation ○ disabled validations ○ libraries: errors & warnings are suppressed anyway ● Monitor
  • 17. Track performance ● Nightly build ● log build times
  • 18. UI responsiveness ● Measuring: detect a blocked UI thread ○ initially Svelto https://github.com/dragos/svelto ○ now our own method & logging ○ Eclipse Mars ● Improvements ○ UI is for drawing only! ○ Make sure everything is cancellable ● Safeguards ○ certain services should never be executed on the UI thread => check & log
  • 19. Lightweight Editor (fallback) ● Syntax-highlighting + markers ● For files > 1 MB ● Based on ContentTypes extension point
  • 20. Two ContentTypes (based on file size) <extension point="org.eclipse.core.contenttype.contentTypes"> <content-type ... describer="com.sigasi...FullVhdlContentDescriber" name="VHDL editor" <describer class="...FullVhdlContentDescriber" /> </content-type> <content-type ... describer="com.sigasi....LightweightVhdlContentDescriber" name="Lightweight VHDL editor" <describer class="...LightweightVhdlContentDescriber" /> </content-type> </extension>
  • 21. Future work ● Continuous process ● Cache global index info per resource? ● Linking without node model? ● Avoid project locking build (Xtext 2.8?)
  • 22. Come talk to us about... ● Documentation generation ● Fancy linking algorithms / type systems ● Graphical views ● Cross-language support ● Testing Xtext-plugins ● Lexical macros ● Manage large amount of validations ● ...
  • 23. +1 0 -1 Sign in: www.eclipsecon.org Evaluate the sessions