Presented by David Hammer, Robert Breske
The GVL aggregates millions of usage reports for musical and cinematographic works from various sources every year. To cope with this amount of incoming data, a transparent and audit-proof data aggregation system was desired to be built. The solution was a DMN rule-based approach embedded in camunda's proccess engine framework, altogether with an audit-proof storage of the process history data. This system is capable of simulating the process flow for any processed data record at any later point in time, in case of a need for revision or validation. The GVL currently runs this scalable system with a troughput of more than one million daily process instances.
2. 2
Robert Breske
robert.breske@cimt-ag.de
developer and software consultant
5+ years project experience
David Hammer
david.hammer@gvl.de
business architect
15+ years overall experience
Society of
collective rights
management
3. 3
Performing artists
Producers of sound recordings
Event organisers
Music video clip producers
Rights users
Distribution
License
revenues
Permission to
administrate rights
Licensing
4. GVL matching business
• Documentation and maintainance of product repertoire
• Matching of around 20 million usages each year
• Clarification of rights conflicts
• Product data deduplication
4
5. 5
Product data from
rights owners
Usage reports from
radio and tv stations
Reported
participations by
artists
Data core
Data reception
and pre-processing
Distribution system
Product fusion process
300M+ Distinct master
data objects
Usage reports from
other royalties
Product data from
other sources
14. 14
• Full history can be rebuild at any point in time
• All process input data is stored in audit database
• Re-execution (=simulation) of process provides
„full history“
15. 15
[1]
• History level set to null
• Process input saved to
denormalized audit database
Runtime
database
History
database
inputVariableA inputVariableB softwareVersion
Custom audit
database
• Rebuild desired history only by
re-executing the process instance
in a sandbox (=simulation)
16. 16
[1]
• History level set to null
• Process input saved to
denormalized audit database
Runtime
database
History
database
inputVariableA inputVariableB softwareVersion
Custom audit
database
• Rebuild desired history only by
re-executing the process instance
in a sandbox (=simulation)
19. 19
Bad practice
Fail forward allows for error
recovery even though all steps
are executed synchronously in
happy path
No asynchronous points to
prevent variable updates on
database
20. 20
Bad practice
Fail forward allows for error
recovery even though all steps
are executed synchronously in
happy path
No asynchronous points to
prevent variable updates on
database
30. Process instances per day
0
200000
400000
600000
800000
1000000
1200000
1400000
0 5 10 15 20 25
30
We stopped scaling at 1M+
process instances per day
31. Process instances per day
0
200000
400000
600000
800000
1000000
1200000
1400000
0 5 10 15 20 25
31
We stopped scaling at 1M+
process instances per day