SlideShare ist ein Scribd-Unternehmen logo
1 von 36
Downloaden Sie, um offline zu lesen
MongoDB@Bayern
Migration from RDBMS, Problems, Unit-Testing
Christian Brensing
Bayerisches Landesamt für
Statistik und Datenverarbeitung
Rechenzentrum Süd
• Template-Processor (~ JSP)


• ODF and RTF


• PDF / PostScript postprocessing


• Groovy, Python, Ruby, Tcl


• In Production using RDBMS
  since 2008


• Central HR system (SAP) uses
  BayText to create output
Client for developing the templates (BayText-IDE)
ODF-Template using Ruby
Generated document
2 Million
Generated documents per year
Tiny dataset
50,000 records and 10 GB BLOB
a small system after all
So why migrate?
Because we can
developers.pop()
     Smaller teams
Easy to learn
Read one book and you're done
Simple mapping
permissions       templates           documents
codes       groups           document_usages
folder_configurations       requests       folders
nodes         structures         template_tags
dynafields       components            roles_roles
roles   bundles        scripts
nodes
  roles
properties
 GridFS
{!
     "_id" : ObjectId(),!
     "_type" : "Document",!
     "updated_at" : ISODate("2007-12-06T10:04:00.543Z"),!
     "updated_by" : "maggie",!
     "version" : 1,!
     "path" : "foo.bar.Document",!
     "description" : "",!
     "language" : "ruby",!
     "format" : "odf",!
     "autoload_uplevel" : -1,!
     "requests" : [!
        {!
           "name" : "Test",!
           "description" : "foo", !
           "xml" : zlib("<?xml ...?>")!
        }, !
        ... !
     ],!
     "structure" : [!
        { !
           "alias" : "some alias",!
           "component_path" : "foo.bar.Document$Document",!
           "parent", -1 !
        },!
        ... !
     ],      !
     }     !
     "tags" : ["foo", "bar", "baz"]!
}
public class RoleReadConverter implements Converter<DBObject, Role> {!
    @Override!
    public Role convert(DBObject source) {!
        Role role = new Role();!
        role.setName((String) source.get("name"));!
        ...!
        return role;!
    }!
}!
!
!
public class RoleWriteConverter implements Converter<Role, DBObject> {!
    @Override!
    public DBObject convert(Role source) {!
        return new BasicDBObject()!
                .append("name", role.getName())!
                .append(...);!
    }!
}!
!
!
// Convert a DBObject to an Entity!
conversionService.convert(collection.findOne(...));




                     Easy mapping using Spring-ConversionService
Performance
   up to 10x
ORM no more
The MongoDB driver is all you need
Obstacles?
Mentality
Referential Integrity
     Multi-Document-Update
{!
     "_id" : ObjectId(),!
     "_type" : "Folder", !
     "updated_at" : ISODate("2007-12-06T10:04:00.543Z"),!
     "updated_by" : "maggie",!
     "version" : 1,!
     "path" : "foo.bar.Folder",!
     "description" : "bla",!
     "bundles" : [!
        ...    !
     ]!
     "document_usages" : [!
        {!
           "name" : "foo",!
           "description" : "bla", !
           "exec_order" : 5, !
           "print_copies" : 2,!
           "type": "FAIR_COPY",!
           "document_path" : "a.b.Document",!
           "bundle" : "bar"!
        },!
        ...!
}



           Linking documents via a path instead of ObjectId
But paths are mutable!
    All references must be updated
Approach

• Pseudo-Transaction with an update locking
  mechanism in a custom oplog collection


• Periodical repair jobs finishing cancelled
  operations


• Extended interpretation of eventual consistency
Unit-Tests
Continuous Integration
F.I.R.S.T.
Isolated
Tests are using a unique DB per JVM
@Configuration!
@Profile("test")!
public class TestSpringConfiguration extends SpringConfiguration {!
    @Override!
    public void init() {!
        setTestProperties();!
        !
        super.init();!
        !
        // Test-DB löschen!
        db().dropDatabase();!
    }!
    !
    private void setTestProperties() {!
        String dbname = env.getProperty("db.name", "baytext_test_" + getSimpleUsername());!
        String host = env.getProperty("db.host", "mongodb-dev.db.rz-sued.bybn.de");!
        String port = env.getProperty("db.port", "27016");!
        !
        cmProperties.setProperty("servers", String.format("%s:%s", host, port));!
        cmProperties.setProperty("name", dbname);!
    }!
!
    // OS-Username without prefix (e.g. maggie instead of lfstad-maggie)!
    private static String getSimpleUsername() {!
        String username = SystemUtils.USER_NAME;!
        int indexOfDash = username.indexOf('-');!
        return indexOfDash != -1 ? username.substring(indexOfDash + 1) : username;!
    }!
}



                                 Test-ApplicationContext
Repeatable
Collections are purged before each test method
public class MongoTestExcecutionListener extends AbstractTestExecutionListener {!
    @Override!
    public void beforeTestMethod(TestContext testContext) throws Exception {!
        purgeCollections();!
    }!
    !
    private void purgeCollections() {!
        DB db = MongoDBHolder.getDB();!
        for (String collectionName : db.getCollectionNames()) {!
            if (collectionName.startsWith("fs.") || collectionName.startsWith("system.")) {!
                continue;!
            }!
            DBCollection collection = db.getCollection(collectionName);!
            if (!collection.isCapped()) {!
                collection.remove(new BasicDBObject());!
            }!
        }!
    }!
}




                              Spring-TestExecutionListener
@RunWith(SpringJUnit4ClassRunner.class)!
@ActiveProfiles({"test"})!
@ContextConfiguration(classes = TestSpringConfiguration.class)!
@TestExecutionListeners({!
    DependencyInjectionTestExecutionListener.class,!
    DirtiesContextTestExecutionListener.class,!
    MongoTestExcecutionListener.class!
})!
public abstract class MongoTestSupport {!
}!
!
!
!
public class DocumentRepositoryTest extends MongoTestSupport {!
}!




                                     Test base class
95% Coverage
Summary

• Developing the data layer is fun again


• Reduced complexity


• Flat learning curve compared to SQL/ORM


• use_mongo() unless transactions_required
Thank you!

Weitere ähnliche Inhalte

Was ist angesagt?

WorkFlow: An Inquiry Into Productivity by Timothy Bolton
WorkFlow:  An Inquiry Into Productivity by Timothy BoltonWorkFlow:  An Inquiry Into Productivity by Timothy Bolton
WorkFlow: An Inquiry Into Productivity by Timothy BoltonMiva
 
Shell Tips & Tricks
Shell Tips & TricksShell Tips & Tricks
Shell Tips & TricksMongoDB
 
Updates to the java api for json processing for java ee 8
Updates to the java api for json processing for java ee 8Updates to the java api for json processing for java ee 8
Updates to the java api for json processing for java ee 8Alex Soto
 
Differential Sync and JSON Patch @ SpringOne2GX 2014
Differential Sync and JSON Patch @ SpringOne2GX 2014Differential Sync and JSON Patch @ SpringOne2GX 2014
Differential Sync and JSON Patch @ SpringOne2GX 2014Brian Cavalier
 
Mongo Presentation by Metatagg Solutions
Mongo Presentation by Metatagg SolutionsMongo Presentation by Metatagg Solutions
Mongo Presentation by Metatagg SolutionsMetatagg Solutions
 
Infinispan,Lucene,Hibername OGM
Infinispan,Lucene,Hibername OGMInfinispan,Lucene,Hibername OGM
Infinispan,Lucene,Hibername OGMJBug Italy
 
Documentation with sphinx @ PyHug
Documentation with sphinx @ PyHugDocumentation with sphinx @ PyHug
Documentation with sphinx @ PyHugJimmy Lai
 
April 2010 - JBoss Web Services
April 2010 - JBoss Web ServicesApril 2010 - JBoss Web Services
April 2010 - JBoss Web ServicesJBug Italy
 
Introduction to node.js
Introduction to node.jsIntroduction to node.js
Introduction to node.jsAdrien Guéret
 
[제1회 루씬 한글분석기 기술세미나] solr로 나만의 검색엔진을 만들어보자
[제1회 루씬 한글분석기 기술세미나] solr로 나만의 검색엔진을 만들어보자[제1회 루씬 한글분석기 기술세미나] solr로 나만의 검색엔진을 만들어보자
[제1회 루씬 한글분석기 기술세미나] solr로 나만의 검색엔진을 만들어보자Donghyeok Kang
 
HTTP Caching and PHP
HTTP Caching and PHPHTTP Caching and PHP
HTTP Caching and PHPDavid de Boer
 
Code decoupling from Symfony (and others frameworks) - PHP Conference Brasil ...
Code decoupling from Symfony (and others frameworks) - PHP Conference Brasil ...Code decoupling from Symfony (and others frameworks) - PHP Conference Brasil ...
Code decoupling from Symfony (and others frameworks) - PHP Conference Brasil ...Miguel Gallardo
 
Fazendo mágica com ElasticSearch
Fazendo mágica com ElasticSearchFazendo mágica com ElasticSearch
Fazendo mágica com ElasticSearchPedro Franceschi
 
Getting started with MongoDB and Scala - Open Source Bridge 2012
Getting started with MongoDB and Scala - Open Source Bridge 2012Getting started with MongoDB and Scala - Open Source Bridge 2012
Getting started with MongoDB and Scala - Open Source Bridge 2012sullis
 

Was ist angesagt? (20)

WorkFlow: An Inquiry Into Productivity by Timothy Bolton
WorkFlow:  An Inquiry Into Productivity by Timothy BoltonWorkFlow:  An Inquiry Into Productivity by Timothy Bolton
WorkFlow: An Inquiry Into Productivity by Timothy Bolton
 
The emerging world of mongo db csp
The emerging world of mongo db   cspThe emerging world of mongo db   csp
The emerging world of mongo db csp
 
Shell Tips & Tricks
Shell Tips & TricksShell Tips & Tricks
Shell Tips & Tricks
 
Updates to the java api for json processing for java ee 8
Updates to the java api for json processing for java ee 8Updates to the java api for json processing for java ee 8
Updates to the java api for json processing for java ee 8
 
Differential Sync and JSON Patch @ SpringOne2GX 2014
Differential Sync and JSON Patch @ SpringOne2GX 2014Differential Sync and JSON Patch @ SpringOne2GX 2014
Differential Sync and JSON Patch @ SpringOne2GX 2014
 
ElasticSearch
ElasticSearchElasticSearch
ElasticSearch
 
Mongodb
MongodbMongodb
Mongodb
 
Nodejs - A-quick-tour-v3
Nodejs - A-quick-tour-v3Nodejs - A-quick-tour-v3
Nodejs - A-quick-tour-v3
 
Mongo Presentation by Metatagg Solutions
Mongo Presentation by Metatagg SolutionsMongo Presentation by Metatagg Solutions
Mongo Presentation by Metatagg Solutions
 
Infinispan,Lucene,Hibername OGM
Infinispan,Lucene,Hibername OGMInfinispan,Lucene,Hibername OGM
Infinispan,Lucene,Hibername OGM
 
Documentation with sphinx @ PyHug
Documentation with sphinx @ PyHugDocumentation with sphinx @ PyHug
Documentation with sphinx @ PyHug
 
April 2010 - JBoss Web Services
April 2010 - JBoss Web ServicesApril 2010 - JBoss Web Services
April 2010 - JBoss Web Services
 
Introduction to node.js
Introduction to node.jsIntroduction to node.js
Introduction to node.js
 
[제1회 루씬 한글분석기 기술세미나] solr로 나만의 검색엔진을 만들어보자
[제1회 루씬 한글분석기 기술세미나] solr로 나만의 검색엔진을 만들어보자[제1회 루씬 한글분석기 기술세미나] solr로 나만의 검색엔진을 만들어보자
[제1회 루씬 한글분석기 기술세미나] solr로 나만의 검색엔진을 만들어보자
 
HTTP Caching and PHP
HTTP Caching and PHPHTTP Caching and PHP
HTTP Caching and PHP
 
Nodejs - A quick tour (v4)
Nodejs - A quick tour (v4)Nodejs - A quick tour (v4)
Nodejs - A quick tour (v4)
 
Code decoupling from Symfony (and others frameworks) - PHP Conference Brasil ...
Code decoupling from Symfony (and others frameworks) - PHP Conference Brasil ...Code decoupling from Symfony (and others frameworks) - PHP Conference Brasil ...
Code decoupling from Symfony (and others frameworks) - PHP Conference Brasil ...
 
Gl qtp day 3 2
Gl qtp day 3   2Gl qtp day 3   2
Gl qtp day 3 2
 
Fazendo mágica com ElasticSearch
Fazendo mágica com ElasticSearchFazendo mágica com ElasticSearch
Fazendo mágica com ElasticSearch
 
Getting started with MongoDB and Scala - Open Source Bridge 2012
Getting started with MongoDB and Scala - Open Source Bridge 2012Getting started with MongoDB and Scala - Open Source Bridge 2012
Getting started with MongoDB and Scala - Open Source Bridge 2012
 

Andere mochten auch

MongoDB: The Operational Big Data by NORBERTO LEITE at Big Data Spain 2014
 MongoDB: The Operational Big Data by NORBERTO LEITE at Big Data Spain 2014 MongoDB: The Operational Big Data by NORBERTO LEITE at Big Data Spain 2014
MongoDB: The Operational Big Data by NORBERTO LEITE at Big Data Spain 2014Big Data Spain
 
Java cro 2016 - From.... to Scrum by Jurica Krizanic
Java cro 2016 - From.... to Scrum by Jurica KrizanicJava cro 2016 - From.... to Scrum by Jurica Krizanic
Java cro 2016 - From.... to Scrum by Jurica KrizanicComsysto Reply GmbH
 
Geospatial applications created using java script(and nosql)
Geospatial applications created using java script(and nosql)Geospatial applications created using java script(and nosql)
Geospatial applications created using java script(and nosql)Comsysto Reply GmbH
 
Caching and JCache with Greg Luck 18.02.16
Caching and JCache with Greg Luck 18.02.16Caching and JCache with Greg Luck 18.02.16
Caching and JCache with Greg Luck 18.02.16Comsysto Reply GmbH
 
SPARK STREAMING Spark Hadoop User Group Munich Meetup 2016
SPARK STREAMING  Spark Hadoop User Group Munich Meetup 2016SPARK STREAMING  Spark Hadoop User Group Munich Meetup 2016
SPARK STREAMING Spark Hadoop User Group Munich Meetup 2016Comsysto Reply GmbH
 

Andere mochten auch (6)

MongoDB: The Operational Big Data by NORBERTO LEITE at Big Data Spain 2014
 MongoDB: The Operational Big Data by NORBERTO LEITE at Big Data Spain 2014 MongoDB: The Operational Big Data by NORBERTO LEITE at Big Data Spain 2014
MongoDB: The Operational Big Data by NORBERTO LEITE at Big Data Spain 2014
 
Java cro 2016 - From.... to Scrum by Jurica Krizanic
Java cro 2016 - From.... to Scrum by Jurica KrizanicJava cro 2016 - From.... to Scrum by Jurica Krizanic
Java cro 2016 - From.... to Scrum by Jurica Krizanic
 
MongoDB
MongoDBMongoDB
MongoDB
 
Geospatial applications created using java script(and nosql)
Geospatial applications created using java script(and nosql)Geospatial applications created using java script(and nosql)
Geospatial applications created using java script(and nosql)
 
Caching and JCache with Greg Luck 18.02.16
Caching and JCache with Greg Luck 18.02.16Caching and JCache with Greg Luck 18.02.16
Caching and JCache with Greg Luck 18.02.16
 
SPARK STREAMING Spark Hadoop User Group Munich Meetup 2016
SPARK STREAMING  Spark Hadoop User Group Munich Meetup 2016SPARK STREAMING  Spark Hadoop User Group Munich Meetup 2016
SPARK STREAMING Spark Hadoop User Group Munich Meetup 2016
 

Ähnlich wie Replacing Oracle with MongoDB for a templating application at the Bavarian government

Spring 3 - An Introduction
Spring 3 - An IntroductionSpring 3 - An Introduction
Spring 3 - An IntroductionThorsten Kamann
 
Ingo Muschenetz: Titanium Studio Deep Dive
Ingo Muschenetz: Titanium Studio Deep DiveIngo Muschenetz: Titanium Studio Deep Dive
Ingo Muschenetz: Titanium Studio Deep DiveAxway Appcelerator
 
Hd insight programming
Hd insight programmingHd insight programming
Hd insight programmingCasear Chu
 
The go-start webframework (GTUG Vienna 27.03.2012)
The go-start webframework (GTUG Vienna 27.03.2012)The go-start webframework (GTUG Vienna 27.03.2012)
The go-start webframework (GTUG Vienna 27.03.2012)ungerik
 
Dealing with Azure Cosmos DB
Dealing with Azure Cosmos DBDealing with Azure Cosmos DB
Dealing with Azure Cosmos DBMihail Mateev
 
BP-6 Repository Customization Best Practices
BP-6 Repository Customization Best PracticesBP-6 Repository Customization Best Practices
BP-6 Repository Customization Best PracticesAlfresco Software
 
Introduction to MongoDB
Introduction to MongoDBIntroduction to MongoDB
Introduction to MongoDBJustin Smestad
 
Building a real life application in node js
Building a real life application in node jsBuilding a real life application in node js
Building a real life application in node jsfakedarren
 
Spring 3 - Der dritte Frühling
Spring 3 - Der dritte FrühlingSpring 3 - Der dritte Frühling
Spring 3 - Der dritte FrühlingThorsten Kamann
 
MongoDB at ZPUGDC
MongoDB at ZPUGDCMongoDB at ZPUGDC
MongoDB at ZPUGDCMike Dirolf
 
Allura - an Open Source MongoDB Based Document Oriented SourceForge
Allura - an Open Source MongoDB Based Document Oriented SourceForgeAllura - an Open Source MongoDB Based Document Oriented SourceForge
Allura - an Open Source MongoDB Based Document Oriented SourceForgeRick Copeland
 
Hands On Spring Data
Hands On Spring DataHands On Spring Data
Hands On Spring DataEric Bottard
 
AMD - Why, What and How
AMD - Why, What and HowAMD - Why, What and How
AMD - Why, What and HowMike Wilcox
 
IndexedDB - Querying and Performance
IndexedDB - Querying and PerformanceIndexedDB - Querying and Performance
IndexedDB - Querying and PerformanceParashuram N
 
Turning a Search Engine into a Relational Database
Turning a Search Engine into a Relational DatabaseTurning a Search Engine into a Relational Database
Turning a Search Engine into a Relational DatabaseMatthias Wahl
 
Serializing EMF models with Xtext
Serializing EMF models with XtextSerializing EMF models with Xtext
Serializing EMF models with Xtextmeysholdt
 
Building Your First MongoDB App
Building Your First MongoDB AppBuilding Your First MongoDB App
Building Your First MongoDB AppHenrik Ingo
 

Ähnlich wie Replacing Oracle with MongoDB for a templating application at the Bavarian government (20)

Spring 3 - An Introduction
Spring 3 - An IntroductionSpring 3 - An Introduction
Spring 3 - An Introduction
 
Ingo Muschenetz: Titanium Studio Deep Dive
Ingo Muschenetz: Titanium Studio Deep DiveIngo Muschenetz: Titanium Studio Deep Dive
Ingo Muschenetz: Titanium Studio Deep Dive
 
Hd insight programming
Hd insight programmingHd insight programming
Hd insight programming
 
The go-start webframework (GTUG Vienna 27.03.2012)
The go-start webframework (GTUG Vienna 27.03.2012)The go-start webframework (GTUG Vienna 27.03.2012)
The go-start webframework (GTUG Vienna 27.03.2012)
 
Dealing with Azure Cosmos DB
Dealing with Azure Cosmos DBDealing with Azure Cosmos DB
Dealing with Azure Cosmos DB
 
BP-6 Repository Customization Best Practices
BP-6 Repository Customization Best PracticesBP-6 Repository Customization Best Practices
BP-6 Repository Customization Best Practices
 
Introduction to MongoDB
Introduction to MongoDBIntroduction to MongoDB
Introduction to MongoDB
 
Building a real life application in node js
Building a real life application in node jsBuilding a real life application in node js
Building a real life application in node js
 
Spring 3 - Der dritte Frühling
Spring 3 - Der dritte FrühlingSpring 3 - Der dritte Frühling
Spring 3 - Der dritte Frühling
 
Es.next
Es.nextEs.next
Es.next
 
MongoDB at ZPUGDC
MongoDB at ZPUGDCMongoDB at ZPUGDC
MongoDB at ZPUGDC
 
Allura - an Open Source MongoDB Based Document Oriented SourceForge
Allura - an Open Source MongoDB Based Document Oriented SourceForgeAllura - an Open Source MongoDB Based Document Oriented SourceForge
Allura - an Open Source MongoDB Based Document Oriented SourceForge
 
Hands On Spring Data
Hands On Spring DataHands On Spring Data
Hands On Spring Data
 
AMD - Why, What and How
AMD - Why, What and HowAMD - Why, What and How
AMD - Why, What and How
 
JS Essence
JS EssenceJS Essence
JS Essence
 
IndexedDB - Querying and Performance
IndexedDB - Querying and PerformanceIndexedDB - Querying and Performance
IndexedDB - Querying and Performance
 
Turning a Search Engine into a Relational Database
Turning a Search Engine into a Relational DatabaseTurning a Search Engine into a Relational Database
Turning a Search Engine into a Relational Database
 
Serializing EMF models with Xtext
Serializing EMF models with XtextSerializing EMF models with Xtext
Serializing EMF models with Xtext
 
Building Your First MongoDB App
Building Your First MongoDB AppBuilding Your First MongoDB App
Building Your First MongoDB App
 
"Javascript" por Tiago Rodrigues
"Javascript" por Tiago Rodrigues"Javascript" por Tiago Rodrigues
"Javascript" por Tiago Rodrigues
 

Mehr von Comsysto Reply GmbH

Architectural Decisions: Smoothly and Consistently
Architectural Decisions: Smoothly and ConsistentlyArchitectural Decisions: Smoothly and Consistently
Architectural Decisions: Smoothly and ConsistentlyComsysto Reply GmbH
 
ljug-meetup-2023-03-hexagonal-architecture.pdf
ljug-meetup-2023-03-hexagonal-architecture.pdfljug-meetup-2023-03-hexagonal-architecture.pdf
ljug-meetup-2023-03-hexagonal-architecture.pdfComsysto Reply GmbH
 
Software Architecture and Architectors: useless VS valuable
Software Architecture and Architectors: useless VS valuableSoftware Architecture and Architectors: useless VS valuable
Software Architecture and Architectors: useless VS valuableComsysto Reply GmbH
 
MicroFrontends für Microservices
MicroFrontends für MicroservicesMicroFrontends für Microservices
MicroFrontends für MicroservicesComsysto Reply GmbH
 
Bable on Smart City Munich Meetup: How cities are leveraging innovative partn...
Bable on Smart City Munich Meetup: How cities are leveraging innovative partn...Bable on Smart City Munich Meetup: How cities are leveraging innovative partn...
Bable on Smart City Munich Meetup: How cities are leveraging innovative partn...Comsysto Reply GmbH
 
Smart City Munich Kickoff Meetup
Smart City Munich Kickoff Meetup Smart City Munich Kickoff Meetup
Smart City Munich Kickoff Meetup Comsysto Reply GmbH
 
Data Reliability Challenges with Spark by Henning Kropp (Spark & Hadoop User ...
Data Reliability Challenges with Spark by Henning Kropp (Spark & Hadoop User ...Data Reliability Challenges with Spark by Henning Kropp (Spark & Hadoop User ...
Data Reliability Challenges with Spark by Henning Kropp (Spark & Hadoop User ...Comsysto Reply GmbH
 
"Hadoop Data Lake vs classical Data Warehouse: How to utilize best of both wo...
"Hadoop Data Lake vs classical Data Warehouse: How to utilize best of both wo..."Hadoop Data Lake vs classical Data Warehouse: How to utilize best of both wo...
"Hadoop Data Lake vs classical Data Warehouse: How to utilize best of both wo...Comsysto Reply GmbH
 
Data lake vs Data Warehouse: Hybrid Architectures
Data lake vs Data Warehouse: Hybrid ArchitecturesData lake vs Data Warehouse: Hybrid Architectures
Data lake vs Data Warehouse: Hybrid ArchitecturesComsysto Reply GmbH
 
Java 9 Modularity and Project Jigsaw
Java 9 Modularity and Project JigsawJava 9 Modularity and Project Jigsaw
Java 9 Modularity and Project JigsawComsysto Reply GmbH
 
Distributed Computing and Caching in the Cloud: Hazelcast and Microsoft
Distributed Computing and Caching in the Cloud: Hazelcast and MicrosoftDistributed Computing and Caching in the Cloud: Hazelcast and Microsoft
Distributed Computing and Caching in the Cloud: Hazelcast and MicrosoftComsysto Reply GmbH
 
Grundlegende Konzepte von Elm, React und AngularDart 2 im Vergleich
Grundlegende Konzepte von Elm, React und AngularDart 2 im VergleichGrundlegende Konzepte von Elm, React und AngularDart 2 im Vergleich
Grundlegende Konzepte von Elm, React und AngularDart 2 im VergleichComsysto Reply GmbH
 
Building a fully-automated Fast Data Platform
Building a fully-automated Fast Data PlatformBuilding a fully-automated Fast Data Platform
Building a fully-automated Fast Data PlatformComsysto Reply GmbH
 
Apache Apex: Stream Processing Architecture and Applications
Apache Apex: Stream Processing Architecture and Applications Apache Apex: Stream Processing Architecture and Applications
Apache Apex: Stream Processing Architecture and Applications Comsysto Reply GmbH
 
Ein Prozess lernt laufen: LEGO Mindstorms Steuerung mit BPMN
Ein Prozess lernt laufen: LEGO Mindstorms Steuerung mit BPMNEin Prozess lernt laufen: LEGO Mindstorms Steuerung mit BPMN
Ein Prozess lernt laufen: LEGO Mindstorms Steuerung mit BPMNComsysto Reply GmbH
 
21.04.2016 Meetup: Spark vs. Flink
21.04.2016 Meetup: Spark vs. Flink21.04.2016 Meetup: Spark vs. Flink
21.04.2016 Meetup: Spark vs. FlinkComsysto Reply GmbH
 
Spark RDD-DF-SQL-DS-Spark Hadoop User Group Munich Meetup 2016
Spark RDD-DF-SQL-DS-Spark Hadoop User Group Munich Meetup 2016Spark RDD-DF-SQL-DS-Spark Hadoop User Group Munich Meetup 2016
Spark RDD-DF-SQL-DS-Spark Hadoop User Group Munich Meetup 2016Comsysto Reply GmbH
 
Machinelearning Spark Hadoop User Group Munich Meetup 2016
Machinelearning Spark Hadoop User Group Munich Meetup 2016Machinelearning Spark Hadoop User Group Munich Meetup 2016
Machinelearning Spark Hadoop User Group Munich Meetup 2016Comsysto Reply GmbH
 
Writing Java EE microservices using WildFly Swarm
Writing Java EE microservices using WildFly SwarmWriting Java EE microservices using WildFly Swarm
Writing Java EE microservices using WildFly SwarmComsysto Reply GmbH
 

Mehr von Comsysto Reply GmbH (20)

Architectural Decisions: Smoothly and Consistently
Architectural Decisions: Smoothly and ConsistentlyArchitectural Decisions: Smoothly and Consistently
Architectural Decisions: Smoothly and Consistently
 
ljug-meetup-2023-03-hexagonal-architecture.pdf
ljug-meetup-2023-03-hexagonal-architecture.pdfljug-meetup-2023-03-hexagonal-architecture.pdf
ljug-meetup-2023-03-hexagonal-architecture.pdf
 
Software Architecture and Architectors: useless VS valuable
Software Architecture and Architectors: useless VS valuableSoftware Architecture and Architectors: useless VS valuable
Software Architecture and Architectors: useless VS valuable
 
MicroFrontends für Microservices
MicroFrontends für MicroservicesMicroFrontends für Microservices
MicroFrontends für Microservices
 
Alles offen = gut(ai)
Alles offen = gut(ai)Alles offen = gut(ai)
Alles offen = gut(ai)
 
Bable on Smart City Munich Meetup: How cities are leveraging innovative partn...
Bable on Smart City Munich Meetup: How cities are leveraging innovative partn...Bable on Smart City Munich Meetup: How cities are leveraging innovative partn...
Bable on Smart City Munich Meetup: How cities are leveraging innovative partn...
 
Smart City Munich Kickoff Meetup
Smart City Munich Kickoff Meetup Smart City Munich Kickoff Meetup
Smart City Munich Kickoff Meetup
 
Data Reliability Challenges with Spark by Henning Kropp (Spark & Hadoop User ...
Data Reliability Challenges with Spark by Henning Kropp (Spark & Hadoop User ...Data Reliability Challenges with Spark by Henning Kropp (Spark & Hadoop User ...
Data Reliability Challenges with Spark by Henning Kropp (Spark & Hadoop User ...
 
"Hadoop Data Lake vs classical Data Warehouse: How to utilize best of both wo...
"Hadoop Data Lake vs classical Data Warehouse: How to utilize best of both wo..."Hadoop Data Lake vs classical Data Warehouse: How to utilize best of both wo...
"Hadoop Data Lake vs classical Data Warehouse: How to utilize best of both wo...
 
Data lake vs Data Warehouse: Hybrid Architectures
Data lake vs Data Warehouse: Hybrid ArchitecturesData lake vs Data Warehouse: Hybrid Architectures
Data lake vs Data Warehouse: Hybrid Architectures
 
Java 9 Modularity and Project Jigsaw
Java 9 Modularity and Project JigsawJava 9 Modularity and Project Jigsaw
Java 9 Modularity and Project Jigsaw
 
Distributed Computing and Caching in the Cloud: Hazelcast and Microsoft
Distributed Computing and Caching in the Cloud: Hazelcast and MicrosoftDistributed Computing and Caching in the Cloud: Hazelcast and Microsoft
Distributed Computing and Caching in the Cloud: Hazelcast and Microsoft
 
Grundlegende Konzepte von Elm, React und AngularDart 2 im Vergleich
Grundlegende Konzepte von Elm, React und AngularDart 2 im VergleichGrundlegende Konzepte von Elm, React und AngularDart 2 im Vergleich
Grundlegende Konzepte von Elm, React und AngularDart 2 im Vergleich
 
Building a fully-automated Fast Data Platform
Building a fully-automated Fast Data PlatformBuilding a fully-automated Fast Data Platform
Building a fully-automated Fast Data Platform
 
Apache Apex: Stream Processing Architecture and Applications
Apache Apex: Stream Processing Architecture and Applications Apache Apex: Stream Processing Architecture and Applications
Apache Apex: Stream Processing Architecture and Applications
 
Ein Prozess lernt laufen: LEGO Mindstorms Steuerung mit BPMN
Ein Prozess lernt laufen: LEGO Mindstorms Steuerung mit BPMNEin Prozess lernt laufen: LEGO Mindstorms Steuerung mit BPMN
Ein Prozess lernt laufen: LEGO Mindstorms Steuerung mit BPMN
 
21.04.2016 Meetup: Spark vs. Flink
21.04.2016 Meetup: Spark vs. Flink21.04.2016 Meetup: Spark vs. Flink
21.04.2016 Meetup: Spark vs. Flink
 
Spark RDD-DF-SQL-DS-Spark Hadoop User Group Munich Meetup 2016
Spark RDD-DF-SQL-DS-Spark Hadoop User Group Munich Meetup 2016Spark RDD-DF-SQL-DS-Spark Hadoop User Group Munich Meetup 2016
Spark RDD-DF-SQL-DS-Spark Hadoop User Group Munich Meetup 2016
 
Machinelearning Spark Hadoop User Group Munich Meetup 2016
Machinelearning Spark Hadoop User Group Munich Meetup 2016Machinelearning Spark Hadoop User Group Munich Meetup 2016
Machinelearning Spark Hadoop User Group Munich Meetup 2016
 
Writing Java EE microservices using WildFly Swarm
Writing Java EE microservices using WildFly SwarmWriting Java EE microservices using WildFly Swarm
Writing Java EE microservices using WildFly Swarm
 

Kürzlich hochgeladen

IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 

Kürzlich hochgeladen (20)

IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 

Replacing Oracle with MongoDB for a templating application at the Bavarian government

  • 1. MongoDB@Bayern Migration from RDBMS, Problems, Unit-Testing
  • 2. Christian Brensing Bayerisches Landesamt für Statistik und Datenverarbeitung Rechenzentrum Süd
  • 3. • Template-Processor (~ JSP) • ODF and RTF • PDF / PostScript postprocessing • Groovy, Python, Ruby, Tcl • In Production using RDBMS since 2008 • Central HR system (SAP) uses BayText to create output
  • 4. Client for developing the templates (BayText-IDE)
  • 9. a small system after all
  • 12. developers.pop() Smaller teams
  • 13. Easy to learn Read one book and you're done
  • 15. permissions templates documents codes groups document_usages folder_configurations requests folders nodes structures template_tags dynafields components roles_roles roles bundles scripts
  • 17. {! "_id" : ObjectId(),! "_type" : "Document",! "updated_at" : ISODate("2007-12-06T10:04:00.543Z"),! "updated_by" : "maggie",! "version" : 1,! "path" : "foo.bar.Document",! "description" : "",! "language" : "ruby",! "format" : "odf",! "autoload_uplevel" : -1,! "requests" : [! {! "name" : "Test",! "description" : "foo", ! "xml" : zlib("<?xml ...?>")! }, ! ... ! ],! "structure" : [! { ! "alias" : "some alias",! "component_path" : "foo.bar.Document$Document",! "parent", -1 ! },! ... ! ], ! } ! "tags" : ["foo", "bar", "baz"]! }
  • 18. public class RoleReadConverter implements Converter<DBObject, Role> {! @Override! public Role convert(DBObject source) {! Role role = new Role();! role.setName((String) source.get("name"));! ...! return role;! }! }! ! ! public class RoleWriteConverter implements Converter<Role, DBObject> {! @Override! public DBObject convert(Role source) {! return new BasicDBObject()! .append("name", role.getName())! .append(...);! }! }! ! ! // Convert a DBObject to an Entity! conversionService.convert(collection.findOne(...)); Easy mapping using Spring-ConversionService
  • 19. Performance up to 10x
  • 20. ORM no more The MongoDB driver is all you need
  • 23. Referential Integrity Multi-Document-Update
  • 24. {! "_id" : ObjectId(),! "_type" : "Folder", ! "updated_at" : ISODate("2007-12-06T10:04:00.543Z"),! "updated_by" : "maggie",! "version" : 1,! "path" : "foo.bar.Folder",! "description" : "bla",! "bundles" : [! ... ! ]! "document_usages" : [! {! "name" : "foo",! "description" : "bla", ! "exec_order" : 5, ! "print_copies" : 2,! "type": "FAIR_COPY",! "document_path" : "a.b.Document",! "bundle" : "bar"! },! ...! } Linking documents via a path instead of ObjectId
  • 25. But paths are mutable! All references must be updated
  • 26. Approach • Pseudo-Transaction with an update locking mechanism in a custom oplog collection • Periodical repair jobs finishing cancelled operations • Extended interpretation of eventual consistency
  • 29. Isolated Tests are using a unique DB per JVM
  • 30. @Configuration! @Profile("test")! public class TestSpringConfiguration extends SpringConfiguration {! @Override! public void init() {! setTestProperties();! ! super.init();! ! // Test-DB löschen! db().dropDatabase();! }! ! private void setTestProperties() {! String dbname = env.getProperty("db.name", "baytext_test_" + getSimpleUsername());! String host = env.getProperty("db.host", "mongodb-dev.db.rz-sued.bybn.de");! String port = env.getProperty("db.port", "27016");! ! cmProperties.setProperty("servers", String.format("%s:%s", host, port));! cmProperties.setProperty("name", dbname);! }! ! // OS-Username without prefix (e.g. maggie instead of lfstad-maggie)! private static String getSimpleUsername() {! String username = SystemUtils.USER_NAME;! int indexOfDash = username.indexOf('-');! return indexOfDash != -1 ? username.substring(indexOfDash + 1) : username;! }! } Test-ApplicationContext
  • 31. Repeatable Collections are purged before each test method
  • 32. public class MongoTestExcecutionListener extends AbstractTestExecutionListener {! @Override! public void beforeTestMethod(TestContext testContext) throws Exception {! purgeCollections();! }! ! private void purgeCollections() {! DB db = MongoDBHolder.getDB();! for (String collectionName : db.getCollectionNames()) {! if (collectionName.startsWith("fs.") || collectionName.startsWith("system.")) {! continue;! }! DBCollection collection = db.getCollection(collectionName);! if (!collection.isCapped()) {! collection.remove(new BasicDBObject());! }! }! }! } Spring-TestExecutionListener
  • 33. @RunWith(SpringJUnit4ClassRunner.class)! @ActiveProfiles({"test"})! @ContextConfiguration(classes = TestSpringConfiguration.class)! @TestExecutionListeners({! DependencyInjectionTestExecutionListener.class,! DirtiesContextTestExecutionListener.class,! MongoTestExcecutionListener.class! })! public abstract class MongoTestSupport {! }! ! ! ! public class DocumentRepositoryTest extends MongoTestSupport {! }! Test base class
  • 35. Summary • Developing the data layer is fun again • Reduced complexity • Flat learning curve compared to SQL/ORM • use_mongo() unless transactions_required