• Gefällt mir
Replacing Oracle with MongoDB for a templating application at the Bavarian government
Nächste SlideShare
Wird geladen in ...5
×

Replacing Oracle with MongoDB for a templating application at the Bavarian government

  • 1,154 Views
Hochgeladen am

Bavarian government runs a document template application (RTF or ODF with Groovy, Python, Ruby or Tcl as scripting language) serving different government offices. Having complex and hierarchical data ...

Bavarian government runs a document template application (RTF or ODF with Groovy, Python, Ruby or Tcl as scripting language) serving different government offices. Having complex and hierarchical data structures to organize the templates, MongoDB was selected to replace the Oracle-based persistence layer. This presentation is about the improvements they have achieved with the migration to MongoDB, problems they had to solve underway and unit testing of the persistence layer in order to keep their quality level. Presentation slides by Christian Brensing, Senior Developer at Rechenzentrum Süd, shown at Munich MongoDB User Group Meetup on 18th September 2012

Mehr in: Technologie , Business
  • Full Name Full Name Comment goes here.
    Sind Sie sicher, dass Sie...
    Ihre Nachricht erscheint hier
    Hinterlassen Sie den ersten Kommentar
Keine Downloads

Views

Gesamtviews
1,154
Bei Slideshare
0
Aus Einbettungen
0
Anzahl an Einbettungen
3

Aktionen

Geteilt
Downloads
6
Kommentare
0
Gefällt mir
2

Einbettungen 0

No embeds

Inhalte melden

Als unangemessen gemeldet Als unangemessen melden
Als unangemessen melden

Wählen Sie Ihren Grund, warum Sie diese Präsentation als unangemessen melden.

Löschen
    No notes for slide

Transcript

  • 1. MongoDB@BayernMigration from RDBMS, Problems, Unit-Testing
  • 2. Christian BrensingBayerisches Landesamt fürStatistik und DatenverarbeitungRechenzentrum Süd
  • 3. • Template-Processor (~ JSP)• ODF and RTF• PDF / PostScript postprocessing• Groovy, Python, Ruby, Tcl• In Production using RDBMS since 2008• Central HR system (SAP) uses BayText to create output
  • 4. Client for developing the templates (BayText-IDE)
  • 5. ODF-Template using Ruby
  • 6. Generated document
  • 7. 2 MillionGenerated documents per year
  • 8. Tiny dataset50,000 records and 10 GB BLOB
  • 9. a small system after all
  • 10. So why migrate?
  • 11. Because we can
  • 12. developers.pop() Smaller teams
  • 13. Easy to learnRead one book and youre done
  • 14. Simple mapping
  • 15. permissions templates documentscodes groups document_usagesfolder_configurations requests foldersnodes structures template_tagsdynafields components roles_rolesroles bundles scripts
  • 16. nodes rolesproperties GridFS
  • 17. {! "_id" : ObjectId(),! "_type" : "Document",! "updated_at" : ISODate("2007-12-06T10:04:00.543Z"),! "updated_by" : "maggie",! "version" : 1,! "path" : "foo.bar.Document",! "description" : "",! "language" : "ruby",! "format" : "odf",! "autoload_uplevel" : -1,! "requests" : [! {! "name" : "Test",! "description" : "foo", ! "xml" : zlib("<?xml ...?>")! }, ! ... ! ],! "structure" : [! { ! "alias" : "some alias",! "component_path" : "foo.bar.Document$Document",! "parent", -1 ! },! ... ! ], ! } ! "tags" : ["foo", "bar", "baz"]!}
  • 18. public class RoleReadConverter implements Converter<DBObject, Role> {! @Override! public Role convert(DBObject source) {! Role role = new Role();! role.setName((String) source.get("name"));! ...! return role;! }!}!!!public class RoleWriteConverter implements Converter<Role, DBObject> {! @Override! public DBObject convert(Role source) {! return new BasicDBObject()! .append("name", role.getName())! .append(...);! }!}!!!// Convert a DBObject to an Entity!conversionService.convert(collection.findOne(...)); Easy mapping using Spring-ConversionService
  • 19. Performance up to 10x
  • 20. ORM no moreThe MongoDB driver is all you need
  • 21. Obstacles?
  • 22. Mentality
  • 23. Referential Integrity Multi-Document-Update
  • 24. {! "_id" : ObjectId(),! "_type" : "Folder", ! "updated_at" : ISODate("2007-12-06T10:04:00.543Z"),! "updated_by" : "maggie",! "version" : 1,! "path" : "foo.bar.Folder",! "description" : "bla",! "bundles" : [! ... ! ]! "document_usages" : [! {! "name" : "foo",! "description" : "bla", ! "exec_order" : 5, ! "print_copies" : 2,! "type": "FAIR_COPY",! "document_path" : "a.b.Document",! "bundle" : "bar"! },! ...!} Linking documents via a path instead of ObjectId
  • 25. But paths are mutable! All references must be updated
  • 26. Approach• Pseudo-Transaction with an update locking mechanism in a custom oplog collection• Periodical repair jobs finishing cancelled operations• Extended interpretation of eventual consistency
  • 27. Unit-TestsContinuous Integration
  • 28. F.I.R.S.T.
  • 29. IsolatedTests are using a unique DB per JVM
  • 30. @Configuration!@Profile("test")!public class TestSpringConfiguration extends SpringConfiguration {! @Override! public void init() {! setTestProperties();! ! super.init();! ! // Test-DB löschen! db().dropDatabase();! }! ! private void setTestProperties() {! String dbname = env.getProperty("db.name", "baytext_test_" + getSimpleUsername());! String host = env.getProperty("db.host", "mongodb-dev.db.rz-sued.bybn.de");! String port = env.getProperty("db.port", "27016");! ! cmProperties.setProperty("servers", String.format("%s:%s", host, port));! cmProperties.setProperty("name", dbname);! }!! // OS-Username without prefix (e.g. maggie instead of lfstad-maggie)! private static String getSimpleUsername() {! String username = SystemUtils.USER_NAME;! int indexOfDash = username.indexOf(-);! return indexOfDash != -1 ? username.substring(indexOfDash + 1) : username;! }!} Test-ApplicationContext
  • 31. RepeatableCollections are purged before each test method
  • 32. public class MongoTestExcecutionListener extends AbstractTestExecutionListener {! @Override! public void beforeTestMethod(TestContext testContext) throws Exception {! purgeCollections();! }! ! private void purgeCollections() {! DB db = MongoDBHolder.getDB();! for (String collectionName : db.getCollectionNames()) {! if (collectionName.startsWith("fs.") || collectionName.startsWith("system.")) {! continue;! }! DBCollection collection = db.getCollection(collectionName);! if (!collection.isCapped()) {! collection.remove(new BasicDBObject());! }! }! }!} Spring-TestExecutionListener
  • 33. @RunWith(SpringJUnit4ClassRunner.class)!@ActiveProfiles({"test"})!@ContextConfiguration(classes = TestSpringConfiguration.class)!@TestExecutionListeners({! DependencyInjectionTestExecutionListener.class,! DirtiesContextTestExecutionListener.class,! MongoTestExcecutionListener.class!})!public abstract class MongoTestSupport {!}!!!!public class DocumentRepositoryTest extends MongoTestSupport {!}! Test base class
  • 34. 95% Coverage
  • 35. Summary• Developing the data layer is fun again• Reduced complexity• Flat learning curve compared to SQL/ORM• use_mongo() unless transactions_required
  • 36. Thank you!