SlideShare ist ein Scribd-Unternehmen logo
Migrating legacy data
A way to migrate data into your rails app
About me
Patrick HĂŒsler
Freelance developer @ http://www.huesler-
informatik.ch
Apple enthusiast
SurïŹng and Kung Fu
Legacy...
Us...
Considerations
 RDBMS

 Keys

 Philosophy

 Structure

 Naming
Tasks
 Connect?

 Retrieve?

 Integrate?

 Map?

 Validate?
Connect
database adapter (hopefully)
add it to database.yml
set_table_name
set_primary_key
set_sequence_name
deïŹne associations
Retrieve
 Create corresponding AR
 models

 Tell AR to connect to the
 legacy database

 Use AR’s features to adjust
 things like table name,
 primary key etc

 Set up associations
1 class Legacy::User < ActiveRecord::Base
2
3   establish_connection("legacydb")
4   set_table_name :users
5
6   belongs_to :login, :class_name => "Legacy::Login"
7 end
1   class Legacy::Base < ActiveRecord::Base
 2     self.abstract_class = true
 3     establish_connection("legacydb")
 4   end
 5
 6   class Legacy::User < Legacy::Base
 7     belongs_to :login, :class_name => "Legacy::Login"
 8   end
 9
10
Integrate
 Put models in their own
 namespace to avoid collisions

 Put them in their own folder
1   class Legacy::Base < ActiveRecord::Base
 2     self.abstract_class = true
 3     establish_connection("legacydb")
 4   end
 5
 6   class Legacy::User < Legacy::Base
 7     belongs_to :login, :class_name => "Legacy::Login"
 8   end
 9
10
Map
Let the models handle the mapping
1 class Legacy::Country < Legacy::Base
2
3   has_many :regions, :class_name => "Legacy::Region"
4
5   def code
6     abbrev
7   end
8 end
Validate
 Just use Active Record

 Create ïŹnders and scopes
 to only migrate the data
 you want

 Use memoization to
 speed up retrieval (e.g.
 fetch only valid records)
1 class Legacy::User < Legacy::Base
2   belongs_to :login, :class_name => "Legacy::Login"
3
4   named_scope :active     , :conditions => ["status = 1"]
5
6   validates_uniqueness_of :email, :scope => :status
7   validates_presence_of :first_name
8   validates_associated :login
9 end
Migrate
 Use migrators/importers
1   # General Importer
 2   # Code removed for the sake of this example
 3   class Migration::Importer
 4     def initialize(_entity_to_migrate,_migrated_entity)
 5       @entity_to_migrate = _entity_to_migrate
 6       @migrated_entity = _migrated_entity
 7     end
 8
 9     def migrate
10       @migrated_entity.attributes.symbolize_keys.keys.each do |attr|
11         if @entity_to_migrate.respond_to?(attr)
12           @migrated_entity.send("#{attr}=",@entity_to_migrate.send(attr))
13         end
14       end
15       @migrated_entity
16     end
17   end
1   # General Importer
 2   # Lines removed for the sake of this example
 3   class Migration::Importer
 4     def self.migrate_all(source_class,entity_name = nil)
 5       migrate_all_from_collection(source_class.send(:all),entity_name)
 6     end
 7
 8     def self.migrate_all_from_collection(collection,entity_name = nil)
 9       entity_name ||= collection.first.class.to_s.pluralize
10       collection.each do |entity|
11         result = self.new(entity).migrate
12         result.save!
13       end
14     end
15   end
1 class Migration::UserImporter < Migration::Importer
2   def initialize(user)
3     super(user,User.new)
4   end
5
6   def self.migrate_all
7     migrate_all_from_collection(Legacy::User.valid)
8   end
9 end
Test
 Integrates seamlessly with
 your rails test suit

 A separate folder helps to
 organize it
1 require File.dirname(__FILE__) + '/../../spec_helper'
 2
 3 describe Migration::UserImporter do
 4
 5   before(:all) do
 6     # setup your test here
 7   end
 8
 9   it "should map email address" do
10    @migrated_user.email.should equal(@user_to_migrate.email)
11   end
12
13   it "should map login to old login" do
14     @migrated_user.old_login.should == @user_to_migrate.login.login
15   end
16
17   it "should not have an empty salt" do
18     @migrated_user.salt.should_not be_blank
19   end
20
21 end
Automate
Rake to the rescue

Let it deal with
dependencies
1 namespace :legacy do
 2   namespace :data do
 3     desc "import data from legacy schema"
 4     task :import => :environment do
 5       Rake::Task['legacy:data:import:users'].invoke
 6     end
 7     namespace :import do
 8       task :fields_of_study do
 9         puts "Migrating Fields of study"
10         Migration::FieldOfStudyImporter.migrate_all
11         puts "Done migrating fields of study"
12       end
13
14       task :universities do
15         puts "Migrating universities"
16         Migration::UniversityImporter.migrate_all
17         puts "Done migrating universities"
18       end
19
20       task :users => [:fields_of_study,:universities] do
21         puts "Migrating users"
22         Migration::UserImporter.migrate_all
23         puts "Done migrating users"
24       end
25     end
26   end
27 end
Deploy
Just put in deploy.rb and let
capistrano handle the rest
1 namespace :deploy do
 2   desc "Migrate data"
 3   task :migrate_legacy_data, :roles => :app do
 4     # Actual one liner that would not fit
 5     # on a presentation screen
 6     command = "cd #{deploy_to}/current;"
 7     command << "RAILS_ENV='production' "
 8     command << "/usr/bin/rake legacy:data:import"
 9     run command
10   end
11 end
Questions?
Contact info
 Available for hire
 http://www.huesler-informatik.ch
 patrick.huesler@gmail.com
 https://www.xing.com/proïŹle/Patrick_Huesler
 http://twitter.com/phuesler
 http://github.com/phuesler

Weitere Àhnliche Inhalte

Was ist angesagt?

Lecture6 display data by okello erick
Lecture6 display data by okello erickLecture6 display data by okello erick
Lecture6 display data by okello erick
okelloerick
 
Lu solr32 34-20110912
Lu solr32 34-20110912Lu solr32 34-20110912
Lu solr32 34-20110912
Koji Sekiguchi
 
React.js or why DOM finally makes sense
React.js or why DOM finally makes senseReact.js or why DOM finally makes sense
React.js or why DOM finally makes sense
Eldar Djafarov
 
Rails 3 overview
Rails 3 overviewRails 3 overview
Rails 3 overview
Yehuda Katz
 
àžšàž—àž—àž”àčˆ 5 àžàžČàžŁàč€àž‚àž”àžąàž™àč‚àž›àžŁàčàžàžŁàžĄàč€àžžàž·àčˆàž­àž„àč‰àž™àž«àžČàčàž„àž°àčàžȘàž”àž‡àž‚àč‰àž­àžĄàžčàž„àžˆàžČàžàžàžČàž™àž‚àč‰àž­àžĄàžčàž„
àžšàž—àž—àž”àčˆ 5  àžàžČàžŁàč€àž‚àž”àžąàž™àč‚àž›àžŁàčàžàžŁàžĄàč€àžžàž·àčˆàž­àž„àč‰àž™àž«àžČàčàž„àž°àčàžȘàž”àž‡àž‚àč‰àž­àžĄàžčàž„àžˆàžČàžàžàžČàž™àž‚àč‰àž­àžĄàžčàž„àžšàž—àž—àž”àčˆ 5  àžàžČàžŁàč€àž‚àž”àžąàž™àč‚àž›àžŁàčàžàžŁàžĄàč€àžžàž·àčˆàž­àž„àč‰àž™àž«àžČàčàž„àž°àčàžȘàž”àž‡àž‚àč‰àž­àžĄàžčàž„àžˆàžČàžàžàžČàž™àž‚àč‰àž­àžĄàžčàž„
àžšàž—àž—àž”àčˆ 5 àžàžČàžŁàč€àž‚àž”àžąàž™àč‚àž›àžŁàčàžàžŁàžĄàč€àžžàž·àčˆàž­àž„àč‰àž™àž«àžČàčàž„àž°àčàžȘàž”àž‡àž‚àč‰àž­àžĄàžčàž„àžˆàžČàžàžàžČàž™àž‚àč‰àž­àžĄàžčàž„
Priew Chakrit
 
SQLite in Adobe AIR
SQLite in Adobe AIRSQLite in Adobe AIR
SQLite in Adobe AIR
Peter Elst
 

Was ist angesagt? (18)

Swift girls20170605
Swift girls20170605Swift girls20170605
Swift girls20170605
 
Rails 3 Beautiful Code
Rails 3 Beautiful CodeRails 3 Beautiful Code
Rails 3 Beautiful Code
 
Cocoa heads testing and viewcontrollers
Cocoa heads testing and viewcontrollersCocoa heads testing and viewcontrollers
Cocoa heads testing and viewcontrollers
 
MassChangeCORPEmail
MassChangeCORPEmailMassChangeCORPEmail
MassChangeCORPEmail
 
Lecture6 display data by okello erick
Lecture6 display data by okello erickLecture6 display data by okello erick
Lecture6 display data by okello erick
 
Lu solr32 34-20110912
Lu solr32 34-20110912Lu solr32 34-20110912
Lu solr32 34-20110912
 
React.js or why DOM finally makes sense
React.js or why DOM finally makes senseReact.js or why DOM finally makes sense
React.js or why DOM finally makes sense
 
Jsp/Servlet
Jsp/ServletJsp/Servlet
Jsp/Servlet
 
Elixir flow: Building and tuning concurrent workflows
Elixir flow: Building and tuning concurrent workflowsElixir flow: Building and tuning concurrent workflows
Elixir flow: Building and tuning concurrent workflows
 
Rails 3 overview
Rails 3 overviewRails 3 overview
Rails 3 overview
 
àžšàž—àž—àž”àčˆ 5 àžàžČàžŁàč€àž‚àž”àžąàž™àč‚àž›àžŁàčàžàžŁàžĄàč€àžžàž·àčˆàž­àž„àč‰àž™àž«àžČàčàž„àž°àčàžȘàž”àž‡àž‚àč‰àž­àžĄàžčàž„àžˆàžČàžàžàžČàž™àž‚àč‰àž­àžĄàžčàž„
àžšàž—àž—àž”àčˆ 5  àžàžČàžŁàč€àž‚àž”àžąàž™àč‚àž›àžŁàčàžàžŁàžĄàč€àžžàž·àčˆàž­àž„àč‰àž™àž«àžČàčàž„àž°àčàžȘàž”àž‡àž‚àč‰àž­àžĄàžčàž„àžˆàžČàžàžàžČàž™àž‚àč‰àž­àžĄàžčàž„àžšàž—àž—àž”àčˆ 5  àžàžČàžŁàč€àž‚àž”àžąàž™àč‚àž›àžŁàčàžàžŁàžĄàč€àžžàž·àčˆàž­àž„àč‰àž™àž«àžČàčàž„àž°àčàžȘàž”àž‡àž‚àč‰àž­àžĄàžčàž„àžˆàžČàžàžàžČàž™àž‚àč‰àž­àžĄàžčàž„
àžšàž—àž—àž”àčˆ 5 àžàžČàžŁàč€àž‚àž”àžąàž™àč‚àž›àžŁàčàžàžŁàžĄàč€àžžàž·àčˆàž­àž„àč‰àž™àž«àžČàčàž„àž°àčàžȘàž”àž‡àž‚àč‰àž­àžĄàžčàž„àžˆàžČàžàžàžČàž™àž‚àč‰àž­àžĄàžčàž„
 
csmju313 L5
csmju313 L5csmju313 L5
csmju313 L5
 
Practical JavaScript Programming - Session 8/8
Practical JavaScript Programming - Session 8/8Practical JavaScript Programming - Session 8/8
Practical JavaScript Programming - Session 8/8
 
Flask - Backend com Python - Semcomp 18
Flask - Backend com Python - Semcomp 18Flask - Backend com Python - Semcomp 18
Flask - Backend com Python - Semcomp 18
 
SQLite in Adobe AIR
SQLite in Adobe AIRSQLite in Adobe AIR
SQLite in Adobe AIR
 
Stored Procedure
Stored ProcedureStored Procedure
Stored Procedure
 
Everything You (N)ever Wanted to Know about Testing View Controllers
Everything You (N)ever Wanted to Know about Testing View ControllersEverything You (N)ever Wanted to Know about Testing View Controllers
Everything You (N)ever Wanted to Know about Testing View Controllers
 
Angular.js Fundamentals
Angular.js FundamentalsAngular.js Fundamentals
Angular.js Fundamentals
 

Andere mochten auch

Ocean rescue power point presentation
Ocean rescue power point presentationOcean rescue power point presentation
Ocean rescue power point presentation
city of dania beach
 
Vooruitblik petrol 2011
Vooruitblik petrol 2011Vooruitblik petrol 2011
Vooruitblik petrol 2011
Rien De Koning
 
Press Release 2 Q01 Tele Celular Sul En
Press Release 2 Q01   Tele Celular Sul EnPress Release 2 Q01   Tele Celular Sul En
Press Release 2 Q01 Tele Celular Sul En
TIM RI
 
Er Verbs in french
Er Verbs in frenchEr Verbs in french
Er Verbs in french
potasz
 
130710 aceh
130710 aceh130710 aceh
130710 aceh
epaper
 
Tips Memasukkan Data Pelajar Ke System Smm V422
Tips Memasukkan Data Pelajar Ke System Smm V422Tips Memasukkan Data Pelajar Ke System Smm V422
Tips Memasukkan Data Pelajar Ke System Smm V422
zafeen zafeen
 
Friendship
FriendshipFriendship
Friendship
smartway01
 
Chapter 1
Chapter 1Chapter 1
Chapter 1
rleegarner
 
Press Release 4 T07 En
Press Release 4 T07 EnPress Release 4 T07 En
Press Release 4 T07 En
TIM RI
 
4juni nas
4juni nas4juni nas
4juni nas
epaper
 
The power of social media uwo
The power of social media uwoThe power of social media uwo
The power of social media uwo
Thomas Clifford
 
Vision Cast (Short)
Vision Cast (Short)Vision Cast (Short)
Vision Cast (Short)
guest90848c3
 
Edisi8nov
Edisi8novEdisi8nov
Edisi8nov
epaper
 

Andere mochten auch (20)

Ocean rescue power point presentation
Ocean rescue power point presentationOcean rescue power point presentation
Ocean rescue power point presentation
 
Vooruitblik petrol 2011
Vooruitblik petrol 2011Vooruitblik petrol 2011
Vooruitblik petrol 2011
 
Prototyping Design Pack
Prototyping Design PackPrototyping Design Pack
Prototyping Design Pack
 
Press Release 2 Q01 Tele Celular Sul En
Press Release 2 Q01   Tele Celular Sul EnPress Release 2 Q01   Tele Celular Sul En
Press Release 2 Q01 Tele Celular Sul En
 
Apresentação Institucional 2T16
Apresentação Institucional 2T16Apresentação Institucional 2T16
Apresentação Institucional 2T16
 
Er Verbs in french
Er Verbs in frenchEr Verbs in french
Er Verbs in french
 
Rodeway
RodewayRodeway
Rodeway
 
130710 aceh
130710 aceh130710 aceh
130710 aceh
 
Tips Memasukkan Data Pelajar Ke System Smm V422
Tips Memasukkan Data Pelajar Ke System Smm V422Tips Memasukkan Data Pelajar Ke System Smm V422
Tips Memasukkan Data Pelajar Ke System Smm V422
 
Friendship
FriendshipFriendship
Friendship
 
Chapter 1
Chapter 1Chapter 1
Chapter 1
 
Panduan Membuat Web Quiz
Panduan Membuat Web QuizPanduan Membuat Web Quiz
Panduan Membuat Web Quiz
 
Press Release 4 T07 En
Press Release 4 T07 EnPress Release 4 T07 En
Press Release 4 T07 En
 
Children and Youth in the Arts
Children and Youth in the ArtsChildren and Youth in the Arts
Children and Youth in the Arts
 
4juni nas
4juni nas4juni nas
4juni nas
 
Social media 101 Appleton North High School
Social media 101 Appleton North High SchoolSocial media 101 Appleton North High School
Social media 101 Appleton North High School
 
The power of social media uwo
The power of social media uwoThe power of social media uwo
The power of social media uwo
 
Ocean lifeguard expo 2010
Ocean lifeguard expo 2010Ocean lifeguard expo 2010
Ocean lifeguard expo 2010
 
Vision Cast (Short)
Vision Cast (Short)Vision Cast (Short)
Vision Cast (Short)
 
Edisi8nov
Edisi8novEdisi8nov
Edisi8nov
 

Ähnlich wie Migrating legacy data

Data herding
Data herdingData herding
Data herding
unbracketed
 
How to disassemble one monster app into an ecosystem of 30
How to disassemble one monster app into an ecosystem of 30How to disassemble one monster app into an ecosystem of 30
How to disassemble one monster app into an ecosystem of 30
fiyuer
 
Rails 3: Dashing to the Finish
Rails 3: Dashing to the FinishRails 3: Dashing to the Finish
Rails 3: Dashing to the Finish
Yehuda Katz
 
Ruby: OOP, metaprogramming, blocks, iterators, mix-ins, duck typing. Code style
Ruby: OOP, metaprogramming, blocks, iterators, mix-ins, duck typing. Code styleRuby: OOP, metaprogramming, blocks, iterators, mix-ins, duck typing. Code style
Ruby: OOP, metaprogramming, blocks, iterators, mix-ins, duck typing. Code style
Anton Shemerey
 
Desarrollando aplicaciones web en minutos
Desarrollando aplicaciones web en minutosDesarrollando aplicaciones web en minutos
Desarrollando aplicaciones web en minutos
Edgar Suarez
 

Ähnlich wie Migrating legacy data (20)

Ruby/Rails
Ruby/RailsRuby/Rails
Ruby/Rails
 
Play vs Rails
Play vs RailsPlay vs Rails
Play vs Rails
 
Why ruby
Why rubyWhy ruby
Why ruby
 
Beyond MVC
Beyond MVCBeyond MVC
Beyond MVC
 
SharePoint Conference 2018 - APIs, APIs everywhere!
SharePoint Conference 2018 - APIs, APIs everywhere!SharePoint Conference 2018 - APIs, APIs everywhere!
SharePoint Conference 2018 - APIs, APIs everywhere!
 
[FT-7][snowmantw] How to make a new functional language and make the world be...
[FT-7][snowmantw] How to make a new functional language and make the world be...[FT-7][snowmantw] How to make a new functional language and make the world be...
[FT-7][snowmantw] How to make a new functional language and make the world be...
 
Data herding
Data herdingData herding
Data herding
 
Data herding
Data herdingData herding
Data herding
 
How to disassemble one monster app into an ecosystem of 30
How to disassemble one monster app into an ecosystem of 30How to disassemble one monster app into an ecosystem of 30
How to disassemble one monster app into an ecosystem of 30
 
Innovative Specifications for Better Performance Logging and Monitoring
Innovative Specifications for Better Performance Logging and MonitoringInnovative Specifications for Better Performance Logging and Monitoring
Innovative Specifications for Better Performance Logging and Monitoring
 
DataMapper
DataMapperDataMapper
DataMapper
 
Rails 3: Dashing to the Finish
Rails 3: Dashing to the FinishRails 3: Dashing to the Finish
Rails 3: Dashing to the Finish
 
Burn down the silos! Helping dev and ops gel on high availability websites
Burn down the silos! Helping dev and ops gel on high availability websitesBurn down the silos! Helping dev and ops gel on high availability websites
Burn down the silos! Helping dev and ops gel on high availability websites
 
Having Fun with Play
Having Fun with PlayHaving Fun with Play
Having Fun with Play
 
All I Need to Know I Learned by Writing My Own Web Framework
All I Need to Know I Learned by Writing My Own Web FrameworkAll I Need to Know I Learned by Writing My Own Web Framework
All I Need to Know I Learned by Writing My Own Web Framework
 
Ruby: OOP, metaprogramming, blocks, iterators, mix-ins, duck typing. Code style
Ruby: OOP, metaprogramming, blocks, iterators, mix-ins, duck typing. Code styleRuby: OOP, metaprogramming, blocks, iterators, mix-ins, duck typing. Code style
Ruby: OOP, metaprogramming, blocks, iterators, mix-ins, duck typing. Code style
 
Desarrollando aplicaciones web en minutos
Desarrollando aplicaciones web en minutosDesarrollando aplicaciones web en minutos
Desarrollando aplicaciones web en minutos
 
SharePoint Saturday Belgium 2018 - APIs, APIs everywhere!
SharePoint Saturday Belgium 2018 - APIs, APIs everywhere!SharePoint Saturday Belgium 2018 - APIs, APIs everywhere!
SharePoint Saturday Belgium 2018 - APIs, APIs everywhere!
 
APIs, APIs Everywhere!
APIs, APIs Everywhere!APIs, APIs Everywhere!
APIs, APIs Everywhere!
 
Developing Applications with MySQL and Java for beginners
Developing Applications with MySQL and Java for beginnersDeveloping Applications with MySQL and Java for beginners
Developing Applications with MySQL and Java for beginners
 

Mehr von Patrick Huesler

Mehr von Patrick Huesler (9)

Technical Challenges of Developing a Facebook Game
Technical Challenges of Developing a Facebook GameTechnical Challenges of Developing a Facebook Game
Technical Challenges of Developing a Facebook Game
 
Culerity and Headless Full Stack Integration Testing
Culerity and Headless Full Stack Integration TestingCulerity and Headless Full Stack Integration Testing
Culerity and Headless Full Stack Integration Testing
 
Client Side Optimization
Client Side OptimizationClient Side Optimization
Client Side Optimization
 
Building and deploying Cocoa applications with ChocTop
Building and deploying Cocoa applications with ChocTopBuilding and deploying Cocoa applications with ChocTop
Building and deploying Cocoa applications with ChocTop
 
Choctop Lightning talk EuRuKo 2010
Choctop Lightning talk EuRuKo 2010Choctop Lightning talk EuRuKo 2010
Choctop Lightning talk EuRuKo 2010
 
Small Cocoa Apps with MacRuby
Small Cocoa Apps with MacRubySmall Cocoa Apps with MacRuby
Small Cocoa Apps with MacRuby
 
Fun with Ruby and Cocoa
Fun with Ruby and CocoaFun with Ruby and Cocoa
Fun with Ruby and Cocoa
 
Erlang, an overview
Erlang, an overviewErlang, an overview
Erlang, an overview
 
Active Record No No's
Active Record No No'sActive Record No No's
Active Record No No's
 

KĂŒrzlich hochgeladen

KĂŒrzlich hochgeladen (20)

To Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMsTo Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
 
What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024
 
Exploring UiPath Orchestrator API: updates and limits in 2024 🚀
Exploring UiPath Orchestrator API: updates and limits in 2024 🚀Exploring UiPath Orchestrator API: updates and limits in 2024 🚀
Exploring UiPath Orchestrator API: updates and limits in 2024 🚀
 
10 Differences between Sales Cloud and CPQ, Blanka DoktorovĂĄ
10 Differences between Sales Cloud and CPQ, Blanka DoktorovĂĄ10 Differences between Sales Cloud and CPQ, Blanka DoktorovĂĄ
10 Differences between Sales Cloud and CPQ, Blanka DoktorovĂĄ
 
Transforming The New York Times: Empowering Evolution through UX
Transforming The New York Times: Empowering Evolution through UXTransforming The New York Times: Empowering Evolution through UX
Transforming The New York Times: Empowering Evolution through UX
 
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
 
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptxWSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
 
Connecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAKConnecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAK
 
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
 
Enterprise Security Monitoring, And Log Management.
Enterprise Security Monitoring, And Log Management.Enterprise Security Monitoring, And Log Management.
Enterprise Security Monitoring, And Log Management.
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
 
AI revolution and Salesforce, Jiƙí Karpíơek
AI revolution and Salesforce, Jiƙí KarpíơekAI revolution and Salesforce, Jiƙí Karpíơek
AI revolution and Salesforce, Jiƙí Karpíơek
 
Designing for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at ComcastDesigning for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at Comcast
 
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
 
Strategic AI Integration in Engineering Teams
Strategic AI Integration in Engineering TeamsStrategic AI Integration in Engineering Teams
Strategic AI Integration in Engineering Teams
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. Startups
 
ODC, Data Fabric and Architecture User Group
ODC, Data Fabric and Architecture User GroupODC, Data Fabric and Architecture User Group
ODC, Data Fabric and Architecture User Group
 
UiPath Test Automation using UiPath Test Suite series, part 2
UiPath Test Automation using UiPath Test Suite series, part 2UiPath Test Automation using UiPath Test Suite series, part 2
UiPath Test Automation using UiPath Test Suite series, part 2
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
 
Optimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through ObservabilityOptimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through Observability
 

Migrating legacy data

  • 1. Migrating legacy data A way to migrate data into your rails app
  • 2. About me Patrick HĂŒsler Freelance developer @ http://www.huesler- informatik.ch Apple enthusiast SurïŹng and Kung Fu
  • 5. Considerations RDBMS Keys Philosophy Structure Naming
  • 6. Tasks Connect? Retrieve? Integrate? Map? Validate?
  • 7. Connect database adapter (hopefully) add it to database.yml set_table_name set_primary_key set_sequence_name deïŹne associations
  • 8. Retrieve Create corresponding AR models Tell AR to connect to the legacy database Use AR’s features to adjust things like table name, primary key etc Set up associations
  • 9. 1 class Legacy::User < ActiveRecord::Base 2 3 establish_connection("legacydb") 4 set_table_name :users 5 6 belongs_to :login, :class_name => "Legacy::Login" 7 end
  • 10. 1 class Legacy::Base < ActiveRecord::Base 2 self.abstract_class = true 3 establish_connection("legacydb") 4 end 5 6 class Legacy::User < Legacy::Base 7 belongs_to :login, :class_name => "Legacy::Login" 8 end 9 10
  • 11. Integrate Put models in their own namespace to avoid collisions Put them in their own folder
  • 12. 1 class Legacy::Base < ActiveRecord::Base 2 self.abstract_class = true 3 establish_connection("legacydb") 4 end 5 6 class Legacy::User < Legacy::Base 7 belongs_to :login, :class_name => "Legacy::Login" 8 end 9 10
  • 13. Map Let the models handle the mapping
  • 14. 1 class Legacy::Country < Legacy::Base 2 3 has_many :regions, :class_name => "Legacy::Region" 4 5 def code 6 abbrev 7 end 8 end
  • 15. Validate Just use Active Record Create ïŹnders and scopes to only migrate the data you want Use memoization to speed up retrieval (e.g. fetch only valid records)
  • 16. 1 class Legacy::User < Legacy::Base 2 belongs_to :login, :class_name => "Legacy::Login" 3 4 named_scope :active , :conditions => ["status = 1"] 5 6 validates_uniqueness_of :email, :scope => :status 7 validates_presence_of :first_name 8 validates_associated :login 9 end
  • 18. 1 # General Importer 2 # Code removed for the sake of this example 3 class Migration::Importer 4 def initialize(_entity_to_migrate,_migrated_entity) 5 @entity_to_migrate = _entity_to_migrate 6 @migrated_entity = _migrated_entity 7 end 8 9 def migrate 10 @migrated_entity.attributes.symbolize_keys.keys.each do |attr| 11 if @entity_to_migrate.respond_to?(attr) 12 @migrated_entity.send("#{attr}=",@entity_to_migrate.send(attr)) 13 end 14 end 15 @migrated_entity 16 end 17 end
  • 19. 1 # General Importer 2 # Lines removed for the sake of this example 3 class Migration::Importer 4 def self.migrate_all(source_class,entity_name = nil) 5 migrate_all_from_collection(source_class.send(:all),entity_name) 6 end 7 8 def self.migrate_all_from_collection(collection,entity_name = nil) 9 entity_name ||= collection.first.class.to_s.pluralize 10 collection.each do |entity| 11 result = self.new(entity).migrate 12 result.save! 13 end 14 end 15 end
  • 20. 1 class Migration::UserImporter < Migration::Importer 2 def initialize(user) 3 super(user,User.new) 4 end 5 6 def self.migrate_all 7 migrate_all_from_collection(Legacy::User.valid) 8 end 9 end
  • 21. Test Integrates seamlessly with your rails test suit A separate folder helps to organize it
  • 22. 1 require File.dirname(__FILE__) + '/../../spec_helper' 2 3 describe Migration::UserImporter do 4 5 before(:all) do 6 # setup your test here 7 end 8 9 it "should map email address" do 10 @migrated_user.email.should equal(@user_to_migrate.email) 11 end 12 13 it "should map login to old login" do 14 @migrated_user.old_login.should == @user_to_migrate.login.login 15 end 16 17 it "should not have an empty salt" do 18 @migrated_user.salt.should_not be_blank 19 end 20 21 end
  • 23. Automate Rake to the rescue Let it deal with dependencies
  • 24. 1 namespace :legacy do 2 namespace :data do 3 desc "import data from legacy schema" 4 task :import => :environment do 5 Rake::Task['legacy:data:import:users'].invoke 6 end 7 namespace :import do 8 task :fields_of_study do 9 puts "Migrating Fields of study" 10 Migration::FieldOfStudyImporter.migrate_all 11 puts "Done migrating fields of study" 12 end 13 14 task :universities do 15 puts "Migrating universities" 16 Migration::UniversityImporter.migrate_all 17 puts "Done migrating universities" 18 end 19 20 task :users => [:fields_of_study,:universities] do 21 puts "Migrating users" 22 Migration::UserImporter.migrate_all 23 puts "Done migrating users" 24 end 25 end 26 end 27 end
  • 25. Deploy Just put in deploy.rb and let capistrano handle the rest
  • 26. 1 namespace :deploy do 2 desc "Migrate data" 3 task :migrate_legacy_data, :roles => :app do 4 # Actual one liner that would not fit 5 # on a presentation screen 6 command = "cd #{deploy_to}/current;" 7 command << "RAILS_ENV='production' " 8 command << "/usr/bin/rake legacy:data:import" 9 run command 10 end 11 end
  • 28. Contact info Available for hire http://www.huesler-informatik.ch patrick.huesler@gmail.com https://www.xing.com/proïŹle/Patrick_Huesler http://twitter.com/phuesler http://github.com/phuesler