SlideShare a Scribd company logo
1 of 54
Download to read offline
Bootstrapping
                        Django with ease
                        OSCON 2012
                        www.concentricsky.com
                        @concentricsky // biglan / wiggins / bothun




Thursday, July 19, 12
Introductions




                        Mike Biglan       Wiggins       Josh Bothun
                            CTO            Architect    Senior Developer

                                         @coffindragger




Thursday, July 19, 12
Areas of Concentric Sky




                        Web/Django       Mobile      Enterprise



Thursday, July 19, 12
(Some) Technologies We Use

                        • Chef          • Backbone.js
                        • Jenkins       • Jinja2
                        • AWS           • Celery
                        • Sphinx        • DebugToolbar
                        • Memcached     • jQuery / jQueryUI
                        • South         • Compass / Sass
                        • git           • Mongo & MySQL

Thursday, July 19, 12
Released CSky Creations



                        • djenesis – Easy Django Bootstrapping
                        • breakdown - Lightweight jinja2 template
                          prototyping server

                        • django-cachemodel – Automatic caching
                          for django models




Thursday, July 19, 12
CSky Creations in the Works



                        • skycloud – Tool for AWS auto-scaling
                        • django-skycms – CMS as a library
                        • django-sky-visitor – Invitations, email
                          confirmation, etc

                        • django-client-admin – Flexible admin for
                          client



Thursday, July 19, 12
CSky Teams


                                       10   Django Team

                        STAFF

            50+
                                       5   Design Team

                                       9   PM Team

                                       3   System Admins


Thursday, July 19, 12
Team Interfacing


                        • Each project is a mix from each team
                        • Members might switch amongst projects
                        • Interfaces between teams
                         • Clean separation of handoffs and
                           responsibilities

                        • Best-practices shared within teams


Thursday, July 19, 12
“Bootstrapping”




                        • Pulling one up by one’s bootstraps
                        • Computers do this much better than humans
                        • Django – starting a project, new or existing



Thursday, July 19, 12
Bootstrapping with Ease


                        Making booting (and rebooting) a
                        django project as easy as possible


                        • Ease is a continually moving target
                        • Shouldn’t sacrifice other goals for ease (e.g.
                          security, modularity, be PHP)




Thursday, July 19, 12
Moving Targets

                        Moving targets create new
                        problems. And new solutions.

                        • CGI – Solving static HTML
                        • JavaScript – Solving non-interactivity
                        • CSS – Solving 100+ HTML pages
                        • Puppet/Chef – Solving 100+ servers

Thursday, July 19, 12
Problem/Solution Lifecycle


                        1. Problem grows

                        2. Many create their own solution; some
                           polish; some release

                        3. Community flocks to one (or a few) solutions

                        4. Solution(s) gets anointed and others die off




Thursday, July 19, 12
Goals of Easy Bootstrapping




                        1. More people using Django
                        2. More people using Django better




Thursday, July 19, 12
More People Using Django

                        • As few commands as possible to try
                          Django out

                        • And to try someone else’s project out.
                        • Must be highly likely to just work
                        • All especially critical for beginners
                        • Django’s runserver & sqlite...great!
                         • It just works...
Thursday, July 19, 12
Making Mistakes should be Hard



                    • Checklists are great
                    • The best checklists
                        are invisible

                    • To automate lists of
                        commands, must
                        have consistency




Thursday, July 19, 12
Consistency Makes things Easier



                        • Ease of replication across servers
                        • Ease of sharing
                         • Both templates and existing projects
                        • Easier to build upon


Thursday, July 19, 12
Consistently Standardized



                        • Standardization: a tool to get to
                          consistency

                        • This can be in an organization. Or the full
                          community

                        • Oh yeah, and...

Thursday, July 19, 12
Standard had Better be Right




Thursday, July 19, 12
Fragmentation




                        • Poor standards lead to fragmentation
                        • Hard to build things on top of
                         fragmented environments

                        • Imagine building for single browser. Not
                         Mosaic.



Thursday, July 19, 12
Android Fragmentation




                        http://www.businessinsider.com/android-fragmentation-chart-from-opensignalmaps-2012-5




Thursday, July 19, 12
Flexibility vs Standardization


                        • Standardization can be rigid; can kill flexibility
                         • Especially as the target moves
                         • E.g. contrib.auth
                        • Flexibility allows preference, innovation
                        • Best of both worlds is standardization with
                          structured options of flexibility

                         • Django does this well
Thursday, July 19, 12
Django


                        • Django is a set of higher-order building blocks
                        • If blocks are too large, sacrifices what can be
                          made (i.e. not flexibile)

                        • If blocks are too small, becomes inefficient
                          and unmanageable

                        • If block types are wrong, not loosely
                          coupled (i.e. non-orthogonal)




Thursday, July 19, 12
3 Stages of Project
            Structure Grief
                 • Denial
                 • Bargaining
                 • Acceptance

Thursday, July 19, 12
Denial


                          I’ll just take the
                          easy way out...

                        • Stuff required libraries in a ‘lib’ directory
                        • Stick django there too
                        • Name everything MyProjectSomething

Thursday, July 19, 12
Denial

                               I’ll just take the easy way out...


                   Why this doesn’t work
                        • project starts to get bloated
                        • version management becomes a headache
                        • upgrade paths become tangled

Thursday, July 19, 12
Bargaining


                          We’ll just adopt some
                          standard practices...
                 1. Have the team argue about which way to do things

                 2. Assign someone to write a document about it

                 3. ???

                 4. Ignore it when you start a new project


Thursday, July 19, 12
Bargaining

                           We’ll just adopt some standard practices...

                   Why this doesn’t work
                        • Projects become inconsistent
                        • It is not in source control
                        • No way to manage changes across projects
                        • There is no history of changes or discussion

Thursday, July 19, 12
Acceptance



              Embrace the community
                        • requirements.txt, version management
                        • also solves the ‘git submodule’ problem
                        • virtualenv is great, always use it


Thursday, July 19, 12
Acceptance



         No, really, use virtualenv
                        • dependency management
                        • concurrent project management
                        • sand-boxing
                        • solves the “chicken-n-egg” problem
                        • replication
Thursday, July 19, 12
What’s in a template?


                        • For a django project,   there are two styles:

                         1. toplevel directory is python code

                         2. toplevel directory is a website project




Thursday, July 19, 12
Toplevel is python code

       ./manage.py
       ./my_project/__init__.py
                                    • Pros
       ./my_project/settings.py
       ./my_project/urls.py
                                     • More pythonic; Everything
       ./my_project/wsgi.py
                                        is python code

       Django’s default template      • It just works
                                    • Cons
                                     • Everything is python code
                                     • Where do non public files
                                        go?


Thursday, July 19, 12
Default Django Project
       ./manage.py
       ./my_project/__init__.py
       ./my_project/settings.py
                                         • Where do apps go?
       ./my_project/urls.py
       ./my_project/wsgi.py              • How about ./my_project/
       ./my_project/my_app/__init__.py     my_app/?
       ./my_project/my_app/models.py

                                          • Name then becomes
                                            ‘my_project.my_app’

                                          • What about reusability?
                                          • What if I want to rename
                                            my_project?


Thursday, July 19, 12
Default Django Project
       ./manage.py
       ./my_project/__init__.py
       ./my_project/settings.py
                                     • Where do apps go?
       ./my_project/urls.py
       ./my_project/wsgi.py          • How about ./my_app/?
                                      • Better for reusability, but...
       ./my_app/__init__.py
       ./my_app/models.py


                                      • Now I have to add ./ to the
                                         PYTHON_PATH




Thursday, July 19, 12
Default Django Project
       ./manage.py
       ./myapp1/models.py
       ./myapp1/urls.py
                                    • What was the name
       ./myapp2/models.py             again?
                                    • Was it ‘myapp1’ or ‘myapp2’
       ./myapp2/urls.py
       ./myapp3/models.py
       ./myapp3/urls.py               that contains the main urls
                                      and settings...




Thursday, July 19, 12
Toplevel is website project

                                                    • Pros
            ./manage.py
            ./wsgi.py
            ./requirements.txt


                                                     • Closer to real world
            ./apps/mainsite/__init__.py
            ./apps/mainsite/settings.py
            ./apps/mainsite/settings_local.py
            ./apps/mainsite/urls.py                      environments
            ./apps/my_app/models.py


                                                       • Consistency across
            ./apps/my_app/urls.py



                                                         projects

                                                    • Cons
                                                     • ??? None I guess!
                        http://github.com/concentricsky/djenesis-template.git



Thursday, July 19, 12
Toplevel is website project
                             ./manage.py
                             ./wsgi.py
                             ./requirements.txt
                             ./apps/mainsite/__init__.py
                             ./apps/mainsite/settings.py
                             ./apps/mainsite/settings_local.py.example
                             ./apps/mainsite/urls.py
                             ./apps/my_app/models.py
                             ./apps/my_app/urls.py




                            Python code is sandboxed
                        • only have to add ./apps to PYTHON_PATH
                        • everything else is ignored by WSGI server

Thursday, July 19, 12
Toplevel is website project
                              ./manage.py
                              ./wsgi.py
                              ./requirements.txt
                              ./apps/mainsite/__init__.py
                              ./apps/mainsite/settings.py
                              ./apps/mainsite/settings_local.py.example
                              ./apps/mainsite/urls.py
                              ./apps/my_app/models.py
                              ./apps/my_app/urls.py



                          Consistent Entry Point Name
                        • I always know where to look first: ./apps/
                          mainsite/urls.py

                        • More important than it may seem at first blush
Thursday, July 19, 12
Toplevel is website project
                                  ./manage.py
                                  ./wsgi.py
                                  ./requirements.txt
                                  ./apps/mainsite/__init__.py
                                  ./apps/mainsite/settings.py
                                  ./apps/mainsite/settings_local.py.example
                                  ./apps/mainsite/urls.py
                                  ./apps/my_app/models.py
                                  ./apps/my_app/urls.py



                                                settings_local
                        •   Loaded after settings.py as overrides.

                        •   Easily set up a local development environment.

                        •   Production credentials are at no risk.

                        •   Easily deploy devel, staging, production stacks.



Thursday, July 19, 12
Toplevel is website project
                             ./manage.py
                             ./wsgi.py
                             ./requirements.txt
                             ./apps/mainsite/__init__.py
                             ./apps/mainsite/settings.py
                             ./apps/mainsite/settings_local.py.example
                             ./apps/mainsite/urls.py
                             ./apps/my_app/models.py
                             ./apps/my_app/urls.py



                              I have requirements.txt
                        • Library dependencies and versioning is simple.
                        • New developers can start working quicker.
                        • Simple virtualenv integration
Thursday, July 19, 12
Okay, but now what?



                        1. Install and use virtualenv

                        2. Design and use a template




Thursday, July 19, 12
How hard can it be?




Thursday, July 19, 12
Taking a closer look

                $ mkvirtualenv myproject && workon myproject
                                            OR
                 $ virtualenv ~/myproject/env && source ~/myproject/env/
                   bin/activate

                                        THEN
                 $ git clone http://github.com/my/template.git /tmp/
                   mytemplate

                 $ pip install django>=1.4

                 $ django-admin.py startproject --template=/tmp/
                   mytemplate ~/myproject/code

                 $ rm -rf /tmp/mytemplate




Thursday, July 19, 12
Our approach




                        Djenesis: our solution to the
                        bootstrapping problem




Thursday, July 19, 12
djenesis


                        • What is it?
                         • A tool that facilitates the process of
                            creating new and setting up existing Django
                            projects.

                        • Available on github & PyPi (apache license)
                         • github.com/concentricsky/djenesis


Thursday, July 19, 12
djenesis - Benefits

                        •   Simplifies routine workflows for:

                            •   Starting a new project

                            •   Getting an existing project up and running with ease

                        •   Encourages developers to use a standard project
                            structure and best python practices (virtualenv)

                        •   Allows external project templates

                            •   Supports git, svn, hg URLs

                            •   Supports Django 1.4 project templates



Thursday, July 19, 12
djenesis - History


                        1. Single zip file of a stub project

                        2. Script that wrapped stub

                        3. Decouples templates from script

                        4. Can use external templates, git URL

                        5. Now: supports Django 1.4 templates




Thursday, July 19, 12
djenesis - External Templates



                        • Inflating a project from a git/svn/hg source
                          means you can (and should) version
                          control your project templates

                        • Collaborate with your internal teams on an
                          ideal project structure




Thursday, July 19, 12
djenesis Stack




Thursday, July 19, 12
djenesis - Setup


                        • Requires virtualenv
                         • Install virtualenv through pip or easy_install
                        • Install djenesis through pip or easy_install




Thursday, July 19, 12
djenesis – New Project

               Before
                 $ virtualenv ~/myproject/env
                 $ source ~/myproject/env/bin/activate
                 $ pip install django
                 $ git clone github.com/my/template.git /tmp/tmpl
                 $ django-admin.py startproject --template=/tmp/tmpl    ~/
                 myproject/code
                 $ rm -rf /tmp/tmpl

               With djenesis
                 $ djenesis myproject git+https://github.com/my/template.git
                 $ source ~/myproject/env/bin/activate




Thursday, July 19, 12
djenesis – Existing Project


             Before
               $    virtualenv ~/myproject/env
               $    source ~/myproject/env/bin/activate
               $    git clone github.com/my/project.git ~/myproject/code
               $    pip install -r ~/myproject/code/requirements.txt

             With djenesis
               $ djenesis myproject -i git+https://github.com/my/project.git
               $ source ~/myproject/env/bin/activate




Thursday, July 19, 12
djenesis - Configuration



                        • Use a default template
                            DJENESIS_DEFAULT_TEMPLATE=/path/to/template


                        • Always use virtualenvwrapper (if installed)
                            DJENESIS_VIRTUALENVWRAPPER=1


                        • Set these in your .bashrc



Thursday, July 19, 12
djenesis – Next?


                        1. startapp template in project template

                        2. setup.py Support

                        3. djenesis post-create hooks for common tasks
                           (copy local settings, syncdb, import fixtures)

                        4. Activate virtualenv when djenesis is called




Thursday, July 19, 12
Bootstrapping
                        Django with ease
                        OSCON 2012
                        www.concentricsky.com
                        @concentricsky // biglan / wiggins / bothun




Thursday, July 19, 12

More Related Content

Similar to Django Bootstrapping with Ease

Cloud4all Architecture Overview
Cloud4all Architecture OverviewCloud4all Architecture Overview
Cloud4all Architecture Overviewicchp2012
 
Introducing Immutant
Introducing Immutant Introducing Immutant
Introducing Immutant Jim Crossley
 
Devops Devops Devops, at Froscon
Devops Devops Devops, at FrosconDevops Devops Devops, at Froscon
Devops Devops Devops, at FrosconKris Buytaert
 
Responsive Web Design & Workflow
Responsive Web Design & WorkflowResponsive Web Design & Workflow
Responsive Web Design & Workflowhouhr
 
Internship dotCloud
Internship dotCloudInternship dotCloud
Internship dotCloudJill Mee
 
Drupal and Devops , the Survey Results
Drupal and Devops , the Survey ResultsDrupal and Devops , the Survey Results
Drupal and Devops , the Survey ResultsKris Buytaert
 
Dan node meetup_socket_talk
Dan node meetup_socket_talkDan node meetup_socket_talk
Dan node meetup_socket_talkIshi von Meier
 
Devconf 2011 - PHP - How Yii framework is developed
Devconf 2011 - PHP - How Yii framework is developedDevconf 2011 - PHP - How Yii framework is developed
Devconf 2011 - PHP - How Yii framework is developedAlexander Makarov
 
Building and supporting drupal websites
Building and supporting drupal websitesBuilding and supporting drupal websites
Building and supporting drupal websitesNTEN
 
Building and Supporting Drupal Websites: In-House, Outhouse, or Both?
Building and Supporting Drupal Websites: In-House, Outhouse, or Both?Building and Supporting Drupal Websites: In-House, Outhouse, or Both?
Building and Supporting Drupal Websites: In-House, Outhouse, or Both?NTEN
 
Fosdem 2009 – improving drupal's page loading performance
Fosdem 2009 – improving drupal's page loading performanceFosdem 2009 – improving drupal's page loading performance
Fosdem 2009 – improving drupal's page loading performanceErup Enolc
 
Devops, Secops, Opsec, DevSec *ops *.* ?
Devops, Secops, Opsec, DevSec *ops *.* ?Devops, Secops, Opsec, DevSec *ops *.* ?
Devops, Secops, Opsec, DevSec *ops *.* ?Kris Buytaert
 
Ux paper prototyping
Ux paper prototypingUx paper prototyping
Ux paper prototypingGrace Ng
 
Realtime Apps with Django
Realtime Apps with DjangoRealtime Apps with Django
Realtime Apps with DjangoRenyi Khor
 
Devops, the future is here, it's just not evenly distributed yet.
Devops, the future is here, it's just not evenly distributed yet.Devops, the future is here, it's just not evenly distributed yet.
Devops, the future is here, it's just not evenly distributed yet.Kris Buytaert
 
Modularizing your Grails Application with Private Plugins - SpringOne 2GX 2012
Modularizing your Grails Application with Private Plugins - SpringOne 2GX 2012Modularizing your Grails Application with Private Plugins - SpringOne 2GX 2012
Modularizing your Grails Application with Private Plugins - SpringOne 2GX 2012kennethaliu
 
시니어가 들려주는 "내가 알고 있는 걸 당신도 알게 된다면"
시니어가 들려주는 "내가 알고 있는 걸 당신도 알게 된다면"시니어가 들려주는 "내가 알고 있는 걸 당신도 알게 된다면"
시니어가 들려주는 "내가 알고 있는 걸 당신도 알게 된다면"InfraEngineer
 
Wfs2009 Leading Change
Wfs2009 Leading ChangeWfs2009 Leading Change
Wfs2009 Leading ChangeGomindSHIFT
 

Similar to Django Bootstrapping with Ease (20)

Cloud4all Architecture Overview
Cloud4all Architecture OverviewCloud4all Architecture Overview
Cloud4all Architecture Overview
 
Introducing Immutant
Introducing Immutant Introducing Immutant
Introducing Immutant
 
Scrum by roofimon
Scrum by roofimonScrum by roofimon
Scrum by roofimon
 
Devops Devops Devops, at Froscon
Devops Devops Devops, at FrosconDevops Devops Devops, at Froscon
Devops Devops Devops, at Froscon
 
Responsive Web Design & Workflow
Responsive Web Design & WorkflowResponsive Web Design & Workflow
Responsive Web Design & Workflow
 
Internship dotCloud
Internship dotCloudInternship dotCloud
Internship dotCloud
 
Drupal and Devops , the Survey Results
Drupal and Devops , the Survey ResultsDrupal and Devops , the Survey Results
Drupal and Devops , the Survey Results
 
Dan node meetup_socket_talk
Dan node meetup_socket_talkDan node meetup_socket_talk
Dan node meetup_socket_talk
 
Devconf 2011 - PHP - How Yii framework is developed
Devconf 2011 - PHP - How Yii framework is developedDevconf 2011 - PHP - How Yii framework is developed
Devconf 2011 - PHP - How Yii framework is developed
 
Devops For Drupal
Devops  For DrupalDevops  For Drupal
Devops For Drupal
 
Building and supporting drupal websites
Building and supporting drupal websitesBuilding and supporting drupal websites
Building and supporting drupal websites
 
Building and Supporting Drupal Websites: In-House, Outhouse, or Both?
Building and Supporting Drupal Websites: In-House, Outhouse, or Both?Building and Supporting Drupal Websites: In-House, Outhouse, or Both?
Building and Supporting Drupal Websites: In-House, Outhouse, or Both?
 
Fosdem 2009 – improving drupal's page loading performance
Fosdem 2009 – improving drupal's page loading performanceFosdem 2009 – improving drupal's page loading performance
Fosdem 2009 – improving drupal's page loading performance
 
Devops, Secops, Opsec, DevSec *ops *.* ?
Devops, Secops, Opsec, DevSec *ops *.* ?Devops, Secops, Opsec, DevSec *ops *.* ?
Devops, Secops, Opsec, DevSec *ops *.* ?
 
Ux paper prototyping
Ux paper prototypingUx paper prototyping
Ux paper prototyping
 
Realtime Apps with Django
Realtime Apps with DjangoRealtime Apps with Django
Realtime Apps with Django
 
Devops, the future is here, it's just not evenly distributed yet.
Devops, the future is here, it's just not evenly distributed yet.Devops, the future is here, it's just not evenly distributed yet.
Devops, the future is here, it's just not evenly distributed yet.
 
Modularizing your Grails Application with Private Plugins - SpringOne 2GX 2012
Modularizing your Grails Application with Private Plugins - SpringOne 2GX 2012Modularizing your Grails Application with Private Plugins - SpringOne 2GX 2012
Modularizing your Grails Application with Private Plugins - SpringOne 2GX 2012
 
시니어가 들려주는 "내가 알고 있는 걸 당신도 알게 된다면"
시니어가 들려주는 "내가 알고 있는 걸 당신도 알게 된다면"시니어가 들려주는 "내가 알고 있는 걸 당신도 알게 된다면"
시니어가 들려주는 "내가 알고 있는 걸 당신도 알게 된다면"
 
Wfs2009 Leading Change
Wfs2009 Leading ChangeWfs2009 Leading Change
Wfs2009 Leading Change
 

Recently uploaded

Spring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfSpring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfAnna Loughnan Colquhoun
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
Things you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceThings you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceMartin Humpolec
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.francesco barbera
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdfJamie (Taka) Wang
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncGenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncObject Automation
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesThousandEyes
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 

Recently uploaded (20)

Spring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfSpring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdf
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
Things you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceThings you didn't know you can use in your Salesforce
Things you didn't know you can use in your Salesforce
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncGenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation Inc
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 

Django Bootstrapping with Ease

  • 1. Bootstrapping Django with ease OSCON 2012 www.concentricsky.com @concentricsky // biglan / wiggins / bothun Thursday, July 19, 12
  • 2. Introductions Mike Biglan Wiggins Josh Bothun CTO Architect Senior Developer @coffindragger Thursday, July 19, 12
  • 3. Areas of Concentric Sky Web/Django Mobile Enterprise Thursday, July 19, 12
  • 4. (Some) Technologies We Use • Chef • Backbone.js • Jenkins • Jinja2 • AWS • Celery • Sphinx • DebugToolbar • Memcached • jQuery / jQueryUI • South • Compass / Sass • git • Mongo & MySQL Thursday, July 19, 12
  • 5. Released CSky Creations • djenesis – Easy Django Bootstrapping • breakdown - Lightweight jinja2 template prototyping server • django-cachemodel – Automatic caching for django models Thursday, July 19, 12
  • 6. CSky Creations in the Works • skycloud – Tool for AWS auto-scaling • django-skycms – CMS as a library • django-sky-visitor – Invitations, email confirmation, etc • django-client-admin – Flexible admin for client Thursday, July 19, 12
  • 7. CSky Teams 10 Django Team STAFF 50+ 5 Design Team 9 PM Team 3 System Admins Thursday, July 19, 12
  • 8. Team Interfacing • Each project is a mix from each team • Members might switch amongst projects • Interfaces between teams • Clean separation of handoffs and responsibilities • Best-practices shared within teams Thursday, July 19, 12
  • 9. “Bootstrapping” • Pulling one up by one’s bootstraps • Computers do this much better than humans • Django – starting a project, new or existing Thursday, July 19, 12
  • 10. Bootstrapping with Ease Making booting (and rebooting) a django project as easy as possible • Ease is a continually moving target • Shouldn’t sacrifice other goals for ease (e.g. security, modularity, be PHP) Thursday, July 19, 12
  • 11. Moving Targets Moving targets create new problems. And new solutions. • CGI – Solving static HTML • JavaScript – Solving non-interactivity • CSS – Solving 100+ HTML pages • Puppet/Chef – Solving 100+ servers Thursday, July 19, 12
  • 12. Problem/Solution Lifecycle 1. Problem grows 2. Many create their own solution; some polish; some release 3. Community flocks to one (or a few) solutions 4. Solution(s) gets anointed and others die off Thursday, July 19, 12
  • 13. Goals of Easy Bootstrapping 1. More people using Django 2. More people using Django better Thursday, July 19, 12
  • 14. More People Using Django • As few commands as possible to try Django out • And to try someone else’s project out. • Must be highly likely to just work • All especially critical for beginners • Django’s runserver & sqlite...great! • It just works... Thursday, July 19, 12
  • 15. Making Mistakes should be Hard • Checklists are great • The best checklists are invisible • To automate lists of commands, must have consistency Thursday, July 19, 12
  • 16. Consistency Makes things Easier • Ease of replication across servers • Ease of sharing • Both templates and existing projects • Easier to build upon Thursday, July 19, 12
  • 17. Consistently Standardized • Standardization: a tool to get to consistency • This can be in an organization. Or the full community • Oh yeah, and... Thursday, July 19, 12
  • 18. Standard had Better be Right Thursday, July 19, 12
  • 19. Fragmentation • Poor standards lead to fragmentation • Hard to build things on top of fragmented environments • Imagine building for single browser. Not Mosaic. Thursday, July 19, 12
  • 20. Android Fragmentation http://www.businessinsider.com/android-fragmentation-chart-from-opensignalmaps-2012-5 Thursday, July 19, 12
  • 21. Flexibility vs Standardization • Standardization can be rigid; can kill flexibility • Especially as the target moves • E.g. contrib.auth • Flexibility allows preference, innovation • Best of both worlds is standardization with structured options of flexibility • Django does this well Thursday, July 19, 12
  • 22. Django • Django is a set of higher-order building blocks • If blocks are too large, sacrifices what can be made (i.e. not flexibile) • If blocks are too small, becomes inefficient and unmanageable • If block types are wrong, not loosely coupled (i.e. non-orthogonal) Thursday, July 19, 12
  • 23. 3 Stages of Project Structure Grief • Denial • Bargaining • Acceptance Thursday, July 19, 12
  • 24. Denial I’ll just take the easy way out... • Stuff required libraries in a ‘lib’ directory • Stick django there too • Name everything MyProjectSomething Thursday, July 19, 12
  • 25. Denial I’ll just take the easy way out... Why this doesn’t work • project starts to get bloated • version management becomes a headache • upgrade paths become tangled Thursday, July 19, 12
  • 26. Bargaining We’ll just adopt some standard practices... 1. Have the team argue about which way to do things 2. Assign someone to write a document about it 3. ??? 4. Ignore it when you start a new project Thursday, July 19, 12
  • 27. Bargaining We’ll just adopt some standard practices... Why this doesn’t work • Projects become inconsistent • It is not in source control • No way to manage changes across projects • There is no history of changes or discussion Thursday, July 19, 12
  • 28. Acceptance Embrace the community • requirements.txt, version management • also solves the ‘git submodule’ problem • virtualenv is great, always use it Thursday, July 19, 12
  • 29. Acceptance No, really, use virtualenv • dependency management • concurrent project management • sand-boxing • solves the “chicken-n-egg” problem • replication Thursday, July 19, 12
  • 30. What’s in a template? • For a django project, there are two styles: 1. toplevel directory is python code 2. toplevel directory is a website project Thursday, July 19, 12
  • 31. Toplevel is python code ./manage.py ./my_project/__init__.py • Pros ./my_project/settings.py ./my_project/urls.py • More pythonic; Everything ./my_project/wsgi.py is python code Django’s default template • It just works • Cons • Everything is python code • Where do non public files go? Thursday, July 19, 12
  • 32. Default Django Project ./manage.py ./my_project/__init__.py ./my_project/settings.py • Where do apps go? ./my_project/urls.py ./my_project/wsgi.py • How about ./my_project/ ./my_project/my_app/__init__.py my_app/? ./my_project/my_app/models.py • Name then becomes ‘my_project.my_app’ • What about reusability? • What if I want to rename my_project? Thursday, July 19, 12
  • 33. Default Django Project ./manage.py ./my_project/__init__.py ./my_project/settings.py • Where do apps go? ./my_project/urls.py ./my_project/wsgi.py • How about ./my_app/? • Better for reusability, but... ./my_app/__init__.py ./my_app/models.py • Now I have to add ./ to the PYTHON_PATH Thursday, July 19, 12
  • 34. Default Django Project ./manage.py ./myapp1/models.py ./myapp1/urls.py • What was the name ./myapp2/models.py again? • Was it ‘myapp1’ or ‘myapp2’ ./myapp2/urls.py ./myapp3/models.py ./myapp3/urls.py that contains the main urls and settings... Thursday, July 19, 12
  • 35. Toplevel is website project • Pros ./manage.py ./wsgi.py ./requirements.txt • Closer to real world ./apps/mainsite/__init__.py ./apps/mainsite/settings.py ./apps/mainsite/settings_local.py ./apps/mainsite/urls.py environments ./apps/my_app/models.py • Consistency across ./apps/my_app/urls.py projects • Cons • ??? None I guess! http://github.com/concentricsky/djenesis-template.git Thursday, July 19, 12
  • 36. Toplevel is website project ./manage.py ./wsgi.py ./requirements.txt ./apps/mainsite/__init__.py ./apps/mainsite/settings.py ./apps/mainsite/settings_local.py.example ./apps/mainsite/urls.py ./apps/my_app/models.py ./apps/my_app/urls.py Python code is sandboxed • only have to add ./apps to PYTHON_PATH • everything else is ignored by WSGI server Thursday, July 19, 12
  • 37. Toplevel is website project ./manage.py ./wsgi.py ./requirements.txt ./apps/mainsite/__init__.py ./apps/mainsite/settings.py ./apps/mainsite/settings_local.py.example ./apps/mainsite/urls.py ./apps/my_app/models.py ./apps/my_app/urls.py Consistent Entry Point Name • I always know where to look first: ./apps/ mainsite/urls.py • More important than it may seem at first blush Thursday, July 19, 12
  • 38. Toplevel is website project ./manage.py ./wsgi.py ./requirements.txt ./apps/mainsite/__init__.py ./apps/mainsite/settings.py ./apps/mainsite/settings_local.py.example ./apps/mainsite/urls.py ./apps/my_app/models.py ./apps/my_app/urls.py settings_local • Loaded after settings.py as overrides. • Easily set up a local development environment. • Production credentials are at no risk. • Easily deploy devel, staging, production stacks. Thursday, July 19, 12
  • 39. Toplevel is website project ./manage.py ./wsgi.py ./requirements.txt ./apps/mainsite/__init__.py ./apps/mainsite/settings.py ./apps/mainsite/settings_local.py.example ./apps/mainsite/urls.py ./apps/my_app/models.py ./apps/my_app/urls.py I have requirements.txt • Library dependencies and versioning is simple. • New developers can start working quicker. • Simple virtualenv integration Thursday, July 19, 12
  • 40. Okay, but now what? 1. Install and use virtualenv 2. Design and use a template Thursday, July 19, 12
  • 41. How hard can it be? Thursday, July 19, 12
  • 42. Taking a closer look $ mkvirtualenv myproject && workon myproject OR $ virtualenv ~/myproject/env && source ~/myproject/env/ bin/activate THEN $ git clone http://github.com/my/template.git /tmp/ mytemplate $ pip install django>=1.4 $ django-admin.py startproject --template=/tmp/ mytemplate ~/myproject/code $ rm -rf /tmp/mytemplate Thursday, July 19, 12
  • 43. Our approach Djenesis: our solution to the bootstrapping problem Thursday, July 19, 12
  • 44. djenesis • What is it? • A tool that facilitates the process of creating new and setting up existing Django projects. • Available on github & PyPi (apache license) • github.com/concentricsky/djenesis Thursday, July 19, 12
  • 45. djenesis - Benefits • Simplifies routine workflows for: • Starting a new project • Getting an existing project up and running with ease • Encourages developers to use a standard project structure and best python practices (virtualenv) • Allows external project templates • Supports git, svn, hg URLs • Supports Django 1.4 project templates Thursday, July 19, 12
  • 46. djenesis - History 1. Single zip file of a stub project 2. Script that wrapped stub 3. Decouples templates from script 4. Can use external templates, git URL 5. Now: supports Django 1.4 templates Thursday, July 19, 12
  • 47. djenesis - External Templates • Inflating a project from a git/svn/hg source means you can (and should) version control your project templates • Collaborate with your internal teams on an ideal project structure Thursday, July 19, 12
  • 49. djenesis - Setup • Requires virtualenv • Install virtualenv through pip or easy_install • Install djenesis through pip or easy_install Thursday, July 19, 12
  • 50. djenesis – New Project Before $ virtualenv ~/myproject/env $ source ~/myproject/env/bin/activate $ pip install django $ git clone github.com/my/template.git /tmp/tmpl $ django-admin.py startproject --template=/tmp/tmpl ~/ myproject/code $ rm -rf /tmp/tmpl With djenesis $ djenesis myproject git+https://github.com/my/template.git $ source ~/myproject/env/bin/activate Thursday, July 19, 12
  • 51. djenesis – Existing Project Before $ virtualenv ~/myproject/env $ source ~/myproject/env/bin/activate $ git clone github.com/my/project.git ~/myproject/code $ pip install -r ~/myproject/code/requirements.txt With djenesis $ djenesis myproject -i git+https://github.com/my/project.git $ source ~/myproject/env/bin/activate Thursday, July 19, 12
  • 52. djenesis - Configuration • Use a default template DJENESIS_DEFAULT_TEMPLATE=/path/to/template • Always use virtualenvwrapper (if installed) DJENESIS_VIRTUALENVWRAPPER=1 • Set these in your .bashrc Thursday, July 19, 12
  • 53. djenesis – Next? 1. startapp template in project template 2. setup.py Support 3. djenesis post-create hooks for common tasks (copy local settings, syncdb, import fixtures) 4. Activate virtualenv when djenesis is called Thursday, July 19, 12
  • 54. Bootstrapping Django with ease OSCON 2012 www.concentricsky.com @concentricsky // biglan / wiggins / bothun Thursday, July 19, 12