1. Data Abstraction for
Large Web Applications
By Brandon Savage

2. Who Am I?
• Software developer at Mozilla working on Socorro
• Author of the PHP Playbook
• Former frequent blogger on PHP topics
• Private pilot in my spare time

3. Data Abstraction For LARGE
Web Applications

4. No magic bullets

5. Once upon a time...
In a galaxy far far away...

6. Eventually the web grew up.
And grew larger.

7. Most webapps still start as
though they’ll always use a
database.

8. We need to change our
thinking.

9. Socorro

10. Socorro Data Sources
• Postgres
• REST API (Middleware)
• Elastic Search
• Hbase
• Bugzilla REST API
• Memcache

11. A database-centric model just
doesn’t work anymore.

12. Solving the problem
• Separate the use of data from the retrieval of data.
• Think in terms of actions.
• Build our applications to be storage agnostic.
• Use the correct data storage medium.

13. #1 Separate the use of data
from the retrieval of data

14. <?php
class MainPage_Controller {
/* ... */
public function do_something(){
/* ... */
$sql = ‘SELECT * FROM database”;
$results = $this->execute($sql);
return $this->executeView(‘index’, array(‘results’ => $results));
}
}

15. <?php
class Data_Model {
/* ... */
public function get_some_data() {
$sql = ‘SELECT * FROM database”;
$results = $this->execute($sql);
/** process results **/
return $processedResults
}
}

16. Processing the data is a
separate layer.

17. <?php
class Data_Model {
/* ... */
public function getSomeData() {
$data = $this->adapter->queryData();
! /** process data here **/
! return $processedData;
}
}
class Data_Model_Adapter extends MySQL_Adapter implements Adapter
{
public function queryData() {
$sql = ‘SELECT * FROM table’;
/** turn into common format **/
return $commonFormatData;
}
}

18. Swapping out data sources
becomes very simple.

19. A cautionary tale

20. Move to middleware in Socorro

21. Make life easier on yourself: do
it right the first time!

22. #2 Think in terms of actions.

23. Actions move beyond SELECT,
INSERT, UPDATE and DELETE.

24. Domain Modeling:
“What are you modeling?”

25. What do I want?
What do I need?
What does this data represent?

26. Django Models:
One model per table.
All methods relate to SQL.
That sucks.

27. <?php
abstract class User_Model {
public function loadUser();
public function authenticateUser();
public function showUserPhones();
}

28. #3 Build our applications to be
storage agnostic

29. Use a standard data format

30. stdClass()

31. Create custom objects for
typehinting or additional
methods

32. Avoid expecting built-ins like
PDOStatement and
MongoCursor outside
retrieval layer

33. #4 Use the correct
storage medium.

34. Example: memcache isn’t for
long-term storage.

35. Example: MongoDB is not for
relational data storage.

36. Relational data goes in relational
databases!

37. Choose the correct NoSQL
database for your needs.

38. Availability, reliability, and
consistency.
Pick two.

39. Consider data storage that isn’t
a database at all.

40. Alternative data options
• Elastic Search
• Redis
• S3
• The File System (Yes! It still exists!)

41. Fix it now or fix it later.
But you will have to fix it.

42. Question time