The Salesforce core stack embraces Open Source Software. Join one of our leading engineers and learn about how we tackle enterprise-grade challenges for our customers using Hadoop, HBase, Jetty, Solr, and Apache QPID. We'll also discuss the process of opening internal libraries like Aura and Phoenix, as well as Salesforce's place in the larger Open Source community.

1. How Open Source Software
Embiggens salesforce.com
What We Use, And What We Contribute To
Ian Varley, salesforce.com, Principal Member of Technical Staff
@thefutureian

2. Safe Harbor
Safe harbor statement under the Private Securities Litigation Reform Act of 1995:
This presentation may contain forward-looking statements that involve risks, uncertainties, and assumptions. If any such uncertainties
materialize or if any of the assumptions proves incorrect, the results of salesforce.com, inc. could differ materially from the results
expressed or implied by the forward-looking statements we make. All statements other than statements of historical fact could be
deemed forward-looking, including any projections of product or service availability, subscriber growth, earnings, revenues, or other
financial items and any statements regarding strategies or plans of management for future operations, statements of belief, any
statements concerning new, planned, or upgraded services or technology developments and customer contracts or use of our services.
The risks and uncertainties referred to above include – but are not limited to – risks associated with developing and delivering new
functionality for our service, new products and services, our new business model, our past operating losses, possible fluctuations in our
operating results and rate of growth, interruptions or delays in our Web hosting, breach of our security measures, the outcome of any
litigation, risks associated with completed and any possible mergers and acquisitions, the immature market in which we operate, our
relatively limited operating history, our ability to expand, retain, and motivate our employees and manage our growth, new releases of our
service and successful customer deployment, our limited history reselling non-salesforce.com products, and utilization and selling to
larger enterprise customers. Further information on potential factors that could affect the financial results of salesforce.com, inc. is
included in our annual report on Form 10-K for the most recent fiscal year and in our quarterly report on Form 10-Q for the most recent
fiscal quarter. These documents and others containing important disclosures are available on the SEC Filings section of the Investor
Information section of our Web site.
Any unreleased services or features referenced in this or other presentations, press releases or public statements are not currently
available and may not be delivered on time or at all. Customers who purchase our services should make the purchase decisions
based upon features that are currently available. Salesforce.com, inc. assumes no obligation and does not intend to update these
forward-looking statements.

3. Who am I?
Ian Varley
3.5 years at salesforce.com
Core Database Team, Big Data Team
@thefutureian

4. Show of hands ...
● Use OSS?
● Contribute to OSS?
● Write their own OSS projects?

5. Developers at salesforce.com
spend all day in open source
software.

6. salesforce.com engineers work on an OSS stack ...
▪ Linux (Ubuntu, RHL)
▪ Java
▪ Eclipse (+ IntelliJ, vim, emacs)
▪ Guava, Apache Commons, more
▪ JUnit, Mockito, Selenium
▪ Git (+ p4)
▪ Memcached

7. What’s so great about open
source software?

8. A rising tide ...
… lifts all boats.

9. It’s a win-win situation.
▪Everyone gets more out than they put in
▪You have control over your own destiny
▪You can attract the industry’s best minds
● The smartest devs seem to gravitate towards open source
● So if you raised your hand before, give yourself a pat on the back.

10. So,
what do we use?

11. Servlet Container
Servlet containers handle routing HTTP requests to code.
▪Started w/ commercial product
▪Feature: “steal” work from overloaded servers
▪(Code name: Hamburglar)
But! Show stopper bug, and no way to fix it ...

12. Solution: Jetty
▪http://www.eclipse.org/jetty/
▪Year-long migration process
▪Tricky with 10+ years of legacy code!
▪Now running Jetty (almost) everywhere.

13. Search Indexing
Indexer takes text (e.g. chatter posts, etc.), makes it searchable.
▪Original implementation: Lucene (forked)
▪But, scale keeps increasing!
▪Bottleneck: single-writer QFS on a SAN
▪Needed solution to scale horizontally

14. Solution: Solr
▪Horizontally scalable, REST interface
▪Query / index on same host, no more SAN
▪New features, core library is (latest) Lucene
We’ve also contributed some small fixes, and contracted a big fix to allow
handling indexers with many cores (10K+!).

15. Contributing is a
win / win.

16. Message Queue
Decouple calling code from its execution.
▪Originally: 10-15 devs had rolled their own
▪Centralized on a transactional queue (Vijay)
▪Commercial product, deeply coupled to DB
▪OK until: the “600” error. 3 years of back and forth.
▪Eventually rewrote our layer to work around it!
▪Scale problems: 50 -> 500 queues
▪CPU contention at head of queue

17. Solution: QPID
▪Apache project, good reputation
▪Separate tier from the DB
▪Ran into bugs …
▪… and fixed them.
▪40% memory savings on client (QPID-4873; thanks Helen Kwong & Brian Toal)

18. Open source lets us bring our experts
to help everybody.

19. Build: Ant
Build tools help get you from “code written” to “code running”.
▪Used Apache Ant for years
▪But, as the # of devs has grown …
▪scale and maintainability problems.

20. Solution: Maven
▪Moving core build to Apache Maven
▪Goal is a more modular and decoupled build structure
▪Declarative dependencies FTW
▪OSGi: Apache Felix

21. Deployment: Home Grown
Deployment tools let you get code out to servers.
▪Salesforce.com has always used home-grown tool, “ReleaseRunner”
▪Required for tight security model (no passwordless ssh, root)
▪But as we scale out, manual methods aren’t cutting it

22. Solution: Puppet, Salt, Razor, Rundeck
Get code out to lots of servers with little manual involvement.
▪Razor: automated machine inventory
▪Puppet: deployment of bits and configuration
▪Salt, Rundeck: service orchestration for restarts
All of this still very much WIP; nobody else does it w/ this level of security

23. Batch Processing
▪Salesforce == RDBMS
▪No great approach for batch processing
▪Especially on sets that don’t fit in memory
▪Working outside relational model very hard

24. Solution: Hadoop
Map/Reduce: ship computations to your data instead vice versa
▪Walter Macklem (Platform CTO); Codename: Gridforce
▪+HDFS (distributed file storage)
▪+Pig (a higher level language)
▪Features: recommendations, search relevance, machine learning
▪Log export pilot ...
(Ask your CSR/CSM/AM to
get nominated for the pilot!)

25. Big Data
Relational databases are powerful ...
▪But, started looking at cost model
▪Lower than average (multi-tenancy)
▪Model is so rich, prohibitive for really large data
▪RDBMS has strict scalability limits per object
▪Hard to scale out because, runs on big iron
What if we could store vast numbers of records, but with fewer capabilities
and assumptions? Scale horizontally, but with the same safety guarantees?

26. Big Data: HBase
Horizontally scalable NoSQL database.
▪Fewer capabilities (no joins, transactions)
▪Scales by adding machines
▪Fault tolerant (on HDFS)
▪Features? Initially, audit & compliance, event tracking
▪Eventually, a lot more: really big objects
▪Got a lot of field history? Join the FHR retention pilot! (Talk to you CSM)
This is my team, so I could talk for hours. But go see Lars Hofhansl’s talk!

27. OK, that’s cool. But, does Salesforce
contribute new projects?

28. Historically: no, not many.
But, this is changing.

29. Aura: UI Framework
Basis for new generation of Salesforce UI
▪High performance client-server architecture
▪Event-driven, MVC architecture
▪https://github.com/forcedotcom/aura

30. Phoenix: a SQL Skin for HBase
“We put the SQL back in NoSQL”
▪A proper subset of SQL
▪Familiar interface, scalable storage
▪Unlike Hive, uses the HBase client API
▪Blazing fast; queries in milliseconds
▪Tons of contribution since we opened it
▪https://github.com/forcedotcom/phoenix

31. Mobile SDK
All SDK dev for Salesforce done in open source
▪https://github.com/forcedotcom/SalesforceMobileSDK-iOS
▪https://github.com/forcedotcom/SalesforceMobileSDK-Android
▪Also: heavy use of Apache Cordova, to blend web & native components

32. Lots more!
So far, we’ve only been talking about Salesforce core.
▪Many salesforce.com companies use tons of Open Source:
•Heroku
- https://github.com/heroku
•Radian6,
Data.com, ExactTarget - you name it, we probably use it somewhere
▪And lots of open source stuff on the platform, too!
•http://boards.developerforce.com/t5/Salesforce-Labs-Open-Source/bd-p/labs
•You
can search github for Apex & Salesforce

33. Salesforce.com isn’t just an OSS user.
We’re an OSS pusher.

34. Committers on dozens of big projects
Salesforce.com actively supports a lot of people who primarily
contribute to open source projects (not just a side thing).
▪ Postgres: Tom Lane (Project Lead)
▪ Ruby: Matz (Project Lead)
▪ Maven: Jason Van Zyl (Project Lead)
▪ HBase: Lars Hofhansl (PMC, release manager); Jesse Yates
▪ Phoenix: James Taylor (Project Lead)
▪ Aura: Doug Chasman (Project Lead)
▪ Pig: Prashant Kommeredi

35. Is Open Source right for everything?
No.

36. It’s great for ...
▪Core components
▪Databases
▪Common algorithms
▪Reusable UI libraries & abstractions
And any case where “the source isn’t the secret sauce”.

37. It’s not great for ...
▪Code entangled with your business model
▪Code you didn’t write with a plan to open up
▪Software that’s “all things to all people”
▪Getting other people to do your work
But, these are kind of anti-patterns anyway, right … ?

38. Most return on investment is from
open sourcing “the interesting bits”,
rather than the whole stack.

39. And embracing the Open Source
approach, particularly in the last 3
years, has been a sea change.

40. In conclusion ...

41. In contributing, we all gain.
Look for more OSS involvement from salesforce.com
in the future!

42. Ian Varley
@thefutureian

43. We want to hear
from YOU!
Please take a moment to complete our
session survey
Surveys can be found in the “My Agenda”
portion of the Dreamforce app