Keynote at Geode Summit 2016 by Dr. Justin Erenkrantz, Bloolmberg LP. Creating the Future of Big Data Through "The Apache Way" and why this matters to the community

1. >>>>>>>>>>>>>>>>>>>>>
CREATING THE FUTURE
OF BIG DATA THROUGH
"THE APACHE WAY”
WHY THIS MATTERS TO THE
COMMUNITY
Dr. Justin R. Erenkrantz, Bloomberg LP
justin@erenkrantz.com / @jerenkrantz

2. WHY SHOULD I PAY ATTENTION?
» Mentor to Apache Geode and HAWQ
» Commi5er to Apache HTTP Server, APR, Subversion, Serf
» Former President and Director of The Apache SoBware
FoundaDon
» Ph.D. from University of California, Irvine
» DissertaDon: "ComputaDonal REST: A New Model for
Decentralized, Internet-Scale ApplicaDons”
» Head of Compute Architecture at Bloomberg LP
» ~50 billion Dcks DAILY flow through our systems
2

3. TECH @ BLOOMBERG: OPEN SOURCE
3
» The core of our Bloomberg Professional plaorm has evolved away
from proprietary code
» FoundaDons of our next-generaDon infrastructure - OpenStack, Ceph,
Hadoop, Spark, Solr, Chromium, Chef - are all open-source
» No longer can vendors tell us that they won’t fix a criDcal bug
» Places a lot of pressure on our partners to collaborate openly
» Giving back to the community - h"ps://github.com/bloomberg/
» Allows us to innovate at the higher levels – helping our customers
make sense of the firehose of informaDon that is available to them

4. TECH @ BLOOMBERG: OPEN CAN BE HARDWARE TOO!
4

5. HISTORY LESSON…
5
» Started as Apache Group with 8 members in Feb 1995
resuming work on NCSA h5pd
» UIUC placed the server code in public domain
» Most of the UIUC team leB to join Netscape
» Webmasters leB in the lurch and joined together
» The Apache SoBware FoundaDon incorporated in 1999
» Today, there are over 350 communiDes affiliated with
Apache performing over 16,000 code commits/month
Why?

6. PHILOSOPHY OF THE APACHE SOFTWARE FOUNDATION
6
» Let the contributors do what they do best: contribute.
FoundaDon exists to do the rest.
» Does not pay for contribuDons
» Many are sponsored by a third-party
» Staff ASF has are focused on infrastructure/PR/etc
» Does not pick “winners” or “losers”
» “CompeDDon” between ASF projects perfectly
acceptable as long as there are healthy
communiDes…think Geode and Ignite (!)

7. ANTI-PHILOSOPHY
7
» “The Apache Way” is not…
» Dumping your code on GitHub
» Single-sponsor contribuDons
» Running a Benevolent Dictatorship (BDFL)
» The Apache SoBware FoundaDon may not be best for
all projects...that’s perfectly OK.
» If you wish to be part of Apache, you need to adhere to
social constructs and norms
» Technical decisions are up to the community to decide

8. ROLE OF APACHE INCUBATOR
8
» Each project (TLP) is run relaDvely autonomously
» Project karma does not automaDcally carry over
» If I can commit to Geode, it doesn’t mean I can
commit to Ignite! (But, I could likely earn it easily!)
» Incubator was formed in 2003 as we were struggling to
scale the foundaDon and repeat the model. It worked.
» If a podling does not have a healthy community, it’ll
never graduate. That’s OK. If the podling does
become a TLP, but later loses its community, it’ll end
up in the Arc. That’s OK, too.

9. TRANSPARENCY & MERITOCRACY
9
» Roy’s Mantra: "If it's not on the list, it didn't happen.”
» Apache in the age of GitHub, JIRA, ReviewBoard, etc.
» Is the mailing list doomed?
» Generation gap may mean email isn’t preferred
» Tools are always secondary to process
» Transparency is the aim: allows others to have a voice
» The tools and process are never about prohibiting face-to-
face contact - but, ensuring that there is equal access for
participation and permitting asynchronous decision making
» Making decisions in a synchronous echo chamber (Slack,
IRC, etc.) is not conducive to transparency

10. MAKING DECISIONS
10
» Voting is the way contributors are (and feel) empowered
» “Binding” votes from recognized contributors (PMC)
» Vote on code, ideas, and, most importantly, releases
» Minimum acceptable quorum: 3 voters
» Minimum acceptable time frame: 72 hours
» The power of the dreaded “-1” (veto)
» Code can be vetoed, but not releases
» Veto should be cast as a last resort; used to foster
discussion

11. GROWING COMMUNITY
11
» ContribuDons can come from anywhere
» Relies upon core contributors being open to ideas
» Yet, there oBen is a set of agreed upon principles
» Going to Geode community and say that you should
remove all consistency code is a non-starter
» This is the power of the mythical "The Apache Way”
» Meritocracy: access based on demonstrated skills
» Michael Young's The Rise of the Meritocracy (1958)
– negaDve connotaDons across an enDre society

12. GROWING COMMUNITY
12
» As a downstream consumer of Apache projects, will
there be someone who is maintaining the code base?
Can I help volunteer to maintain it?
» A codebase by itself is inert
» Code is never perfect, but a healthy and inclusive
community will be improving the code constantly
based upon feedback and others
» “Community over Code”

13. ROLES IN INCUBATOR
13
» Think of a podling as being provided a set of training
wheels as they learn the rules of the road.
» Required quarterly reporDng is one of the few
mechanisms that the Board imposes to all projects to
ensure that the community is healthy.
» If no one submits the report, no one may be home!
» Mentors are around to answer quesDons, share
knowledge , and best pracDces. Mentors are not there
to contribute code – though, oBen we could; but, that
role is disDnct.

14. NORMS OF THE COMMUNITY
14
» Over the years, most disputes I have seen come down
to norms that were not agreed upon or documented
» Forming an explicit consensus on release versioning
and compaDbility rules up-front is so incredibly helpful.
» Projects always have a tension between “new
features” and compaDbility. Decide where the
community wants to be early on.
» The Geode wiki secDon is great. Keep it up!

15. EXPECTATIONS FOR CONTRIBUTORS
15
» Explicitly communicaDng to contributors who are not
yet in PMC what the expectaDons are for receiving
commit access (vote) to a project is extremely helpful.
» It’s painful to see contributors who do not feel
empowered by the community. It’s a huge red flag.
» Each project can and should set its own bar.
» My gut feeling now is to err on the side of inclusiveness
and give commit rights earlier than I did. It’s all under
version control anyway. Worst case, revoke that
person’s bit.

16. GRADUATION
16
» When will Apache Geode graduate from Incubator ?
» “ When it's ready” is the only honest answer.
» Geode community needs to demonstrate that it can
govern itself and be inclusive and transparent
» It doesn’t have to be perfect – no community is.
» This is where the Board can be extremely helpful.
» I am extremely happy to see the progress that Geode
has made so far and wish it the very best on its path.

17. Join the Apache Geode Community!
• Check out: http://geode.incubator.apache.org
• Subscribe: user-subscribe@geode.incubator.apache.org
• Download: http://geode.incubator.apache.org/releases/

18. 18
THANKS!
Dr. Justin R. Erenkrantz, Bloomberg LP
justin@erenkrantz.com / @jerenkrantz