A production of software stacks is an important part of a healthy software ecosystem. This talk is about most advanced open technology for the software stacks creation and validation, provided by Apache BigTop (incubating). I am going to discuss the advantages of the project, challenges our project and community is facing, and future plans. Presenter: Konstantin Boudnik, PhD

1. What's stacking?
Or:
How I Learned to Stop Worrying
and Love the BigTop
Dr. Konstantin Boudnik
Apache Hadoop committer since 2009
Co-author of iTest (BigTop's kernel)
Senior architect @Karmasphere
cos@apache.org

2. Distributed under CC3.0
https://creativecommons.org/licenses/by/3.0/

3. Disclaimer!
All characters appearing in this work are fictitious. Any
resemblance to real creatures, living or dead, is purely
coincidental

4. Can you see the forrest
behind the trees?

5. Not a technical talk...
What's the real intrinsic value of BigTop
What is our philosophical proposition

6. Why Hadoop data stack is important?

7. The elephant is yesterday's news...
An article over at Forbes:
“...it is not a database, it is not an analytics
environment, and it is not a visualization tool.
By itself, it is not a solution for helping
businesses make better decisions.”
Some self-proclaimed hot-shot dude

8. Sorry, I am allergic to bullshit...

9. There are limits, all right...
● Maximum size of the file system
● High availability
● Recoverability

10. But there is a lot to pick from...
● Do you want to experiment with
– CEPH
– Cassandra
– Ganglia
– Puppet

11. Why I am wasting your time?
● You are an ASP and need to ship a stack

12. Why I am wasting your time?
● You are an ASP and need to ship a stack
● You need a higher level abstraction for MR

13. Why I am wasting your time?
● You are an ASP and need to ship a stack
● You need a higher level abstraction for MR
● You are in need of virtual deployment

14. Why I am wasting your time?
● You are an ASP and need to ship a stack
● You need a higher level abstraction for MR
● You are in need of virtual deployment
Hey!
“I am gonna just pick up a distro from B or S”1)
1) “B” - Big Ego Distributor
“S” - Small Ego Distributor

15. Yeah, right! (case study)
Commercial vendor “BS” packs Hive 0.7.1 into
a release

16. We know how it is full of it

17. Good news!
“Our next release comes with Hive 0.8.1”
BS' VP of Product

18. But may be they just forgot this?..
public int executeUpdate(String sql, String[]
columnNames) throws SQLException {
throw new SQLException("Method not
supported");
}

19. So, no worries! We can deliver!

20. But can they?

21. Do you REALLY have to be at mercy
of a distribution vendors?

22. DIY: and there's always “but”...
“I have no resource to do it myself”

23. Maybe ASP or ISV business isn't for
you?

24. Or maybe
BigTop stack: take it and go...
Modify a stack BOM
– Build
– Deploy
– Configure with Puppet (included)
– Test (scenarios are provided or easy to add)
– Grab a virtual appliance if short of hardware

25. Hit a problem?
Just rinse and repeat (C)

26. Many thanks to the good folks from
Hadoop community!
● Many components
● Gazillions of versions
● Lot of patches if you like it hot and dirty

27. WAIT!
Where is the philosophy?

28. Competition is great!
● Competition enabling open platform
● Diversity of stacks
● No more vendors lock-ups
● Don't be afraid to try
● Be your own Sprechstallmeister

29. May be something wrong with this?

30. News And Updates

31. 5 releases of BigTop!
Three are out
Two more in the work

32. Strong community and growing!
20 committers and many contributors
We are looking for more!

33. And sorry: no more binaries for you
BigTop won't be publishing convenience
binary artifacts anymore

34. Might as well use this expertise:
http://www.magnatempusgroup.net

35. Come and join us!
https://incubator.apache.org/bigtop/
https://blogs.apache.org/bigtop/
Powered By Bigtop:
Cloudera
EMC
HortonWorks
MagnaTempusGroup
Trend Micro

36. May be one day...

37. Q&A