Sergey Lukjanov, Data Processing PTL, outlines the changes made in the Icehouse release as well as upcoming updates for Juno. Learn more about Data Processing (Sahara) here: https://wiki.openstack.org/wiki/Sahara

1. PTL
Sergey Lukjanov
Data Processing Update
OpenStack Sahara (ex. Savanna)

2. To provide a scalable data processing stack
and associated management interfaces
● provision & operate Hadoop clusters
● schedule & operate Hadoop jobs / workloads

3. Elastic Data Processing (EDP) is Sahara’s take
on data processing workflow management.

4. Icehouse release overview
● 700+ code commits from 50 people
● 57 blueprints implemented
● ~5000 code reviews
● ~140 bugs fixed
details: https://launchpad.net/sahara/icehouse

5. Heat-based cluster provisioning

6. Hadoop 2 support
● in both Vanilla and HDP plugins
● EDP supports Hadoop 2

7. EDP improvements
● HBase and Sqoop via the HDP plugin
● Streaming & Java MapReduce
● External HDFS

8. CLI @ python-saharaclient

9. Sahara @ integrated gate

10. Juno plans
● Spark plugin supported by EDP
● Merge dashboard to Horizon
● Sahara as resources in Heat
● Heat engine by default
● Testing improvements

11. Juno plans
● Spark plugin supported by EDP
● Merge dashboard to Horizon
● Sahara as resources in Heat
● Heat engine by default
● Testing improvements