Anthill: A Distributed DBMS Based On MapReduce

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],分析型数据库相关背景 “ As a data processing paradigm, MapReduce represents a giant step backwards. ” From David J. Dewitt

Hadoop 架构 JobTracker TaskTracker MapTask ReduceTask TaskTracker MapTask ReduceTask TaskTracker MapTask ReduceTask Hadoop 客户端

[object Object],[object Object],[object Object],[object Object],[object Object],HadoopDB 的缺陷

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],研究目标

Anthill 背后的理论分析 ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],“ MapReduce advantages over parallel databases include storage-system independence and fine-grain fault tolerance for large jobs.” From Jeffery Dean

Anthill 架构设计 Hadoop 集群 JobTracker TaskTracker MonetDB 实例 MapTask ReduceTask TaskTracker MonetDB 实例 MapTask ReduceTask TaskTracker MonetDB 实例 MapTask ReduceTask 提交 MapReduce 作业 Anthill 服务器元数据库分析器优化器规划器执行器 Anthill 客户端网络

Anthill 服务器 ,[object Object],[object Object],[object Object],[object Object],[object Object],TOK_TAB_COL SELECT TOK_FROM TOK_TAB pokes TOK_INSERT TOK_DEST TOK_SELECT TOK_SELEXPR TOK_FUNC count 1 TOK_TMP TOK_SELEXPR TOK_GROUPBY foo TOK_TAB_COL foo TOK_WHERE = TOK_TAB_COL bar ‘ hello’ SELECT foo, count(1) FROM pokes WHERE bar=’hello’ GROUP BY foo;

Anthill 对 Hadoop 的扩展 ,[object Object],[object Object],[object Object],<<Interface>> InputFormat MonetDBInputFormat FileInputFormat TextInputFormat DBInputFormat SequenceFileInputFormat HiveInputFormat <<Interface>> OutputFormat MonetDBOutputFormat FileOutputFormat TextOutputFormat DBOutputFormat SequenceFileOutputFormat NullOutputFormat

两类表存储 ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

分布策略 ,[object Object],[object Object],“ Put the computation near the data.” From Jim Gray

分布式连接策略 ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Anthill 的其它相关设计 ,[object Object],[object Object]

实验结果与分析 ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

未来展望 ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],HDFS “ 推” ( 最终结果 ) Reduce Map 本地内存 ( 文件系统 ) 同步“拉” “ 推” HDFS MapReduce 数据流同步“拉” 同步“拉” MonetDB 集群 “ 推” ( 最终结果 ) Reduce Map 本地内存 ( 缓存 + 文件系统 ) “ 推” MonetDB 集群 Anthill 数据流同步“拉” 同步“拉” “ 推” 同步“拉” “ 推”

Anthill: A Distributed DBMS Based On MapReduce

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Ähnlich wie Anthill: A Distributed DBMS Based On MapReduce

Ähnlich wie Anthill: A Distributed DBMS Based On MapReduce (20)

Anthill: A Distributed DBMS Based On MapReduce