3. 3
*概念:软件架构主题
2009年软件架构主题!
架构模式:3&4层New
服务架构:SOA WOA
存储方式:RDB NoSQL
编程语言:编译型动态语言
应用类型:普通社会化
处理方式:集中式P2P
架构方式:封闭架构开放服务链
2010年软件架构主题?
架构模式:云计算五层架构
The Hardware Layer
The Virtualization Layer
The IaaS Layer
The PaaS Layer
The SaaS Layer
服务架构:ROA
存储方式:NOSQL
【一家之言】
NoSQL Non-Relational DB
NOSQL Not-Only RDB
4. 4
*概念:什么是DAL
概念:数据访问层DAL 【Layer=Services】
【维基百科】A Data Access Layer (DAL) is a layer of a computer program which provides simplified access to
data stored in persistent storage of some kind, such as an entity-relational database. This data access layer is
used in turn by other program modules to access and manipulate the data within the data store without having to
deal with the complexities inherent in this access.
DAL是一系列服务的集合,是DAO在大型系统中的自然延伸,是SOA的具体实现。
好处:
简化客户端接口,规范数据操作【透明性分片】
保持客户端接口,动态升级服务端实现【SOA】
支持对巨量数据的访问【缓存;读写分离;分片(分库/分表)】
服务的治理【认证/管理/权限/监控/分析/优化/…】
功能需求:
访问代理【Proxy】
读写分离【RAIDb-1镜像,每个库都是全库。写库是单点】
数据分片【RAIDb-0分区,每个库都是子库。】
高可用性【支持Quorum-NWR模型——多个写库】
特性需求:
平台中立【DAL服务器,可部署在多种平台上】
语言中立【DAL客户端,支持Java、C/C++、PHP、Flex等多种语言。】
数据库中立【1)多种数据库类型;2)多个数据库版本?】
资源共享【1) 共享DataSource连接池; 2)共享Cache。】
访问日志【1)提供SqlLog可用于性能瓶颈分析;2)提供BackLog可用于数据恢复。】
访问控制【1)用户认证;2)连接管理;3)权限控制。】
读写分离【FullDB】
数据分片【分表,分库】
访问集群【负载均衡:HA-Proxy、JGroup】
43. 43
+实践:结果集分页技术
方式:
1,iBatis或Hibernate
2,直接修改psqlid配置文件的sql语句。
样例:通过PoGenerator生成PO对象以及SQL模板
代码:见【实践:客户端(生成PO和属性注入) 】
方法:
String sql_query_first(String dbtype,String table,String fields,String wheres,String orders,int n);
String sql_query_limit(String dbtype,String table,String fields,String wheres,String orders,int
offset,int n);
类型:MySQL/PostgreSQL/SQLite/HSQLDB/H2
queryFirst = SELECT {fields} FROM tusers WHERE {wheres} ORDER BY {orders} LIMIT 15
queryPage = SELECT {fields} FROM tusers WHERE {wheres} ORDER BY {orders} LIMIT 15 OFFSET 0
类型:DB2
queryFirst = SELECT {fields} FROM umpay.t_user WHERE {wheres} ORDER BY {orders} FETCH FIRST 15
ROWS ONLY
queryPage = SELECT * FROM (SELECT RANK() OVER( ORDER BY {orders}) AS i_,{fields} FROM
umpay.t_user WHERE {wheres}) AS T_ WHERE i_>0 FETCH FIRST 15 ROWS ONLY
#queryPage = SELECT * FROM (SELECT ROWNUMBER() OVER( ORDER BY {orders}) AS i_,{fields} FROM
umpay.t_user WHERE {wheres}) AS T_ WHERE i_>0 FETCH FIRST 15 ROWS ONLY 【ROWNUMBER()=ROW_NUMBER() 】
类型:Oracle
queryFirst = SELECT {fields} FROM mobile.tarea WHERE {wheres} ORDER BY {orders} WHERE ROWNUM<=1
queryPage = SELECT * FROM(SELECT T.*,ROWNUM RN FROM (SELECT {fields} FROM mobile.tarea WHERE
{wheres} ORDER BY {orders}) T WHERE ROWNUM<=15) WHERE RN>0
参考:
http://en.wikipedia.org/wiki/Select_(SQL)
64. 64
参考:Dynamo关键技术
分布式Key-Value存储Dynamo:
特点
完全去中心化,人工管理工作很小。
总是可写,可根据读写情况进行调整NWR来优化
采用的都比较成熟的技术来处理可扩展性和可用性问题
问题:Partitioning
技术:改进的Consistent Hashing
优点:Incremental Scalability
问题:High Availability for writes
技术:Vector clocks with reconciliation during reads
技术:Version size is decoupled from update rates.
问题:Handling temporary failures
技术:Sloppy Quorum and hinted handoff
技术:Provides high availability and durability guarantee
when some of the replicas are not available.
问题:Recovering from permanent failures
技术:Anti-entropy using Merkle trees
技术:Synchronizes divergent replicas in the background.
问题:Membership and failure detection
技术:Gossip-based membership protocol and failure
detection.
技术:Preserves symmetry and avoids having a centralized
registry for storing membership and node liveness
information.
参考:
http://hi.baidu.com/zeorliu/blog/item/984867f0a4b55bc97931aaf2.h
tml 介绍Amazon的Dynamo
http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html
『amazon-dynamo-sosp2007.pdf』
68. 68
+容错:BTF Basic protocol
BFT: Protocol overview
Client c sends m = <REQUEST,o,t,c>σc to the primary. (o=operation,
t=monotonic timestamp)
Primary p assigns seq#n to m and sends <PRE-PREPARE,v,n,m> σp
PBFT main ideas:
•Static configuration (same 3f+1 nodes)
•To deal with malicious primary
to other replicas. (v=current view, i.e., replica set)
If replica i accepts the message, it sends <PREPARE,v,n,d,i> σi to
other replicas. (d is hash of the request). Signals that i agrees to
assign n to m in v.
•Use a 3-phase protocol to agree on sequence number
Once replica i has a pre-prepare and 2f+1 matching prepare
•To deal with loss of agreement
messages, it sends <COMMIT,v,n,d,i> σi to other replicas. At this
point, correct replicas •Use agree a bigger on quorum an order (2f+of 1 requests out of 3f+within 1 nodes)
a view.
•Need to authenticate communications
Once replica i has 2f+1 matching prepare and commit messages, it
executes m, then sends <REPLY,v,t,c,i,r> σi to the client. (The need
for this last step has to do with view changes.)
78. 78
参考:开源C-JDBC概念0
综合方案1:定制jdbc驱动;定制jdbc服务;
Sequoia是c-jdbc项目的延续(因和SUN的jdbc商标冲
突而改名),该项目发端于INRIA(法国国家计算机和
自动化研究院)的sardes项目。2003年提出了RAIDb的
概念,并开发c-jdbc作为软件实现,RAIDb借鉴了RAID
的思想,采用灵活的数据库复制技术和集成中间件,
以廉价、异构的数据库构建冗余数据库集群,从而提
供高可扩展、灵活的高性能集群。
参考
http://c-jdbc.ow2.org/ HA DB clusters with JDBC(已
改名Sequoia 2.10.10 http://sequoia.continuent.org
http://linux.softpedia.com/progDownload/Sequoia-
Download-477.html下载
http://forge.continuent.org/jira/browse/SEQUOIA
问题,代码库:cvs -z3 -
d:pserver:anonymous@sequoiadb.cvs.sourcefo
rge.net:/cvsroot/sequoiadb co -P modulename
)
http://liuye.javaeye.com/blog/224496 异构数据库中间
件sequoia(c-jdbc)初体验
http://www.continuent.com/community/lab-projects/
sequoia Important Update on Sequoia
Note: Sequoia is no longer actively developed or
supported by Continuent. Our main open source
project is Continuent Tungsten. Please consider
Continuent Tungsten as an alternative.
It is important to understand that Sequoia has a
number of trade-offs that limit its usefulness
because of the performance impact and lack of
application transparency. Tungsten processes
updates more quickly, handles a wider range of
SQL, works over a WAN, and permits direct
database access without proxies.
79. 79
参考:开源c-jdbc概念1
RAIDb
RAIDb concept
Redundant Array of Inexpensive
Databases
RAIDb controller
gives the view of a single
database to the client
balance the load on the database
backends
RAIDb levels offers various tradeoff
of performance and fault tolerance
RAIDb levels
RAIDb-0 【分区】
Partitioning
no duplication and no fault
tolerance
at least 2 nodes
RAIDb-1 【镜像】
mirroring
performance bounded by write
broadcast
at least 2 nodes
RAIDb-2
partial replication
at least 2 copies of each table for
fault tolerance
at least 3 nodes
SQL requests
RAIDb controller
table 1 table 2 & 3 table ...
table n-1 table n
SQL requests
RAIDb controller
Full DB Full DB Full DB
Full DB Full DB
SQL requests
RAIDb controller
Full DB table x table y
table x & y table z
80. 80
参考:开源c-jdbc概念2
RAIDb
RAIDb levels composition
RAIDb-1-0
no limit to the composition
deepness
C-JDBC
overview
Middleware implementing RAIDb
100% Java implementation
open source (LGPL)
Two components
generic JDBC driver (C-JDBC
driver)
C-JDBC Controller
Read-one, Write all approach
provides eager (strong)
consistency
Supports heterogeneous databases
【异构DB】
RAIDb-0 controller
table w table x & y
SQL requests
RAIDb-1 controller
table z
RAIDb-0 controller
table w table y
RAIDb-0 controller
table x table y
table z
table x & z
table w