1. DAY2%/%Track%3%/
11:00~11:45

2. 김요한

3. CONTENTS
- Consistent Hashing
- 분산 코드네이터
5. 더 고민해야 할 부분
- 공유 데이터 관리
- 모듈 기반 어플리케이션
- 대용량 Connection 관리

4. 1. Scalability

5. 1 Scalability
When you add twice as many servers, are you twice as fast
(or have twice the capacity)?
- LiveJournal
https://www.usenix.org/legacy/events/usenix07/tech/slides/fitzpatrick.pdf
Replacing all components of a car while driving it at
100mph.
- Instagram
https://www.unboundid.com/blog/2012/06/27/scaling-at-100mph

6. 1 Scalability

7. 1 Scalability
http://stalk.io
<script type="text/javascript" src="http://www.stalk.io/stalk.js" charset="utf-8" ></script>
<script language="javascript">
STALK.init();
</script>

8. 2. Async Event Looping Server

9. 2 Async Event Looping Server
Single'Thread'로'Event'Looping'처리'하는'비동기'서버
core'당'하나의'thread'/'저사양'/'고성
능
/'WebSocket'추상화
cf.'DB'연결'Transaction'관리'/'좀'더'오래'걸리는'작업'/'일반적인'업무'시스템

10. 2 Async Event Looping Server
http://www.techempower.com/benchmarks/

11. 2 Async Event Looping Server
서버만$많다면$될까?

12. 3. Reverse Proxy Server

13. 3 Reverse Proxy Server
L4#/#L7#Switch
?

14. 3 Reverse Proxy Server
Reverse&Proxy

15. 3 Reverse Proxy Server
Reverse&Proxy
L4&Switch

16. 3 Reverse Proxy Server
HAProxy + Varnish
Fully Dynamic Contents
HaProxy
Varnish

17. 3 Reverse Proxy Server
참고로,
Job Queueing
Gearman, RabbitMQ + Celery (instagram, pinterest)
Job$Worker
Job$Worker
Job$Queue$Server
Job$Worker
비동기 처리
Job$Worker
병렬 프로세싱

18. 3 Reverse Proxy Server
public!class!ReverseProxyServer!extends!Verticle!{
!!public!void!start()!!{
!!!!HttpClient!clients[]!=
!!{vertx.createHttpClient().setHost("10.21.32.01").setPort(8282),
:::vertx.createHttpClient().setHost("10.21.32.02").setPort(8282),
:::vertx.createHttpClient().setHost("10.21.32.03").setPort(8282),:.:.:.
};
Reverse&Proxy&서버는&
“&부하&분산&”&은&어떻게?
“&무중단&서버&추가&”&는&어떻게?
!!!!vertx.createHttpServer().requestHandler(new!Handler()!{
!!!!!!public!void!handle(final!HttpServerRequest!req)!{
!!!!!!!!final!HttpClientRequest!cReq!=!clients.get().request(req,
!!!!!!!!!new!Handler<HttpClientResponse>()!{
!!!!!!!!!!public!void!handle(HttpClientResponse!cRes)!{
!!!!!!!!!!!!cRes.dataHandler(new!Handler<Buffer>()!{
!!!!!!!!!!!!!!req.response.write(data);
!!!!!!!!!!!!});
신규서버&추가&/&장애&서버&처리&.&.&.&.&Auto&Scale&Out&?
!!!!!!!!!!!!cRes.endHandler(new!SimpleHandler()!{
!!!!!!!!!!!!!req.response.end();
!!!!!!!!!!!!});
!!!!!!!!!!}
!!!!!!!!});
!!!!!!}
!!!!}).listen(8080);
vert.x 로 Reverse
!!}
Proxy 서버 구현하기.

19. 4. 분산서버 노드 관리
- Consistent Hashing
- 분산 코드네이터

20. 4.1 Consistent Hashing
부하 분산 방법 ?
Consistent(Hashing

21. 4.1 Consistent Hashing
Service&Server
A
D
Token&Ring
C
value&=&HASH(
B
)

22. 4.1 Consistent Hashing
A
D
value#=#HASH(
B
장애#발생#!!
C
A#-#C#구간이#넓어서#C#에#부하가#많을#수#있다#!
)

23. 4.1 Consistent Hashing
B C
D
C
B
A
D
D A
B C
A
D
value&=&HASH(
)
A
B
C
D
B
A
C
A
B
C
B
D
A
D
C
C
B A
C D
B
A
value&=&HASH(
)

24. 4.1 Consistent Hashing
B C
D
C
B
A
D
D A
B C
A
D
value&=&HASH(
)
A
B
C
D
B
A
C
A
B
C
D
B
A
D
C
C
B A
C D
B
A
value&=&HASH(
)

25. 4.1 Consistent Hashing
public!class!ConsistentHash<T>{
ConsistentHash.add(‘nodeA’,...);
!!!!!!!!private!final!HashFunction!hashFunction!=!Hashing.md5();
!!!!!!!!private!final!SortedMap<Long,!T>!circle!=!new
TreeMap<Long,!T>();
ConsistentHash.add(‘nodeB’,...);
!!!!!!!!public!void!add(String!name,!T!node)!{
!!!!!!!!!!!!!!!!!!!!!!!
circle.put(hashFunction.hashString(name).asLong(),node);
!!!!!!!!public!void!remove(String!name)!{
!!!!!!!!}
!!!!!!!!!!!!!!!!!!!!!!!
circle.remove(hashFunction.hashString(name).asLong());
ConsistentHash.add(‘nodeC’,...);
!!!!!!!!}
!!!!!!!!public!T!get(String!value)!{
!!!!!!!!!!!!!!!!long!hash!=!hashFunction.hashString(value).asLong();
!!!!!!!!!!!!!!!!if!(!circle.containsKey(hash))!{
!!!!!!!!!!!!!!!!!!!!!!!!SortedMap<Long,!T>!tailMap!=!circle.tailMap(hash);
!!!!!!!!!!!!!!!!!!!!!!!!hash!=
ConsistentHash.add(‘nodeD’,...);
tailMap.isEmpty()?circle.firstKey():tailMap.firstKey();
!!!!!!!!!!!!!!!!}
!!!!!!!!!!!!!!!!return!circle.get(hash);
!!!!!!!!}

26. 4.1 Consistent Hashing
public!class!ConsistentHash<T>{
!!!!!!!!private!final!HashFunction!hashFunction!=!Hashing.md5();
!!!!!!!!private!final!SortedMap<Long,!T>!circle!=!new!TreeMap<Long,!T>();
!!!!!!!!private!final!int!numberOfReplicas!=!100;
!!!!!!!!public!void!add(String!name,!T!node)!{
!!!!!!!!!!!!!!!!for!(int!i!=!0;!i!<!numberOfReplicas;!i++)!{
!!!!!!!!!!!!!!!!!!!!!!!!circle.put(hashFunction.hashString(name!+
i).asLong(),node);
!!!!!!!!!!!!!!!!}
!!!!!!!!}
!!!!!!!!public!void!remove(String!name)!{
!!!!!!!!!!!!!!!!for!(int!i!=!0;!i!<!numberOfReplicas;!i++)!{
!!!!!!!!!!!!!!!!!!!!!!!!circle.remove(hashFunction.hashString(name!+
i).asLong());
!!!!!!!!!!!!!!!!}
!!!!!!!!}
!!!!!!!!public!T!get(String!value)!{
!!!!!!!!!!!!!!!!long!hash!=!hashFunction.hashString(value).asLong();
!!!!!!!!!!!!!!!!if!(!circle.containsKey(hash))!{
!!!!!!!!!!!!!!!!!!!!!!!!SortedMap<Long,!T>!tailMap!=!circle.tailMap(hash);
!!!!!!!!!!!!!!!!!!!!!!!!hash!=
tailMap.isEmpty()?circle.firstKey():tailMap.firstKey();
!!!!!!!!!!!!!!!!}

27. 4.1 Consistent Hashing
public!class!ConsistentHash<T>{
!!!!!!!!private!final!HashFunction!hashFunction!=!Hashing.md5();
!!!!!!!!private!final!SortedMap<Long,!T>!circle!=!new!TreeMap<Long,!T>();
private!final!int!numberOfReplicas!=
!!!!!!!!private!final!int!numberOfReplicas!=!100;
100;
모두#똑같이#해야#하나?
!!!!!!!!public!void!add(String!name,!T!node)!{
!!!!!!!!!!!!!!!!for!(int!i!=!0;!i!<!numberOfReplicas;!i++)!{
!!!!!!!!!!!!!!!!!!!!!!!!circle.put(hashFunction.hashString(name!+
서버의#CPU#Core#수#나#네트워크#구간#등#고려
i).asLong(),node);
!!!!!!!!!!!!!!!!}
!!!!!!!!}
!!!!!!!!public!void!remove(String!name)!{
!!!!!!!!!!!!!!!!for!(int!i!=!0;!i!<!numberOfReplicas;!i++)!{
!!!!!!!!!!!!!!!!!!!!!!!!circle.remove(hashFunction.hashString(name!+
i).asLong());
!!!!!!!!!!!!!!!!}
!!!!!!!!}
!!!!!!!!public!T!get(String!value)!{
!!!!!!!!!!!!!!!!long!hash!=!hashFunction.hashString(value).asLong();
!!!!!!!!!!!!!!!!if!(!circle.containsKey(hash))!{
!!!!!!!!!!!!!!!!!!!!!!!!SortedMap<Long,!T>!tailMap!=!circle.tailMap(hash);
!!!!!!!!!!!!!!!!!!!!!!!!hash!=
tailMap.isEmpty()?circle.firstKey():tailMap.firstKey();
!!!!!!!!!!!!!!!!}

28. 4.1 Consistent Hashing
replics
(standard deviation)
서버#노드#10#개를#1#부터#500#개#까지#relicas#로#배치#10,000#번의#consistent#Hashing#결과
https://weblogs.java.net/blog/tomwhite/archive/2007/11/consistent_hash.html

29. 4.2 분산 코디네이터
zkClient.get().create("/serviceServers/nodeA",%%%//%서버정보를%가지고%있는%zNode%Path5
55555555555555555555555555555555"1.1.1.1:8080".getBytes(),5//%zNode%의%데이터,%서버%상세정보%(Null%이어도
됨)
55555555555555555555555555555555CreateMode.EPHEMERAL);%%%%%//%zNode%타입%(EPHEMERAL%:%임시노드%/
PERSISTENT%:%영구생성)
EPHEMERAL(타입은(Zookeeper(서버에(접속이(끊어지면,(자동으로(삭제(!!
서비스(서버(장애로(Shutdown(되면(Zookeeper(목록에서(자동으로(삭제(가능
zkClient.get().create("/serviceServers/nodeB",%.%.%.%.
.
zkClient.get().create("/serviceServers/nodeC",%.%.%.%.
.
zkClient.get().create("/serviceServers/nodeD",%.%.%.%.
.
Apache ZooKeeper™
/serviceServers/nodeA
55555555555555/nodeB
55555555555555/nodeC
55555555555555/nodeD

30. 4.2 분산 코디네이터
Apache ZooKeeper™
/serviceServers/nodeA
--------------/nodeB
--------------/nodeC
--------------/nodeD
-----------------:
-----------------:
노드"삭제
create"node
서버"목록을"watching"하면서"
2
Proxy"서버의"서비스"서버"목록을"동기화"!! watching
reverse"proxy"server
3
1
서비스"서버가"startup"될때"zookeeper"Server"에"연
결해서"EPHEMERAL"노드"생성
장애"발생""
zookeeper"와"연결이"자동으로"끊어짐!!
서비스"서버"목록을"Ring"에"추가
Consistent"Hashing
<"service"servers">

31. 4.2 분산 코디네이터
장애#발생시#자동#삭제#될#수#있도록!CreateMode.EPHEMERAL!로#Znode#생성
?
하지만,#또#다른#문제들.
-#네트워크#문제로#znode#가#삭제
-#Disk#Full,#Memory,#CPU#.#.#.#.#?
CreateMode.PERSISTENT!로#Znode#생성#후#데몬으로#서버들의#상태#관리.
Auto#Scaling#이#가능하도록#해야#할#것.

32. 5. 더 고민해야 할 부분
- 공유데이터관리
- 모듈 기반 어플리케이션
- 대용량 Connection 관리

33. 5.1 공유 데이터 관리
WAS$의$Cluster$는$장애$/$성능$저하의$주된$원인$!
Tomcat$의$session$replication$:$DeltaManager$or$BackupManager
해결책$중$하나$-$NAVER$의$TripleS$(Shared$Session$System)
http://helloworld.naver.com/helloworld/233847
?

34. 5.1 공유 데이터 관리
를"활용한"공유"데이터"(session)"공유하기
ZooKeeper™
Redis Cluster Manager
create/delete
redis"node
redis"서버"목록을"Ring"에
주가하고"동기화"한다.
watching
heartbeat
redis"상태"체크!!
.".".
Reverse"Proxy
server
Consistent"Hashing
shard 1
.".".
get"/"set"session"datas
shard 2
.".".
<"servers">
shard 3

35. 5.1 공유 데이터 관리
공유하지%않는%것%(shared%nothing)%이%가장%좋은%방법
!!
<%servers%>
Reverse%Proxy
server
체팅방명(키)이%hash%key
HASH%(“체팅방%A”)
HASH%(“체팅방%B”)
Consistent%Hashing
이%서비스%서버에%“체팅방%A”%에%입장한
모든%접속자%목록은%Local%메모리에%저
장하고%있다.

36. 5.2 모듈 기반 어플리케이션
var!http!!!!!=!require('http');
vertx.deployModule("webCserverCv1.2",!config,!1);
var!sender!!!=!require('./lib/messageSenderMod');
vertx.deployModule("modCmessageSenderCv0.1",!config,!1);
var!receiver!=!require('./lib/messageReceiverMod');
vertx.deployModule("modCmessageReceiverCv0.1",!config,!1);
var!auth!!!!!=!require('./lib/userAuthMod');
vertx.deployModule("modCuserAuthCv0.1",!config,!1);
var!buddy!!!!=!require('./lib/buddyMgrMod');
vertx.deployModule("modCbuddyMgsCv0.1",!config,!1);
.&.&.&.&.&.&.&.&.&.&.
.&.&.&.&.&.&.&.&.&.&.
서버의$기능들을$분리해$구현한$모듈들