AWS 클라우드는 단순하고 작은 규모의 서비스 초기부터 사용자가 점차 증가되는 과정에 따라 다양하게 선택할 수 있는 서비스 아키텍처를 제공해 드리고 있습니다. 소규모 Amazon SageMaker 도입과 PoC부터 천만 사용자 대상 서비스로 비즈니스가 확장되었을 때 AI/ML 서비스 개발팀이 조직구조, 아키텍처, ML 운영 관리에 어떤 고민이 필요한지 이야기하고, 규모의 변화에 맞는 Amazon SageMaker 활용과 모범 사례 등을 소개합니다.

1. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S E O U L | M A Y 4 , 2 0 2 3

2. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
SageMaker
Solutions Architect
AWS
AI/ML Expert Solutions Architect
AWS

3. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AI/ML
1 1,000 10,000 100,000 500,000 1000,000
+
AI
3
↑ ↑ ↑
↑

4. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AI/ML
“AI/ML
AWS ?”
”AI/ML
?”
4

5. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
•
•
•
A yak painted by Gogh, generated by Stable Diffusion

6. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Day 1

7. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
>1

8. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
User
Instance database
Product Manager
Business Analyst
Data engineer
ML engineer
Developer
CTO
•
•
&
AI/ML
1
2
1
8

9. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AI/ML
56%
AI
-
1,600x
2019 ~2022
(
), AI
75%
2024 AI
-
100,000+
9

10. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AI/ML
10

11. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AI/ML
•
• /
•
•
•
A man at a computer with a lot of problems painted by Gogh,
generated by Stable Diffusion

12. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
•
• /
•
• ML
Engineers in front of white board in meeting room painted by Gogh,
generated by Stable Diffusion

13. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
> 1000

14. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AZ 3Tier
• AZ
( )
• S3
• ,
Data Engineer
Developer
1
2
3
Amazon S3
Amazon Athena
VPC
User
Instance
DB(replica)
Load balancer
Availability Zone Availability Zone
Instance
DB(master)
Instance
Instance
Auto Scaling group
1
2
AI/ML
14

15. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
!
15

16. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
/
s Data
Amazon S3
Virtual Private Cloud
S3 Pipeline
(Exploratory
Data Analysis)
(ad-hoc)
Athena queries
Analytics
1
2
EC2 Storage
Notebook AMI
Instance Data
File system
3
16

17. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
• Structure
vs Unstructured
• vs
•
•
Amazon ECR Amazon S3 Amazon EMR

18. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS
/ Lo g
3 r d Party
IOT /
D W/ D a t a L a k e
Amazon
Aurora
Amazon Kinesis
& Amazon MSK
Amazon S3
Amazon Redshift
Amazon
Redshift
Amazon
EMR
Amazon
QuickSight
Amazon
SageMaker
Amazon
DynamoDB
AWS Glue
AWS Lake Formation, Amazon DataZone
& |
18

19. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
>10,000

20. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
• S3
• AI/ML
•
Data Analyst
Developer
Data Scientist
1
2
3
AI/ML
20

21. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
No code / Low code ML on AWS
Amazon SageMaker Canvas
PREPARE
데이터 탐색, 전처리, 피처 엔지니어링
BUILD
모델 개발
TRAIN & TUNE
모델 학습 및 최적화
DEPLOY & MANAGE
실시간 또는 배치 추론:
모델 호스팅 및 모니터링
Amazon SageMaker Data Wrangler
Amazon SageMaker Autopilot
Amazon SageMaker JumpStart
다양한 배포옵션
데이터 엔지니어 또는 과학자, ML 엔지니어
데이터 분석가 또는 비즈니스 분석가
21

22. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
SageMaker Data Wrangler
Spark Serverless processing job
No Code / Low code ,
300+ ,
SageMaker
,
, ,
Schedule Job, Feature Store, Workflow
SageMaker
22

23. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
SageMaker Canvas
UI
AutoML
SageMaker
SageMaker Canvas
23

24. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
A I 서 비 스
코드 개발
Amazon CodeGuru
Amazon DevOps Guru
비지니스
Amazon Personalize
Amazon Forecast
검색
Amazon Kendra
산업용 AI
Amazon Monitron
Amazon Lookout for Equipment
헬스케어
Amazon HealthLake
Amazon Comprehend Medical
Amazon Transcribe Medical
Amazon Fraud Detector
Amazon Lookout for Metrics
Amazon Lookout for Vision
산업 특화
핵심
챗봇
Amazon Lex
텍스트 및 문서
Amazon Translate
Amazon Comprehend
Amazon Textract
음성
Amazon Polly
Amazon Transcribe
Amazon Transcribe Call Analytics
컴퓨티 비전
Amazon Rekognition
AWS Panorama
Amazon CodeWhisperer
Realtime
Face Matching
A M A Z O N
S A G E M A K E R
M L 프 레 임 워 크 및
인 프 라
PyTorch, Apache
MXNet, TensorFlow
Amazon EC2 CPUs GPUs AWS Trainium
Amazon Elastic
Inference
AWS Inferentia FPGA
Habana
Gaudi
엣지
디바이스
관리
데이터
준비
피처
스토어
편향성
탐지
노트북
기반 개발
관리 및
모니터링
원클릭 모델
학습
모델 배포
운영
파라미터
튜닝
모델 테스트
예측
CI/CD
데이터
라벨링
노코드
비지니스
분석
CANVAS
무료
ML 테스트
및 학습
STUDIO
LAB
STUDIO IDE
GROUND
TRUTH
Geospatial ML
Stability AI Models
ML 거버넌스 기능 (Role Manager, Model
Cards, Model Dashboard)
기본 제공 데이터 준비, 실시간 협업
및 자동화된 배포
The AWS ML Stack
24

25. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
>100,000

26. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
• ,
•
• AI/ML
Data Analyst
Developer
Data Scientist
1
2
ML engineer
AI/ML
3
26

27. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
SageMaker
Developer
Data Scientists
End-users
SageMaker
Training
SageMaker
Hosting
Amazon ECR
Amazon
SageMaker
Amazon S3 Amazon
CloudFront
Amazon
API Gateway
AWS
Lambda
27

28. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
SageMaker JumpStart
350 , STATE-OF-THE-ART ,FOUNDATION
TASKS A lgorithms /M o del
s
VISION
TEXT
TABULAR
Audio
28

29. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
ML Governancetools
SageMaker
SageMaker
Role Manager
SageMaker
Model Cards
,
SageMaker
Model Dashboard
29

30. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
>500,000

31. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
, X 30 …
• AI/ML
,
•
/
1
Developers
Data Engineers
Data scientists
ML Engineers
2
Analytics
Data
Engineer
Data Scientist ML Engineer
/ Developer
Amazon SageMaker
EndPoint
Production
AI/ML
31

32. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker Pipelines
/ MLOps
SageMaker Projects
Build, train
deploy
SageMaker
Model Registry
,
,
&
SageMaker Traning Pipelines
Data
Evaluate
Train
Register
Data
Evaluate
Train
Register
Amazon
API
Gateway
Elastic
Load
Balancing
Client
Mode
l
End
point
Auto
Scaling group
AZ1
AZ2
AZ3
ML
32

33. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
SageMaker Project
Amazon SageMaker Pipelines
/ MLOps
CloudFormation
Template
SageMaker
Project
AWS Service
Catalog
AWS
CloudFormation
Code
Pipeline
SageMaker Projects
Build, train
deploy
ML
SageMaker
Studio Role
SageMaker
Studio
CodeBuild
Configration
CodeCommit
Artifact Repo
Model
Registry
33

34. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
SageMaker Projects
SageMaker
Model
Registry
SageMaker Pipelines
Build, train, deploy
OU Model Dev, DS1
OU Production, STG
OU Production, PRD
Organizations
Root
OrganizationalUnit
Model Dev
Account Account
Account
Organizationalunit
Production
Account
Account
OU Model Dev, DS2
34
Organizations
Root
OrganizationalUnit
/Account

35. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
>1,000,000

36. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
NLP/
• ,
•
1
2
/ / /
“ ”
Developers
Data Engineers
Data scientists
ML engineers
ML Operators
AI/ML
36

37. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon
S3
FSx
for Lustre
Dataset
Notebook Amazon
S3
…..
Dataset
Amazon ECR
( )
Model
Registry
Endpoint
Training
container
image
400 Gbps GPUDirect
RDMA(Remote Direct
Memory Access)
Elastic Fabric
Adapter(EFA)
•
SageMaker
Distributed Training Lib
(~60% )
•
Amazon SageMaker
1
2
1
trn1n trn1n trn1n
2
37

38. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker
GPU Out-of-memory error
38
Data record
Data record
Data record
Data record
Dataset
Data record
Data record
Data record
Data record
Data record
Data record
Data record
Data record
Data record
…
Data record
Data record
Dataset k(batch)
Data record
(Data parallelism)
1.1 (instance 0)
2
L11(gpu0)
L12 (gpu1)
L13 (gpu2)
L14 (gpu3)
L21(gpu4)
L22 (gpu5)
L23 (gpu6)
L24 (gpu7)
1.n (instance n)
5 6
L51(gpu0)
L52 (gpu1)
L53 (gpu2)
L54 (gpu3)
L61(gpu4)
L62 (gpu5)
L63 (gpu6)
L64 (gpu7)
…
1
(Tensor
parallelism)
k.1 (instance 0)
2
L11(gpu0)
L12 (gpu1)
L13 (gpu2)
L14 (gpu3)
L21(gpu4)
L22 (gpu5)
L23 (gpu6)
L24 (gpu7)
k.n (instance n)
5 6
L51(gpu0)
L52 (gpu1)
L53 (gpu2)
L54 (gpu3)
L61(gpu4)
L62 (gpu5)
L63 (gpu6)
L64 (gpu7)
…
1
GPU
(Model parallelism)
Out of Memory

39. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
SageMaker
Amazon S3
Endpoint
Amazon SageMakerEndpoint
ML Instance
Multi Model Endpoint Container
Users
•
• CPU GPU
1
2
39

40. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
+

41. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
ML
41
Celebration party with colleagues at night with stars painted by Gogh,
generated by Stable Diffusion

42. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
ML
•
! ( )
•
• API AI/ML
1
2
AI/ML
30ms
일요일 월요일 화요일 수요일 목요일 금요일 토요일
42
3

43. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
A/B ,
•
Auto Scaling
• A/B
,
•
1
2 Input Data
(Request)
Prediction
(Response)
Auto Scaling group
Amazon SageMaker Endpoint
Model
Endpoint
A
ML
GPU
ProductionVariant1, Weight0.7
B
ML
GPU
ProductionVariant2, Weight0.3
C
ML
inf2
Shadow Variant
70%
30%
Cloned
100%
Request
1
3
2
3
Amazon S3
Response
AZ1
AZ2
AZ3
43

44. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon
GPU 70%
44
AWS Inferentia
(DL)
Amazon EC2
70%
AWS Trainium
diffusion
Amazon EC2
50%
AWS Inferentia2
diffusion
/
Amazon EC2
40%

45. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
3
•
AWS AI NoCode, LowCode
( , , )
( , , white paper) (On/Off , )
•
Amazon SageMaker
• AI/ML, AWS
45

46. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
NoCode, LowCode
AI
AI/ML !
NoCode, LowCode
?
2
Session 6 17:20
AI
AWS AI/ML
2
Session 4 17:20
46

47. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.