Can We Benefit from Solid State Drives in Rich Multimedia Content Processing, Storage and Streaming?

.

.

The Big Picture: YALMS

M.Zhanikeev -- maratishe@gmail.com

Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024

2 /17
2/17

.

.

YALMS: Yet Another LMS
• feature:

input, processing, and delivery in equal shares 17
HTML5
Viewers

Step 1: Record and Stream

A

Creator

S

V

C

Screen
Record
Streaming

Step 2: Annotate

HTML5
Muxer

Content
Server

S
Lecturer
Autoannotate

17 M.Zhanikeev+1 "YALMS: Yet Another Lecture Management System..." IEICE・ET研 (July 2013)

3 /17
3/17

.

.

A Business Day at YALMS
screen and video streams uploaded by users 15
• many concurrent video annotation tasks (PDF + VIDEO) 16
• constantly changing popularity distribution of content, needs
dynamic relocations
• many

15 "YALMS Project Home" https://github.com/maratishe/yalms (June 2013)
16 "YALMS Annotation Robot (software behind this paper)" https://github.com/maratishe/yalms.annotation (September 2013)


4 /17
4/17

.

.

YALMS Topics



5 /17
5/17

.

.

YALMS: The Big SSD Question
.
The Question is
.
... can such a rich

multimedia architecture benefit from SSDs?

• WHY1: SSDs have properties
◦ read/write, random/sequential
• WHY2: SSDs are becoming
• WHY3:

different from those of HDDs

cheaper (MLC) so why not to use them?

parallel/multicore processing is not kind to HDDs but SSDs

can cope
• WHY4:

50Mbytes/s throughput is not enough any more



6 /17
6/17

.

.

Where Are We Now? Existing
Research



7 /17
7/17

.

.

Existing Research (1)
• roughly, two groups:
1.

SSD versus HDD research

• filesystem abstraction and scheduling -- load balancing between SSD and HDD 01
• SSD as cache memory 0203

2.

parallel processing and the multicore
• job graph partitioning 18
• local Big Data processing 19
• multicore streaming 11 -- 14

01 J.Kim+4 "Disk Schedulers for Solid State Drives" 7th ACM EMSOFT (2009)
02 S.Rizvi+1 "Flash SSD vs HDD: High Performance .... Multimedia Storage Systems" 2nd ICCET (2010)
03 Y.Lee+1 "Mapping granularity and performance tradeoffs for solid state drive" The Journal of Supercomputing (2013)
18 X.Sui+3 "Parallel Graph Partitioning on Multicore Architectures" 23rd LCPC (2010)
19 R.Chen+2 "Tiled-MapReduce: Optimizing Resource Usages of Data-parallel Applications on Multicore..." 19th PACT (2010)
11 Y.Wei+3 "Energy-Efficient Real-Time Scheduling of Multimedia Tasks on Multi-Core Processors" ACM SAC (2010)
14 V.Galiano+3 "SSD architecture such as multi-channel and multi-way" EURASIP Journal ... Signal Processing (2013)


8 /17
8/17

.

.

Existing Research: What is Missing?
.
In a Nutshell, Existing Research is...
.
• extremely specialized and is built for one application only
.

• ... or, in business, the use is

very primitive -- a temporary memory buffer

• ... at the same time.. YALMS needs
◦ one design fits all

generic load balancing

• changes along the timeline should cause changes --

dynamic optimization

• ...



9 /17
9/17

.

.

The YALMS.hiperf project



10 /17
10/17

.

.

Meet the YALMLS.hiperf Project
pool of SSDs,
HDDs, etc.

• a

Manage

Cores
Cores
Cores

• a separate pool of
Viewers
Multimedia
System

Creators


SSDs
HDDs
HDDs

CPU cores

• steady arrival and

departure rates of
HDDs
HDDs
HDDs

tasks/processes


11 /17
11/17

.

.

YALMS.hiperf : The Overall Design
A Multimedia System

YALMS
.hiperf
Robot

Process
Process
Process
Process
Process
Process
Process
Req
(details)

YALMS.hiperf
Agents
Agents
Agents
Agents
Agents
SSDs
HDD
SSDs
HDD
SSDs
HDD
SSDs
HDD
SSDs
HDDs
SSDs


• a

Rep
(agent)

firmware approach

• tasks/processes send
•

requests
replies point to agents

….


12 /17
12/17

.

.

YALMS.hiperf : The Basic Unit
Multicore

Input

Processing
logic

Fork

parallel
processing

• traditional

Read/write?
Seq./rand.?
(Chunk) size

formulation

Next
step

ENGINE

SSD

HiPerf/Normal?
Longer plans?

HDD

Join

•

engine performs
runtime

optimization
…

• tasks are assigned

Network
storage

classes

01 J.Kim+4 "Disk Schedulers for Solid State Drives" 7th ACM EMSOFT (2009)
02 S.Rizvi+1 "Flash SSD vs HDD: High Performance .... Multimedia Storage Systems" 2nd ICCET (2010)


13 /17
13/17

.

.

YALMS.hiperf : Runtime Optimization
.
Basic Formulation
.
start from zero, collect
optimize accordingly
.

history, then

• tasks should naturally have the

few

heavy hitters distribution
◦ if not, tasks should be partitioned

• then simply

minimize

∑ ∑

lagi,j

(1)

i∈tasks j∈time

• ... details in future publications

03 Y.Lee+1 "Mapping granularity and performance tradeoffs for solid state drive" The Journal of Supercomputing (2013)


14 /17
14/17

.

.

Wrapup



15 /17
15/17

.

.

How Far is The Project?
• software for multicore video
◦ mostly reads, very few writes
• currently working on the

VLC
ffmpeg

screen
video

annotation on top of SSD

generic version of software, will release soon

scenes

cleanup
I-Frames

I-Frames

End product
slices

ANNOTATION
LaTeX/Beamer
Keynote
PowerPoint
…

Slides
PDF


split

timestamps
PNGs

pdf2png
pages

PNGs


16 /17
16/17

.

.

That’s all, thank you ...



17 /17
17/17

.

.

[01] J.Kim+4 (2009)
Disk Schedulers for Solid State Drives
7th ACM EMSOFT
[02] S.Rizvi+1 (2010)
Flash SSD vs HDD: High Performance .... Multimedia Storage Systems
2nd ICCET
[03] Y.Lee+1 (2013)
Mapping granularity and performance tradeoffs for solid state drive
The Journal of Supercomputing
[04]

(2012)
Data Compression in the Intel Solid-State Drive 520 Series
Intel Technology Brief

[05]

(2013)
Cisco Content Delivery Engine 250
Cisco Data Sheet



17 /17
17/17

.

.

[06] B.Sajadi+3 (2011)
...SSDs for Large-Scale Interactive Graphics Applications
Symp. on 3D Graphics and Games (I3D)
[07] D.Zhao+1 (2013)
HyCache: a User-Level Caching Middleware for Distributed File Systems
ACM HPDC
[08] M.Ryu+2 (2011)
Impact of Flash Memory on Video-on-Demand Storage: Analysis of Tradeoffs
2nd ACM MMSys
[09]

(2013)
Performance ... On Demand
Concurrent Computer Corporation (STEC), Case Study

[10] M.Ryu+2 (2012)
Why ...flash-based multi-tiered storage systems performing poorly for HTTP video
streaming?
22nd NOSSDAV


17 /17
17/17

.

.

[11] Y.Wei+3 (2010)
Energy-Efficient Real-Time Scheduling of Multimedia Tasks on Multi-Core
Processors
ACM SAC
[12] S.Sankaraiah+3 (2011)
GOP Level Parallelism on H.264 Video Encoder for Multicore Architecture
IPCSIT
[13] T.Dias+2 (2010)
H.264/AVC framework for multi-core embedded video encoders
SoC
[14] V.Galiano+3 (2013)
SSD architecture such as multi-channel and multi-way
EURASIP Journal ... Signal Processing
[15]

(June 2013)
YALMS Project Home
https://github.com/maratishe/yalms



17 /17
17/17

.

.

[16]

(September 2013)
YALMS Annotation Robot (software behind this paper)
https://github.com/maratishe/yalms.annotation

[17] M.Zhanikeev+1 (July 2013)
YALMS: Yet Another Lecture Management System...
IEICE・ET研
[18] X.Sui+3 (2010)
Parallel Graph Partitioning on Multicore Architectures
23rd LCPC
[19] R.Chen+2 (2010)
Tiled-MapReduce: Optimizing Resource Usages of Data-parallel Applications on
Multicore...
19th PACT



17 /17
17/17

Can We Benefit from Solid State Drives in Rich Multimedia Content Processing, Storage and Streaming?

Recommended

Recommended

More Related Content

Similar to Can We Benefit from Solid State Drives in Rich Multimedia Content Processing, Storage and Streaming?

Similar to Can We Benefit from Solid State Drives in Rich Multimedia Content Processing, Storage and Streaming? (20)

More from Tokyo University of Science

More from Tokyo University of Science (20)

Recently uploaded

Recently uploaded (20)

Can We Benefit from Solid State Drives in Rich Multimedia Content Processing, Storage and Streaming?