A short talk I gave to R&D department in Digimax, Inc. It's dedicated to briefly describe how to parallelize your routine work in animation industry by making use of some tools.

1. 平行化你的工作
Part 1
Drake Guan
Digimax, Inc.
2013/06
(http://about.me/drakeguan)

2. http://en.wikipedia.org/wiki/Parallel_computing
https://en.wikipedia.org/wiki/Distributed_computing

3. Parallel instead of Distributed
http://bit.ly/17WQ0gR

4. Scope
! GNU Parallel
! Parallel Python (pp)
! Alfred
What can you expect? It’s
just a part 1. Actually, it is
animation studio oriented.
Creativity is more important
than simply techniques.

5. GNU Parallel
GNU parallel is a shell tool for executing jobs in parallel using one or
more computers.
http://www.gnu.org/software/parallel/

6. http://www.gnu.org/software/parallel/

7. Online Doc Examples 是你的好朋友
http://www.gnu.org/software/parallel/

8. 你只要記得這兩個範例就行了
http://www.gnu.org/software/parallel/man.html#example__working_as_xargs__n1__argument_appending

9. 範例1: 轉檔
# straightforward but sequential way
~: for i in *.exr; do convert $i –depth 8 –alpha off –gamma 2.2 –quality 100
$(basename $i .exr).jpg; done
# parallel way
~: ls *.exr | parallel do convert {} –depth 8 –alpha off –gamma 2.2 –quality
100 {.}.jpg
• 8 bits: -depth 8
• No alpha: -alpha off
• Gamma encoding: -gamma 2.2
• JPEG quality: -q 100

10. 範例2: 更改檔名
~: parallel mv {} {.}.html ::: *.htm
~: find /shows/qq/mov/ -name ‘*.avi’ | parallel mv {} {.}.mov
~: find . | parallel mv {} backup_{}
~: parallel mv {} {.}_backup.jpg ::: *.jpg

11. 範例4: 瘋狂 grep
RenderMan Studio 的文件很多，你想找出裏頭僅存的 Alfred 資料.
~: find /rms_doc/ -type f | parallel –k –n 1000 grep –H –n Alfred {}
• File only: -type f
• Keeping order: -k
• Multiple arguments: -n 1000

12. Parallel Processing and
Multiprocessing in Python
http://wiki.python.org/moin/ParallelProcessing

13. 多到爆的模組，看了就不想用!

14. Parallel Python (pp)
http://www.parallelpython.com/

15. Quick start guide 是我們的好朋友

16. mkmov: images -> video
! Supering: logo, footage info, author, date, frame count, …
! Resizing: to the extend of 16 if necessary (player’s constraint)
! Gamma encoding
! HDR -> LDR
! Movie maker

17. mkmov code snippets

18. 你要懂的 pickling
! Python 先天上有 GIL (Global Interpreter Lock) 的限制：一
個 thread 要被 Python interpreter 執行，得先要個這個 lock.
! Parallel Python and others, 都是走 multi process (or fork).
! Picking: 醃製/醬瓜，用來保存食物的方法(酸，但可以吃)
! Python Pickle: 用來保存(serializing)記憶體狀態的方法
! Parallel Python modules 幾乎都用 pickle 來處理平行化時的
message passing.

19. Alfred

20. Alfscript

21. dgAlfred
! 讓每個人都可以透過 alfred 使用我們的 render farm.
! dgAlfred 幫你翻譯(產生) alfscript.
! 所有 alfscript 的語法都支援. 每個指令都有個 class 的對應.
! 是 ypcat 的傑作! 程式碼非常短!

23. dgalfred
-> genTask(…)

24. Applications of dgalfred
! rfMaya.py
! rfNuke.py
! rfShake.py
! rfMkDoremi.py
! rfMopBake
! rf -> render farm

25. That’s Part 1.