3D mathematics OpenGL ES 2.0 overview Vertex/Fragment shader basic understanding Further study.

1. Opening ES 2.0/3.0 Programming
Introduction
Champ Yen
http://champyen.blogspot.com
1

2. Outline
●
3D mathematics
●
OpenGL ES 2.0 overview
●
Vertex/Fragment shader basic understanding
●
Further study.
02/07/14
2

3. Basic Matrix/Vector Review
●
Vector Notation
●
Vector Operations
●
Matrix Notation
●
Matrix Operations
●
Orthogonality
●
Right-Handed Coordinate
●
Transformation/Translation
02/07/14
3

4. Vectors - Notation
1, 2, 3
1
2
3
02/07/14
row vector
column vector
a
b
4

5. Operations of Vectors
●
●
●
●
Negate (vector with opposite direction)
Magnitude (length of a vector)
Multiply by a scalar (scale length)
Normalize (vector with same direction, length = 1)
●
+, -, distance
●
Dot, Cross
02/07/14
5

6. Operations of Vectors - Dot
Geometric meaning
Dot product tells the similarity between
two vectors.
02/07/14
6

7. Operations of Vectors - Cross
Geometric meaning
Cross product yields a vector that is perpendicular
to the original two vectors.
02/07/14
7

8. Matrices - Notation
Mr×c
row
m11 m12 m13 … m1(c-1) m1c
m21 m22 m23 … m2(c-1) m2c
m31 m32 m33 … m3(c-1) m3c
………………………...
m(r-1)1 m(r-1)2 m(r-1)3 … m(r-1)(c-1) m(r-1)c
mr1 mr2 mr3 …. mr(c-1) mrc
column
02/07/14
8

9. Matrices Operations
●
Identity matrix
●
Transposition (for Mrc, MTcr, mTij = mji)
●
Multiply with a scalar
●
Multiply two matrices
●
Multiply a vector with a matrix
●
Multiply a matrix with a vector
02/07/14
9

10. Orthogonality
●
●
Vector : a, b are orthogonal if a dot b = 0
Matrix : matrix m is orthogonal if m*mT = I => MT is M’s
inverse Matrix (M-1)
02/07/14
10

11. Coordinate – Right-Handed
+y
+x
+z
02/07/14
11

12. Linear Tranformation
+y
+y
’
A linear transformation
preserves straight and parallel
lines, and there is no
translation—the origin does
not move.
Applications:
 Rotation
+x  Scaling
 Reflection
…..
+z
’
+z
02/07/14
+x
’
12

13. 2D Rotation
The coordinate of P’ in XY space is:
(x, y) => (x*cos + y*(-sin), x*sin + y*cos)
+y’
+y
P’
p
+x’
θ
02/07/14
cos -sin
sin cos
x
y
+x
13

14. Rotation in 3D
Mx=
1
0
0
My=
cos
0
-sin
Mz =
0
0
cos -sin
sin cos
0
1
0
cos -sin
sin cos
0
0
Rotation around X axis
sin
0
cos
Rotation around Y axis
0
0
1
Rotation around Z axis
v’ = Mv
02/07/14
14

15. Translation
+y
P’ - P =
P’
P
+x
P’ =
+z
02/07/14
dx
dy
dz
1
0
0
0
0
1
0
0
0
0
1
0
dx
dy
dz
1
P
1
15

16. Combination – Transformation & Translation
Rotation then Translation
1
0
0
0
0
1
0
0
0
0
1
0
dx
dy
dz
1
cos -sin 0
sin cos 0
0
0
1
0
0
0
0
0
0
1
x
y
z
1
1
0
0
0
dx
dy
dz
1
x
y
z
1
Translation then Rotation
cos -sin 0
sin cos 0
0
0
1
0
0
0
0
0
0
1
0
1
0
0
0
0
1
0
Yes! We can do multiple things in ONE matrix.
02/07/14
16

17. What is OpenGL ES?
●
OpenGL-based API for Embedded Systems
●
Removed redundant and expensive functionality
●
Kept as compatible with OpenGL as possible
●
Added features needed for embedded systems
02/07/14
17
http://developer.amd.com/media/gpu_assets/GDCMobile2006-Ginsburg-OpenGLES2.0.pdf

18. OpenGL ES 2.0 programming
●
●
OpenGL ES 1.1/2.0 pipeline
initialization

●
setup



●
window system
geometry
shader
texture
per-frame rendering


02/07/14
uniform
drawing primitives
18

19. OpenGL (ES) 1.x Pipeline
Triangles/Lines/Points
API
API
Primitive
Primitive
Processing Vertices
Processing
Vertex
Vertex
Buffer
Buffer
Objects
Objects
Alpha
Alpha
Test
Test
02/07/14
Transform
Transform
and
and
Lighting
Lighting
Primitive
Primitive
Assembly
Assembly
Rasterizer
Rasterizer
Texture
Texture
Environment
Environment
Colour
Colour
Sum
Sum
Fog
Fog
Depth
Depth
Stencil
Stencil
Color
Color
Buffer
Buffer
Blend
Blend
Dither
Dither
http://www.khronos.org/opengles/2_X/
Frame Buffer
19

20. OpenGL (ES) 2.0 Pipeline
Triangles/Lines/Points
API
API
Primitive
Primitive
Processing Vertices
Processing
Vertex
Vertex
Shader
Shader
Vertex
Vertex
Buffer
Buffer
Objects
Objects
Primitive
Primitive
Assembly
Assembly
Rasterizer
Rasterizer
Fragment
Fragment
Shader
Shader
Depth
Depth
Stencil
Stencil
Color
Color
Buffer
Buffer
Blend
Blend
Dither
Dither
Frame Buffer
This isn’t your father’s OpenGL! (*)
02/07/14
*) OpenGL SuperBible 4/E
20

21. OpenGL ES 2.0 Overview
●
Shader only
●
Not backwards compatible to 1.x
●
ES GLSL forms core of API
●
●
Shifts some burden to application But puts more power in
your hands
Convergence of desktop and handheld!
02/07/14
21
http://developer.amd.com/media/gpu_assets/GDCMobile2006-Ginsburg-OpenGLES2.0.pdf

22. Initialize
●
●
●
EGL
Play the same role as glx/wgl/cgl.
glDepthRange(n, f);
Specify the depth of window coordinate.
glViewport(x, y, w, h);
Specify the bottom-left corner and display width/height of
window coordinate.
02/07/14
22

23. What you see may not be what you think.
Normalized Device Coordinate
(-1, 1, 1)
Window Coordinate
(1, 1, 1)
glViewPort
glDepthRange
(0, h, 1)
(0, h, 0)
(w, h, 1)
(w, h, 0)
Viewport
Transform
(-1, -1, 1)
(1, -1, 1)
(0, 0, 1)
(0, 0, 0)
(-1, -1, -1)
02/07/14
eye
(1, -1, -1)
(w, 0, 1)
(w, 0, 0)
eye
23

24. What’s shader?
●
●
●
●
a set of software instructions which is used primarily to
calculate rendering effects. (*)
Apply to each vertex or pixel.
Expose the capability to program a specific stage in
graphics pipeline.
Leverage CPU burden to GPU.
02/07/14
*) http://en.wikipedia.org/wiki/Shader
24

25. Vertex & Fragments
02/07/14
25

26. Shader Create/Compilation
GLuint LoadShader(GLenum type, const char *shaderSrc)
{
GLuint shader;
GLint compiled;
// Create the shader object, type must be GL_VERTEX_SHADER or GL_FRAGMENT_SHADER
shader = glCreateShader(type);
if(shader == 0)
return 0;
// Load the shader source
glShaderSource(shader, 1, &shaderSrc, NULL); // Compile the shader
glCompileShader(shader);
// Check the compile status
glGetShaderiv(shader, GL_COMPILE_STATUS, &compiled);
if(!compiled){
………
glDeleteShader(shader);
return 0;
}
return shader;
}
02/07/14
26

27. Program Create/Link
// Create the program object
programObject = glCreateProgram();
if(programObject == 0)
return 0;
glAttachShader(programObject, vertexShader);
glAttachShader(programObject, fragmentShader);
// Link the program
glLinkProgram(programObject);
// Check the link status
glGetProgramiv(programObject, GL_LINK_STATUS, &linked);
if(!linked){
……
glDeleteProgram(programObject);
return FALSE;
}
…
// Use the program object
glUseProgram(programObject);
02/07/14
Linked Program
Vertex
Shader
Object
Fragment
Shader
Object
27

28. Vertex Shaders
output to fragment shader
02/07/14
28

29. GLSL
●
OpenGL Shading Language
●
C-Like Grammar
●
consistent version # with OpenGL since 3.3
●
GLSL ES 1.00 is based on GLSL 1.20
02/07/14
29

30. GLSL ES – Basic Types
vector/matrix types
textures
02/07/14
30
http://www.khronos.org/registry/gles/specs/2.0/GLSL_ES_Specification_1.0.17.pdf

31. Vertex Shader - Example
// uniforms used by the shaders
uniform mat4 u_mvp; // matrix to convert P from
// model space to clip space.
// attributes input to the vertex shader
attribute vec4 a_position; // input position value
attribute vec4 a_color;
// input vertex color
// varying variables – input to the fragment shader
varying vec4 v_color;
// output vertex color
void main()
{
v_color = a_color;
gl_Position = u_mvp * a_position;
}
02/07/14
31

32. Vertex Shader
Attributes: position, color,
normal, texture coordinate
… etc
Uniform: transform matrix,
light direction … etc
per-vertex info.
global info.
(0, 0.5, 0)
(-0.5, -0.5, 0)
02/07/14
(0.5, -0.5, 0)
32

33. Vertice Attributes
/* vertex without VBO, copy to GPU before
rendering each time*/
glEnableVertexAttribArray(vtxIdx);
glVertexAttribPointer(vtxIdx, vtxSize,
GL_FLOAT, GL_FALSE, vtxStride,
vtxBuf);
……
glBindAttribLocation(programObject,
vtxIdx, “a_position”);
glDrawElements(GL_TRIANGLES,
numIdx, GL_UNSIGNED_SHORT,
indices);
02/07/14
/* vertex with VBO, stored in GPU side */
glGenBuffer(1, &vboIds);
glBindBuffer(GL_ARRAY_BUFFER,
vboIdx);
glBufferData(GL_ARRAY_BUFFER,
vtxStride*numVtx, vtxBuf,
GL_STATIC);
offset = 0;
glEnableVertexAttribArray(vtxIdx);
glVertexAttribPointer(vtxIdx, vtxSize,
GL_FLOAT, GL_FALSE, vtxStride,
(void*)offset);
……
glBindAttribLocation(programObject,
vtxIdx, “a_position”);
glDrawElements(GL_TRIANGLES,
numIdx, GL_UNSIGNED_SHORT,
indices);
33

34. Primitives & Rasterization
02/07/14
34

35. Uniforms
GLint location;
GLfloat mvp[16];
…… (calculate ModelViewProjection matrix)
location = glGetUniformLocation(programObject, “u_mvp”);
glUniform4fv(location, mvp);
……
103/02/07
35

36. Coordinate Transformation
world space
e
ey
eye space
model space
View Matrix
Rotate
Translation
Model Matrix
Rotate/Scaling
Translation
Perspective
Projection
103/02/07
Projection
Matrix
(Mproj * (Mview * (Mmodel * vertex))) = (Mproj * Mview * Mmodel) * vertex
= Mmvp * vertex
NDC, clip coordinate
36

37. View Matrix(cont.)
world space’ = world space – Peye
+y
+y’
+y
+x’
+z’
world space
e
ey
+x’
+y’
+z’
Translation
ey
e
+x
+x
+z
+z
103/02/07
37

38. View Matrix(cont.)
+y
M
+y’
X’
Y’
Z’
=
X
Y
Z
+x’
+z’
world space’
I
e
ey
+x
=
-1
∴
M=
X’
Y’
1
0
0
Z’
=
T
X’
Y’
Z’
0
1
0
0
0
1
X’
=
Y’
Z’
103/02/07
38

39. Perspective Projection Matrix
103/02/07
http://www.songho.ca/opengl/gl_projectionmatrix.html
http://read.pudn.com/downloads130/doc/556616/perspective.doc
39

40. View Matrix
Mview
X’
Y’
Z’
0 0 0
0
0
0
1
Mview =
103/02/07
1
0
0
0
0
1
0
0
0
0 -Peye
1
0 1
xw
yw
zw
1
=
xe
ye
ze
1
X’
-dot(X’,Pe)
Y’
-dot(Y’,Pe)
Z’
-dot(Z’,Pe)
0 0 0
1
40

41. Perspective Projection Matrix
103/02/07
http://www.songho.ca/opengl/gl_projectionmatrix.html
http://read.pudn.com/downloads130/doc/556616/perspective.doc
41

42. Fragment Shader
Color of a pixel in a fragment
103/02/07
42

43. Fragment Shader - Example
varying vec4 v_color;
// input vertex color
void main(void)
{
gl_FragColor = v_color;
}
103/02/07
43

44. Fragment Shader
The varying inputs of fragment shader from vertex shader
is interpolated for each pixel of a fragment.
103/02/07
44

45. Per-fragment Lightinng
Per-vertex Lighting
Per-fragment Lighting
103/02/07
45
http://www.opengl.org/discussion_boards/ubbthreads.php?ubb=showflat&Number=222753

46. Textures – 2D Texture
// Generate a texture object
glGenTextures(1, &textureId);
// Bind the texture object
glBindTexture(GL_TEXTURE_2D, textureId);
// Load the texture
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, width, height,
0, GL_RGB,GL_UNSIGNED_BYTE, pixels);
// Set the filtering mode
glTexParameteri(GL_TEXTURE_2D,
GL_TEXTURE_MIN_FILTER, GL_NEAREST);
glTexParameteri(GL_TEXTURE_2D,
GL_TEXTURE_MAG_FILTER, GL_NEAREST);
…..
// Get the sampler locations
location = glGetUniformLocation(programObject, "s_texture");
// …
// Bind the texture
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, textureId);
// Set the sampler texture unit to 0
glUniform1i(location, 0);
103/02/07
●
Only Texture2D is supported
(0.0, 1.0)
(1.0, 1.0)
(0.0, 0.0)
(1.0, 0.0)
46

47. Fragment Shader – Texture Example
varying vec2 v_textcoord;
uniform sampler2D s_texture;
void main(void)
{
gl_FragColor
= texture2D(s_texture, v_textcoord);
}
103/02/07
47

48. Texture - Environment Cube Mapping
Environment mapping provides good reflection effect.
103/02/07
48

49. Depth Test
glEnable(GL_DEPTH_TEST);
without depth test
103/02/07
with depth test
49

50. Culling
Culling opertaion discards the specific side of each triagles.
glFrontFace(GL_CW);
//clockwise
glCullFace(GL_BACK);
glEnable(GL_CULL_FACE);
Clockwise
V0
103/02/07
Counter-Clockwise
V1
V2
V2
V1
V0
50

51. Scissor/Stencil/Dithering
Scissor
Stencil
Dithering
103/02/07
http://angra.blog31.fc2.com/blog-entry-52.html
http://iphone-3d-programming.labs.oreilly.com/ch06.html
51

52. Frame Buffer Objects – release the power of GPU
motion blur
Depth of Field
Bloom lighting
Light Scattering
103/02/07
http://developer.nvidia.com/docs/IO/8230/GDC2003_OpenGLShaderTricks.pdf
http://en.wikipedia.org/wiki/Bloom_%28shader_effect%29
http://developer.amd.com/media/gpu_assets/Scheuermann_DepthOfField.pdf
http://http.developer.nvidia.com/GPUGems3/gpugems3_ch13.html
http://www.fabiensanglard.net/lightScattering/index.php
52

53. More Power on Desktop
●
●
●
●
●
●
●
●
3D texture
Uniform Buffer Object
Floating-Point Depth/Texture/Render buffer
Double Precision
Shader subroutines
Geometry Shader
Tessellation Shader
……..
103/02/07
53

54. Further Study
OpenGL ES 2.0 Programming Guide by Aaftab Munshi, Aaftab Munshi and Aaftab Munshi,
Addison-Wesley, ISBN13: 978-0321502797
OpenGL ES SuperBible 4th/5th edition by Richard S. Wright, Nicholas S Haemel, Graham
Sellers and Benjamin Lipchak, Addison-Wesley, ISBN13: 978-0321712615
3D Math Primer for Graphics and Game Development by Fletcher Dunn and Ian Parberry,
Jones & Bartlett Publishers , ISBN13: 978-1556229114
GPU Gems 3 by Hubert Nguyen, Addison-Wesley, ISBN13: 978-0321515261
Free Online: http://developer.nvidia.com/object/gpu-gems-3.html
103/02/07
54

55. Further Study
GPU Pro 2 edited by Wolfgang Engel, A K Peters/CRC Press (GPU Pro Series)
Original ShaderX series (http://tog.acm.org/resources/shaderx/)
OpenGL ES 2.0 specification (API 2.0.25, GLSL ES 1.0.17)
http://www.khronos.org/registry/gles/
OpenGL 4.1 Specification
http://www.opengl.org/registry/
NeHe
http://nehe.gamedev.net/
LightHouse3D
http://www.lighthouse3d.com/opengl/
103/02/07
55

56. Q&A
Thanks.
103/02/07
56