This document provides an overview of matrix algebra concepts including: - Matrix addition is defined as adding corresponding elements and is commutative and associative. - Matrix multiplication is defined as taking the dot product of rows and columns. It is associative but not commutative. - The transpose of a matrix is obtained by flipping rows and columns. - Properties of matrix operations like addition, multiplication, and transposition are discussed.

1. Lesson 5
Matrix Algebra and The Transpose
Math 20
September 28, 2007
Announcements
Thomas Schelling at IOP (79 JFK Street), Wednesday 6pm
Problem Set 2 is on the course web site. Due October 3
Problem Sessions: Sundays 6–7 (SC 221), Tuesdays 1–2 (SC
116)
My office hours: Mondays 1–2, Tuesdays 3–4, Wednesdays
1–3 (SC 323)

2. Remember the definition of matrix addition
Definition
Let A = (aij )m×n and B = (bij )m×n be matrices. The sum of A
and B is the matrix C = (cij )m×n defined by
cij = aij + bij
That is, C is obtained by adding corresponding elements of A And
B.

3. Remember the definition of matrix addition
Definition
Let A = (aij )m×n and B = (bij )m×n be matrices. The sum of A
and B is the matrix C = (cij )m×n defined by
cij = aij + bij
That is, C is obtained by adding corresponding elements of A And
B.
We do not define A + B if A and B do not have the same
dimension.

4. Properties of Matrix Addition
Rules
Let A, B, C, and D be m × n matrices.
(a) A + B = B + A

5. Properties of Matrix Addition
Rules
Let A, B, C, and D be m × n matrices.
(a) A + B = B + A (so addition is commutative)

6. Properties of Matrix Addition
Rules
Let A, B, C, and D be m × n matrices.
(a) A + B = B + A (so addition is commutative)
(b) A + (B + C) = (A + B) + C

7. Properties of Matrix Addition
Rules
Let A, B, C, and D be m × n matrices.
(a) A + B = B + A (so addition is commutative)
(b) A + (B + C) = (A + B) + C (so addition is associative)

8. Properties of Matrix Addition
Rules
Let A, B, C, and D be m × n matrices.
(a) A + B = B + A (so addition is commutative)
(b) A + (B + C) = (A + B) + C (so addition is associative)
(c) There is a unique m × n matrix O such that
A+O=A
for any m × n matrix A. The matrix O is called the m × n
additive identity matrix.
(d) For each m × n matrix A, there is a unique m × n matrix D
such that
A+D=O
(We write D = −A.) So additive inverses exist.

9. Proof
Proof.
Let’s prove (b). We need to show that every entry of A + (B + C)
is equal to the corresponding entry of (A + B) + C.

10. Proof
Proof.
Let’s prove (b). We need to show that every entry of A + (B + C)
is equal to the corresponding entry of (A + B) + C. Well,
[A + (B + C)]ij = aij + [B + C]ij = aij + (bij + cij )
= (aij + bij ) + cij as real numbers
= [A + B]ij + cij = [(A + B) + C]ij .

11. Remember the definition of matrix multiplication
Definition
Let A = (aij )m×n and B = (bij )n×p . Then the matrix product of
A and B is the m × p matrix whose jth column is Abj . In other
words, the (i, j)th entry of AB is the dot product of ith row of A
and the jth column of B. In symbols
n
(AB)ij = aik bkj .
k=1

12. Remember the definition of matrix multiplication
Definition
Let A = (aij )m×n and B = (bij )n×p . Then the matrix product of
A and B is the m × p matrix whose jth column is Abj . In other
words, the (i, j)th entry of AB is the dot product of ith row of A
and the jth column of B. In symbols
n
(AB)ij = aik bkj .
k=1
Another way to look at it:
A b1 b2 . . . bp = Ab1 Ab2 . . . Abp

13. Examples
Let
12 02
A= B=
−1 2 31
 
0 120
1 −1 2
D = −1 0 2 0
C=
2 3 −1
3 100
Compute
(a) (A + B)C (d) (AB)C
(b) AC + BC (e) AC and CA
(c) A(BC) (f) AB and BA

14. Does matrix multiplication distribute?
Solution to (a) and (b)
9 11 −2
(A + B)C = = AC + BC
87 1

15. Does matrix multiplication distribute?
Solution to (a) and (b)
9 11 −2
(A + B)C = = AC + BC
87 1
In fact, this is true in general:
Distributive rules for matrices
If A, B, and C are of appropriate sizes, then
(A + B)C = AC + BC.
If A, B, and C are of appropriate sizes, then
A(B + C) = AB + AC

16. Is matrix multiplication associative?
Solution to (c) and (d)
14 6 8
A(BC) = = (AB)C
6 −6 12

17. Is matrix multiplication associative?
Solution to (c) and (d)
14 6 8
A(BC) = = (AB)C
6 −6 12
In fact, this is true in general:
Associative rules for matrix multiplication
If A, B, and C are of appropriate sizes, then
(AB)C = A(BC)

18. Is matrix multiplication commutative?
Solution to (e)
AC =

19. Is matrix multiplication commutative?
Solution to (e)
550
AC = ,
3 7 −4

20. Is matrix multiplication commutative?
Solution to (e)
550
AC = , but CA is not even defined. We cannot
3 7 −4
multiply a 2 × 3 matrix by a 2 × 2 matrix.

21. Is matrix multiplication commutative?
Solution to (e)
550
AC = , but CA is not even defined. We cannot
3 7 −4
multiply a 2 × 3 matrix by a 2 × 2 matrix.
Solution to (f)
BA =

22. Is matrix multiplication commutative?
Solution to (e)
550
AC = , but CA is not even defined. We cannot
3 7 −4
multiply a 2 × 3 matrix by a 2 × 2 matrix.
Solution to (f)
−2 4
BA = ,
28

23. Is matrix multiplication commutative?
Solution to (e)
550
AC = , but CA is not even defined. We cannot
3 7 −4
multiply a 2 × 3 matrix by a 2 × 2 matrix.
Solution to (f)
−2 4
BA = , but AB =
28

24. Is matrix multiplication commutative?
Solution to (e)
550
AC = , but CA is not even defined. We cannot
3 7 −4
multiply a 2 × 3 matrix by a 2 × 2 matrix.
Solution to (f)
−2 4 64
BA = , but AB = .
28 60

25. Is matrix multiplication commutative?
Solution to (e)
550
AC = , but CA is not even defined. We cannot
3 7 −4
multiply a 2 × 3 matrix by a 2 × 2 matrix.
Solution to (f)
−2 4 64
BA = , but AB = .
28 60
So matrix multiplication is (usually) not commutative, even when
it’s possible to test.

26. The Transpose
But there is another operation on matrices, which is just flipping
rows and columns.
Definition
Let A = (aij )m×n be a matrix. The transpose of A is the matrix
A = (aij )n×m whose (i, j)th entry is aji .

27. The Transpose
But there is another operation on matrices, which is just flipping
rows and columns.
Definition
Let A = (aij )m×n be a matrix. The transpose of A is the matrix
A = (aij )n×m whose (i, j)th entry is aji .
Example 
12
Let A = 3 4. Then
56
A=

28. The Transpose
But there is another operation on matrices, which is just flipping
rows and columns.
Definition
Let A = (aij )m×n be a matrix. The transpose of A is the matrix
A = (aij )n×m whose (i, j)th entry is aji .
Example 
12
Let A = 3 4. Then
56
135
A= .
246

29. Examples
Let
12 02
A= B=
−1 2 31
 
0 120
1 −1 2
D = −1 0 2 0
C=
2 3 −1
3 100
Compute
(a) (A ) (c) (AC) and A C
(b) (A + B) and A + B (d) (AC) and C A

30. Solution to (a).
(A ) =

31. Solution to (a).
(A ) = A

32. Solution to (a).
(A ) = A
In fact, this is true in general.

33. Does the transpose distribute over addition?
Solution to (b).
12
(A + B) = =A +B
43

34. Does the transpose distribute over addition?
Solution to (b).
12
(A + B) = =A +B
43
In fact, this is true in general.

35. Does the transpose distribute over multiplication?
Solution  (c) and (d).
to 
53
5 7 , but A C is not defined.
(AC) =
0 −4

36. Does the transpose distribute over multiplication?
Solution  (c) and (d).
to 
53
5 7 , but A C is not defined. On the other hand,
(AC) =
0 −4
 
53
C A = 5 7  = (AC) .
0 −4

37. Does the transpose distribute over multiplication?
Solution  (c) and (d).
to 
53
5 7 , but A C is not defined. On the other hand,
(AC) =
0 −4
 
53
C A = 5 7  = (AC) .
0 −4
This is true in general.

38. Remember these properties (including which ones aren’t true)
because they are the rules of the game in linear algebra!

39. Transpose and dot product
Notice:
p·q=pq

40. Transpose and dot product
Notice:
p·q=pq
Also,
Theorem
Let A be an n × n matrix and v, w vectors in Rn . Then
v · Aw = (A v) · w.

41. Transpose and dot product
Notice:
p·q=pq
Also,
Theorem
Let A be an n × n matrix and v, w vectors in Rn . Then
v · Aw = (A v) · w.
Proof.
Remember v · Aw is a scalar, so is equal to its own transpose.

42. Transpose and dot product
Notice:
p·q=pq
Also,
Theorem
Let A be an n × n matrix and v, w vectors in Rn . Then
v · Aw = (A v) · w.
Proof.
Remember v · Aw is a scalar, so is equal to its own transpose. Then
v · Aw = (v Aw) = (Aw) (v ) = w A v = w · A v = (A v) · w.

43. Example
Some special cases which are useful. Let
32 1 0
A= e1 = e2 =
−1 1 0 1
10 01 00 00
E11 = E12 = E21 = E22 =
00 00 10 01
20 10
D= I=
0 −1 01
Compute
(a) Ae1 and Ae2 (c) AE12 and E12 A
(b) AE11 and E11 A (d) AD and DA

44. Solution to (a)
3 2
Ae1 = Ae2 =
−1 1

45. Solution to (a)
3 2
Ae1 = Ae2 =
−1 1
In general,
Aei is the ith column of A.
ej A is the jth row of A.

46. Solution to (b) and (c)
30 32
AE11 = E11 A =
−1 0 00
−1 1
03
AE12 = E12 A =
0 −1 00

47. Solution to (b) and (c)
30 32
AE11 = E11 A =
−1 0 00
−1 1
03
AE12 = E12 A =
0 −1 00
In general,
AEij has all zeros except for the jth column, which is the ith
column of A
Eij A has all zeros except for the ith row, which is the jth row
of A

48. Solution to (d)
6 −2
64
DA = AD =
1 −1 2 −1

49. Solution to (d)
6 −2
64
DA = AD =
1 −1 2 −1
In general,
a diagonal matrix times a matrix scales every row of the
right-hand matrix by the corresponding diagonal entries
a matrix times a diagonal matrix scales every column of the
left-hand matrix by the corresponding diagonal entries.
diagonal matrices do commute