The standard inner product is the dot product.
Definition 1 Given two vectors ~v, ~w ∈ Rn, we define their standard inner product h~v, ~wi by h~v, ~wi = ~v>( ~w) ∈ R. We sometimes use the notation ~v · ~w for h~v, ~wi, and call the operation the dot product.
Warning 2 Note that ~v>( ~w) 6= ~w(~v>): one is a number, while the other is an n × n matrix.
Question 3 Make sure for yourself, by using the definition, that
x1
x2
... xn
·
y1
y2
... yn
= x1y1+ x2y2+ x3y3+ · · · + xnyn.
Prove the following facts about the dot product. ~u, ~v, ~w ∈ Rn and a ∈ R (a) ~v · ~w = ~w · ~v (The dot product is commutative)
(b) (~u + ~v) · ~w = ~u · ~w + ~v · ~w and (a~v) · ~w = a(~v · ~w) (The dot product is linear in the first argument)
(c) ~u · (~v + ~w) = ~u · ~v + ~u · ~w and ~v · (a ~w) = a(~v · ~w) (The dot product is linear in the second argument)
(d) ~v · ~v ≥ 0 (We say that the dot product is “positive definite”)
(e) if ~v · ~z = 0 for all ~z ∈ Rn, then ~v = ~0 (The dot product is nondegenerate) 1. ~v · ~w = v1w1+ v2w2+ ... + vnwn= w1v1+ w2v2+ ... + wnvn = w · v, so the dot product is commutative.
(skipping item 2 for now) 3.
~
u · (v + ~w) = ~u>(v + ~w) by definition
= ~u>(v) + ~u>( ~w) since ~u>: Rn → R is linear
= ~u · v + ~u · ~w by definition and
~
u · (a ~w) = ~u>(a ~w) by definition
= a~u>( ~w) since ~u>: Rn→ R is linear
= a~u · ~w by definition
18 Dot product
2. follows from 3 and 1
4. ~v · ~v = v12+ v22+ v32+ ... + v2n, and the square of a real number is nonnega-tive, so the sum of these squares is also nonnegative.
5. is perhaps the trickiest fact to prove. Observe that if ~v · ~z = 0 for every ~z ∈ Rn, then this formula is true in particular for z = ~ej. But ~v · ~ej= vj. Thus, by dotting with all of the standard basis vectors, we see that every coordinate of ~v must be 0.
Thus ~v is the zero vector
The fact that the dot product is linear in two separate vector variables means that it is an example of a “bilinear form”. We will make a careful study of bi-linear forms later in this course: it will turn out that the second derivative of a multivariable function gives a bilinear form at each point.
So far, the inner product feels like it belongs to the realm of pure algebra. In the next few exercises, we will start to see some hints of its geometric meaning.
Question 4 Let v =5 1
. Solution
Hint: h~v, ~vi = 52+ 12= 26 h~v, ~vi = 26
Let’s think about this a bit more abstractly. Set v =x y
. Solution
Hint: h~v, ~vi = x2+ y2 h~v, ~vi = x2+ y2
Notice that the length of the line segment from (0, 0) to (x, y) is p
x2+ y2 by the Pythagorean theorem.
19 Length
The inner product provides a way to measure the length of a vector.
You should have discovered that v · v is the square of the length of the vector v when viewed as an arrow based at the origin. So far, you have only shown this in the 2-dimensional case. See if you can do it in three dimensions.
Show that the length of the line segment from (0, 0, 0) to (x, y, z) is
Until now, you may not have seen a treatment of length in higher dimensions.
Generalizing the results above, we define:
Definition 1 The length of a vector ~v ∈ Rn is defined by |v| =√ v · v.
Question 2 Solution The length of the vector
Question 3 Solution
Hint: By the Pythagorean theorem, we can see that the distance isp
(5 − 2)2+ (9 − 3)2
Hint: We could also view this as the length of the vector 3 6
which “points” from (2, 3) to (5, 9).
The distance between the points (2, 3) and (5, 9) is sqrt(32+ 62)
Definition 4 The distance between two points p and q in Rn is defined to be the length of the “displacement” vector ~p − ~q.
Question 5 Solution
Hint: The displacement vector between these points is
Hint: The length of the displacement vector isp
32+ 12+ 62+ 72
The distance between the points (2, 7, 3, 1) and (5, 6, 9, 8) is sqrt(32+ 1 + 62+ 72)
Question 6 Write an equation for the sphere centered at (0, 0, 0, 0) in R4of radius r using the coordinates x, y, z, w on R4.
Solution
19 Length
Hint: For a point p = (x, y, z, w) to be on the sphere of radius r centered at (0, 0, 0, 0), the distance from p to the origin must be r
Hint: r =p
x2+ y2+ z2+ w2
Hint: x2+ y2+ z2+ w2= r2 x2+ y2+ z2+ w2 = r2
Question 7 Write an inequality stating that the point (x, y, z, w) is more than 4 units away from the point (2, 3, 1, 9)
Solution
Hint: The distance between the point (x, y, z, w) and (2, 3, 1, 9) isp
(x − 2)2+ (y − 3)2+ (z − 1)2+ (w − 9)2.
Hint: So we needp
(x − 2)2+ (y − 3)2+ (z − 1)2+ (w − 9)2> 4 sqrt((x − 2)2+ (y − 3)2+ (z − 1)2+ (w − 9)2) > 4
Prove that |a~v| = |a||~v| for every a ∈ R.
Warning 8 These two uses of | · | are distinct: |a| means the absolute value of a, and |~v| is the length of ~v.
|a~v| =pha~v, a~vi by definition
=p
a2h~v, ~vi by the linearity of the inner product in each slot
=
√
a2ph~v, ~vi
= |a||~v|
20 Angles
Dot products can be used to compute angles.
Question 1 Give a vector of length 1 which points in the same direction as ~v =
1 2
(i.e. is a positive multiple of ~v).
Solution
Hint: Remember that you just argued that |a~v| = |a|~v for any a ∈ R. What positive a could you choose to make |a||~v| = 1?
Hint: We need to take a = 1
|~v|
Hint: The length of ~v isp
12+ 22=√ 5
Hint: The vector
√1 25
√5
points in the same direction as ~v, but has length 1.
Now that we understand the relationship between the inner product and length of vectors, we will attempt to establish a connection between the inner product and the angle between two vectors.
Do you remember the law of cosines? It states the following:
Theorem 2 If a triangle has side lengths a, b, and c, then c2= a2+b2−2ab cos(θ), where θ is the angle opposite the side with length c.
Prove the law of cosines. You may want to read the lovely proof at mathproofs1. You can find a beautiful proof here2.
We can rephrase this in terms of vectors, since geometrically if ~v and ~w are vectors, the third side of the triangle is the vector ~w − ~v.
Theorem 3 For any two vectors v, w ∈ Rn, |w − v|2= |w|2+ |v|2− 2|v||w| cos(θ), where θ is the angle between v and w.
(For you sticklers, this is really being taken as the definition of the angle between two vectors in arbitrary dimension.)
Rewrite the theorem above by using our definition of length in terms of the dot product. Performing some algebra you should obtain a nice expression for v · w in terms of |v|, |w|, and cos(θ).
1http://mathproofs.blogspot.com/2006/06/law-of-cosines.html
2http://mathproofs.blogspot.com/2006/06/law-of-cosines.html
20 Angles
|w − v|2= |v|2+ |w|2− 2|v||w| cos(θ) hw − v, w − vi = |v|2+ |w|2− 2|v||w| cos(θ)
hw, w − vi − hv, w − vi = |v|2+ |w|2− 2|v||w| cos(θ) by the linearity of the inner product in the first slot hw, wi − hw, v − hv, wi + hv, vi = |v|2+ |w|2− 2|v||w| cos(θ) by the linearity of the inner product in the second slot
|w|2− 2hv, wi + |v|2= |v|2+ |w|2− 2|v||w| cos(θ) hv, wi = |v||w| cos(θ)
You should have discovered the following theorem:
Theorem 4 For any two vectors v, w ∈ Rn, v · w = |v||w| cos(θ). In words, the dot product of two vectors is the product of the lengths of the two vectors, times the cosine of the angle between them.
This gives an almost totally geometric picture of the dot product: Given two vectors ~v and ~w, |~v cos(θ)| can be viewed as the length of the projection of ~v onto the line containing ~w. So |~v|| ~w| cos(θ) is the “length of the projection of ~v in the direction of ~w times the length of ~w”.
As mentioned above, this theorem is really being used to define the angle be-tween two vectors. This is not quite rigorous: how do we even know that v · w
|v||w| is even between −1 and 1, so that it could be the cosine of an angle? This is clear from the “Euclidean Geometry” perspective, but not as clear from the “Carte-sian Geometry” perspective. To make sure that everything is okay, we prove the
“Cauchy-Schwarz” theorem which reconciles these two worlds.