Vector Calculus: Understanding the Dot Product

Blog Articles
21

This can be a really tough one, and to be honest I don’t find many of the analogies very useful. I’m a video game programmer so I need the dot product a lot, and I still find it easiest to think in terms of projection, as sterile as it may sound. I think of the dot product as the component of the first vector in the direction of the second. Or, if you think of the second vector as a hyperplane (more sterile, mathematical terms), how long would the first vector be projected onto it?

I love this site, and I don’t comment often, but I think you do a great job, and I’m sure I’ll be pointing my son at your articles before too long (he’s nearly 1)

As a matter of professional curiosity, are you certain that the Mario Kart boost pads only give you a boost proportional to your alignment? Or do they just give you a constant boost so long as you are crossing them in the correct direction?

22

Add my thanks to the many expressed already. The shadow anaology worked well. It cemented the notion that the “thing” (vector) is fixed, while it’s shadow is falling onto surfaces that “move” / rotate from one coordinate system to another. The shadow concept has an analog in orbital mechanics; ignoring perturbations, a satellite goes “round and round” in a fixed path, it’s the earth below it that moves.

23

Oh my gahd! I’ve been trying to figure out the concept for quite a long time. You explanation helped me to visualize the concept and let my intuition flow. Thanks you very much!

24

I’d love to feature this post in the Math and Multimedia Carnival #21 on my site, Math Concepts Explained. Please let me know! My site is at http://sk19math.blogspot.com. Thanks!

25

@Shaun: Definitely! I’ve emailed you as well, but feel free to use the content as you need. Thanks for asking!

26

How does the cross product work?? I mean, I get that it gives us the orthogonal vector to a plane but how does it do this? Seems kind of magical to me. What’s the intuition?

27

I get the projection aspect of it – I see why projecting vector a onto vector b results in cos(theta) * |a|. But I don’t understand intuitively why you multiply that by |b|. I mean I see it in the proof via the law of cosines and your explanation, but why intuitively is that part of the dot product. I don’t if this is too confusing of a question. So when you multiply by the magnitude of the vector that you are projecting on to, what does that give you? What does that scalar quantity mean? Is that the total “Force” if you want to think of it that way?

28

Kalid, thanks for the great article and even more for your wonderful “mission” :slight_smile:

There’s still something that doesn’t click to me about the dot product, hopefully you’ll be able to give me a hint.

Formally, I can see that the two definitions, axbx+ayby and |a||b|cos(t), are equivalent. Given what I expect to be the meaning of the dot product, I can figure out why summing the products of pairwise parallel components and multiplying the vector lengths times cos(t) both “make sense”, but it’s not straightforward for my intuition to realize why do they make exactly the same sense. Why are those reasonable measures of “weighted parallelism” (i’m just trying to give a more meaningful name to the dot product than “dot product”, but it doesn’t really matter) exactly the same?

My only way to help intuition is to have intuitive arguments to derive (1) the latter from the former and (2) the former from the latter (I’m not talking about formal proofs here, those are available everywhere). Thanks to your great article, I can now see path (1), but I still miss an intuition for path (2).

More precisely, I can see that axbx+ayby is invariant wrt to rotation, and therefore I can choose one particular frame where bx=|b|, by=0 and ax=a*cos(t), which leads to |a||b|cos(t). That’s path (1) and it gives my intuition a strong hint on the equivalence of the two definitions. Thank you for that.

However, should I start from |a||b|cos(t) and try to derive axbx+ayby, I’d be kind of lost. I could make it to the point that, being |a||b|cos(t) obviously invariant wrt to rotation as well, I could fix one frame where |b|=bx and |a|cos(t)=ax, and that would cover the axbx part, which in this particular frame gives the value of the dot product. But what’s the intuition that makes me generalize it to axbx+ayby? How do I come to the point that I tell myself "hey, of course I need to add the "ayby" there!"?

Thank you once again for your great contributions :slight_smile:

Regards,

Andrea

29

I thought some more about this. Your explanation lead me to try and wrap my head around line integrals and their use of the dot product. But…

You are absolutely right that the dot product in itself is just multiplication taking direction into account. Changing the vector notation into matrix notation makes that clear. And yet…

I asked myself: Why do we use the dot product? Well, we want an answer to a specific question, don’t we? That got me thinking about what vectors represent; which is “magnitude” and “direction”. So, when we multiply vectors we multiply both magnitude and direction.

Except…we don’t always care equally about magnitude and direction! When we only care about the magnitude of vectors, we don’t even bother with vector notation!

In the case of the dot product we care about the direction of vector ‘a’ with respect to vector ‘b’ (the direction of vector ‘b’ with respect to vector ‘a’). In other words we care about the RELATIVE DIRECTION—which explains why it is a scalar.

This raises the question: “How do we multiply vectors when we care about the absolute direction?” For a two dimensional vector the answer is easy; we convert the vector to a complex number and multiply complex number ‘a’ with complex number ‘b’.

In a similar way, this explains the multiplication of a Nx1 matrix with a 1xN matrix to get a NxN matrix. But in this case, we care about multiple direction (instead of relative direction, or absolute direction). The Jacobian is a good example.

Similarly, the cross product. We care about direction, but we only care about the direction perpendicular to both vector ‘a’ and vector ‘b’.

So vector multiplication is about multiplying a pair of numbers (magnitude and direction), whereby we take into account ‘direction’ to varying degrees.

To summarise, I would suggest changing “…the dot product is ‘multiplication, taking direction into account’.” into:

The dot product is multiplication while taking relative direction (or the difference in direction) into account.

Anyway, thank you for explanations like these. They help me to understand the basic mathematical idea—instead of the more specific physical idea—behind mathematical concepts like these.

Thanks.

30

@Avo: Thanks so much. That’s exactly what I wanted – math as a story, something interesting to follow, not a dry series of steps. Really appreciate the encouragement!

Heh, it’s not a choice between learning and video games, you can do both :).

31

I forgot to mention that the context of the dot product determines whether its symmetry is important; whether its parametrisation is important; … and so on. The context also determines what we’re multiplying—and what our result represents (e.g. your: “What is the x-coordinate of a, assuming b is the x-axis?”).

Which is why I always had difficulty with the dot product. The dot product of a vector with itself, is conceptually completely different. Especially when you contrast it with the use of a dot product in line integrals.

Thanks.

P.S. Love your integral explanations:

Integrals are “multiplication, taking changes into account”; and…
the dot product is “multiplication, taking direction into account”.; and…
Line integrals are “multiplication, taking into account changes both in magnitude and direction”.

32

One of the most difficult things in mathematics isn’t failing to understand the initial concept (the one we’re first introduced to at school). Instead, the difficult thing in mathematics is to understand the generalised concept. You seem to have covered everything—and it was useful to me—but I don’t believe you’ve explained very well why it becomes a scalar.

Lets rewrite the first equation a bit:

vec{a} cdot vec{b} divided by |vec{b}| = |vec{a}| cos(theta)

or in words (arguing from the right side of the equation): to get the magnitude of the component of vector a in the direction of vector b, take the dot product of vectors a and b and divide by the magnitude of vector b. (Remember that “vec{b} / |vec{b}|” is the unit vector e_b.) This sounds remarkably similar to a coordinate transformation, specifically from polar to Cartesian coordinates. (I know you mention it, sorry)

But there is a second transformation happening at the same time. We’re changing from vectors to scalars. The “projection” doesn’t really explain that (at least it never did to me). Your question is better: “What is the x-coordinate of a, assuming b is the x-axis?” The thing to remember though, is that we’re simultaneously applying a parametrisation! That is why the dot product occurs in Line Integrals!

To summarise:
Step 1: take the component of vector a in the direction of vector b (it’s still a vector)
Step 2: alter the size of said component by multiplying it with |b|. (it’s still a vector)
Step 3: apply parametrisation to change it from a vector to a scalar.

That last part makes sense out of Line Integrals involving Work (and Energy): we’re parametrising vector a along curve C (which is defined by position vector b)

Hope it helps! (And I hope I didn’t make any mistakes.)

33

[…] much, exactly? Well, this is a job for the dot product, which is the projection of the field onto the surface. The dot product gives us a number (from 0 […]

34

@Brian: Glad you’re enjoying it. Don’t let this site stop you though, I want to read your insights too (love seeing how people approach things).

Great question on why we don’t end up with a vector. There’s a few ways to interpret how one vector “overlaps” with another. The dot product is a horrible name because it doesn’t tell anything about the overlap, just the name of the symbol used (woohoo). The idea is to say “How much does vector A push in the direction of the vector B?”. There is an implicit vector here (the reference vector, B), so the operation just gives the size.

The adding of components is basically breaking each vector (A and B) into parts, and seeing how each part might possibly push in the same direction. It turns out, only the horizontal parts can push in the same direction (ax * bx) and only the vertical parts can push in the same direction (ay * by). The combined “push” is the sum of how much each axis is pushing. Hope this helps!

35

Great post - I really liked the stpe-by-step breakdown of the different ways to look at it and I could imagine that the mario kart example would go down really well in class.

36

I found this discussion refreshing especially when combined with Professor Strang’s Linear Algebra Course on MIT open courses. Also having spent my first Saturday Night with AdaFruit I loved the kill your fruit analogy. As you fill this out a comparison with the cross product and some of the geometry of “in the plane” versus out of the plane would be great. Before I started to regularly come to this site to have my mind blown I did it on such simple realization as “the cross product of any two vectors in a plane gets you a vector normal to the plane.” Off to make some fruit salad…

37

@mark: Thanks for dropping by! Ah, I remember we used Prof. Strang’s book in our course too, sadly I don’t remember enough linear algebra (especially an intuition for eigenvectors / eigenvalues) but I’m looking forward to getting re-acquainted with it :).

@Daniel: Thanks – if you end up trying it, I’d love to know how it worked out! I’m still looking for really vivid analogies of what’s happening.

38

@Anonymous: Another good analogy – it’s how much effort will be transferred when you pull the cart at that angle.

39

@alex:

Yep, totally agree with the grouping – it shows the symmetry by thinking of a onto b, or b onto a. And yep, with line integrals, you usually have a defined Force and Path vector (it’s not as intuitive to project the Path onto the Force).

I love breaking down complicated ideas into easy-to-digest parts… yep, the integral is a wrapper around the idea of “applying” something piece by piece (of which multiplication and repeated addition are very simple base cases).

On the vector notation: I guess it depends on how deep you want to go :). For me, I take it as an axiom that “we want to figure out how much one vector pushes in the direction of the other, it will be a single number (called the dot product) and here’s why it should be what it is (i.e., x and y components interacting)”

Yes, relative direction is a good way to put it – the absolute doesn’t matter. In a system of relative coordinates (just angles between them), what do we know about the force of one onto the other? (Interestingly, I stumbled upon the concept of coordinate-free geometry today).

When we care about absolute direction, yep, we want different results when vectors that are 90 degrees apart but oriented differently are combined. I think we need to be careful with complex numbers though (for terminology) since the typical dot product gives 0 when perpendicular, but complex numbers do a rotation.

I might tweak it to “the dot product is multiplication, taking the difference in direction into account”. (I’ll have to tweak it). WIth all of these 1-liners, it’s a tradeoff between an immediate click vs. getting into too much detail.

And the feeling for these discussions is mutual, I love exploring these ideas!

@Joe: Thanks Joe. I agree, there’s an interplay between theory and application (I learned vector calc before E&M, and having physics examples definitely solidified my earlier understanding). I don’t think you can teach in some pyramid where the base ideas are learned, divorced from examples, and then the applications are sprinkled in later.

Appreciate the encouragement, I’d love to do the cross product & determinant in the future! (Two concepts I really, really want to get an intuition for & not just the mechanical calculation).

40

[…] http://betterexplained.com/articles/vector-calculus-understanding-the-dot-product/ RedditBufferShareEmailPrintFacebookDiggStumbleUpon […]