Dot products are a nice geometric tool for understanding projection. But now that we know about linear transformations, we can get a deeper feel for what's going on with the dot product, and the connection between its numerical computation and its geometric interpretation.
Full series: http://3b1b.co/eola
Future series like this are funded by the community, through Patreon, where supporters get early access as the series is being produced.
3blue1brown is a channel about animating math, in all senses of the word animate. And you know the drill with YouTube, if you want to stay posted about new videos, subscribe, and click the bell to receive notifications (if you're into that).
If you are new to this channel and want to see more, a good place to start is this playlist: https://goo.gl/WmnCQZ
Various social media stuffs:
I highly respect and love this professor. However, to be honest, this video is very confusing. Please check out this you-tube video, which explains dot product much more clearer: https://www.youtube.com/watch?v=FrDAU2N0FEg&feature=youtu.be
This video was needlessly overcomplicated for such an easy topic. It is sometimes fascinating that some unrelated things can be corresponding to each other, but that's just a bonus and IMO should not be the default explanation for the dot product of vectors. I was expecting to see an intuitive way to grasp why x1*x2+y1*y2+z1*z2 were exactly |a||b|cos(alpha). Similarly instead of 'bending the two-dimensional space' to explain this you could just use the fact that bcos(alpha) is the b's projection onto an axis under exactly alpha angle. That could be explained intuitively too. While this video in my opinion does not address the essence of dot product and the fundamental 'why' still is unanswered by it.
is that actually the liner algebra we took in school, i mean i took it as a simple way to find x1,x2,x3 etc and learned about the matrices on my own, but still everything i'v learned isn't like that piece of art kind of playlist.
Hey this series has helped me understand linear algebra to a degree that I didn’t think I ever could. That being said, this video is the point where I got confused. I see a lot of people in this comment section with the same visualization problem I had and I thought this might help you because it helped me.
The dot product alone isn’t the projection of one vector onto another in itself. The formula for projection of u onto v is ( (u•v)/|v^2| )*v where |v^2| is the length of v squared.
So this means that after you take the dot product you still need to divide the result by the magnitude of the initial vector and then set it in the direction of the initial vector (hence the *v at the end of the formula and the second division of the magnitude of v resulting in v^2 being on the bottom)
Hope this helps
The overall series in great but I lost myself in this video. There's one fundamental thing that I don't grasp. In 10:10 the 2x1 matrix transform a vector in a point that is in the diagonal number line. But in 12:40 an apparently similar 2x1 matrix transform the vector in a point on X axe.. The point is the key because the diagonal line is the reason why we can say that a 2x1 matrix and a 1x2 vector are the same thing
I went through it for more than 1 hour pausing at each point made. I have never attended college and learned through textbooks recommended by first class colleges and the Khan Academy teachings (in particular) to get my engineering credentials for a living. I jammed into my head all these math proofs like an Ape and try to relate them to the real world of tangible things. Thank God that I found You to light up the bulbs and cast away the shadows (vague conjectures of realities).
Example on dot product. You sum it beautifully as a linear transformation from 2D space not defined as numerical vectors but projecting space onto a diagonal copy of a number line. I take this and others as wings to fly now.
Yes. I signed onto Patreon as a small token of appreciation.
If the dot product of two vectors is defined as the LENGTH of the projection on a vector multiplied by the LENGTH of the vector being projected on then how can the resulting value ever be negative? I thought that length could not be negative!?!
Here you just finished talking about how you can get to [ 3u_x 3u_y ] by thinking of how i-hat and j-hat get scaled when applying the whole linear transformation of projecting and scaling all at once. But I think it's worth breaking it down into the successive application of projecting onto u-hat (which you already describe is the linear transformation given by [ u_x u_y ]), and then applying the transformation of scaling that resulting line by 3. When you focus on that last part, you can ask the same question we've been asking the whole time when dealing with linear transformations: what happens to the unit vector basis of our coordinate system? That is, what happens to u-hat? Well, it's clear that if the transformation is simply a stretching of a factor of 3, then u-hat (a unit vector) must go to 3. That means that we can write our second linear transformation as the one-by-one matrix that just represents the one-dimensional number that our one-dimensional unit vector ends up at: [ 3 ]. So, our full project-and-stretch linear transformation can be written as the successive application of the projection matrix [ u_x u_y ] and our stretching matrix [ 3 ], which (happy day!) gives us the same result [ 3u_x 3u_y ]! And this way we get to approach the same problem but using the composition of two linear transformations that you already beautifully defined!
Edit: My only problem with what I just said is it feels like I have to go FROM treating u-hat as just the unit vector in two-dimensional space that happens to lie on the line we're projecting to, TO treating u-hat as the new one-dimensional basis vector of our new coordinate space. Maybe someone here can tell me if that's an okay thing to do or not :)
I'm a bit irritated because of the order of the videos. If you look at the chapter number in the name, they are in the wrong order, but if you look at the announcement of the next video, the order seams to fit. Are they named wrong?
(Edit: I use the word "basis" wrong, because u1 and u2 are linearly dependent, but I guess you know what I want to say: they're the "replacements" of the old basis vectors)
What you are really doing when you do the dot product of a vector v with the vector u is: You change v's basis. Normally, you can write v as a linear combination v=v1*i^+v2*j^.
Now - as we've seen - the dot product is just a linear transformation from two (or any) dimensions into one, where u (written as a row vector) is the transformation matrix, which means that i^ and j^, our old basis vectors, are transformed into u1 and u2 - one dimensional vectors (= numbers). So they're our new basis, just numbers instead of vectors. Now our linear combination of v is v1*u1+v2*u2. The dot product.
The projection onto the number line through u in this video is just a nice way of visualizing it.
But why does it work? Well:
Our new basis vectors u1 and u2 are one dimensional - we can place them on any number line we want - usually we take the x-axis, because it is horizontal and therefore easy to use. So u1 and u2 sit on the x-axis. Now we use a trick: Projecting our OLD basis vectors i^ and j^ onto the line and then scaling them with the length of u^ (here =1, because u^ is a unit vector) is the same as rotating u1 and u2 onto it (which wouldn't need any scaling, because u1 and u2 themselves are already the right size). This trick works because of the symmetry shown in the video.
So what we are doing when we project i^ and j^ onto this line is just putting our new one dimensional basis vectors u1 and u2 onto their correspondent place on the new number line that goes through u^.
So what we're basically doing is doing the linear transformation described by the dot product and then rotating the number line on which we land so that it goes through u^.
Now if we have any vector v=(v1,v2)=v1*i^+v2*j^, we can also just project it onto the line because it's the same as first projecting the unit vectors and then combining them to v (because the transformation is linear).
Of course, if u^ isn't a unit vector (length 1), we also need to scale by its length.
Damn, my comment turned out longer than I thought. But I hope it helps some of you who had similar problems with understanding this as I did in the beginning.
This seems to be very similar to talking about covectors and stacks in tensor algebra. Is it, in fact, the same? I believe all your animation videos make mathematics might easier for those of us who perhaps struggle more to think visually in our heads.
Another way of thinking the dot product is to compute the projection of w on v by argmin_alpha ||w- alpha*v||^2, which amounts to find the closest scaling of v to w. When one sets the derivative with respect to alpha equal to zero, one gets alpha = v.w/||v||.
Your videos are amazing! I go to the University of Michigan Ann Arbor and I wish they explained things like you did. Honestly, I wish they just showed us your videos before each lecture since it would have saved me lots of stress and anger. This has completely changed my idea of linear algebra from hating it to enjoying it. (This lecture was one of the harder ones to follow but easy to learn to compute) Bravo and thank you!
I can see the significance of dot product for normalized vectors (i.e. the unit vector) - where 1 means the vectors point in the same direction, 0 means their perpendicular and -1 means they point in opposite direction - and any value in the range is used to measure how much wide apart they are (this has some application, for example in modeling light touching the normal of a surface in computer graphics). But what meaning does a dot product of 10 has, say compared with 8? Is there any application that gives meaning to differentiating a dot product of 8 from that of 10?
(BTW - Many thanks for this wonderful series.)
Duality....very confusing that part is. You are speaking way too fast, and you need to explain the idea at least twice in 2 different ways....i am not catching anything at all. @5:30, what are you even doing transfroming the grids like that? Why transfrom it that way and not some other way???
1) Not 2 different ways, although yes, I've had to watch the video many times to grasp what he's saying. What he leads up to is that the 2D vector being projected onto in each projection visualization on the 2D plane acts as a scalar multiple of the 1D unit basis for the vector's own copy of the 1D line (you're supposed to imagine that the vector lies on its own 1D line at the same time as lying on the 2D plane), with the vector starting at zero/the origin in both spaces.
5:30 is purely a non-specific example of a 2D-to-1D linear transformation (he's transforming the space he's working with from 2D to 1D using the rule he made up at that moment for this example). He's just showing an example to introduce the idea of the rest of the video.
5:45 J-Hat was positive (on the Y-axis) on the 2D plane, but he transformed the 2D plane into a 1D line in the same manner as 5:30, using a different rule this time. His rule says to make J-Hat negative on his new 1D line. J-Hat is now negative on the 1D line, but WAS positive (on the Y-axis) on the 2D plane.
6:24 -2 is implied by his visualization, and then verified by his matrix multiplication.
Here's what it boils down to:
A linear transformation takes the basis components of a vector (I-Hat, J-Hat, etc.), scales the same dimension of each component (scaling to negative is also possible by just flipping the orientation, e.g. right-to-left) and then adds these together to get the total magnitude in this same dimension for the transformed vector (because you're ultimately adding the transformed components back together to get the new transformed vector, but you're doing it more thoroughly and modularly by dimension). Repeat if doing more dimensions.
Notice that the way we're transforming each individual dimension is exactly what 3b1b is demonstrating in the video. He's doing 2D-to-1D, so it means he does one dimension, but then doesn't do the other one. He effectively zeroed out the 2nd dimension. To demonstrate, try scaling the same dimension of each component of a 2D vector to zero and then adding these together to get the total magnitude in this dimension. It's zero! This dimension ends up disappearing! That's why he transforms the grids like that.
+Jo Reven No no, why J-Hat lands on the POSITIVE axis of the 1d number line but not on the negative axis of the 1d number line. He didn't explain that at all. And then @5:45, he said that J-Hat is -2, but he showed on the graph that J is positive. It doesn't make any sense. And then @6:24, he says that it 'implies -2', what on earth is he doing moving the J arrow around w/o explaining? Ridiculous.
Great video again but I’m struggling on one concept that’s stopping me from getting the full picture.
How is it allowed that we can represent a number line with a direction within 2D space and have it only represent scalar quantities?
He's not putting the number line in 2D space, but laying it on top. It remains separate. It's like taking two transparent slides, each with something drawn of them, and laying them on top of each other and then looking at them directly from above. You'd see the combined result, but they remain separate. It's also like layers in Photoshop.
Great series so far, here is where I start to struggle. 10:57 "Since it's all linear this implies more generally that the new matrix can be interpreted as projecting any vector onto the number line copy and multiply where it lands by 3." Why can we do this? What's the effect of linearity here? I don't get the reasoning behind this statement which isn't further explained I think.
For anyone who's lost like I was hours ago, I think I'm beginning to grasp it.
See the number line as a 1D vector space with basis vector u-hat, where u-hat has coordinates ux and uy. The basis vectors i-hat and j-hat define the 2D coordinate system. The coordinates ux and uy are projections of u-hat onto the x-axis and y-axis respectively.
Say you want to transform this 2D space into the 1D number line, such that L(i-hat) and L(j-hat) are projections of i-hat and j-hat onto the number line. You'll need a 1x2 matrix that will look like [L(i-hat) L(j-hat)]. As I wrote earlier, ux and uy are projections of u-hat onto the x-axis (i-hat) and y-axis (j-hat). Because of symmetry, the projections of i-hat and j-hat onto u-hat (the number line) will also be ux and uy. So, L(i-hat) = ux, L(j-hat) = uy! The matrix will be [ux uy].
Note that ANY vector, v, in the original 2D space is a linear combination of the basis vectors i-hat and j-hat. After the transformation, this still holds true: v = c1 i-hat + c2 j-hat and L(v) = c1 L(i-hat) + c2 L(j-hat). So, because L(i-hat) and L(j-hat) are projections of i-hat and j-hat, L(v) is a projection of v onto the number line!
Now, the vector equivalent of the transformation matrix [ux uy] is just u-hat. If you take the dot product between a vector v and u-hat, you do exactly the same as you do with the transformation; you project v onto u-hat!
So, to generalize: Taking the dot product of vectors v and w is equivalent to transforming the vector v by the matrix [wx wy]. But also equivalent to transforming the vector w by [vx vy].
This video was harder to follow as the motivation was not crystal clear to me all the way through, but it was AMAZING. Soooo just to recap...
If you put a random line through the origin, the MATRIX that represents a particular TRANSFORMATION of PROJECTING any vector in the 2D space onto that line (i.e. collapsing the plane onto the line, but PERPENDICULARLY so to say) is given by the [ ux uy ], where ux and uy are x and y components of u hat, a unit length vector oriented parallel to that random line. This means that this particular unit vector U HAT is in this DUALITY context associated to a TRANSFORMATION matrix that collapses whole 2D plane onto THAT EXACT LINE - and does that PERPENDICULARLY.
However, if the vector is non-unit, say just u, without the hat, the transformation matrix will essentially collapse the whole 2D plane onto the line again, but this time, the lengths will be stretched and the collapsing process will not be perpendicular, but there is still a duality. AND as an added bonus, this TRANSFORMATION will land any random vector on EXACTLY the value of the DOT PRODUCT of THAT vector and the vector u...
There are a few cases later in this video where you are piling it on with difficult terminology and moving a lot of things in the animations. This can become overwhelming. Try to limit comments when presenting lots of animations, and limit animations when you need to make complicated narrations.
I would love to see more moments in the videos where you are asking us to answer questions or apply our understanding that we are building from the animations. People learn by doing, and the act of passively watching a video lacks the opportunity to exercise this learning mechanism.
Well, you forgot to mention an aspect that makes the fact with the projections even clearer and it's something important, because most high school teachers will first teach students the dot product not in the algebraic way but in this way: the geometric definition of the dot product. Basically, the dot product of two vectors v and u, v • u = |v| • |u| • cos α, where α is the angle between the 2 vectors. The algebraic definition can then be demonstrated starting from the geometric definition in this way. (There is a general demonstration for ℝⁿ, but I will show it for ℝ² for simplicity). Let v = vₓ î + vᵧ ĵ and u = uₓ î + uᵧ ĵ. Imagine projecting v on to u (Just as another point for intuition about projecting, then I'll go back to the proof). The projection's length is equal to |v| cos α. Now, u • v = (vₓ î + vᵧ ĵ)(uₓ î + uᵧ ĵ) = (vₓ î • uₓ î )+ (vₓ î • uᵧ ĵ) + (vᵧ ĵ • uₓ î) + (vᵧ ĵ • uᵧ ĵ). On one hand î • î, by the geometric definition is equal to 1 • 1 • cos 0 (which is 1) = 1. Same for ĵ • ĵ = 1. On the other hand, î • ĵ = 1 • 1 • cos 90° (which is 0) = 0. So u • v = vₓ uₓ + vᵧ uᵧ.
A tip for those that have found this lesson difficult. My case. English is my second language. Play the video at 0.8 speed, and pause the video after each sentence. Just move to the other sentence after getting the concept. After a couple of times it helped me. :)
The deepest parts of the ocean are the least explored places on our planet. Immense water pressure and a complete absence of sunlight make this environment nearly as unwelcoming as outer space. Yet locked in these dark depths could be clues that will help us better understand our world.
When the DEEPSEA CHALLENGER dropped nearly 7 miles (11 kilometers) to the Challenger Deep in the Mariana Trench, its mission was one of scientific exploration and discovery, part of an expedition designed to help scientists study some of the least explored places on our planet. Immense water pressure and a complete absence of sunlight make this environment nearly as unwelcoming as outer space, and yet scientists hoped that the expedition would bring back clues to help us better understand our world.
Photograph by Mark ThiessenThe crew tests one of two unmanned landers that will be released to the bottom prior to the sub.