Dot Product #

Linear AlgebraDifficulty: ★★☆☆☆Depth: 2Unlocks: 13

Sum of component products. Measures angle between vectors.

Interactive Visualization #

⏮◀◀▶▶STEP0.25x1xZOOM

t=0s

Core Concepts #

-Algebraic definition: dot product = sum of corresponding component products (produces a scalar)
-Geometric definition: dot product = product of magnitudes times cosine of the angle between vectors (measures angle)

Key Symbols & Notation #

dot operator: '·' (binary operator between vectors)

Essential Relationships #

-Algebraic equals geometric: sum-of-products = |u||v|cos(theta)
-Orthogonality test: dot product = 0 if and only if vectors are perpendicular

Prerequisites (1) #

Vectors Introduction6 atoms

Unlocks (4) #

Orthogonalitylvl 3 Normslvl 2 Projectionslvl 3 Kernel Methodslvl 4

Advanced Learning Details

Graph Position #

Depth Cost

Fan-Out (ROI)

Bottleneck Score

Chain Length

Cognitive Load #

Atomic Elements

Total Elements

Percentile Level

Atomic Level

All Concepts (5) #

- Dot product (algebraic): an operation that multiplies corresponding components of two vectors and sums those products to produce a single scalar
- Dot product (geometric): the dot product encodes angle information between vectors - its sign and magnitude relate to whether vectors are acute/obtuse and how closely aligned they are
- Scalar result: the dot product of two vectors is a scalar (not a vector)
- Orthogonality criterion: two vectors are perpendicular exactly when their dot product is zero
- Self-dot equals squared magnitude: dotting a vector with itself gives the square of its length (magnitude)

Teaching Strategy #

Self-serve tutorial - low prerequisites, straightforward concepts.

The dot product is the bridge between “lists of numbers” and “geometry.” It turns two vectors into a single scalar that tells you how aligned they are—crucial for angles, lengths, projections, and many ML similarity measures.

TL;DR:

For vectors a, b ∈ ℝⁿ, the dot product is a · b = ∑ᵢ aᵢbᵢ (a scalar). Geometrically, a · b = ‖a‖‖b‖cos θ, where θ is the angle between them. Positive means “mostly same direction,” zero means perpendicular, negative means “mostly opposite.”

What Is Dot Product? #

Why this concept exists (motivation) #

When you have two vectors a and b, you often want a single number that answers: “How similar are their directions?”

•If a and b point the same way, you want a large positive number.
•If they are perpendicular, you want 0.
•If they point opposite ways, you want a negative number.

The dot product is designed to do exactly that, while still behaving nicely with algebra (linearity).

Algebraic definition (component-wise) #

For a = (a₁, a₂, …, aₙ) and b = (b₁, b₂, …, bₙ) in ℝⁿ, the dot product is

a · b = ∑ᵢ₌₁ⁿ aᵢ bᵢ

It is a scalar (a single number), not a vector.

To keep the pacing clear, here’s the 2D and 3D special cases:

•In ℝ²: (a₁, a₂) · (b₁, b₂) = a₁b₁ + a₂b₂
•In ℝ³: (a₁, a₂, a₃) · (b₁, b₂, b₃) = a₁b₁ + a₂b₂ + a₃b₃

Geometric definition (angle + lengths) #

If θ is the angle between nonzero vectors a and b, then

a · b = ‖a‖ ‖b‖ cos θ

This is powerful because it connects an algebraic computation (sum of products) to geometry (angles).

Interpreting the sign and magnitude #

From a · b = ‖a‖‖b‖cos θ:

•If 0 < θ < π/2, then cos θ > 0 ⇒ a · b > 0 (acute angle, aligned).
•If θ = π/2, then cos θ = 0 ⇒ a · b = 0 (perpendicular).
•If π/2 < θ < π, then cos θ < 0 ⇒ a · b < 0 (obtuse angle, opposed).

So the dot product is a “direction agreement score,” scaled by how long the vectors are.

A useful mental model: shadow / projection #

Think of a · b as: “How much of a lies along b (or vice versa), times ‖b‖.”

More precisely, if b ≠ 0, then

a · b = ‖b‖ · (signed length of the projection of a onto the direction of b)

We’ll make that precise later, but it’s a great intuition for what dot products mean.

Core Mechanic 1: Computing Dot Products (and the algebra you can trust) #

How to compute it reliably #

Given a, b ∈ ℝⁿ:

1)Pair up corresponding components (aᵢ with bᵢ).
2)Multiply each pair.
3)Sum the products.

Example structure:

a · b = a₁b₁ + a₂b₂ + … + aₙbₙ

Key properties (why we like dot product) #

These properties are the reason dot product becomes the default “inner product” in ℝⁿ.

1) Commutative #

a · b = b · a

Because multiplication of real numbers is commutative: aᵢbᵢ = bᵢaᵢ.

2) Distributive over addition #

a · (b + c) = a · b + a · c

Show it component-wise:

b + c = (b₁ + c₁, …, bₙ + cₙ)

Then

a · (b + c)

= ∑ᵢ aᵢ(bᵢ + cᵢ)

= ∑ᵢ (aᵢbᵢ + aᵢcᵢ)

= ∑ᵢ aᵢbᵢ + ∑ᵢ aᵢcᵢ

= a · b + a · c

This “linearity” is essential for projections and least squares later.

3) Homogeneous with scalars #

(ka) · b = k(a · b) and a · (kb) = k(a · b)

Component-wise:

(ka) · b = ∑ᵢ (kaᵢ)bᵢ = k∑ᵢ aᵢbᵢ = k(a · b)

4) Positive definiteness (dot with itself) #

a · a ≥ 0, and a · a = 0 ⇔ a = 0

Because

a · a = ∑ᵢ aᵢ²

A sum of squares can’t be negative, and it’s zero only if every aᵢ = 0.

Dot product and length (preview of norms) #

This property gives you Euclidean length:

‖a‖ = √(a · a)

Check quickly:

a · a = a₁² + … + aₙ²

‖a‖ = √(a₁² + … + aₙ²)

This connection is a big reason the dot product is everywhere: it defines the usual geometry of ℝⁿ.

Table: operations to keep straight #

Object	Example	Result type	Notes
Scalar	3	number	no direction
Vector	a = (1, 2)	vector	magnitude + direction
Scalar multiply	3a	vector	scales length
Vector add	a + b	vector	head-to-tail
Dot product	a · b	scalar	measures alignment

Takeaway: dot product is the main way we turn “two directions” into “one number.”

Core Mechanic 2: Geometry—Angles, Orthogonality, and Cosine #

Why geometry matters here #

The algebraic definition ∑ᵢ aᵢbᵢ is easy to compute, but it doesn’t look like an angle measure.

The geometric form

a · b = ‖a‖‖b‖cos θ

explains what the dot product is doing: it compares directions.

Deriving a practical angle formula #

If a ≠ 0 and b ≠ 0, you can solve for cos θ:

a · b = ‖a‖‖b‖cos θ

Divide both sides by ‖a‖‖b‖:

cos θ = (a · b) / (‖a‖‖b‖)

Then

θ = arccos( (a · b) / (‖a‖‖b‖) )

This is how you compute angles between vectors in any dimension.

Orthogonality criterion (the dot-product test) #

Two vectors are orthogonal (perpendicular) exactly when their dot product is zero:

a ⟂ b ⇔ a · b = 0

Geometric reason:

a · b = ‖a‖‖b‖cos(π/2) = ‖a‖‖b‖·0 = 0

Algebraic usefulness: you don’t need to “draw” vectors to test perpendicularity.

Unit vectors and simplifying the dot product #

If you normalize vectors to unit length:

û = a / ‖a‖, v̂ = b / ‖b‖

Then

û · v̂ = cos θ

So for unit vectors, the dot product directly equals cosine similarity.

Cauchy–Schwarz inequality (why cosine stays in range) #

A critical fact (you’ll use it constantly later) is:

|a · b| ≤ ‖a‖‖b‖

This implies

-1 ≤ (a · b) / (‖a‖‖b‖) ≤ 1

So arccos is well-defined (up to floating-point errors in practice).

We won’t fully prove Cauchy–Schwarz here, but intuitively: the projection of one vector onto another can’t be longer than the original.

A “projection-flavored” identity to remember #

For b ≠ 0, the scalar projection (signed) of a onto b is

comp_{b}(a) = (a · b) / ‖b‖

And the vector projection is

proj_{b}(a) = ((a · b) / (‖b‖²)) b

Notice how the dot product is the engine: it extracts the amount of alignment needed to scale b.

This is the gateway to the node Projections.

Application/Connection: Similarity, Projections, and Kernels #

1) Similarity in data (ML intuition) #

In machine learning and information retrieval, vectors often represent data:

•A document as a “bag-of-words” vector
•A user as a preference vector
•An embedding as a feature vector

The dot product a · b becomes a similarity score: larger means more aligned.

But there’s a catch: dot product depends on length. If one vector has huge magnitude, it can dominate the score.

That motivates cosine similarity:

cos θ = (a · b) / (‖a‖‖b‖)

This measures alignment independent of scale.

2) Projections and least squares (linear algebra intuition) #

Suppose you want the “part of a in the direction of b.” The projection formula

proj_{b}(a) = ((a · b) / (‖b‖²)) b

is fundamental for:

•projecting onto a line or subspace
•deriving least-squares solutions
•understanding orthogonality of residuals

In least squares, the key condition is often: residual r is orthogonal to the subspace, meaning r · v = 0 for all basis vectors v in the subspace.

3) Norms and distances #

Because ‖a‖ = √(a · a), dot product defines Euclidean geometry.

A classic identity connects dot products and distances:

‖a − b‖² = (a − b) · (a − b)

Expand:

(a − b) · (a − b)

= a · a − a · b − b · a + b · b

= ‖a‖² − 2(a · b) + ‖b‖²

This shows dot products are enough to compute squared distances.

4) Kernel methods: dot products in disguise #

Kernel methods (like SVMs with kernels) revolve around computing

ϕ(x) · ϕ(z)

without explicitly forming the feature map ϕ.

A kernel function K(x, z) is essentially a dot product in some (possibly huge) feature space:

K(x, z) = ϕ(x) · ϕ(z)

So understanding dot products deeply is a prerequisite for the “kernel trick” intuition: learning with geometry in feature space while doing computations in input space.

Where this node points next #

•Orthogonality is the “dot product equals zero” worldview.
•Norms use √(a · a) to define length.
•Projections use (a · b) / (‖b‖²) as the scaling factor.
•Kernel methods generalize dot products to nonlinear similarity measures.

Worked Examples (3) #

Compute a dot product and interpret the sign #

Let a = (2, −1, 3) and b = (4, 0, −2). Compute a · b and interpret what it suggests about their directions.

Write the component-wise formula:
a · b = a₁b₁ + a₂b₂ + a₃b₃
Substitute values:
a · b = (2)(4) + (−1)(0) + (3)(−2)
Compute each product:
(2)(4) = 8
(−1)(0) = 0
(3)(−2) = −6
Sum:
a · b = 8 + 0 − 6 = 2

Insight: The dot product is positive (2), so the angle θ between a and b is acute (θ < π/2). They are somewhat aligned, but not strongly—if they were strongly aligned, the dot product would be large relative to ‖a‖‖b‖.

Find the angle between two vectors #

Let a = (1, 2) and b = (2, 1). Find the angle θ between them.

Compute the dot product:
a · b = (1)(2) + (2)(1) = 2 + 2 = 4
Compute lengths:
‖a‖ = √(a · a) = √(1² + 2²) = √5
‖b‖ = √(b · b) = √(2² + 1²) = √5
Use cos θ = (a · b) / (‖a‖‖b‖):
cos θ = 4 / (√5 · √5) = 4 / 5
Solve for θ:
θ = arccos(4/5)
≈ 0.6435 radians
≈ 36.87°

Insight: Even in 2D, the dot product gives an angle formula that generalizes perfectly to ℝⁿ. The computation used only ∑ᵢ aᵢbᵢ and square roots.

Use dot product to compute a projection onto a direction #

Project a = (3, 4) onto b = (1, 0).

Compute a · b:
a · b = (3)(1) + (4)(0) = 3
Compute ‖b‖²:
‖b‖² = b · b = 1² + 0² = 1
Apply projection formula:
proj_{b}(a) = ((a · b) / (‖b‖²)) b
Substitute:
proj_{b}(a) = (3/1)(1, 0) = (3, 0)

Insight: Since b points along the x-axis, the projection should be “keep the x part, drop the y part.” The dot product extracts exactly that aligned amount.

Key Takeaways #

✓
Dot product maps two vectors to a scalar: a · b = ∑ᵢ aᵢbᵢ.
✓
Geometric meaning: a · b = ‖a‖‖b‖cos θ, so it measures directional alignment.
✓
a · b = 0 ⇔ a and b are orthogonal (perpendicular).
✓
‖a‖ = √(a · a) connects dot product to Euclidean length.
✓
Angle formula: θ = arccos((a · b) / (‖a‖‖b‖)) for nonzero vectors.
✓
Projection relies on dot product: proj_{b}(a) = ((a · b) / ‖b‖²) b.
✓
Dot products underlie cosine similarity, distances, least squares geometry, and kernels.

Common Mistakes #

✗
Forgetting the dot product returns a scalar (not a vector), and trying to treat it like component-wise multiplication.
✗
Mixing up a · b with ‖a‖‖b‖ (missing the cos θ factor). Alignment matters, not just lengths.
✗
Computing an angle when one vector is 0 (θ is undefined because you can’t divide by ‖a‖‖b‖).
✗
Assuming a large dot product always means “very similar” without considering magnitudes (cosine similarity fixes this).

Practice #

easy

Compute a · b for a = (−2, 5, 1) and b = (3, 1, 4).

Hint: Multiply corresponding components and sum: (−2)(3) + 5(1) + 1(4).

Show solution

a · b = (−2)(3) + (5)(1) + (1)(4)

= −6 + 5 + 4

= 3

medium

Find a nonzero vector x in ℝ² that is orthogonal to a = (3, −6).

Hint: Solve a · x = 0. Let x = (x₁, x₂) and enforce 3x₁ + (−6)x₂ = 0.

Show solution

Let x = (x₁, x₂). Orthogonality means:

a · x = 0

(3, −6) · (x₁, x₂) = 3x₁ − 6x₂ = 0

So 3x₁ = 6x₂ ⇒ x₁ = 2x₂.

Choose x₂ = 1 ⇒ x₁ = 2.

One valid answer is x = (2, 1). (Any nonzero scalar multiple also works.)

hard

Let a = (1, 2, 2) and b = (2, 0, 1). Compute cos θ between them and decide whether the angle is acute, right, or obtuse.

Hint: Compute a · b, then ‖a‖ and ‖b‖, then cos θ = (a · b) / (‖a‖‖b‖). The sign of cos θ tells acute/obtuse.

Show solution

Compute dot product:

a · b = (1)(2) + (2)(0) + (2)(1) = 2 + 0 + 2 = 4

Compute norms:

‖a‖ = √(1² + 2² + 2²) = √(1 + 4 + 4) = √9 = 3

‖b‖ = √(2² + 0² + 1²) = √(4 + 0 + 1) = √5

Compute cosine:

cos θ = 4 / (3√5)

Since 4 / (3√5) > 0, the angle is acute (θ < π/2).

Connections #

Next nodes you can unlock:

•Orthogonality: Uses the criterion a · b = 0 as the definition of perpendicularity.
•Norms: Builds Euclidean length from √(a · a) and compares other norm choices.
•Projections: Uses dot product to compute components along directions/subspaces and leads to least squares.
•Kernel Methods: Generalizes dot products into kernels K(x, z) = ϕ(x) · ϕ(z).

Quality: A (4.5/5)

← back to tree browse all →