Lengths and Angles of Vectors

1. The Cauchy-Schwarz Inequality

Core Concept

The Cauchy-Schwarz inequality is a fundamental theorem in inner product spaces that establishes a relationship between the inner product of two vectors and their individual magnitudes. The inequality states that:

$⟨x,y{⟩}^2\le ⟨x,x⟩⟨y,y⟩$

This means the square of the inner product of two vectors is always less than or equal to the product of their self-inner products.

Note: This Inequalit holds only for linearly dependent vectors.

Detailed Proof

The proof follows several steps:

1. Base Case: When x = 0

   • The left side becomes $(0,y{)}^2=0$
   • The right side is $(0,0)(y,y)=0$
   • Therefore, the inequality holds trivially

2. Main Case: When x ≠ 0

   • We consider the expression $(tx+y,tx+y)$ for any scalar t
   • This must be ≥ 0 due to the positive definiteness of inner products
   • Expanding this:
     • $⟨tx+y,tx+y⟩=t^2⟨x,x⟩+2t⟨x,y⟩+⟨y,y⟩$

3. Quadratic Analysis:

   • The expression forms a quadratic in t
   • Since it’s always ≥ 0, this quadratic either: a) Has no real roots b) Has exactly one repeated real root
   • Both conditions require the discriminant to be ≤ 0
   • Discriminant formula: $4(x,y{)}^2-4(x,x)(y,y)\le 0$
   • Simplifying: $(x,y{)}^2\le (x,x)(y,y)$

Significance

This inequality is crucial because:

1. It allows us to define angles between vectors
2. It establishes bounds on inner products
3. It generalizes the concept of angle cosine from Euclidean geometry

2. Vector Magnitude and Length

Definition and Properties

The magnitude (or length) of a vector x in an inner product space is defined as:

$\Vert x\Vert =\sqrt{⟨x,x⟩}$

This definition generalizes our intuitive understanding of length from Euclidean space to any inner product space.

Key Properties:

1. Non-negativity: $\Vert x\Vert \ge 0$

   • This follows from the positive definiteness of inner products
   • Equality holds iff x = 0

2. Scalar Multiplication: $\Vert cx\Vert =|c|\Vert x\Vert$

   • This shows how scaling affects length
   • Reflects our intuitive understanding of scaling

3. Triangle Inequality: $\Vert x+y\Vert \le \Vert x\Vert +\Vert y\Vert$

   • Generalizes the triangle inequality from Euclidean geometry
   • Fundamental for establishing metric properties

Distance Function

The distance between vectors is defined as: $d(x,y)=\Vert x-y\Vert$

This definition leads to several important properties:

1. Non-negativity: $d(x,y)\ge 0$
2. Identity of Indiscernibles: $d(x,y)=0$ iff x = y
3. Symmetry: $d(x,y)=d(y,x)$
4. Triangle Inequality: $d(x,z)\le d(x,y)+d(y,z)$

Normalized Inner Product:

From Cauchy-Schwarz: $-1\le \frac{⟨x,y⟩}{\Vert x\Vert \Vert y\Vert }\le 1$

Derivation:

1. Start with Cauchy-Schwarz: $|⟨x,y⟩{|}^2\le \Vert x{\Vert }^2\Vert y{\Vert }^2$
2. Take square root: $|⟨x,y⟩|\le \Vert x\Vert \Vert y\Vert$
3. Divide by $\Vert x\Vert \Vert y\Vert$ (assuming both non-zero): $\frac{|⟨x,y⟩|}{\Vert x\Vert \Vert y\Vert }\le 1$
4. Since this is a ratio of real numbers: $-1\le \frac{⟨x,y⟩}{\Vert x\Vert \Vert y\Vert }\le 1$

hence, there is a unique number $\theta \in [0,\pi ]$ such that $\cos \theta =\frac{⟨x,y⟩}{\Vert x\Vert \Vert y\Vert }$

3. Angles Between Vectors

Mathematical Definition

The angle between vectors is defined through the normalized inner product:

$\cos \theta =\frac{⟨x,y⟩}{\Vert x\Vert \Vert y\Vert }$

Detailed Analysis

1. Domain Restriction:

   • The Cauchy-Schwarz inequality ensures:
   • $-1\le \frac{(x,y)}{\Vert x\Vert \Vert y\Vert }\le 1$
   • This makes $\theta$ well-defined in $[0,\pi ]$

2. Geometric Interpretation:

   • For $\theta =0$: vectors point in same direction
   • For $\theta =\pi /2$: vectors are perpendicular
   • For $\theta =\pi$: vectors point in opposite directions

Example Working

Consider the example in ℝ² with inner product $⟨x,y⟩=2x_1{y}_1+3x_2{y}_2$:

For x = (1,2) and y = (1,0):

1. Calculate $\Vert x\Vert$:

   • $(x,x)=2(1{)}^2+3(2{)}^2=2+12=14$
   • $\Vert x\Vert =\sqrt{14}$

2. Calculate $\Vert y\Vert$:

   • $(y,y)=2(1{)}^2+3(0{)}^2=2$
   • $\Vert y\Vert =\sqrt{2}$

3. Calculate $(x,y)$:

   • $(x,y)=2(1)(1)+3(2)(0)=2$

4. Therefore: $\cos \theta =\frac{2}{\sqrt{14}\sqrt{2}}$

4. Orthogonality

Definition and Properties

Two vectors are orthogonal if their inner product is zero: $⟨x,y⟩=0$

Key Theorems

1. Lemma 5.2:

   • Statement: A vector x is orthogonal to all vectors in inner product space $V$ iff x = 0
   • Proof:
     • Forward: If x = 0, clearly orthogonal to all vectors
     • Reverse: If orthogonal to all vectors, then $⟨x,y⟩=0$ for all $y\in V$ which implies $⟨x,x⟩=0$ By positive definiteness, x must be 0

2. Corollary 5.3:

   • For a basis $\alpha =\{v_1,...,v_n\}$ of inner product space $V$
   • If x is orthogonal to all basis vectors, then x = 0
   • Proof:
     • If $⟨x,v_i⟩=0$ for $i=0,1,2,...,n$ then $⟨x,y⟩={\sum }_{i=1}^n{y}_i⟨x,v_i⟩=0$ for any $y={\sum }_{i=1}^n{y}_i{v}_i\in V$
     • Consider an arbitrary vector y ∈ V. Since α is a basis, any $y\in V$ can be written as a linear combination of basis vectors: $y={\sum }_{i=1}^n{y}_i{v}_i$ where $y_i$ are scalars
     • Examine inner product of x with y $⟨x,y⟩=⟨x,{\sum }_{i=1}^n{y}_i{v}_i⟩$
     • By linearity in second argument: $⟨x,y⟩={\sum }_{i=1}^n{y}_i⟨x,v_i⟩$
     • Apply orthogonality assumption. Since $⟨x,v_i⟩=0$ for all i. Therefore: $⟨x,y⟩={\sum }_{i=1}^n{y}_i(0)=0$
     • We’ve shown that x is orthogonal to every vector y ∈ V. By Lemma 5.2 (which states that if a vector is orthogonal to all vectors in V, it must be zero). Therefore, x = 0

The Pythagorean Theorem

Let $V$ be an inner product space, and let $\mathbf{x}$ and $\mathbf{y}$ be any two vectors in $V$ with the angle $\theta$. Then, $⟨\mathbf{x},\mathbf{y}⟩=\Vert \mathbf{x}\Vert \Vert \mathbf{y}\Vert \cos \theta$ gives the equality

$$\Vert \mathbf{x}+\mathbf{y}{\Vert }^2=\Vert \mathbf{x}{\Vert }^2+\Vert \mathbf{y}{\Vert }^2+2\Vert \mathbf{x}\Vert \Vert \mathbf{y}\Vert \cos \theta .$$

Moreover, it deduces the Pythagorean theorem:

$$\Vert \mathbf{x}+\mathbf{y}{\Vert }^2=\Vert \mathbf{x}{\Vert }^2+\Vert \mathbf{y}{\Vert }^2$$

for any orthogonal vector $\mathbf{x}$ and $\mathbf{y}$. □

5. Linear Independence and Orthogonality

Statement: If $\{{\mathbf{x}}_1,{\mathbf{x}}_2,\dots ,{\mathbf{x}}_k\}$ nonzero vectors in an inner product space $V$ are mutually orthogonal (i.e., each vector is orthogonal to every other vector), then they are linearly independent.

Detailed Proof:

1. Suppose $c_1{x}_1+c_2{x}_2+...+c_k{x}_k=0$
2. Take inner product with $x_i$: $⟨c_1{x}_1+...+c_k{x}_k,x_i⟩=0$
3. By orthogonality: $c_i\Vert x_i{\Vert }^2=0$
4. Since $x_i\ne 0$, $c_i=0$
5. Therefore, vectors are linearly independent

This provides a powerful way to verify linear independence through orthogonality.