Norms

Concept of Length and Norms: When we consider a geometric vector (a directed line segment originating from a point like the origin), its “length” intuitively corresponds to the distance from the origin to the endpoint of the segment. In vector spaces, the notion of “length” is formalized through the concept of a norm, which quantifies the size or magnitude of a vector.

Definition of a Norm

A norm on a vector space $V$ is a function $\Vert \cdot \Vert :V\to \mathbb{R}$ that assigns a real number $\Vert x\Vert$ to each vector $x\in V$, representing its “length” or “magnitude.” Formally, a function $\Vert \cdot \Vert$ qualifies as a norm if it satisfies the following properties for all $x,y\in V$ and for any scalar $\lambda \in \mathbb{R}$:

1. Absolute Homogeneity:

   $$\Vert \lambda x\Vert =|\lambda |\Vert x\Vert$$

   This property ensures that scaling a vector by a real number $\lambda$ scales its norm by $|\lambda |$. The norm is proportional to the size of the scalar multiplication, which is key to preserving the concept of length across scaled vectors.

2. Triangle Inequality:

   $$\Vert x+y\Vert \le \Vert x\Vert +\Vert y\Vert$$

   The triangle inequality implies that the length of the sum of two vectors is no greater than the sum of their lengths. Geometrically, this means the shortest path between two points is the direct path rather than a detour, represented by the edges of a triangle in vector addition.

3. Positive Definiteness:

   $$\Vert x\Vert \ge 0\text{and}\Vert x\Vert =0⟺x=0$$

   This property ensures that the norm is non-negative, and it is only zero if the vector itself is the zero vector. This captures the intuitive sense that only the zero vector has no “length.”

Together, these properties ensure that the norm consistently measures the “length” of vectors, adhering to our geometric intuition about distances and magnitudes.

Application to Finite-Dimensional Vector Spaces

While the definition of a norm applies broadly to any vector space $V$, in practice, we often deal with finite-dimensional vector spaces, specifically ${\mathbb{R}}^n$. In ${\mathbb{R}}^n$, each vector $x=(x_1,x_2,\dots ,x_n)$ is represented by its components, and norms help quantify various notions of “length” or “distance” in such spaces.

Examples of Norms

Example 1: Manhattan Norm (ℓ₁ Norm) The Manhattan norm, also known as the ${\ell }_1$ norm, is defined on ${\mathbb{R}}^n$ as:

$$\Vert x{\Vert }_1:=\sum _{i=1}^n|x_i|$$

This norm sums the absolute values of a vector’s components, measuring the total distance traveled along each axis (similar to navigating a grid-like city, hence the name Manhattan).

In two dimensions, the set of all vectors $x\in {\mathbb{R}}^2$ with $\Vert x{\Vert }_1=1$ forms a diamond-shaped unit ball. This diamond is the set of points equidistant from the origin according to the Manhattan distance.

Example 2: Euclidean Norm (ℓ₂ Norm) The Euclidean norm, or ${\ell }_2$ norm, is defined as:

$$\Vert x{\Vert }_2:=\sqrt{\sum _{i=1}^n{x}_i^2}=\sqrt{x^{\top }x}$$

This norm represents the Euclidean distance from the vector’s endpoint to the origin, computed as the square root of the sum of squares of its components. The Euclidean norm is our intuitive notion of “straight-line” distance in ${\mathbb{R}}^n$.

In two dimensions, the set of all vectors $x\in {\mathbb{R}}^2$ with $\Vert x{\Vert }_2=1$ forms a circular unit ball. This set captures the idea that all points on the circle are equidistant from the origin in terms of the Euclidean distance.

Types of Distances

Notational Remark

The Euclidean norm is often used by default unless stated otherwise, given its natural geometric interpretation and simplicity in common vector and matrix calculations.

Geometric Interpretation of the Triangle Inequality

In geometric terms, the triangle inequality (a norm property) states that for any triangle, the sum of the lengths of any two sides is always at least the length of the third side. This property preserves the shortest distance path concept and is foundational in both Euclidean geometry and general vector space theory.

Summary

In summary, norms are essential tools in vector spaces for measuring vector lengths or magnitudes, with properties that ensure consistency with geometric intuition. The Manhattan norm provides a “taxi-cab” or axis-aligned measure, while the Euclidean norm offers a direct, radial distance from the origin, forming the basis for much of the analysis in finite-dimensional vector spaces.