Vectors and linear combinations

1.1 – What Is Linear Algebra?

It feels deeply intuitive to speak of our thoughts as if they were distributed through space: we can talk about the closeness of ideas, an argument following a particular path, or a line of reasoning that moves us away from certainty. This almost natural tendency to imagine thought as wandering through space finds in algebra a way to become rigorous. Algebra allows us to construct abstract landscapes inhabited by complex entities such as meanings or decisions, no longer as vague shadows of the mind, but as inhabitants of a territory that can be described, measured, and traversed.

This idea lies at the very heart of contemporary AI. Much of what we now call Artificial Intelligence consists precisely in learning how to place complex entities within spaces where proximity is meaningful, where distance expresses similarity or difference, where certain directions correspond to interpretable changes, and where paths of transformation can even be traced from one representation to another.

Within this territory, Linear Algebra emerges not merely as a branch of mathematics, but as a way of giving structure to things that have no physical form. A vector is not just an arrow drawn on a plane: it can become the numerical footprint of a word, a person, a film, or a face, occupying a position in an abstract space whose dimensions are no longer the width, height, or depth we are accustomed to in three-dimensional space.

For this reason, learning Linear Algebra is not merely about learning operations with matrices and vectors; it is about learning to view knowledge as a landscape. We will begin this journey with the most basic idea from which the algebra grows: the concept of a vector.

1.2 – What Is a Vector?

From a certain point of view, a vector is nothing more than an ordered list of numbers. This generality makes Linear Algebra a ubiquitous discipline, common to fields such as engineering, economics, and even the social sciences. Originally used to represent motion and displacement through space, vectors are now, with the rise of Data Science, used to represent many other kinds of information.

Any entity that can be represented by two or more numbers can be treated as a vector. Each number within the vector is known as a component. For example, \( \left( 15, 167 \right) \) would be a two-component vector: the first component could represent the age and the second the height of a particular person. Throughout this series, we will frequently switch between horizontal and vertical vector notation. The previous example would appear vertically as \( \begin{bmatrix} 15 \\ 167 \end{bmatrix} \). Using the first notation, a general two-component vector can be written as:

$$ \mathbf{v} = \left( v_1, v_2 \right)$$

Depending on the field of application, it is common to work with vectors containing more than two or three components, even though this forces us to abandon our spatial intuition. The general notation is:

$$ \mathbf{v} = \left( v_1, v_2, \cdots v_n\right)$$

for a vector with \( n \) components.

Thinking of vectors as numerical lists is still only an approximation. The concept is actually more general: what truly characterizes vectors is not just their form, but the operations we can perform on them. In the next sections, we will see that vectors can be added and also multiplied by numbers. The entire structure of Linear Algebra emerges from these two simple operations.

Vector Space

This plane represents a two-dimensional Cartesian space, with two axes. In Data Science, for example, vectors allow us to describe numerical facts. Using an example deliberately removed from displacement across a plane, we can interpret this vector as a representation of a transaction in the oil market. Volume is represented on the horizontal axis, while price (let us assume in dollars) is represented on the vertical axis. One barrel of oil is approximately 159 liters. If we move the vector upward while keeping its horizontal position fixed, we can represent different prices for the same volume. Movements along the horizontal axis would correspond to transactions involving larger or smaller volumes.

1.3 – Vector Addition

The most elementary of these operations is vector addition. It is performed by adding the corresponding components:

$$ \mathbf{v} = \left( v_1, v_2 \right)$$

$$ \mathbf{u} = \left( u_1, u_2 \right)$$

$$ \mathbf{v} + \mathbf{u} = \left( v_1 + u_1, v_2 + u_2 \right)$$

If we interpret vectors as displacements, vector addition is equivalent to combining those two displacements: the resulting vector represents the total displacement.

Vector addition is commonly described using the parallelogram rule, a visual tool for understanding the operation. The parallelogram is formed by translating each vector so that it begins at the tip of the other. The diagonal running from the origin to the opposite vertex is precisely the sum vector \( \mathbf{v} + \mathbf{u} \).

In many cases, we will represent vectors as ‘arrows’ emerging from the origin of the plane. The parallelogram rule is easy to understand using this representation. However, the reader should keep in mind that this is only one way of visualizing the concept of a vector, which can just as well be conceived as a simple point in the plane. In either case, an origin is always required as a reference point.

Let us continue with the interpretation from the previous section. The price of a barrel of oil is not the same in every market in the world (Brent, WTI, etc…). Two vectors with different slopes would represent different market prices. Even so, we can add them together to calculate the combined volume and price of both transactions.

1.4 – Scalar Multiplication

Scalar multiplication consists of multiplying a vector by a real number.

$$ a\mathbf{v} = \left( av_1, av_2 \right)$$

Here we are no longer combining two displacements as in vector addition, but extending or ‘scaling’ a single displacement. All scalar multiples of a given vector share the same direction, which can be visualized as the line extending indefinitely in both directions along the vector.

Positive values of \(a\) lengthen the vector, while negative values also reverse its orientation, always preserving the same direction. Values of \(a\) between \(0\) and \(1\), by contrast, shorten it, and multiplying by \(0\) returns the zero vector located at the origin.

The reader should remember that vector addition and scalar multiplication are not limited to vectors with two or three components. Vectors are much more than conceptual tools used to describe displacements in the three-dimensional space with which we are familiar.

Scaling a vector allows us to represent transactions within the same market, that is, while preserving the same price-to-volume relationship. Both dimensions now grow proportionally. Scaling does not change the slope of the vector, so its direction always remains the same.

1.5 – Linear Combination

Now that vector addition and scalar multiplication have been established, we can combine them in what is known as a linear combination, perhaps the most important concept in this chapter, and possibly in all of Linear Algebra. A linear combination of vectors is simply the sum of two or more vectors, each scaled by a real number. The result is another vector:

$$ \mathbf{p} = a\mathbf{v} + b\mathbf{u}$$

Linear combination is the operation that allows us to move on to another fundamental concept. If we speak of positions and displacements, it is because we have implicitly assumed a space in which these concepts make sense. Vectors presuppose a space that they ‘inhabit’ and can ‘traverse’. Linear combinations allow us to formulate this idea of space more precisely. The set of all possible linear combinations of certain vectors constitutes the space spanned by those vectors. In other words, from just a few vectors we can describe every position that can be reached by combining them through addition and scalar multiplication.

Consider, for example, the vectors \( \begin{bmatrix} 1 \\ 0 \end{bmatrix} \) and \( \begin{bmatrix} 0 \\ 1 \end{bmatrix} \). Any pair of vectors inhabiting the two-dimensional space \( \mathbb{R}^2 \) can be described from these two as a particular linear combination, simply by changing the values of the scalars \( a \) and \( b \).

$$ \mathbf{p}= a\begin{bmatrix} 1 \\ 0 \end{bmatrix} + b\begin{bmatrix} 0 \\ 1 \end{bmatrix}$$

For example, the point \( \begin{bmatrix} 7 \\ 9 \end{bmatrix} \) in the plane would be

$$\begin{bmatrix} 7 \\ 9 \end{bmatrix}= 7\begin{bmatrix} 1 \\ 0 \end{bmatrix} + 9\begin{bmatrix} 0 \\ 1 \end{bmatrix}$$

Spanning a space means having the vectors needed to traverse it completely and describe any position within it.

Any transaction can now be represented as a linear combination of two vectors. The scalars in the animation are restricted. But if we allowed any real number, we would see that every point in the space is accessible using two vectors.

This last statement is only partially true. If we place one of the source vectors on top of the other so that they have the same direction, we can observe that the possible results of their linear combinations are confined to that direction in the plane. No matter how much we scale either of them, we will never escape that line, leaving the rest of the two-dimensional plane inaccessible. For two vectors combined linearly to allow us to reach the entire two-dimensional plane, they must satisfy one property: independence.

1.6 – Linear Independence

In the two-dimensional plane \( \mathbb{R}^2 \), two vectors do not always span the entire plane. There are two cases in which this is not possible. The first occurs when one of the vectors is the zero vector, whose components are \( 0 \), and therefore contributes no new direction. The other occurs when one vector is a scalar multiple of the other. In this case, the vectors are superimposed along the same direction. Since they do not provide a second direction, the two-dimensional plane becomes inaccessible. Although the vectors still have two components, the space they span is not two-dimensional, but a line.

It is therefore important to distinguish the components of a vector from the dimensions of the space it spans when combined with other vectors. Those dimensions depend on the number of directions generated by those vectors. In the same way that moving across a flat sheet of paper requires not only moving from left to right but also from top to bottom, spanning the two-dimensional plane requires two distinct directions.

Two vectors contribute distinct directions as long as they exhibit linear independence. We say that a set of vectors is linearly independent when none of them can be obtained as a linear combination of the others. When considering only two vectors, this simply means that neither is a scalar multiple of the other.

From now on, we will use vertical notation, which will be useful for representing linear combinations. An example of two dependent vectors would be:

since they are scalar multiples:

Two independent vectors could be:

since there is no scalar value that transforms one into the other.

In the case of three vectors, they are linearly dependent if one of them is a linear combination of the other two. For example:

since:

Since one of the vectors is linearly dependent, this set spans only a two-dimensional space, despite having three components.

Although three-dimensional visualization is somewhat more complex, we show it here to illustrate that these properties hold regardless of the dimensionality of the space. The buttons above allow you to choose different sets of vectors with different numbers of independent vectors.

With three linearly independent vectors, the space spanned by their linear combinations includes every point in three-dimensional space. With only two independent vectors, one of the vectors is a linear combination of the others and contributes no new direction (specifically, \( \mathbf{w} = \mathbf{u} + \mathbf{v} \)), so the combination of all three can span only a two-dimensional space. The turquoise vector cannot leave the plane. A single independent vector, in turn, spans only a line, that is, a one-dimensional space.

We can also observe that, even with three independent vectors, if one of the vectors in the set is the zero vector, which is obtained by scaling by \( 0 \), then the set automatically becomes linearly dependent, and you will see that the spanned space is once again limited to two dimensions.

1.7 – Bases

The concept of a basis is essentially an obvious consequence of the ideas introduced previously. If anything, it results from applying a certain minimalism to them. We have seen that a two-dimensional space can be described using two vectors, but also using three, four, and so on. We call a set a basis of a space when it contains the minimum number of vectors needed to span that space. In other words, any set of \( n \) linearly independent vectors is a basis of an \( n \)-dimensional space. Two vectors form a basis of the 2D plane as long as they are linearly independent. More vectors could also span a two-dimensional space, but they would no longer form a basis, since at least one vector would be linearly dependent and the set would therefore contain redundant directions.

For every \( n \)-dimensional space there are infinitely many possible bases, but there is one special case: the standard basis. It is unique to each coordinate space, and each basis vector has a component equal to \( 1 \) along one direction. For 2D space, it is:

and for 3D:

The solid vectors show the basis of the plane, with the standard basis displayed by default. By keeping the basis vectors fixed and scaling their values, it is easy to see that any vector or point in the plane is a linear combination of those basis vectors. And this is the fundamental idea behind bases: they are the building blocks of every other vector in a space.

One important detail is that the same point in space can be reached using different bases. This anticipates the idea of change of basis, which we will explore later.