Linear Algebra
Seven ideas from a second course in linear algebra, done the way Axler does it: operators first, determinants last. Drag vectors to test subspaces and change bases, bend the plane with a matrix, hunt for eigenvectors, project onto subspaces, watch circles become ellipses, then check yourself with concept questions, practice problems and exercises from the book.
Follows the chapter order of S. Axler, Linear Algebra Done Right, 4th ed. (Springer, 2024; open access), Chapters 1–3 and 5–9. Result numbers refer to that book; the questions and examples here are written for this page. Exercises from the book appear at the end of each module, with their numbers from the book and their statements restated in our own words. Exercises from the book appear at the end of each module, with their numbers from the book and their statements restated in our own words. For how this subject feeds the others on this site, see the roadmap.
Vector spaces and subspaces
A vector space is anywhere you can add things and scale them, with the usual rules. A subspace is a subset you can't leave by adding or scaling. That one closure condition rules out most sets you might draw.
The idea
Examples: , , lines and planes through the origin in , the polynomials of degree at most inside all polynomials, the continuous functions inside all functions . The space need not be made of arrows.
Explore
Worked example
Let and in . Is a direct sum? What is ?
Compute .
forces . So , a line, not .
So the sum is not direct. Exhibit a vector with two decompositions.
, with the first term in and the second in both times.
Find .
Every vector in either has , so , a 3-dimensional space. By Axler 2.43, , so they are equal.
Check your understanding
Which of these is a subspace of ?
- It doesn't contain 0.
- leaves the set.
Is the set of polynomials of degree exactly 3 a subspace of ?
- Yes.Add and .
- No.
Three lines through 0 in , pairwise intersecting only at 0. Is their sum direct?
- Yes, the pairwise intersections are .That test is only valid for two subspaces.
- No: the sum is , and 0 can be written as a nonzero sum.
Practice
Functions as vectorsA subspace you can't draw
Show that is a subspace of , but is not.
Solution
The first condition is linear: if and , then ; and 0 satisfies it, as do sums of continuous functions. The second fails at 0: .
Even and oddA direct sum of function spaces
Show that , where and are the even and odd functions.
Solution
Write : even plus odd. If is both even and odd, , so ; by Axler 1.46 the sum is direct.
From Axler, Linear Algebra Done Right (4th ed., Springer, 2024; open access)
Numbers match the book. Each question is restated in our own words; the hints and solutions are written for this page.
Exercise 1C.10Intersections of subspaces
If and are subspaces of , prove that is a subspace of .
Hint
Check the three conditions; each holds in both subspaces separately.
Solution
and , so . If , then is in (closed under addition) and in , so in the intersection. The same for . The argument works for the intersection of any collection of subspaces, which is why "the smallest subspace containing a set" (its span) makes sense.
Exercise 1C.12When is a union a subspace?
Prove that the union of two subspaces of is a subspace if and only if one of them contains the other.
Hint
If neither contains the other, pick and and ask where lives.
Solution
If , the union is , a subspace; likewise if .
Conversely, suppose is a subspace but neither contains the other. Take and . Then . If , then , a contradiction. If , then , a contradiction. (The axes in the figure of this module are the picture.)
Exercise 1C.20A complement in 𝐅⁴
For , find a subspace of with .
Hint
is 2-dimensional, so should be too, and should meet only in 0. Try fixing coordinates 2 and 4.
Solution
Take . Every vector splits: . And if , then , so . By Axler 1.46 the sum is direct.
There are many other choices: any 2-dimensional with works.
Span, independence and bases
A basis is a coordinate system: every vector is a combination of the basis vectors in exactly one way. Spanning gives "at least one way", independence gives "at most one", and the key inequality between them makes dimension well defined.
The idea
The proof is a swap: insert the independent vectors one at a time into the spanning list, and each time the linear dependence lemma (Axler 2.19) lets you throw out one of the original spanning vectors. You never run out, so there were at least as many of them.
Explore
Worked example
Find a basis of and its dimension.
Guess a natural list in .
: all vanish at 6.
Why is it independent?
The degrees are 1, 2, 3, 4: in a combination equal to 0, the top-degree term can't cancel, so its coefficient is 0; repeat downward.
Why does it span ? (Use dimension instead of solving.)
(the constant 1 is missing), so . It contains 4 independent vectors, so , and an independent list of the right length is a basis (Axler 2.38).
Check your understanding
Can four vectors in be linearly independent?
- Yes, if chosen carefully.The standard basis is a spanning list of length 3.
- No.
and are 5-dimensional subspaces of . What is the smallest possible ?
- 0Use .
- 2
- 5That's the largest.
What is ?
- Count .
Practice
Extending to a basisAxler 2.32
Extend to a basis of .
Solution
Add and discard dependent vectors. is not in the span (a combination has middle entry and first entry , last ; gives middle 1). So is independent of length 3, hence a basis.
Complements always existAxler 2.33
Show every subspace of a finite-dimensional has a subspace with . Is unique?
Solution
Extend a basis of to a basis of , and let . Every splits, and by independence. Not unique: in , any line other than itself is a complement of a line . An orthogonal complement (module 5) picks a canonical one.
From Axler, Linear Algebra Done Right (4th ed., Springer, 2024; open access)
Numbers match the book. Each question is restated in our own words; the hints and solutions are written for this page.
Exercise 2A.1Four vectors spanning a plane
Find four distinct vectors in whose span is .
Hint
The plane is 2-dimensional. Any two independent vectors in it span it, and extra vectors in it don't enlarge the span.
Solution
Take , , , . Each lies in the plane. The first two are independent (look at the last two coordinates), so they span a 2-dimensional subspace of the 2-dimensional plane, hence all of it. Adding the other two, which lie in the plane, doesn't change the span. (The plane is 2-dimensional because it is the null space of the nonzero map onto , by the fundamental theorem of linear maps.)
Exercise 2A.6Dependent exactly when c = 8
Show that is linearly dependent in if and only if .
Hint
The first two vectors are independent, so the list is dependent iff the third is a combination of the first two.
Solution
and are not multiples of each other, so they are independent, and by the linear dependence lemma the list is dependent iff for some . The first two coordinates give and , so , . The third coordinate then forces .
Exercise 2C.14Three big subspaces must meet
Let be 10-dimensional and subspaces with . Prove .
Hint
Apply twice, using .
Solution
. Then . So the triple intersection contains a nonzero vector.
Linear maps and rank–nullity
A linear map is determined by what it does to a basis, and whatever it loses (the null space) is exactly what it fails to reach (in dimension count). That is the fundamental theorem of linear maps.
The idea
Consequences: a map to a smaller space can't be injective; a map to a larger space can't be surjective; an operator on a finite-dimensional space is injective iff surjective iff invertible (Axler 3.65). The matrix of records as its columns, and on the determinant is the signed area of the image of the unit square (Axler 9.50: is invertible iff ).
Explore
Worked example
Let be differentiation. Find its null space and range, and check the fundamental theorem.
Null space?
iff is constant: , dimension 1.
Range?
Derivatives of degree- polynomials have degree , and every such is for an antiderivative . So , dimension 4.
Check.
. And since is not injective, it is not surjective either, as the theorem predicts for an operator.
Check your understanding
Can a linear map be injective?
- Yes.Its range has dimension at most 3.
- No.
sends the unit square to a parallelogram of area 3, with orientation reversed.
- 3Orientation reversal flips the sign.
- −3
- 1/3That's up to sign.
, . Injective? Surjective?
- Injective, not surjective.
- Both, since it's an operator.That equivalence needs finite dimension.
Practice
Prescribing a map on a basisAxler 3.4
Is there a linear with , ? With , ?
Solution
First: yes, uniquely, since is a basis; it is . Second: no, since would force .
Rank of a productComposition
Show .
Solution
. Also , the image of a space of dimension under a linear map, so its dimension is at most that (fundamental theorem applied to restricted to ).
From Axler, Linear Algebra Done Right (4th ed., Springer, 2024; open access)
Numbers match the book. Each question is restated in our own words; the hints and solutions are written for this page.
Exercise 3B.1Prescribed null space and range
Give an example of a linear map with and .
Hint
The fundamental theorem says the domain must be 5-dimensional.
Solution
, . Its null space is , of dimension 3, and it is onto .
Exercise 3B.4Maps with a big null space don't form a subspace
Show that is not a subspace of .
Hint
Find two such maps whose sum has a small null space.
Solution
Let and . Both have null spaces of dimension 3. But has null space , of dimension 1. The set is not closed under addition.
Exercise 3B.10Images of a spanning list span the range
If spans and , show spans .
Hint
Write an arbitrary vector of the range as and expand .
Solution
Each , so their span is inside the range. Conversely, any is with , so . This is why the columns of a matrix span the range of the map it represents.
Eigenvalues and eigenvectors
An eigenvector is a direction the operator doesn't turn, only stretches. Find enough of them and the operator becomes a list of numbers acting on independent axes.
The idea
Axler's proof of existence avoids determinants: for nonzero , the vectors are dependent, so some nonzero polynomial has ; factor over into linear factors, and one of the factors must fail to be injective.
Explore
Worked example
Diagonalize and compute .
Find the eigenvalues.
is not invertible iff , so .
Eigenvectors?
: . : . They form a basis, so is diagonalizable with matrix in this basis.
Compute .
, so . In the eigenbasis, powers are just powers of numbers.
Check your understanding
Rotation of by 60° has how many real eigenvalues?
- None.
- One.Every nonzero vector is turned.
- Two.
Is the shear diagonalizable?
- Yes, it is upper triangular.Triangular is weaker than diagonalizable.
- No: its only eigenvalue is 1 and the eigenspace is a line.
An operator on has eigenvalues 1, 2, 3, 4. Must it be diagonalizable?
- Yes.
- Not necessarily.Four distinct eigenvalues on a 4-dimensional space.
Practice
Invariant subspacesAxler 5.2
Show that for any , and are invariant under , and that a one-dimensional subspace is invariant iff it is spanned by an eigenvector.
Solution
If then . If , then . A line is invariant iff for some .
Eigenvalues of a polynomial in TSpectral mapping
If , show for every polynomial .
Solution
by induction, then take linear combinations. Over the converse holds too: every eigenvalue of is for some eigenvalue of .
From Axler, Linear Algebra Done Right (4th ed., Springer, 2024; open access)
Numbers match the book. Each question is restated in our own words; the hints and solutions are written for this page.
Exercise 5A.5A map with no real eigenvalues
Find the eigenvalues of , .
Hint
Compute .
Solution
, so . If with , then , so , impossible for real . So has no eigenvalues. (Regarded on , the eigenvalues are .)
Exercise 5A.10Eigenvectors of x d/dx
Define by . Find all eigenvalues and eigenvectors of .
Hint
Try monomials.
Solution
. So are eigenvalues, with eigenvectors . These are five distinct eigenvalues on a 5-dimensional space, so there are no others (Axler 5.12), each eigenspace is 1-dimensional, and the eigenvectors are exactly the nonzero multiples .
Exercise 5D.1Diagonalizability from a polynomial equation
For on a finite-dimensional complex : (a) if , show is diagonalizable; (b) if , show is diagonalizable; (c) give with that is not diagonalizable.
Hint
The minimal polynomial divides any polynomial that kills . is diagonalizable iff its minimal polynomial has distinct roots (Axler 5.62).
Solution
(a) The minimal polynomial divides , which has distinct roots, so so does the minimal polynomial.
(b) It divides with , again distinct roots.
(c) : , so . Its minimal polynomial is , a repeated root; its only eigenvector direction is .
Inner products and orthogonal projection
An inner product adds lengths and angles. Its most useful consequence: the closest point of a subspace to a vector is its orthogonal projection, which is how least squares, Fourier series and best approximations all work.
The idea
Explore: least squares as a projection
Worked example
Find the polynomial minimizing .
Gram–Schmidt on with .
. , with norm , so .
Project .
, . So .
Check orthogonality.
: and . The residual is orthogonal to , as Axler 6.61 requires.
Check your understanding
When does equality hold in Cauchy–Schwarz?
- When one vector is a scalar multiple of the other.
- When they are orthogonal.Then the left side is 0.
If is the orthogonal projection onto , then
- Only if .
- 0Only if .
In the least squares figure, the sum of the residuals is always 0 for the line model. Why?
- The residual vector is orthogonal to the all-ones column.
- By coincidence of the data.Drag points: it stays 0.
Practice
Pythagoras and the parallelogram lawAxler 6.12, 6.21
Show , and that on does not come from an inner product.
Solution
Expand both squares; the cross terms cancel. For , take , : .
Orthogonal complementsAxler 6.49, 6.51
Show and for finite-dimensional .
Solution
with the second term in ; and since . Then dimensions add (Axler 3.94).
From Axler, Linear Algebra Done Right (4th ed., Springer, 2024; open access)
Numbers match the book. Each question is restated in our own words; the hints and solutions are written for this page.
Exercise 6B.1A converse to Pythagoras
Suppose for all scalars . Show that is orthonormal.
Hint
First take a single nonzero coefficient, then two.
Solution
Taking and the rest 0 gives . For and a scalar , take , : , and the hypothesis says this is . So for every . With the real part of vanishes, and with (in the complex case) the imaginary part does. So .
Exercise 6B.14Exactly 2ᵐ Gram–Schmidt outputs
In a real inner product space, let be independent. Prove there are exactly orthonormal lists with for every .
Hint
The condition determines up to sign: it must be a unit vector in a 1-dimensional subspace.
Solution
Let . If the list works, then and (since ). Now is 1-dimensional (Axler 6.51 inside ), so is one of its two unit vectors. That gives at most lists.
Conversely, Gram–Schmidt gives one valid list , and changing any signs keeps it orthonormal with the same spans. So all sign choices work, and there are exactly .
Exercise 6C.1The perp of a list is the perp of its span
For , prove .
Hint
One inclusion is immediate since each is in the span. For the other, use linearity of the inner product.
Solution
Being orthogonal to the larger set implies being orthogonal to each , so the right side is inside the left. If for every , then , so the left side is inside the right.
The spectral theorem and SVD
Self-adjoint operators are diagonal in an orthonormal basis: perpendicular axes, real stretch factors. Every linear map, self-adjoint or not, is diagonal between two orthonormal bases, which is the singular value decomposition, and it is why every invertible map sends circles to ellipses.
The idea
Geometrically (Axler 7.99): sends the unit ball to an ellipsoid whose semi-axes are , and the are the directions that land on those axes.
Explore
Worked example
Find the singular values of the shear .
Compute .
.
Its eigenvalues?
Trace 3, determinant 1: .
Singular values, and a sanity check.
: the golden ratio and its reciprocal. Their product is , the area scale. The eigenvalues of are both 1, very different from the singular values: is not normal.
Check your understanding
For a self-adjoint operator, the singular values are
- the absolute values of its eigenvalues.
- its eigenvalues.Singular values are never negative.
Which matrices are normal?
- Rotations.
- The shear . here.
The best rank-1 approximation of in operator norm is
- .
- the rank-1 piece with the smallest singular value.Keep the biggest, drop the rest.
Practice
Eigenvalues of self-adjoint operators are realAxler 7.12
Prove it, and prove eigenvectors for distinct eigenvalues are orthogonal.
Solution
. If , with real: , so .
Operator normAxler 7.85
Show .
Solution
With (extended to an orthonormal basis), , with equality at .
From Axler, Linear Algebra Done Right (4th ed., Springer, 2024; open access)
Numbers match the book. Each question is restated in our own words; the hints and solutions are written for this page.
Exercise 7A.4Invariant subspaces and the adjoint
For and a subspace , prove that is invariant under if and only if is invariant under .
Hint
Use , and for the converse apply the first half to .
Solution
Suppose . For and : , since . So .
Conversely, if is invariant under , the first half applied to and says is invariant under (Axler 6.52, 7.5).
Exercise 7E.1Zero singular values
Show is 0 if and only if all its singular values are 0.
Hint
The singular values are the square roots of the eigenvalues of , and .
Solution
If , then , whose eigenvalues are all 0. Conversely, if all singular values are 0, the self-adjoint operator has only the eigenvalue 0. By the spectral theorem it is diagonal with zeros in an orthonormal basis, so . Then for every , so .
Exercise 7E.3Eigenvalues 0, 0 but singular values 5, 0
Give whose only eigenvalue is 0 and whose singular values are 5 and 0.
Hint
A nilpotent map with a large entry.
Solution
, i.e. . It is upper triangular with zero diagonal, so 0 is its only eigenvalue. , so the singular values are 5 and 0. Eigenvalues and singular values agree for normal operators, but this is far from normal.
Generalized eigenvectors and Jordan form
When there aren't enough eigenvectors, generalized ones fill the gap: vectors killed by some power of . Every complex operator splits along them, and on each piece it is a scalar plus a nilpotent part. That nilpotent part is what makes powers grow like .
The idea
For a Jordan block, since . The extra factor is invisible in the eigenvalues but very visible in the dynamics, as the figure shows.
Explore
Worked example
Find the generalized eigenspaces and a Jordan basis for .
Eigenvalues?
The matrix is upper triangular with diagonal , so the eigenvalues are 6 and 7 (Axler 5.41), with multiplicities 2 and 1.
and ?
has matrix with rows ; its square kills and , so , while only. is spanned by (solve ).
Jordan basis?
Take , , and . In the basis , has blocks and .
Check your understanding
A nilpotent operator on satisfies
- .
- .A single Jordan block has .
For a Jordan block with , for every ?
- Yes, though it can grow first.
- No, the term blows up. when .
Over , the trace of equals
- the sum of the eigenvalues, with multiplicity.
- the sum of the distinct eigenvalues.Multiplicity counts.
Practice
Cayley–Hamilton in 2×2Axler 8.29
Verify for .
Solution
. Subtract and add : every entry becomes 0, e.g. .
Square rootsAxler 8.39–8.41
Find a square root of where .
Solution
Use truncated: . In general, the Taylor series truncates because is nilpotent, which is why every invertible complex operator has a square root.
From Axler, Linear Algebra Done Right (4th ed., Springer, 2024; open access)
Numbers match the book. Each question is restated in our own words; the hints and solutions are written for this page.
Exercise 8A.3When null space and range are complementary
For , prove if and only if .
Hint
The sum is direct iff ; dimensions then add to by the fundamental theorem.
Solution
(⇒) Let . Then and , so . Hence . The other inclusion is always true.
(⇐) If , write . Then , so , and . So the sum is direct, and its dimension is (Axler 3.21), so it is all of .
Exercise 8B.1Generalized eigenspaces of a rotation
For on , find the generalized eigenspaces for its distinct eigenvalues.
Hint
. Since the eigenvalues are distinct, generalized eigenspaces equal eigenspaces.
Solution
, so eigenvalues satisfy : . means , i.e. : . Similarly . These two lines already span , so and . On this is the 90° rotation, which has no eigenvectors at all.
Exercise 8C.1A nilpotent operator with no square root
Prove that on has no square root.
Hint
A square root of a nilpotent operator is nilpotent, and nilpotent operators on satisfy .
Solution
and . Suppose . Then , so is nilpotent, and by Axler 8.16, . Then , a contradiction. Compare Axler 8.39: always has a square root, but itself need not.