# Basics in Matrix Algebra

**8 The Inverse
8.1 Inverting 2-by-2 Matrices**

The inverse of a n-by-n matrix G is an n-by-n matrix F such that FG = I

_{n}. If this matrix

exists, we will denote it by G

^{-1}. Let us first look at an example of a 2-by-2 matrix. We have

to find coefficients a, b, c and d such that the following equation holds:

We can do this by just stating the equations for the
single components:

Solving for the coefficients of F yields:

Note that now we can solve the system of linear equations that we had in the
last lecture

by matrix-algebra operations:

Notice that this method will work for any system of linear
equations where we leave the

left-hand side unchanged (i.e. G stays as defined before) and change the vector
h on the right-hand

side. This suggests that a system of linear equations always has a unique
solution given

that the matrix on the left-hand side is invertible. In fact, this is a result
that holds for any

system of n linear equations with n unknowns, as we will see later on.

However, in general these systems do not always have a unique solution. In some
cases

they do not have any solution, in other cases they may have infinitely many
solutions. This

is related to the invertibility of the matrix on the left-hand side of the
system. Consider the

following example:

This result tells us that there are no coefficients a, b,
c and d such that DC = I. We say

that the matrix C is not invertible or singular. If you look at systems of
equations involving

the matrix C you will see that they either have no solution or infinitely many,
depending on the

vector on the right-hand side. Later, we will have a look at the general
relationship between

the invertibility of a matrix and the number of solutions of systems of linear
equations in that

matrix.

**8.2 The Determinant**

It turns out that the invertability of a matrix is related to its determinant.
In the case of a

2-by-2 matrix, the determinant is defined as follows:

Let us see what happens if we calculate the determinants
of the two matrices we tried to

invert before:

In fact, this result is no coincidence: If the determinant
of a matrix is 0, then it is singular,

i.e. not invertible. If the determinant is distinct from 0, then the matrix is
invertible (or

non-singular). In order to check this for higher-order matrices, however, we
need the general

definition of the determinant. In order to do this, it is useful to introduce
the following notation:

A_{ij} is defined as the matrix that is obtained when the ith row and the jth
column of the matrix

A are deleted. An example:

With this notation at hand, we can write down a relatively
short formula for the determinant:

Note, however, that this formula requires that we know the determinants of all
terms A_{ij} .

Note that so far, we only know them for 2-by-2 matrices (from our definition
above). Hence,

we can now calculate the determinant of a general 3-by-3 matrix:

Now that we know how to get the determinant of a 3-by-3
matrix, we could go on and

calculate the determinant of a 4-by-4 matrix, and so on. We say that this
definition of the

determinant is recursive.

You may have seen other ways of obtaining the determinant. We will not cover
them in

this course. You are free to use them throughout the course (as long as they are
correct, of

course. . .).

**8.3 Systems of n Linear Equations in n Unknowns**

Let us go back to the general representation of a system of n linear equations
in n unknowns

that we introduced in the last lecture:

It turns out that the determinant of G gives us a lot of
information about the number of

solutions of the system. It also depends on the nature of h: If h = 0 (i.e. all
its components

are zero) we are dealing with a homogeneous system of equations, in all other
cases (h ≠ 0,

i.e. at least one component of h is not equal zero) with an inhomogeneous system
of equations.

The number of solutions can be summarized in the following table:

**8.4 The Adjoint**

In order to write down the general formula for the inverse of a matrix, we need
the concept of

the adjoint. The adjoint of a matrix A is defined as the n-by-n matrix whose row
j, column i

element is given by (notice the change of the indeces j and i,
which amounts to

something like a transpose). For a 2-by-2matrix this works as follows (note that
the determinant

of a 1-by-1 matrix is given by its single entry):

**8.5 The Inverse of a General n-by-n Matrix**

Now, we can finally write down the formula for the inverse of a general n-by-n
matrix. If the

determinant of an n-by-n matrix A is not equal to zero, then its inverse exists
and is found by

dividing the adjoint by the determinant:

**8.6 Rules for Calculus with Inverses**

The following rules apply for the calculus with the inverse:

**9 Derivatives with Respect to Vectors**

To find the minimum (or the maximum) of some function that contains a vector, we
will often

be interested in the derivative of this function with respect to every single
component of the

vector. We define the derivative of a (scalar-valued) function with respect to a
column vector

as the collection of all these derivatives in a single vector:

A very simple function of a vector x is its product with
another vector a. We can get the

derivative of this function by multiplying out and then taking the derivatives
with respect to

the single components of x:

It gets a bit more complicated if we take the derivative
of a quadratic form x'Ax. Let us

derive the general formula by looking at the simplest example where x is 2 by 2:

**10 Eigenvalues and Eigenvectors**

Eigenvalues of an n-by-n matrix A are values of q that solve the following
problem:

Note that we have a free choice for both q and the vector
x, as long as x ≠ 0. Let us

perform some operations on this equation to get it into the form that we are
used to:

This is a homogeneous system of n equations in n unknowns. That is, this system
can only

have a solution with x ≠ 0 if the following holds (remember the rules for the
systems of linear

equations from above!):

This condition is called the characteristic equation. Let us have a look at the
characteristic

equation if A is 2 by 2:

This is a polynomial of order 2 in the unknown q. In
general, if A is n by n then the

characteristic equation is a polynomial of order n in q. It may (or may not)
have solutions in

the space of real numbers. [Aside: If we extend our analysis to the space of
complex numbers,

however, we can always find solutions to for the characteristic equation.] The
q’s that solve the

characteristic equation are called the eigenvalues of A.

When we take one solution back to the original problem, we can solve for the
corresponding

eigenvector x. Notice that there are many possible eigenvectors related with one
eigenvalue

– since we have chosen such that the
matrix A - qI has determinant 0, this
system has

infinitely many solutions for x now. The possible eigenvectors are related to
each other: they

are multiples of one another.

Let us go through all this for the following example to get a feeling for this:

The following are important properties of eigenvalues and eigenvectors: