# Linear Algebra

Linear algebra is the branch of mathematics that studies vector spaces and linear transformations between them like adding and scaling. It uses matrixes to represent linear maps. Linear algebra is widely used in science and engineering.

In the Compute Engine matrices are represented as lists of lists.

For example the matrix above is represented as the following list of lists:

```
["List", ["List", 1, 3, ["List", 5, 0]]]
```

An **axis** is a dimension of a tensor. A vector has one axis, a matrix has two
axes, a tensor (multi-dimensional matrix) has more than two axes.

The **shape** of a tensor is the length of each of its axes. For example, a
matrix with 3 rows and 4 columns has a shape of `(3, 4)`

.

The **rank** of a tensor is the number of axes of the tensor. For example, a
matrix has a rank of 2. Note that rank is sometimes used with a different
meaning. For example, the rank of a matrix can also be defined as the maximum
number of linearly independent rows or columns of the matrix. In the Compute
Engine, **rank** is always used with the meaning of the number of axes.

In the Compute Engine, matrices are stored in **row-major** order. This means that
the first element of the outer axis is the first row of the matrix, the second element
of the list is the second row of the matrix, etc.

An extension of linear algebra is tensor algebra which studies tensors, which are multidimensional arrays.

For example, a grayscale image can be represented by a matrix of grayscale values. But a color image is represented by a rank 3 tensor, an array of RGB triplets. Tensors are also represented as nested lists.

The Compute Engine provides a number of functions for working with matrices.

### Representing Matrices

Vectors (row vectors) are represented as lists, that is an expression with the
head `List`

.

Matrixes are represented as lists of lists.

Tensors (multi-dimensional matrixes) are represented as nested lists.

Tensors are represented internally using an optimized format that is more
efficient than nested lists. Because of this, some operations on tensors
such as `Reshape`

and `Transpose`

can be done in O(1) time.

`Vector`

is a convenience function that interprets a list of elements as a
column vector.

`Matrix`

is an optional “tag” inert function that is used to influence the visual
representation of a matrix. It has not impact on the value of the matrix.

In LaTeX notation, a matrix is represented with “environments” (with command
`\begin`

and `\end`

) such as `pmatrix`

or `bmatrix`

.:

In LaTeX, each column is separated by an `&`

and each row is separated by
`\\`

.

### Vector

["**Vector**", *x-1*, …*x-2*]

`Vector`

interprets the elements *x-1*… as a column vector

This is essentially a shortcut for `["Matrix", ["List", ["List", _x-1_], ["List, _x-2_], ...]]]`

.

```
["Vector", 1, 3, 5, 0]
```

A row vector can be represented with a simple list or a tuple.

```
["List", 1, 3, 5, 0]
```

### Matrix

["**Matrix**", *matrix*]

["**Matrix**", *matrix*, *delimiters*, *columns-format*]

`Matrix`

is an inert function: its value is the value of its first argument.
It influences the visual representation of a matrix.

*matrix* is a matrix represented by a list of rows. Each row is represented
by a list of elements.

*delimiters* is an optional string of two characters.
The first character represent the opening delimiter and the second character
represents the closing delimiter.

The delimiters can be any of the following characters:

`(`

,`)`

,`[`

,`]`

,`{`

,`}`

,`<`

,`>`

`⟦`

(`U+27E6`

),`⟧`

(`U+27E7`

)`|`

,`‖`

(`U+2016`

)`\\`

`⌈`

(`U+2308`

),`⌉`

(`U+2309`

),`⌊`

(`U+230A`

),`⌋`

(`U+230B`

)`⌜`

(`U+231C`

),`⌝`

(`U+231D`

),`⌞`

(`U+231E`

),`⌟`

(`U+231F`

)`⎰`

(`U+23B0`

),`⎱`

(`U+23B1`

).

In addition, the character `.`

can be used to indicate no delimiter.

Some commom combinations may be represented using some standard LaTeX environments:

Delimiters | LaTeX Environment | Example |
---|---|---|

`()` |
`pmatrix` |
\[ \begin{pmatrix} a & b \\ c & d \end{pmatrix} \] |

`[]` |
`bmatrix` |
\[ \begin{bmatrix} a & b \\ c & d \end{bmatrix} \] |

`{}` |
`Bmatrix` |
\[ \begin{Bmatrix} a & b \\ c & d \end{Bmatrix} \] |

`||` |
`vmatrix` |
\[ \begin{vmatrix} a & b \\ c & d \end{vmatrix} \] |

`‖‖` |
`Vmatrix` |
\[ \begin{Vmatrix} a & b \\ c & d \end{Vmatrix} \] |

`{.` |
`dcases` |
\[ \begin{dcases} a & b \\ c & d \end{dcases} \] |

`.}` |
`rcases` |
\[ \begin{rcases} a & b \\ c & d \end{rcases} \] |

*columns_format* is an optional string indicating the format of each column.
A character `=`

indicates a centered column, `<`

indicates a left-aligned
column, and `>`

indicates a right-aligned column.

A character of `|`

indicate a solid line between two
columns and `:`

indicate a dashed lines between two columns.

## Matrix Properties

### Shape

["**Shape**", *matrix*]

Returns the shape of a matrix, a tuple of the lengths of the matrix along each of its axis.

A list (or vector) has a single axis. A matrix has two axes. A tensor has more than two axes.

For a scalar, `Shape`

returns an empty tuple.

```
["Shape", 5]
// ➔ ["Tuple"]
["Shape", ["List", 5, 2, 10, 18]]
// ➔ ["Tuple", 4]
["Shape", ["List", ["List", 5, 2, 10, 18], ["List", 1, 2, 3]]]
// ➔ ["Tuple", 2, 4]
```

**Note:** The shape of a matrix is also sometimes called “dimensions”.
However, this terminology is ambiguous because the word “dimension” is also
used to refer to the length of a matrix along a specific axis.

### Rank

["**Rank**", *matrix*]

Returns the number of axes of a matrix.

A scalar (a number, for example) has **rank 0**.

A vector or list has **rank 1**.

A matrix has **rank 2**, a tensor has **rank 3**, etc.

The rank is the length of the shape of the tensor.

```
["Rank", 5]
// ➔ 0
["Rank", ["List", 5, 2, 10, 18]]
// ➔ 1
["Rank", ["List", ["List", 5, 2, 10], ["List", 1, 2, 3]]]
// ➔ 2
```

## Accessing the content of Tensors

### At

["**At**", *matrix*, *index-1*, *index-2*, …]

Returns the element of the matrix at the specified indexes.

*index-1*, … is a sequence of integers, one for each axis of the matrix.

Indexes start at 1. Negative indexes count elements from the end. A negative
index is equivalent to adding the length of the axis to the index. So `-1`

is
the last element of the axis, `-2`

is the second to last element, etc.

```
["At", ["List", 5, 2, 10, 18], 3]
// ➔ 10
["At", ["List", ["List", 5, 2, 10, 18], ["List", 1, 2, 3]], 2, 3]
// ➔ 3
["At", ["List", ["List", 5, 2, 10, 18], ["List", 1, 2, 3]], 2, -1]
// ➔ 3
```

In a list (or vector), there is only one axis, so there is only one index.

In a matrix, the first index is the row, the second is the column.

In LaTeX, accessing the element of a matrix is done with a subscript or square brackets following a matrix.

## Transforming Matrixes

### Flatten

["**Flatten**", *matrix*]

Returns a list of all the elements of the matrix, recursively, in row-major order.

This function can also be applied to any collection.

Only elements with the same head as the collection are flattened.
Matrixes have a head of `List`

, so only other `List`

elements
are flattened.

```
["Flatten", ["List", ["List", 5, 2, 10, 18], ["List", 1, 2, 3]]]
// ➔ ["List", 5, 2, 10, 18, 1, 2, 3]
```

`Flatten`

is similar to the APL `,`

Ravel operator or `numpy.ravel`

Numpy.

### Reshape

["**Reshape**", *matrix*, *shape*]

Returns a matrix with the specified shape.

*matrix* can be a list, a matrix, a tensor or a collection.

*shape* is a tuple of integers, one for each axis of the result matrix.

`Reshape`

can be used to convert a list or collection into a matrix.

```
["Reshape", ["Range", 9], ["Tuple", 3, 3]]
// ➔ ["List", ["List", 1, 2, 3], ["List", 4, 5, 6], ["List", 7, 8, 9]]
```

This is similar to the APL `⍴`

Reshape operator or `numpy.reshape`

Numpy.

The result may have fewer or more elements than the original tensor.

When reshaping, the elements are taken from the original tensor in row-major
order, that is the order of elements as returned by `Flatten`

.

If the result has fewer elements, the elements are dropped from the end of the element list. If the result has more elements, the lists of elements is filled cyclically.

This is the same behavior as APL, but other environment may behave differently.
For example, by default Mathematica `ArrayReshape`

will fill the missing elements
with zeros.

### Transpose

["**Transpose**", *matrix*]

Returns the transpose of the matrix.

```
["Transpose", ["List", ["List", 1, 2, 3], ["List", 4, 5, 6]]]
// ➔ ["List", ["List", 1, 4], ["List", 2, 5], ["List", 3, 6]]
```

["**Transpose**", *tensor*, *axis-1*, *axis-2*]

Swap the two specified axes of the tensor. Note that axis indexes start at 1.

### ConjugateTranspose

["**ConjugateTranspose**", *matrix*]

Returns the conjugate transpose of the matrix, that is the transpose of the matrix with all its (complex) elements conjugated. Also known as the Hermitian transpose.

```
["ConjugateTranspose", ["List", ["List", 1, 2, 3], ["List", 4, 5, 6]]]
// ➔ ["List", ["List", 1, 4], ["List", 2, 5], ["List", 3, 6]]
```

["**ConjugateTranspose**", *matrix*, *axis-1*, *axis-2*]

Swap the two specified axes of the *matrix*. Note that axis
indexes start at 1. In addition, all the (complex) elements
of the tensor are conjugated.

### Inverse

["**Inverse**", *matrix*]

Returns the inverse of the matrix.

```
["Inverse", ["List", ["List", 1, 2], ["List", 3, 4]]]
// ➔ ["List", ["List", -2, 1], ["List", 1.5, -0.5]]
```

### PseudoInverse

["**PseudoInverse**", *matrix*]

Returns the Moore-Penrose pseudoinverse of the matrix.

```
["PseudoInverse", ["List", ["List", 1, 2], ["List", 3, 4]]]
// ➔ ["List", ["List", -2, 1], ["List", 1.5, -0.5]]
```

### Diagonal

["**Diagonal**", *matrix*]

Returns the diagonal of the matrix, that is the list of all the elements on the diagonal of the matrix.

```
["Diagonal", ["List", ["List", 1, 2], ["List", 3, 4]]]
// ➔ ["List", 1, 4]
```

## Calculating with Matrixes

### Determinant

["**Determinant**", *matrix*]

Returns the determinant of the matrix.

```
["Determinant", ["List", ["List", 1, 2], ["List", 3, 4]]]
// ➔ -2
```

### AdjugateMatrix

["**AdjugateMatrix**", *matrix*]

Returns the adjugate matrix of the input matrix, that is the inverse of the cofactor matrix.

The cofactor matrix is a matrix of the determinants of the minors of the matrix multiplied by \( (-1)^{i+j} \). That is, for each element of the matrix, the cofactor is the determinant of the matrix without the row and column of the element.

```
["AdjugateMatrix", ["List", ["List", 1, 2], ["List", 3, 4]]]
// ➔ ["List", ["List", 4, -2], ["List", -3, 1]]
```

### Trace

["**Trace**", *matrix*]

Returns the trace of the matrix, the sum of the elements on the diagonal of the matrix. The trace is only defined for square matrices. The trace is also the sum of the eigenvalues of the matrix.

```
["Trace", ["List", ["List", 1, 2], ["List", 3, 4]]]
// ➔ 5
```