Linear Algebra with Numpy¶

Try me¶

## Introduction This tutorial provides an overview of Linear Algebra operations that can be performed with Numpy. Until now, we have learned to perform the same operations to lists of numbers using repetition loop. However, linear algebra operations with arrays allow us to build the same programming logic more effectively, just by using one operation. This paradigm is known as array programming. This notebook provides an introduction to linear algebra with Numpy and compares code snippets performing the same operations with linear algebra operations and repetition loops.

\[c = [c_1, c_2, c_3, ..., c_n]\]

\[v = [v_1, v_2, v_3, ...., v_n]\]

\[d = c·v = c_1*v_1 + c_2*v_2 + c_3*v_3 + ... + c_n*v_n\]

\[d = c·v = \sum_{i_1}{n}{c_i*v_i}\]

To use Numpy, we need to import the library. We will use the alias np to refer to Numpy:

[ ]:

import numpy as np

Linear algebra operations¶

Dot product¶

Recall that, in linear algebra, the dot product or scalar product of two vectors is the sum of the product of the members in the same position of the array. That is, imagine we have two vectors with n elements, the dot vector can be defined like: #### Dot product in Python from scratch Implementing the dot product in Python from scratch is rather straight-forward, for instance, we can initialise the result variable to zero and use a for loop to iteratively sum the product of the members of the two vectors, like:

[2]:

an_array= [1, 2,3, 5]
another_array = [6, 7, 8, 9]
dot_product = 0
for j in range(len(an_array)):
  dot_product += an_array[j]*another_array[j]
print(dot_product)

Dot Product using Numpy¶

Numpy provides convenient methods to implement the dot product. You can use the standard Python matrix product operator ‘@’ to perform the scalar product of two vectors:

[4]:

a = np.array([1, 2, 3, 4, 5])
b = np.array([1, 2, 3, 4, 5])

c = a@b
print(c)

The dot function works just the same:

[5]:

c = np.dot(a,b)
print(c)

Matrix product¶

The Matrix product is an extension of the scalar product to 2D vectors or matrix. The result of the matrix product is another array where each member is the scalar product of the elements of the first matrix in the same position along the first axis (or row) and the elements of the second matrix in the same position along the second axis (or column). Using mathematical notation, we can define the matrix product as:

\[\begin{split}A = \begin{bmatrix} a_{11} & a_{12} & ... & a_{1p}\\ a_{21} & a_{22} & ... & a_{2p}\\ ... & ... & ... & ... \\ a_{m1} & a_{m2} & ... & a_{mp} \end{bmatrix}\end{split}\]

\[\begin{split}B = \begin{bmatrix} b_{11} & b_{12} & ... & b_{1n}\\ a_{21} & a_{22} & ... & b_{2n}\\ ... & ... & ... & ... \\ a_{p1} & b_{p2} & ... & b_{pn} \end{bmatrix}\end{split}\]

\[\begin{split}C = A·B = \begin{bmatrix} c_{11} & c_{12} & ... & c_{1n}\\ c_{21} & c_{22} & ... & c_{2n}\\ ... & ... & ... & ... \\ c_{m1} & c_{m2} & ... & c_{mn} \end{bmatrix}\end{split}\]

\[c_{ij} = \sum_{c=1}^{p}{a_{ic}*b_{jc}}\]

That, is, given a matrix \(A\) of shape \(mxp\) and a matrix \(B\) of shape \(pxn\) the result of the matrix product is another matrix of shape \(mxn\) where the element in row \(i\) and column \(j\) is the scalar product of the \(i^{th}\) row (row in position i) of \(A\) and the \(j^{th}\) column of \(B\) (\(i \in [1, 2, ..., m], j \in [1, 2, ..., n]\). Note that for the matrix product to work, the number of elements of the rows of A must be equal to the number of elements of the columns of B.

Matrix product from scratch¶

If we need to implement the matrix product from scratch we can use nested for loops to travel the axis of the matrices (one for loop to traverse the rows (first axis) of \(A\), and another one to traverse the columns (second axis) of \(B\)), and an additional for loop to implement the scalar product, just as in the previous section:

[5]:

a_matrix = [[1, 2, 3],[4, 5, 6]]
another_matrix = [[1, 2],[3, 4],[5, 6]]
m = len(a_matrix)
p= len(a_matrix[0])
n = len(another_matrix[0])

#initialise the matrix as a zero matrix using comprehension
matrix_product = [ [ 0 for i in range(m) ] for j in range(n) ]

for i in range(m):
  for j in range(n):
    for c in range(p):
      matrix_product[i][j]+=a_matrix[i][c]*another_matrix[c][j]

print(matrix_product)

[[22, 28], [49, 64]]

Matrix product with Numpy¶

In Numpy, the implementation of the matrix product is equivalent to the implementation of the dot product. The standard Python operator @ or the np.dot function can be used to perform a matrix product, but the @ operator is preferred:

[8]:

A = np.array([[1, 3], [3,4]])  # this is a 2x2 matrix
b = np.array([[1], [2]])       #this is a column vector (2x1 matrix)

C = np.dot(A, b)
print(C)

C = A @ b
print(C)

[[ 7]
 [11]]
[[ 7]
 [11]]

Transpose¶

The transpose() method returns the transpose of a matrix or vector. Recall that the transpose of a matrix of shape \(mxn\) is another matrix of shape \(nxm\). In mathematical notation, the transpose of matrix \(A\) is noted as \(A^T\), for example:

\[\begin{split}A = \begin{bmatrix} 1 & 2 & 3 & 4\\ 5 & 6 & 7 & 8 \end{bmatrix}\end{split}\]

\[\begin{split}A^T = \begin{bmatrix} 1 & 5\\ 2 & 6\\ 3 & 7\\ 4 & 8 \end{bmatrix}\end{split}\]

In Numpy, the method transponse() of a Numpy transpose object returns the transpose:

[10]:

A = np.array([[1, 3], [3,4], [5, 6]])  # this is a 3x2 matrix
print("A is: ")
print(A)
print("And it´s shape is:")
print(A.shape)
print("The transpose is:")
A_t = A.transpose()
print(A_t)
print("And its shape:")
print(A_t.shape)

A is:
[[1 3]
 [3 4]
 [5 6]]
And it´s shape is:
(3, 2)
The transpose is:
[[1 3 5]
 [3 4 6]]
And its shape:
(2, 3)

Reshape¶

Besides the transpose method, we can change the shape of any array to an arbitrary shape using the method reshape changes the shape of a numpy array without changing the values of its elements:

[13]:

a = np.arange(1, 28) # an array with 27 elements

A = a.reshape(3,3,3) #A has the same elements but shape 3x3x3

print(A)

[[[ 1  2  3]
  [ 4  5  6]
  [ 7  8  9]]

 [[10 11 12]
  [13 14 15]
  [16 17 18]]

 [[19 20 21]
  [22 23 24]
  [25 26 27]]]

Array functions¶

Numpy also provides an extensive list of array functions that work on the dimensions of the array:

sum(): Returns the sum of all elements.
min(): Returns the minimum value within the array
max(): Returns the maximum value within the array
mean(): Returns the mean of an array
median(): Returns the median value of the array
cumsum(): Returns the cumulative sum of the elements of the array.

All of the functions above support the additional axis parameter to work on a specific dimension, instead of working on all dimensions.

[13]:

x =np.array([[1,2,3,4],[5,6,7,8]])
y =np.array([[9,10,11,12],[13,14,15,16]])

print("sum of all elements in x:")
print(np.sum(x))

print("mean value of y:")
print(np.mean(y))

sum of all elements in x:
36
mean value of y:
12.5

Other functions take two arrays as arguments and perform element wise operations:

minimum(): Returns an array with the minimum value in each position of the array
maximum(): Returns an array with the maximum value in each position of the array

[14]:

b=np.linspace(0,1,10)
r = c=np.random.random(10)
print(np.minimum(b,r))

[0.         0.11111111 0.22222222 0.0069694  0.44444444 0.3403326
 0.19167794 0.71257103 0.78045669 0.64287305]