Linear Algebra for Data Science I

Introduction

We [Irving Kaplansky and Halmos] share a philosophy about linear algebra: we think basis-free, we write basis-free, but when the chips are down we close the office door and compute with matrices like fury ~ Kaplansky

Linear Algebra is a branch of mathematics that focus on linear transformations on vector spaces and matrices

Why do we need to study linear algebra ?

The core concepts of basic algorithms such as linear regression, support vector machine to more advanced ones deep learning all rely heavily on linear algebra and vector calculus.

Before we make a deep dive into linear algebra ,let us try to understand the building blocks of vector spaces and matrices.

Building Blocks

Binary Operations

A binary operation on a set S is a map from S*S to S
Suppose (x,y) is an ordered pair then image is denoted by x o y .
e.g: Division is not a binary operation why ?

A binary operation o on S is associative if (x o y)o z =x o(y o z) for all x,y,z in S

Group

A group is a set S with an associative binary operation on S such that there is an identity element and every element has inverse.

Ring

A ring is a set R with two binary operations + and . on R with following conditions

* (R,+) is a group
* . is associative
* x.(y+z)=(x.y)+(x.z) for all x,y,z in R,
(y+z).x=(y.x)+(z.x) for all x,y,z in R

Commutative Ring : If for all x,y xy=yx

Field

A field is a commutative ring such that non zero elements form a group under multiplication

Vector Spaces

A vector space over a field F is a quadruple
(V, +,’, F)t satisfying the following axioms for all a, ß belong to F and x, y, z belong to V

I (V, +) is a commutative group, that is

(a) + is a map from V x V to V (we write the image of (x,y) as
x + y for convenience) (Closure with respect to +)

(b) (x + y) + z = x + (y + z) (Associativity of +)

© there exists an element 0 of V such that x + 0 = 0 + x = x
for all x belong to V (Existence of 0)

(d) for each x in V there exists an element -x in V such that
x + (-x) = (-x) + x = 0 (Existence of negative)

(e) x + y = Y + x (Commutativity of +)

II (.) is a map from F x V to V (we write the image of (a, x) as a· x
for convenience) (Closure with respect to .)

III a· (ß . x) = (aß) . x

IV 1· x = x

V (a + ß) . x = (a· x) + (ß· x) (Distributivity)

VI a· (x + y) = (a· x) + (a· y) (Distributivity)

Here + denotes vector addition except in the LHS of (V) where it
denotes addition in F. The elements of V are called vectors and the
elements of F are called scalars. F itself is called the base field of the vector space.

Example

Some other concepts which are useful like subspaces , independences , basis and dimensions . I have uploaded my notebook notes as it contains lots of mathematical symbols