Deep Learning

Deep Learning  By Bharat Sreeram:
whatsapp: 6309613028

Deep Learning Session 1:


https://register.gotowebinar.com/recording/169761692275062022 

Deep Learning Session 2:
https://register.gotowebinar.com/recording/7258036079617821453

Deep Learning Session 3:
https://register.gotowebinar.com/recording/2545748045784130311


Deep Learning Session 4:
Topic : Implementation of building and train a Simple neural network.
https://register.gotowebinar.com/recording/8006219657435466754
Deep Learning Session 5:
Topic : converting Simple neural network into Deep neural network and train the network
https://register.gotowebinar.com/recording/7666410091490655500
Deep Learning Session 6:
Topic : develop predict function for DNN,
         predicting for new data.
         testing accuracy
https://register.gotowebinar.com/recording/1785324704657948936
Deep Learning Session 7:
Topic : predicting diabetic status based on age weight height using deep neural networks.
https://register.gotowebinar.com/recording/6662266803795545869

Deep Learning Session 8 :
Topic :  predicting diabetic status part 2
https://register.gotowebinar.com/recording/2111160577525697036

Deep Learning Session 9:

Topic : Polynomial Classifications using Deep Neural networks

Deep Learning Session 9 Recorded Session

How many output neurons for output layer , if classifiers are more than 2. 

example:

     Hand Written digit Recognization problem we   have 9 classifiers, 0 to 9. 

--------------------------------------

f = open('D:/mystuff/patients2.txt')

hd = f.readline()

lines = f.readlines()

a =[]

wt=[]

h=[]

d=[]

for line in lines:

    w = line.strip().split(',')

    a.append(float(w[1]))

    wt.append(float(w[2]))

    h.append(float(w[-2]))

    db=0

    if w[-1]=='moderate':

        db=1

    elif w[-1]=='high':

        db=2

    d.append(db)

print(a)

print(wt)

print(h)

print(d)

    

Output:

[25.0, 26.0, 45.0, 56.0, 55.0, 56.0, 55.0, 56.0, 49.0, 49.0, 52.0]
[60.0, 67.0, 78.0, 80.0, 81.0, 79.0, 79.0, 89.0, 70.0, 75.0, 80.0]
[5.9, 6.0, 5.5, 5.9, 6.0, 5.11, 5.4, 5.9, 5.6, 5.6, 5.7]
[2, 1, 2, 0, 0, 1, 2, 2, 1, 2, 2]

"""
converting numrical labels into binary array. 
0 -> 1 0 0
1 -> 0 1 0
2 -> 0 0 1
"""
import numpy as np
def binaryArray(l):
    nlabels = len(set(l))
    
    binary = []
    for v in l:
        arr = np.zeros(nlabels)
        arr[v]=1
        binary.append(arr)
    return np.array(binary)

binaryArray(d)

    

output:

array([[0., 0., 1.],
       [0., 1., 0.],
       [0., 0., 1.],
       [1., 0., 0.],
       [1., 0., 0.],
       [0., 1., 0.],
       [0., 0., 1.],
       [0., 0., 1.],
       [0., 1., 0.],
       [0., 0., 1.],
       [0., 0., 1.]])

"""
Developing a function , to convert network predictions 
 into numerical labels. 
 network predictions are probability values (0 to 1)
"""
yc = np.array([[0.002,0.001,0.990],
    [0.4,0.9,0.3],
     [0.99,0.001,0.004]])
def predictionToLabels(ycap):
    ycap[ycap>0.5]=1
    ycap[ycap<0.5]=0
    labels=[]
    for v in ycap:
        l = np.where(v==1)[0][0]
        labels.append(l)
    return np.c_[labels]
print(yc)
print(predictionToLabels(yc))   

output:

[[0.002 0.001 0.99 ]
 [0.4   0.9   0.3  ]
 [0.99  0.001 0.004]]
[[2]
 [1]
 [0]]

Deep Learning Session 10:

Topic: Polynomial Classifictions Using Deep Neural Networks : Part 2

How to train network and devoloping related functions. 

Deep Learning Session 10 Recorded Session

"""

preparing input matrix. 

input features. 

  1. age

  2. weight

  3. height

  we have 11 rows. 

   input matrix should be 11X3 matrix. 

   

   if you want to include bias, it should be 11X4 matrix. 

   first column should be ones. 

"""

X = np.c_[a,wt,h]

print(X)

print(X.shape)

output:

[[25.   60.    5.9 ]
 [26.   67.    6.  ]
 [45.   78.    5.5 ]
 [56.   80.    5.9 ]
 [55.   81.    6.  ]
 [56.   79.    5.11]
 [55.   79.    5.4 ]
 [56.   89.    5.9 ]
 [49.   70.    5.6 ]
 [49.   75.    5.6 ]
 [52.   80.    5.7 ]]
(11, 3)

"""
preparing label matrix. 
it should be binary array. 
it should be 11X3 matrix, bcoz, 3 classifiers. 
"""
Y = binaryArray(d)
print(Y)
print(Y.shape)

output:

[[0. 0. 1.]
 [0. 1. 0.]
 [0. 0. 1.]
 [1. 0. 0.]
 [1. 0. 0.]
 [0. 1. 0.]
 [0. 0. 1.]
 [0. 0. 1.]
 [0. 1. 0.]
 [0. 0. 1.]
 [0. 0. 1.]]
(11, 3)

"""
building deep neural network. 
one input layer. --> 3 neurons. 
three hidden layers ---> 
   first hidden ---> 5 neurons. 
   2nd hidden ---> 7 neurons. 
   3rd hidden ---> 5 neurons. 
output layer ---> 3 neurons. 
Total number of layers : 5

number weight matrices required. 4. 

"""
np.random.seed(101)
W1 = 2*np.random.random((3,5))-1
W2 = 2*np.random.random((5,7))-1
W3 = 2*np.random.random((7,5))-1
W4 = 2*np.random.random((5,3))-1
print("Weight matrix between input and first hidden layers")
print(W1)
print("_"*40)
print("Weight matrix between first hidden  and 2nd Hidden layers")
print(W2)
print("_"*40)
print("Weight matrix between 2nd  hidden and 3rd hidden layers")
print(W3)
print("_"*40)
print("Weight matrix between 3rd  hidden and output  layers")
print(W4)
print("_"*40)

output:

Weight matrix between input and first hidden layers
[[ 0.03279726  0.14133517 -0.94305155 -0.65695669  0.37055396]
 [ 0.66779373 -0.38606756  0.78722616  0.44308772 -0.62012209]
 [ 0.10845518 -0.29573609 -0.63621519  0.57120352  0.93096644]]
________________________________________
Weight matrix between first hidden  and 2nd Hidden layers
[[-0.53529268 -0.83287713  0.20709684  0.45798551 -0.44752234  0.37061266
   0.03573495]
 [-0.90303093 -0.72426152 -0.62606515  0.9886358   0.04133079  0.15757907
   0.46963812]
 [ 0.08392354  0.82630712  0.6158403  -0.19400434 -0.28555131  0.90575343
  -0.31273684]
 [ 0.73019963  0.66055542  0.0763229   0.84493875 -0.80570704 -0.79430501
   0.40301459]
 [ 0.78095974 -0.6808794  -0.44885491  0.34498306 -0.67139375  0.40274227
  -0.02472956]]
________________________________________
Weight matrix between 2nd  hidden and 3rd hidden layers
[[ 3.61355536e-01  4.30963847e-02 -9.13206611e-01 -5.52126793e-01
   1.50410174e-01]
 [-7.59132680e-01  2.33427602e-04 -7.23980863e-01 -8.94383198e-01
  -6.43446155e-01]
 [-1.15263737e-01  7.55174649e-01  8.98528258e-01 -4.36651664e-02
  -7.77613154e-02]
 [ 2.74578062e-01 -3.50784007e-01 -7.64843814e-01 -8.97798007e-01
   2.75317306e-01]
 [ 6.24531790e-01  3.40520841e-01  3.03535407e-01 -1.50862113e-01
   3.13190672e-01]
 [-5.81677002e-01  3.19849038e-01  5.92467975e-02  4.97040740e-01
  -8.12486288e-01]
 [ 5.69043700e-01  3.74484075e-01  3.90156993e-01 -6.26696081e-03
   9.50722254e-01]]
________________________________________
Weight matrix between 3rd  hidden and output  layers
[[-0.59294476 -0.40195918 -0.54468821]
 [-0.90366223  0.80794271 -0.83979292]
 [ 0.21443254  0.26169341 -0.24411614]
 [-0.97351798  0.68443882 -0.23012451]
 [ 0.10333169  0.42107628  0.3505577 ]]
________________________________________

W = np.array([W1,W2,W3,W4])

"""

Train the network

using Sigmoid activation function. 

how many activation functions?

  5 layers --> 5-1 --> 4 activation functions. 

  7 functions need to develop.

  

  1. sigmoid

  2. derivative for cross entropy

  3. mse (mean square error --> just to monitor whether error is decreassing or not for every iteration. )

  4. to scale input features. 

  5. predict

  6. accuracy

  7. training

"""

def sigmoid(x):

    """

       x --> dot product of input and weight matrix

    """

    return 1/(1+np.exp(-x))

def derivative(x):

 """

   x-->  output of a activation function. 

 """

 return x*(1-x)

def mse(y,ycap):

    return ((y-ycap)**2).mean()

def stddev(x):

    return (((x-x.mean())**2).sum()/(x.size-1))**0.5

def scale(x):

    return (x-x.mean())/stddev(x)

def scaleMatrix(x):

    ncol = x.shape[1]

    for v in range(ncol):

        col = x[:,v]

        x[:,v]= scale(col)

    return x

def predict(x,w):

    """

     here x is input matrix

         w is list of all weight matrices

    """

    res = x

    for v in w:

        res = sigmoid(res.dot(v))

    return res

def accuracy(y,ycap):

    bools = y==ycap

    pcnt =  bools[bools==True].size

    n = y.size

    acc = pcnt/n * 100

    return acc

        

def train(x,y,w,iter,conv=0.0000001):

  perr = 0  

  for i in range(iter):

    l1 = predict(x,[w[0]])

    l2 = predict(l1,[w[1]])

    l3 = predict(l2,[w[2]])

    l4 = predict(l3,[w[3]])

    cerr = mse(y,l4)

    diff = abs(perr-cerr)

    j = 0

    if diff<=conv:

        print("Training completed after ", i+1, " iterations")

        j = 1

        break

    if i%250==0 :

        print("Current error ", cerr)

    e4 = y-l4

    delta4 = e4 * derivative(l4)

    e3 = delta4.dot(w[3].T)

    delta3 = e3 * derivative(l3)

    e2 = delta3.dot(w[2].T)

    delta2 = e2* derivative(l2)

    e1 = delta2.dot(w[1].T)

    delta1 = e1 * derivative(l1)

    w[0] += x.T.dot(delta1)

    w[1] += l1.T.dot(delta2)

    w[2] += l2.T.dot(delta3)

    w[3] += l3.T.dot(delta4)

    perr = cerr

  if j==0:

    print("Training not completed ")

  return w

input = scaleMatrix(X)

theta = train(input, Y, W,10000)

ycap = predict(input, theta)

print(ycap)

output:

[[1.15922558e-02 1.28518549e-02 9.82054291e-01]
 [2.04100968e-05 9.89775798e-01 1.41329614e-02]
 [2.35118473e-03 8.12241952e-03 9.91954327e-01]
 [9.87993170e-01 2.52737021e-05 1.76653962e-02]
 [9.87498670e-01 2.59311107e-05 1.84960708e-02]
 [1.83102055e-05 9.89716543e-01 1.25169542e-02]
 [3.23231165e-03 5.80688071e-03 9.92234298e-01]
 [7.75060247e-03 3.04138128e-03 9.88399186e-01]
 [2.87704806e-05 9.83882607e-01 1.86484050e-02]
 [1.87651939e-03 1.71341919e-02 9.84045999e-01]
 [8.07656340e-03 2.96907550e-03 9.86808767e-01]]

predictions = predictionToLabels(ycap)
print(predictions)

output:

[[2]
 [1]
 [2]
 [0]
 [0]
 [1]
 [2]
 [2]
 [1]
 [2]
 [2]]

accuracy(np.c_[d], predictions )

output:

100.0