Part 3: TRAINING & PREDICTING VALUES FROM THE NEURAL NETWORK FRAMEWORK WE CREATED

Now, we will finally test our very simple framework, I have made 2 example files, that can be imported and used in our main.py(which will be the main execution file for our package).

I want you to understand and learn the code on your own. Hence, I am sharing this open source repository, so that all of you can clone it and tinker with it.

NOTE : IF YOU FIND ANY PROBLEMS WITH THE CODE OR WANT TO CONTRIBUTE, PLEASE CREATE AN ISSUE AND FORK THE REPOSITORY.

angadsinghsandhu/tensortorch

If you have not gone through the part 1 & part 2 of this series available here :

PART 1 : UNDERSTANDING NEURAL NETWORKS USING AN EXAMPLE

PART 2 : WRITING A DYNAMIC NEURAL NETWORK FRAMEWORK FROM SCRATCH IN PYTHON

File Structure

Photo by Markus Spiske on Unsplash

The following files are in the framework I created for now and these includes 2 types of training you can do right out of the gate, i.e. Regression and Classification.

- Data_Creation
| - odd_even_data.py
| - regression_data.py
- Examples
| - odd_even_eg.py
| - regression_eg.py
- Framework
| - ClassificationFramework.py
| - RegressionFramework.py
| - normalize.py
| - predict.py
__init__.py
README.md
main.py

TheData_Creation file is used to import or initialize our training data

TheExamples folder houses our created models files that we actually train to get our output

TheFramework folder has the framework classes that we wrote that do most of our training. This folder also has helper functions such as normalize.py and predict.py that we use to alter our input data or test our model.

Themain.py file is actually executed to run our code, this was done to maintain a clear relative import system.

I have also written multiple display methods inside our frameworks, to display information such as the feedforward matrices, backprop matrices and our loss after each iteration over our data.

We will see how we can implement our framework and tweak it to our personal taste. Will will see these examples now and also what can we do to improve them.

Classification Model

Here, we will take a very rudimentary approach where we use data going from 0 to 100 and and our outputs 0 or 1, where the output is 1 for an even input and 0 for an odd one. You can find the code where we create this data in theodd_even_data.py file.

This is how our Model looks like :

odd_even_eg.py

Finally, This is how our Model executes like :

Classification Model

First we get our imports that are : the framework, our predict function, our normalize function, our data creation method

Secondly, we define a run method and get our data by calling the data() function inside our data creation file. For our data, we can specify the number of examples we want (default is 100, if you leave it empty).

Thirdly, we normalize out input data to lie in-between the values of 0 and 1. We also get the values of our offset and factor which can be important for getting correct results during the prediction part of our model.

Fourth, we create our Neural Network object. We only need to input the input and output data respectively. As default, our learning rate is set to 0.06 and the number of our hidden layers is 2 , but we can also specify this if we wanted to.

Fifth, we display information about the network we created by calling the displayNN() method on out neural network object. This displays the initialized arrays we will use before training. Example :

===========Neural Network Properties===========
The number of layers in this NN are : 4
The learning rate of this NN are : 0.06
The number of nodes in each NN are : [1, 4, 5, 7, 1]
The weights of each layer of the NN is :
w1 :
[...]
w2 :
[...]
w3 :
[...]
w4 :
[...]
The shape weight matrices of each layer are :
W1 shape : [(4, 1)]
W2 shape : [(5, 4)]
W3 shape : [(7, 5)]
W4 shape : [(1, 7)]
===========Neural Network Properties===========

Sixth, we finally execute a for loop in which we start training our network. We call the train() method to go over our data. In the end we use the resetloss() method to re-initialize our loss to 0.

In-between our train and reset loss functions, we can call multiple display functions to find out how our arrays are being updated.

displayNN_forward() - Display the matrices we use during frorwardprop [W, b]
displayNN_backward() - Display the matrices we use during backwardprop [dZ, dW, db]
displayNN_loss() - Display the loss during each iteration

Seventh, Finally we call call our predict method to test our trained data

# predicting our data
predict(network, offset, factor)

Regression Model

Here, we will take an even more rudimentary approach where we use data going from 0 to 100 and and our outputs 1 more than 2 times our input, (eg. for an input of 4 our output will be 9). You can find the code where we create this data in theregression_data.py file.

This is how our Model looks like :

regression_eg.py

Finally, This is how our Model executes like :

Regression Model

First we get our imports that are : the framework, our predict function, our normalize function, our data creation method

Secondly, we define a run method and get our data by calling the data() function inside our data creation file. For our data, we can specify the number of examples we want (default is 100, if you leave it empty).

NOTE : FOR A REGRESSION BASED MODEL, WE WILL NOT NORMALIZE OUR INPUT DATA.

Thirdly, we create our Neural Network object. We only need to input the input and output data respectively. As default, our learning rate is set to 0.06 and the number of our hidden layers is 2 , but we can also specify this if we wanted to.

Fourth, we display information about the network we created by calling the displayNN() method on out neural network object. This displays the initialized arrays we will use before training. Example :

===========Neural Network Properties===========
The number of layers in this NN are : 4
The learning rate of this NN are : 0.06
The number of nodes in each NN are : [1, 4, 5, 7, 1]
The weights of each layer of the NN is :
w1 :
[...]
w2 :
[...]
w3 :
[...]
w4 :
[...]
The shape weight matrices of each layer are :
W1 shape : [(4, 1)]
W2 shape : [(5, 4)]
W3 shape : [(7, 5)]
W4 shape : [(1, 7)]
===========Neural Network Properties===========

Fifth, we finally execute a for loop in which we start training our network. We call the train() method to go over our data. In the end we use the resetloss() method to re-initialize our loss to 0.

In-between our train and reset loss functions, we can call multiple display functions to find out how our arrays are being updated.

displayNN_forward() - Display the matrices we use during frorwardprop [W, b]
displayNN_backward() - Display the matrices we use during backwardprop [dZ, dW, db]
displayNN_loss() - Display the loss during each iteration

Sixth, Finally we call call our predict method to test our trained data

# predicting our data
predict(network)

What’s Next

We can upgrade the framework by creating new files in Framework folder inside the repository. These upgrades might be

• Maybe new loss functions
• Adding options for Batch, mini-Batch and Stochastic Gradient Descent
• New architectures to handle CNNs and RNNs.
• Adding support for Unsupervised Learning Models.

Follow Me

Angad Sandhu — Medium