5 Fundamental Functions in PyTorch Every Deep Learning beginner should know
Torch your Deep Learning world alight with these Tensor related functions
5 Fundamental Functions in Pytorch, Every Deep Learning beginner should know
Being able to build neural networks in a accurate and efficient way is one of the most sought after skills recruiters look for in a deep learning engineer. For those who don’t know, PyTorch is a Python library with a wide variety of functions and operations, mostly used for deep learning. One of the most basic yet important part of PyTorch is the ability to create Tensors. A tensor is a number, vector, matrix, or any n-dimensional array. As we know one must avoid using explicit loops while building a neural network to ease the computation speed. Instead we use vectorized operations to avoid such loops. Hence the ability to compute matrix operations fast enough is crucial while building neural networks.
Now the question might be, ‘why not use NumPy library instead?’
For Deep Learning, we would need to compute the derivative of elements of the parameters of our model. PyTorch provides the ability to keep track of deravitives while backpropagating, which NumPy does not. This is called ‘Auto Grad.’ PyTorch also provides built in support for fast execution using GPU. This is essential when it comes to training models. As Numpy lacks the ability to shift its computation to GPU, the time to train the model eventually becomes very large.
All Deep Learning projects using PyTorch start with creating a tensor. Let’s see a few MUST KNOW functions which are the backbone of any Deep Learning project that involves building a neural network.
torch.tensor()torch.sum()torch.index_select()torch.stack()torch.mm()
Before we begin, let’s install and import PyTorch
Function 1 — torch.tensor()
First, we define a helper function, describe(x), that will summarize various properties of a tensor x, such as the type of the tensor, the dimensions of the tensor, and the contents of the tensor.
Creating a tensor in PyTorch with torch.Tensor
PyTorch allows us to create tensors in many different ways using the torch package. One way to create a tensor is to initialize a random one by specifying its dimensions
Creating a tensor declaratively by using Python lists
We can also create tensors using python lists. We just need to pass the list as parameter to the function and we have the tensor form of it.
Creating Tensors using NumPy arrays
And, of course, we can always go from a PyTorch tensor to a NumPy array, as well. Notice that the type of the tensor is DoubleTensor instead of the default FloatTensor. This corresponds with the data type of the NumPy random matrix, a float64,as presented below.
What we can’t do with a tensor
We cannot break the regularity of a tensor. A tensor must be a real or a complex number, it should not be a string or character.
Overview:
torch.tensor() forms the core of any PyTorch project, quite literally, as it forms tensors.
Function 2 — torch.sum()
This functions returns the sum of all elements in the input tensor.
If you are coming from NumPy background you might have already noticed, for a 2D tensor we represent rows as the dimension 0 and columns as dimension 1. torch.sum() function allows us to calculate the sum of the rows and columns.
Note: That we are also passing True for keepdims to retain the dimension in the result. By defining dim = 1 we tell the function to collapse the array by columns.
Overview
This function is very useful in computing the cost functions.
Function 3 — torch.index_select()
This function returns a new tensor which indexes the input tensor along dimension dim using the entries in index which is a LongTensor.
We can pass the indices as a tensor and define the axis as 1, and the function returns a new tensor of size
rows_of_original_tensor x length_of_indices_tensor.
We can pass the indices as a tensor and define the axis as 0, and the function returns a new tensor of size columns_of_original_tensor x length_of_indices_tensor.
Overview
This function is useful in complex indexing: noncontiguous indexing of a tensor.
Function 4 — torch.stack()
This Concatenates a sequence of tensors along a new dimension.
We can pass the tensors we want to concatenate as a list of tensors with the dim as 0 to stack it along the rows.
We can pass the tensors we want to concatenate as a list of tensors with the dim as 1 to stack it along the columns.
Overview
This function is very useful in combination with torch.index_select() to flatten the image matrix.
Function 5 — torch.mm()
This function performs a matrix multiplication of the matrices.
We can easily perdorm matrix multiplication by just passing the matrices as parameters and the function will result a new tensor as a product of the two matrices.
In the above example we define a NumPy array and then convert in to tensor of float32 type. And Now we can successfully perform matrix multiplication on the the tensors. It is necessary that the data types of both the tensors match for a successful operation.
In order to perform successful matrix multiplication operation, it is necessary that the columns no. of matrix 1 and row no. of matrix 2 should match. It is the basic rule of matrix multiplication that torch.mm() function follows. Even if the matrices are of same order it will still not automatically multiply with the transpose of the other matrix, user have to manually define it.
Overview
In order to calculate derivative in while backward propagating it is necessary to be able to perform matrix multiplication efficiently, this is where torch.mm() comes into picture.
Conclusion
This concludes our look at 5 fundamental PyTorch functions. From basic tensor creation to advanced and lesser known functions with specific use cases like torch.index_select(), PyTorch provides many such functions which make job of a data science enthusiast easier. This was a beginner friendly introduction to PyTorch. There is much more. From here it would be a good idea to explore the documentation and create your own tensors. Play around and have some fun. As we get a tighter grip on the basics, we can move forward to Neural Networks and Deep Learning.
Reference Links
Links to my references and other interesting articles about tensors
- Official documentation for tensor operations: https://pytorch.org/docs/stable/torch.html
- PyTorch Tensor Tutorial: https://pytorch.org/tutorials/beginner/examples_tensor/two_layer_net_tensor.html
