Basic introduction to Pytorch

What is Pytorch?

Pytorch is an open-source machine learning library developed by Facebook. It allows flexibility and speed for scientific computing for deep learning. Consider it as a replacement of Numpy so that the GPU capability can be used.

Tensors?

Tensor is the core element of Pytorch and are basically a n-dimensional data container much like Numpy's n-dimensional array. A tensor could be a number, vector, matrix or a n-dimensional array.

import torch

# Scalar - Integer
t1 = torch.tensor(5)
print("Tensor is {} with shape {} and data type {}".format(t1, t1.shape, t1.dtype))

Tensor is 5 with shape torch.Size([]) and data type torch.int64

# Scalar - Float
t2 = torch.tensor(5.)
print("Tensor is {} with shape {} and data type {}".format(t2, t2.shape, t2.dtype))

Tensor is 5.0 with shape torch.Size([]) and data type torch.float32

# Vector
t3 = torch.tensor([1., 2, 3])
print("Tensor is {} with shape {} and data type {}".format(t3, t3.shape, t3.dtype))

t4 = torch.tensor([1, 2, 3])
print("Tensor is {} with shape {} and data type {}".format(t4, t4.shape, t4.dtype))

Tensor is tensor([1., 2., 3.]) with shape torch.Size([3]) and data type torch.float32
Tensor is tensor([1, 2, 3]) with shape torch.Size([3]) and data type torch.int64

# Matrix
t5 = torch.tensor([[1., 2, 3],[4, 5, 6]])
print("Tensor is {} with shape {} and data type {}".format(t5, t5.shape, t5.dtype))

t6 = torch.tensor([[1, 2, 3],[4, 5, 6]])
print("Tensor is {} with shape {} and data type {}".format(t6, t6.shape, t6.dtype))

Tensor is tensor([[1., 2., 3.],
        [4., 5., 6.]]) with shape torch.Size([2, 3]) and data type torch.float32
Tensor is tensor([[1, 2, 3],
        [4, 5, 6]]) with shape torch.Size([2, 3]) and data type torch.int64

Creating tensors using torch.Tensor & torch.tensor :

torch.Tensor is the main Tensor class and everything is an instance of it. You can use it to create an empty tensor which is not possible with torch.tensor.

# Creating tensors using torch.Tensor class 

# Create a tensor with 3 elements and random data
t7 = torch.Tensor(3)
print("Tensor is {} with shape {} and data type {}".format(t7, t7.shape, t7.dtype))

# Matrix - 2X3 with random data
t8 = torch.Tensor(2, 3)
print("Tensor is {} with shape {} and data type {}".format(t8, t8.shape, t8.dtype))

# 2X3 Matrix with data
t9 = torch.Tensor([[1, 2], [3, 4]])
print("Tensor is {} with shape {} and data type {}".format(t9, t9.shape, t9.dtype))

Tensor is tensor([9.8755e-35, 1.4013e-45, 1.3633e-41]) with shape torch.Size([3]) and data type torch.float32
Tensor is tensor([[ 0.0000e+00, -2.0000e+00,  0.0000e+00],
        [-2.0000e+00,  8.4078e-45, -2.0000e+00]]) with shape torch.Size([2, 3]) and data type torch.float32
Tensor is tensor([[1., 2.],
        [3., 4.]]) with shape torch.Size([2, 2]) and data type torch.float32

# Arithemetic operations

a = torch.tensor(1)
b = torch.tensor(2)
c = torch.tensor(3)
y = a + b + c
print(y)

# Derivatives 
a = torch.tensor(1.)
b = torch.tensor(2., requires_grad=True)
c = torch.tensor(3., requires_grad=True)
y = a + b + c
print(y)

tensor(6)
tensor(6., grad_fn=<AddBackward0>)

Pytorch allows you to not only run the tensors in CPU but also in GPU for speed. Also, it got some unique features to track operations applied on them by forming backward graph for the tensors having requires_grad property set to True.

So basically, you can calculate the derivative of the "y" with respect to the other tensors having requires_grad by calling .backward method. Derivative is stored in a variable .grad associated with each tensor.

y.backward()

print('dy/da:', a.grad)
print('dy/db:', b.grad)
print('dy/dc:', c.grad)

dy/da: None
dy/db: tensor(1.)
dy/dc: tensor(1.)

Numpy and Pytorch

We can quickly create tensors from Numpy arrays.

import numpy as np

# Convert Numpy array to tensor

a1 = np.array([[1, 2, 4.], [5, 6, 7]])

t = torch.from_numpy(a1)

print("Tensor is {} with shape {} and data type {}".format(t, t.shape, t.dtype))

Tensor is tensor([[1., 2., 4.],
        [5., 6., 7.]], dtype=torch.float64) with shape torch.Size([2, 3]) and data type torch.float64

# Convert tensor to Numpy array

a2 = t.numpy()
print(a2)

[[1. 2. 4.]
 [5. 6. 7.]]