How to use torch.no_grad to perform operations?

 In the context of PyTorch, you can use torch.no_ grad () to temporarily disable gradient computation which doesn’t need gradient tracking. Imagine a situation in which you are performing a model inference after training. By using the torch.no_grad() you can easily prevent or get rid of unnecessary calculations of the gradient. It will further save memory and the resources like computational. For example, if you are evaluating the model on a dataset validation post-training, using the torch.no.grad() will ensure that gradient should not computed for these validation purposes which can lead to enhancement of the efficiency. Here is the instance given:-

Import torch

# Define some tensors for demonstration
Weights = torch.tensor([3.0], requires_grad=True)
Bias = torch.tensor([1.0], requires_grad=True)
Inputs = torch.tensor([[1.0], [2.0], [3.0]])

# An example of performing operations without tracking gradients using torch.no_grad()

With torch.no_grad():
    # Forward pass without tracking gradients
    Predictions = inputs * weights + bias
# Here, any operations within the torch.no_grad() block won’t track gradients
# If you try to access gradients of tensors outside the torch.no_grad() block, it will raise an error:
# print(weights.grad)  # This will raise an error because gradients weren’t tracked
# Outside the torch.no_grad() block, gradients are still tracked as usual:
# For instance, let’s perform a backward pass to compute gradients for weights and bias
Loss = torch.tensor([[1.0], [0.5], [2.0]])
Predictions.requires_grad_()
Loss.backward(predictions)
# Access gradients outside the torch.no_grad() block
Print(“Gradients of weights:”, weights.grad)
Print(“Gradients of bias:”, bias.grad)

Therefore by applying a smart torch.no_ grad(), you can create a balance between computational efficiency and tracking of the gradients. It will streamline the process without affecting negatively the parameters of the training module.

For more information and knowledge about this particular topic and also others you can join our online master data science  program.