Step-by-Step Guide to Configuring GPU in Azure N-Series Virtual Machines

Overview

The Azure GPU VM “NC4as_T4_v3” is equipped with a single NVIDIA T4 GPU, offering 16 GB of GPU memory optimized for AI inference, machine learning, and graphics-intensive workloads. It features 4 vCPUs and 28 GB of RAM, making it suitable for cost-effective GPU-accelerated tasks. This VM is part of the NVv4-series and supports GPU passthrough for improved performance.

1. Build the NC4as_T4_v3 VM

   • In the Azure Portal, create a new Virtual Machine. Under “Size,” select one of the N‑Series VMs, then pick Standard NC4as T4 v3 (4 vCPU, 28 GB RAM, 16 GB GPU VRAM, Premium SSD, Ubuntu 22.04 LTS).

   • Enable SSH (port 22) so you can connect (optional); Secure Boot can remain enabled. Or you can do a serial console also.

   • Once deployed, download the SSH key or password you configured, and note the VM’s public IP.

2. Connect & Prepare the OS

   • From your local shell, if you connect via SSH:

ssh azureuser@<your-vm-ip>

• Or, if you are using a serial console, there is no need to connect with ssh, you just need to directly login via username and password.

• Update packages and install the driver helper:

sudo apt update && sudo apt upgrade -y sudo apt install -y ubuntu-drivers-common

• Confirm you have a CUDA‑capable GPU:

lspci | grep -i NVIDIA

To confirm, that the GPU is CUDA compatible or not

3. Install NVIDIA Driver & CUDA Toolkit

   • Install the latest signed NVIDIA driver (works with Secure Boot):

sudo ubuntu-drivers install sudo reboot

• Install the CUDA 12.5 toolkit (using the Debian package from NVIDIA’s repo):

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb sudo apt install -y ./cuda-keyring_1.1-1_all.deb sudo apt update sudo apt install -y cuda-toolkit-12-5 sudo reboot

• After reboot, verify the driver with:

nvidia-smi

To Verify, Nvidia Drivers are Installed Successfully

4. Set Up PyTorch & Verify CUDA GPU Access

   • Create and activate a Python 3 venv:

python3 -m venv env source env/bin/activate

• Install PyTorch with the CUDA 12.1 build (compatible with your 12.2 driver):

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

Note: The Cuda version might be different due to different cuda hardware, such as, A100, H100, etc.

5. Create a file setup_validation.py

import torch
print("PyTorch version:", torch.__version__)
print("CUDA available:", torch.cuda.is_available())
if torch.cuda.is_available():
    print("Number of GPUs:", torch.cuda.device_count())
    print("Current GPU device name:", torch.cuda.get_device_name(torch.cuda.current_device()))
    print("GPU Capability:", torch.cuda.get_device_capability(torch.cuda.current_device()))
    print("CUDA version PyTorch was compiled with:", torch.version.cuda)
    print("GPU memory allocated:", torch.cuda.memory_allocated() / 1024**2, "MB")
    print("GPU memory cached:", torch.cuda.memory_reserved() / 1024**2, "MB")
else:
    print("CUDA is NOT available. Check driver or installation.")

CUDA accessed successfully

6. Conclusion & Use Cases

With the Azure NC4as_T4_v3 GPU VM successfully configured—featuring the NVIDIA T4 GPU, CUDA Toolkit, and PyTorch with GPU support—you’re now equipped to run GPU-accelerated workloads efficiently. This VM is ideal for fine-tuning deep learning models, deploying trained models as scalable services, and executing GPU-intensive tasks such as inference, image/video processing, or scientific computation. Whether you're developing machine learning pipelines, experimenting with large models, or hosting AI services, this VM offers the right balance of performance and cost for many production and research scenarios.