| Tool/Framework | Description |
|---|---|
| TensorRT-LLM | Python API to define Large Language Models (LLMs) and build TensorRT engines |
| nvtop | GPU & Accelerator process monitoring for AMD, Apple, Huawei, Intel, NVIDIA |
| nvitop | An interactive NVIDIA-GPU process viewer and beyond, the one-stop solution for GPU process management |
NCCL
The NVIDIA Collective Communications Library (NCCL, pronounced “Nickel”) is a library providing inter-GPU communication primitives that are topology-aware and can be easily integrated into applications.
- See if NCLL is installed correctly.
find /usr -name "libnccl.so*" 2>/dev/null