Denpex

# Denpex > Denpex diagnoses distributed GPU training and inference failures. Given the logs a failed run produced, it correlates available evidence across ranks, looks for the first supported failure, and classifies the failure against 16,400+ known signatures. Diagnosis depends on the supplied evidence; incomplete logs may not establish a root cause. It covers NCCL collective timeouts, CUDA out-of-memory, NVIDIA Xid hardware faults, silent data corruption, stragglers and gray failures across PyTorch, FSDP, DeepSpeed, Megatron, Ray, Slurm, Kubernetes and vLLM. Denpex focuses on failure diagnosis. It complements DCGM device diagnostics, Prometheus metrics, Slurm scheduling, Ray orchestration, and Weights & Biases experiment tracking. Consult the product documentation for the scope of each integration and deployment mode. ## Start here - [/vscode](https://denpex.com/vscode): VS Code extension, local diagnosis workflow, and cloud review options. - [/failures](https://denpex.com/failures): The failure encyclopedia index. Root cause and fix per failure mode. - [/nccl-errors](https://denpex.com/nccl-errors): Every common NCCL error and what it actually indicates, with the environment variables that tell them apart. - [/cuda-errors](https://denpex.com/cuda-errors): CUDA errors and why the traceback usually points at the wrong line. Asynchronous reporting explained. - [/slurm-gpu-errors](https://denpex.com/slurm-gpu-errors): Slurm GPU failures by phase: allocation, cgroup creation, and inside the workload. - [/deepspeed-errors](https://denpex.com/deepspeed-errors): DeepSpeed failures separated by ZeRO stage and lifecycle phase. - [/vllm-errors](https://denpex.com/vllm-errors): vLLM and TensorRT-LLM inference serving failures. - [/xid](https://denpex.com/xid): NVIDIA Xid error reference. Which codes mean a bad GPU and which do not. - [/solutions/best-tools-gpu-cluster-debugging](https://denpex.com/solutions/best-tools-gpu-cluster-debugging): The GPU debugging stack tool