Skip to content

feat: add get_gpu_errors tool for ECC and XID error reporting #28

Description

@pmady

Summary

Add a get_gpu_errors MCP tool that surfaces GPU error state — ECC memory errors, XID errors, and driver health.

Background

GPUs can have silent errors that degrade inference accuracy or cause crashes. NVML exposes:

  • nvmlDeviceGetMemoryErrorCounter — ECC single-bit and double-bit errors
  • nvmlDeviceGetTotalEccErrors — total corrected/uncorrected errors
  • XID errors via event monitoring

Requirements

  • New get_gpu_errors tool returning per-device error counts
  • Fields: ecc_single_bit, ecc_double_bit, retired_pages, xid_errors (if available)
  • Graceful handling when ECC is not supported (consumer GPUs)

Suggested output

{
  "index": 0,
  "uuid": "GPU-aaaa-1111",
  "ecc_supported": true,
  "ecc_single_bit_errors": 0,
  "ecc_double_bit_errors": 0,
  "retired_pages_single_bit": 0,
  "retired_pages_double_bit": 0
}

Implementation hints

  • Add Errors() ([]ErrorInfo, error) to the Collector interface
  • Implement in gpu/nvml.go, stub in gpu/stub.go, add to gpu/mock.go
  • Register tool in server/server.go following existing pattern
  • Add tests in server/server_test.go using mock

Acceptance criteria

  • New tool registered and callable
  • Graceful fallback for non-ECC GPUs
  • Tests using mock (no GPU required)
  • README tools table updated

Difficulty: Intermediate — requires understanding NVML error APIs

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestgpuGPU hardware related

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions