Expand description
Utilities for safe PyTorch operations with proper error handling
§Panic Handling and FFI Boundaries
PyTorch’s C++ backend (libtorch) throws exceptions on OOM and other errors.
The tch-rs bindings translate these C++ exceptions into Rust panics during
operations like tensor allocation (Tensor::zeros) and device transfers
(tensor.to_device).
This module provides wrappers that use catch_unwind to catch these panics
and convert them to proper Result errors with actionable error messages.
Important: This only works because tch-rs performs the C++ exception → Rust
panic translation. Raw libtorch panics from C++ cannot be caught by catch_unwind.
§Thread Safety
These utilities are safe to call from multiple threads, but be aware that catching panics during tensor operations may leave PyTorch’s CUDA/ROCm allocator in an inconsistent state. After catching an OOM panic, it’s recommended to avoid further allocations on the same device, or restart the application to reset allocator state.
Functions§
- check_
gpu_ memory - Check available GPU memory before attempting an operation
- estimate_
tensor_ size_ mb - Estimate tensor size in megabytes
- preflight_
gpu_ check - Pre-flight check before loading a model
- safe_
tensor_ op - Safely execute a tensor operation that might panic (e.g., OOM)
- safe_
to_ device - Safely transfer a tensor to a device (CPU to GPU)
- safe_
zeros - Safely allocate a tensor (zeros, ones, randn, etc.)