Skip to main content

Module torch_utils

Module torch_utils 

Source
Expand description

Utilities for safe PyTorch operations with proper error handling

§Panic Handling and FFI Boundaries

PyTorch’s C++ backend (libtorch) throws exceptions on OOM and other errors. The tch-rs bindings translate these C++ exceptions into Rust panics during operations like tensor allocation (Tensor::zeros) and device transfers (tensor.to_device).

This module provides wrappers that use catch_unwind to catch these panics and convert them to proper Result errors with actionable error messages.

Important: This only works because tch-rs performs the C++ exception → Rust panic translation. Raw libtorch panics from C++ cannot be caught by catch_unwind.

§Thread Safety

These utilities are safe to call from multiple threads, but be aware that catching panics during tensor operations may leave PyTorch’s CUDA/ROCm allocator in an inconsistent state. After catching an OOM panic, it’s recommended to avoid further allocations on the same device, or restart the application to reset allocator state.

Functions§

check_gpu_memory
Check available GPU memory before attempting an operation
estimate_tensor_size_mb
Estimate tensor size in megabytes
preflight_gpu_check
Pre-flight check before loading a model
safe_tensor_op
Safely execute a tensor operation that might panic (e.g., OOM)
safe_to_device
Safely transfer a tensor to a device (CPU to GPU)
safe_zeros
Safely allocate a tensor (zeros, ones, randn, etc.)