Feature Request / Bug: Robotic buzzing (electric shaver sound) and VRAM overflow during Batch Inference on 3GB/low-VRAM GPUs

[LebedevIV]

When running Batch Inference on full-length audio tracks using a 3GB GPU (NVIDIA GeForce GTX 1060 3GB), the output audio turns into a permanent synthetic/robotic buzzing sound (resembling an electric shaver). The system does not throw a CUDA out-of-memory error, but the inference breaks because of underlying PyTorch 2.7+ compatibility quirks with legacy Pascal architectures.

The problem is twofold:

1. FP16 Breakdown on 3GB: The framework automatically forces FP16 (Half Precision), which silently corrupts tensors into NaN zeros on older 3GB cards.
2. VRAM Overload: Long audio files completely choke the available 3GB buffer when loaded all at once.

I solved this by modifying configs/config.py to force FP32 precision and offload the data caching pipeline to the host system RAM via CPU environment flags:

if torch.cuda.is_available():
    infer_device = torch.device("cuda:0")
    infer_dtype = torch.float32
    infer_gpu_mem = 3.0
    # FIX FOR 3GB/4GB GPUS:
    is_half = False
    os.environ["RVC_AUDIO_FORCE_CPU"] = "1"

I request to expose these two parameters directly into the WebUI "Model Inference" / "Batch Inference" tabs as checkboxes or advanced toggles:

• Force FP32 Precision (Disable is_half)
• Offload DataLoader Caching to System RAM (CPU)

This will instantly allow thousands of users with budget or older GPUs (GTX 1060 3GB, 1050Ti, 960) to safely process full-length songs in near real-time using Windows Shared GPU Memory without crashing the pipeline.

Additional context

• OS: Windows 10 / 11 x64
• Python: 3.12+ (Python 3.12 branch)
• PyTorch: 2.7.1+cu118
• GPU: NVIDIA GeForce GTX 1060 3GB