Error: Too Many Open Files

Issue

HVR jobs fail with the following error:

Too many open files

Environment

• HVR 6
• Platform: Windows-based hub server
• Channels: 342
• Jobs: Capture, integrate, and stats

Resolution

To troubleshoot this issue:

1. On the hub host, run hvr schedstutter.hvr -a to enable stutter mode. This configures hvrscheduler.exe to open, write to, and close each log file for every message instead of keeping the file open.
2. Confirm that the scheduler logs the following message and that the errors stop: Activating 'stutter' i/o to %d logfile%s.
3. If the number of concurrent jobs approaches 2,042, distribute the channels across multiple hub instances.
4. Exclude HVR directories from antivirus, backup, monitoring, and indexing processes to reduce file lock or handle contention.
5. Use a tool such as Process Explorer to monitor the handle counts for hvrscheduler.exe and hvrhubserver.exe. A steadily increasing count may indicate a leak or contention.

Cause

This issue occurs when hvrscheduler.exe exceeds the Windows C runtime file descriptor limit of 2,048. It is more likely to occur when the hub manages many channels and concurrent jobs because the scheduler keeps multiple log files open.