Skip to content

KB-016: Kopia repository server OOM = repository size, not a maintenance failure

Status: Resolved (raised to 2Gi); revisit as the repository grows.

Symptom

The volsync-system/kopia repository server OOMKills in a crashloop (exit 137) at its memory limit, even a trivial kopia maintenance info exec OOMs. The server log shows Found too many index blobs ... run kopia maintenance, which looks like maintenance is broken.

Cause

It's repository size, not maintenance. The VolSync Kopia repository had grown to ~320 GB / 3.4M in-use contents / ~3,462 index blobs. The server loads the full index into memory on start, which exceeds the limit. The "too many index blobs" message is a red herring: maintenance is actually running fine:

  • The KopiaMaintenance CR drives the kopia-maint-daily-* cronjob every 4h. (Raised to hourly after this incident: index blobs were still being created faster than epoch compaction reclaimed them, so the schedule is now 30 * * * *.)
  • Its job log shows it owns maintenance, runs quick then full maintenance (GC + epoch compaction), and reports MAINTENANCE_STATUS: SUCCESS in a few minutes.
  • The maintenance job pod has no resource limits, so it doesn't OOM, only the long-lived server did.

Fix

Raise the server's memory in kubernetes/apps/volsync-system/kopia/app/helmrelease.yaml:

resources:
  requests:
    memory: 1Gi      # floor
  limits:
    memory: 2Gi

The repository keeps growing, so if it OOMs again at 2Gi, go 3-4Gi. (Maintenance config lives at kubernetes/apps/volsync-system/volsync/maintenance/.)

References