Summary
As backup/cluster administrator
I want to limit the amount of pending backups created by a schedule
So that I avoid creating large amounts of pending backups if something is preventing them from running/completing.
Context
I've recently had some nodes in my cluster fail, and due to affinity rules some of the workloads that were running on these nodes could not be scheduled on any of the remaining nodes. As a result of this the backup jobs created by K8up could also not be scheduled anywhere and got stuck in Pending state. I have failedJobsHistoryLimit and successfulJobsHistoryLimit both set to 1 for this schedule, but there appears to not be a limit on the pending jobs, so over Christmas dozens of jobs were created for each schedule.
The same issue could probably also happen if the backups start taking longer than the configured interval in the schedule & concurrentRunsAllowed is set to false.
I can't think of any reason to ever want to have more than 1 pending backup, so I would propose simply preventing schedules from creating a backup job if the previous one is still pending. Alternatively (if anyone else does see a use for wanting to have multiple pending backups) a pendingJobsLimit setting could be created.
Out of Scope
No response
Further links
No response
Acceptance Criteria
- Before creating a
Backup from a Schedule check whether the previous Backup job is still Pending, and skip creating the new Backup if this is the case.
Implementation Ideas
No response
Summary
As backup/cluster administrator
I want to limit the amount of pending backups created by a schedule
So that I avoid creating large amounts of pending backups if something is preventing them from running/completing.
Context
I've recently had some nodes in my cluster fail, and due to affinity rules some of the workloads that were running on these nodes could not be scheduled on any of the remaining nodes. As a result of this the backup jobs created by K8up could also not be scheduled anywhere and got stuck in
Pendingstate. I havefailedJobsHistoryLimitandsuccessfulJobsHistoryLimitboth set to1for this schedule, but there appears to not be a limit on the pending jobs, so over Christmas dozens of jobs were created for each schedule.The same issue could probably also happen if the backups start taking longer than the configured interval in the schedule &
concurrentRunsAllowedis set tofalse.I can't think of any reason to ever want to have more than 1 pending backup, so I would propose simply preventing schedules from creating a backup job if the previous one is still pending. Alternatively (if anyone else does see a use for wanting to have multiple pending backups) a
pendingJobsLimitsetting could be created.Out of Scope
No response
Further links
No response
Acceptance Criteria
Backupfrom aSchedulecheck whether the previousBackupjob is stillPending, and skip creating the newBackupif this is the case.Implementation Ideas
No response