#3958 - Opportunistic scheduler

Identifier #3958
Issue type Feature request or suggestion
Title Opportunistic scheduler
Status Open
Tags

Type: Performance (custom)

Handling member Deleted
Addon core
Description Some background Cron hooks are not very time critical, but may have a performance impact. Ideally we would run these hooks only when the server is under low load.

Code in a detector function to find if the server is under 'low load', based on I/O load, CPU usage (uptime command on Linux), and memory usage (configurable thresholds).
This check would be run before every Cron hook that is algorithmically determined as 'optional'.

Make the scheduler (Composr CRON system) use this to determine whether some low priority hooks should get done. For example, updating the sitemap. CRON hooks would specify:
- a "frequency" (currently this logic is hard-coded in v10, and already implemented as the 'minutes_between_runs' setting in v11)
- a "fluff" factor (to determine how much things should be allowed to be late, e.g. 1.6x -- tasks being deferred if not in a low load state)
- a "priority" (i.e. tasks would run in priority order [well actually anything already late past it's fluff factor would be elevated to high priority])
...and the system would automatically launch CRON hooks accordingly.

Hooks should be able to keep calling back and checking to see if they should keep going. Not all hooks would implement that, but particularly long running low priority re-entrant ones (background caching/indexing) would.
Steps to reproduce

Related to

#3501 - Poor-mans-Cron
Funded? No
The system will post a comment when this issue is modified (e.g., status changes). To be notified of this, click "Enable comment notifications".

Rating

Unrated