Specific timezones cause high CPU load on UNIX systems using croner <= 7.x about croner HOT 10 CLOSED

I have the same job scheduled for all these timezones:

Europe/Andorra
Asia/Dubai
Asia/Kabul
America/Puerto_Rico
America/Puerto_Rico
Europe/Tirane
Asia/Yerevan
Africa/Lagos
Antarctica/Casey
America/Argentina/Buenos_Aires
Pacific/Pago_Pago
Europe/Vienna
Antarctica/Macquarie
America/Puerto_Rico
Europe/Helsinki
Asia/Baku
Europe/Belgrade
America/Barbados
Asia/Dhaka
Europe/Brussels
Africa/Abidjan
Europe/Sofia
Asia/Qatar
Africa/Maputo
Africa/Lagos
America/Puerto_Rico
Atlantic/Bermuda
Asia/Kuching
America/La_Paz
America/Puerto_Rico
America/Araguaina
America/Toronto
Asia/Thimphu
Africa/Maputo
Europe/Minsk
America/Belize
America/Cambridge_Bay
Asia/Yangon
Africa/Lagos
Africa/Lagos
Africa/Lagos
Europe/Zurich
Africa/Abidjan
Pacific/Rarotonga
America/Punta_Arenas
Africa/Lagos
Asia/Shanghai
America/Bogota
America/Costa_Rica
America/Havana
Atlantic/Cape_Verde
America/Puerto_Rico
Asia/Bangkok
Asia/Famagusta
Europe/Prague
Europe/Berlin
Africa/Nairobi
Europe/Berlin
America/Puerto_Rico
America/Santo_Domingo
Africa/Algiers
America/Guayaquil
Europe/Tallinn
Africa/Cairo
Africa/El_Aaiun
Africa/Nairobi
Africa/Ceuta
Africa/Nairobi
Europe/Helsinki
Pacific/Fiji
Atlantic/Stanley
Pacific/Guadalcanal
Atlantic/Faroe
Europe/Paris
Africa/Lagos
Europe/London
America/Puerto_Rico
Asia/Tbilisi
America/Cayenne
Europe/London
Africa/Abidjan
Europe/Gibraltar
America/Danmarkshavn
Africa/Abidjan
Africa/Abidjan
America/Puerto_Rico
Africa/Lagos
Europe/Athens
Atlantic/South_Georgia
America/Guatemala
Pacific/Guam
Africa/Bissau
America/Guyana
Asia/Hong_Kong
America/Tegucigalpa
Europe/Belgrade
America/Port-au-Prince
Europe/Budapest
Asia/Jakarta
Europe/Dublin
Asia/Jerusalem
Europe/London
Asia/Kolkata
Indian/Chagos
Asia/Baghdad
Asia/Tehran
Africa/Abidjan
Europe/Rome
Europe/London
America/Jamaica
Asia/Amman
Asia/Tokyo
Africa/Nairobi
Asia/Bishkek
Asia/Bangkok
Pacific/Kanton
Africa/Nairobi
America/Puerto_Rico
Asia/Pyongyang
Asia/Seoul
Asia/Riyadh
America/Panama
Asia/Almaty
Asia/Bangkok
Asia/Beirut
America/Puerto_Rico
Europe/Zurich
Asia/Colombo
Africa/Monrovia
Africa/Johannesburg
Europe/Vilnius
Europe/Brussels
Europe/Riga
Africa/Tripoli
Africa/Casablanca
Europe/Paris
Europe/Chisinau
Europe/Belgrade
America/Puerto_Rico
Africa/Nairobi
Pacific/Kwajalein
Europe/Belgrade
Africa/Abidjan
Asia/Yangon
Asia/Choibalsan
Asia/Macau
Pacific/Guam
America/Martinique
Africa/Abidjan
America/Puerto_Rico
Europe/Malta
Indian/Mauritius
Indian/Maldives
Africa/Maputo
America/Bahia_Banderas
Asia/Kuching
Africa/Maputo
Africa/Windhoek
Pacific/Noumea
Africa/Lagos
Pacific/Norfolk
Africa/Lagos
America/Managua
Europe/Brussels
Europe/Berlin
Asia/Kathmandu
Pacific/Nauru
Pacific/Niue
Pacific/Auckland
Asia/Dubai
America/Panama
America/Lima
Pacific/Gambier
Pacific/Bougainville
Asia/Manila
Asia/Karachi
Europe/Warsaw
America/Miquelon
Pacific/Pitcairn
America/Puerto_Rico
Asia/Gaza
Atlantic/Azores
Pacific/Palau
America/Asuncion
Asia/Qatar
Asia/Dubai
Europe/Bucharest
Europe/Belgrade
Asia/Anadyr
Africa/Maputo
Asia/Riyadh
Pacific/Guadalcanal
Asia/Dubai
Africa/Khartoum
Europe/Berlin
Asia/Singapore
Africa/Abidjan
Europe/Belgrade
Europe/Berlin
Europe/Prague
Africa/Abidjan
Europe/Rome
Africa/Abidjan
Africa/Nairobi
America/Paramaribo
Africa/Juba
Africa/Sao_Tome
America/El_Salvador
America/Puerto_Rico
Asia/Damascus
Africa/Johannesburg
America/Grand_Turk
Africa/Ndjamena
Asia/Dubai
Africa/Abidjan
Asia/Bangkok
Asia/Dushanbe
Pacific/Fakaofo
Asia/Dili
Asia/Ashgabat
Africa/Tunis
Pacific/Tongatapu
Europe/Istanbul
America/Puerto_Rico
Pacific/Tarawa
Asia/Taipei
Africa/Nairobi
Europe/Kyiv
Africa/Nairobi
Pacific/Pago_Pago
America/Adak
America/Montevideo
Asia/Samarkand
Europe/Rome
America/Puerto_Rico
America/Caracas
America/Puerto_Rico
America/Puerto_Rico
Asia/Bangkok
Pacific/Efate
Pacific/Tarawa
Pacific/Apia
Asia/Riyadh
Africa/Nairobi
Africa/Johannesburg
Africa/Maputo
Africa/Maputo

from croner.

Thanks, Will have a look asap

from croner.

I tested this both on a production linux server running Ubuntu 22.04.3 LTS and a intel macbook running Ventura 13.4.1.

Croner is ^7.0.5 and node is v21.4.0

I could observe the same behavior in both cases, all the logic of my script is done in callbacks there is nothing running aside from these callbacks.

A few suggestions to keep testing:

• Try scheduling multiple jobs on the same timezones
• Try scheduling like 10-20 jobs at once

from croner.

Yep that worked! Both on the production server and my macbook! Thanks for the quick replies and help!

from croner.

Oh! Which timezone, if it's needed for debugging?

from croner.

Looking into this. Parsing seem to be fine:

I'm also trying this code, with an additional log to ensure that there is no internal loop going bananas:

import { Cron } from "./src/croner.js";

console.log("Starting normal cron");
new Cron("* * * * * *", { maxRuns: 5}, async () => {
    console.log("Running normal cron")
});

setTimeout(() => {
    console.log("Starting timezoned cron");
    new Cron("* * * * * *", { maxRuns: 5, timezone: "Asia/Calcutta"}, async () => {
        console.log("Running timezoned cron")
    });        
}, 6000);

And the results seem fine, without any surge in CPU usage:

Using your cron pattern 0 8 * * * i get this output:

Which essentially means that croner is chunking up the waiting in 30 second pieces. Internally there is a setTimeout started which is waiting at most 30 seconds then re-checking how much time is left. Shouln't normally cause high cpu.

@Earu Which environment are you using (runtime and version, croner version, linux distro)?

from croner.

Thanks, i'll have a look using a similar setup 👍

from croner.

Currently using Debian 12, Node 21.4.0, Croner ^7.0.5, and running 2000 instances on an old laptop. Running this gives me a total system CPU usage of about 3-5% (which is unchanged from prior to starting the process).

hexagon@hexabox:~/git/test> uname -a
Linux hexabox 6.1.0-13-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.55-1 (2023-09-29) x86_64 GNU/Linux
hexagon@hexabox:~/git/test> cat /etc/debian_version
12.2
hexagon@hexabox:~/git/test> node --version
v21.4.0
hexagon@hexabox:~/git/test> cat package.json | grep croner
    "croner": "^7.0.5"
hexagon@hexabox:~/git/test> cat test.js
import { Cron } from "croner";

for(let i = 0; i < 2000; i++) {
    Cron("0 8 * * *", {
        timezone: "Europe/London",
    }, async () => {
        console.log('test');
    });

However! With this next test, I do actually see a surge in CPU usage:

import { Cron } from "croner";

let uniqueTimeZones = [
    "Europe/Andorra",
    "Asia/Dubai",
    "Asia/Kabul",
    "America/Puerto_Rico",
    "Europe/Tirane",
    "Asia/Yerevan",
    "Africa/Lagos",
    "Antarctica/Casey",
    "America/Argentina/Buenos_Aires",
    "Pacific/Pago_Pago",
    "Europe/Vienna",
    "Antarctica/Macquarie",
    "Europe/Helsinki",
    "Asia/Baku",
    "Europe/Belgrade",
    "America/Barbados",
    "Asia/Dhaka",
    "Europe/Brussels",
    "Africa/Abidjan",
    "Europe/Sofia",
    "Asia/Qatar",
    "Africa/Maputo",
    "Atlantic/Bermuda",
    "Asia/Kuching",
    "America/La_Paz",
    "America/Araguaina",
    "America/Toronto",
    "Asia/Thimphu",
    "Europe/Minsk",
    "America/Belize",
    "America/Cambridge_Bay",
    "Asia/Yangon",
    "Europe/Zurich",
    "Pacific/Rarotonga",
    "America/Punta_Arenas",
    "Asia/Shanghai",
    "America/Bogota",
    "America/Costa_Rica",
    "America/Havana",
    "Atlantic/Cape_Verde",
    "Asia/Bangkok",
    "Asia/Famagusta",
    "Europe/Prague",
    "Europe/Berlin",
    "Africa/Nairobi",
    "America/Santo_Domingo",
    "Africa/Algiers",
    "America/Guayaquil",
    "Europe/Tallinn",
    "Africa/Cairo",
    "Africa/El_Aaiun",
    "Africa/Ceuta",
    "Pacific/Fiji",
    "Atlantic/Stanley",
    "Pacific/Guadalcanal",
    "Atlantic/Faroe",
    "Europe/Paris",
    "Europe/London",
    "Asia/Tbilisi",
    "America/Cayenne",
    "Europe/Gibraltar",
    "America/Danmarkshavn",
    "Europe/Athens",
    "Atlantic/South_Georgia",
    "America/Guatemala",
    "Pacific/Guam",
    "Africa/Bissau",
    "America/Guyana",
    "Asia/Hong_Kong",
    "America/Tegucigalpa",
    "America/Port-au-Prince",
    "Europe/Budapest",
    "Asia/Jakarta",
    "Europe/Dublin",
    "Asia/Jerusalem",
    "Asia/Kolkata",
    "Indian/Chagos",
    "Asia/Baghdad",
    "Asia/Tehran",
    "Europe/Rome",
    "America/Jamaica",
    "Asia/Amman",
    "Asia/Tokyo",
    "Asia/Bishkek",
    "Pacific/Kanton",
    "Asia/Pyongyang",
    "Asia/Seoul",
    "Asia/Riyadh",
    "America/Panama",
    "Asia/Almaty",
    "Asia/Beirut",
    "Asia/Colombo",
    "Africa/Monrovia",
    "Africa/Johannesburg",
    "Europe/Vilnius",
    "Europe/Riga",
    "Africa/Tripoli",
    "Africa/Casablanca",
    "Europe/Chisinau",
    "Pacific/Kwajalein",
    "Asia/Choibalsan",
    "Asia/Macau",
    "America/Martinique",
    "Europe/Malta",
    "Indian/Mauritius",
    "Indian/Maldives",
    "America/Bahia_Banderas",
    "Africa/Windhoek",
    "Pacific/Noumea",
    "Pacific/Norfolk",
    "America/Managua",
    "Asia/Kathmandu",
    "Pacific/Nauru",
    "Pacific/Niue",
    "Pacific/Auckland",
    "America/Lima",
    "Pacific/Gambier",
    "Pacific/Bougainville",
    "Asia/Manila",
    "Asia/Karachi",
    "Europe/Warsaw",
    "America/Miquelon",
    "Pacific/Pitcairn",
    "Asia/Gaza",
    "Atlantic/Azores",
    "Pacific/Palau",
    "America/Asuncion",
    "Europe/Bucharest",
    "Asia/Anadyr",
    "Pacific/Guadalcanal",
    "Africa/Khartoum",
    "Asia/Singapore",
    "Europe/Prague",
    "America/Paramaribo",
    "Africa/Juba",
    "Africa/Sao_Tome",
    "America/El_Salvador",
    "Asia/Damascus",
    "America/Grand_Turk",
    "Africa/Ndjamena",
    "Asia/Dushanbe",
    "Pacific/Fakaofo",
    "Asia/Dili",
    "Asia/Ashgabat",
    "Africa/Tunis",
    "Pacific/Tongatapu",
    "Europe/Istanbul",
    "Pacific/Tarawa",
    "Asia/Taipei",
    "Europe/Kyiv",
    "America/Adak",
    "America/Montevideo",
    "Asia/Samarkand",
    "America/Caracas",
    "Pacific/Efate",
    "Pacific/Apia"
];

const cronPattern = "0 8 * * *";

const instancesPerTimeZone = 10;

console.info(`Registering ${instancesPerTimeZone} (instances) * ${uniqueTimeZones.length} (time zones) = ${instancesPerTimeZone*uniqueTimeZones.length} cron instances running pattern '${cronPattern}'`);

for(let tzIdx = 0; tzIdx < uniqueTimeZones.length; tzIdx++) {
    for(let inst = 0; inst < instancesPerTimeZone; inst++) {
        Cron(cronPattern, {
            timezone: uniqueTimeZones[tzIdx],
        }, async () => {
            console.log('Run');
        });
    }
}

@Earu, I've narrowed it down to "America/Puerto_Rico" as one of the time zones causing this issue. However, it seems to be resolved in 8.0.0. If you're using Node 18 or higher in all your environments, can you try upgrading to ^8.0.0 to see if that fixes the issue for you?

Even if it does, please leave this issue open. I'll need to address this in the 7.x channel for legacy solutions, and I'll also have to figure out a safeguard to prevent this from happening in the 8.x channel.

from croner.

Resolved in 8.0.0 and potentially hotfixed for 7.x in [email protected] and channel croner@legacy-node-support.

from croner.

Now resolved in the 7.x branch through 7.0.7

from croner.