Crontabs with ranges fail to parse in uwsgi

[chr15m]

tl;dr: we should probably mention in the docs that uwsgi doesn't support ranges (and check exactly what it does support).

Discussed in #330

^{Originally posted by jfmatth October 5, 2023}
Noticed that my cron line isn't being picked up correctly by Piku, and not being the crontab guru, hoping to find out if i'm right?

current

cron: * 6-13 * * 1-5 ./bin/cron_job.sh

git push output from piku

cron.1 | clock source: unix
cron.1 | pcre jit disabled
cron.1 | detected number of CPU cores: 1
cron.1 | current working directory: /home/piku/.piku/uwsgi-enabled
cron.1 | detected binary path: /usr/bin/uwsgi-core
cron.1 | chdir() to /home/piku/.piku/apps/stockprices
cron.1 | your processes number limit is 63209
cron.1 | your memory page size is 4096 bytes
cron.1 | detected max file descriptor number: 1024
cron.1 | lock engine: pthread robust mutexes
cron.1 | thunder lock: disabled (you can enable it with --thunder-lock)
cron.1 | your mercy for graceful operations on workers is 60 seconds
cron.1 | [uwsgi-cron] command "1-5 ./bin/cron_job.sh" registered as cron task
cron.1 | *** Operational MODE: no-workers ***
cron.1 | !!!!!!!!!!!!!! WARNING !!!!!!!!!!!!!!
cron.1 | no request plugin is loaded, you will not be able to manage requests.
cron.1 | you may need to install the package for your language of choice, or simply load it with --plugin.
cron.1 | !!!!!!!!!!! END OF WARNING !!!!!!!!!!
cron.1 | gracefully (RE)spawned uWSGI master process (pid: 119472)

Seems to only be picking up the cron.1 | [uwsgi-cron] command "1-5 ./bin/cron_job.sh" registered as cron task schedule?

Looking at the regex defined in piku.py

CRON_REGEXP = "^((?:(?:\*\/)?\d+)|\*) ((?:(?:\*\/)?\d+)|\*) ((?:(?:\*\/)?\d+)|\*) ((?:(?:\*\/)?\d+)|\*) ((?:(?:\*\/)?\d+)|\*) (.*)$"

Thoughts?

I'm trying to have the job run from 6am to 1pm every minute M-F, if my crontab is wrong, let me know too.

shelling into the app, i see the uwsgi line is

cron = -1 6-13 -1 -1 1-5 ./bin/cron_job.sh

Updates

After troubleshooting a lot, don't know why the jobs aren't running like they should

issue 1 - UTC counts, so I changed the job to run on UTC time, not local, so changed to run UTC, for just one hour and it works.
issue 2 - doesn't seem to like ranges?

cron: * 18-20 * * * ./bin/cron_job.sh

This doesn't work for whatever reason, although UWSGI see's it OK according to the file and the docs

piku@ct-piku-1:~/.piku/uwsgi-enabled$ more stockprices_cron.1.ini
[uwsgi]
...
virtualenv = /home/piku/.piku/envs/stockprices
cron = -1 18-20 -1 -1 -1 ./bin/cron_job.sh

Really need ranges to work, but not sure what I'm doing wrong :(

Updates 2

OK, digging into the UWSGI C code, looks like they only support INT's on hours, so my guess is they are expecting a single number, uggh.

I'll assume that's the case, and create multiple cron jobs and see how that goes, uggh

Final solution

Just in case anyone else run's into this...

Kudo's to the Piku team for sending my cron string over to UWSGI, but unfortunately, UWSGI doesn't provide full crontab regex, which is fine. I ended up using Python's APSCHEDULER.

APscheduler is an amazing library, and even has a 3rd party add-on for Django. With a little work in apscheduler, I used their from_crontab trigger type:

trigger=CronTrigger.from_crontab("* 13-21 * * 1-5"),

This worked like I needed it to.

I

[github-actions]

This issue is stale because it has been open for 90 days with no activity.

[jfmatth]

go ahead and close this.

[chr15m]

I think there are three actions required:

• Update the documentation to mention it (need to check if we did this already because I can't remember).
• Update the documentation to recommend alternative solutions for running crontabs in different languages.
• Update the code to warn the user if a range is detected.

[rcarmo]

Yeah, I just got bitten by this too. I'm going to see if I can review the docs

[jfmatth]

Would a summary of the limitation be enough and a workaround for the docs?

[chr15m]

I think that makes sense at least in the short term.