Scheduler does not start with Gunicorn in v1.12.3

[cryptaliagy]

In case you found this by googling: You can fix this by downgrading to 1.12.1

I have been running into the issue wherein if I use the Flask development server with flask run, the scheduler will start, but by running gunicorn it did not. I tried adding the job (with no persistent memory store) in the create_app function, pre-loading, not-pre-loading, adding the tasks in the @app.before_first_request decorated function, and even starting the scheduler 3 separate times in my code (once in create_app, once in the before_first_request, and once in a wsgi.py file). In all cases, querying the /scheduler api always returned a running=false state.

Here's the command I used to start gunicorn:

gunicorn --bind "$FLASK_RUN_HOST:$FLASK_RUN_PORT" --worker-connections 2 --threads 4 --reload --preload 'backend.wsgi:app';

environment configs:

FLASK_APP=/app/backend/__init__.py
FLASK_ENV=development
FLASK_RUN_HOST=0.0.0.0
FLASK_RUN_PORT=80

Using docker image python:3.9-slim-buster, running Python 3.9.10

I've spent a few hours slamming my head against a wall on this one, and I think it comes down to the PR #140 that resolved #139. I'd probably recommend adding something like a SCHEDULER_ALLOW_RELOADER environment variable to bypass it, and specifying it in the documentation.

[christopherpickering]

Any chance you can share your wsgi and init files for your app?

[cryptaliagy]

I guess, though I would be surprised if it gives you any insight deeper than what is above

# __init__.py
import logging
import os
from typing import (
    Optional,
)

from flask import (
    Flask,
    request,
)

from backend.routes import root
from backend.extensions import (
    db,
    migrate,
    cron,
)
from backend.utils import (
    configure_logger,
    do_task,
)


def create_app(config_object_path: Optional[str] = None) -> Flask:
    '''
    Flask app factory function. Creates and initializes the app.

    Args:
        config_object_path: an optional string defining the path of the
            configuration object

    Returns:
        A configured Flask app
    '''
    app = Flask(__name__)

    if config_object_path is None:
        app.config.from_object('backend.config.EnvironmentConfiguration')  # type: ignore
    else:
        app.config.from_object(config_object_path)

    configure_logger(app.logger, debug=app.config['ENV'] == 'development')

    return initialize_app(app)


def initialize_app(app: Flask) -> Flask:
    '''
    Sets up all the applicable extensions and routes from blueprints

    Args:
        app: A pre-configured application object

    Returns:
        The same object, after all routes and initializations
    '''
    app.register_blueprint(root)

    db.init_app(app)
    migrate.init_app(app, db)
    cron.init_app(app)

    @app.before_request
    def _():
        app.logger.info(
            'Hit route path %s',
            request.path
        )

    @cron.task('interval', id='task', minutes=10, misfire_grace_time=900)
    def _():
        with app.app_context():
            do_task()

    cron.start()

    return app

# wsgi.py
from backend import create_app

app = create_app()

[christopherpickering]

Hmm I use gunicorn as well and haven't had that problem. The only difference in my startup is that I put my register_blueprint inside a with app.app_context():.

Here's how I call gunicorn.

gunicorn --worker-class=gevent --workers 1 --threads 30  --bind  unix:website.sock --umask 007 scheduler:app

[cryptaliagy]

Hmm I use gunicorn as well and haven't had that problem. The only difference in my startup is that I put my register_blueprint inside a with app.app_context():.

Here's how I call gunicorn.

gunicorn --worker-class=gevent --workers 1 --threads 30  --bind  unix:website.sock --umask 007 scheduler:app

Is it potentially due to not using reload that you don't experience this issue? Again, I do think it's just due to that change in #140 - It silently causes the scheduler to not start if the process is the reloader (but isn't documented), which is exactly the use case that I'm trying to do.

[cryptaliagy]

On some further inspection, I think #163 might be connected to this issue as well: the user who reported it seems to likewise be utilizing the reloader thread

[christopherpickering]

Ah yep I'm sure you are correct. Feb 22, 2022, 8:28 PM by ***@***.***:

…

On some further inspection, I think > #163 <#163>> might be connected to this issue as well: the user who reported it seems to likewise be utilizing the reloader thread — Reply to this email directly, > view it on GitHub <#198 (comment)>> , or > unsubscribe <https://github.com/notifications/unsubscribe-auth/AEHW6IQOXGBEZD25D4ISFJLU4RA3ZANCNFSM5OYMROGQ>> . Triage notifications on the go with GitHub Mobile for > iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>> or > Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>> . You are receiving this because you commented.> Message ID: > <viniciuschiele/flask-apscheduler/issues/198/1048396283> @> github> .> com>

[cryptaliagy]

Decided to play around a little bit with this, so I wrote a test app to reproduce the issue: https://github.com/taliamax/flask-test

On further inspection, I think I was very wrong about the actual cause, though correct about the location. The offending line seems to be

flask-apscheduler/flask_apscheduler/scheduler.py

Line 97 in 5f878c4

if flask.helpers.get_debug_flag() and not werkzeug.serving.is_running_from_reloader():

The problem seems to be that FLASK_ENV is set to development, but using gunicorn will always cause werkzeug.serving.is_running_from_reloader() to be False, since gunicorn never seems to set the werkzeug environment variables - which makes sense.

A potential fix might be to either:

1. Use some other configuration option (or environment variable), such as "SCHEDULER_ALWAYS_START_SCHEDULER" or something of the sort to override the options used on the line above or
2. Check the server version (which might not be set depending on when .start() is called? some quick testing with flask run instead of gunicorn didn't have werkzeug set the SERVER_VERSION env variable during create_app, whereas gunicorn had) to make sure that the thing serving the app is actually werkzeug before trying to use .is_running_from_reloader()

[christopherpickering]

This is the expected behavior of the scheduler > using debug + a non debug server will prevent the scheduler from running. See #220 (comment) for more information.

[cryptaliagy]

It seems incredibly strange to me that the guidance offered here is "use only a specific server". Is there really no room to even enable a configuration flag for bypassing that?

[viniciuschiele]

Because there is no reliable way to detect if werkzeug is being used, I think we should remove this condition and recommend users to disable Flask reloader (app.run(use_reloader=False) or flask run --no-reload) when running locally, we could also log a warning if is_running_from_reloader returns True to alert/guide the devs.

Thoughts?

cc: @Gkirito

[Gkirito]

Because there is no reliable way to detect if werkzeug is being used, I think we should remove this condition and recommend users to disable Flask reloader (app.run(use_reloader=False) or flask run --no-reload) when running locally, we could also log a warning if is_running_from_reloader returns True to alert/guide the devs.

Thoughts?

cc: @Gkirito

Good idea!
Yea, I found too many different environments, I agree with your thoughts. Let us delete this condition