how can I add multiple redis url and redbeat_broker_url rabbitmq

[oguzhannkayaa]

Hi, I am trying to add multiple redis (for high availability) url to redbeat_redis_url and I also wants to add broker_url for rabbitmq.

I am trying to use rabbitmq as a message broker. and redis as a beat data store. because of clustering.

app.conf.update(
broker_url = 'amqp://localhost:5672/',
redbeat_redis_url = 'redis://master1:6379/1',
redbeat_redis_options = {
'sentinels': [('slave2:6379'),
('slave3.18:6379')],
'socket_timeout': 0.1,
'retry_period': 0.1

}
)
After I close master, beat stop. Does go to red_beat_redis_options

and also I am getting this error when I run celery beat -S redbeat.RedBeatScheduler.

beat raised exception : ConnectionError('Error -2 connecting to redis-sentinel:26379. Name or service not known.',)

How can I create a service_name and password in redis-sentinel I am not trying to use redis as a message broker. I am using celery-redbeat to store celerybeat data in redis-sentinel cluster from this page.https://pypi.org/project/celery-redbeat/

and

from this configuration

redbeat_redis_url = 'redis-sentinel://redis-sentinel:26379/0'
redbeat_redis_options = {
'sentinels': [('192.168.1.1', 26379),
('192.168.1.2', 26379),
('192.168.1.3', 26379)],
'socket_timeout': 0.1,
}

I add 192.168.1.1:26379 instead of redis-sentinel:/26379 but when master node down in redis-sentinel cluster beat is down too.

redbeat_redis_url = 'redis-sentinel://192.168.1.1:26379/0'
redbeat_redis_options = {
'sentinels': [('192.168.1.2', 26379),
('192.168.1.3', 26379)],
'socket_timeout': 0.1,
}

[rive-n]

wow no upd over here. What a nice project...

[man4red]

I know this is old but hope it would help

the key is to set redbeat_redis_url to sentinel dsn
like for example, your sentinel dsn might look like this:
sentinel://:redis@redis-sentinel:26379/0;sentinel://:redis@redis-sentinel2:26379/0;sentinel://:redis@redis-sentinel3:26379/0

but redbeat/scheduler.py:124 checks for

        elif conf.redis_url.startswith('redis-sentinel') and 'sentinels' in redis_options:
            ...

so you have to change your sentinel dsn (for redbeat only) to reidis-sentinel

please note that it is perfectly valid of celeryconfig to use sentinel as a schema within the dsn but not for redbeat tho

anyways, if your sentinel requires auth - provide password to sentinel_kwargs
the other options are for redis connection only

my celeryconfig.py looks like that

# at this point we would just dump it from celery_settings (pydantic model)
locals().update(celery_settings.model_dump())

# now we can override some
if redis_settings.redis_mode == "sentinel":
    broker_url = redis_settings.sentinel_dsn
    result_backend_transport_options = REDIS_PARAMS
    broker_transport_options = REDIS_PARAMS
    redbeat_redis_url = redis_settings.sentinel_dsn.replace("sentinel://", "redis-sentinel://")
    redbeat_redis_options = {
        "sentinels": redis_settings.sentinels,
        "sentinel_kwargs": {},
        "password": redis_settings.password,
        "db": redis_settings.db,
        "service_name": redis_settings.master_name,
        "retry_on_timeout": True,
        "socket_timeout": 1,
        "socket_connect_timeout": 1,
        "socket_keepalive": True,
        "socket_keepalive_options": {socket.TCP_KEEPCNT: 5, socket.TCP_KEEPIDLE: 60, socket.TCP_KEEPINTVL: 10},
        "retry_period": 10,
    }

debug output would look like that

Configuration ->
    . broker -> sentinel://:**@redis-sentinel:26379/0
    . loader -> celery.loaders.default.Loader
    . scheduler -> redbeat.schedulers.RedBeatScheduler
       . redis -> redis-sentinel://:**@redis-sentinel:26379/0;redis-sentinel://:redis@redis-sentinel2:26379/0;redis-sentinel://:redis@redis-sentinel3:26379/0
       . lock -> `redbeat::lock` 25.00 minutes (1500s)
    . logfile -> [stderr]@%WARNING
    . maxinterval -> 5.00 minutes (300s)