Clustering - Job running always on 1 instance

[s3YwCf2ZbfJG4SHAfjQMAjtsf]

V3.5.0
Same machine, 2 sites.
Instance Name the same
Instance Id shown from sql statement below.
Cluster with 2 instances
1 job shows in the Crystal Dashboard properly. (Once per minute execution)
Clustering enabled
Disallow concurrency

Issue: I was hoping it would switch between instances but it seems like the job is always running on 1 instance.

(Separate problem) According to my logs, after restarting the sites over a few minutes, it seems like the job was running on both sites.

Thanks for any suggestions. Please let me know if additional information is needed.

SCHEDULER_STATE table:

Scheduler start code:

                _scheduler = await SchedulerBuilder.Create(properties)
                    .WithId(SchedulerInstanceId)
                    .WithName(SchedulerGroupName)
                    .UseDefaultThreadPool(x => x.MaxConcurrency = 1)
                    .UsePersistentStore(x =>
                    {
                        x.UseProperties = true;
                        x.UseClustering();
                        x.UseOracle(proper connection);
                        x.UseJsonSerializer();
                    })
                    .BuildScheduler();

                await _scheduler.Start();

Single Job:

            var details = JobBuilder.Create<MainJobHandler>()
                          .WithIdentity(name, JobGroup)
                          .WithDescription(name)
                          .DisallowConcurrentExecution(true)
                          .Build();

                ITrigger trigger = TriggerBuilder.Create()
                                    .WithIdentity(triggerData.JobName, JobGroup)
                                    .StartAt(triggerData.StartOn)
                                    .WithDescription($"Executing {triggerData.JobName} Notification")
                                    .WithCronSchedule("0 0/1 * 1/1 * ? *", action =>
                                    {
                                        action.InTimeZone(triggerData.tzInfo);
                                        if (triggerData.MisfireHandlingDoNothing)
                                        {
                                            action.WithMisfireHandlingInstructionDoNothing();
                                        }
                                    })
                                    .Build();

[aleksandar-angelov]

As written in the Quartz documentation, the node that will fire the trigger is quite random.

Load-balancing occurs automatically, with each node of the cluster firing jobs as quickly as it can. When a trigger’s firing time occurs, the first node to acquire it (by placing a lock on it) is the node that will fire it.
The load balancing mechanism is near-random for busy schedulers (lots of triggers) but favors the same node for non-busy (e.g. few triggers) schedulers.

https://www.quartz-scheduler.net/documentation/quartz-3.x/configuration/reference.html#clustering

[s3YwCf2ZbfJG4SHAfjQMAjtsf]

Thanks for the clarification. It sounds like my implementation is appropriate but clustering works as clustering works. I can live with that.