What is the best way to manage a ruleset with more than 1000 rules?

[restena-imo]

Hello,

I'm working to create a set of Ansible tasks to manage the policy about more than 1000 rules.
I want to have the rules described in the inventory with the following structure. So all the rules combined in 1 inventory file.

- description: My Description
  enabled: true
  interface:
  - opt1
  source_net: any
  destination_net: 1.2.3.4/32
  destination_port: 53
  source_port: any
  protocol: UDP
  ip_protocol: inet
  action: pass
  log: true

My Ansible seems no to be really optimised. When I try from scratch to all all the rules, it takes more than 30min. So not really ready for a production environment.
Please note that I'm using the _multi modules and the last version 25.7.8 of the collection and OPNsense 25.7.2.
So I probably need to review it or to review my process.

Does someone is managing his firewall policy with Ansible ?

Thank you for you feedbacks,

Have a nice day,

[superstes]

Thank you for the info.
I'll have to add tests for that amount of rules - I personally have not yet tested it with so many.

[superstes]

FYI: I've added a ticket for it: #369

[restena-imo]

Hi Superstes,
Thank you for taking in consideration my post.
By the way, are you using Ansible to manage Opnsense firewall rules in a daily basis? I yes, could you share your approach?
I can share my approach as well....
As I mentioned, I've put all the rules into a single inventory file based on the structure I put above.
then, in term of Ansible tasks I'm doing the following steps:

1. Pulling the running policy with oxlorg.opnsense.list and set a fact
2. Build the candidate policy (based on the inventory) add the sequence number calculated from the position rule within the inventory and set a fact.
3. Compare the running_policy and the candidate_policy to know what to do: add, modify or delete rule(s)

What I'm not really enthusiast is when a rule must be removed or added, my ansible re-calculate the sequence number of all next rules, purge and re add them. Let me put a quick example bellow just to be clear.

Let's imagine I've the following running policy and I need to remove the rule C.
1 - rule A
2 - rule B
3 - rule C
4 - rule D
5 - rule E

So the candidate policy is as follow (rule C removed and rule D and E sequence number recalculted)
1 - rule A
2 - rule B
3 - rule D
4 - rule E

In that case, my Ansible is going to execute a purge task with oxlorg.opnsense.rule_multi to remove rule C, D and E.
And then execute and adding task (also with _multi) to add rule D and E with new sequence number. And finally execute a last task to apply all changes.

So with more that 1000 rules it could be a really a long time to just remove 1 rule due to the fact that my Ansible re-add a lot of rules due to the change of the sequence number.

Probably my approach is not really the most efficient one.

Thank you for sharing your point of view..

[superstes]

• Currently I have no setup where I can/need-to manage that many rules - no.
  But I am managing other Firewalls like NFTables via Ansible - where I got a few hundred rules. But there we do not have to care about single rules as we can just write a whole config-file.. that simplifies the logic a lot.

• I have to create a simulated setup that allows me to tackle these challenges .
  I will be reporting back one I found time for it.

• As mentioned in Problem: Improve performance for Rule Mass-Management #369 I know that the module's internal logic is not yet optimized for speed. I plan to add asynchronous API-calls where required to speed-up processing.

• What do you use as match_fields? These are the fields that the Ansible-Module uses to match a configured rule with an existing one.
  As an example: When you set it to match_fields=['description'] you should be able to set a unique rule-name in the description-field (that cannot be changed later on..) which allows you to:

  • Modify a rule-sequence-number without deleting and re-adding a rule
  • Thus delete a rule without purging others

[restena-imo]

Hi,

For the match_fields, we use a combination of description and sequence_number.

When you say:
Modify a rule-sequence-number without deleting and re-adding a rule
Thus delete a rule without purging others
Does it mean that if we change a rule-sequence-number, the other rules are automatically updated ?

[superstes]

As the API is not designed for mass-management, we can only do so much..