Cilium CNI make limits optional

[TlalkaM]

What would you like to be added

Variable/variables to disable adding cpu/memory under limits section or whole limit section in the daemonset yaml, eg.

resources:
{% if cilium_enable_cpu_limit or cilium_enable_memory_limit %}
  limits:
{% if cilium_enable_cpu_limit %}
    cpu: {{ cilium_cpu_limit }}
{% endif %}
{% if cilium_enable_memory_limit %}
    memory: {{ cilium_enable_memory_limit }}
{% endif %} 
{% endif %}

This can also be achieved by checking if the cilium_cpu_limit and cilium_memory_limit are defined and changing the default values to {{ undef() }}

Why is this needed

WIth nodes that hosts ~1k pods every node drain causes very high spike in cilium utilization of resources, setting high limits and wasting cpu cycles for limiting cpu for cilium pod seems like workaround :)