Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dynamically adjust LiveSim resources according to namespace #610

Closed
tanyaveksler opened this issue Sep 26, 2023 · 1 comment
Closed

Dynamically adjust LiveSim resources according to namespace #610

tanyaveksler opened this issue Sep 26, 2023 · 1 comment
Labels
enhancement New feature or request Istio Issues related to Istio support

Comments

@tanyaveksler
Copy link
Member

Currently, whenever found a missing resource (ingress controller pods, ingress/egress gateway pods), LiveSim adds previously prepared resources (which are hardcoded). Change resource parameters (e.g., namespace) according to what is expected in the current example.

@tanyaveksler tanyaveksler added enhancement New feature or request Istio Issues related to Istio support labels Sep 26, 2023
tanyaveksler added a commit that referenced this issue Oct 3, 2023
…ned with livesim, until #610 is implemented).

Extending IngressPolicy to both directions (ingress/egress)
Splitting livesim ingress/egress gateway resources to separated files.

Signed-off-by: Tanya <[email protected]>
tanyaveksler added a commit that referenced this issue Oct 22, 2023
…ned with livesim, until #610 is implemented).

Extending IngressPolicy to both directions (ingress/egress)
Splitting livesim ingress/egress gateway resources to separated files.

Signed-off-by: Tanya <[email protected]>
tanyaveksler added a commit that referenced this issue Nov 12, 2023
* Bug fix in livesim: Changed missing pods labels dictionary to set of pairs (key, value) to allow missing pods having the same label key but different label values.
Bug fix in IngressNetworkLayer._allowed_xgress_conns_optimized: all_allowed_conns should always include allowed_conns.

Signed-off-by: Tanya <[email protected]>

* Small fixes and preparations for implementing istio egress gateway.
Bug fix: considering DNS entries when computing allowed connections in the Ingress layer.
Removing duplicates from the list of missing livesim resources.

Signed-off-by: Tanya <[email protected]>

* More fixes of handling DNS entries in optimized solution.

Signed-off-by: Tanya <[email protected]>

* Uppdating expected results of some livesim tests following the addition of istio-egressgateway resource to livesim, and changing the namespace of istio-ingressgateway to istio-system.

Signed-off-by: Tanya <[email protected]>

* Small fixes.

Signed-off-by: Tanya <[email protected]>

* Referencing istio ingress gateway as istio:ingressgateway (to be aligned with livesim, until #610 is implemented).
Extending IngressPolicy to both directions (ingress/egress)
Splitting livesim ingress/egress gateway resources to separated files.

Signed-off-by: Tanya <[email protected]>

* Changing expected results of some istio ingress/egress gateway tests according to the change in livesim implementation of these gateways.
Temporarily commenting out other expected results, until ingress/egress gateway implementation is completes.

Signed-off-by: Tanya <[email protected]>

* Changing expected results of some istio ingress/egress gateway tests according to the change in livesim implementation of these gateways.

Signed-off-by: Tanya <[email protected]>

* Renamed IngressPolicy to IstioGatewayPolicy.

Signed-off-by: Tanya <[email protected]>

* Renamed IstioGatewayPolicy to GatewayPolicy.

Signed-off-by: Tanya <[email protected]>

* Update nca/Resources/GatewayPolicy.py

Co-authored-by: Adi Sosnovich <[email protected]>

* Update nca/NetworkConfig/NetworkConfigQuery.py

Co-authored-by: Adi Sosnovich <[email protected]>

* Renamed Ingress layer to Gateway layer.

Signed-off-by: Tanya <[email protected]>

---------

Signed-off-by: Tanya <[email protected]>
Co-authored-by: Adi Sosnovich <[email protected]>
tanyaveksler added a commit that referenced this issue Dec 31, 2023
* Bug fix in livesim: Changed missing pods labels dictionary to set of pairs (key, value) to allow missing pods having the same label key but different label values.
Bug fix in IngressNetworkLayer._allowed_xgress_conns_optimized: all_allowed_conns should always include allowed_conns.

Signed-off-by: Tanya <[email protected]>

* Small fixes and preparations for implementing istio egress gateway.
Bug fix: considering DNS entries when computing allowed connections in the Ingress layer.
Removing duplicates from the list of missing livesim resources.

Signed-off-by: Tanya <[email protected]>

* More fixes of handling DNS entries in optimized solution.

Signed-off-by: Tanya <[email protected]>

* Uppdating expected results of some livesim tests following the addition of istio-egressgateway resource to livesim, and changing the namespace of istio-ingressgateway to istio-system.

Signed-off-by: Tanya <[email protected]>

* The second part of supporting istio egress gateways: parsing and building policy rules.

Signed-off-by: Tanya <[email protected]>

* Fixed lint error

Signed-off-by: Tanya <[email protected]>

* Small fixes.

Signed-off-by: Tanya <[email protected]>

* Referencing istio ingress gateway as istio:ingressgateway (to be aligned with livesim, until #610 is implemented).
Extending IngressPolicy to both directions (ingress/egress)
Splitting livesim ingress/egress gateway resources to separated files.

Signed-off-by: Tanya <[email protected]>

* More concise implementation of ingress/egress gateways.

Signed-off-by: Tanya <[email protected]>

* Changing expected results of some istio ingress/egress gateway tests according to the change in livesim implementation of these gateways.
Temporarily commenting out other expected results, until ingress/egress gateway implementation is completes.

Signed-off-by: Tanya <[email protected]>

* Changing expected results of some istio ingress/egress gateway tests according to the change in livesim implementation of these gateways.

Signed-off-by: Tanya <[email protected]>

* Change in test: changing istio ingress gateway namespace to istio-system

Signed-off-by: Tanya <[email protected]>

* Renamed IngressPolicy to IstioGatewayPolicy.

Signed-off-by: Tanya <[email protected]>

* Fixing lint errors

Signed-off-by: Tanya <[email protected]>

* Generalized istio ingress/egress gateway policies creation.

Updated istio-ingress-test and its expected results.

Signed-off-by: Tanya <[email protected]>

* Added egress gateway test.
Renamed Ingress policy kind and Ingress layer name to IngressEgressGateway policy type and layer name.
Renamed IngressNetworkLayer to GatewayLayer.
Changed creation of deny policy - a single deny policy with one rule.
Handling denied connections in GatewayLayer; using captured_cond_func (to avoid denied connections affect captured).

Signed-off-by: Tanya <[email protected]>

* Differentiating between Ingress policy and IstioGatewayPolicy.
Fixing lint errors.

Signed-off-by: Tanya <[email protected]>

* Added new test runtime to expected runtimes.

Signed-off-by: Tanya <[email protected]>

* Added new test runtime to expected runtimes.

Signed-off-by: Tanya <[email protected]>

* Removed unneeded namespace attribute in parser.

Signed-off-by: Tanya <[email protected]>

* Split Gateway and VirtualService resources to separate files.

Signed-off-by: Tanya <[email protected]>

* Fixed lint errors.

Signed-off-by: Tanya <[email protected]>

* Split IstioTrafficResourcesYamlParser to Gateway and VirtualService parsers.

Signed-off-by: Tanya <[email protected]>

* Fixed lint errors.

Signed-off-by: Tanya <[email protected]>

* Small fix.

Signed-off-by: Tanya <[email protected]>

* Fixed lint errors.

Signed-off-by: Tanya <[email protected]>

* Added warning about ignoring local (mesh-to-mesh) traffic defined by VirtualService.

Signed-off-by: Tanya <[email protected]>

* Fixed lint errors.

Signed-off-by: Tanya <[email protected]>

* Updated expected number of warnings following the edition of more warning messages.

Signed-off-by: Tanya <[email protected]>

* Updated documentation.

Signed-off-by: Tanya <[email protected]>

* Extended istio-egress test and updated its expected results accordingly.
Updated documentation.
Refined deny rule - denying only connections to DNS entries mentioned in the virtual services.
Refined calculation of egress pods - not assuming only one egress gateway. Calculating egress pods out of currently processed gateway.

Signed-off-by: Tanya <[email protected]>

* Split Resources folder to PolicyResources and SpecialResources.
Improved documentation.
Removed unused imports.

Signed-off-by: Tanya <[email protected]>

* Fixed lint errors.

Signed-off-by: Tanya <[email protected]>

* Separated IstioGatewayPolicyGenerator into a separate file.

Signed-off-by: Tanya <[email protected]>

* Fixed lint error.

Signed-off-by: Tanya <[email protected]>

* Renamed SpecialResources folder to OtherResources.

Signed-off-by: Tanya <[email protected]>

* Supporting only standard istio ingress/egress gateways, which are identified by 'istio: ingressgateway' and 'istio: egressgateway' selectors, and egress gateway destination is identified by 'istio-egressgateway.istio-system.svc.cluster.local' host.

Signed-off-by: Tanya <[email protected]>

* Supporting only standard istio ingress/egress gateways, which are identified by 'istio: ingressgateway' and 'istio: egressgateway' selectors, and egress gateway destination is identified by 'istio-egressgateway.istio-system.svc.cluster.local' host.
Updated relevant tests and expected results correspondingly.

Signed-off-by: Tanya <[email protected]>

* Added a combined test of sidecars together with egress gateway.

Signed-off-by: Tanya <[email protected]>

* Added a combined test of sidecars together with egress gateway.

Signed-off-by: Tanya <[email protected]>

* Added more explanation comments.

Signed-off-by: Tanya <[email protected]>

* Split GatewayLayer into two layers: K8sGatewayLayer and IstioGatewayLayer. In IstioGatewayLayer, splitting TCP and non-TCP connections, similarly to IstioLayer.
Updated expected results correspondingly.

Signed-off-by: Tanya <[email protected]>

* Fixed lint errors.

Signed-off-by: Tanya <[email protected]>

* Updated expected results, after splitting TCP and non-TCP connections in Istio gateway connectivity output.

Signed-off-by: Tanya <[email protected]>

* Updated expected results, after splitting TCP and non-TCP connections in Istio gateway connectivity output.

Signed-off-by: Tanya <[email protected]>

* Avoid blocking connections to DNS entries in K8s and Calico.
Added multilayer test with Istio gateways.

Signed-off-by: Tanya <[email protected]>

* Update nca/Resources/PolicyResources/GatewayPolicy.py

Co-authored-by: Adi Sosnovich <[email protected]>

* Update nca/Parsers/IstioVirtualServiceYamlParser.py

Co-authored-by: Adi Sosnovich <[email protected]>

* Update nca/Parsers/IstioVirtualServiceYamlParser.py

Co-authored-by: Adi Sosnovich <[email protected]>

* Update nca/Parsers/IstioVirtualServiceYamlParser.py

Co-authored-by: Adi Sosnovich <[email protected]>

* Renamed GenericIngressLikeYamlParser to GenericGatewayYamlParser.

Signed-off-by: Tanya <[email protected]>

* Fixed indentation error.

Signed-off-by: Tanya <[email protected]>

* Fixed lint noise.

Signed-off-by: Tanya <[email protected]>

* Added comment.

Signed-off-by: Tanya <[email protected]>

* Update nca/Parsers/IstioVirtualServiceYamlParser.py

Co-authored-by: Adi Sosnovich <[email protected]>

* Update nca/Parsers/IstioVirtualServiceYamlParser.py

Co-authored-by: Adi Sosnovich <[email protected]>

* Update nca/Parsers/IstioVirtualServiceYamlParser.py

Co-authored-by: Adi Sosnovich <[email protected]>

* Update nca/Parsers/IstioVirtualServiceYamlParser.py

Co-authored-by: Adi Sosnovich <[email protected]>

* Update nca/Resources/OtherResources/VirtualService.py

Co-authored-by: Adi Sosnovich <[email protected]>

* Update nca/Resources/OtherResources/VirtualService.py

Co-authored-by: Adi Sosnovich <[email protected]>

* Update nca/Parsers/IstioGatewayYamlParser.py

Co-authored-by: Adi Sosnovich <[email protected]>

* Update nca/Parsers/IstioGatewayYamlParser.py

Co-authored-by: Adi Sosnovich <[email protected]>

* Update nca/Parsers/IstioGatewayYamlParser.py

Co-authored-by: Adi Sosnovich <[email protected]>

* Update nca/Parsers/IstioGatewayPolicyGenerator.py

Co-authored-by: Adi Sosnovich <[email protected]>

* Update nca/Parsers/IstioGatewayPolicyGenerator.py

Co-authored-by: Adi Sosnovich <[email protected]>

* Update nca/Parsers/IstioGatewayPolicyGenerator.py

Co-authored-by: Adi Sosnovich <[email protected]>

* Update nca/Parsers/IstioGatewayPolicyGenerator.py

Co-authored-by: Adi Sosnovich <[email protected]>

* Update nca/Parsers/IstioGatewayPolicyGenerator.py

Co-authored-by: Adi Sosnovich <[email protected]>

* Update nca/Parsers/IstioGatewayPolicyGenerator.py

Co-authored-by: Adi Sosnovich <[email protected]>

* Update nca/Parsers/IstioGatewayPolicyGenerator.py

Co-authored-by: Adi Sosnovich <[email protected]>

* Update nca/Parsers/IstioGatewayPolicyGenerator.py

Co-authored-by: Adi Sosnovich <[email protected]>

* Added more documentation and in-code comments.

Signed-off-by: Tanya <[email protected]>

* Fixed lint errors.

Signed-off-by: Tanya <[email protected]>

* Fixed printing warning for unused routes.

Signed-off-by: Tanya <[email protected]>

* Small fixes.

Signed-off-by: Tanya <[email protected]>

* Added a check that both parts of egress flow are present.

Signed-off-by: Tanya <[email protected]>

* Added two tests checking partial egress flow.
Adding empty gateway policy in case no policies were generated, in order to keep findings.

Signed-off-by: Tanya <[email protected]>

* Added expected runtime for the new tests.

Signed-off-by: Tanya <[email protected]>

* Added expected runtime for the new tests.

Signed-off-by: Tanya <[email protected]>

* Added expected runtime for the new tests.

Signed-off-by: Tanya <[email protected]>

* Made more detailed warning message about missing part of egress flow

Signed-off-by: Tanya <[email protected]>

* Avoid printing redundant warning.

Signed-off-by: Tanya <[email protected]>

---------

Signed-off-by: Tanya <[email protected]>
Co-authored-by: Adi Sosnovich <[email protected]>
tanyaveksler added a commit that referenced this issue Jan 2, 2024
* Bug fix in livesim: Changed missing pods labels dictionary to set of pairs (key, value) to allow missing pods having the same label key but different label values.
Bug fix in IngressNetworkLayer._allowed_xgress_conns_optimized: all_allowed_conns should always include allowed_conns.

Signed-off-by: Tanya <[email protected]>

* Small fixes and preparations for implementing istio egress gateway.
Bug fix: considering DNS entries when computing allowed connections in the Ingress layer.
Removing duplicates from the list of missing livesim resources.

Signed-off-by: Tanya <[email protected]>

* More fixes of handling DNS entries in optimized solution.

Signed-off-by: Tanya <[email protected]>

* Uppdating expected results of some livesim tests following the addition of istio-egressgateway resource to livesim, and changing the namespace of istio-ingressgateway to istio-system.

Signed-off-by: Tanya <[email protected]>

* The second part of supporting istio egress gateways: parsing and building policy rules.

Signed-off-by: Tanya <[email protected]>

* Fixed lint error

Signed-off-by: Tanya <[email protected]>

* Small fixes.

Signed-off-by: Tanya <[email protected]>

* Referencing istio ingress gateway as istio:ingressgateway (to be aligned with livesim, until #610 is implemented).
Extending IngressPolicy to both directions (ingress/egress)
Splitting livesim ingress/egress gateway resources to separated files.

Signed-off-by: Tanya <[email protected]>

* More concise implementation of ingress/egress gateways.

Signed-off-by: Tanya <[email protected]>

* Changing expected results of some istio ingress/egress gateway tests according to the change in livesim implementation of these gateways.
Temporarily commenting out other expected results, until ingress/egress gateway implementation is completes.

Signed-off-by: Tanya <[email protected]>

* Changing expected results of some istio ingress/egress gateway tests according to the change in livesim implementation of these gateways.

Signed-off-by: Tanya <[email protected]>

* Change in test: changing istio ingress gateway namespace to istio-system

Signed-off-by: Tanya <[email protected]>

* Renamed IngressPolicy to IstioGatewayPolicy.

Signed-off-by: Tanya <[email protected]>

* Fixing lint errors

Signed-off-by: Tanya <[email protected]>

* Generalized istio ingress/egress gateway policies creation.

Updated istio-ingress-test and its expected results.

Signed-off-by: Tanya <[email protected]>

* Added egress gateway test.
Renamed Ingress policy kind and Ingress layer name to IngressEgressGateway policy type and layer name.
Renamed IngressNetworkLayer to GatewayLayer.
Changed creation of deny policy - a single deny policy with one rule.
Handling denied connections in GatewayLayer; using captured_cond_func (to avoid denied connections affect captured).

Signed-off-by: Tanya <[email protected]>

* Differentiating between Ingress policy and IstioGatewayPolicy.
Fixing lint errors.

Signed-off-by: Tanya <[email protected]>

* Added new test runtime to expected runtimes.

Signed-off-by: Tanya <[email protected]>

* Added new test runtime to expected runtimes.

Signed-off-by: Tanya <[email protected]>

* Removed unneeded namespace attribute in parser.

Signed-off-by: Tanya <[email protected]>

* Split Gateway and VirtualService resources to separate files.

Signed-off-by: Tanya <[email protected]>

* Fixed lint errors.

Signed-off-by: Tanya <[email protected]>

* Split IstioTrafficResourcesYamlParser to Gateway and VirtualService parsers.

Signed-off-by: Tanya <[email protected]>

* Fixed lint errors.

Signed-off-by: Tanya <[email protected]>

* Small fix.

Signed-off-by: Tanya <[email protected]>

* Fixed lint errors.

Signed-off-by: Tanya <[email protected]>

* Added warning about ignoring local (mesh-to-mesh) traffic defined by VirtualService.

Signed-off-by: Tanya <[email protected]>

* Fixed lint errors.

Signed-off-by: Tanya <[email protected]>

* Updated expected number of warnings following the edition of more warning messages.

Signed-off-by: Tanya <[email protected]>

* Updated documentation.

Signed-off-by: Tanya <[email protected]>

* Extended istio-egress test and updated its expected results accordingly.
Updated documentation.
Refined deny rule - denying only connections to DNS entries mentioned in the virtual services.
Refined calculation of egress pods - not assuming only one egress gateway. Calculating egress pods out of currently processed gateway.

Signed-off-by: Tanya <[email protected]>

* Split Resources folder to PolicyResources and SpecialResources.
Improved documentation.
Removed unused imports.

Signed-off-by: Tanya <[email protected]>

* Fixed lint errors.

Signed-off-by: Tanya <[email protected]>

* Separated IstioGatewayPolicyGenerator into a separate file.

Signed-off-by: Tanya <[email protected]>

* Fixed lint error.

Signed-off-by: Tanya <[email protected]>

* Renamed SpecialResources folder to OtherResources.

Signed-off-by: Tanya <[email protected]>

* Supporting only standard istio ingress/egress gateways, which are identified by 'istio: ingressgateway' and 'istio: egressgateway' selectors, and egress gateway destination is identified by 'istio-egressgateway.istio-system.svc.cluster.local' host.

Signed-off-by: Tanya <[email protected]>

* Supporting only standard istio ingress/egress gateways, which are identified by 'istio: ingressgateway' and 'istio: egressgateway' selectors, and egress gateway destination is identified by 'istio-egressgateway.istio-system.svc.cluster.local' host.
Updated relevant tests and expected results correspondingly.

Signed-off-by: Tanya <[email protected]>

* Added a combined test of sidecars together with egress gateway.

Signed-off-by: Tanya <[email protected]>

* Added a combined test of sidecars together with egress gateway.

Signed-off-by: Tanya <[email protected]>

* Added more explanation comments.

Signed-off-by: Tanya <[email protected]>

* Split GatewayLayer into two layers: K8sGatewayLayer and IstioGatewayLayer. In IstioGatewayLayer, splitting TCP and non-TCP connections, similarly to IstioLayer.
Updated expected results correspondingly.

Signed-off-by: Tanya <[email protected]>

* Fixed lint errors.

Signed-off-by: Tanya <[email protected]>

* Updated expected results, after splitting TCP and non-TCP connections in Istio gateway connectivity output.

Signed-off-by: Tanya <[email protected]>

* Updated expected results, after splitting TCP and non-TCP connections in Istio gateway connectivity output.

Signed-off-by: Tanya <[email protected]>

* Avoid blocking connections to DNS entries in K8s and Calico.
Added multilayer test with Istio gateways.

Signed-off-by: Tanya <[email protected]>

* Update nca/Resources/PolicyResources/GatewayPolicy.py

Co-authored-by: Adi Sosnovich <[email protected]>

* Update nca/Parsers/IstioVirtualServiceYamlParser.py

Co-authored-by: Adi Sosnovich <[email protected]>

* Update nca/Parsers/IstioVirtualServiceYamlParser.py

Co-authored-by: Adi Sosnovich <[email protected]>

* Update nca/Parsers/IstioVirtualServiceYamlParser.py

Co-authored-by: Adi Sosnovich <[email protected]>

* Renamed GenericIngressLikeYamlParser to GenericGatewayYamlParser.

Signed-off-by: Tanya <[email protected]>

* Fixed indentation error.

Signed-off-by: Tanya <[email protected]>

* Fixed lint noise.

Signed-off-by: Tanya <[email protected]>

* Added comment.

Signed-off-by: Tanya <[email protected]>

* Added istio traffic resources to readme documentation

Signed-off-by: Tanya <[email protected]>

* Update docs/IstioSupport.md

Co-authored-by: Adi Sosnovich <[email protected]>

---------

Signed-off-by: Tanya <[email protected]>
Co-authored-by: Adi Sosnovich <[email protected]>
@zivnevo
Copy link
Member

zivnevo commented Aug 1, 2024

Moved to NP-Guard repo. See here

@zivnevo zivnevo closed this as not planned Won't fix, can't repro, duplicate, stale Aug 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Istio Issues related to Istio support
Projects
None yet
Development

No branches or pull requests

2 participants