Html out #531

shmfr · 2023-06-14T09:18:51Z

No description provided.

…ubeSet) holding all connections including src_peers, dst_peers and protocols Signed-off-by: Tanya <[email protected]>

Signed-off-by: Tanya <[email protected]>

Added optimized_denied_ingress_props and optimized_denied_egress_props (in addition to allowed ones). Improved non_captured_conns computation Signed-off-by: Tanya <[email protected]>

Signed-off-by: Tanya <[email protected]>

…_HC_set # Conflicts: # nca/CoreDS/CanonicalHyperCubeSet.py

Signed-off-by: Tanya <[email protected]>

Signed-off-by: Tanya <[email protected]> Signed-off-by: Tanya <[email protected]>

Signed-off-by: Tanya <[email protected]>

Fixed protocol handling: 0 is a valid protocol number (HOPOPT). Allowing any protocol in the range [0...255], though ProtocolNameResolver does not contain names of all of the possible 256 protocols. Fixed handling non-captured peers in K8S (cannot be handled as denied). Signed-off-by: Tanya <[email protected]>

…r dst_peers dimensions is all values). Added optimization for fw_rules_map - join different entries having the same values (fw_rules). Signed-off-by: Tanya <[email protected]>

1. To represent No connections, do not build TcpLikeProperties (TcpLikeProperties with no dimensions represent All connections). 2. Support subsets in query in optimized solution. 3. For comparison of optimized solution to the original one, add connections from peers to themselves. 4. More accurate comparison for 'dot' connectivity queries. 5. Generalized convert_named_ports (to not assume dimensions order) 6. Handling the possibility when projection on one dimension is empty. Signed-off-by: Tanya <[email protected]>

Properly handling HostEPs in optimized TcpLikeProperties. Signed-off-by: Tanya <[email protected]>

…perties) Fixed handling of non-captured pods in optimized solution. Added using True/False HC_set (make_all_properties()/make_empty_properties()) Signed-off-by: Tanya <[email protected]>

'false' - only original run 'true' - only optimized run 'debug'- both runs and comparison of their results. Printing parsing time, queries time and total run time. Signed-off-by: Tanya <[email protected]>

Workaround for the bug in HC set: using mutual contained_in, instead of == Signed-off-by: Tanya <[email protected]>

Signed-off-by: Tanya <[email protected]>

Signed-off-by: Tanya <[email protected]> Signed-off-by: Tanya <[email protected]>

…for -opt=debug option) Added more debug prints. Better handling of peer_set copying in TcpLikeProperties. Signed-off-by: Tanya <[email protected]>

Further optimization - calculating ref_ip_blocks only in non-optimized run. Signed-off-by: Tanya <[email protected]>

Improved comments. Signed-off-by: Tanya <[email protected]>

Signed-off-by: Tanya <[email protected]>

Signed-off-by: Shmulik Froimovich <[email protected]>

…ml-out

adisos

added a few initial comments.

add basic tests with output of svg file

adisos · 2023-06-26T08:54:06Z

nca/nca_cli.py

@@ -321,7 +325,7 @@ def nca_main(argv=None):
 help='A list of labels to subset the query by')
 parser.add_argument('--ghe_token', '--gh_token', type=str, help='A valid token to access a GitHub repository')
 parser.add_argument('--output_format', '-o', type=str,
- help='Output format specification (txt, txt_no_fw_rules, csv, md, dot, jpg or yaml). '
+ help='Output format specification (txt, txt_no_fw_rules, csv, md, dot, jpg, html or yaml). '


add the new html format in the relevant readme files, possibly also include some explanation (and also assumption that graphviz is installed?)

adisos · 2023-06-26T09:01:05Z

nca/FWRules/InteractiveConnectivityGraph.py

@@ -327,6 +393,218 @@ def create_output(self, elements_relations):
 self._highlight_tag(related_tag, related_tag_info.t_class)
 self._save_tag_file(tag_soup, tag_info)

+ HTML_TEMPLATE = '''


can this template be saved in a separate file?

Signed-off-by: Shmulik Froimovich <[email protected]>

…_HCS

Signed-off-by: Shmulik Froimovich <[email protected]>

adisos · 2023-06-26T10:04:51Z

nca/FWRules/InteractiveConnectivityGraph.py

@@ -59,13 +60,13 @@ class ElementRelations:
 highlights: set = field(default_factory=set)
 explanation: list = field(default_factory=set)

- def __init__(self, svg_file_name, output_directory):
+ def __init__(self, svg_file_name, output_directory, expl_xml=None):


add the new arg to documentation.

adisos · 2023-07-24T11:38:58Z

nca/Utils/OutputConfiguration.py

@@ -62,6 +64,23 @@ def print_query_output(self, output, supported_output_formats=None):
 print(f'Command {dot_cmd_string}\n did not create {path}\n', file=sys.stderr)
 if os.path.isfile(tmp_dot_file):
 os.remove(tmp_dot_file)
+ elif self.outputFormat == 'html':


trying to run the following command from cli:
python -m nca --connectivity -r tests/fw_rules_tests/podlist/ns_list.json -r tests/fw_rules_tests/podlist/pods_list_4.json -r tests/fw_rules_tests/policies/test2-networkpolicy.yaml -o html -f test2.html

but getting:

Failed to create a svg file: test2.html 'NoneType' object is not iterable Command dot test2.html.nca_tmp.dot -Tsvg -otest2.html.nca_tmp.svg did not create test2.html

I do have dot installed on my env

we should avoid the error 'NoneType' object is not iterable even if dot is not installed

Signed-off-by: haim-kermany <[email protected]>

nca/Parsers/IstioSidecarYamlParser.py

 if alphabet_str:
- fqdn_regex = r"(?=.{1,254}$)[A-Za-z0-9]([-A-Za-z0-9]*[A-Za-z0-9])?(\.[A-Za-z0-9]([-A-Za-z0-9]*[A-Za-z0-9])?)*[.]?"
+ fqdn_regex = "^((?!-)[A-Za-z0-9-]+(?<!-).)+[A-Za-z0-9.]+"


tanyaveksler added 30 commits November 15, 2022 17:42

Initial implementation of the optimized TcpLikeProperties (and HyperC…

74ec605

…ubeSet) holding all connections including src_peers, dst_peers and protocols Signed-off-by: Tanya <[email protected]>

Extended testcase3 to produce connectivity_map

c279de6

Signed-off-by: Tanya <[email protected]>

Fixed a problem in HyperCubeSet (wrongly changing self in _and_aux.

1b1ebbe

Added optimized_denied_ingress_props and optimized_denied_egress_props (in addition to allowed ones). Improved non_captured_conns computation Signed-off-by: Tanya <[email protected]>

Small fix

09cfad1

Signed-off-by: Tanya <[email protected]>

Further fix of the hyper cube set

94fb117

Signed-off-by: Tanya <[email protected]>

Further fix of the hyper cube set

520377a

Signed-off-by: Tanya <[email protected]>

Merge remote-tracking branch 'origin/Optimized_HC_set' into Optimized…

af37b10

…_HC_set # Conflicts: # nca/CoreDS/CanonicalHyperCubeSet.py

Avoiding redundant and heavy copy of layers.

0339215

Signed-off-by: Tanya <[email protected]>

Merge branch 'master' into Optimized_HC_set

564d667

General changes from the Optimized_HC_set branch.

b9810b3

Signed-off-by: Tanya <[email protected]>

General changes from the Optimized_HC_set branch.

e82ac99

Signed-off-by: Tanya <[email protected]>

Avoiding lint warnings

6b3b364

Signed-off-by: Tanya <[email protected]> Signed-off-by: Tanya <[email protected]>

Avoiding lint warnings

cfb5ee7

Signed-off-by: Tanya <[email protected]> Signed-off-by: Tanya <[email protected]>

Merge branch 'Small-fixes-for-master' into Optimized_HC_set

01a9d40

Added support of IpBlocks in optimized hyper cube set implementation.

28e211b

Signed-off-by: Tanya <[email protected]>

More accurate update of base_peer_set.

d5d97a6

Signed-off-by: Tanya <[email protected]>

Added dedundant fw_rules creation for testing (to be further removed).

0a2cb45

Signed-off-by: Tanya <[email protected]>

Fixed the construction of connectivity graph (when some of src_peer o…

05a496c

…r dst_peers dimensions is all values). Added optimization for fw_rules_map - join different entries having the same values (fw_rules). Signed-off-by: Tanya <[email protected]>

Properly handling 'False' represented by TcpLikeProperties.

0d4bd28

Properly handling HostEPs in optimized TcpLikeProperties. Signed-off-by: Tanya <[email protected]>

Added support to ICMP data in optimized HC set (handled in TCPLikePro…

47d5bd0

…perties) Fixed handling of non-captured pods in optimized solution. Added using True/False HC_set (make_all_properties()/make_empty_properties()) Signed-off-by: Tanya <[email protected]>

Added command line flag optimized_run, having 3 possibilities:

0a8d53f

'false' - only original run 'true' - only optimized run 'debug'- both runs and comparison of their results. Printing parsing time, queries time and total run time. Signed-off-by: Tanya <[email protected]>

Small fixes;

183eaeb

Workaround for the bug in HC set: using mutual contained_in, instead of == Signed-off-by: Tanya <[email protected]>

Small fixes;

8e34573

Signed-off-by: Tanya <[email protected]>

Fixed building tcp_like_properties form connectivity graph

9957228

Signed-off-by: Tanya <[email protected]> Signed-off-by: Tanya <[email protected]>

Optimized the comparison between original and optimized connections (…

004742b

…for -opt=debug option) Added more debug prints. Better handling of peer_set copying in TcpLikeProperties. Signed-off-by: Tanya <[email protected]>

Implemented optimized Istio policy handling.

a27899d

Further optimization - calculating ref_ip_blocks only in non-optimized run. Signed-off-by: Tanya <[email protected]>

Added Ingress policy support in the optimized solution.

ebae572

Improved comments. Signed-off-by: Tanya <[email protected]>

Added Istio Ingress policy support in the optimized solution.

25118de

Signed-off-by: Tanya <[email protected]>

dot header fix

d8e0812

Signed-off-by: Shmulik Froimovich <[email protected]>

shmfr marked this pull request as ready for review June 18, 2023 12:33

shmfr requested a review from adisos June 18, 2023 12:33

shmfr added 8 commits June 20, 2023 14:04

removing explanation for ipblocks

c402d73

Signed-off-by: Shmulik Froimovich <[email protected]>

adding policies in one place

d835df7

Signed-off-by: Shmulik Froimovich <[email protected]>

adding some documentations and test

b7ceb53

Signed-off-by: Shmulik Froimovich <[email protected]>

after merge with master

895d75d

Signed-off-by: Shmulik Froimovich <[email protected]>

updated expected results

dea4134

Signed-off-by: Shmulik Froimovich <[email protected]>

fixed run_all_tests path problem

c4761fc

Signed-off-by: Shmulik Froimovich <[email protected]>

duplicate line removed

fd40627

Signed-off-by: Shmulik Froimovich <[email protected]>

Merge remote-tracking branch 'origin/explainability_with_HCS' into ht…

cef2195

…ml-out

adisos requested changes Jun 26, 2023

View reviewed changes

shmfr added 11 commits June 26, 2023 18:33

updated testes and some docs.

cdaa329

Signed-off-by: Shmulik Froimovich <[email protected]>

little better explanation

658811e

Signed-off-by: Shmulik Froimovich <[email protected]>

some small fixes

7c6ac51

Signed-off-by: Shmulik Froimovich <[email protected]>

base_name added

a468689

Signed-off-by: Shmulik Froimovich <[email protected]>

better documentation for using IP-blocks

224e43b

Signed-off-by: Shmulik Froimovich <[email protected]>

added a ip-block test

cae6397

Signed-off-by: Shmulik Froimovich <[email protected]>

base_name check

7dcfc67

Signed-off-by: Shmulik Froimovich <[email protected]>

base_name check

c33d88a

Signed-off-by: Shmulik Froimovich <[email protected]>

Merge remote-tracking branch 'origin/master' into explainability_with…

43f494b

…_HCS

base_name and sync

2a6e68f

Signed-off-by: Shmulik Froimovich <[email protected]>

after merge with expl branch

f4f7394

Signed-off-by: Shmulik Froimovich <[email protected]>

Base automatically changed from explainability_with_HCS to master July 6, 2023 06:40

adisos reviewed Jul 24, 2023

View reviewed changes

haim-kermany added 2 commits December 5, 2023 12:13

empty merge

0f6cec8

from master

e0b70dd

Signed-off-by: haim-kermany <[email protected]>

github-advanced-security bot found potential problems Dec 25, 2023

View reviewed changes

adisos closed this Jan 15, 2024

shmfr linked an issue Jan 15, 2024 that may be closed by this pull request

connect interactive map to NCA #456

Closed

This pull request was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Html out #531

Html out #531

shmfr commented Jun 14, 2023

adisos left a comment

adisos Jun 26, 2023

adisos Jun 26, 2023

adisos Jun 26, 2023

adisos Jul 24, 2023 •

edited

Loading

Html out #531

Html out #531

Conversation

shmfr commented Jun 14, 2023

adisos left a comment

Choose a reason for hiding this comment

adisos Jun 26, 2023

Choose a reason for hiding this comment

adisos Jun 26, 2023

Choose a reason for hiding this comment

adisos Jun 26, 2023

Choose a reason for hiding this comment

adisos Jul 24, 2023 • edited Loading

Choose a reason for hiding this comment

adisos Jul 24, 2023 •

edited

Loading