Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Smb hashmap/v5 #11786

Closed
wants to merge 5 commits into from
Closed

Smb hashmap/v5 #11786

wants to merge 5 commits into from

Conversation

victorjulien
Copy link
Member

@victorjulien victorjulien commented Sep 17, 2024

Adds use of the lru crate to turn guid2name_map into a LruCache. In this test code it's hard coded to 1024 entries max.

Continuation of #11721.

PR for initial LRU usage review and qa feedback.

Todos:

  • config for guid2name limit (1024 hardcoded now)
  • undo timestamp storage in guid2name
  • git cleanup
  • review comments
  • docs

Don't tag the session as gap'd when the GAP is in a precise location:

1. in "skip" data, where the GAP just fits the skip data

2. in file data, where we pass the GAP on to the file

This reduces load of GAP post-processing that is unnecessary in these
case.
@suricata-qa
Copy link

ERROR:

ERROR: QA failed on build_asan.

Pipeline 22678

Copy link

codecov bot commented Sep 17, 2024

Codecov Report

Attention: Patch coverage is 92.50000% with 18 lines in your changes missing coverage. Please review.

Project coverage is 82.55%. Comparing base (79aa486) to head (ef475aa).
Report is 125 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #11786      +/-   ##
==========================================
- Coverage   82.63%   82.55%   -0.08%     
==========================================
  Files         919      919              
  Lines      248943   248976      +33     
==========================================
- Hits       205716   205550     -166     
- Misses      43227    43426     +199     
Flag Coverage Δ
fuzzcorpus 60.35% <92.08%> (-0.51%) ⬇️
livemode 18.71% <0.83%> (-0.04%) ⬇️
pcap 44.12% <77.08%> (-0.05%) ⬇️
suricata-verify 61.84% <77.50%> (-0.03%) ⬇️
unittests 58.99% <14.58%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

@victorjulien victorjulien force-pushed the smb-hashmap/v5 branch 2 times, most recently from e1a935e to c12f6a5 Compare September 17, 2024 12:28
@victorjulien victorjulien mentioned this pull request Sep 17, 2024
@suricata-qa
Copy link

WARNING:

field baseline test %
SURI_TLPR1_stats_chk
.uptime 649 628 96.76%

Pipeline 22679

Use `lru` crate.

Add `app-layer.protocols.smb.max-guid-cache-size` to control the max
size of the LRU cache.
@suricata-qa
Copy link

WARNING:

field baseline test %
SURI_TLPW2_autofp_stats_chk
.flow.end.tcp_state.last_ack 0 1 -

Pipeline 22680

@suricata-qa
Copy link

Information: QA ran without warnings.

Pipeline 22681

@victorjulien
Copy link
Member Author

Probably best to convert the other hashmaps in the smb state as well.

@@ -732,7 +734,7 @@ fn smb1_response_record_one(state: &mut SMBState, r: &SmbRecord, command: u8, an
fid.extend_from_slice(&u32_as_bytes(r.ssn_id));
SCLogDebug!("SMB1_COMMAND_NT_CREATE_ANDX fid {:?}", fid);
SCLogDebug!("fid {:?} name {:?}", fid, p);
state.guid2name_map.insert(fid, p);
_ = state.guid2name_map.put(fid, p);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we handle somehow the return value ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what to do with it. IIRC it returns the evicted value.

@@ -732,7 +734,7 @@ fn smb1_response_record_one(state: &mut SMBState, r: &SmbRecord, command: u8, an
fid.extend_from_slice(&u32_as_bytes(r.ssn_id));
SCLogDebug!("SMB1_COMMAND_NT_CREATE_ANDX fid {:?}", fid);
SCLogDebug!("fid {:?} name {:?}", fid, p);
state.guid2name_map.insert(fid, p);
_ = state.guid2name_map.put(fid, p);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps for consistency with let _name, use let _evicted?

@jasonish
Copy link
Member

Just for observation sake, this brings in some new crates:

  • ahash: this one is new to me
  • hashbrown: common, well supported
  • once_cell: I believe this to be common and well supported to

@jasonish
Copy link
Member

Still a draft, but us of the lru crate looks good, and in general this crate is in very wide use.

@victorjulien victorjulien mentioned this pull request Oct 24, 2024
@victorjulien
Copy link
Member Author

continued testing in #12028

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

4 participants