Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dpdk: add initial unittests for DPDK codebase v4 #11728

Closed
wants to merge 7 commits into from

Conversation

lukashino
Copy link
Contributor

Redmine ticket: https://redmine.openinfosecfoundation.org/issues/6927
Follow-up of: #11494

Describe changes:
v4

  • fixed live device cleaning function
  • silenced 23.11 DPDK bond warning about "deprecated" (experimental) function on Fedora 40 builds
  • rebased

v3

  • comments from Philippe
  • I left TAILQ_INIT in the finalize/cleanup functions - all elements of the arrays are cleaned but not the base pointer of the array itself. TAILQ_* functions don't offer reset function but TAILQ_INIT provides the desired outcome
  • moved guards around - per Philippe's suggestions

v2

  • added a FatalError check on the number of LiveDevices
  • changed #if HAVE_DPDK to #if defined(HAVE_DPDK) && defined(UNITTESTS)
  • enabled unit tests in the Github workflow runs on Ubuntu and Fedora tasks

v1

  • function-guarded variable
  • fix the CPU exclude logic
  • add DPDK unit tests

Lukas Sismis added 7 commits September 8, 2024 19:29
To better control the values within the variables and to
prepare for the follow-up unit tests, the variable was moved
into global scope and should accessed only by functions.
This allows reinstantination of the variable value - needed for
unit tests.
The function would incorrectly perform XOR operation. While it
worked when the worker cores were occupying all cores, it is
still not correct operation. The example might be when a core
would be affined to only management and not worker threads.
With the XOR operation it would set affinity to also worker set.
(1 XOR 0 -> 1 where in fact the desired outcome is 0)
For the upcoming changes, skipping a unit test might be beneficial
when testing a function that retrieves hardware data. This can e.g. depend
on the number of CPU cores and systems that does not meet the required
test criteria will need to omit the tests.
The tests should always target minimal system requirements
Copy link

codecov bot commented Sep 9, 2024

Codecov Report

Attention: Patch coverage is 92.80822% with 21 lines in your changes missing coverage. Please review.

Project coverage is 82.64%. Comparing base (685baa9) to head (a6cae2c).
Report is 152 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #11728      +/-   ##
==========================================
+ Coverage   82.63%   82.64%   +0.01%     
==========================================
  Files         919      919              
  Lines      248925   249204     +279     
==========================================
+ Hits       205703   205961     +258     
- Misses      43222    43243      +21     
Flag Coverage Δ
fuzzcorpus 60.85% <16.66%> (-0.04%) ⬇️
livemode 18.87% <76.00%> (+0.15%) ⬆️
pcap 44.06% <33.33%> (-0.08%) ⬇️
suricata-verify 61.85% <33.33%> (-0.03%) ⬇️
unittests 58.78% <92.80%> (-0.23%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

@suricata-qa
Copy link

Information:

ERROR: QA failed on SURI_TLPR1_alerts_cmp.

field baseline test %
SURI_TLPR1_stats_chk
.uptime 642 620 96.57%

Pipeline 22482

*/
#define SKIP(reason) \
do { \
SCLogInfo("Test skipped: %s", reason); \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need to SCLogInfo as we will have the printf("skip\n"); after ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

./src/suricata -u -U DPDKRunmodeSetThreads01
Test DPDKRunmodeSetThreads01                                      : Info: affinity: Test skipped: not enough cpus in the system, the test requires at least 32 cores [UnitTestsUtilAffinityVerifyCPURequirement:util-affinity.c:347]
skip
==== TEST RESULTS ====
PASSED: 0
SKIPPED: 1
FAILED: 0
======================

This was the idea - to only tell the user that the test was skipped but a responsible test writer should explain why the test was skipped and what prerequirements were not met for the test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The macro SKIP_INCOMPATIBLE_ENVIRONMENT could skip the tests not only based on CPU count but potentially on other requirements as well.

@@ -52,7 +52,10 @@ uint16_t BondingMemberDevicesGet(
{
#ifdef HAVE_DPDK_BOND
#if RTE_VERSION >= RTE_VERSION_NUM(23, 11, 0, 0)
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wdeprecated-declarations"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need this ? Why cannot we have a non deprecated declaration ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non deprecated declaration is not possible as DPDK in 23.11 marked the old API (slave) as deprecated and the new API (member) as experimental. On both versions of this API therefore a warning is thrown. I circumvented this by allowing experimental API in the Github Actions.

@@ -88,10 +88,11 @@ ThreadsAffinityType * GetAffinityTypeFromName(const char *name);

uint16_t AffinityGetNextCPU(ThreadsAffinityType *taf);
uint16_t UtilAffinityGetAffinedCPUNum(ThreadsAffinityType *taf);
#ifdef HAVE_DPDK
#if defined(__linux__)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there #if defined(UNITTESTS) missing ?
It is in the C file, but not here...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, fixed

Copy link
Member

@victorjulien victorjulien left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see Philippe's comments inline

@lukashino
Copy link
Contributor Author

goto #12081

@lukashino lukashino closed this Nov 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

4 participants