[BUG] regexp_multi changed behavior in master

[aduernberger]

Prerequisites

• Put an X between the brackets on this line if you have done all of the following:
  • Read about bug reporting in general: https://rspamd.com/doc/faq.html#how-to-report-bugs-found-in-rspamd
  • Enabled relevant debugging logs: https://rspamd.com/doc/faq.html#how-to-debug-some-module-in-rspamd
  • Checked the FAQs about Core files in case of fatal crash: https://rspamd.com/doc/faq.html#how-to-figure-out-why-rspamd-process-crashed
  • Tried ASAN package and obtained the ASAN report (if possible): https://rspamd.com/doc/faq.html#asan-builds
  • Checked that your issue isn't already filed: https://github.com/issues?utf8=%E2%9C%93&q=is%3Aissue+user%3Arspamd
  • Checked that there is not already an experimental package or master branch

Describe the bug

Configuring a regexp_multi multimap does not set the correct result score. The Symbol score is shown correctly.

Steps to Reproduce

Configure multimap

bad_word {
    type = "content";
    filter = "text";
    description = "Bad Word filter";
    map = "regexp_multi;file:///etc/rspamd/local.d/bad_word.txt";
    score = 1.0;
    symbol = "BAD_WORD";
    regexp = true;
}

bad_word.txt
spam_body.txt

Call rspamc symbols spam_body.txt.

Score: 19.26 / 6.00
Symbol: BAD_WORD (4.00)
Symbol: BAYES_SPAM (0.86)[80.00%]
Symbol: BROKEN_HEADERS (8.00)
Symbol: HFILTER_HOSTNAME_UNKNOWN (2.50)
Symbol: MIME_GOOD (-0.10)[text/plain]
Symbol: MISSING_ESSENTIAL_HEADERS (7.00)

The Symbol BAD_WORD shows a score of 4.00. But the result score is 19.26 instead of 4.00 + 0.86 + 8.00 + 2.50 + -0.10 + 7.00 = 22,26

Expected behavior

Doing the same with 3.8.4 yields the correct score.

Score: 22.26 / 6.00
Symbol: BAD_WORD (4.00)
Symbol: BAYES_SPAM (0.86)[80.00%]
Symbol: BROKEN_HEADERS (8.00)
Symbol: HFILTER_HOSTNAME_UNKNOWN (2.50)
Symbol: MIME_GOOD (-0.10)[text/plain]
Symbol: MISSING_ESSENTIAL_HEADERS (7.00)

Versions

Broken: Rspamd daemon version 3.9.0 (current master built today, same problem with master built one month ago)
Working: Rspamd daemon version 3.8.4

Both tests were done on an updated Ubuntu 22.04 Server.

Additional Information

Maybe the same issue as #4978

[dragoangel]

Hi @aduernberger, yep, I assume issue is same that I faced. I closed my issue, to not create a duplicates.

[dragoangel]

@vstakhov can you please help with this issue?

[dragoangel]

Hi @fatalbanana thank you for a Autotests, do you have ideas whenever this big was introduced?

[fatalbanana]

Hi @fatalbanana thank you for a Autotests, do you have ideas whenever this big was introduced?

ae10366

[vstakhov]

I don't see how that could be related.

[vstakhov]

Also, guys, I'm on a vacation now, please do not expect any significant attention from me these days.

[fatalbanana]

I don't see how that could be related.

git bisect fingered this commit

test repo

I've tried to manually verify that it wasn't perhaps fixed again by related commits.

[dragoangel]

Hi @fatalbanana thank you for a Autotests, do you have ideas whenever this big was introduced?

ae10366

I build locally Rspamd with basic reverse commit and get things working again. So yes, this is the root case. Problem somewhere there.