Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import error on importing SIGMA rules that contain a single quote and double quotes #2947

Open
FrankShorty opened this issue Oct 17, 2023 · 0 comments
Assignees
Labels

Comments

@FrankShorty
Copy link

Describe the bug
Importing a collection SIGMA rules using the TSCTL import-sigma-rules function results in YAML parser errors.

For example, importing SIGMA rule Unsigned AppX Installation Attempt Using Add-AppxPackage

results in the error below

yaml.parser.ParserError: while parsing a flow mapping in "<unicode string>", line 1, column 1: {'title': 'Unsigned AppX Install ... ^ expected ',' or '}', but got '<scalar>' in "<unicode string>", line 1, column 285: ... of the "Add-AppxPackage" or it\'s alias "Add-AppPackage" to inst ...

This is due to the fact that prior to parsing, the sigma_util.py script typecasts to string the 'doc' extracted from rule_yaml_data in 'parse_sigma_rule_by_text':

  try:
        rule_yaml_data = yaml.safe_load_all(rule_text)

        for doc in rule_yaml_data:
            parser = sigma_collection.SigmaCollectionParser(
                str(doc), sigma_conf_obj, None
            )
            parsed_sigma_rules = parser.generate(sigma_backend)
            rule_return.update(doc)

this turns a string like this:
Detects usage of the "Add-AppxPackage" or it's alias "Add-AppPackage" to install unsigned AppX packages

into this:
'Detects usage of the "Add-AppxPackage" or it\'s alias "Add-AppPackage" to install unsigned AppX packages'

This breaks the YAML parser as it treats it like a control char, not a quote.

To solve this issue, we recommended to use StringIO from the io library to stringify the yaml document, rather then typecast to string:

from io import StringIO
...
        for doc in rule_yaml_data:
            yaml_stream = StringIO(yaml.dump(doc))
            parser = sigma_collection.SigmaCollectionParser(
                yaml_stream, sigma_conf_obj, None
            )

To Reproduce
Steps to reproduce the behavior:

  1. Copy the sigma rule from the page mentioned
  2. Copy the rule into the sigma rules folder.
  3. Run the TSCTL command to import the rules folder
  4. See error

Expected behavior
I expect the rule to be parsed correctly and not throw a parsing error.

Desktop (please complete the following information):

  • OS: Windows
  • Timesketch version: LATEST
@jkppr jkppr assigned jkppr and jaegeral and unassigned jkppr Nov 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants