Skip to content

New linux plugin: modxview #1330

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 37 commits into from
Jan 18, 2025
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
d5a0b93
add TAINT_FLAGS constant
Abyss-W4tcher Oct 31, 2024
e1b343a
add module taints parsing apis
Abyss-W4tcher Oct 31, 2024
c88ebe8
introduce modxview linux plugin
Abyss-W4tcher Oct 31, 2024
9440f53
use a dict of dataclasses for taint_flags
Abyss-W4tcher Nov 1, 2024
9d08c46
add module offset to seen_addresses
Abyss-W4tcher Nov 1, 2024
b209ea3
remove slashes in columns
Abyss-W4tcher Nov 1, 2024
485ef89
remove taints_value overload attr
Abyss-W4tcher Nov 8, 2024
dd3542b
explicit loop iterator
Abyss-W4tcher Nov 8, 2024
a388895
Merge branch 'volatilityfoundation:develop' into modxview_plugin
Abyss-W4tcher Dec 22, 2024
f3d7647
unify Tainting parsing capabilities
Abyss-W4tcher Dec 22, 2024
dda104b
move out Tainting capabilities
Abyss-W4tcher Jan 2, 2025
2a5f38e
introduce versioned Linux utilities
Abyss-W4tcher Jan 2, 2025
8bc6259
initial tainting utilities
Abyss-W4tcher Jan 2, 2025
3105a31
leverage Tainting from separated Linux utilities
Abyss-W4tcher Jan 2, 2025
6e4213e
update tainting requirements to new versioned utilities
Abyss-W4tcher Jan 2, 2025
0a35026
2.13.0 -> 2.14.0 bump
Abyss-W4tcher Jan 2, 2025
89b8da8
make self.kernel private and call parent __init__
Abyss-W4tcher Jan 2, 2025
32a9d16
Merge branch 'develop' into modxview_plugin
ikelos Jan 3, 2025
4a34b98
minor readability adjustments
Abyss-W4tcher Jan 3, 2025
38c5cc1
bump framework req to 2.16.0
Abyss-W4tcher Jan 3, 2025
a7b4e2f
version check_modules
Abyss-W4tcher Jan 5, 2025
2d262e7
cut unnecessary intermediate LinuxUtilityInterface
Abyss-W4tcher Jan 5, 2025
5c70356
version check_modules requirement
Abyss-W4tcher Jan 5, 2025
302f9fd
cut unnecessary plugin runner functions
Abyss-W4tcher Jan 5, 2025
4115c26
bump framework req to 2.18.0
Abyss-W4tcher Jan 5, 2025
bd82f4f
2.16.0 -> 2.18.0 bump
Abyss-W4tcher Jan 5, 2025
d956742
remove typing.Set import
Abyss-W4tcher Jan 5, 2025
0e4e751
stateless classmethods
Abyss-W4tcher Jan 11, 2025
b447bfa
remove module tainting proxies
Abyss-W4tcher Jan 16, 2025
94704c6
2.16.0 -> 2.17.0 bump
Abyss-W4tcher Jan 16, 2025
cd8690a
require framework version 2.17.0
Abyss-W4tcher Jan 16, 2025
ca98aa2
Merge branch 'develop' into modxview_plugin
Abyss-W4tcher Jan 18, 2025
cc9486c
pre-process module triaging to improve readability
Abyss-W4tcher Jan 18, 2025
3b679cb
explicit None check
Abyss-W4tcher Jan 18, 2025
bb6556d
correct arguments for pre_4_10_rc1
Abyss-W4tcher Jan 18, 2025
0b82f73
functools caching and doc.
Abyss-W4tcher Jan 18, 2025
8095924
typo
Abyss-W4tcher Jan 18, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion volatility3/framework/constants/_version.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# We use the SemVer 2.0.0 versioning scheme
VERSION_MAJOR = 2 # Number of releases of the library with a breaking change
VERSION_MINOR = 15 # Number of changes that only add to the interface
VERSION_MINOR = 18 # Number of changes that only add to the interface
VERSION_PATCH = 0 # Number of changes that do not change the interface
VERSION_SUFFIX = ""

Expand Down
55 changes: 55 additions & 0 deletions volatility3/framework/constants/linux/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
Linux-specific values that aren't found in debug symbols
"""
from enum import IntEnum, Flag
from dataclasses import dataclass

KERNEL_NAME = "__kernel__"

Expand Down Expand Up @@ -352,3 +353,57 @@ def flags(self) -> str:
MODULE_MAXIMUM_CORE_SIZE = 20000000
MODULE_MAXIMUM_CORE_TEXT_SIZE = 20000000
MODULE_MINIMUM_SIZE = 4096


@dataclass
class TaintFlag:
shift: int
desc: str
when_present: bool
module: bool


TAINT_FLAGS = {
"P": TaintFlag(
shift=1 << 0, desc="PROPRIETARY_MODULE", when_present=True, module=True
),
"G": TaintFlag(
shift=1 << 0, desc="PROPRIETARY_MODULE", when_present=False, module=True
),
"F": TaintFlag(shift=1 << 1, desc="FORCED_MODULE", when_present=True, module=False),
"S": TaintFlag(
shift=1 << 2, desc="CPU_OUT_OF_SPEC", when_present=True, module=False
),
"R": TaintFlag(shift=1 << 3, desc="FORCED_RMMOD", when_present=True, module=False),
"M": TaintFlag(shift=1 << 4, desc="MACHINE_CHECK", when_present=True, module=False),
"B": TaintFlag(shift=1 << 5, desc="BAD_PAGE", when_present=True, module=False),
"U": TaintFlag(shift=1 << 6, desc="USER", when_present=True, module=False),
"D": TaintFlag(shift=1 << 7, desc="DIE", when_present=True, module=False),
"A": TaintFlag(
shift=1 << 8, desc="OVERRIDDEN_ACPI_TABLE", when_present=True, module=False
),
"W": TaintFlag(shift=1 << 9, desc="WARN", when_present=True, module=False),
"C": TaintFlag(shift=1 << 10, desc="CRAP", when_present=True, module=True),
"I": TaintFlag(
shift=1 << 11, desc="FIRMWARE_WORKAROUND", when_present=True, module=False
),
"O": TaintFlag(shift=1 << 12, desc="OOT_MODULE", when_present=True, module=True),
"E": TaintFlag(
shift=1 << 13, desc="UNSIGNED_MODULE", when_present=True, module=True
),
"L": TaintFlag(shift=1 << 14, desc="SOFTLOCKUP", when_present=True, module=False),
"K": TaintFlag(shift=1 << 15, desc="LIVEPATCH", when_present=True, module=True),
"X": TaintFlag(shift=1 << 16, desc="AUX", when_present=True, module=True),
"T": TaintFlag(shift=1 << 17, desc="RANDSTRUCT", when_present=True, module=True),
"N": TaintFlag(shift=1 << 18, desc="TEST", when_present=True, module=True),
}
"""Flags used to taint kernel and modules, for debugging purposes.

Map based on 6.12-rc5.

Documentation :
- https://www.kernel.org/doc/Documentation/admin-guide/sysctl/kernel.rst#:~:text=guide/sysrq.rst.-,tainted,-%3D%3D%3D%3D%3D%3D%3D%0A%0ANon%2Dzero%20if
- https://www.kernel.org/doc/Documentation/admin-guide/tainted-kernels.rst#:~:text=More%20detailed%20explanation%20for%20tainting
- taint_flag kernel struct
- taint_flags kernel constant
"""
1 change: 1 addition & 0 deletions volatility3/framework/plugins/linux/check_modules.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
class Check_modules(plugins.PluginInterface):
"""Compares module list to sysfs info, if available"""

_version = (1, 0, 0)
_required_framework_version = (2, 0, 0)

@classmethod
Expand Down
172 changes: 172 additions & 0 deletions volatility3/framework/plugins/linux/modxview.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,172 @@
# This file is Copyright 2024 Volatility Foundation and licensed under the Volatility Software License 1.0
# which is available at https://www.volatilityfoundation.org/license/vsl-v1.0
#
import logging
from typing import List, Dict, Set, Iterator

Check failure on line 5 in volatility3/framework/plugins/linux/modxview.py

View workflow job for this annotation

GitHub Actions / lint

Ruff (F401)

volatility3/framework/plugins/linux/modxview.py:5:32: F401 `typing.Set` imported but unused
from volatility3.plugins.linux import lsmod, check_modules, hidden_modules
from volatility3.framework import interfaces
from volatility3.framework.configuration import requirements
from volatility3.framework.renderers import format_hints, TreeGrid, NotAvailableValue
from volatility3.framework.symbols.linux import extensions
from volatility3.framework.constants import architectures
from volatility3.framework.symbols.linux.utilities import tainting

vollog = logging.getLogger(__name__)


class Modxview(interfaces.plugins.PluginInterface):
"""Centralize lsmod, check_modules and hidden_modules results to efficiently
spot modules presence and taints."""

_version = (1, 0, 0)
_required_framework_version = (2, 18, 0)

@classmethod
def get_requirements(cls) -> List[interfaces.configuration.RequirementInterface]:
return [
requirements.ModuleRequirement(
name="kernel",
description="Linux kernel",
architectures=architectures.LINUX_ARCHS,
),
requirements.VersionRequirement(
name="linux-tainting", component=tainting.Tainting, version=(1, 0, 0)
),
requirements.PluginRequirement(
name="lsmod", plugin=lsmod.Lsmod, version=(2, 0, 0)
),
requirements.PluginRequirement(
name="check_modules",
plugin=check_modules.Check_modules,
version=(1, 0, 0),
),
requirements.PluginRequirement(
name="hidden_modules",
plugin=hidden_modules.Hidden_modules,
version=(1, 0, 0),
),
requirements.BooleanRequirement(
name="plain_taints",
description="Display the plain taints string for each module.",
optional=True,
default=False,
),
]

@classmethod
def flatten_run_modules_results(
cls, run_results: Dict[str, List[extensions.module]], deduplicate: bool = True
) -> Iterator[extensions.module]:
"""Flatten a dictionary mapping plugin names and modules list, to a single merged list.
This is useful to get a generic lookup list of all the detected modules.

Args:
run_results: dictionary of plugin names mapping a list of detected modules
deduplicate: remove duplicate modules, based on their offsets

Returns:
Iterator of modules objects
"""
seen_addresses = set()
for modules in run_results.values():
for module in modules:
if deduplicate and module.vol.offset in seen_addresses:
continue
seen_addresses.add(module.vol.offset)
yield module

@classmethod
def run_modules_scanners(
cls,
context: interfaces.context.ContextInterface,
kernel_name: str,
run_hidden_modules: bool = True,
) -> Dict[str, List[extensions.module]]:
"""Run module scanning plugins and aggregate the results.

Args:
run_hidden_modules: specify if the hidden_modules plugin should be run
Returns:
Dictionary mapping each plugin to its corresponding result
"""

kernel = context.modules[kernel_name]
run_results = {}
# lsmod
run_results["lsmod"] = list(lsmod.Lsmod.list_modules(context, kernel_name))
# check_modules
sysfs_modules: dict = check_modules.Check_modules.get_kset_modules(
context, kernel_name
)
## Convert get_kset_modules() offsets back to module objects
run_results["check_modules"] = [
kernel.object(object_type="module", offset=m_offset, absolute=True)
for m_offset in sysfs_modules.values()
]
# hidden_modules
if run_hidden_modules:
known_modules_addresses = set(
context.layers[kernel.layer_name].canonicalize(module.vol.offset)
for module in run_results["lsmod"] + run_results["check_modules"]
)
modules_memory_boundaries = (
hidden_modules.Hidden_modules.get_modules_memory_boundaries(
context, kernel_name
)
)
run_results["hidden_modules"] = list(
hidden_modules.Hidden_modules.get_hidden_modules(
context,
kernel_name,
known_modules_addresses,
modules_memory_boundaries,
)
)

return run_results

def _generator(self):
kernel_name = self.config["kernel"]
run_results = self.run_modules_scanners(self.context, kernel_name)
modules_offsets = {}
for key in ["lsmod", "check_modules", "hidden_modules"]:
modules_offsets[key] = set(module.vol.offset for module in run_results[key])

seen_addresses = set()
for modules_list in run_results.values():
for module in modules_list:
if module.vol.offset in seen_addresses:
continue
seen_addresses.add(module.vol.offset)

if self.config.get("plain_taints"):
taints = module.get_taints_as_plain_string()
else:
taints = ",".join(module.get_taints_parsed())

yield (
0,
(
module.get_name() or NotAvailableValue(),
format_hints.Hex(module.vol.offset),
module.vol.offset in modules_offsets["lsmod"],
module.vol.offset in modules_offsets["check_modules"],
module.vol.offset in modules_offsets["hidden_modules"],
taints or NotAvailableValue(),
),
)

def run(self):
columns = [
("Name", str),
("Address", format_hints.Hex),
("In procfs", bool),
("In sysfs", bool),
("Hidden", bool),
("Taints", str),
]

return TreeGrid(
columns,
self._generator(),
)
26 changes: 25 additions & 1 deletion volatility3/framework/symbols/linux/extensions/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
from volatility3.framework.objects import utility
from volatility3.framework.symbols import generic, linux, intermed
from volatility3.framework.symbols.linux.extensions import elf

from volatility3.framework.symbols.linux.utilities import tainting

vollog = logging.getLogger(__name__)

Expand Down Expand Up @@ -279,6 +279,30 @@ def get_symbol_by_address(self, wanted_sym_address):

return None

def get_taints_as_plain_string(self) -> str:
"""Convert the module's taints value to a 1-1 character mapping.
Convenient wrapper around framework's Tainting capabilities.

Returns:
The raw taints string.
"""
return tainting.Tainting(
self._context,
linux.LinuxUtilities.get_module_from_volobj_type(self._context, self).name,
).get_taints_as_plain_string(self.taints, True)

def get_taints_parsed(self) -> List[str]:
"""Convert the module's taints string to a 1-1 descriptor mapping.
Convenient wrapper around framework's Tainting capabilities.

Returns:
A comprehensive (user-friendly) taint descriptor list.
"""
return tainting.Tainting(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Straight pass throughs like this don't need adding, people should just depend directly on tainting.Tainting. We only need them for older methods that used to live here which people may try calling because the plugin hasn't been updated.

Copy link
Contributor Author

@Abyss-W4tcher Abyss-W4tcher Jan 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These (new) straight pass through are convenient functions to allow plugins devs to do:

my_module.get_taints_as_plain_string()

instead of

tainting.Tainting(context,vmlinux.name).get_taints_as_plain_string(my_module.taints, True)

It also manages the correct attribute and parameters to pass to fit the module use case. This also applies to volshell module inspection use cases. If you still think that it is overkill, please let me know.

Regarding the need for an API bump, even if it is a passthrough, it still is a non-private method addition to the API, no matter its content ?

Copy link
Contributor Author

@Abyss-W4tcher Abyss-W4tcher Jan 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some reason I can't answer the comment about API versioning so I'll do it here.

The point of these new APIs is not only for modules, but for later plugins. We plan on doing a kernel tainting plugin, so putting these in a sub-LinuxUtilities is not beneficial right now but will be in the future.

API minor bumps (re-counted after merge of develop into this branch):

  • 1 (new) : tainting.Tainting.get_taints_*
  • 2 (new) : module.get_taints_*

Which gives : 2.16.0 -> 2.18.0

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, sorry, I thought these were on LinuxUtilities, rather than on module.

Having said that, module is already very heavy (I had to scroll quite a while to see which class they were attached to). So now what happens if suddenly the method requires an extra parameter? Typically we leave the old one around but with a deprecation method, and then implement a new method that takes the additional parameter. We also don't really have a way of versioning modules, which means it's either the framework version goes up, or we find a way of versioning the module class, which then everyone has to check before they make a call. So... as long as you're happy that the api encompasses everything you want to do now and will want to do in the future, then it's ok to go in...

tainting.Tainting will be version itself, so no need for a framework bump, the module's either there and versioned or it'll throw an import error (which will rule out the plugin that's trying to import it).

That leaves only the bump on module.get_trains_* which is needed so that a plugin being run under the framework fails early rather than trying to make calls to an API that doesn't exist.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically, that proxy call also needs to check that the Tainting module is the right version (although it would only prevent errors if the Tainting module MAJOR version went up and someone forgot to change this method. Should be spotted quickly, but the question is, is all that worth the convenience of having it defined on the module. (Also, why is the tainting object taking parameters for construction, is that so it gets cached?

Copy link
Contributor Author

@Abyss-W4tcher Abyss-W4tcher Jan 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's noted for module.get_taints_*, but doesn't adding Tainting for the first time technically be considered a new framework API (still making it 2.18.0), even if obviously it would raise an ImportError on failure.

Removing this passthrough might prevent users/devs exploring the module API and unfamiliar with tainting to apply it. Also, I think we still have to define what should be the procedure to apply, if someone adds a new versioned LinuxUtility and wants to call it inside a linux.extensions class like inode or socket.

Copy link
Contributor Author

@Abyss-W4tcher Abyss-W4tcher Jan 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ikelos, I made a PoC to set requirements on (unversionable internal) APIs with a wrapper:

# Requirements can be stacked
# Raises an exception (at plugin runtime) on requirement failure
@requirements.VersionRequirement.version_requirement_wrapper(
    name="linux-tainting", component=tainting.Tainting, version=(1, 0, 0)
)
def get_taints_as_plain_string(self) -> str:

Disregarding the underlying machinery, do you think this design could be an appropriate solution to flexibly emit requirements ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, you're probably right, it should get at least an additive MINOR version bump. I think we've managed ok before but for the sake of accuracy you're right.

I'm not sure that a module not being exposed as a method on a class is that good an excuse to just keep heaping stuff on the module. Plus if we don't start now, we'll end up doing it for every new feature in existance, and that'll become unwieldy down the line (assuming it isn't already unwieldy with all the methods it has at the moment).

If people don't know about tainting, they're not going to notice it in the module documentation, and if they know that's what they're looking for, they'll find it quickly enough (particularly with existing plugins in the framework that make use of it). I don't think that argument holds up as to why it needs to be part of the module interface.

So at the moment, still not convinced it should be part of module.

I'm actually not against the version_requirement_wrapper idea, although I don't think it helps as much as we'd want. It'll do the code to fail when a requirement isn't met, and it makes it visible at the start of a function, but what about a class that makes use of several different versions of a function? There's no way of using that information in something calling the plugin, so it will likely just perpetuate an intricate web rather than doing anything all that useful to keep it managable. So for example, let's say the module now requires "Tainting-1.0.0", does the plugin know that calling module will require that? I think tagging the point of calling with a dependency doesn't help, because by the time you've called it, it's too late. You want to know up front, before you've invested any processing power, whether something's going to fail later on or not.

So tagging individual subfunctions within module will just create an object you can't easily use without checking each of the individual components in it that you need. That's why I'd like to keep the functionality separate, rather than tying all of their functionality to the highest requirement of any of the subcomponents...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can understand that the extra-complexity for the sake of convenience might not be worthy, and as its counterpart is not that much longer I think it is best to stop here and move on 👍. I'll remove the proxies.

Currently, internal APIs calling LinuxUtilities aren't version restrained:

yield linux.LinuxUtilities.container_of(

To fix this, we would need to basically make all internal classes inherit from VersionRequirement, so that they get automatically checked at instantiation. The decorator is only here as a "panic/debug" safeguard to help us understand immediately where a bug originates if we forgot to update API callers. Indeed, it can get "messy" if we start versioning internal APIs one-by-one with it.

I think this versioning issue is already in the framework unfortunately, but I do not have an immediate way of solving it properly without updating each LinuxUtilities call. I opened an issue (#1554) as it starts to diverge from the original modxview plugin.

self._context,
linux.LinuxUtilities.get_module_from_volobj_type(self._context, self).name,
).get_taints_parsed(self.taints, True)

@property
def section_symtab(self):
if self.has_member("kallsyms"):
Expand Down
Empty file.
Loading
Loading