Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix OverflowError when System.DateTime has DateTimeKind bits set #88

Merged
merged 1 commit into from
Oct 19, 2024

Conversation

nightlark
Copy link
Contributor

@nightlark nightlark commented Oct 7, 2024

I've had several users encounter dotnet assemblies (mostly resource dlls) that cause dnfile to throw an OverflowError: date value out of range exception (stack trace below). Looking into the dotnet System.DateTime data type, the ticks value is actually just stored using the lower 62-bits. The upper 2 bits are a DateTimeKind that is either unspecified, UTC, or Local.

Zeroing out the upper 2 bits resolves these cases for dotnet binaries with a "valid" System.DateTime (defined as up to 12/31/9999 23:59:59.9999999 according to a comment in the dotnet source code). However there is still a possibility that a malformed binary could encode a 62-bit value that still results in an OverflowError in Python's date time module, so that exception should be caught and handled slightly gracefully so parsing the rest of the file can at least continue.

I was trying to decide if anything special needs to be done to handle the DateTimeKind, but there isn't enough information to know what the local timezone was when the file got created for converting between time zones.

Unfortunately, the sample files I have aren't ones that I'm able to share.

Comment in the dotnet runtime sourcecode about the format for System.DateTime: https://github.com/dotnet/runtime/blob/17c55f1/src/libraries/System.Private.CoreLib/src/System/DateTime.cs#L130-L138

>>> dnfile.dnPE("Example.resources.dll")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/uname/Code/dnfile/src/dnfile/__init__.py", line 64, in __init__
    super().__init__(name, data, fast_load)
  File "/Users/mast9/Code/dnfile/venv/lib/python3.12/site-packages/pefile.py", line 2941, in __init__
    self.__parse__(name, data, fast_load)
  File "/Users/uname/Code/dnfile/src/dnfile/__init__.py", line 132, in __parse__
    super().__parse__(fname, data, fast_load)
  File "/Users/uname/Code/dnfile/venv/lib/python3.12/site-packages/pefile.py", line 3362, in __parse__
    self.full_load()
  File "/Users/uname/Code/dnfile/venv/lib/python3.12/site-packages/pefile.py", line 3479, in full_load
    self.parse_data_directories()
  File "/Users/uname/Code/dnfile/src/dnfile/__init__.py", line 178, in parse_data_directories
    value = entry[1](dir_entry.VirtualAddress, dir_entry.Size)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/uname/Code/dnfile/src/dnfile/__init__.py", line 221, in parse_clr_structure
    return ClrData(self, rva, size, self.clr_lazy_load)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/uname/Code/dnfile/src/dnfile/__init__.py", line 526, in __init__
    self._init_resources(pe)
  File "/Users/mast9/Code/dnfile/src/dnfile/__init__.py", line 574, in _init_resources
    rsrc.parse()
  File "/Users/uname/Code/dnfile/src/dnfile/resource.py", line 291, in parse
    rs.parse()
  File "/Users/uname/Code/dnfile/src/dnfile/resource.py", line 437, in parse
    rsrc_factory.read_rsrc_data_v2(self._data, e_data_offset, self.resource_types, e)
  File "/Users/uname/Code/dnfile/src/dnfile/resource.py", line 132, in read_rsrc_data_v2
    d, v = self.type_str_to_type(entry.type_name, data, offset)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/uname/Code/dnfile/src/dnfile/resource.py", line 232, in type_str_to_type
    dt = datetime.datetime(1, 1, 1) + delta
         ~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~
OverflowError: date value out of range

@nightlark nightlark changed the title Fix OverflowError when System.DateTime has DateTimeKind bit set Fix OverflowError when System.DateTime has DateTimeKind bits set Oct 7, 2024
@nightlark nightlark force-pushed the fix-datetime-parsing branch from fac90ac to fd94b25 Compare October 7, 2024 23:37
@malwarefrank
Copy link
Owner

Thank you for letting me know about the DateTime problem and contributing a fix. I think we should expose the DateTimeKind value, but I can add that after merging your changes.

@nightlark
Copy link
Contributor Author

nightlark commented Oct 15, 2024

How would you want the DateTimeKind value to be exposed? The two options I see are returning a (named)tuple or custom dnfile class with the two fields as the final_value.

The main downside I see is that it would be a breaking change, which I was trying to avoid -- but if you're fine with that, I could make that change here.

edit: Would it be possible to get a patch release that maintains backwards compatibility? Then I can add the breaking change that also returns the DateTimeKind in a follow up PR that can be merged after this one.

@malwarefrank malwarefrank merged commit 1c4007b into malwarefrank:master Oct 19, 2024
2 checks passed
@nightlark nightlark deleted the fix-datetime-parsing branch October 19, 2024 03:45
@nightlark nightlark restored the fix-datetime-parsing branch October 19, 2024 03:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants