Skip to content

Commit 935ba58

Browse files
paulnoalhytPaul Noalhytrbs-jacobwhyitfor
authored
Feature/debian12 (#502)
* Moved image to Debian 12. build_image.py now uses DOCKER_BUILDKIT=1. Add some type hints. Updated Lief version. Fixed tests. Stopped using package-manager toolchains. * Ran linter * Update notebook outputs for tests to pass * Build m68k GNU toolchain from source with crosstool-ng. Remove files from /tmp after building toolchains. * Apply suggestions from code review Co-authored-by: Jacob Strieb <[email protected]> * Fix Dockerstub * Update build_image.py. Removed duplicate env setting * Updated changelogs * Update CHANGELOGs --------- Co-authored-by: Paul Noalhyt <[email protected]> Co-authored-by: Jacob Strieb <[email protected]> Co-authored-by: Wyatt <[email protected]>
1 parent 3f94e44 commit 935ba58

File tree

19 files changed

+170
-88
lines changed

19 files changed

+170
-88
lines changed

build_image.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
import yaml
1111

1212
DEFAULT_PYTHON_IMAGE = (
13-
"python:3.8-bullseye@sha256:e1cd369204123e89646f8c001db830eddfe3e381bd5c837df00141be3bd754cb"
13+
"python:3.9-bookworm@sha256:a23efa04a7f7a881151fe5d473770588ef639c08fd5f0dcc6987dbe13705c829"
1414
)
1515
BASE_DOCKERFILE = "base.Dockerfile"
1616
FINISH_DOCKERFILE = "finish.Dockerfile"

disassemblers/ofrak_binary_ninja/ofrak_binary_ninja/components/blocks/unpackers.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
from typing import Optional
55
from warnings import warn
66

7-
from binaryninja import BinaryView, Endianness, TypeClass
7+
from binaryninja import BinaryView, Endianness, TypeClass, ReferenceSource, DataVariable
88
from ofrak_type.architecture import InstructionSetMode
99
from ofrak_type.range import Range
1010

@@ -120,7 +120,7 @@ def _binary_ninja_get_complex_blocks(
120120
# Add literal pools/data by iterating over data word candidates after the function's
121121
# code boundaries, and checking if there are code references to those candidates from
122122
# the function's code ranges
123-
data_refs = list()
123+
data_refs: List[ReferenceSource] = list()
124124

125125
# Adjust literal pool start address by accounting alignment "nop" instructions
126126
while binaryview.get_disassembly(end_ea) == "nop":
@@ -228,7 +228,7 @@ async def unpack(self, resource: Resource, config: Optional[ComponentConfig] = N
228228
literal_pool_search_addr += binaryview.get_instruction_length(literal_pool_search_addr)
229229

230230
while literal_pool_search_addr < cb_start_vaddr + cb_view.size:
231-
data_var = binaryview.get_data_var_at(literal_pool_search_addr)
231+
data_var: Optional[DataVariable] = binaryview.get_data_var_at(literal_pool_search_addr)
232232
if data_var is None or data_var.type.width == 0:
233233
literal_pool_search_addr += 1
234234
continue

disassemblers/ofrak_ghidra/CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
77

88
### Changed
99
- Minor update to OFRAK Community License, add OFRAK Pro License ([#478](https://github.com/redballoonsecurity/ofrak/pull/478))
10+
- Move to OpenJDK version 17 with the docker container move to Debian 12 ([#502](https://github.com/redballoonsecurity/ofrak/pull/502))
1011

1112
### Fixed
1213
- Speedup: do not run Ghidra auto-analysis upon importing a program. ([#473](https://github.com/redballoonsecurity/ofrak/pull/473))

disassemblers/ofrak_ghidra/Dockerstub

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
# Download & install java and supervisor
2-
RUN apt-get update && apt-get install -y openjdk-11-jdk supervisor
2+
RUN apt-get update && apt-get install -y openjdk-17-jdk supervisor
33

44
# Download & install ghidra
55
RUN mkdir -p /opt/rbs && \

ofrak_core/CHANGELOG.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,10 +45,13 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
4545
- `build_image.py` supports building Docker images with OFRAK packages from any ancestor directory. ([#425](https://github.com/redballoonsecurity/ofrak/pull/425))
4646
- Partially reverted [#150](https://github.com/redballoonsecurity/ofrak/pull/150) so entropy C code is called with `ctypes` again, but maintaining the current API and automatic compilation by `setup.py`. ([#482](https://github.com/redballoonsecurity/ofrak/pull/482))
4747
- Minor update to OFRAK Community License, add OFRAK Pro License ([#478](https://github.com/redballoonsecurity/ofrak/pull/478))
48+
- Update python to 3.9 as main version used and tested (including in default docker image build) ([#502](https://github.com/redballoonsecurity/ofrak/pull/502))
49+
- Update OpenJDK to version 17, remove unused qemu package ([#502](https://github.com/redballoonsecurity/ofrak/pull/502))
4850

4951
### Security
5052
- Update aiohttp to 3.10.11 ([#522](https://github.com/redballoonsecurity/ofrak/pull/522))
5153
- Update pycryptogrpahy to version 43.0.3. ([#525](https://github.com/redballoonsecurity/ofrak/pull/525))
54+
- Bump `lief` dependency to 0.15.1 to address [vulnerability](https://github.com/redballoonsecurity/ofrak/security/dependabot/31) in lower versions ([#502](https://github.com/redballoonsecurity/ofrak/pull/502))
5255

5356
## [3.2.0](https://github.com/redballoonsecurity/ofrak/compare/ofrak-v3.1.0...ofrak-v3.2.0)
5457
### Added

ofrak_core/Dockerstub

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,6 @@ RUN apt-get -y update && \
1414
mtd-utils \
1515
pigz \
1616
zip \
17-
qemu \
1817
qemu-user-static \
1918
u-boot-tools \
2019
unar \
@@ -24,7 +23,7 @@ RUN apt-get -y update && \
2423
RUN python3 -m pip install python-lzo
2524

2625
# Install apktool and uber-apk-signer
27-
RUN apt-get -y update && apt-get -y install openjdk-11-jdk
26+
RUN apt-get -y update && apt-get -y install openjdk-17-jdk
2827
RUN wget https://raw.githubusercontent.com/iBotPeaches/Apktool/v2.3.3/scripts/linux/apktool -O /usr/local/bin/apktool && \
2928
wget https://bitbucket.org/iBotPeaches/apktool/downloads/apktool_2.3.3.jar -O /usr/local/bin/apktool.jar && \
3029
wget https://github.com/patrickfav/uber-apk-signer/releases/download/v1.0.0/uber-apk-signer-1.0.0.jar -O /usr/local/bin/uber-apk-signer.jar && \

ofrak_core/ofrak/core/apk.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -25,10 +25,10 @@
2525
APKTOOL = ComponentExternalTool("apktool", "https://ibotpeaches.github.io/Apktool/", "-version")
2626
JAVA = ComponentExternalTool(
2727
"java",
28-
"https://openjdk.org/projects/jdk/11/",
28+
"https://openjdk.org/projects/jdk/17/",
2929
"-help",
30-
apt_package="openjdk-11-jdk",
31-
brew_package="openjdk@11",
30+
apt_package="openjdk-17-jdk",
31+
brew_package="openjdk@17",
3232
)
3333

3434

ofrak_core/ofrak/core/elf/lief_modifier.py

Lines changed: 17 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -38,29 +38,31 @@ class LiefAddSegmentModifier(Modifier[LiefAddSegmentConfig]):
3838
targets = (Elf,)
3939

4040
async def modify(self, resource: Resource, config: LiefAddSegmentConfig) -> None:
41-
binary: lief.ELF.Binary = lief.parse(await resource.get_data())
41+
binary: Optional[lief.Binary] = lief.parse(await resource.get_data())
42+
if not binary or not isinstance(binary, lief.ELF.Binary):
43+
raise ValueError("Lief failed parsing binary.")
4244

4345
segment = lief.ELF.Segment()
44-
segment.type = lief.ELF.SEGMENT_TYPES.LOAD
45-
segment.content = config.content
46+
segment.type = lief.ELF.Segment.TYPE.LOAD
47+
segment.content = memoryview(bytearray(config.content))
4648
segment.alignment = config.alignment
4749
segment.virtual_address = config.virtual_address
4850
if config.physical_address is not None:
4951
segment.physical_address = config.physical_address
5052
if "r" in config.rwx_flags:
51-
segment.add(lief.ELF.SEGMENT_FLAGS.R)
53+
segment.add(lief.ELF.Segment.FLAGS.R)
5254
if "w" in config.rwx_flags:
53-
segment.add(lief.ELF.SEGMENT_FLAGS.W)
55+
segment.add(lief.ELF.Segment.FLAGS.W)
5456
if "x" in config.rwx_flags:
55-
segment.add(lief.ELF.SEGMENT_FLAGS.X)
57+
segment.add(lief.ELF.Segment.FLAGS.X)
5658

5759
if config.replace_note:
5860
# instead of adding a segment to the binary, replace a useless NOTE segment
5961
# see https://github.com/lief-project/LIEF/issues/98
6062
# and https://github.com/lief-project/LIEF/issues/143
61-
if not binary.has(lief.ELF.SEGMENT_TYPES.NOTE):
63+
if not binary.has(lief.ELF.Segment.TYPE.NOTE):
6264
raise ValueError("Binary must have a NOTE section to add a new section")
63-
segment = binary.replace(segment, binary[lief.ELF.SEGMENT_TYPES.NOTE])
65+
segment = binary.replace(segment, binary[lief.ELF.Segment.TYPE.NOTE])
6466
if config.physical_address is not None:
6567
segment.physical_address = config.physical_address
6668
else:
@@ -86,10 +88,12 @@ class LiefAddSectionModifer(Modifier[LiefAddSectionModifierConfig]):
8688
targets = (Elf,)
8789

8890
async def modify(self, resource: Resource, config: LiefAddSectionModifierConfig):
89-
binary: lief.ELF.Binary = lief.parse(await resource.get_data())
91+
binary: Optional[lief.Binary] = lief.parse(await resource.get_data())
92+
if not binary or not isinstance(binary, lief.ELF.Binary):
93+
raise ValueError("Lief failed parsing binary.")
9094
section: lief.ELF.Section = lief.ELF.Section()
9195
section.name = config.name
92-
section.content = list(config.content)
96+
section.content = memoryview(bytearray(config.content))
9397
section.flags = config.flags
9498
binary.add(section)
9599

@@ -111,7 +115,9 @@ class LiefRemoveSectionModifier(Modifier[LiefRemoveSectionModifierConfig]):
111115
targets = (Elf,)
112116

113117
async def modify(self, resource: Resource, config: LiefRemoveSectionModifierConfig):
114-
binary: lief.ELF.Binary = lief.parse(await resource.get_data())
118+
binary: Optional[lief.Binary] = lief.parse(await resource.get_data())
119+
if not binary or not isinstance(binary, lief.ELF.Binary):
120+
raise ValueError("Lief failed parsing binary.")
115121
section: lief.ELF.Section = binary.get_section(config.name)
116122
if section is None:
117123
raise AttributeError(f"No section with name {config.name}")

ofrak_core/requirements.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ importlib-metadata>=4.13
1010
intervaltree==3.1.0
1111
keystone-engine==0.9.2
1212
jefferson==0.4.5;python_version>="3.8"
13-
lief==0.12.3
13+
lief==0.15.1
1414
orjson~=3.9.15
1515
pefile==2023.2.7
1616
pycdlib==1.12.0
Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
version https://git-lfs.github.com/spec/v1
2-
oid sha256:73c7c1514bcfc624e277ba3e210e5f3475e60b0e32820747e79f026c1d0c6fbe
3-
size 1839792
2+
oid sha256:88024463c275711c29f6ac2920a5d3174349d8802720e2df1656637c18d8efba
3+
size 1922136

0 commit comments

Comments
 (0)