configure Ghidrathon using absolute path of Python interpreter used t…

…o install Jep (#85)

* configure Ghidrathon using absolute path of Python interpreter

* update license

* fixup README

.github/workflows/tests.yml

@@ -0,0 +1,77 @@
+name: CI
+
+on:
+  push:
+    branches: [ main ]
+  pull_request:
+    branches: [ main ]
+
+jobs:
+
+  tests:
+    name: Tests in ${{ matrix.python-version }} on ${{ matrix.os }}
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        ghidra-release-url: ["https://github.com/NationalSecurityAgency/ghidra/releases/download/Ghidra_10.3.2_build/ghidra_10.3.2_PUBLIC_20230711.zip", "https://github.com/NationalSecurityAgency/ghidra/releases/download/Ghidra_11.0_build/ghidra_11.0_PUBLIC_20231222.zip"]
+        jep-jar-release-url: ["https://github.com/ninia/jep/releases/download/v4.2.0/jep-4.2.0.jar"]
+        os: [ubuntu-latest, windows-latest, macos-latest]
+        python-version: ["3.8", "3.12"]
+    steps:
+    - name: Checkout Ghidrathon
+      uses: actions/checkout@v4
+    - name: Setup Java
+      uses: actions/setup-java@v4
+      with:
+        distribution: "temurin"
+        java-version: "17"
+    - name: Setup Gradle
+      uses: gradle/actions/setup-gradle@v3
+      with:
+        gradle-version: "7.3"
+    - name: Setup Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v5
+      with:
+        python-version: ${{ matrix.python-version }}
+    - name: Configure temp folder
+      run: mkdir ../tmp
+    - name: Install Python Jep
+      run: |
+        pip install numpy
+        pip install jep==4.2.0
+        python -c "import importlib.util;import pathlib;print(pathlib.Path(importlib.util.find_spec('jep').origin).parent)"
+    - name: Download dependencies Linux/macOS
+      if : ${{ matrix.os != 'windows-latest' }}
+      run: |
+        wget ${{ matrix.ghidra-release-url }} -O ../tmp/ghidra.zip
+        unzip ../tmp/ghidra.zip -d ../tmp/ghidra
+        mv ../tmp/ghidra/$(ls ../tmp/ghidra) ../tmp/ghidra/ghidra_PUBLIC
+        wget ${{ matrix.jep-jar-release-url }} -O ./lib/jep-4.2.0.jar
+    - name: Download dependencies Windows
+      if : ${{ matrix.os == 'windows-latest' }}
+      shell: pwsh
+      run: |
+        Invoke-WebRequest -URI "${{ matrix.ghidra-release-url }}" -OutFile "../tmp/ghidra.zip"
+        mkdir ../tmp/ghidra
+        tar -xf ../tmp/ghidra.zip -C ../tmp/ghidra
+        Rename-Item -Path "../tmp/ghidra/$((Get-ChildItem -Path "../tmp/ghidra").Name)" -NewName "ghidra_PUBLIC"
+        Invoke-WebRequest -URI "${{ matrix.jep-jar-release-url }}" -OutFile "./lib/jep-4.2.0.jar"
+    - name: Build Ghidrathon
+      run: gradle -PGHIDRA_INSTALL_DIR=${{ github.workspace }}/../tmp/ghidra/ghidra_PUBLIC
+    - name: Install Ghidrathon Linux/macOS
+      if : ${{ matrix.os != 'windows-latest' }}
+      run: |
+        unzip ./dist/$(ls ./dist) -d ../tmp/ghidra/ghidra_PUBLIC/Ghidra/Extensions/
+    - name: Install Ghidrathon Windows
+      if : ${{ matrix.os == 'windows-latest' }}
+      shell: pwsh
+      run: |
+        Rename-Item -Path "./dist/$((Get-ChildItem -Path "./dist").Name)" -NewName "Ghidrathon.zip"
+        tar -xf ./dist/Ghidrathon.zip -C ../tmp/ghidra/ghidra_PUBLIC/Ghidra/Extensions/
+    - name: Set Ghidrathon Python interpreter
+      run: python util/ghidrathon_configure.py ../tmp/ghidra/ghidra_PUBLIC
+    - name: Run tests
+      run: |
+        ../tmp/ghidra/ghidra_PUBLIC/support/analyzeHeadless ${{ github.workspace }}/../tmp/ghidra test -Import ${{ github.workspace }}/../tmp/ghidra/ghidra_PUBLIC/GPL/DemanglerGnu/os/linux_x86_64/demangler_gnu_v2_24 -PostScript ${{ github.workspace }}/data/python/tests/hello.py -PostScript ${{ github.workspace }}/data/python/tests/runall.py
+        python -c "import pathlib, sys; sys.exit(0 if pathlib.Path('hello.txt').exists() else -1)"

LICENSE.txt

@@ -187,7 +187,7 @@
       same "printed page" as the copyright notice for easier
       identification within third-party archives.
 
-   Copyright (C) 2022 Mandiant, Inc.
+   Copyright (C) 2024 Mandiant, Inc.
 
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.

README.md

@@ -1,8 +1,9 @@
 # Ghidrathon
 
 [![License](https://img.shields.io/badge/license-Apache--2.0-green.svg)](LICENSE.txt)
+[![CI](https://github.com/mandiant/ghidrathon/actions/workflows/tests.yml/badge.svg)](https://github.com/mandiant/ghidrathon/actions/workflows/tests.yml)
 
-Ghidrathon is a Ghidra extension that adds Python 3 scripting capabilities to Ghidra. Why? Ghidra natively supports scripting in Java and Jython. Unfortunately many open-source analysis tools, like [capa](https://github.com/mandiant/capa), [Unicorn Engine](https://github.com/unicorn-engine/unicorn), [angr](https://github.com/angr/angr), etc., are written in Python 3 making it difficult, and in some cases, impossible to use these tools in Ghidra. More so the security community has released several great plugins for other SRE frameworks like IDA Pro and Binary Ninja, but again, because many of these plugins use Python 3 it is difficult to port them to Ghidra. Ghidrathon helps you use existing and develop new Python 3 tooling in Ghidra and script Ghidra using modern Python in a way that tightly integrates with Ghidra's UI.
+Ghidrathon is a Ghidra extension that adds Python 3 scripting capabilities to Ghidra. Why? Ghidra natively supports scripting in Java and Jython. Unfortunately, many open-source analysis tools, like [capa](https://github.com/mandiant/capa), [Unicorn Engine](https://github.com/unicorn-engine/unicorn), [angr](https://github.com/angr/angr), etc., are written in Python 3 making it difficult, and in some cases, impossible to use these tools in Ghidra. More so the security community has released several great plugins for other SRE frameworks like IDA Pro and Binary Ninja, but again, because many of these plugins use Python 3 it is difficult to port them to Ghidra. Ghidrathon helps you use existing and develop new Python 3 tooling in Ghidra and script Ghidra using modern Python in a way that tightly integrates with Ghidra's UI.
 
 Check out:
 
@@ -54,7 +55,7 @@ INFO  REPORT: Post-analysis succeeded for file: /example.o (HeadlessAnalyzer)
 INFO  REPORT: Save succeeded for processed file: /example.o (HeadlessAnalyzer)
 ```
 
-For more information on running Ghidra in headless mode check out `<ghidra_install>/support/analyzeHeadlessREADME.html`.
+For more information on running Ghidra in headless mode check out `<absolute_path_to_ghidra_install_dir>/support/analyzeHeadlessREADME.html`.
 
 ## Third-Party Python Modules
 
@@ -74,87 +75,44 @@ Ghidrathon links your local Python installation to Ghidra using the open-source
 
 For more information on how Jep works to embed Python in Java see their documentation [here](https://github.com/ninia/jep/wiki/How-Jep-Works).
 
-## OS Support
-
-Ghidrathon supports the following operating systems:
-
-* Linux
-* Windows
-* macOS (x86_64)
-
-## Requirements
-
-The following tools are needed to build, install, and run Ghidrathon:
+## Installing Ghidrathon
 
+### Requirements
 Tool | Version |Source |
 |---|---|---|
-| Ghidra | `>= 10.3` | https://ghidra-sre.org |
-| Jep | `4.2.0` | https://github.com/ninia/jep |
-| Gradle | `>= 7.3` | https://gradle.org/releases |
-| Python | `>= 3.8` | https://www.python.org/downloads |
-
-Note: Ghidra >= 10.2 requires [JDK 17 64-bit](https://adoptium.net/temurin/releases/).
-
-## Python Virtual Environments
+| Ghidrathon | `>= 4.0.0` | https://github.com/mandiant/Ghidrathon/releases |
+| Python | `>= 3.8.0` | https://www.python.org/downloads |
+| Jep | `4.2.0` | https://github.com/ninia/jep/releases |
+| Ghidra | `>= 10.3.2` | https://github.com/NationalSecurityAgency/ghidra/releases |
+| Java | `>= 17.0.0` | https://adoptium.net/temurin/releases/ |
 
-Ghidrathon supports Python virtual environments. To use a Python virtual environment, simply build Ghidrathon inside your virtual environment **and** execute Ghidra inside the **same** virtual environment.
-
-## Building Ghidrathon
-
-**Note:** Review [Python Virtual Environments](#python-virtual-environments) before building if you would like to use a Python virtual environment for Ghidrathon.
-
-**Note**: Building Ghidrathon requires building Jep. If you are running Windows, this requires installing the Microsoft C++ Build Tools found [here](https://visualstudio.microsoft.com/visual-cpp-build-tools/). See Jep's documentation [here](https://github.com/ninia/jep/wiki/Windows) for more information on installing Jep on Windows.
-
-Use the following steps to build Ghidrathon for your environment:
-
-* Install Ghidra using the documentation [here](https://htmlpreview.github.io/?https://github.com/NationalSecurityAgency/ghidra/blob/stable/GhidraDocs/InstallationGuide.html#InstallationNotes)
-* Install Gradle from [here](https://gradle.org/releases)
-* Download the latest Ghidrathon source release from [here](https://github.com/mandiant/Ghidrathon/releases)
-* Run the following command from the Ghidrathon source directory:
-    * **Note:** Ghidrathon defaults to the Python binary found in your path. You can specify a different Python binary by adding the optional argument `-PPYTHON_BIN=<absolute path to Python binary>` to the command below
-    * **Note:** you may optionally set an environment variable named `GHIDRA_INSTALL_DIR` instead of specifying `-PGHIDRA_INSTALL_DIR`
+Use the following steps to install Ghidrathon to your Ghidra environment:
 
+1. Install Jep:
 ```
-$ gradle -PGHIDRA_INSTALL_DIR=<absolute path to Ghidra install>
+$ python -m pip install jep==4.2.0
 ```
+2. Execute `ghidrathon_configure.py`:
+```
+$ python ghidrathon_configure.py <absolute_path_to_ghidra_install_dir>
+```
+3. Download and unzip the latest Ghidrathon [release](https://github.com/mandiant/Ghidrathon/releases)
+4. Install the Ghidrathon extension (`.zip`) into Ghidra:
+   * Using Ghidra's UI:
+        * Navigate to `File > Install Extensions...`
+        * Click the green `+` button
+        * Navigate to the Ghidrathon extension (`.zip`)
+        * Click `Ok`
+    * Using a limited environment:
+        * Extract the Ghidrathon extension (`.zip`)  to `<absolute_path_to_ghidra_install_dir>\Ghidra\Extensions`
 
-This command installs Jep, configures Ghidrathon with the necessary Jep binaries, and builds Ghidrathon. If successful, you will find a new directory in your Ghidrathon source directory named `dist` containing your Ghidrathon extension (`.zip`). Please open a new issue if you experience any issues building Ghidrathon.
-
-## Installing Ghidrathon
-
-Use the following steps to install your Ghidrathon extension using the Ghidra UI:
-
-* Start Ghidra
-* Navigate to `File > Install Extensions...`
-* Click the green `+` button
-* Navigate to your Ghidrathon extension built earlier (`.zip`)
-* Click `Ok`
-* Restart Ghidra
-
-**OR**
-
-Extract your Ghidrathon extension (`.zip`) directly to `<absolute path to Ghidra install>\Ghidra\Extensions` to automatically enable Ghidrathon the next time that Ghidra is started. This method works great if you do not have access to the Ghidra UI when installing Ghidrathon.
-
-### Disabling Jython
-
-Ghidrathon disables the built-in Jython script provider to avoid conflicts when Ghidra decides which provider should handle scripts with the `.py` file extension. This means existing Jython scripts cannot be executed with Ghidrathon installed. We recommend completely disabling the Jython extension.
-
-Use the following steps to disable the Jython extension:
-
-* Open a CodeBrowser window (*not the Project Manager window*)
-* Navigate to `File > Configure...`
-* Click `Ghidra Core`
-* Un-check `PythonPlugin`
-
-Use the following steps to enable the Jython extension:
+### Switching Python Interpreters
 
-* Uninstall Ghidrathon
-* Enable the Jython extension using the steps outlined above
-* Restart Ghidra
+You can switch Ghidrathon to use a different Python interpreter by running `ghidrathon_configure.py` using the new Python interpreter.
 
-## Using Ghidrathon
+### Python Virtual Environments
 
-See [Python 3 Interpreter Window](#python-3-interpreter-window), [Ghidra Script Manager Integration](#ghidra-script-manager-integration), and [Ghidra Headless Mode](#ghidra-headless-mode) for more information about using Ghidrathon.
+Ghidrathon supports Python virtual environments. To use a Python virtual environment, complete steps `1` and `2` using the Python interpreter that is configured for your environment. Do the same when running `ghidrathon_configure.py` to switch the Ghidrathon to use a different interpreter.
 
 ## Considerations
 

build.gradle

@@ -1,4 +1,4 @@
-// Copyright (C) 2022 Mandiant, Inc. All Rights Reserved.
+// Copyright (C) 2024 Mandiant, Inc. All Rights Reserved.
 // Licensed under the Apache License, Version 2.0 (the "License");
 //  you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at: [package root]/LICENSE.txt
@@ -22,39 +22,6 @@
 // application.gradle.version property in <GHIDRA_INSTALL_DIR>/Ghidra/application.properties
 // for the correction version of Gradle to use for the Ghidra installation you specify.
 
-def javaHome
-def pythonBin
-
-if (project.hasProperty("PYTHON_BIN")) {
-	pythonBin = project.getProperty("PYTHON_BIN")
-}
-else {
-	pythonBin = "python"
-}
-
-// we need to install Jep; this requires C++ build tools on Windows (see README); we need to define
-// the env variable "JAVA_HOME" containing absolute path to Java JDK configured for Ghidra
-task installJep(type: Exec) {
-    environment "JAVA_HOME", System.getProperty("java.home")
-    commandLine pythonBin, '-m', 'pip', 'install', 'jep==4.2'
-}
-
-// we need to copy the Jep native binaries built in installJep to our extension directory; we use a small
-// utility script written in Python
-task copyJepNativeBinaries(type: Exec) {
-    dependsOn installJep
-    workingDir "${projectDir}"
-    commandLine pythonBin, "util${File.separator}configure_jep_native_binaries.py"
-}
-
-// make all tasks not matching copyJepNativeBinaries or installJep depend on copyJepNativeBinaries; mostly
-// used to ensure our tasks run before Ghidra buildExtension task
-tasks.matching { it.name != 'copyJepNativeBinaries' && it.name != 'installJep' }.all { Task task ->
-    task.dependsOn copyJepNativeBinaries
-}
-
-// from here we use the standard Gradle build provided by Ghidra framework
-
 //----------------------START "DO NOT MODIFY" SECTION------------------------------
 def ghidraInstallDir
 

data/GhidrathonConfig.xml

@@ -3,10 +3,5 @@
     <ARRAY NAME="JAVA_EXCLUDE_LIBS" TYPE="string">
         <A VALUE="pdb" />
     </ARRAY>
-    <ARRAY NAME="PYTHON_SHARED_MODULES" TYPE="string">
-        <A VALUE="numpy" />
-    </ARRAY>
-    <ARRAY NAME="PYTHON_INCLUDE_PATHS" TYPE="string">
-    </ARRAY>
 </GHIDRATHON_CONFIG>
 

data/python/jepeval.py

@@ -1,4 +1,4 @@
-# Copyright (C) 2022 Mandiant, Inc. All Rights Reserved.
+# Copyright (C) 2024 Mandiant, Inc. All Rights Reserved.
 # Licensed under the Apache License, Version 2.0 (the "License");
 #  you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at: [package root]/LICENSE.txt

data/python/jeprunscript.py

@@ -1,4 +1,4 @@
-# Copyright (C) 2022 Mandiant, Inc. All Rights Reserved.
+# Copyright (C) 2024 Mandiant, Inc. All Rights Reserved.
 # Licensed under the Apache License, Version 2.0 (the "License");
 #  you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at: [package root]/LICENSE.txt

data/python/jepwelcome.py

@@ -1,4 +1,4 @@
-# Copyright (C) 2022 Mandiant, Inc. All Rights Reserved.
+# Copyright (C) 2024 Mandiant, Inc. All Rights Reserved.
 # Licensed under the Apache License, Version 2.0 (the "License");
 #  you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at: [package root]/LICENSE.txt

data/python/tests/hello.py

@@ -0,0 +1,16 @@
+# Run Ghidrathon CI tests.
+# @author Mike Hunhoff ([email protected])
+# @category Python 3
+# Copyright (C) 2024 Mandiant, Inc. All Rights Reserved.
+# Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at: [package root]/LICENSE.txt
+# Unless required by applicable law or agreed to in writing, software distributed under the License
+#  is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and limitations under the License.
+
+import pathlib
+import jep
+
+path = pathlib.Path("hello.txt")
+path.write_text(f"Hello from Jep {jep.__version__}", encoding="utf-8")

data/python/tests/runall.py

@@ -1,7 +1,7 @@
 # Run Ghidrathon unit tests.
-# @author Mike Hunhoff (michael.hunhoff@mandiant.com)
+# @author Mike Hunhoff (mehunhoff@google.com)
 # @category Python 3
-# Copyright (C) 2022 Mandiant, Inc. All Rights Reserved.
+# Copyright (C) 2024 Mandiant, Inc. All Rights Reserved.
 # Licensed under the Apache License, Version 2.0 (the "License");
 #  you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at: [package root]/LICENSE.txt

data/python/tests/test_cpython.py

@@ -1,4 +1,4 @@
-# Copyright (C) 2022 Mandiant, Inc. All Rights Reserved.
+# Copyright (C) 2024 Mandiant, Inc. All Rights Reserved.
 # Licensed under the Apache License, Version 2.0 (the "License");
 #  you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at: [package root]/LICENSE.txt

data/python/tests/test_jepbridge.py

@@ -1,4 +1,4 @@
-# Copyright (C) 2022 Mandiant, Inc. All Rights Reserved.
+# Copyright (C) 2024 Mandiant, Inc. All Rights Reserved.
 # Licensed under the Apache License, Version 2.0 (the "License");
 #  you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at: [package root]/LICENSE.txt

doc/building.md

@@ -0,0 +1,19 @@
+# Building Ghidrathon
+
+## Requirements
+
+Tool | Version |Source |
+|---|---|---|
+| Ghidrathon | `>= 4.0.0` | https://github.com/mandiant/Ghidrathon/releases |
+| Ghidra | `>= 10.3.2` | https://github.com/NationalSecurityAgency/ghidra/releases |
+| Java | `>= 17.0.0` | https://adoptium.net/temurin/releases/ |
+| Gradle | `>= 7.3` | https://gradle.org/releases |
+
+Use the following steps to build Ghidrathon:
+1. Download the [supported Jep JAR release](https://github.com/ninia/jep/releases/download/v4.2.0/jep-4.2.0.jar) to `<absolute_path_to_ghidrathon_source_dir>\lib`
+2. Execute gradle from `<absolute_path_to_ghidrathon_source_dir>`:
+```
+$ gradle -PGHIDRA_INSTALL_DIR=<absolute_path_to_Ghidra_install_dir>
+```
+
+The extension is stored in `<absolute_path_to_ghidrathon_source_dir>\dist`.

extension.properties

@@ -1,5 +1,5 @@
 name=Ghidrathon
 description=The FLARE team's open-source extension to add Python 3 scripting to Ghidra.
-author=Mike Hunhoff (michael.hunhoff@mandiant.com)
+author=Mike Hunhoff (mehunhoff@google.com)
 createdOn=03/31/2022
 version=@extversion@

ghidra_scripts/ghidrathon_example.py

@@ -1,8 +1,8 @@
 # Print function basic block and instruction counts.
-# @author Mike Hunhoff (michael.hunhoff@mandiant.com)
+# @author Mike Hunhoff (mehunhoff@google.com)
 # @category Python 3
 
-# Copyright (C) 2022 Mandiant, Inc. All Rights Reserved.
+# Copyright (C) 2024 Mandiant, Inc. All Rights Reserved.
 # Licensed under the Apache License, Version 2.0 (the "License");
 #  you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at: [package root]/LICENSE.txt

src/main/java/ghidrathon/GhidrathonClassEnquirer.java

@@ -1,4 +1,4 @@
-// Copyright (C) 2022 Mandiant, Inc. All Rights Reserved.
+// Copyright (C) 2024 Mandiant, Inc. All Rights Reserved.
 // Licensed under the Apache License, Version 2.0 (the "License");
 //  you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at: [package root]/LICENSE.txt