+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Skip to content
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ GHOSTS Content Servers Overview¶
+GHOSTS content servers are an evolving part of the framework. They exist for several reasons:
+-
+
- On an air-gapped network, where we are simulating some subset of the internet, we want more types of browsable content within that range — documents, spreadsheets, presentations, pdf files, streamed movies, and the like. +
- We want a broad range of URLs within a site we are representing in the range. +
- We want to simulate a document store, such as SharePoint, OneCloud, or similar, but without the hassle of installing and maintaining those actual systems. +
++Research by Global WebIndex claims that globally, 59% of the world's population uses social media, and that the average daily use is 2 hours and 29 minutes (July 2022).
+
Air-gaps¶
+Many ranges are air-gapped, which means they have no access to the wider internet. In these cases, recreating a reasonable facsimile of the internet is key to the training experience. While there are many systems that do this well, we often want to augment the scenario with a wider array of URL traffic, or we want to introduce more of certain kinds of content going across the wire. PANDORA was created to address these concerns. Shortly later, we added a social server as well.
+Having to know valid URLs¶
+The other problem is that the internet works by the client having to “know” the location of some resource via:
+-
+
- Actually knowing the URL +
- Being referred from another page - I might know google.com and search for something, which gives me a reference to another page I was not aware of previously. +
- Inferring the URL from some like resource - If one is poking around and looking for something on a server, a slight change of URL often gives hints that get you to where you wanted to go. +
- Guessing - the proliferation of .com domains means that for something new, it's often fruitful to just try that thing.com and see if it works! +
The problem here is that currently, clients must know valid URLs that actually exist out in a simulated greyspace (via TopGen, GreyBox, or otherwise), which limits the array of potential requests and creates range work to maintain. So we created GHOSTS PANDORA, which serves whatever clients ask for - if the request is for a doc file, the server creates a random doc file on the fly — in memory — and serves it back to the client. Pandora serves the following content types:
+-
+
- html +
- css +
- js +
- doc|x +
- ppt|x +
- xls|x +
- mp4 +
- gif +
- jpg +
- png +
- zip +
- msi +
- iso +
- other binary formats, etc. +
Pandora has generic request handlers for each HTTP verb (GET, POST, etc.) and is deployed as a simple docker container. We configure it to handle a particular IP on a multiple-IP-enabled host machine. It works off any URL, but part of the solution involved introducing more randomness in the GHOSTS clients as well. Those clients now support a creative parameter-built URL system that can be configured to look something like this:
+sharepoint.hello.com/{org}/{report_type}/{uuid}/{file_name}.{file_type}
+These variables are processed at runtime, and produce a final url that might look something like:
+sharepoint.hello.com/operations/maintenance/80e6af4e-5107-43b5-832f-0d8027efbd76/report.docx
+In addition, Pandora supports “bad” payloads via configuration. Here the server responds to specific configured URLs to deploy planted injects. So clients can download malware in an exercise in a manner that is hard to differentiate based on URLs already seen within the event. The configuration looks like:
+[payloads]
+1=/bad/payload/url/,some_bad.zip,application/zip
+Where id=url,payload file, MIME response type.
+ + + + + + + + +
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/content/pandora/index.html b/content/pandora/index.html
new file mode 100644
index 00000000..d8098513
--- /dev/null
+++ b/content/pandora/index.html
@@ -0,0 +1,1532 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Skip to content
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ GHOSTS PANDORA SERVER Overview¶
+
+
+Pandora is part of GHOSTS
+Pandora is within the GHOSTS Source Code Repository hosted on GitHub.
+GHOSTS PANDORA is a web server that responds to a myriad of request types with randomized content generated in real-time. Used in conjunction with GHOSTS NPCs, the two can provide for agents that are periodically downloading content other than simple HTML and associated image, CSS, and js files.
+
Running this server¶
+As a Docker Container¶
+Docker is the preferred way to run Pandora - mostly because this is how we run and test it before version releases.
+-
+
- Review the repository docker-compose.yml file +
- Run the following in your terminal +
mkdir ghosts-pandora
+cd ghosts-pandora
+curl https://raw.githubusercontent.com/cmu-sei/GHOSTS/master/src/ghosts.pandora/docker-compose.yml -o docker-compose.yml
+docker-compose up -d
+Bare metal¶
+This assumes the host server is a common Linux distribution. For images to render correctly, PIL or the more recent Pillow library is necessary. See here for more information on Pillow installation and configuration.
+-
+
- Using a Python 3 distribution >= 3.6.2 +
- In the terminal run:
pip install -r requirements.txt
+ - Then run
python app.py
+
Capabilities¶
+Handling requests by directory¶
+-
+
- /api - All requests beginning with
/apiautomatically respond with json. This includes:-
+
/api/users
+/api/user/a320f971-b3d9-4b79-bb8d-b41d02572942
+/api/reports/personnel
+
+ - /csv - All requests beginning with
/csvautomatically respond with csv. Like the above, this includes urls such as:-
+
/csv/users
+/csv/user/winx.jalton
+/csv/reports/HR/payroll
+
+ - /i, /img, /images - All requests beginning with these directories automatically respond with a random image of type [gif, jpg, png]. Examples:
-
+
/i/v1/a9f6e2b7-636c-4821-acf4-90220f091351/f8f8b1f0-9aa5-4fc7-8880-379e3192748e/small
+/images/products/184f3515-f49b-4e07-8c8b-7f978666df0e/view
+/img/432.png
+
+ - /pdf - All requests respond with a random pdf document. Examples:
-
+
/pdf/operations/SOP_Vault/a7f48bd5-84cb-43a1-8d3d-cd2c732ddff6
+/pdf/products
+
+ - /docs - All requests respond with a random word document +
- /slides - All requests respond with a random powerpoint document +
- /sheets - All requests respond with a random excel document +
Handling requests by type¶
+For requests indicating a specific file type, there are several specific handlers built to respond with that particular kind of file, such as:
+-
+
- .csv +
- Image requests [.gif, .ico, .jpg, .jpeg, .png] +
- .json +
- Office document requests +
- .doc, .docx +
- .ppt, .pptx +
- .xls, .xlsx +
So that a URL such as /users/58361185-c9f2-460f-ac45-cb845ba88574/profile.pdf would return a pdf document typically rendered right in the browser.
All unhandled request types, urls without a specific file indicator, or requests made outside specifically handled directories (from the preceding section) are returned as html, including:
+-
+
/docs/by_department/operations/users
+/blog/d/2022/12/4/blog_title-text
+/hello/index.html
+
Hiding malicious payloads for red-teaming¶
+Pandora also can hide payloads in a particular request for things like red-teaming and such. This is done in the configuration file, and looks like this:
+[payloads]
+1=/1/,a.zip,application/zip
+2=/2/users,b.zip,application/zip
+3=/3/some/report/url,c.zip,application/zip
+Each record must be an incrementing integer with no duplication. The values are:
+-
+
- The URL that this payload responds to +
- The local file (stored in
./payloads/) to be returned
+ - The MIME type of the response +
So for 1 in the example above, requests to /1/ return the a.zip file as an application/zip file.
+ + + + + + + + +
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/content/social/index.html b/content/social/index.html
new file mode 100644
index 00000000..1935f39c
--- /dev/null
+++ b/content/social/index.html
@@ -0,0 +1,1387 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Skip to content
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ GHOSTS PANDORA SOCIAL Overview¶
+
+
+PANDORA SOCIAL is still very early beta
+Here be dragons
+The place where GHOSTS agents come to share their thoughts and information.
+In the spirit of the original PANDORA, this server also responds to a very broad array of URLs but enables clients to POST/PUT/DELETE to it as well, for example:
+| Request | +Response | +
|---|---|
POST /images |
+responds with a url to the saved image file | +
POST / |
+responds with a randomly-generated streamed video | +
POST /users/michelle_smith/af2d00aa-4a89-4af3-baff-1746b556e7a1/ |
+responds with a reply to the original user's social post | +
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/core/api/index.html b/core/api/index.html
new file mode 100644
index 00000000..c13876f8
--- /dev/null
+++ b/core/api/index.html
@@ -0,0 +1,1500 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Skip to content
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ GHOSTS Core API Overview¶
+
+
+GHOSTS Source Code
+The GHOSTS Source Code Repository is hosted on GitHub.
+The GHOSTS API enables the control and orchestration of non-player characters (NPCs) within a deployment. It supports logging, reporting, and managing individual, groups of, or entire deployments of client installs.
+Installation¶
+-
+
- Install 🐳 Docker +
- Install Docker Compose +
- Run the following commands - we'll use this docker-compose.yml file +
mkdir ghosts
+cd ghosts
+curl https://raw.githubusercontent.com/cmu-sei/GHOSTS/master/src/Ghosts.Api/docker-compose.yml -o docker-compose.yml
+docker-compose up -d
+The required containers will be downloaded and configured automatically.
+Once the last command completes, if you open http://localhost:5000/api/home in your browser, you should see the initial API page outlining the version of the install, and a few test machine entries. If this page renders, your API is up, running, and available.
+You will still need to set up Grafana. Beware that you must often chown the host location of the container as listed in the docker-compose file or the container will just continually restart in error due to insufficient permissions.
Configuring the API¶
+The API generally has good defaults to get you up and running quickly, but there are some considerations in the appconfig.json file:
"ClientSettings": {
+ "OfflineAfterMinutes": 30, ...
+ "MatchMachinesBy": null,
+Can be fqdn|host|resolvedhost|null - null tells the API to match incoming requests with machine records by the machine name. For installations where multiple domains are reporting into the same API, you probably want to use FQDN in order to avoid machines being duplicated.
+"QueueSyncDelayInSeconds": 10,
+"NotificationsQueueSyncDelayInSeconds": 10,
+This is how often the synch job runs. Incoming machine requests are not real-time in order to best bundle like records together.
+Configuring Grafana¶
+-
+
- Grafana will be running (if containerized) on port 3000, and we can access it via the same URL we use for the API. +
- The default login is admin/admin. +
- The first step is to set up a datasource named "ghosts" to the ghosts Postgres database. +
- Now import your choice of the grafana json files in this repository. It creates the default GHOSTS dashboard. +
Webhooks¶
+The GHOSTS API provides webhook callbacks based on the configuration on the endpoint: /api/webhooks. The payload for creating a webhook is in the format:
{
+ "status": 0,
+ "description": "some description",
+ "postbackUrl": "http://localhost/endpoint:port",
+ "postbackMethod": 0, (0 == get, 1 == post)
+ "postbackFormat": "see below"
+}
+Payloads can be any format — here is a sample:
+{
+ 'machine':'[machinename]',
+ 'created':'[datetime.utcnow]',
+ 'type':'[messagetype]',
+ 'payload':'[messagepayload]'
+}
+On send, the payload will be converted into the correct JSON format:
+{
+ "machine":"some_guid",
+ "created":"some_datetime",
+ "type":"some_message",
+ "payload":"some_payload"
+}
+If the postback method is POST, the payload will be sent as the message body. If the postback method is GET, the payload will be sent as part of the querystring value ?message=payload.
The following events are reported via webhooks:
+-
+
- Timeline delivered (with the timeline that was delivered as payload) to a machine via API (original API posting of timeline only holds timeline in wait - the client still must check-in in order for that timeline to be delivered) +
- Machine requested updates ("checked in") from API +
- Machine posted results to API +
Troubleshooting¶
+++Is the API up and running?
+
-
+
- Go to
/api/homein the browser, it should return the current API version and the number of machines and groups under management. If it says relationship not found, restart the API application and it should create the database automatically.
+ - Run
docker ps --alland see that all containers are running normally. If one or more is not running, look at the logs for that machine viadocker logs [machine name].
+
++The ClientId, ClientResults, and other Client* endpoints are failing.
+
The Client* endpoints are for the Clients to use only. There are specific header values set by the client in the request that is used to authenticate the request. If you are not using the client, you will not have these headers set, and these endpoints will fail.
+ + + + + + + + +
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/core/api/timelines/index.html b/core/api/timelines/index.html
new file mode 100644
index 00000000..755d68ca
--- /dev/null
+++ b/core/api/timelines/index.html
@@ -0,0 +1,1479 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Skip to content
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Managing Timelines¶
+One of the primary capabilities of the API is to manage multiple client timelines. Timelines are the mechanism for clients to perform actions on behalf of a user. For example, a client might execute a timeline to browse a website, create a document, or other Handler activities.
+Clients are configured to check in periodically with the API to report their current state and to check if any updates are available. We can have the API hold an update for the next time a client, group of clients, or all clients check in. That update will then be processed by the client and the client will report back to the API that the update has been processed. This allows the API to manage the state of the client and to ensure that the client is always up to date. So the process is:
+-
+
- We configure an update for a client or group of clients in the API. +
- Clients check in and are told that an update is available. +
- The client processes the update and reports back to the API that the update has been processed. +
Configuring a Timeline for a Client¶
+Client timelines are updated via POST to /api/timelines in the following format:
+{
+ "machineId": "3fa85f64-5717-4562-b3fc-2c963f66afaf",
+ "type": 0,
+ "activeUtc": "2023-03-08T16:38:04.002Z",
+ "status": 0,
+ "update": {
+ "status": 0,
+ "timeLineHandlers": [
+ {
+ "handlerType": 0,
+ "initial": "string",
+ "utcTimeOn": {
+ "ticks": 0
+ },
+ "utcTimeOff": {
+ "ticks": 0
+ },
+ "handlerArgs": {
+ "additionalProp1": "string",
+ "additionalProp2": "string",
+ "additionalProp3": "string"
+ },
+ "loop": true,
+ "timeLineEvents": [
+ {
+ "trackableId": "string",
+ "command": "string",
+ "commandArgs": [
+ "string"
+ ],
+ "delayAfter": 0,
+ "delayBefore": 0
+ }
+ ]
+ }
+ ]
+ }
+}
+The key values to consider here are: The machine ID that you want to update, the other key value is the type of timeline. The type is an integer value that represents the type of update. The following are the types of updates that are currently supported:
-
+
- Timeline = 0 // this +
- Health = 1 +
- TimelinePartial = 10 // this the agent is currently doing off its default timeline +
- RequestForTimeline = 20 +
| ID | +Type | +Description | +
|---|---|---|
| 0 | +Timeline | +Replaces the client's default timeline stored within ./config/timeline.json | +
| 1 | +Health | +This updates a client's health instructions | +
| 10 | +TimelinePartial | +Does not replace the default timeline, rather this timeline is executed immediately on separate threads from whatever | +
| 20 | +RequestForTimeline | +Use this to instruct the client to send its current default timeline up to the API | +
The remainder of the settings are for the timeline - basically what we are doing here is sending a client a new timeline, with the above values to indicate, which machine and what to change in the update node.
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/core/client/index.html b/core/client/index.html
new file mode 100644
index 00000000..d5181a9b
--- /dev/null
+++ b/core/client/index.html
@@ -0,0 +1,1733 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Skip to content
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ GHOSTS Core Client Overview¶
+
+
+GHOSTS Source Code
+The GHOSTS Source Code Repository is hosted on GitHub.
+The GHOSTS client simulates what anyone might do at a computer given their particular role or membership within some team. It creates documents, browses websites, downloads files, and uses all sorts of popular applications on many versions of Windows and Linux machines. Whether you're a friendly administrator or a powerful cyber adversary, GHOSTS can replicate your expected behavior.
+
GHOSTS has many use cases in cyber training and exercises, most notably for bringing non-player characters (NPCs) to life, but it can also be used for other purposes where realistic activity on a computer or network is needed - testing or generating datasets, for example.
+
+
+Do I need the API?
+Although clients are fully functional without the API server, the latter enables logging, reports, and remote administration capabilities. Without the API, changes to clients must be managed in some other manner.
+The client's configuration and command system are file-based, so if you do not wish to use the API, you can manage these through some other tool, such as Ansible or similar. All of these files are under the client's install location.
+
+
+Run as the NPC you're representing
+The GHOSTS client should typically be run as a user, and not as administrator or root - training or exercising teams will notice this immediately.
+The Windows Client¶
+GHOSTS on Windows (Win10 and Win7 currently supported) is a .NET Console application that performs user activity on client workstations - web browsing, working with office documents, using the command prompt or PowerShell, etc. Our team typically installs the client in an out-of-game directory (c:\exercise\ghosts\), where no event or injects will originate. It is recommended you verify one working client before deploying to multiple machines. You will need the base URL from the API installation for the client to communicate home.
Windows Installation¶
+-
+
-
+
Your client Windows machine will need to have (at least) the Microsoft DotNet 4.6.1 runtime installed . Again, note that you only need the runtime, not the full SDK. We continue to use 4.6.1 on Windows to maintain backward compatibility.
+
+ - + + +
Unzip to your client machine in a directory such as c:\exercise\ghosts. You will need to adjust the configuration in config/application.json for your client to talk to your an installed instance of the GHOSTS API server.
+
+Additional configuration required for web browsing
+For any client utilizing the Firefox or Chrome web browser, an automation driver is necessary to be included in the same folder as the GHOSTS binary. For Firefox, download the appropriate 🦎 Geckodriver for your version of the browser here . For Chrome, download the appropriate Chromedriver for your version of the browser here .
+
+
+Additional configuration required for email
+Using the Windows client email functions requires the use of Redemption which provides robust Outlook automation. The full Redemption library should be found in /lib.
Linux Client¶
+Your client Linux machine will need to have the latest Microsoft dotnetcore runtime installed (Note that for the GHOSTS client, there are versions for dotnetcoreapp3.1 - this will eventually go away — and dotnet6.0, which is long term support (LTS) and will stick around for a while). Again, note that you only need the runtime installed, not the full SDK.
+Linux Installation¶
+ +Unzip to a folder such as ~/ghosts for the user that you want GHOSTS to run as.
Note that on Linux machines running the client as root and utilizing web browsing may result in failures due to Gecko/Chromedriver display issues.
+Client Directory Structure¶
+
+
+Do not copy the instance folder
+You should never copy the instance folder from one machine to another.
| Folder | +Description | +
|---|---|
config/ |
+configuration files are stored here. | +
instance/ |
+generated files and information relative to this particular installed instance of ghosts is stored here. This folder should never be copied from one machine to another | +
lib/ |
+third-party libraries used by ghosts are stored here | +
logs/ |
+output logs for the installed instance (logs/app.log), and logs that are transferred to the server (logs/clientupdates.log) |
+
Configuration Quick Start¶
+After unzipping the GHOSTS client, we can simply double-click it to run. Note that the console window likely printed a few messages, but then disappeared. This is normal, in production mode GHOSTS hides itself. We can see it in the Windows Task Manager, however, and we can kill the process from there. We can also run the included kill-ghosts.bat file that closes the application and any applications it might control.
Beyond this initial step of verifying that the client will run, there are two files that we might need to adjust to fit many deployments:
+application.json¶
+In this file, often all we need to change are the URLs for the API, IdUrl, ClientResultsUrl, ClientUpdatesUrl, and the like. Change the hostname to your installed API location, and GHOSTS should check in as expected.
+{
+ "IdEnabled": true, //enabled in order to command and control from api (C2) server
+ "IdUrl": "http://yourapiurl.com/api/clientid", //url for API endpoint to get clientid
+ "ClientResultsEnabled": true, //enabled to report results to C2
+ "ClientResultsUrl": "http://yourapiurl.com/api/clientresults", //url for API endpoint to report results
+ "ClientResultsCycleSleep": 90, //report results every x ms
+ "ClientUpdatesEnabled": true, //enabled to get updates from C2
+ "ClientUpdatesUrl": "http://yourapiurl.com/api/clientupdates", //url for API endpoint to get updates
+ "ClientUpdatesCycleSleep": 90, //check for updates every x ms
+ "Survey": { //survey is a local report of processes running, etc.
+ "IsEnabled": true, //on/off
+ "Frequency": "once", //how often to survey
+ "MaxAgeInHours": 168, //how long to wait until new survey
+ "OutputFormat": "indent" //compact/fancy(indent)
+ },
+ "HealthIsEnabled": true, //enable local health checks
+ "HandlersIsEnabled": true, //enable local timeline activity
+ "ChromeExtensions": "", //comma separated local extensions (used for injects in the past)
+ "FirefoxInstallLocation": "", //geckodriver needs this for non-standard installs (is pesky)
+ "FirefoxMajorVersionMinimum": 48, //geckodriver is picky about versions
+ "OfficeDocsMaxAgeInHours": 6, //cleanup kills docs in the documents folder older than this setting
+}
+timeline.json¶
+The other file we may want to adjust is the default timeline. This is what the agent does all day, including browsing the internet, creating documents, and similar. The defaults hopefully give you a good idea of what is possible, and of course, the array of configurations here is endless - be creative!
+The primary item is the HandlerType. This tells GHOSTS to run a command (Command), use Firefox to browse an array of websites (BrowserFirefox), create Excel documents (Excel) and so on. Some of the other items related to a handler's configuration are:
+-
+
- Initial: The initial command for a handler to execute. For a web browser, you might enter either a URL or "about:blank". +
- UtcTimeOn | UtcTimeOff: "00:00:00": "24:00:00" to not shut off. Otherwise, enter an on and an off time to simulate things such as office hours of 9-5, etc. There are 30 minutes of jitter plus or minus from the time entered. +
- Loop: Set this to true to continue to execute this same command on a loop, or false to execute something just one time. +
{
+ "HandlerType": "Command",
+ "Initial": "",
+ "UtcTimeOn": "00:00:00",
+ "UtcTimeOff": "24:00:00",
+ "Loop": "True",
+ "TimeLineEvents": [
+ {
+ "Command": "NETSTAT",
+ "CommandArgs": [],
+ "DelayAfter": 900000,
+ "DelayBefore": 0
+ }
+ ]
+}
+To access a network share file, the command might be: net use X:\SERVER\Share
+To RDP to another machine: mstsc.exe {ConnectionFile | /v:ServerName[:Port]} [/console] [/f] [/w:Width/h:Height]
+/v - specifies the remote computer and port (optional) you wish to connect to
+/console – connects to the console of a Windows Server 2003 based system
+/f – starts the remote desktop connection in full screen mode
+/w & /h – specifies the width and height of the remote desktop connection
Actions can also be created for standard copy/move/deletion of files via their respective commands.
+Chrome
+We have to pass the browser window an initial value. If we don't want it to go anywhere at start, we could pass about:blank, otherwise we'd pass a url. These can be http or https.
+{
+ "HandlerType": "BrowserChrome",
+ "Initial": "http://google.com",
+ "UtcTimeOn": "00:00:00",
+ "UtcTimeOff": "24:00:00",
+ "Loop": "True",
+ "TimeLineEvents": [
+ {
+ "Command": "random",
+ "CommandArgs": [
+ "http://google.com",
+ "http://facebook.com",
+ ],
+ "DelayAfter": 1000,
+ "DelayBefore": 0
+ }
+ ]
+}
+Excel, PowerPoint, Word
+{
+ "HandlerType": "Word",
+ "Initial": "",
+ "UtcTimeOn": "00:00:00",
+ "UtcTimeOff": "24:00:00",
+ "Loop": "True",
+ "TimeLineEvents": [
+ {
+ "Command": "create",
+ "CommandArgs": [ "%homedrive%%homepath%\\Documents" ],
+ "DelayAfter": 900000,
+ "DelayBefore": 0
+ }
+ ]
+}
+Trackables¶
+For specific Timeline Events where the outcome is needed to be tracked, like for example, a client machine spawned inject, use a Trackable (via TrackableId in the following example):
+{
+ "TimeLineHandlers": [
+ {
+ "HandlerType": "BrowserChrome",
+ "Initial": "about:blank",
+ "UtcTimeOn": "00:00:00",
+ "UtcTimeOff": "24:00:00",
+ "Loop": false,
+ "TimeLineEvents": [
+ {
+ "Command": "browse",
+ "CommandArgs": [ "https://dl.dafont.com/dl/?f=italian_breakfast" ],
+ "DelayAfter": 0,
+ "DelayBefore": 0
+ },
+ {
+ "Command": "download",
+ "CommandArgs": [ "//a[contains(@class, 'dl')]" ],
+ "TrackableId": "<guid id from trackables table/>",
+ "DelayAfter": 0,
+ "DelayBefore": 0
+ }
+ ]
+ },
+ {
+ "HandlerType": "Command",
+ "Initial": "",
+ "UtcTimeOn": "00:00:00",
+ "UtcTimeOff": "24:00:00",
+ "Loop": false,
+ "TimeLineEvents": [
+ {
+ "Command": "cd %homedrive%%homepath%\\Downloads",
+ "CommandArgs": [
+ "powershell expand-archive -Path italian_breakfast.zip -destinationpath x",
+ "cd x",
+ "dir"
+ ],
+ "TrackableId": "<guid id from trackables table/>",
+ "DelayAfter": 10,
+ "DelayBefore": 10000
+ }
+ ]
+ }
+ ]
+}
+Troubleshooting¶
+++Clients aren't running (immediately exiting, throwing copious exceptions, or similar)
+
-
+
- Is the dotnet framework runtime 4.x installed on the machine? +
- If GPO is doing white-listing of what apps can run, is
ghosts.exewhite-listed?
+ - Will the client run by simply double-clicking on the exe? +
- Does it report anything to the windows application event logs? +
- What's in
logs/app.log?
+ - Is the ghosts executable set to execute automatically when the machine restarts? +
++Clients aren't reporting their activity to the API
+
-
+
- Is the client running correctly? (if not, see above) +
- Is there entries in the logs/clientupdates.log? +
- If there are, is the file too large? (Try removing it, ghosts might be hung trying to process a lot of log data) +
- If the folder
instancecreated? Does the fileinstance/id.jsonexist? (If it does and has an ID within, then ghosts has reported home to the api at least once)
+ - The file
logs/app.logindicating any fatal issues? (Logging can be ratcheted up and down via nlog configuration)
+
++Can I update what clients are doing?
+
-
+
- Clients operate off their
config/timeline.jsonfile and this can be updated via Powershell, Ansible, or other means - it's just a file.
+ - Clients can also do just-in-time activities via the
instance/timeline/infolder. Anything placed here will be picked up, executed, and moved to the corresponding out folder once complete. This does not affect any activity currently controlled with the default timeline file.
+
++Can I reset a client on a box?
+
-
+
- Yes, launching a new instance of Ghosts kills the previous one and all associated tasks from the timeline (any instances of Word, PowerShell, etc.). Only one instance of Ghosts will be running on a client box at any time. We can also run the .bat script
kill-ghosts.batincluded in the distribution to clean everything up.
+
++What is the easiest way to determine the running version of the client?
+
-
+
- run the version flag:
ghosts.exe --version
+
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/core/handlers/blog_helper/index.html b/core/handlers/blog_helper/index.html
new file mode 100644
index 00000000..8fb41ce7
--- /dev/null
+++ b/core/handlers/blog_helper/index.html
@@ -0,0 +1,1358 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Blogs/Drupal
+ + + + + + + + + + +
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/core/handlers/browser/index.html b/core/handlers/browser/index.html
new file mode 100644
index 00000000..0b6638fd
--- /dev/null
+++ b/core/handlers/browser/index.html
@@ -0,0 +1,1418 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Skip to content
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Web Browser (Firefox|Chrome) Configuration¶
+A sample timeline for each browser is available in a code repository directory of the same name.
+Timeline URL Variables¶
+An example of a URL using variables is:
+http://craigslist.org/{org}/{group}/{uuid}/{verb}/{type}/{n}?{c}={now}
+All variables are in the format {variable}. There are several standard variables:
-
+
{now}= short datetime (mm/dd/yyyy format)
+{uuid}= uuid
+{c}= a single character from a-z and A-Z
+{n}= number between 1 and 1000
+
We can also configure additional variables by adding the following to a browser handler in a timeline:
+"url-replace": [
+ {"verb": ["order", "enable", "engage"]},
+ {"group": ["operations", "logistics", "medical"]},
+ {"org": ["army", "command", "brigade", "battalion"]},
+ {"type": ["document", "doc", "files", "vault", "filevault"]}
+ ]
+Therefore:
+https://www.cmu.edu/{org}/{group}/{verb}/{type}/{uuid}/version_{n}?{c}={now}
+might be rendered as:
+https://www.cmu.edu/command/operations/order/doc/bcc396b5-47d0-4665-93c8-0a314cec13e1/version_55?d=6/21/2022
+
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/core/handlers/clicks/index.html b/core/handlers/clicks/index.html
new file mode 100644
index 00000000..21c67203
--- /dev/null
+++ b/core/handlers/clicks/index.html
@@ -0,0 +1,1356 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Clicks
+ + + + + + + + + + +
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/core/handlers/cmd/index.html b/core/handlers/cmd/index.html
new file mode 100644
index 00000000..0dbcee7a
--- /dev/null
+++ b/core/handlers/cmd/index.html
@@ -0,0 +1,1356 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Cmd
+ + + + + + + + + + +
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/core/handlers/excel/index.html b/core/handlers/excel/index.html
new file mode 100644
index 00000000..5d8d3ec8
--- /dev/null
+++ b/core/handlers/excel/index.html
@@ -0,0 +1,1363 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Skip to content
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Excel Configuration¶
+ + + + + + + + +
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/core/handlers/index.html b/core/handlers/index.html
new file mode 100644
index 00000000..e92f171e
--- /dev/null
+++ b/core/handlers/index.html
@@ -0,0 +1,1406 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Skip to content
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Basic Handler Configuration¶
+The following is the format for a basic timeline handler:
+"TimeLineHandlers": [
+ {
+ "HandlerType": "Watcher",
+ "UtcTimeOn": "00:00:00",
+ "UtcTimeOff": "24:00:00",
+ "Loop": true,
+ "TimeLineEvents": [
+ {
+ "Command": "folder",
+ "CommandArgs": [ "path:%HOMEDRIVE%%HOMEPATH%\\Downloads", "size:2000", "deletionApproach:oldest" ],
+ "DelayAfter": 0,
+ "DelayBefore": 0
+ }
+ ]
+ }
+]
+Some of the key-value pairs are self-explanatory, but let's review a few important ones:
+| Key | +Value | +
|---|---|
HandlerType |
+The application or major function we want to control. This could be FireFox, the Cmd terminal, or Word. | +
UtcTimeOn |
+The time the handler will begin to run. To simulate an agent coming into the office at 0900, we set this to your time zone's UTC value. | +
UtcTimeOff |
+The time the handler will stop. To simulate an agent leaving the office at 1700, we set this to your time zone's UTC value. | +
Loop |
+true or false - a handler could be repeated or just run one-time. | +
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/core/handlers/notepad/index.html b/core/handlers/notepad/index.html
new file mode 100644
index 00000000..5f349b73
--- /dev/null
+++ b/core/handlers/notepad/index.html
@@ -0,0 +1,1356 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Notepad
+ + + + + + + + + + +
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/core/handlers/npc_system/index.html b/core/handlers/npc_system/index.html
new file mode 100644
index 00000000..262c1f3f
--- /dev/null
+++ b/core/handlers/npc_system/index.html
@@ -0,0 +1,1356 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ NPCSystem
+ + + + + + + + + + +
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/core/handlers/outlook/index.html b/core/handlers/outlook/index.html
new file mode 100644
index 00000000..1779df62
--- /dev/null
+++ b/core/handlers/outlook/index.html
@@ -0,0 +1,1358 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Outlook
+ + + + + + + + + + +
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/core/handlers/pidgin/index.html b/core/handlers/pidgin/index.html
new file mode 100644
index 00000000..6df7c13a
--- /dev/null
+++ b/core/handlers/pidgin/index.html
@@ -0,0 +1,1356 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Pidgin
+ + + + + + + + + + +
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/core/handlers/powerpoint/index.html b/core/handlers/powerpoint/index.html
new file mode 100644
index 00000000..d5e70df0
--- /dev/null
+++ b/core/handlers/powerpoint/index.html
@@ -0,0 +1,1363 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Skip to content
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ PowerPoint Configuration¶
+ + + + + + + + +
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/core/handlers/print/index.html b/core/handlers/print/index.html
new file mode 100644
index 00000000..e6075276
--- /dev/null
+++ b/core/handlers/print/index.html
@@ -0,0 +1,1356 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Print
+ + + + + + + + + + +
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/core/handlers/rdp/index.html b/core/handlers/rdp/index.html
new file mode 100644
index 00000000..579f2cfb
--- /dev/null
+++ b/core/handlers/rdp/index.html
@@ -0,0 +1,1356 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ RDP
+ + + + + + + + + + +
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/core/handlers/reboot/index.html b/core/handlers/reboot/index.html
new file mode 100644
index 00000000..4e3e1edb
--- /dev/null
+++ b/core/handlers/reboot/index.html
@@ -0,0 +1,1356 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Reboot
+ + + + + + + + + + +
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/core/handlers/sftp/index.html b/core/handlers/sftp/index.html
new file mode 100644
index 00000000..b17aaaa1
--- /dev/null
+++ b/core/handlers/sftp/index.html
@@ -0,0 +1,1356 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Sftp
+ + + + + + + + + + +
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/core/handlers/sharepoint_helper/index.html b/core/handlers/sharepoint_helper/index.html
new file mode 100644
index 00000000..7f81640d
--- /dev/null
+++ b/core/handlers/sharepoint_helper/index.html
@@ -0,0 +1,1358 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Sharepoint
+ + + + + + + + + + +
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/core/handlers/ssh/index.html b/core/handlers/ssh/index.html
new file mode 100644
index 00000000..31df3349
--- /dev/null
+++ b/core/handlers/ssh/index.html
@@ -0,0 +1,1356 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Ssh
+ + + + + + + + + + +
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/core/handlers/watcher/index.html b/core/handlers/watcher/index.html
new file mode 100644
index 00000000..fcfaa382
--- /dev/null
+++ b/core/handlers/watcher/index.html
@@ -0,0 +1,1356 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Watcher
+ + + + + + + + + + +
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/core/handlers/wmi/index.html b/core/handlers/wmi/index.html
new file mode 100644
index 00000000..52f93b56
--- /dev/null
+++ b/core/handlers/wmi/index.html
@@ -0,0 +1,1356 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Wmi
+ + + + + + + + + + +
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/core/handlers/word/index.html b/core/handlers/word/index.html
new file mode 100644
index 00000000..3da8e623
--- /dev/null
+++ b/core/handlers/word/index.html
@@ -0,0 +1,1363 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Skip to content
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Word Configuration¶
+ + + + + + + + +
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/index.html b/index.html
new file mode 100644
index 00000000..4b54d2da
--- /dev/null
+++ b/index.html
@@ -0,0 +1,1457 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Skip to content
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ The GHOSTS NPC Framework¶
+Developed by Carnegie Mellon University's Software Engineering Institute (SEI), The GHOSTS Framework is an open-source (OSS) software framework that enables creating, deploying, and orchestrating complex non-player character (NPC) activity within training, exercise, simulation, and educational environments. Within GHOSTS there are several systems and applications, all of which are modular and extendable to fit a wide array of use cases and needs.
+If you've never seen GHOSTS in action, watch this quick three-minute introductory video on YouTube:
+ + +Documentation¶
+This is the GHOSTS documentation site for the framework and all of its components. Each major component's detail is accessible from the main navigation. If anything is unclear or you still have questions, please do not hesitate to start a discussion — our community is growing and eager to help!
+Cyber Ranges and Crucible¶
+GHOSTS is typically run on machines within a virtualized network, often referred to as "the range". This network can be as simple or as complex as required for training and exercise purposes.
+CERT's Cyber Workforce Development (CWD) team has a great deal of experience in building cyber ranges for training and exercise, captured in our technical report Foundation of Cyber Ranges. The report details the design considerations and execution plan for building high-fidelity, realistic virtual cyber ranges that deliver maximum training and exercise value for cyberwarfare participants.
+
+
+Run GHOSTS on the Crucible Framework
+
+GHOSTS runs very well within the SEI's Crucible Framework (Source Code and Docs) - which we use extensively for training and exercise here at Carnegie Mellon University (CMU)'s Software Engineering Institute, and particularly, within CERT.
Building a sufficiently complex range for training and exercise purposes is often challenging. Crucible is a modular framework for creating, deploying, and managing virtual environments to support training, education, and exercises. Crucible is designed to be easy to use, extensible, and customizable to meet the needs of a wide variety of use cases.
+Philosophy¶
+GHOSTS evolved in our quest to create more realistic NPCs within cyberwarfare training and exercise. In 2018, we outlined our thoughts in a technical report entitled GHOSTS in the Machine: A Framework for Cyber-Warfare Exercise NPC Simulation.1 In that report, we outline how the GHOSTS framework accomplishes the creation of simulations in which NPCs realistically represent a vast array of possible encounters and outcomes. We have continued to follow our initial path since. The litmus has always been that if we looked over someone's shoulder while they were using a computer, that is what GHOSTS should look like.2
+Since then the framework has expanded to include tools that serve content in simulated environments, create NPCs with details about them that we can programmatically use to drive their decision-making, and machine learning agent preference engines. If it is related to replicating human behavior within a cyber training or exercise scenario, we are interested in how GHOSTS can contribute within that space.
+Reporting Bugs¶
+Found a bug? Please report all bugs - including bugs for the individual components - in the cmu-sei/ghosts issue tracker. Include as much detail as possible including steps to reproduce, the specific components involved, and any error messages you may have found.
+Requesting Features¶
+Have a good idea for a new feature? Submit all new feature requests through the cmu-sei/ghosts issue tracker. Include the reasons why you're requesting the new feature and how it might benefit other users.
+License¶
+[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution.
+Copyright 2018 Carnegie Mellon University. See the LICENSE.md file for details.
+
+
+
+
+
+
+
+
+
+
+
+ +
-
+
-
+
This paper is influenced by our previous paper R-EACTR: A Framework for Designing Realistic Cyber Warfare Exercises which outlines a design framework for cyber warfare exercises. It ensures that designs of team-based exercises factor realism into all aspects of the participant experience. Both of these papers are natural extensions to The CERT Approach to Cybersecurity Workforce Development. ↩
+
+ -
+
There is also a GHOSTS video presentation from FloCon 2021 that provides a general introduction to the framework. ↩
+
+
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/new/index.html b/new/index.html
new file mode 100644
index 00000000..a1ec12f3
--- /dev/null
+++ b/new/index.html
@@ -0,0 +1,1415 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Skip to content
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ What's New¶
+Welcome to what's new in the GHOSTS framework. Use this page to review the latest changes.
+GHOSTS Core v7.0¶
+-
+
- 10+ significant features built by 6+ contributors! +
- We surpassed 175 issues/discussions! +
- These new docs are hosted on GitHub Pages +
- GHOSTS PANDORA - a new tool for delivering randomized content within a range +
- GHOSTS PANDORA SOCIAL - a new tool for managing social media content within a range +
In the Core Client, v7 Improves:
+-
+
- Performance +
- The Clean up of created artifacts +
- Logging +
- Cron-like scheduling (in development for all handlers) +
- New handlers for Firefox and Chrome Browsers +
- Complete forms +
- Post payloads (images, files, etc.) +
- Better UA string handling +
- SharePoint +
- Drupal Blog Management +
- Jabber (XMPP) +
- RDP +
- sFTP +
- SSH +
- Client Now Supports AutoIT +
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/quickstart/index.html b/quickstart/index.html
new file mode 100644
index 00000000..35d98940
--- /dev/null
+++ b/quickstart/index.html
@@ -0,0 +1,1454 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Skip to content
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Quick Start¶
+
+
+No Compilers Necessary
+This section details the installation and configuration of GHOSTS from precompiled binaries for both the server and the client.
+If you are just checking out the project for the first time and want to see NPCs performing activities, skip to the client section below. Otherwise, it is easier to install the API first and then a client that will connect to that installed API instance.
+GHOSTS API Server¶
+Easy installation requires:
+-
+
- Install 🐳 Docker +
- Install Docker Compose +
-
+
We'll use this docker-compose.yml file in the following command block which will download the required containers will automatically. The commands to stand up the GHOSTS API containers is:
++mkdir ghosts-api +cd ghosts-api +curl https://raw.githubusercontent.com/cmu-sei/GHOSTS/master/src/Ghosts.Api/docker-compose.yml -o docker-compose.yml +docker-compose up -d +
+ -
+
Once the command completes, if you open http://localhost:5000/api/home in your browser, you should see the initial API page outlining the version of the install, and a few test machine entries. If this page renders, your API is up, running, and available. If the page does not render, follow the advice in the API troubleshooting section.
+
+
You will still need to set up Grafana. Beware that you must often chown g_data, which is the host location for the Graphana container as listed in the docker-compose file. Otherwise, the Grafana container will just continually restart in error due to insufficient permissions (detailed in API troubleshooting).
GHOSTS Clients¶
+For any of the clients utilizing the browser, an automation driver is necessary to be included in the same folder as the GHOSTS binary. For Firefox, download the appropriate 🦎 Geckodriver for your version of the browser here. For Chrome, download the appropriate Chromedriver for your version of the browser here.
+Note there are additional configuration steps for Outlook email automation.
+The GHOSTS client should typically be run as a specific user, and not as an administrator or root account.
+Windows Client¶
+-
+
- Your client machine will need to have (at least) the Microsoft DotNet 4.6.1 runtime installed. You do not need the full SDK. +
- Download the latest Windows client. +
- Unzip to your client machine in a directory such as
c:\exercise\ghosts. You will need to adjust the configuration inconfig/application.jsonfor your client to talk to the already installed API server from above.
+
Linux Client¶
+-
+
- Your client Linux machine will need to have the latest Microsoft dotnetcore runtime installed (Note that for the GHOSTS client, there are versions for dotnetcoreapp3.1 - this will eventually go away — and dotnet6.0, which is LTS and should stick around for a while). Again, note that you only need the runtime installed, not the full SDK. +
- Download the latest Linux client zip file. Unzip to a folder such as ~/ghosts for the user that you want GHOSTS to run as. +
Note that on Linux machines running the client as root and utilizing web browsing may result in failures due to Gecko/Chromedriver display issues.
+ + + + + + + + +
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/search/search_index.json b/search/search_index.json
new file mode 100644
index 00000000..e3efcca3
--- /dev/null
+++ b/search/search_index.json
@@ -0,0 +1 @@
+{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"The GHOSTS NPC Framework","text":"Developed by Carnegie Mellon University's Software Engineering Institute (SEI), The GHOSTS Framework is an open-source (OSS) software framework that enables creating, deploying, and orchestrating complex non-player character (NPC) activity within training, exercise, simulation, and educational environments. Within GHOSTS there are several systems and applications, all of which are modular and extendable to fit a wide array of use cases and needs.
If you've never seen GHOSTS in action, watch this quick three-minute introductory video on YouTube:
"},{"location":"#documentation","title":"Documentation","text":"This is the GHOSTS documentation site for the framework and all of its components. Each major component's detail is accessible from the main navigation. If anything is unclear or you still have questions, please do not hesitate to start a discussion \u2014 our community is growing and eager to help!
"},{"location":"#cyber-ranges-and-crucible","title":"Cyber Ranges and Crucible","text":"GHOSTS is typically run on machines within a virtualized network, often referred to as \"the range\". This network can be as simple or as complex as required for training and exercise purposes.
CERT's Cyber Workforce Development (CWD) team has a great deal of experience in building cyber ranges for training and exercise, captured in our technical report Foundation of Cyber Ranges. The report details the design considerations and execution plan for building high-fidelity, realistic virtual cyber ranges that deliver maximum training and exercise value for cyberwarfare participants.
Run GHOSTS on the Crucible FrameworkGHOSTS runs very well within the SEI's Crucible Framework (Source Code and Docs) - which we use extensively for training and exercise here at Carnegie Mellon University (CMU)'s Software Engineering Institute, and particularly, within CERT.
Building a sufficiently complex range for training and exercise purposes is often challenging. Crucible is a modular framework for creating, deploying, and managing virtual environments to support training, education, and exercises. Crucible is designed to be easy to use, extensible, and customizable to meet the needs of a wide variety of use cases.
"},{"location":"#philosophy","title":"Philosophy","text":"GHOSTS evolved in our quest to create more realistic NPCs within cyberwarfare training and exercise. In 2018, we outlined our thoughts in a technical report entitled GHOSTS in the Machine: A Framework for Cyber-Warfare Exercise NPC Simulation.1 In that report, we outline how the GHOSTS framework accomplishes the creation of simulations in which NPCs realistically represent a vast array of possible encounters and outcomes. We have continued to follow our initial path since. The litmus has always been that if we looked over someone's shoulder while they were using a computer, that is what GHOSTS should look like.2
Since then the framework has expanded to include tools that serve content in simulated environments, create NPCs with details about them that we can programmatically use to drive their decision-making, and machine learning agent preference engines. If it is related to replicating human behavior within a cyber training or exercise scenario, we are interested in how GHOSTS can contribute within that space.
"},{"location":"#reporting-bugs","title":"Reporting Bugs","text":"Found a bug? Please report all bugs - including bugs for the individual components - in the cmu-sei/ghosts issue tracker. Include as much detail as possible including steps to reproduce, the specific components involved, and any error messages you may have found.
"},{"location":"#requesting-features","title":"Requesting Features","text":"Have a good idea for a new feature? Submit all new feature requests through the cmu-sei/ghosts issue tracker. Include the reasons why you're requesting the new feature and how it might benefit other users.
"},{"location":"#license","title":"License","text":"[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution.
Copyright 2018 Carnegie Mellon University. See the LICENSE.md file for details.
-
This paper is influenced by our previous paper R-EACTR: A Framework for Designing Realistic Cyber Warfare Exercises which outlines a design framework for cyber warfare exercises. It ensures that designs of team-based exercises factor realism into all aspects of the participant experience. Both of these papers are natural extensions to The CERT Approach to Cybersecurity Workforce Development.\u00a0\u21a9
-
There is also a GHOSTS video presentation from FloCon 2021 that provides a general introduction to the framework.\u00a0\u21a9
Welcome to what's new in the GHOSTS framework. Use this page to review the latest changes.
"},{"location":"new/#ghosts-core-v70","title":"GHOSTS Core v7.0","text":"- 10+ significant features built by 6+ contributors!
- We surpassed 175 issues/discussions!
- These new docs are hosted on GitHub Pages
- GHOSTS PANDORA - a new tool for delivering randomized content within a range
- GHOSTS PANDORA SOCIAL - a new tool for managing social media content within a range
In the Core Client, v7 Improves:
- Performance
- The Clean up of created artifacts
- Logging
- Cron-like scheduling (in development for all handlers)
- New handlers for Firefox and Chrome Browsers
- Complete forms
- Post payloads (images, files, etc.)
- Better UA string handling
- SharePoint
- Drupal Blog Management
- Jabber (XMPP)
- RDP
- sFTP
- SSH
- Client Now Supports AutoIT
This section details the installation and configuration of GHOSTS from precompiled binaries for both the server and the client.
If you are just checking out the project for the first time and want to see NPCs performing activities, skip to the client section below. Otherwise, it is easier to install the API first and then a client that will connect to that installed API instance.
"},{"location":"quickstart/#ghosts-api-server","title":"GHOSTS API Server","text":"Easy installation requires:
- Install \ud83d\udc33 Docker
- Install Docker Compose
-
We'll use this docker-compose.yml file in the following command block which will download the required containers will automatically. The commands to stand up the GHOSTS API containers is:
mkdir ghosts-api\ncd ghosts-api\ncurl https://raw.githubusercontent.com/cmu-sei/GHOSTS/master/src/Ghosts.Api/docker-compose.yml -o docker-compose.yml\ndocker-compose up -d\n -
Once the command completes, if you open http://localhost:5000/api/home in your browser, you should see the initial API page outlining the version of the install, and a few test machine entries. If this page renders, your API is up, running, and available. If the page does not render, follow the advice in the API troubleshooting section.
You will still need to set up Grafana. Beware that you must often chown g_data, which is the host location for the Graphana container as listed in the docker-compose file. Otherwise, the Grafana container will just continually restart in error due to insufficient permissions (detailed in API troubleshooting).
For any of the clients utilizing the browser, an automation driver is necessary to be included in the same folder as the GHOSTS binary. For Firefox, download the appropriate \ud83e\udd8e Geckodriver for your version of the browser here. For Chrome, download the appropriate Chromedriver for your version of the browser here.
Note there are additional configuration steps for Outlook email automation.
The GHOSTS client should typically be run as a specific user, and not as an administrator or root account.
"},{"location":"quickstart/#windows-client","title":"Windows Client","text":"- Your client machine will need to have (at least) the Microsoft DotNet 4.6.1 runtime installed. You do not need the full SDK.
- Download the latest Windows client.
- Unzip to your client machine in a directory such as
c:\\exercise\\ghosts. You will need to adjust the configuration inconfig/application.jsonfor your client to talk to the already installed API server from above.
- Your client Linux machine will need to have the latest Microsoft dotnetcore runtime installed (Note that for the GHOSTS client, there are versions for dotnetcoreapp3.1 - this will eventually go away \u2014 and dotnet6.0, which is LTS and should stick around for a while). Again, note that you only need the runtime installed, not the full SDK.
- Download the latest Linux client zip file. Unzip to a folder such as ~/ghosts for the user that you want GHOSTS to run as.
Note that on Linux machines running the client as root and utilizing web browsing may result in failures due to Gecko/Chromedriver display issues.
"},{"location":"advanced/","title":"GHOSTS Advanced Features Overview","text":"The SEI is a research institute, and so we often are thinking about how to use GHOSTS in new ways to drive insight for our customers. Some of this work makes it here, with the caveat that it might be early beta editions or require some amount of engineer hand-holding in its current state. Some call it pre-release, we call it advanced features.
"},{"location":"advanced/cyclone/","title":"GHOSTS CYCLONE Overview","text":"UnreleasedComing soon
"},{"location":"advanced/enchanter/","title":"GHOSTS ENCHANTER Overview","text":"UnreleasedComing soon
"},{"location":"advanced/necromancer/","title":"GHOSTS NECROMANCER Overview","text":"UnreleasedComing soon
"},{"location":"animator/","title":"GHOSTS ANIMATOR Overview","text":"GHOSTS ANIMATOR Source CodeThe GHOSTS ANIMATOR Source Code Repository is hosted on GitHub
Animator brings NPCs to life in two ways:
-
Initial Creation
Animator creates the initial NPC profile, including details such as name, address, career, finances, and family members. Based on configuration, it can place users in a multi-level organizational structure, and establish relationships between users.
-
Animation Jobs
Via jobs that can be run during training and exercise events, Animator can update the NPC's preferences, beliefs, and relationships. This enables dynamic NPCs that change over time.
At its core, Animator is a realistic user detail generator. Its primary function is to create sufficiently realistic identities and accompanying verbose portfolios of personal information. Each generated user, or NPC (Non-Player Character) as we call them, has numerous categories of details associated with them, and a great deal of metadata that define who they are. Each piece of information is generated using sourced datasets in an attempt to distribute characteristics realistically. We like to say it creates, \"NPCs so real, they sell for a premium on the dark web.\"1
"},{"location":"animator/#quick-start","title":"Quick Start","text":"git clone <https://github.com/cmu-sei/GHOSTS-ANIMATOR>\ncd ghosts-animator/src\ndocker build . -t ghosts/animator\ndocker compose up -d\nor if you don't want to build and just run the latest docker-compose file:
mkdir ghosts-animator\ncd ghosts-animator\ncurl https://github.com/cmu-sei/GHOSTS-ANIMATOR/blob/master/src/docker-compose.yml -o docker-compose.yml\ndocker compose up -d\nNow browse to http://localhost:5000/
"},{"location":"animator/#using-animator-to-create-npcs","title":"Using Animator to Create NPCs","text":"The data generated by Animator can be leveraged in multiple areas, but is particularly applicable in four key areas:
-
Training Machine Learning Algorithms - Animator creates larges sets of hyper-realistic user data. It can be leveraged to generate data sets that can be used for training machine learning algorithms. This enables the rapid training of anthropology-related ML algorithms that can leverage one or more of the hundred-plus data points generated by Animator.2
-
Honeypot Payloads - NPC details generated by Animator are designed to be as realistic as possible given the available relevant open source information. This makes the user data convincingly real while still being completely fabricated. Therefore, the data is ideal for use in applications like honeypots, where the goal is to trick an attacker into thinking they are compromising an asset with real user data. This data is also perfect for any other application that would benefit from extremely realistic user information.
-
Insider Threat Modeling - Each Animator NPC is given an Insider Threat Profile. This profile determines how likely it is that the NPC is an insider threat by incorporating the CDSE's Insider Threat Potential Indicators. As we continue developing Animator, it will be possible to configure NPCs to be more or less likely to be insider threats based on factors like their finances, criminal history, foreign contacts, and mental health.
-
Social Network and Relationship Modeling - Animator can establish relationships between the NPCs it generates. As we increase the fidelity of inter-NPC relationships, Animator NPCs create larger and more realistic social networks. By leveraging Animator's ability to quickly generate thousands of inter-related NPCs, Animator can easily be used to perform social networking modeling and research.
- Once Animator receives a request to create NPCs, it starts by creating an empty NPC Profile.
- Animator then iterates through all 100+ data points for the NPC and generates synthetic data to be associated with that NPC.
- Example data points are name, address, mental health, career, finances, and family members.
- Data points are either generated at random or are generated using weighted randomization. Weighted randomization involves leveraging verified datasets to influence the distribution of randomly generated data points to match much more closely to reality.
- Animator will complete this process for as many users as were selected by the request. This information can be exported through the API, or stored in a local database
Animator currently supports storing NPC data in a local Mongo Database. This feature is still being actively improved.
-
The GHOSTS development team highly recommends Nick Bilton's book American Kingpin for insight into the early days of the dark web.\u00a0\u21a9
-
A key developer from the Animator team went on to a position in the SEI's AI division. AI models need data. You connect the dots.\u00a0\u21a9
So, now we have Animator-generated NPCs, and they have profile information and preferences.
Animator has a job system that might enables us to push our simulation further:
- What motivates an agent?
- What does an agent know and how did they learn that? How does their knowledge grow over time?
- What relationships does an agent have off-network? How might this influence what they do on the computer?
- What does an agent believe? How did they come to that belief?
Jobs operate on a \"per cycle\" or \"step\" basis. For each cycle, the job processes a list of agents, and the actions or determinations programmed for each.
"},{"location":"animator/jobs/#decision-making-framework","title":"Decision-Making Framework","text":"We can use any combination of the following to drive agent decision-making:
"},{"location":"animator/jobs/#motivation","title":"Motivation","text":"We implement the Reiss Motivational Profile (RMP) - which is a mathematical framework for reasoning about agent comparative motivations - agent A is twice as motivated by X than agent B - that is baselined every few years.
"},{"location":"animator/jobs/#relationships","title":"Relationships","text":"Agents build relationships with other agents in the cohort. These get better or worse over time.
How this works is that each agent has the potential to interact with n other agents (they can also potentially transfer knowledge as a result). The more an agent knows about a particular subject, maybe the more likely they are to transfer information to another agent.
"},{"location":"animator/jobs/#knowledge","title":"Knowledge","text":"Agents build knowledge across an array of subjects that may alter their preferences. Within Animator, there are two main ways to learn:
- Independently through study or by utilizing resources such as books or videos
- Through relationships with others at the coffee counter, through mentorship, or group-based learning (a classroom or team for example). Here NPCs learn via interactions with other agents, and the system tracks what was learned and from whom.
What an agent believes can directly influence their behavior. Beliefs shape understanding of the world and guide decision-making and problem-solving. Agents come to belief utilizing Bayes Theorem, which is a mathematical framework for reasoning about probability of evidence.
So what does this all mean? Here is an example where an agent shares bits of information on social media:
Some tweets contain no insight about the agent. Some disclose some bit of information:
- Agent knows X fact
- Agent interacted with Y agent
- Agent decided to disclose some personal detail Z
Other agents \u2014 and adversaries \u2014 can see and infer from this information!
"},{"location":"animator/run/","title":"Running Animator Animations","text":"Animator is a simulation of a population of agents. Animator runs in cycles, and for each cycle, the agents make decisions based on their attributes, preferences, motivations, and behaviors.
"},{"location":"animator/run/#setup","title":"Setup","text":"- Get the Animator API up and running as outlined here
- Edit the
appsettings.jsonfile to enable Animations:
\"ApplicationSettings\": {\n\"GhostsApiUrl\": \"http://localhost:52388/\", // this should be root url of your ghosts API server\n\"Animations\": {\n\"IsEnabled\": false, // set this to true to enable animation jobs\n\"SocialGraph\": {\n\"IsEnabled\": false, // set this to true to enable the entire module (inc. web gui access)\n\"IsInteracting\": false, // set this to true to actually run the interactions\n\"MaximumSteps\": 4000, // animator stops when it reaches this many steps \n\"TurnLength\": 900, // by default, a step is a cpu cycle, the higher this number, the slower the simulation \n\"ChanceOfKnowledgeTransfer\": 0.3,\n\"Decay\": {\n\"StepsTo\": 10,\n\"ChanceOf\": 0.05\n}\n},\n\"SocialSharing\": {\n\"IsEnabled\": false, // set this to true to enable the entire module (inc. web gui access)\n\"IsInteracting\": false, // set this to true to actually run the interactions\n\"IsSendingTimelinesToGhostsApi\": false, // set this to true to actually send the timeline commands to the ghosts API\n\"IsChatGptEnabled\": false, // this is still under development, here be dragons\n\"SocializerUrl\": \"http://socializer.com\", // change this to the root url of your in-game social server\n\"MaximumSteps\": 14000,\n\"TurnLength\": 9000 // by default, a step is a cpu cycle, the higher this number, the slower the simulation \n},\n\"SocialBelief\": {\n\"IsEnabled\": false, // set this to true to enable the entire module (inc. web gui access)\n\"IsInteracting\": false, // set this to true to actually run the interactions\n\"MaximumSteps\": 14000,\n\"TurnLength\": 9000\n}\n}\n},\n...\nAfter you update the appsettings.json file, you will need to restart the Animator API server via:
docker restart animator-api\nGHOSTS content servers are an evolving part of the framework. They exist for several reasons:
- On an air-gapped network, where we are simulating some subset of the internet, we want more types of browsable content within that range \u2014 documents, spreadsheets, presentations, pdf files, streamed movies, and the like.
- We want a broad range of URLs within a site we are representing in the range.
- We want to simulate a document store, such as SharePoint, OneCloud, or similar, but without the hassle of installing and maintaining those actual systems.
Research by Global WebIndex claims that globally, 59% of the world's population uses social media, and that the average daily use is 2 hours and 29 minutes (July 2022).
"},{"location":"content/#air-gaps","title":"Air-gaps","text":"Many ranges are air-gapped, which means they have no access to the wider internet. In these cases, recreating a reasonable facsimile of the internet is key to the training experience. While there are many systems that do this well, we often want to augment the scenario with a wider array of URL traffic, or we want to introduce more of certain kinds of content going across the wire. PANDORA was created to address these concerns. Shortly later, we added a social server as well.
"},{"location":"content/#having-to-know-valid-urls","title":"Having to know valid URLs","text":"The other problem is that the internet works by the client having to \u201cknow\u201d the location of some resource via:
- Actually knowing the URL
- Being referred from another page - I might know google.com and search for something, which gives me a reference to another page I was not aware of previously.
- Inferring the URL from some like resource - If one is poking around and looking for something on a server, a slight change of URL often gives hints that get you to where you wanted to go.
- Guessing - the proliferation of .com domains means that for something new, it's often fruitful to just try that thing.com and see if it works!
The problem here is that currently, clients must know valid URLs that actually exist out in a simulated greyspace (via TopGen, GreyBox, or otherwise), which limits the array of potential requests and creates range work to maintain. So we created GHOSTS PANDORA, which serves whatever clients ask for - if the request is for a doc file, the server creates a random doc file on the fly \u2014 in memory \u2014 and serves it back to the client. Pandora serves the following content types:
- html
- css
- js
- doc|x
- ppt|x
- xls|x
- mp4
- gif
- jpg
- png
- zip
- msi
- iso
- other binary formats, etc.
Pandora has generic request handlers for each HTTP verb (GET, POST, etc.) and is deployed as a simple docker container. We configure it to handle a particular IP on a multiple-IP-enabled host machine. It works off any URL, but part of the solution involved introducing more randomness in the GHOSTS clients as well. Those clients now support a creative parameter-built URL system that can be configured to look something like this:
sharepoint.hello.com/{org}/{report_type}/{uuid}/{file_name}.{file_type}\nThese variables are processed at runtime, and produce a final url that might look something like:
sharepoint.hello.com/operations/maintenance/80e6af4e-5107-43b5-832f-0d8027efbd76/report.docx\nIn addition, Pandora supports \u201cbad\u201d payloads via configuration. Here the server responds to specific configured URLs to deploy planted injects. So clients can download malware in an exercise in a manner that is hard to differentiate based on URLs already seen within the event. The configuration looks like:
[payloads]\n1=/bad/payload/url/,some_bad.zip,application/zip\nWhere id=url,payload file, MIME response type.
"},{"location":"content/pandora/","title":"GHOSTS PANDORA SERVER Overview","text":"Pandora is part of GHOSTSPandora is within the GHOSTS Source Code Repository hosted on GitHub.
GHOSTS PANDORA is a web server that responds to a myriad of request types with randomized content generated in real-time. Used in conjunction with GHOSTS NPCs, the two can provide for agents that are periodically downloading content other than simple HTML and associated image, CSS, and js files.
"},{"location":"content/pandora/#running-this-server","title":"Running this server","text":""},{"location":"content/pandora/#as-a-docker-container","title":"As a Docker Container","text":"Docker is the preferred way to run Pandora - mostly because this is how we run and test it before version releases.
- Review the repository docker-compose.yml file
- Run the following in your terminal
mkdir ghosts-pandora\ncd ghosts-pandora\ncurl https://raw.githubusercontent.com/cmu-sei/GHOSTS/master/src/ghosts.pandora/docker-compose.yml -o docker-compose.yml\ndocker-compose up -d\nThis assumes the host server is a common Linux distribution. For images to render correctly, PIL or the more recent Pillow library is necessary. See here for more information on Pillow installation and configuration.
- Using a Python 3 distribution >= 3.6.2
- In the terminal run:
pip install -r requirements.txt - Then run
python app.py
- /api - All requests beginning with
/apiautomatically respond with json. This includes:/api/users/api/user/a320f971-b3d9-4b79-bb8d-b41d02572942/api/reports/personnel
- /csv - All requests beginning with
/csvautomatically respond with csv. Like the above, this includes urls such as:/csv/users/csv/user/winx.jalton/csv/reports/HR/payroll
- /i, /img, /images - All requests beginning with these directories automatically respond with a random image of type [gif, jpg, png]. Examples:
/i/v1/a9f6e2b7-636c-4821-acf4-90220f091351/f8f8b1f0-9aa5-4fc7-8880-379e3192748e/small/images/products/184f3515-f49b-4e07-8c8b-7f978666df0e/view/img/432.png
- /pdf - All requests respond with a random pdf document. Examples:
/pdf/operations/SOP_Vault/a7f48bd5-84cb-43a1-8d3d-cd2c732ddff6/pdf/products
- /docs - All requests respond with a random word document
- /slides - All requests respond with a random powerpoint document
- /sheets - All requests respond with a random excel document
For requests indicating a specific file type, there are several specific handlers built to respond with that particular kind of file, such as:
- .csv
- Image requests [.gif, .ico, .jpg, .jpeg, .png]
- .json
- Office document requests
- .doc, .docx
- .ppt, .pptx
- .xls, .xlsx
So that a URL such as /users/58361185-c9f2-460f-ac45-cb845ba88574/profile.pdf would return a pdf document typically rendered right in the browser.
All unhandled request types, urls without a specific file indicator, or requests made outside specifically handled directories (from the preceding section) are returned as html, including:
/docs/by_department/operations/users/blog/d/2022/12/4/blog_title-text/hello/index.html
Pandora also can hide payloads in a particular request for things like red-teaming and such. This is done in the configuration file, and looks like this:
[payloads]\n1=/1/,a.zip,application/zip\n2=/2/users,b.zip,application/zip\n3=/3/some/report/url,c.zip,application/zip\nEach record must be an incrementing integer with no duplication. The values are:
- The URL that this payload responds to
- The local file (stored in
./payloads/) to be returned - The MIME type of the response
So for 1 in the example above, requests to /1/ return the a.zip file as an application/zip file.
"},{"location":"content/social/","title":"GHOSTS PANDORA SOCIAL Overview","text":"PANDORA SOCIAL is still very early betaHere be dragons
The place where GHOSTS agents come to share their thoughts and information.
In the spirit of the original PANDORA, this server also responds to a very broad array of URLs but enables clients to POST/PUT/DELETE to it as well, for example:
Request ResponsePOST /images responds with a url to the saved image file POST / responds with a randomly-generated streamed video POST /users/michelle_smith/af2d00aa-4a89-4af3-baff-1746b556e7a1/ responds with a reply to the original user's social post"},{"location":"core/api/","title":"GHOSTS Core API Overview","text":"GHOSTS Source Code The GHOSTS Source Code Repository is hosted on GitHub.
The GHOSTS API enables the control and orchestration of non-player characters (NPCs) within a deployment. It supports logging, reporting, and managing individual, groups of, or entire deployments of client installs.
"},{"location":"core/api/#installation","title":"Installation","text":"- Install \ud83d\udc33 Docker
- Install Docker Compose
- Run the following commands - we'll use this docker-compose.yml file
mkdir ghosts\ncd ghosts\ncurl https://raw.githubusercontent.com/cmu-sei/GHOSTS/master/src/Ghosts.Api/docker-compose.yml -o docker-compose.yml\ndocker-compose up -d\nThe required containers will be downloaded and configured automatically.
Once the last command completes, if you open http://localhost:5000/api/home in your browser, you should see the initial API page outlining the version of the install, and a few test machine entries. If this page renders, your API is up, running, and available.
You will still need to set up Grafana. Beware that you must often chown the host location of the container as listed in the docker-compose file or the container will just continually restart in error due to insufficient permissions.
The API generally has good defaults to get you up and running quickly, but there are some considerations in the appconfig.json file:
\"ClientSettings\": {\n\"OfflineAfterMinutes\": 30, ...\n\"MatchMachinesBy\": null,\nCan be fqdn|host|resolvedhost|null - null tells the API to match incoming requests with machine records by the machine name. For installations where multiple domains are reporting into the same API, you probably want to use FQDN in order to avoid machines being duplicated.
\"QueueSyncDelayInSeconds\": 10,\n\"NotificationsQueueSyncDelayInSeconds\": 10,\nThis is how often the synch job runs. Incoming machine requests are not real-time in order to best bundle like records together.
"},{"location":"core/api/#configuring-grafana","title":"Configuring Grafana","text":"- Grafana will be running (if containerized) on port 3000, and we can access it via the same URL we use for the API.
- The default login is admin/admin.
- The first step is to set up a datasource named \"ghosts\" to the ghosts Postgres database.
- Now import your choice of the grafana json files in this repository. It creates the default GHOSTS dashboard.
The GHOSTS API provides webhook callbacks based on the configuration on the endpoint: /api/webhooks. The payload for creating a webhook is in the format:
{\n\"status\": 0,\n\"description\": \"some description\",\n\"postbackUrl\": \"http://localhost/endpoint:port\",\n\"postbackMethod\": 0, (0 == get, 1 == post)\n\"postbackFormat\": \"see below\"\n}\nPayloads can be any format \u2014 here is a sample:
{\n'machine':'[machinename]',\n'created':'[datetime.utcnow]',\n'type':'[messagetype]',\n'payload':'[messagepayload]'\n}\nOn send, the payload will be converted into the correct JSON format:
{\n\"machine\":\"some_guid\",\n\"created\":\"some_datetime\",\n\"type\":\"some_message\",\n\"payload\":\"some_payload\"\n}\nIf the postback method is POST, the payload will be sent as the message body. If the postback method is GET, the payload will be sent as part of the querystring value ?message=payload.
The following events are reported via webhooks:
- Timeline delivered (with the timeline that was delivered as payload) to a machine via API (original API posting of timeline only holds timeline in wait - the client still must check-in in order for that timeline to be delivered)
- Machine requested updates (\"checked in\") from API
- Machine posted results to API
Is the API up and running?
- Go to
/api/homein the browser, it should return the current API version and the number of machines and groups under management. If it says relationship not found, restart the API application and it should create the database automatically. - Run
docker ps --alland see that all containers are running normally. If one or more is not running, look at the logs for that machine viadocker logs [machine name].
The ClientId, ClientResults, and other Client* endpoints are failing.
The Client* endpoints are for the Clients to use only. There are specific header values set by the client in the request that is used to authenticate the request. If you are not using the client, you will not have these headers set, and these endpoints will fail.
"},{"location":"core/client/","title":"GHOSTS Core Client Overview","text":"GHOSTS Source CodeThe GHOSTS Source Code Repository is hosted on GitHub.
The GHOSTS client simulates what anyone might do at a computer given their particular role or membership within some team. It creates documents, browses websites, downloads files, and uses all sorts of popular applications on many versions of Windows and Linux machines. Whether you're a friendly administrator or a powerful cyber adversary, GHOSTS can replicate your expected behavior.
GHOSTS has many use cases in cyber training and exercises, most notably for bringing non-player characters (NPCs) to life, but it can also be used for other purposes where realistic activity on a computer or network is needed - testing or generating datasets, for example.
Do I need the API?Although clients are fully functional without the API server, the latter enables logging, reports, and remote administration capabilities. Without the API, changes to clients must be managed in some other manner.
The client's configuration and command system are file-based, so if you do not wish to use the API, you can manage these through some other tool, such as Ansible or similar. All of these files are under the client's install location.
Run as the NPC you're representingThe GHOSTS client should typically be run as a user, and not as administrator or root - training or exercising teams will notice this immediately.
"},{"location":"core/client/#the-windows-client","title":"The Windows Client","text":"GHOSTS on Windows (Win10 and Win7 currently supported) is a .NET Console application that performs user activity on client workstations - web browsing, working with office documents, using the command prompt or PowerShell, etc. Our team typically installs the client in an out-of-game directory (c:\\exercise\\ghosts\\), where no event or injects will originate. It is recommended you verify one working client before deploying to multiple machines. You will need the base URL from the API installation for the client to communicate home.
-
Your client Windows machine will need to have (at least) the Microsoft DotNet 4.6.1 runtime installed . Again, note that you only need the runtime, not the full SDK. We continue to use 4.6.1 on Windows to maintain backward compatibility.
-
Download the appropriate latest client
Unzip to your client machine in a directory such as c:\\exercise\\ghosts. You will need to adjust the configuration in config/application.json for your client to talk to your an installed instance of the GHOSTS API server.
For any client utilizing the Firefox or Chrome web browser, an automation driver is necessary to be included in the same folder as the GHOSTS binary. For Firefox, download the appropriate \ud83e\udd8e\u00a0 Geckodriver for your version of the browser here . For Chrome, download the appropriate Chromedriver for your version of the browser here .
Additional configuration required for emailUsing the Windows client email functions requires the use of Redemption which provides robust Outlook automation. The full Redemption library should be found in /lib.
Your client Linux machine will need to have the latest Microsoft dotnetcore runtime installed (Note that for the GHOSTS client, there are versions for dotnetcoreapp3.1 - this will eventually go away \u2014 and dotnet6.0, which is long term support (LTS) and will stick around for a while). Again, note that you only need the runtime installed, not the full SDK.
"},{"location":"core/client/#linux-installation","title":"Linux Installation","text":"- Download the latest Linux client
Unzip to a folder such as ~/ghosts for the user that you want GHOSTS to run as.
Note that on Linux machines running the client as root and utilizing web browsing may result in failures due to Gecko/Chromedriver display issues.
"},{"location":"core/client/#client-directory-structure","title":"Client Directory Structure","text":"Do not copy the instance folderYou should never copy the instance folder from one machine to another.
config/ configuration files are stored here. instance/ generated files and information relative to this particular installed instance of ghosts is stored here. This folder should never be copied from one machine to another lib/ third-party libraries used by ghosts are stored here logs/ output logs for the installed instance (logs/app.log), and logs that are transferred to the server (logs/clientupdates.log)"},{"location":"core/client/#configuration-quick-start","title":"Configuration Quick Start","text":"After unzipping the GHOSTS client, we can simply double-click it to run. Note that the console window likely printed a few messages, but then disappeared. This is normal, in production mode GHOSTS hides itself. We can see it in the Windows Task Manager, however, and we can kill the process from there. We can also run the included kill-ghosts.bat file that closes the application and any applications it might control.
Beyond this initial step of verifying that the client will run, there are two files that we might need to adjust to fit many deployments:
"},{"location":"core/client/#applicationjson","title":"application.json","text":"In this file, often all we need to change are the URLs for the API, IdUrl, ClientResultsUrl, ClientUpdatesUrl, and the like. Change the hostname to your installed API location, and GHOSTS should check in as expected.
{\n\"IdEnabled\": true, //enabled in order to command and control from api (C2) server\n\"IdUrl\": \"http://yourapiurl.com/api/clientid\", //url for API endpoint to get clientid\n\"ClientResultsEnabled\": true, //enabled to report results to C2\n\"ClientResultsUrl\": \"http://yourapiurl.com/api/clientresults\", //url for API endpoint to report results\n\"ClientResultsCycleSleep\": 90, //report results every x ms\n\"ClientUpdatesEnabled\": true, //enabled to get updates from C2\n\"ClientUpdatesUrl\": \"http://yourapiurl.com/api/clientupdates\", //url for API endpoint to get updates\n\"ClientUpdatesCycleSleep\": 90, //check for updates every x ms\n\"Survey\": { //survey is a local report of processes running, etc.\n\"IsEnabled\": true, //on/off\n\"Frequency\": \"once\", //how often to survey\n\"MaxAgeInHours\": 168, //how long to wait until new survey\n\"OutputFormat\": \"indent\" //compact/fancy(indent)\n},\n\"HealthIsEnabled\": true, //enable local health checks\n\"HandlersIsEnabled\": true, //enable local timeline activity\n\"ChromeExtensions\": \"\", //comma separated local extensions (used for injects in the past)\n\"FirefoxInstallLocation\": \"\", //geckodriver needs this for non-standard installs (is pesky)\n\"FirefoxMajorVersionMinimum\": 48, //geckodriver is picky about versions\n\"OfficeDocsMaxAgeInHours\": 6, //cleanup kills docs in the documents folder older than this setting\n}\nThe other file we may want to adjust is the default timeline. This is what the agent does all day, including browsing the internet, creating documents, and similar. The defaults hopefully give you a good idea of what is possible, and of course, the array of configurations here is endless - be creative!
The primary item is the HandlerType. This tells GHOSTS to run a command (Command), use Firefox to browse an array of websites (BrowserFirefox), create Excel documents (Excel) and so on. Some of the other items related to a handler's configuration are:
- Initial: The initial command for a handler to execute. For a web browser, you might enter either a URL or \"about:blank\".
- UtcTimeOn | UtcTimeOff: \"00:00:00\": \"24:00:00\" to not shut off. Otherwise, enter an on and an off time to simulate things such as office hours of 9-5, etc. There are 30 minutes of jitter plus or minus from the time entered.
- Loop: Set this to true to continue to execute this same command on a loop, or false to execute something just one time.
{\n\"HandlerType\": \"Command\",\n\"Initial\": \"\",\n\"UtcTimeOn\": \"00:00:00\",\n\"UtcTimeOff\": \"24:00:00\",\n\"Loop\": \"True\",\n\"TimeLineEvents\": [\n{\n\"Command\": \"NETSTAT\", \"CommandArgs\": [],\n\"DelayAfter\": 900000,\n\"DelayBefore\": 0\n}\n]\n}\nTo access a network share file, the command might be: net use X:\\SERVER\\Share
To RDP to another machine: mstsc.exe {ConnectionFile | /v:ServerName[:Port]} [/console] [/f] [/w:Width/h:Height]
/v - specifies the remote computer and port (optional) you wish to connect to /console \u2013 connects to the console of a Windows Server 2003 based system /f \u2013 starts the remote desktop connection in full screen mode /w & /h \u2013 specifies the width and height of the remote desktop connection
Actions can also be created for standard copy/move/deletion of files via their respective commands.
Chrome
We have to pass the browser window an initial value. If we don't want it to go anywhere at start, we could pass about:blank, otherwise we'd pass a url. These can be http or https.
{\n\"HandlerType\": \"BrowserChrome\",\n\"Initial\": \"http://google.com\",\n\"UtcTimeOn\": \"00:00:00\",\n\"UtcTimeOff\": \"24:00:00\",\n\"Loop\": \"True\",\n\"TimeLineEvents\": [\n{\n\"Command\": \"random\",\n\"CommandArgs\": [\n\"http://google.com\",\n\"http://facebook.com\",\n],\n\"DelayAfter\": 1000,\n\"DelayBefore\": 0\n}\n]\n}\nExcel, PowerPoint, Word
{\n\"HandlerType\": \"Word\",\n\"Initial\": \"\",\n\"UtcTimeOn\": \"00:00:00\",\n\"UtcTimeOff\": \"24:00:00\",\n\"Loop\": \"True\",\n\"TimeLineEvents\": [\n{\n\"Command\": \"create\",\n\"CommandArgs\": [ \"%homedrive%%homepath%\\\\Documents\" ],\n\"DelayAfter\": 900000,\n\"DelayBefore\": 0\n}\n]\n}\nFor specific Timeline Events where the outcome is needed to be tracked, like for example, a client machine spawned inject, use a Trackable (via TrackableId in the following example):
{\n\"TimeLineHandlers\": [\n{\n\"HandlerType\": \"BrowserChrome\",\n\"Initial\": \"about:blank\",\n\"UtcTimeOn\": \"00:00:00\",\n\"UtcTimeOff\": \"24:00:00\",\n\"Loop\": false,\n\"TimeLineEvents\": [\n{\n\"Command\": \"browse\",\n\"CommandArgs\": [ \"https://dl.dafont.com/dl/?f=italian_breakfast\" ],\n\"DelayAfter\": 0,\n\"DelayBefore\": 0\n},\n{\n\"Command\": \"download\",\n\"CommandArgs\": [ \"//a[contains(@class, 'dl')]\" ],\n\"TrackableId\": \"<guid id from trackables table/>\",\n\"DelayAfter\": 0,\n\"DelayBefore\": 0\n}\n]\n},\n{\n\"HandlerType\": \"Command\",\n\"Initial\": \"\",\n\"UtcTimeOn\": \"00:00:00\",\n\"UtcTimeOff\": \"24:00:00\",\n\"Loop\": false,\n\"TimeLineEvents\": [\n{\n\"Command\": \"cd %homedrive%%homepath%\\\\Downloads\",\n\"CommandArgs\": [\n\"powershell expand-archive -Path italian_breakfast.zip -destinationpath x\",\n\"cd x\",\n\"dir\"\n],\n\"TrackableId\": \"<guid id from trackables table/>\",\n\"DelayAfter\": 10,\n\"DelayBefore\": 10000\n}\n]\n}\n]\n}\nClients aren't running (immediately exiting, throwing copious exceptions, or similar)
- Is the dotnet framework runtime 4.x installed on the machine?
- If GPO is doing white-listing of what apps can run, is
ghosts.exewhite-listed? - Will the client run by simply double-clicking on the exe?
- Does it report anything to the windows application event logs?
- What's in
logs/app.log? - Is the ghosts executable set to execute automatically when the machine restarts?
Clients aren't reporting their activity to the API
- Is the client running correctly? (if not, see above)
- Is there entries in the logs/clientupdates.log?
- If there are, is the file too large? (Try removing it, ghosts might be hung trying to process a lot of log data)
- If the folder
instancecreated? Does the fileinstance/id.jsonexist? (If it does and has an ID within, then ghosts has reported home to the api at least once) - The file
logs/app.logindicating any fatal issues? (Logging can be ratcheted up and down via nlog configuration)
Can I update what clients are doing?
- Clients operate off their
config/timeline.jsonfile and this can be updated via Powershell, Ansible, or other means - it's just a file. - Clients can also do just-in-time activities via the
instance/timeline/infolder. Anything placed here will be picked up, executed, and moved to the corresponding out folder once complete. This does not affect any activity currently controlled with the default timeline file.
Can I reset a client on a box?
- Yes, launching a new instance of Ghosts kills the previous one and all associated tasks from the timeline (any instances of Word, PowerShell, etc.). Only one instance of Ghosts will be running on a client box at any time. We can also run the .bat script
kill-ghosts.batincluded in the distribution to clean everything up.
What is the easiest way to determine the running version of the client?
- run the version flag:
ghosts.exe --version
The following is the format for a basic timeline handler:
\"TimeLineHandlers\": [\n{\n\"HandlerType\": \"Watcher\",\n\"UtcTimeOn\": \"00:00:00\",\n\"UtcTimeOff\": \"24:00:00\",\n\"Loop\": true,\n\"TimeLineEvents\": [\n{\n\"Command\": \"folder\",\n\"CommandArgs\": [ \"path:%HOMEDRIVE%%HOMEPATH%\\\\Downloads\", \"size:2000\", \"deletionApproach:oldest\" ],\n\"DelayAfter\": 0,\n\"DelayBefore\": 0\n}\n]\n}\n]\nSome of the key-value pairs are self-explanatory, but let's review a few important ones:
Key ValueHandlerType The application or major function we want to control. This could be FireFox, the Cmd terminal, or Word. UtcTimeOn The time the handler will begin to run. To simulate an agent coming into the office at 0900, we set this to your time zone's UTC value. UtcTimeOff The time the handler will stop. To simulate an agent leaving the office at 1700, we set this to your time zone's UTC value. Loop true or false - a handler could be repeated or just run one-time."},{"location":"core/api/timelines/","title":"Managing Timelines","text":"One of the primary capabilities of the API is to manage multiple client timelines. Timelines are the mechanism for clients to perform actions on behalf of a user. For example, a client might execute a timeline to browse a website, create a document, or other Handler activities.
Clients are configured to check in periodically with the API to report their current state and to check if any updates are available. We can have the API hold an update for the next time a client, group of clients, or all clients check in. That update will then be processed by the client and the client will report back to the API that the update has been processed. This allows the API to manage the state of the client and to ensure that the client is always up to date. So the process is:
- We configure an update for a client or group of clients in the API.
- Clients check in and are told that an update is available.
- The client processes the update and reports back to the API that the update has been processed.
Client timelines are updated via POST to /api/timelines in the following format:
{\n\"machineId\": \"3fa85f64-5717-4562-b3fc-2c963f66afaf\",\n\"type\": 0,\n\"activeUtc\": \"2023-03-08T16:38:04.002Z\",\n\"status\": 0,\n\"update\": {\n\"status\": 0,\n\"timeLineHandlers\": [\n{\n\"handlerType\": 0,\n\"initial\": \"string\",\n\"utcTimeOn\": {\n\"ticks\": 0\n},\n\"utcTimeOff\": {\n\"ticks\": 0\n},\n\"handlerArgs\": {\n\"additionalProp1\": \"string\",\n\"additionalProp2\": \"string\",\n\"additionalProp3\": \"string\"\n},\n\"loop\": true,\n\"timeLineEvents\": [\n{\n\"trackableId\": \"string\",\n\"command\": \"string\",\n\"commandArgs\": [\n\"string\"\n],\n\"delayAfter\": 0,\n\"delayBefore\": 0\n}\n]\n}\n]\n}\n}\nThe key values to consider here are: The machine ID that you want to update, the other key value is the type of timeline. The type is an integer value that represents the type of update. The following are the types of updates that are currently supported:
- Timeline = 0 // this
- Health = 1
- TimelinePartial = 10 // this the agent is currently doing off its default timeline
- RequestForTimeline = 20
The remainder of the settings are for the timeline - basically what we are doing here is sending a client a new timeline, with the above values to indicate, which machine and what to change in the update node.
A sample timeline for each browser is available in a code repository directory of the same name.
"},{"location":"core/handlers/browser/#timeline-url-variables","title":"Timeline URL Variables","text":"An example of a URL using variables is:
http://craigslist.org/{org}/{group}/{uuid}/{verb}/{type}/{n}?{c}={now}\nAll variables are in the format {variable}. There are several standard variables:
{now}= short datetime (mm/dd/yyyy format){uuid}= uuid{c}= a single character from a-z and A-Z{n}= number between 1 and 1000
We can also configure additional variables by adding the following to a browser handler in a timeline:
\"url-replace\": [\n{\"verb\": [\"order\", \"enable\", \"engage\"]},\n{\"group\": [\"operations\", \"logistics\", \"medical\"]},\n{\"org\": [\"army\", \"command\", \"brigade\", \"battalion\"]},\n{\"type\": [\"document\", \"doc\", \"files\", \"vault\", \"filevault\"]}\n]\nTherefore:
https://www.cmu.edu/{org}/{group}/{verb}/{type}/{uuid}/version_{n}?{c}={now}\nmight be rendered as:
https://www.cmu.edu/command/operations/order/doc/bcc396b5-47d0-4665-93c8-0a314cec13e1/version_55?d=6/21/2022\nThe GHOSTS SPECTRE Source Code Repository is hosted on GitHub
SPECTRE is our attempt to reduce that agent browsing patterns appear as \"computer random\".1 This module modifies user web browser timelines so that they more closely match that agent's preferences over time. We expect this simple model of ML to be used for other types of agent activity in the future.
Agent browsing patterns are no longer random and match different types of users on a network. These patterns improve over time.
Also, this simple model of ML can now be used for other types of agent activity. This makes for an exciting future on the GHOSTS platform.
Using Machine Learning to Increase NPC Fidelity with Dynamic Preferences Used in Forward-Looking Decisions
As GHOSTS agents make more informed, and hopefully, more complex decisions, there is a need for each agent to have a system of preferences existing at the time the agent is created, and for an ability to update those preferences over time as the agent continues to make decisions and measure the outcome of those decisions afterwards.
SPECTRE provides GHOSTS enables agents to make preferenced decisions and to use the outcome of those decisions to learn and evaluate future choices more intelligently.
"},{"location":"spectre/#how-it-works","title":"How it works","text":"SPECTRE currently has two components:
"},{"location":"spectre/#preference-engine","title":"Preference Engine","text":"- An administrator creates the types of personas they wish to have GHOSTS agents represent
- GHOSTS agents report in their browser history to the GHOSTS C2 API
- If the agent has no persona, SPECTRE assigns one at random
- The persona assignment randomly creates different preferences for the agent based on persona settings (not unlike character creation in D&D)
- Preferences are a preference and its numeric value (-100 to 100) which roughly represents how much an agent likes or dislikes something
- Preferences can be any
key:valuepair, such as: \"sports\":50 \"kali\":35 \"127.0.0.1\":77 - Preferences of 0 mean that the agent is indifferent (or has no preference)
- All preference history is stored so that in the (near) future we can track an agent's preferences over time
- It is assumed that high negative preferences would have agents avoiding those values (e.g. for \"onions\":-100 one would assume an agent is avoiding eating onions at all costs.)
The incoming GHOSTS agent browsing activity can be attenuated to individual agent preferences after they are assigned some number of preferences based on default persona profiles. SPECTRE will aggregate this information periodically and perform model training and testing against that browsing activity, and recommend new browsing patterns for that agent to execute. This creates a new activity timeline for the agent. This cycle is referred to as a \"Test\". After any given test, that information would be used to inform the next round of ML testing to be done.
"},{"location":"spectre/#installation","title":"Installation","text":"SPECTRE installs alongside an existing GHOSTS installation as a separate docker container. With a few configuration changes, you should be up and running in minimal time.
There are only two configuration settings, both contained within appsettings.json:
\"DefaultConnection\": \"Host=localhost;Port=5432;Database=preferences;User Id=ghosts;Password=scotty@1;Pooling=true;Command Timeout=9900\",\n\"GhostsApiUrl\": \"http://localhost:5000\"\nThe first setting is for your connection to a necessary Postgres database, which SPECTRE will use for its operations. It is fine to host this on the same machine or container that you might be using for GHOSTS itself. In this case, either use the same user as GHOSTS or create a new user for SPECTRE.
The second setting is for SPECTRE to access the GHOSTS API endpoints. This is used to get information about machines and to update their timelines, based on SPECTRE findings and executions.
"},{"location":"spectre/#quick-start","title":"Quick Start","text":"Spectre is rather like an add-on -- it sits alongside the core API and uses that system to get its initial agent information, process their timelines, and then post updated timelines back to the API for dissemination to the clients.
- There is one setting in the spectre
appsettings.jsonfile that may need updating based on the install - it is how Spectre will connect to the GHOSTS Core API:\"GhostsApiUrl\": \"http://host.docker.internal:52388\"\u2014\u00a0update this to your core API host and port. - Now go to Spectre's host:port/swagger to bring up the API.
GET /Agentswill show you what you have under SPECTRE control - at the start, it is likely no agents. We need to sync with core GHOSTS Core API to get its agents into SPECTRE.- So, we need to run
POST /Agents/synconce to pull the agents in the core API over to SPECTRE. - Now Spectre's
GET /Agentswill show you the same agents from ghosts Core API. - You can now run a browse recommendations job.
-
This work is detailed in the technical report Using Machine Learning to Increase NPC Fidelity. Some of the team also discussed this project in an SEI podcast episode, entitled ML-Driven Decision-Making in Realistic Cyber Exercises.\u00a0\u21a9
+
+
+
+ Skip to content
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ GHOSTS SPECTRE Overview¶
+
+
+GHOSTS SPECTRE Source Code
+The GHOSTS SPECTRE Source Code Repository is hosted on GitHub
+SPECTRE is our attempt to reduce that agent browsing patterns appear as "computer random".1 This module modifies user web browser timelines so that they more closely match that agent's preferences over time. We expect this simple model of ML to be used for other types of agent activity in the future.
+Agent browsing patterns are no longer random and match different types of users on a network. These patterns improve over time.
+Also, this simple model of ML can now be used for other types of agent activity. This makes for an exciting future on the GHOSTS platform.
+Using Machine Learning to Increase NPC Fidelity with Dynamic Preferences Used in Forward-Looking Decisions
+As GHOSTS agents make more informed, and hopefully, more complex decisions, there is a need for each agent to have a system of preferences existing at the time the agent is created, and for an ability to update those preferences over time as the agent continues to make decisions and measure the outcome of those decisions afterwards.
+SPECTRE provides GHOSTS enables agents to make preferenced decisions and to use the outcome of those decisions to learn and evaluate future choices more intelligently.
+How it works¶
+SPECTRE currently has two components:
+Preference Engine¶
+-
+
- An administrator creates the types of personas they wish to have GHOSTS agents represent +
- GHOSTS agents report in their browser history to the GHOSTS C2 API +
- If the agent has no persona, SPECTRE assigns one at random +
- The persona assignment randomly creates different preferences for the agent based on persona settings (not unlike character creation in D&D) +
- Preferences are a preference and its numeric value (-100 to 100) which roughly represents how much an agent likes or dislikes something +
- Preferences can be any
key:valuepair, such as: + "sports":50 + "kali":35 + "127.0.0.1":77
+ - Preferences of 0 mean that the agent is indifferent (or has no preference) +
- All preference history is stored so that in the (near) future we can track an agent's preferences over time +
- It is assumed that high negative preferences would have agents avoiding those values (e.g. for "onions":-100 one would assume an agent is avoiding eating onions at all costs.) +
Machine Learning¶
+The incoming GHOSTS agent browsing activity can be attenuated to individual agent preferences after they are assigned some number of preferences based on default persona profiles. SPECTRE will aggregate this information periodically and perform model training and testing against that browsing activity, and recommend new browsing patterns for that agent to execute. This creates a new activity timeline for the agent. This cycle is referred to as a "Test". After any given test, that information would be used to inform the next round of ML testing to be done.
+Installation¶
+SPECTRE installs alongside an existing GHOSTS installation as a separate docker container. With a few configuration changes, you should be up and running in minimal time.
+There are only two configuration settings, both contained within appsettings.json:
"DefaultConnection": "Host=localhost;Port=5432;Database=preferences;User Id=ghosts;Password=scotty@1;Pooling=true;Command Timeout=9900",
+"GhostsApiUrl": "http://localhost:5000"
+The first setting is for your connection to a necessary Postgres database, which SPECTRE will use for its operations. It is fine to host this on the same machine or container that you might be using for GHOSTS itself. In this case, either use the same user as GHOSTS or create a new user for SPECTRE.
+The second setting is for SPECTRE to access the GHOSTS API endpoints. This is used to get information about machines and to update their timelines, based on SPECTRE findings and executions.
+Quick Start¶
+Spectre is rather like an add-on -- it sits alongside the core API and uses that system to get its initial agent information, process their timelines, and then post updated timelines back to the API for dissemination to the clients.
+-
+
- There is one setting in the spectre
appsettings.jsonfile that may need updating based on the install - it is how Spectre will connect to the GHOSTS Core API:"GhostsApiUrl": "http://host.docker.internal:52388"— update this to your core API host and port.
+ - Now go to Spectre's host:port/swagger to bring up the API. +
GET /Agentswill show you what you have under SPECTRE control - at the start, it is likely no agents. We need to sync with core GHOSTS Core API to get its agents into SPECTRE.
+- So, we need to run
POST /Agents/synconce to pull the agents in the core API over to SPECTRE.
+ - Now Spectre's
GET /Agentswill show you the same agents from ghosts Core API.
+ - You can now run a browse recommendations job. +
+
+
+
+
+
+
+
+
+
+
+ +
-
+
-
+
This work is detailed in the technical report Using Machine Learning to Increase NPC Fidelity. Some of the team also discussed this project in an SEI podcast episode, entitled ML-Driven Decision-Making in Realistic Cyber Exercises. ↩
+
+
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/stylesheets/extra.css b/stylesheets/extra.css
new file mode 100644
index 00000000..34d19a1f
--- /dev/null
+++ b/stylesheets/extra.css
@@ -0,0 +1,3 @@
+:root > * { --md-primary-fg-color: green; }
+img[alt="Crucible Logo"] {width:4.5%; float:left; margin-right:7px; }
+img[alt="Animator Logo"] {width: 4.5%; }
\ No newline at end of file