Merge pull request #288 from cmu-sei/v8

updates docs

docs/content/index.md

@@ -8,9 +8,9 @@ GHOSTS content servers are an evolving part of the framework. They exist for sev
 
 > Research by Global WebIndex claims that globally, 59% of the world's population uses social media, and that the average daily use is 2 hours and 29 minutes (July 2022).
 
-### Air-gaps
+### Air-gapped networks
 
-Many ranges are air-gapped, which means they have no access to the wider internet. In these cases, recreating a reasonable facsimile of the internet is key to the training experience. While there are many systems that do this well, we often want to augment the scenario with a wider array of URL traffic, or we want to introduce more of certain kinds of content going across the wire. PANDORA was created to address these concerns. Shortly later, we added a social server as well.
+Many ranges have no access to the wider internet. In these cases, recreating a reasonable facsimile of the internet is key to the training experience. While there are many systems that do this well, we often want to augment the scenario with a wider array of URL traffic, or we want to introduce more of certain kinds of content going across the wire. PANDORA was created to address these concerns. Shortly later, we added a social server as well.
 
 ### Having to know valid URLs
 

docs/core/client.md

@@ -1,4 +1,4 @@
-# GHOSTS Core Client Overview
+# GHOSTS Client Overview
 
 ???+ info "GHOSTS Source Code"
     The [GHOSTS Source Code Repository](https://github.com/cmu-sei/GHOSTS) is hosted on GitHub.
@@ -69,28 +69,58 @@ Beyond this initial step of verifying that the client will run, there are two fi
 
 In this file, often all we need to change are the URLs for the API, IdUrl, ClientResultsUrl, ClientUpdatesUrl, and the like. Change the hostname to your installed API location, and GHOSTS should check in as expected.
 
-```javascript
+```json
 {
-    "IdEnabled": true,                                                      //enabled in order to command and control from api (C2) server
-    "IdUrl": "http://yourapiurl.com/api/clientid",                          //url for API endpoint to get clientid
-    "ClientResultsEnabled": true,                                           //enabled to report results to C2
-    "ClientResultsUrl": "http://yourapiurl.com/api/clientresults",          //url for API endpoint to report results
-    "ClientResultsCycleSleep": 90,                                          //report results every x ms
-    "ClientUpdatesEnabled": true,                                           //enabled to get updates from C2
-    "ClientUpdatesUrl": "http://yourapiurl.com/api/clientupdates",          //url for API endpoint to get updates
-    "ClientUpdatesCycleSleep": 90,                                          //check for updates every x ms
-    "Survey": {                                                             //survey is a local report of processes running, etc.
-        "IsEnabled": true,                                                  //on/off
-        "Frequency": "once",                                                //how often to survey
-        "MaxAgeInHours": 168,                                               //how long to wait until new survey
-        "OutputFormat": "indent"                                            //compact/fancy(indent)
-    },
-    "HealthIsEnabled": true,                                                //enable local health checks
-    "HandlersIsEnabled": true,                                              //enable local timeline activity
-    "ChromeExtensions": "",                                                 //comma separated local extensions (used for injects in the past)
-    "FirefoxInstallLocation": "",                                           //geckodriver needs this for non-standard installs (is pesky)
-    "FirefoxMajorVersionMinimum": 48,                                       //geckodriver is picky about versions
-    "OfficeDocsMaxAgeInHours": 6,                                           //cleanup kills docs in the documents folder older than this setting
+  "ApiRootUrl": "http://localhost:5000/api",    // there is now just one api url to change in order to point where you've hosted your api
+  "Sockets": {                                  // this is the new websockets configuration, turn it off if you're not using it
+    "IsEnabled": true,
+    "Heartbeat": 50000
+  },
+  "Id": {
+    "IsEnabled": true,
+    "Format": "guestlocal",
+    "FormatKey": "guestinfo.id",
+    "FormatValue": "$formatkeyvalue$-$machinename$",
+    "VMWareToolsLocation": "C:\\progra~1\\VMware\\VMware Tools\\vmtoolsd.exe"
+  },
+  "AllowMultipleInstances": false,
+  "EncodeHeaders": true,
+  "ClientResults": {
+    "IsEnabled": true,
+    "IsSecure": false,
+    "CycleSleep": 300000
+  },
+  "ClientUpdates": {
+    "IsEnabled": true,
+    "CycleSleep": 300000
+  },
+  "Survey": {
+    "IsEnabled": false,
+    "IsSecure": false,
+    "Frequency": "once",
+    "CycleSleepMinutes": 5,
+    "OutputFormat": "indent"
+  },
+  "Timeline": {
+    "Location": "config/timeline.json"
+  },
+  "Content": {
+    "EmailsMax": 20,
+    "EmailContent": "",
+    "EmailReply": "",
+    "EmailDomain": "",
+    "EmailOutside": "",
+    "BlogContent": "",
+    "BlogReply": "",
+    "FileNames": "",
+    "Dictionary": ""
+  },
+  "ResourceControl": {
+    "ManageProcesses": true
+  },
+  "HealthIsEnabled": false,
+  "HandlersIsEnabled": true,
+  "DisableStartup": false
 }
 ```
 
@@ -104,7 +134,7 @@ The primary item is the HandlerType. This tells GHOSTS to run a command (Command
 - UtcTimeOn | UtcTimeOff: "00:00:00": "24:00:00" to not shut off. Otherwise, enter an on and an off time to simulate things such as office hours of 9-5, etc. There are 30 minutes of jitter plus or minus from the time entered.
 - Loop: Set this to true to continue to execute this same command on a loop, or false to execute something just one time.
 
-```javascript
+```json
 {
    "HandlerType": "Command",
    "Initial": "",
@@ -137,7 +167,7 @@ Chrome
 
 We have to pass the browser window an initial value. If we don't want it to go anywhere at start, we could pass about:blank, otherwise we'd pass a url. These can be http or https.
 
-```javascript
+```json
 {
    "HandlerType": "BrowserChrome",
    "Initial": "http://google.com",
@@ -160,7 +190,7 @@ We have to pass the browser window an initial value. If we don't want it to go a
 
 Excel, PowerPoint, Word
 
-```javascript
+```json
 {
    "HandlerType": "Word",
    "Initial": "",
@@ -182,7 +212,7 @@ Excel, PowerPoint, Word
 
 For specific Timeline Events where the outcome is needed to be tracked, like for example, a client machine spawned inject, use a Trackable (via TrackableId in the following example):
 
-```javascript
+```json
 {
     "TimeLineHandlers": [
         {

docs/index.md

@@ -1,6 +1,6 @@
 # The GHOSTS NPC Framework
 
-Developed by Carnegie Mellon University's Software Engineering Institute (SEI), The GHOSTS Framework is an open-source (OSS) software framework that enables creating, deploying, and orchestrating complex non-player character (NPC) activity within training, exercise, simulation, and educational environments. Within GHOSTS there are several systems and applications, all of which are modular and extendable to fit a wide array of use cases and needs.
+Developed by Carnegie Mellon University's Software Engineering Institute (SEI), The GHOSTS Framework is an open-source (OSS) software framework that enables creating, deploying, and orchestrating complex non-player character (NPC) activity within training, exercise, simulation, and educational environments.
 
 If you've never seen GHOSTS in action, watch this quick three-minute introductory video on YouTube:
 
@@ -12,9 +12,9 @@ This is the [GHOSTS documentation site](https://cmu-sei.github.io/GHOSTS/) for t
 
 ## :material-cog: Cyber Ranges and Crucible
 
-GHOSTS is typically run on machines within a virtualized network, often referred to as "the range". This network can be as simple or as complex as required for training and exercise purposes.
+GHOSTS is typically run on machines within a virtualized network, often referred to as "the range". This network can be as simple or as complex as required for training, exercise, modeling, or simulation purposes.
 
-CERT's Cyber Workforce Development (CWD) team has a great deal of experience in building cyber ranges for training and exercise, captured in our technical report :material-file-document:[_Foundation of Cyber Ranges_](https://resources.sei.cmu.edu/library/asset-view.cfm?assetid=734198){:target="_blank"}. The report details the design considerations and execution plan for building high-fidelity, realistic virtual cyber ranges that deliver maximum training and exercise value for cyberwarfare participants.
+CERT's Cyber Mission Readiness (CMR) team has a great deal of experience in building cyber ranges for training, exercise, and simulation — captured in our technical report :material-file-document:[_Foundation of Cyber Ranges_](https://resources.sei.cmu.edu/library/asset-view.cfm?assetid=734198){:target="_blank"}. The report details the design considerations and execution plan for building high-fidelity, realistic virtual cyber ranges that deliver maximum training and exercise value for cyberwarfare participants.
 
 ???+ tip "Run GHOSTS on the Crucible Framework"
     ![Crucible Logo](assets/img/crucible-icon-c-alpha.svg)
@@ -26,7 +26,7 @@ CERT's Cyber Workforce Development (CWD) team has a great deal of experience in
 
 GHOSTS evolved in our quest to create more realistic NPCs within cyberwarfare training and exercise. In 2018, we outlined our thoughts in a technical report entitled [_GHOSTS in the Machine: A Framework for Cyber-Warfare Exercise NPC Simulation_](https://resources.sei.cmu.edu/library/asset-view.cfm?assetid=534316){:target="_blank"}.[^1] In that report, we outline how the GHOSTS framework accomplishes the creation of simulations in which NPCs realistically represent a vast array of possible encounters and outcomes. We have continued to follow our initial path since. The litmus has always been that if we looked over someone's shoulder while they were using a computer, that is what GHOSTS should look like.[^2]
 
-Since then the framework has expanded to include tools that [serve content in simulated environments](content/index.md), [create NPCs with details about them that we can programmatically use to drive their decision-making](animator/index.md), and [machine learning agent preference engines](spectre/index.md). If it is related to replicating human behavior within a cyber training or exercise scenario, we are interested in how GHOSTS can contribute within that space.
+Since then the framework has expanded to include tools that [serve content in simulated environments](content/index.md), [create NPCs with details about them that we can programmatically use to drive their decision-making](animator/index.md), and [machine learning agent preference engines](spectre/index.md). If it is related to replicating human behavior in terms of computing, we are interested in how GHOSTS can contribute within that space. Although these efforts listed here began separately, they are now all part of the GHOSTS framework proper.
 
 ## :material-bug: Reporting Bugs
 

docs/new.md

@@ -1,28 +1,28 @@
-# What's New 🆕
+# What's New 🆕 in GHOSTS v8 👻
 
-Welcome to what's new in the GHOSTS framework. Use this page to review the latest changes.
+???+ info "Welcome to GHOSTS"
+    Welcome to what's new in the GHOSTS framework. Use this page to review the latest changes.
 
-## GHOSTS v8 👻
+## Enhancements:
 
-- Beginning of moving to websockets — NPCs are now "always connected" 📶 and do not need to "check in" in order to execute activities, althought these are still being built out, and the old check-in system is still in place for the time being.
-- Adds ability to configure random timespans to a timeline for delaybefore or delayafter some activity is executed.
+- ANIMATOR and SPECTRE functionality merged into the GHOSTS API proper. Those projects are now archived. 📁
+- We have moved off mongo — all that data is now stored in Postgres ❤️.
+- General docker-compose cleanup in hopes install and initial configuration problems are greatly minimized. 🧹
+- Animator job management is now done through the UI. 🖥️
+- This release is the beginning of moving from web requests to websockets — NPCs are now "always connected" 📶 and do not need to "check in" or poll the server in order to execute activities, although these are still being built out, and the old system where clients poll the API server is still in place.
+- Adds ability to configure random timespans to a timeline for delay before or delay after some activity is executed. ⏳
 - Cleans up and simplifies configuration on client and server systems (breaking change). 🤖
-- Updates all framework and dependency versions to latest (.NET8).
-  - NetOffice binary update, particularly negatively impacting Excel, forcing it to continually restart. Updated to latest (and new source) for NetOffice libraries across all Office products.
-
-- "One docker-compose To Rule Them All" — we were seeing installs fail because of piecemeal installation of various ghosts modules. Now, there is a single docker-compose file that will install all of the necessary components for a GHOSTS system.
-  - Animator and SPECTRE merged into ghosts api proper. 
-- Animator job management is now done through the UI.
-- API endpoints have been re-organized in a more logical fashion. 🗂
+- "One docker-compose To Rule Them All" 💍 — we were seeing installs fail because of piecemeal installation of various ghosts modules. Now, there is a single docker-compose file that will install all of the necessary components for a GHOSTS system.
+- API endpoints have been re-organized in a more logical fashion. 🗂️
 - Added a favicon. 💅
 
-## Bug Fixes 🐛
+## Bug Fixes:
 
-- Updates Grafana docker compose to not use root.
-- Cleans up containers and ensures all are amd64 (not arm!). 
-- Fixes a bug to get the cmd window to stay open after running a command, despite attempts to use parameters to get it to stay open.
-  - Cmd now stays open, but this can have different outcomes based on the types of commands passed in, plus ghosts will reap windows in order to save on machine resources, so a used command window won't always be there later.
-- New improved RDP, fixes an array of different connectivity bugs.
-- Fixes bugs in delay before and after with fixed integer/long values over int maximum.
-- Adds "log" level to application logs.
-- Fixes documentation bug in machine group timelines.
+- Updates Grafana docker compose to not use root. 🚫👤
+- Cleans up containers and ensures all are amd64 (not arm!). 💻
+- Fixes a bug to get the cmd window to stay open after running a command, despite attempts to use parameters to get it to stay open. 🪟
+- Cmd now stays open, but this can have different outcomes based on the types of commands passed in, plus ghosts will reap windows in order to save on machine resources, so a used command window won't always be there later. ⚙️
+- New improved RDP, fixes an array of different connectivity bugs. 🌐
+- Fixes bugs in delay before and after with fixed integer/long values over int maximum. 🔢
+- Adds "log" level to application logs. 📝
+- Fixes documentation bug in machine group timelines. 🐛

docs/quickstart.md

@@ -20,7 +20,7 @@ Easy installation requires:
     docker-compose up -d
     ```
 
-4. Once the command completes, if you open [http://localhost:5000/api/home](http://localhost:5000/api/home){:target="_blank"} in your browser, you should see the initial API page outlining the version of the install, and a few test machine entries. If this page renders, your API is up, running, and available. If the page does not render, follow the advice in the [API troubleshooting section](core/api.md#troubleshooting).
+4. Once the command completes, if you open [http://localhost:5000/](http://localhost:5000/){:target="_blank"} in your browser, you should see the initial API page outlining the version of the install, and a few test machine entries. If this page renders, your API is up, running, and available. If the page does not render, follow the advice in the [API troubleshooting section](core/api.md#troubleshooting).
 
 [You will still need to set up Grafana](core/api.md#configuring-grafana). Beware that you must often `chown g_data`, which is the host location for the Graphana container as listed in the docker-compose file. Otherwise, the Grafana container will just continually restart in error due to insufficient permissions (detailed in [API troubleshooting](core/api.md#troubleshooting)).
 
@@ -40,7 +40,7 @@ The GHOSTS client should typically be run as a specific user, and not as an admi
 
 ### :material-linux: Linux Client
 
-1. Your client Linux machine will need to have the latest [Microsoft dotnetcore runtime](https://dotnet.microsoft.com/download){:target="_blank"} installed. Again, note that you only need the runtime installed, not the full SDK.
+1. Your client Linux machine will need to have version 6 of the [Microsoft dotnetcore runtime](https://dotnet.microsoft.com/download){:target="_blank"} installed. Again, note that you only need the runtime installed, not the full SDK.
 2. [Download the latest Linux client](https://github.com/cmu-sei/GHOSTS/releases/latest){:target="_blank"} zip file. Unzip to a folder such as ~/ghosts for the user that you want GHOSTS to run as.
 
 Note that on Linux machines running the client as root and utilizing web browsing may result in failures due to Gecko/Chromedriver display issues.

docs/stylesheets/extra.css

@@ -1,3 +1,3 @@
-:root > * { --md-primary-fg-color: green; }
+:root > * { --md-primary-fg-color: purple; }
 img[alt="Crucible Logo"] {width:4.5%; float:left; margin-right:7px; }
 img[alt="Animator Logo"] {width: 4.5%; }

mkdocs.yml

@@ -17,14 +17,14 @@ theme:
       toggle:
         icon: material/brightness-7
         name: Switch to dark mode
-      primary: green
+      primary: purple
 
     # Palette toggle for dark mode
     - scheme: slate
       toggle:
         icon: material/brightness-4
         name: Switch to light mode
-      primary: green
+      primary: purple
 markdown_extensions:
   # Python Markdown
   - abbr
@@ -65,7 +65,7 @@ nav:
   - Home: index.md
   - What's New: new.md
   - Quick Start: quickstart.md
-  - Core Client: 
+  - Client: 
     - core/client.md
     - Client Handlers:
       - Basic Configuration: core/handlers.md
@@ -90,15 +90,14 @@ nav:
         - Blogs/Drupal: core/handlers/blog_helper.md
         - Sharepoint: core/handlers/sharepoint_helper.md
       - Wmi: core/handlers/wmi.md
-  - Core API: 
+  - API: 
       - core/api.md
       - Capabilities:
         - Managing Timelines: core/api/timelines.md
-  - Animator: 
+  - Animations:
     - animator/index.md
     - Animation Jobs: animator/jobs.md
     - Running Animations: animator/run.md
-  - Spectre: spectre/index.md
   - Content Servers: 
     - content/index.md
     - Pandora: content/pandora.md