Merge branch 'develop-cfs-extravars' into develop. Close #106.

**Description**

The generated cFS application may need definitions that come from external
files, such as message IDs from other applications, or structs defined
elsewhere.

Users may also want to customize additional aspects of the generated
application without having to modify the template, such as passing additional
variables to be expanded in the template.

There should be a setting to include additional headers or other variables so
that the generated app never has to be modified.

**Type**

- Feature: facilitate customizing generated applications.

**Additional context**

None.

**Requester**

- Ivan Perez.

**Method to check presence of bug**

Not applicable (not a bug).

**Expected result**

Ogma provides a mechanism to provide additional variables to be expanded in cFS
applications.

The following dockerfile first checks that passing the additional extra
variables files does not affect the result unless the variable is mentioned in
the template, by comparing the result to a run without passing any extra vars.
It later adds a file with a custom variable, and then passes an extra template
variable using the new flag, after which it prints the message "Success":

```
FROM ubuntu:focal

ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update

RUN apt-get install --yes \
      curl g++ gcc git libgmp3-dev libz-dev make pkg-config

RUN mkdir -p $HOME/.local/bin
ENV PATH=$PATH:/root/.local/bin/

RUN curl https://downloads.haskell.org/~ghcup/0.1.17.7/x86_64-linux-ghcup-0.1.17.7 -o $HOME/.local/bin/ghcup
RUN chmod a+x $HOME/.local/bin/ghcup

ENV PATH=$PATH:/root/.ghcup/bin/
RUN ghcup install ghc 9.10
RUN ghcup install cabal 3.12
RUN ghcup set ghc 9.10.1
RUN cabal update

SHELL ["/bin/bash", "-c"]
CMD git clone $REPO \
    && cd $NAME \
    && git checkout $COMMIT \
    && cabal install ogma-cli:ogma \
    && ogma cfs --variable-file ogma-cli/examples/cfs-variables --variable-db ogma-cli/examples/cfs-variable-db --app-target-dir original --handlers-file ogma-cli/examples/cfs-handlers \
    && cp -r ogma-core/templates/copilot-cfs custom \
    && find custom -iname '.gitignore' -delete \
    && echo '{ "message": "Success" }' >> template-vars \
    && ogma cfs --variable-file ogma-cli/examples/cfs-variables --variable-db ogma-cli/examples/cfs-variable-db --app-target-dir new --handlers-file ogma-cli/examples/cfs-handlers \
        --app-template-dir custom --template-vars template-vars \
    && diff -rq original new \
    && echo '{{message}}' >> custom/fsw/result \
    && rm -rf new \
    && ogma cfs --variable-file ogma-cli/examples/cfs-variables --variable-db ogma-cli/examples/cfs-variable-db --app-target-dir new --handlers-file ogma-cli/examples/cfs-handlers \
        --app-template-dir custom --template-vars template-vars \
    && cat new/fsw/result
```

Command (substitute variables based on new path after merge):
```sh
$ docker run -e "REPO=https://github.com/NASA/ogma" -e "NAME=ogma" -e "COMMIT=<HASH>" -it ogma-verify-106
```

**Solution implemented**

Update `ogma-core` to expect a new argument with a JSON file listing
additional keys to expand as variables in the template.

Update `ogma-cli` to expose the new argument to the user via an optional
flag.

Update `ogma-cli` to document new feature in the README.

**Further notes**

None.

ogma-cli/CHANGELOG.md

@@ -7,6 +7,7 @@
 * Update installation instructions to use cabal install (#149).
 * Update README with new cFS template variables (#229).
 * Expose handlers-file argument to cFS backend (#234).
+* Expose template-vars argument to cFS backend (#106).
 
 ## [1.6.0] - 2025-01-21
 

ogma-cli/README.md

@@ -135,6 +135,8 @@ be made available to the monitor.
 and the message they are included with.
 - `--handlers-file FILENAME`: a file containing a list of known fault handlers
   or triggers.
+- `--template-vars FILENAME`: a JSON file containing a list of additional
+  variables to expand in the template.
 
 The following execution generates an initial cFS application for runtime
 monitoring using Copilot:
@@ -235,6 +237,15 @@ the processing to an auxiliary function.
   by the monitoring application to notify of faults or updates from the
   monitoring system.
 
+Additionally, the `cfs` command accepts a file with a JSON object listing
+additional variables to be expanded in the template. To make use of this
+feature, we encourage users to modify the template to fit their needs, and pass
+the file as an argument via the flag `--template-vars`. Values passed to the
+template are also respected in file names; for example, if the JSON file
+contains an object key `"APP_NAME"` with the value `"Monitor"`, a file or
+directory in the template named `My_{{APP_NAME}}` will be expanded in the
+destination directory as `My_Monitor`.
+
 We understand that this level of customization may be insufficient for your
 application. If that is the case, feel free to reach out to our team to discuss
 how we could make the template expansion system more versatile.

ogma-cli/src/CLI/CommandCFSApp.hs

@@ -56,11 +56,12 @@ import Command.CFSApp ( ErrorCode, cFSApp )
 
 -- | Options needed to generate the cFS application.
 data CommandOpts = CommandOpts
-  { cFSAppTarget      :: String
-  , cFSAppTemplateDir :: Maybe String
-  , cFSAppVarNames    :: String
-  , cFSAppVarDB       :: Maybe String
-  , cFSAppHandlers    :: Maybe String
+  { cFSAppTarget       :: String
+  , cFSAppTemplateDir  :: Maybe String
+  , cFSAppVarNames     :: String
+  , cFSAppVarDB        :: Maybe String
+  , cFSAppHandlers     :: Maybe String
+  , cFSAppTemplateVars :: Maybe String
   }
 
 -- | Create <https://cfs.gsfc.nasa.gov/ NASA core Flight System> (cFS)
@@ -76,6 +77,7 @@ command c =
     (cFSAppVarNames c)
     (cFSAppVarDB c)
     (cFSAppHandlers c)
+    (cFSAppTemplateVars c)
 
 -- * CLI
 
@@ -122,6 +124,13 @@ commandOptsParser = CommandOpts
             <> help strCFSAppHandlerListArgDesc
             )
         )
+  <*> optional
+        ( strOption
+            (  long "template-vars"
+            <> metavar "FILENAME"
+            <> help strCFSAppTemplateVarsArgDesc
+            )
+        )
 
 -- | Argument target directory to cFS app generation command
 strCFSAppDirArgDesc :: String
@@ -146,3 +155,8 @@ strCFSAppVarDBArgDesc =
 strCFSAppHandlerListArgDesc :: String
 strCFSAppHandlerListArgDesc =
   "File containing list of Copilot handlers used in the specification"
+
+-- | Argument template variables to cFS app generation command
+strCFSAppTemplateVarsArgDesc :: String
+strCFSAppTemplateVarsArgDesc =
+  "JSON file containing additional variables to expand in template"

ogma-core/CHANGELOG.md

@@ -10,6 +10,7 @@
 * Update Copilot struct code generator to use new function names (#231).
 * Simplify Copilot struct definitions by using generics (#199).
 * Update cFS backend to process a handlers file (#234).
+* Update cFS backend to process a template variables file (#106).
 
 ## [1.6.0] - 2025-01-21
 

ogma-core/src/Command/CFSApp.hs

@@ -49,7 +49,9 @@ module Command.CFSApp
 import qualified Control.Exception    as E
 import           Control.Monad.Except ( ExceptT (..), liftEither, runExceptT,
                                         throwError )
-import           Data.Aeson           ( ToJSON (..), decode )
+import           Data.Aeson           ( ToJSON (..), Value (Null, Object),
+                                        decode, eitherDecode, object )
+import           Data.Aeson.KeyMap    ( union )
 import           Data.List            ( find )
 import           Data.Text            ( Text )
 import           Data.Text.Lazy       ( unpack )
@@ -58,6 +60,7 @@ import           System.FilePath      ( (</>) )
 
 -- Internal imports: auxiliary
 import Command.Result         ( Result (..) )
+import Data.ByteString.Extra  as B ( safeReadFile )
 import Data.Location          ( Location (..) )
 import System.Directory.Extra ( copyTemplate )
 
@@ -76,8 +79,11 @@ cFSApp :: FilePath       -- ^ Target directory where the application
        -> Maybe FilePath -- ^ File containing a list of handlers used in the
 			 --   Copilot specification. The handlers are assumed
                          --   to receive no arguments.
+       -> Maybe FilePath -- ^ File containing additional variables to make
+                         --   available to the template.
        -> IO (Result ErrorCode)
-cFSApp targetDir mTemplateDir varNameFile varDBFile handlersFile = do
+cFSApp targetDir mTemplateDir varNameFile varDBFile handlersFile
+       templateVarsF = do
 
   -- We first try to open the two files we need to fill in details in the CFS
   -- app template.
@@ -92,10 +98,14 @@ cFSApp targetDir mTemplateDir varNameFile varDBFile handlersFile = do
 
   handlersE <- parseOptionalRequirementsListFile handlersFile
 
-  case (varDBE, handlersE) of
-    (Left  e, _)                  -> return $ cannotOpenDB varDBFile e
-    (_,       Left e)             -> return $ cannotOpenHandlers handlersFile e
-    (Right varDB, Right handlers) -> do
+  templateVarsE <- parseOptionalTemplateVarsFile templateVarsF
+
+  case (varDBE, handlersE, templateVarsE) of
+    (Left  e, _     , _)      -> return $ cannotOpenDB varDBFile e
+    (_,       Left e, _)      -> return $ cannotOpenHandlers handlersFile e
+    (_,       _,      Left e) -> return $ cannotOpenTemplateVars templateVarsF e
+
+    (Right varDB, Right handlers, Right templateVars) -> do
 
       -- The variable list is mandatory. This check fails if the filename
       -- provided does not exist or if the file cannot be opened. The condition
@@ -126,10 +136,11 @@ cFSApp targetDir mTemplateDir varNameFile varDBFile handlersFile = do
             -- This is a Data.List.unzip4
             let (vars, ids, infos, datas) = foldr f ([], [], [], []) varNames
 
-            let subst = toJSON $ appComponents vars ids infos datas handlers
+            let subst  = toJSON $ appComponents vars ids infos datas handlers
+                subst' = mergeObjects subst templateVars
 
             -- Expand template
-            copyTemplate templateDir subst targetDir
+            copyTemplate templateDir subst' targetDir
 
             return Success
 
@@ -224,6 +235,22 @@ parseOptionalRequirementsListFile :: Maybe FilePath
 parseOptionalRequirementsListFile Nothing   = return $ Right []
 parseOptionalRequirementsListFile (Just fp) = E.try $ lines <$> readFile fp
 
+-- | Process a JSON file with additional template variables to make available
+-- during template expansion.
+parseOptionalTemplateVarsFile :: Maybe FilePath
+                              -> IO (Either String Value)
+parseOptionalTemplateVarsFile Nothing   = return $ Right $ object []
+parseOptionalTemplateVarsFile (Just fp) = do
+  content <- B.safeReadFile fp
+  let value = eitherDecode =<< content
+  case value of
+    Left  _          -> return value
+    Right (Object _) -> return value
+    Right Null       -> return value
+    Right _          -> return $
+      Left "The value passed in the JSON file is not an object."
+
+
 -- * Exception handlers
 
 -- | Exception handler to deal with the case in which the variable DB cannot be
@@ -283,6 +310,21 @@ cannotOpenHandlers (Just file) _e =
     msg =
       "cannot open handlers file " ++ file
 
+-- | Exception handler to deal with the case in which the template vars file
+-- cannot be opened.
+cannotOpenTemplateVars :: Maybe FilePath -> String -> Result ErrorCode
+cannotOpenTemplateVars Nothing _e =
+    Error ecCannotOpenTemplateVarsCritical msg LocationNothing
+  where
+    msg =
+      "cannotOpenTemplateVars: this is a bug. Please notify the developers"
+cannotOpenTemplateVars (Just file) e =
+    Error ecCannotOpenTemplateVarsUser msg (LocationFile file)
+  where
+    msg =
+      "Cannot open file with additional template variables: " ++ file ++
+      ": " ++ e
+
 -- * Error codes
 
 -- | Encoding of reasons why the command can fail.
@@ -306,6 +348,14 @@ ecCannotOpenHandlersCritical = 2
 ecCannotOpenHandlersUser :: ErrorCode
 ecCannotOpenHandlersUser = 1
 
+-- | Internal error: template vars file cannot be opened.
+ecCannotOpenTemplateVarsCritical :: ErrorCode
+ecCannotOpenTemplateVarsCritical = 2
+
+-- | Error: the template vars file provided by the user cannot be opened.
+ecCannotOpenTemplateVarsUser :: ErrorCode
+ecCannotOpenTemplateVarsUser = 1
+
 -- | Error: the variable file provided by the user cannot be opened.
 ecCannotOpenVarFile :: ErrorCode
 ecCannotOpenVarFile = 1
@@ -318,3 +368,14 @@ ecCannotEmptyVarList = 1
 -- permissions or some I/O error.
 ecCannotCopyTemplate :: ErrorCode
 ecCannotCopyTemplate = 1
+
+-- * Auxiliary Functions
+
+-- | Merge two JSON objects.
+--
+-- Fails if the values are not objects or null.
+mergeObjects :: Value -> Value -> Value
+mergeObjects (Object m1) (Object m2) = Object (union m1 m2)
+mergeObjects obj         Null        = obj
+mergeObjects Null        obj         = obj
+mergeObjects _           _           = error "The values passed are not objects"