How can we trace where chunks came from?

[balacij]

Let's say we have an arbitrary Doc. For all of our artifacts, we use a Doc to represent its (textual) body. The Doc is 'final' in the sense that it is purely text and contains no explicit links to where it came from, nor how it came to be. This notion extends to any of our chunks when created.

When rendering in certain areas, we have a chunk database in scope which we rely on for chunk information. For example, Sentence carries UIDs that, when rendered into a Doc, is coupled with a 'term renderer' to pull the term of whatever chunk is associated with that UID. However, once we have that final Doc, the information of the term renderers and the original UIDs/chunks is 'forgotten' and we can't use the Doc to figure out what chunks it relied on when rendered. I think this whole notion extends to any of our chunks.

In other areas, where we don't have that chunk database in scope, we really want one to be. For example, when creating our Doxygen configs:

(Doxygen code here, just hidden to save space, see lines 36, 55, 86)

Drasil/code/drasil-code/lib/Language/Drasil/Code/Imperative/Doxygen/Import.hs

Lines 27 to 87 in 2372a12

	-- | Converts a 'Verbosity' to a rendered value that can be used in a Doxygen
	-- configuration file.
	verbosityToDoc :: Verbosity -> Doc
	verbosityToDoc Verbose = no
	verbosityToDoc Quiet = yes
	-- | Renderings of comments regarding the default tag values for various Doxygen
	-- settings.
	--
	-- TODO: `makeDoxConfig` needs access to a `ChunkDB` so that we can re-write
	-- these `Doc`s as `Sentence`s and render them back into `Doc`s appropriately.
	defNo, defYes :: Doc
	defNo = defaultValSentence <> no <> text "."
	defYes = defaultValSentence <> yes <> text "."
	-- | Renders a Doxygen configuration file.
	--
	-- * Sets the PROJECT_NAME field according to the given name.
	-- * Sets the INPUT field to the header files contained in the given GOOLState.
	-- * Sets the OPTIMIZE_OUTPUT_JAVA field according to the OptimizeChoice parameter.
	-- * Sets the QUIET field according to the given Verbosity.
	makeDoxConfig :: ProjName -> GOOLState -> OptimizeChoice -> Verbosity -> Doc
	makeDoxConfig prog s opt v =
	let fs = nub (s ^. headers ++ maybeToList (s ^. mainMod))
	in vcat [
	text "# Doxyfile 1.8.15",
	blank,
	text "# This file describes the settings to be used by the documentation system",
	text "# doxygen (www.doxygen.org) for a project.",
	text "#",
	text "# All text after a double hash (##) is considered a comment and is placed in",
	text "# front of the TAG it is preceding.",
	text "#",
	text "# All text after a single hash (#) is considered a comment and will be ignored.",
	text "# The format is:",
	text "# TAG = value [value, ...]",
	text "# For lists, items can also be appended using:",
	text "# TAG += value [value, ...]",
	text "# Values that contain spaces should be placed between quotes (\\\" \\\").",
	blank,
	text "#---------------------------------------------------------------------------",
	text "# Project related configuration options",
	text "#---------------------------------------------------------------------------",
	blank,
	text "# This tag specifies the encoding used for all characters in the configuration",
	text "# file that follow. The default is UTF-8 which is also the encoding used for all",
	text "# text before the first occurrence of this tag. Doxygen uses libiconv (or the",
	text "# iconv built into libc) for the transcoding. See",
	text "# https://www.gnu.org/software/libiconv/ for the list of possible encodings.",
	text "# The default value is: UTF-8.",
	blank,
	text "DOXYFILE_ENCODING = UTF-8",
	blank,
	text "# The PROJECT_NAME tag is a single word (or a sequence of words surrounded by",
	text "# double-quotes, unless you are using Doxywizard) that should identify the",
	text "# project for which the documentation is generated. This name is used in the",
	text "# title of most generated pages and in a few other places.",
	text "# The default value is: My Project.",
	blank,
	text $ "PROJECT_NAME = \"" ++ prog ++ "\"",
	blank,

For another example, without our chunk database in scope, we have "string" (chunks rendered about some function, really) duplication/hard-coded (hence PRs like #3642) in our Docs or other various chunks.

@smiths created the image (which I often use) that shows how various artifacts (such as constants, terms, authors, etc.) are shared against a particular originating SRS. I suppose a part of my question with this discussion post is: how can we actually show that same image without needing to manually trace the relations ourselves?

To get this traceability information, I get the feeling that we either (a) want some sort of 'lazy' variant of each of our chunks that we can use to calculate that traceability information, (b) want to use some TemplateHaskell-magic to calculate this information for us [this sounds like it would be brutal], or (c) just want features of Haskell that we can't have.

[balacij]

I think I'm struggling to articulate this all. There are 2 issues (arguably just 1) I'm trying to bring up:

1. when we create chunks, we don't store any information in them explaining where they came to be (I previously wanted this to come about in UIDs, but I disagree with that view now) and without going meta-Haskell, we can't tell where they came from [i.e., we don't have any real trace information from our chunks -- we can't pick an arbitrary chunk and search for the tree of knowledge it depends on], and
2. whenever creating chunks without a chunk database in scope, we rely on Haskell-level variables and otherwise as an alternative to referencing chunks and how we want to use them, which further breaks the chain of traceability.

[balacij]

Am I overloading 'traceability'? In the context of the SRS, we say that "the trace of X is a list of chunks that X references," but here I'm using our other definition: "the trace of X is a list of chunks that X relies on to be whole". The second definition doesn't necessarily need any sort of reference to the originating chunk.

For example, a theory that instantiates the abstract variables of another 'higher-level' theory doesn't really need to reference the originating theory in its description. The derivation sections of theories aren't hard-required, so we don't necessarily always have that information either unless the transcription was fully complete.

Another example is code fragments. When we generate code (specifically the chunks that go into that code), we don't hold on to any references to the original inputs to the generation of the code.

[smiths]

I think you are on to something very important here @balacij. We definitely do want two-way traceability. We want to know to be able to look at a chunk and trace back to what knowledge was used to create it. This applies for the theories in the SRS and the code fragements.

We don't unofficially enforce it now in our SRS examples for the theories. I've tried to make sure that our examples point to all the pieces (assumptions, definitions, other theories) that they depend on. If the notes section isn't adequate to capture this, we add a detailed derivation. Right now this is something that we fully rely on the human authors to do, but I think in our refined theories version of the SRS we'll be able to make this more explicit. I hope we will be able to automatically check to see if the theory is properly justified/explained/supported. For instance, we can have the assumptions propagate through the theories. If Theory A uses Theory B, then Theory A inherits all the assumptions from Theory B. It will also add its own assumptions. Some of the assumptions from Theory B may be replaced by new assumptions from Theory A, but this will have to be made explicit by the creator of the SRS. We want to be able to capture the "tree of knowledge" (I like that term that you used above!) that a theory depends on.

For the code fragments we want the same thing. We want to be able to capture the "tree of knowledge" that led to the code. That way we can say that if anything changes in the tree of knowledge, the code will likely change. If nothing changes, then the code will not change. If we can achieve this, then we will have achieved something really important for project certification. Right now, projects, like those at OPG, are not recertified because the expense is too high. To be conservative, any changes to the theory, code, equipment, at OPG requires redoing the entire certification exercise. If they knew exactly how changes propagated, then they would only have to recertify the parts impacted by the change.

I do not know how we do the kind of traceability you are asking for, that is a question I would ask you and @JacquesCarette. 😄 I do know that full two-way traceability would be an amazing feature that we could definitely brag about! 😄

[JacquesCarette]

Right now, we definitely do not trace where chunks came from. Some, of course, as 'ab initio' (i.e. created from nothing). But our combinators do not leave behind any kind of traceability information at this particular level. Only higher up to we leave behind more information that we can use for traceability.

Having said that, we get a lot of traceability information from the 'assembling' information (right now, buried in the SRS). That is not necessarily a bad way to do it - but we'd have to upgrade our "weaving programs" language to be much more explicit about this.