# ------------------------------------------------------------
# cawk is subjet to a MIT open-source licence
# please refer to the MIT licence file for further information
# ------------------------------------------------------------
# cawk is Copyright (C) 2024-2025 by Cedric Llorens
# ------------------------------------------------------------

# -----------------
# ---- introduction
# -----------------

cawk objective is to provide to the community a complete list of tests allowing to check 
network configurations whatever the supplier. Moreover, cawk is ONLY based on 3 well-known
packages: 
- gnu m4 aka m4
- gnu make aka gmake
- gnu awk aka gawk

these packages are very powerful today and the cawk project intends to ONLY use these
packages. No other langage, no database, no configure, etc. cawk try to keep things simple 
and understandable for the whole. The package size is very very low with a strong power :-).

All the AIs can help and build new tests because Cawk relies on well-known languages, 
Òmaking development and maintenance easier.

you can play with two predefined assessment <repo> and <run> included in cawk package, but
you can create/delete as many different assessments as you want <run_{audit_name}> based on 
your own different networks or customers.

In summary, in gawk && gmake && gm4 we trust :-) 

# ----------------------
# ---- cawk installation
# ----------------------

jump to a specific directory and extract cawk, cd to cawk root directory and execute the <gmake> 
command in order to have help on the cawk gmake targets. 

note : you may have to change the file support/tests.sed for finding the gawk path at your system 
level required for buidling tests:
	-%SED_GAWK_PATH% must point out the right path for gawk 
	( we set <!/usr/bin/env -S gawk -f> for a generic finding )

# ---------------------
# ---- cawk directories
# ---------------------

cawk has the following core directories:

- checkdiff : contains a cawk compliance print to compare when running <gmake check>, 
to be launched only after the first installation

- common : contains a <kind of library> (set of functions) included in the tests and 
others common useful scripts

- m4 : contains a <kind of m4 libraries> (set of m4 functions) that may be used at 
exceptions or tests level

- database : contains all the flat files databases used to manage various cawk options like sync
confs, sending emeails, etc.

	- scripts : contains scripts used for various usages as described : generally it requires the
	use of the internal inventory

		- each time sync_audit audit=AUDIT_NAME target is called, database_sync.script is launched 
		allowing to update device scope files. these file scopes are/must be stored sync_scopes with 
		the name audit(=AUDIT_NAME assessment)_inventory_sync_scope.txt
	
	- sync-scopes : sync scopes files generated by the scripts described above. It could be useful
	to define various sub-assessments for a as same audit=AUDIT_NAME like
		- audit_network_part1 assessment with scope associated to
		- audit_network_part2 assessment with scope associated to
		- etc.

- tests : contains a collection of individual tests <*.gawk.template> or <*.gawk.m4> per supplier :

	- there are 3 types of core tests directories:
		- tests/repo : contains the full collection of cawk coded tests, that can be used in 
		tests/run or tests/run_{audit_name} directories
		- tests/run  : a full copy of repo by default (you may copy tests from tests/repo 
		or add your own tests)
		-- audit=AUDIT_NAME assessment --
		- tests/run_{audit_name} : contains a full copy of repo tests when created (you can 
		remove or add other tests or add your own tests)
		-- audit=AUDIT_NAME assessment --

	- inside each core tests directories, you have a full set of tests supplier directories

	- a test has <.template> suffix or <.m4> suffix, but the test is converted to <.gawk> with the 
	support of support/tests.sed and the cawk root Makefile. this <step> allows to write tests more 
	easily && to enforce env system portability, so each test may to %SED_VAR% aka:
		- %SED_BLOCK_JUNIPER% : space identation used for block hierarchy
		- %SED_COMMON_PATH% = to point out the common <kind of library>
		- %SED_GAWK_PATH% = to point out the right path for gawk 
		etc.
		these values can be changed thanks to the file support/tests.sed

		moreover, a cawk m4 parse block macro allows to parse any type configuration without 
		managing the block hierarchy level as it is automatically generated by the macro when 
		the <.gawk> is generated

- confs : contains a collection of configurations per supplier

	- there are 3 types of core confs directories:
		- confs/repo : contains a collection of cawk tests confs
		- confs/run : a full copy of repo by default (you may copy configurations from confs/repo 
		or add your own configurations)
		-- audit=AUDIT_NAME assessment --
		- confs/run_{audit_name} : a full copy of repo confs when created (you can remove or 
		add other configurations)
		-- audit=AUDIT_NAME assessment --

	- inside each core conf directories, you have a full set of configuration supplier directories

	- inside each core conf directories, you have a sync directory allowing to sync confs with a
	  central repository, please refer to the cawk sync section for further information

- exceptions : contains a collection of exceptions per supplier applied for reporting

	- there are 3 types of core exceptions directories:
		- exceptions/repo : contains a collection of up-and-running exceptions
		- exceptions/run : contains a collection of up-and-running exceptions (a full copy of repo)
		-- audit=AUDIT_NAME assessment --
		- exceptions/run_{audit_name} : contains a full copy of repo exceptions when created
		-- audit=AUDIT_NAME assessment --

	- inside each core exception directories, you have a full set of exception supplier files

- logs : to store all the cawk logs if needed

- reports : contains assessment reports (and summary), each report has the same format

	- there are 3 types of core report directories:
		- report/repo : empty by default, contains the repo assessment results
		- report/run : empty by default, contains the run assessment results
		-- audit=AUDIT_NAME assessment --
		- report/run_{audit_name} : empty by default, contains the run_{audit_name} assessment results
		-- audit=AUDIT_NAME assessment --

	- each of these directories has a sub-directory <archives>. each time an asessment is performed
	  a tar.gz file is built and stored in this directory including the date in filename

- support : contains files helping for building cawk:

	- tests.sed : used when building the tests in order to make change of the set %SED_VAR%

- backup : contains tar.gz files when using cawk backup targets, please refer to the cawk backup
           section for further information

# -----------------------
# ---- cawk gmake targets
# -----------------------

just execute <gmake> in the cawk root directory and all the cawk targets are detailed

# ------------------------
# ---- cawk gmake parallel
# ------------------------

in standard mode, cawk performs assessment not in parallel mode. to use prarallel mode,
you have to modify the Makefile.support.mk file thanks to these gmake VARS:

	# --------------- cawk parallel options
	# enable parallel yes/no
	MAKE_PARALLEL = yes
	# number of files to process per target (all targets are processed in parallel)
	MAKE_FILES_PER_TARGET = 100

	# --------------- gmake parallel options
	# gmake number of jobs
	MAKE_J = 4
	# gmake load average
	MAKE_LOAD_AVG = 3

once cawk parallel mode is enabled, before performing assessment, cawk build one Makefile
per os in tmp directory. once done, it performs each Makefile in parallel mode to offer 
enhanced performances for a huge set of files.

# -----------------------------------
# ---- cawk risk level && status code
# -----------------------------------

cawk allows the following risk levels:
	- high   : for (high impacts) security item
	- medium : for (medium impacts) security item
	- low    : for (low impacts) security item
	- info   : for audit/information item

cawk allows the following status codes:
	- pass    : Check passed successfully
	- error   : Check failed or error occurred
	- warning : check passed with warning

# ---------------------------
# ---- cawk assessment format
# ---------------------------

cawk follows an assessment report format consisting of six fields:
	1) conf_name: The name of the configuration
	2) test_name: The name of the test
	3) error_line: The output describing the error
	4) line_nb: The line number
	5) risk_level: The risk level (high, medium, low, info)
	6) status_code: The status code (pass, error, warning)

# ---------------------------------
# ---- cawk security key indicators
# ---------------------------------

cawk computes two key security indicators in the summary report:
	- security Compliance : expressed as a percentage between 0% and 100% (100% best score)
	- average number of errors per Device : measured from 0 up to an upper bound (0 best score)

# --------------------
# ---- cawk first use
# --------------------

after install, cd to the cawk root directory, execute:

	- gmake : provide all cawk gmake targets

	- gmake clean check_repo view_repo : it applies the test repo to conf repo 
	and see results, you may check reports/repo assessment files 

	- gmake clean check_run view_run : it applies the test run to conf run and 
	see results, you may check reports/run assessment files

	-- audit=AUDIT_NAME assessment --

	- gmake create_audit audit=client1 : it creates client1 assessment, all tests 
	from repo are copied, all exceptions from repo are copied, all confs from repo 
	are copied. now, you may remove or/and add tests, remove and/or add true confs 
	and run this assessment as such

          	- gmake check_run audit=client1 : it applies the test run_client1 to conf 
		run_client1,you may also check reports/run_client1 assessment files
          
          	- gmake check_run view_run audit=client1 : it applies the test run_client1 
		to conf run_client1 and see results,you may also check reports/run_client1 
		assessment files

          	- gmake check_run view_run audit=client1 supplier=cisco-ios : same as previous
		but only for cisco-ios configurations 

          	- gmake delete_audit audit=client1 : remove the client1 assessment

	- gmake list_audit : list all the AUDIT_NAMEs (audit=AUDIT_NAME)

	- gmake run_audit : run assessments for all the AUDIT_NAMEs (audit=AUDIT_NAME)

	-- audit=AUDIT_NAME assessment --

# ------------------------------------------
# ---- cawk database
# ------------------------------------------

cawk allows to manage flat file databases in order to use cawk options like sync confs,
sending emeails, etc. this is only available for audit=AUDIT_NAME assessment

	- gmake database_view : view all the databases

	-- sync database --

	- the cawk sync database format is the following where fields are separated by spaces :

		each time sync_audit audit=AUDIT_NAME target is called, database/scripts/database_sync.script
		is launched allowing to update device scope files that can be used as the field number 4 described 
		hereafter. these file scopes are/must be stored in database/sync_scopes with the name :
		audit(=AUDIT_NAME assessment)_inventory_sync_scope.txt

		- 1 field is the audit name (i.e. audit=AUDIT_NAME)
		- 2 field is the various sync paths separated by comma (no space) 
		    like /conf/ or /conf/cawk/,/conf/cawk_2/ (i.e. dir=SYNC_PATH_DIR)
		- 3 field is an extended regex to select devices pattern matching 
		    like .* or .*switch.* (i.e. regex=REGEX_PATTERN)
		 - 4 field is a file containing a list of devices matching a device scope based on internal inventory

	- gmake database_sync_(add,update) audit=AUDIT_NAME dir=SYNC_PATH regex=REGEX_PATTERN/.* scope=SCOPE_FILE/none: 
	  add/update an entry in the cawk sync database

	  note : regex may allow to build sub-scope of an assessment like
		     audit=cawk_customer1_router with regex matching only routers
			 audit=cawk_customer1_switch with regex matching only switches
	
	- gmake database_sync_del audit=AUDIT_NAME : delete an entry in the cawk sync database
	
	-- email database --

	- the email sync database format is the following where fields are separated by spaces :

		- 1 field is the audit name"
		- 2 field is the dst list of emails separated by comma (no space) like email1,email2,email3"
		- 3 field is the cc list of emails separated by comma (no space) like email1,email2,email3"

	- gmake database_email_(add,update) audit=AUDIT_NAME dst=EMAIL_LIST/none cc=EMAIL_LIST/none: 
	  add/update an entry in the cawk email database

	- gmake database_email_del audit=AUDIT_NAME : delete an entry in the cawk email database

# ------------------------------------------
# ---- cawk sync audit=AUDIT_NAME assessment
# ------------------------------------------

cawk allows to sync confs only for audit=AUDIT_NAME assessments. the way of working is:

	- use the gmake database_sync_(add,del,update) targets to update the cawk sync database
	  for audit=AUDIT_NAME assessment

	- gmake sync_run audit=AUDIT_NAME 	

	(CAUTION) local confs of the audit=AUDIT_NAME are removed 

	confs soft links from the central confs repositories to cawk confs/run_audit/confs.os 
	are automatically built and pushed in the right confs.os directory. In fact, each conf 
	is analyzed and os detected like cisco-ios, cisco-xe, etc.

	- gmake sync_run_audit (sync all the audit=AUDIT_NAMEs)

note : we do recomment that the central confs repository is owner by an different user than
the cawk package in order to enforce that the cawk user can only read the central confs
repository.

# -------------------------------------------
# ---- cawk email audit=AUDIT_NAME assessment
# -------------------------------------------

cawk allows to email audit=AUDIT_NAME assessments in zip file format. the way of working is:

	- use the gmake database_email_(add,del,update) targets to update the cawk email database
	  for audit=AUDIT_NAME assessment

	- gmake email_send audit=AUDIT_NAME : send email by refering to cawk email database 	

	- gmake email_send_audit (email all the audit=AUDIT_NAMEs)

# ---------------------------------------------
# ---- cawk backup audit=AUDIT_NAME assessment
# ---- cawk restore audit=AUDIT_NAME assessment
# ---------------------------------------------

cawk allows to build backup/restore only for audit=AUDIT_NAME assessments. 
it saves all tests, exceptions, confs, reports linked to this assessment. 

backup one audit assessment --
for example, to backup an audit=cawk assessment, execute : 
 - gmake backup_run audit=cawk
   the backup file is stored in backup/run_cawk.<date>.tar.gz

restore one audit assessment --
copy your backup in the cawk backup directory, execute: 
 - gmake restore_run audit=cawk file=backup_path_file 
as such, the audit=cawk assessment is restored.

backup all audit assessments --
you can backup cawk database and all your audit=AUDIT_NAME assessments in one command, execute :
 - gmake backup_run_audit
   the backup file is stored in backup/run_audit.<date>.tar.gz

restore all audit assessments --
copy your backup in the cawk backup directory, execute: 
 - gmake restore_run_audit file=backup_path_file 
as such, all the assessments are restored.

# ---------------------------
# ---- cawk version migration
# ---------------------------

- backup all your audit=AUDIT_NAME assessments and save it in a secure directory
- install a new version of cawk
- copy your backup in the cawk backup directory
- restore your all audit=AUDIT_NAME assessments

# -------------------
# ---- cawk community
# -------------------

if someone intends to submit a test and a configuration associated to, if approved, then the 
test will added to the package and the name of author will be added to the AUTHORS list.

the requester must use a Pull Request to submit an evolution as such:

	# project clone
	git clone https://github.com/cedricllorens/cawk.git
	cd cawk

	# build your own develop branch
	git checkout -b cawk_name_update

	# perform your changes
	git add .
	git commit -m "Update ..."

	# push your update
	git push origin cawk_name_update

	# create the push request at Github

enjoy in particating to cawk or to simply use cawk, 
cedric llorens.