report-converter is a Python tool which can be used to create a CodeChecker
report directory from the given code analyzer output which can be stored to
a CodeChecker server.
- Install guide
- Usage
- Supported analyzer outputs
- Plist to html tool
- Report hash generation module
- License
# Create a Python virtualenv and set it as your environment.
make venv
source $PWD/venv/bin/activate
# Build and install report-converter package.
make package$ report-converter --help (click to expand)
usage: report-converter [-h] -o OUTPUT_DIR -t TYPE [--meta [META [META ...]]]
[--filename FILENAME] [-c] [-v]
file
Creates a CodeChecker report directory from the given code analyzer output
which can be stored to a CodeChecker web server.
positional arguments:
file Code analyzer output result file which will be parsed
and used to generate a CodeChecker report directory.
optional arguments:
-h, --help show this help message and exit
-o OUTPUT_DIR, --output OUTPUT_DIR
This directory will be used to generate CodeChecker
report directory files.
-t TYPE, --type TYPE Specify the format of the code analyzer output.
Currently supported output types are: asan, clang-
tidy, coccinelle, cppcheck, cpplint, eslint,
fbinfer, golint, kernel-doc, lsan, mdl, msan,
pyflakes, pylint, smatch, sparse, sphinx, spotbugs,
tsan, tslint, ubsan.
--meta [META [META ...]]
Metadata information which will be stored alongside
the run when the created report directory will be
stored to a running CodeChecker server. It has the
following format: key=value. Valid key values are:
analyzer_command, analyzer_version. (default: None)
--filename FILENAME This option can be used to override the default plist
file name output of this tool. This tool can produce
multiple plist files on the given code analyzer output
result file. The problem is if we run this tool
multiple times on the same file, it may override some
plist files. To prevent this we can generate a unique
hash into the plist file names with this option. For
example: '{source_file}_{analyzer}_{file_hash}_xxxxx'.
{source_file}, {analyzer} and {file_hash} are special
values which will be replaced with the current
analyzer, source file name and hash of the absolute
file path where the bug was found. (default:
{source_file}_{analyzer}_{file_hash})
-c, --clean Delete files stored in the output directory. (default:
False)
-v, --verbose Set verbosity level. (default: False)
Supported analyzers:
asan - AddressSanitizer, https://clang.llvm.org/docs/AddressSanitizer.html
clang-tidy - Clang Tidy, https://clang.llvm.org/extra/clang-tidy
coccinelle - Coccinelle, https://github.com/coccinelle/coccinelle
cppcheck - Cppcheck, http://cppcheck.sourceforge.net
cpplint - cpplint, https://github.com/cpplint/cpplint
eslint - ESLint, https://eslint.org/
fbinfer - Facebook Infer, https://fbinfer.com
golint - Golint, https://github.com/golang/lint
kernel-doc - Kernel-Doc, https://github.com/torvalds/linux/blob/master/scripts/kernel-doc
lsan - LeakSanitizer, https://clang.llvm.org/docs/LeakSanitizer.html
mdl - Markdownlint, https://github.com/markdownlint/markdownlint
msan - MemorySanitizer, https://clang.llvm.org/docs/MemorySanitizer.html
pyflakes - Pyflakes, https://github.com/PyCQA/pyflakes
pylint - Pylint, https://www.pylint.org
smatch - Smatch, https://repo.or.cz/w/smatch.git
sparse - Sparse, https://git.kernel.org/pub/scm/devel/sparse/sparse.git
sphinx - Sphinx, https://github.com/sphinx-doc/sphinx
spotbugs - spotbugs, https://spotbugs.github.io
tsan - ThreadSanitizer, https://clang.llvm.org/docs/ThreadSanitizer.html
tslint - TSLint, https://palantir.github.io/tslint
ubsan - UndefinedBehaviorSanitizer, https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html
- Compile with
-gand-fno-omit-frame-pointerto get proper debug information in your binary. - Run your program with environment variable
UBSAN_OPTIONS=print_stacktrace=1. - Set the
UBSAN_SYMBOLIZER_PATHenvironment variable to point to thellvm-symbolizerbinary (or make surellvm-symbolizeris in your$PATH). - Run the
report-converterconverter from the directory where your binary is because the output of theUndefinedBehaviorSanitizerwill contain relative paths.
# Compile your program.
clang++ -fsanitize=undefined -g -fno-omit-frame-pointer ubsanitizer.cpp
# Run your program and redirect the output to a file.
UBSAN_OPTIONS=print_stacktrace=1 \
UBSAN_SYMBOLIZER_PATH=/usr/lib/llvm-6.0/bin/llvm-symbolizer \
./a.out > ubsan.output 2>&1
# Generate plist files from the output.
report-converter -t ubsan -o ./ubsan_results ubsan.output- Compile with
-gand-fno-omit-frame-pointerto get proper debug information in your binary. - Set the
ASAN_SYMBOLIZER_PATHenvironment variable to point to thellvm-symbolizerbinary (or make surellvm-symbolizeris in your$PATH).
# Compile your program.
clang++ -fsanitize=address -g -fno-omit-frame-pointer asan.cpp
# Run your program and redirect the output to a file.
ASAN_SYMBOLIZER_PATH=/usr/lib/llvm-6.0/bin/llvm-symbolizer \
./a.out > asan.output 2>&1
# Generate plist files from the output.
report-converter -t asan -o ./asan_results asan.output- Compile with
-gand-fno-omit-frame-pointerto get proper debug information in your binary. - Set the
MSAN_SYMBOLIZER_PATHenvironment variable to point to thellvm-symbolizerbinary (or make surellvm-symbolizeris in your$PATH).
# Compile your program.
clang++ -fsanitize=memory -g -fno-omit-frame-pointer msan.cpp
# Run your program and redirect the output to a file.
MSAN_SYMBOLIZER_PATH=/usr/lib/llvm-6.0/bin/llvm-symbolizer \
./a.out > msan.output 2>&1
# Generate plist files from the output.
report-converter -t msan -o ./msan_results msan.output- Compile with
-gto get proper debug information in your binary.
# Compile your program.
clang++ -fsanitize=thread -g tsan.cpp
# Run your program and redirect the output to a file.
./a.out > tsan.output 2>&1
# Generate plist files from the output.
report-converter -t tsan -o ./tsan_results tsan.output- Compile with
-gand-fsanitize=addressto get proper debug information in your binary.
# Compile your program.
clang -fsanitize=address -g lsan.c
# Run your program and redirect the output to a file.
ASAN_OPTIONS=detect_leaks=1 ./a.out > lsan.output 2>&1
# Generate plist files from the output.
report-converter -t lsan -o ./lsan_results lsan.outputCppcheck is a static analysis tool for
C/C++ code.
The recommended way of running the Cppcheck tool is to use a JSON Compilation Database file.
The following example shows how to log the compilation commands with CodeChecker, run CppCheck and store the results found by CppCheck to the CodeChecker database.
# Collect the compilation commands with absolute path conversion.
# Absolute path conversion is required so the reports generated by Cppcheck can
# be stored.
CC_LOGGER_ABS_PATH=1 CodeChecker log -o compile_command.json -b "make"
# Create a directory for the reports.
mkdir cppcheck_reports
# Run Cppcheck by using the generated compile command database.
cppcheck --project=compile_command.json --plist-output=./cppcheck_reports
# Use 'report-converter' to create a CodeChecker report directory from the
# analyzer result of Cppcheck. It will generate a unique report identifier
report-converter -t cppcheck -o ./codechecker_cppcheck_reports ./cppcheck_reports
# Store the Cppcheck reports with CodeChecker.
CodeChecker store ./codechecker_cppcheck_reports -n cppcheckNotes:
- The missing unique report identifiers will be generated by the
report-convertertool. - Analysis related information can not be stored since it is not collected by
CppCheck:
analysis statistics,analysis duration,cppcheck commandetc.
For more information about logging checkout the log section in the user guide.
Spotbugs is a static analysis tool for Java
code.
The recommended way of running the Spotbugs tool is to generate an xml output
file with messages (-xml:withMessages).
The following example shows you how to run SpotBugs and store the results found by SpotBugs to the CodeChecker database.
# Run SpotBugs.
# Use the '-xml:withMessages' option to generate xml output.
spotbugs -xml:withMessages -output ./bugs.xml -textui /path/to/your/project
# Use 'report-converter' to create a CodeChecker report directory from the
# analyzer result of SpotBugs.
report-converter -t spotbugs -o ./codechecker_spotbugs_reports ./bugs.xml
# Store the SpotBugs reports with CodeChecker.
CodeChecker store ./codechecker_spotbugs_reports -n spotbugsFacebook Infer is a static analysis tool developed by
Facebook which supports multiple programming languages such as C/C++, Java
etc.
The recommended way of running the Facebook Infer tool is to generate an
infer-out directory which will contain a report.json file.
The following example shows you how to run Facebook Infer and store the results found by Infer to the CodeChecker database.
# Run Infer.
infer capture -- clang++ main.cpp
infer analyze
# Use 'report-converter' to create a CodeChecker report directory from the
# analyzer result of FB Infer.
report-converter -t fbinfer -o ./codechecker_fbinfer_reports ./infer-out
# Store the Infer reports with CodeChecker.
CodeChecker store ./codechecker_fbinfer_reports -n fbinferESLint is a static analysis tool for JavaScript.
The recommended way of running the ESLint tool is to generate a json output file.
The following example shows you how to run ESLint and store the results found by ESLint to the CodeChecker database.
# Run ESLint.
eslint -o ./eslint_reports.json -f json /path/to/my/project
# Use 'report-converter' to create a CodeChecker report directory from the
# analyzer result of ESLint.
report-converter -t eslint -o ./codechecker_eslint_reports ./eslint_reports.json
# Store the ESLint reports with CodeChecker.
CodeChecker store ./codechecker_eslint_reports -n eslintPylint is a static analysis tool for Python.
The recommended way of running the Pylint tool is to generate a json output
file.
The following example shows you how to run Pylint and store the results found by Pylint to the CodeChecker database.
# Run Pylint.
pylint -f json /path/to/my/project > ./pylint_reports.json
# Use 'report-converter' to create a CodeChecker report directory from the
# analyzer result of Pylint.
report-converter -t pylint -o ./codechecker_pylint_reports ./pylint_reports.json
# Store the Pylint reports with CodeChecker.
CodeChecker store ./codechecker_pylint_reports -n pylintPyflakes is a static analysis tool for
Python code.
The recommended way of running Pyflakes is to redirect the output to a file and give this file to the report converter tool.
The following example shows you how to run Pyflakes and store the results found by Pyflakes to the CodeChecker database.
# Run Pyflakes.
pyflakes /path/to/your/project > ./pyflakes_reports.out
# Use 'report-converter' to create a CodeChecker report directory from the
# analyzer result of Pyflakes.
report-converter -t pyflakes -o ./codechecker_pyflakes_reports ./pyflakes_reports.out
# Store the Pyflakes reports with CodeChecker.
CodeChecker store ./codechecker_pyflakes_reports -n pyflakesTSLint is a static analysis tool for
TypeScript.
The recommended way of running the TSLint tool is to generate a json output file.
The following example shows you how to run TSLint and store the results found by TSLint to the CodeChecker database.
# Run TSLint.
tslint --format json /path/to/my/ts/file -o ./tslint_reports.json
# Use 'report-converter' to create a CodeChecker report directory from the
# analyzer result of TSLint.
report-converter -t tslint -o ./codechecker_tslint_reports ./tslint_reports.json
# Store the TSLint reports with CodeChecker.
CodeChecker store ./codechecker_tslint_reports -n tslintGolint is a static analysis tool for Go
code.
The recommended way of running Golint is to redirect the output to a file and give this file to the report converter tool.
The following example shows you how to run Golint and store the results found by Golint to the CodeChecker database.
# Run Golint.
golint /path/to/your/project > ./golint_reports.out
# Use 'report-converter' to create a CodeChecker report directory from the
# analyzer result of Golint.
report-converter -t golint -o ./codechecker_golint_reports ./golint_reports.out
# Store the Golint reports with CodeChecker.
CodeChecker store ./codechecker_golint_reports -n golintMarkdownlint is a static analysis tool for markdown files.
The recommended way of running Markdownlint is to redirect the output to a file and give this file to the report converter tool.
The following example shows you how to run Markdownlint and store the results found by Markdownlint to the CodeChecker database.
# Run Markdownlint.
mdl /path/to/your/project > ./mdl_reports.out
# Use 'report-converter' to create a CodeChecker report directory from the
# analyzer result of Markdownlint.
report-converter -t mdl -o ./codechecker_mdl_reports ./mdl_reports.out
# Store Markdownlint reports with CodeChecker.
CodeChecker store ./codechecker_mdl_reports -n mdlCoccinelle allows programmers to easily write some complex style-preserving source-to-source transformations on C source code, like for instance to perform some refactorings.
The recommended way of running Coccinelle is to redirect the output to a file and give this file to the report converter tool.
Note: the checker name will be the file name of the .cocci file along with cocinelle prefix.
The following example shows you how to run Coccinelle on kernel sources and store the results found by Coccinelle to the CodeChecker database.
# Change Directory to your project
cd path/to/linux/kernel/repository
# Run Coccicheck
make coccicheck MODE=report V=1 > ./coccinelle_reports.out
# Use 'report-converter' to create a CodeChecker report directory from the
# analyzer result of Coccicheck
report-converter -t coccinelle -o ./codechecker_coccinelle_reports ./coccinelle_reports.out
# Store the Cocccinelle reports with CodeChecker.
CodeChecker store ./codechecker_coccinelle_reports -n coccinelleSmatch is a static analysis tool for C that is used on the kernel.
The recommended way of running Smatch is to redirect the output to a file and give this file to the report converter tool.
The following example shows you how to run Smatch on kernel sources and store the results found by Smatch to the CodeChecker database.
# Change Directory to your project
cd path/to/linux/kernel/repository
# Run Smatch
# Note: The warnings will be stored by default into smatch_warns.txt after executing the following command
path/to/smatch/smatch_scripts/test_kernel.sh
# Use 'report-converter' to create a CodeChecker report directory from the
# analyzer result of Smatch
report-converter -t smatch -o ./codechecker_smatch_reports ./smatch_warns.txt
# Store the Smatch reports with CodeChecker.
CodeChecker store ./codechecker_smatch_reports -n smatchKernel-Doc structure is extracted from the comments, and proper Sphinx C Domain function and type descriptions with anchors are generated from them. The descriptions are filtered for special kernel-doc highlights and cross-references.
The recommended way of running Kernel-Doc is to redirect the output to a file and give this file to the report converter tool.
The following example shows you how to run Kernel-Doc on kernel sources and store the results found by Kernel-Doc to the CodeChecker database.
# Change Directory to your project
cd path/to/linux/kernel/repository
# Run Kernel-Doc
# Note: The output of the following command will be both of sphinx and kernel-doc,
# but the parser will parse only kernel-doc output
make htmldocs 2>&1 | tee kernel-docs.out
# Use 'report-converter' to create a CodeChecker report directory from the
# analyzer result of Kernel-Doc
report-converter -t kernel-doc -o ./codechecker_kernel_doc_reports ./kernel-docs.out
# Store the Kernel-Doc reports with CodeChecker.
CodeChecker store ./codechecker_kernel_doc_reports -n kernel-docSphinx Sphinx is a documentation generator or a tool that translates a set of plain text source files into various output formats, automatically producing cross-references, indices, etc.
The recommended way of running Sphinx is to redirect the output to a file and give this file to the report converter tool.
The following example shows you how to run Sphinx on kernel sources and store the results found by Sphinx to the CodeChecker database.
# Change Directory to your project
cd path/to/linux/kernel/repository
# Run Sphinx
# Note: The output of the following command will be both of sphinx and kernel-doc,
# but the parser will parse only sphinx output
make htmldocs 2>&1 | tee sphinx.out
# Use 'report-converter' to create a CodeChecker report directory from the
# analyzer result of Sphinx
report-converter -t sphinx -o ./codechecker_sphinx_reports ./sphinx.out
# Store the Sphinx reports with CodeChecker.
CodeChecker store ./codechecker_sphinx_reports -n sphinxSparse is a semantic checker for C programs; it can be used to find a number of potential problems with kernel code.
The recommended way of running Sparse is to redirect the output to a file and give this file to the report converter tool.
The following example shows you how to run Sparse on kernel sources and store the results found by Sparse to the CodeChecker database.
# Change Directory to your project
cd path/to/linux/kernel/repository
# Run Sparse
make C=1 2>&1 | tee sparse.out
# Use 'report-converter' to create a CodeChecker report directory from the
# analyzer result of Sparse
report-converter -t sparse -o ./codechecker_sparse_reports ./sparse.out
# Store the Sparse reports with CodeChecker.
CodeChecker store ./codechecker_sparse_reports -n sparsecpplint is a lint-like tool which checks C++ code against Google C++ Style Guide.
The recommended way of running cpplint is to redirect the output to a file and give this file to the report converter tool.
The following example shows you how to run cpplint and store the results found by cpplint to the CodeChecker database.
# Change Directory to your project
cd path/to/your/project
# Run cpplint
cpplint sample.cpp > ./sample.out 2>&1
# Use 'report-converter' to create a CodeChecker report directory from the
# analyzer result of cpplint
report-converter -t cpplint -o ./codechecker_cpplint_reports ./sample.out
# Store the cpplint reports with CodeChecker.
CodeChecker store ./codechecker_cpplint_reports -n cpplintplist-to-html is a python tool which parses and creates HTML files from one
or more .plist result files.
$ plist-to-html --help (click to expand)
usage: plist-to-html [-h] -o OUTPUT_DIR [-l LAYOUT_DIR]
file/folder [file/folder ...]
Parse and create HTML files from one or more '.plist' result files.
positional arguments:
file/folder The plist files and/or folders containing analysis
results which should be parsed.
optional arguments:
-h, --help show this help message and exit
-o OUTPUT_DIR, --output OUTPUT_DIR
Generate HTML output files in the given folder.
(default: None)
-l LAYOUT_DIR, --layout LAYOUT_DIR
Directory which contains dependency HTML, CSS and
JavaScript files. (default: plist_to_html/../static)
A report hash identifies a specific bug in the analyzed code. For example if
a function contains some bug and this function is called from several parts of
the program then essentially these are the same bug. These bugs get the same
report hash which indicates a connection between them. CodeChecker web
interface also helps to group these findings. However, in some special cases
the hash should be built from specific information which makes bug
identification sensitive on some different things (for example indentation of
the code). We recommend using CONTEXT_FREE which works in most cases.
Multiple hash types are available:
You can use this library to generate report hash for these types by using the
get_report_hash function.
get_report_hash function can be used to generate report hash with bug path
if the hash type parameter is PATH_SENSITIVE.
High level overview of the hash content:
file_namefrom the main diag section.checker name.checker message.line contentfrom the source file if can be read up.column numbersfrom the main diag section.range column numbersonly from the control diag sections if column number in the range is not the same as the previous control diag section number in the bug path. If there are no control sections event section column numbers are used.
Note: as the main diagnostic section the last element from the bug path is used.
get_report_hash function can be used to generate report hash without bug path
if the hash type parameter is CONTEXT_FREE.
High level overview of the hash content:
file_namefrom the main diag section.checker message.line contentfrom the source file if can be read up. All the whitespaces from the source content are removed.column numbersfrom the main diag sections location.
get_report_hash function can be used to generate report hash with bug event
messages if the hash type parameter is DIAGNOSTIC_MESSAGE.
High level overview of the hash content:
- Same as
CONTEXT_FREE(file name, checker message etc.) bug step messagesfrom all events.
Note: this is an experimental hash and it is not recommended to use it on your project because this hash can change very easily for example on variable / function renames.
get_report_path_hash can be used to get path hash for the given bug path
which can be used to filter deduplications of multiple reports.
The project is licensed under Apache License v2.0 with LLVM Exceptions. See LICENSE.TXT for details.