Caution

The tis-report tool is not available in this version.

tis-report Manual

This manual explains how to use tis-report to generate analyses reports. These reports are user-friendly, interactive, self-contained web-pages which showcase most of the useful information extracted from analyses.

Since a report is a self-contained .html file, it’s particularly well suited as a CI artifact.

Quick Start

Here’s a short but complete usage example :

$ tis-analyzer {your usual options} -tis-report
$ tis-report _results/
$ xdg-open tis_report.html

Step by step

Exporting results

tis-report feeds itself off of files generated by the analyzer.

To export the files needed by tis-report, run tis-analyzer with the -tis-report option on (starts with one dash) :

$ tis-analyzer {your usual options} -tis-report

Running the analyzer with this option will create a _results folder in the current working directory and populate it with the files needed.

Tip

_results/ is the default output directory of the -tis-report option, you can specify a different one with -tis-report-directory ...

tis-report works just as well when aggregating results from several analyses. If you want to create an aggregated report of multiple analyses, just run them all with the -tis-report option on, and make sure all the generated files will be created in the same folder.

Tip

The tool tries to generate the base name of the result files:

  • in case a configuration file is used, the name field of the selected configuration is used if any, otherwise, the configuration filename is used together with the selected number.
  • when no configuration file is used, a unique filename, is generated.

You can specity the base name of the result files with -tis-report-basename.

Warning: if the same configuration is used for several analysis, the -tis-report-basename option MUST be used to get the results in different files.

Prior to 1.43

The -tis-report option was added in version 1.43 of the analyzer, prior to this version, you have to create a folder yourself, chose a unique name for each analysis, and use both :

  • -info-json-results {folder}/{name}_results.json
  • -info-csv-all {folder}/{name}

Replacing {name} and {folder} with your choices of name and folder

As an example, let’s say you are only analyzing a single file called hw.c in interpreter mode, you want to store the generated data in a folder called _results and you want to use the prefix XXX for this analysis. Then you would have to do the following :

$ mkdir _results
$ tis-analyzer hw.c -val -val-profile interpreter -info-json-results _results/XXX_results.json -info-csv-all _results/XXX

Generating the report

Once the data files needed have been generated, run tis-report by pointing it to the folder they are contained in :

$ tis-report _results/

Reading the report

The generated report is named tis_report.html and can be viewed in a web-browser.

As of 1.43, a report generated by tis-report looks like this :

An Overwiew of the output of tis-report

Analyses Summary

A close-up view of the "Analyses Summary" section

This section presents general figures.

It has no interactive parts.

Consolidated Undefined Behaviors

A close-up view of the "Consolidated Undefined Behaviors" section

This table lists each unique undefined behavior found. This means redundant UBs found by different analyses are deduplicated before being displayed in this table.

Clicking on a column header will cycle through the sorting options for this column :

  • ▲/▼ : Unsorted
  • ▼ : Descending
  • ▲ : Ascending

Clicking on a row in the table reveals more details :

An expanded row in the "Consolidated Undefined Behaviors" table

Individual Analyses

Close-up of the "Individual Analyses" table

Each row in this table presents code coverage, time, memory, and UBs information for a specific analysis.

This table can be sorted by clicking on a column header.

Clicking on the row of a given analysis reveals the list of alarms raised by this specific analysis, each presented with the same look as in the Consolidated Undefined Behaviors section.

Code Coverage

Close-up of the "Code Coverage" section

This section presents code coverage metrics at different granularity levels :

  • per functions
  • per lines
  • per statements

The toggle on the upper right-hand corner allows switching between displaying Line and Statement coverage

The first table is a table to totals. The counts update live according to what is currently being display in the second table

The second table shows coverage information for each function encounter during the analyses. It can be sorted by clicking on the column header like the previous tables. It can also be filtered by filling in the text boxes underneath the headers. The rows displayed in this table update live as you type in the text boxes.

Clicking on the row of a given function in this table will reveal its source code with color-coded lines to indicate coverage :

Close-up of the "Code Coverage" section

When in “Statements” mode, partially covered lines are highlighted in yellow :

Close-up of the "Code Coverage" section