per fuzzer coverage information in project overview

[phi-go]

Adds per fuzzer summary.json (based on https://storage.googleapis.com/oss-fuzz-coverage/{0}/reports-by-target/{1}/{2}/linux/summary.json) to the project timestamp data and generates per fuzzer coverage plots for the project overview, example below.

This PR is mainly to understand if an approach like this would be acceptable, though to implement some metrics in #2054 it is probably needed to have a separate stage that takes a look at project data over time and not only in report generation.

Still this PR addresses an issue but it might take too much screen space (especially for large numbers of fuzzers), it might also require too much storage space, data ingress and runtime for data generation, and could have noticeable effects on webpage load time as well.

There is a overview as well now: #2102 (comment).

[phi-go]

I will fix the CI issues tomorrow.

Is there a command to run these CI tests locally? I ran the formatting shell script but it returned success.

[DavidKorczynski]

This looks good! Could you use single quotes instead of double quotes where possible?

Once the CI is green then we can land this. We don't have helper scripts for the ci yet. In general, if you have typing and run yapf -i on the modules, then you should be good. We'd be happy to land some scripts that does this locally though, it's about automating https://github.com/ossf/fuzz-introspector/blob/main/.github/workflows/main.yml and https://github.com/ossf/fuzz-introspector/blob/main/.github/workflows/mypy.yml

Just to confirm, are you able to see the output of the failed CI runs?

[phi-go]

This looks good! Could you use single quotes instead of double quotes where possible?

Good to hear. Of course, will update.

Once the CI is green then we can land this. We don't have helper scripts for the ci yet.

Ok, sounds good. I'll see if I get around to those scripts.

Just to confirm, are you able to see the output of the failed CI runs?

Yes, I can see the details here and here.

[phi-go]

I fixed the formatting. However, two coverage reports that are needed for the webapp-api-test are not available, so this check does not succeed, removing those projects from the must_include file "fixes" this, how should we proceed?

https://storage.googleapis.com/oss-fuzz-coverage/abseil-cpp/reports/20250216/linux/summary.json
issue: https://issues.oss-fuzz.com/issues/390244458

https://storage.googleapis.com/oss-fuzz-coverage/leveldb/reports/20250216/linux/summary.json
which just awaits a PR google/leveldb#1251

[phi-go]

I reworked the project report a bit now there is a per fuzzer summary table, see below (with some example errors). This also should be more efficient when displaying the report and the summary is now available in all-project-current.json.

[DavidKorczynski]

We shouldn't fail if there is no coverage in this manner. Something like the following solves it:

$ git diff ./
diff --git a/tools/web-fuzzing-introspection/app/static/assets/db/web_db_creator_from_summary.py b/tools/web-fuzzing-introspection/app/static/assets/db/web_db_creator_from_summary.py
index 42f3470..4b98b52 100644
--- a/tools/web-fuzzing-introspection/app/static/assets/db/web_db_creator_from_summary.py
+++ b/tools/web-fuzzing-introspection/app/static/assets/db/web_db_creator_from_summary.py
@@ -319,7 +319,8 @@ def prepare_code_coverage_dict(
         project_language: str) -> Optional[Dict[str, Any]]:
     """Gets coverage URL and line coverage total of a project"""
     line_total_summary = extract_code_coverage_data(code_coverage_summary)
-
+    if not line_total_summary:
+        return None
     coverage_url = oss_fuzz.get_coverage_report_url(project_name,
                                                     date_str.replace('-', ''),
                                                     project_language)
@@ -457,7 +458,9 @@ def extract_local_project_data(project_name, oss_fuzz_path,
 
     code_coverage_data_dict = prepare_code_coverage_dict(
         code_coverage_summary, project_name, '', project_language)
-
+    if not code_coverage_data_dict:
+        return
+    
     if cov_fuzz_stats is not None:
         all_fuzzers = cov_fuzz_stats.split("\n")
         if all_fuzzers[-1] == '':
@@ -719,7 +722,9 @@ def extract_project_data(project_name, date_str, should_include_details,
 
     code_coverage_data_dict = prepare_code_coverage_dict(
         code_coverage_summary, project_name, date_str, project_language)
-
+    if not code_coverage_data_dict:
+        return
+    
     per_fuzzer_cov = {}
     if cov_fuzz_stats is not None:
         all_fuzzers = cov_fuzz_stats.split("\n")

[phi-go]

Oof, I see, my bad. Thank you for taking a look!

Only the first return should be necessary, neither the local nor the normal run needed this check before and no logic was changed in that regard. I updated as such.