Checked for duplicates

Yes - I've already checked

Alternatives considered

Yes - and alternatives don't suffice

Related problems

Currently, monitoring scripts generate only PNG visualizations. Users cannot programmatically access or analyze the underlying data without parsing images or re-running CMR queries. This limits custom analysis, historical trends, external tool integration, and data archival.

Feature Description

Summary

Update monitoring chart generation scripts to export CSV files containing the underlying data, in addition to existing PNG images.

Affected Scripts

• monitoring/opera_daily_products_query.py - Daily products chart
• monitoring/latency_graph.py - Latency chart

Proposed Implementation

1. Update Python Scripts

Modify both scripts to generate CSV files alongside PNG files in monitoring/ directory:

• opera_daily_products_query.csv
• opera_latency_query.csv

CSV Format Specifications:

Daily Products CSV:

• Date (YYYY-MM-DD)
• Collection name
• Product count (total)
• Regional product count (region-specific, e.g., North America + Central America where applicable)
• Statistical values (mean, 2-sigma bounds)

Latency CSV:

• Product type
• Latency metric type (output-input revision, output-input temporal, input revision-temporal)
• Individual latency values (days)
• Statistical measure (mean)

2. Update GitHub Workflows

Modify workflow file patterns to include CSV files:

• .github/workflows/opera_daily_products_query.yml: 'monitoring/*.{png,csv}'
• .github/workflows/opera_daily_latency.yml: 'monitoring/*.{png,csv}'

3. Update README.md

Add CSV download links near charts:

![OPERA Daily Products](monitoring/opera_daily_products_query.png)
[Download CSV Data](monitoring/opera_daily_products_query.csv)

![OPERA Products Latency](monitoring/opera_latency_query.png)
[Download CSV Data](monitoring/opera_latency_query.csv)

Storage Location

CSV files stored in monitoring/ folder, committed to repository. Future migration to S3 planned.

Acceptance Criteria

• opera_daily_products_query.py generates CSV with complete chart data
• latency_graph.py generates CSV with complete chart data
• CSV files saved to monitoring/ with correct naming
• CSV files use standard format (comma-delimited, UTF-8)
• GitHub workflows automatically commit and push CSV files
• README.md includes CSV download links near respective charts
• CSV files are human-readable and importable to Excel/Google Sheets
• Existing PNG generation and workflows continue working unchanged