Add narrow vs wide parquet performance benchmark tutorial

[rdhyee]

Summary

• Adds interactive browser-based benchmarks comparing narrow (691MB, 11.6M rows) vs wide (275MB, 2.5M rows) parquet schemas
• Three benchmarks: entity counts, site aggregation, material distribution
• Multiple runs with median timing for reliability
• Environment info display and data validity checks
• Technical notes on caching, cold starts, and memory limits

Context

This addresses Eric's question about whether the 2-3x speedup seen locally also holds "over the wire" with HTTP range requests via DuckDB-WASM.

Test plan

• Run quarto preview on the new tutorial
• Verify "Run All Benchmarks" button executes successfully
• Check all three benchmarks show timing comparisons
• Verify results summary table populates correctly

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR adds an interactive browser-based performance benchmark tutorial comparing narrow vs wide parquet schema formats for the iSamples property graph data. The tutorial uses DuckDB-WASM to execute queries via HTTP range requests and demonstrates the performance differences between the two schema approaches.

Key changes:

• Interactive benchmarking suite with three different query types (entity counts, site aggregation, material distribution)
• Real-time performance measurement with cold/warm run tracking and median calculation
• Data validity checks to confirm schema equivalence
• Environment information display and comprehensive technical documentation

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The median calculation is incorrect for arrays with an even number of elements. For an array with 3 elements (odd), Math.floor(3/2) = 1 correctly returns the middle element. However, if the number of runs changes to an even number, this implementation would not calculate the true median (average of the two middle values).

Consider using a more robust median calculation:

const median = arr => {
    const sorted = [...arr].sort((a, b) => a - b);
    const mid = Math.floor(sorted.length / 2);
    return sorted.length % 2 === 0 
        ? (sorted[mid - 1] + sorted[mid]) / 2 
        : sorted[mid];
};

Suggested change

	return sorted[Math.floor(sorted.length / 2)];
	const mid = Math.floor(sorted.length / 2);
	return sorted.length % 2 === 0
	? (sorted[mid - 1] + sorted[mid]) / 2
	: sorted[mid];

[Copilot]

The median calculation is duplicated from Benchmark 1 (lines 236-239). Consider extracting this function to avoid code duplication and improve maintainability. The same issue with even-length arrays also applies here.

Suggested change

	const median = arr => {
	const sorted = [...arr].sort((a, b) => a - b);
	return sorted[Math.floor(sorted.length / 2)];
	};
	// Median calculation extracted for reuse and correctness
	function median(arr) {
	// (function defined above)
	return (sorted[mid - 1] + sorted[mid]) / 2;
	} else {
	return sorted[mid];
	}
	}

[Copilot]

The median calculation is duplicated from Benchmarks 1 and 2. Consider extracting this function to avoid code duplication and improve maintainability. The same issue with even-length arrays also applies here.

Suggested change

	const median = arr => {
	const sorted = [...arr].sort((a, b) => a - b);
	return sorted[Math.floor(sorted.length / 2)];
	};
	// Median calculation extracted and fixed for even-length arrays
	function median(arr) {
	const sorted = [...arr].sort((a, b) => a - b);
	const mid = Math.floor(sorted.length / 2);
	if (sorted.length % 2 === 0) {
	return (sorted[mid - 1] + sorted[mid]) / 2;
	} else {
	return sorted[mid];
	}
	}

[Copilot]

[nitpick] The browser detection logic is overly simplistic and may incorrectly identify browsers. For example, Chrome's user agent string contains "Safari", so the order matters. Consider checking for more specific patterns or using a library for accurate browser detection.

The current logic will work for most cases since Chrome is checked first, but Edge (Chromium-based) would be detected as "Chrome" rather than "Edge", which may not be the desired behavior.

Suggested change

	const browser = ua.includes('Chrome') ? 'Chrome' : ua.includes('Firefox') ? 'Firefox' : ua.includes('Safari') ? 'Safari' : 'Unknown';
	// Improved browser detection logic
	let browser = 'Unknown';
	if (ua.includes('Edg/')) {
	browser = 'Edge';
	} else if (ua.includes('Chrome/') && !ua.includes('Edg/')) {
	browser = 'Chrome';
	} else if (ua.includes('Firefox/')) {
	browser = 'Firefox';
	} else if (ua.includes('Safari/') && !ua.includes('Chrome/') && !ua.includes('Edg/')) {
	browser = 'Safari';
	}

[Copilot]

The speedup calculation narrowMedian / wideMedian could produce Infinity or NaN if wideMedian is 0 or if either value is undefined. While unlikely in practice, consider adding a guard condition or handling this edge case gracefully in the display logic to prevent confusing output.

[Copilot]

The speedup calculation narrowMedian / wideMedian could produce Infinity or NaN if wideMedian is 0. The same issue exists in Benchmark 1 (line 252).

Suggested change

	speedup: narrowMedian / wideMedian
	speedup: wideMedian === 0 ? null : narrowMedian / wideMedian

[Copilot]

The speedup calculation narrowMedian / wideMedian could produce Infinity or NaN if wideMedian is 0. The same issue exists in Benchmarks 1 and 2.

Suggested change

	speedup: narrowMedian / wideMedian
	speedup: wideMedian === 0 ? null : narrowMedian / wideMedian

[rdhyee]

Combined Review: Codex + Copilot Analysis

Summary

Copilot gave a positive high-level overview. Codex provided detailed code analysis identifying several methodology and implementation issues worth addressing.

Critical Issues Identified by Codex

1. Eager Data Loading (lines 105-123)

   • DuckDB instances with read_parquet views are created on page load
   • Immediately fetches metadata from 691MB/275MB files before user clicks "Run All Benchmarks"
   • Fix: Gate DB initialization behind the button or a dedicated "Load data" action

2. Concurrent Benchmark Execution

   • All benchmarks run simultaneously when runBenchmarks increments
   • Validity COUNT(*) queries also fire at the same time
   • Timers measure overlapping work, not isolated runs
   • Fix: Chain benchmarks sequentially (benchmark2 waits on benchmark1, etc.)

3. Cold/Warm Methodology Flaw

   • Medians include the cold run (all 3 values)
   • Narrow always runs first - order effects and HTTP cache priming skew results
   • Fix: Run explicit cold pass per schema, discard it, compute medians over warm runs; randomize/alternate query order

4. Cache Warming Effects

   • Earlier benchmarks warm caches for later ones
   • Later queries may report unrealistically fast "over-the-wire" times
   • Fix: Add cache-busting or document that results after first benchmark are warmed

5. Minimal Error Handling

   • Network failures or DuckDB instantiation errors stall the page silently
   • Fix: Wrap each benchmark in try/catch with user-visible status

6. Unpinned Dependency

   • @observablehq/duckdb@latest could break with future releases
   • Fix: Pin to a known version

Nice-to-Have Improvements

• Surface navigator.connection + range-request byte counts to back "2-3x faster" claim with transfer evidence
• Show bytes-read stats from DuckDB if available

Decision Needed

These are valid methodological concerns. Options:

1. Address before merge - Fix the critical issues above
2. Merge as-is with follow-up - Document limitations, create follow-up issue for improvements
3. Partial fix - Address eager loading and error handling, defer sequencing/methodology changes

What's the preferred approach?

[rdhyee]

Fixes Applied (commit 9f7ebc6)

Addressed all critical issues from the Codex review:

1. Lazy Data Loading ✅

• DB initialization now gated behind button click (not on page load)
• Uses mutable dbState to cache instances after first initialization
• Shows loading indicator during initialization

2. Sequential Benchmark Execution ✅

• Each benchmark waits for the previous one to complete:
  • validityCheck runs first
  • benchmark1 waits for validityCheck
  • benchmark2 waits for benchmark1
  • benchmark3 waits for benchmark2

3. Improved Cold/Warm Methodology ✅

• warmMedian function now excludes the cold (first) run
• Cold run still reported separately for visibility
• Updated methodology docs to explain the approach

4. Error Handling ✅

• All benchmark cells wrapped in try/catch
• Errors display in user-visible #error_display div
• Error propagation stops subsequent benchmarks from running

5. Pinned DuckDB Version ✅

• Changed from @latest to @0.7.1 for reproducibility

Updated Methodology Documentation

- **Cold run**: First query (includes metadata fetch, JIT compilation) - reported separately
- **Warm runs**: Runs 2-3 (metadata cached, JIT warmed up)
- **Warm median**: Median of warm runs only (excludes cold run for fair comparison)
- **Sequential execution**: Benchmarks run one after another, not concurrently

Ready for re-review!

[rdhyee]

Third Round: Error Handling Fixes

Codex reviewed the second round of fixes and found remaining issues with error handling in render blocks. These have now been addressed:

Fixed (commit 43896b1):

1. Error guards in render blocks - validityCheck, benchmark1, benchmark2, benchmark3 now gracefully display error messages instead of crashing when they return {error} objects

2. Summary aggregation guards - allResults now:

   • Filters out failed benchmarks before computing speedup
   • Shows "N/A" for failed benchmark rows
   • Handles null avgSpeedup in display

3. Documentation drift - Updated Pitfalls table: "3 iterations and report the median" → "3 iterations and report the warm-run median (exclude cold run)"

4. warmMedian calculation - Fixed to average two elements instead of floor selection (which overstated times for 2 warm runs)

All Codex Review Issues Resolved ✅

The implementation now handles error cases gracefully and the documentation accurately reflects the methodology.