feat(otel): OpenTelemetry Integration for py-shiny

[schloerke]

Summary

Adds comprehensive OpenTelemetry (OTel) instrumentation to py-shiny, enabling distributed tracing and observability for Shiny applications. This implementation follows the 10-phase plan outlined in Epic #2131.

What's Included

This PR integrates all completed phases of the OTel implementation:

✅ Phase 1: Foundation (Core OTel Infrastructure) - #2142

• Core OTel infrastructure with lazy tracer/logger initialization
• Collection level management system (5 granular levels: NONE, SESSION, REACTIVE_UPDATE, REACTIVITY, ALL)
• Span wrapper utilities for async context propagation
• Environment variable configuration via SHINY_OTEL_COLLECT
• Comprehensive test suite (35 unit tests, >90% coverage)

🚧 Remaining Phases (Not Yet Included)

• Phase 2: Session Lifecycle Instrumentation
• Phase 3: Reactive Flush Instrumentation
• Phase 4: Reactive Execution Instrumentation
• Phase 5: Value Updates and Logging
• Phase 6: Error Handling and Sanitization
• Phase 7: User-Facing API (otel_collect context manager)
• Phase 8: Testing Infrastructure
• Phase 9: Documentation
• Phase 10: Follow-up Evaluation

Key Features

Collection Levels

from shiny.otel import OtelCollectLevel

# NONE (0): No telemetry collected
# SESSION (1): Session lifecycle spans only
# REACTIVE_UPDATE (2): SESSION + reactive flush cycles
# REACTIVITY (3): REACTIVE_UPDATE + individual reactive computations
# ALL (4): All available telemetry (default)

Foundation API

from shiny.otel import (
    OtelCollectLevel,
    get_otel_tracer,
    get_otel_logger,
    is_otel_tracing_enabled,
    get_otel_collect_level,
    should_otel_collect,
)

Configuration

# Control collection level via environment variable
export SHINY_OTEL_COLLECT=session  # or: none, reactive_update, reactivity, all
python app.py

Implementation Details

• Lazy initialization: Tracer and logger are initialized on first use, allowing apps to configure OTel before importing shiny
• No-op behavior: When OTel SDK is not configured, all functions return no-op objects with zero overhead
• Type safety: Full type annotations for pyright compliance
• Context propagation: Proper async context handling for span parent-child relationships
• Validation: should_otel_collect(NONE) raises ValueError to catch programming errors early

Testing

• ✅ All 35 OTel foundation tests pass
• ✅ >90% test coverage for shiny/otel/ code
• ✅ Full test suite passes (373 tests total)
• ✅ Type checking passes (0 errors with pyright)
• ✅ Code formatting and linting clean (black, isort, flake8)

Files Added (Phase 1)

• shiny/otel/__init__.py - Public API exports with comprehensive documentation
• shiny/otel/_core.py - Core tracer/logger initialization
• shiny/otel/_collect.py - Collection level management
• shiny/otel/_span_wrappers.py - Span context managers
• tests/pytest/test_otel_foundation.py - Unit tests
• pyproject.toml - Added opentelemetry-api>=1.20.0 dependency

Breaking Changes

None. This is a purely additive feature.

Migration Guide

No migration required. OTel support is opt-in:

1. Install the OpenTelemetry SDK:

   pip install opentelemetry-sdk

2. Configure the SDK before importing shiny:

   from opentelemetry import trace
   from opentelemetry.sdk.trace import TracerProvider
   from opentelemetry.sdk.trace.export import ConsoleSpanExporter, SimpleSpanProcessor
   
   provider = TracerProvider()
   provider.add_span_processor(SimpleSpanProcessor(ConsoleSpanExporter()))
   trace.set_tracer_provider(provider)
   
   # Now import and use shiny
   from shiny import App, ui

3. Control collection level:

   export SHINY_OTEL_COLLECT=all
   python app.py

Related Issues

• Epic: epic: OpenTelemetry Integration for py-shiny #2131
• Phase 1: otel: Foundation (Core OTel Infrastructure) #2132 (✅ Completed)
• Phase 2: otel: Session Lifecycle Instrumentation #2133
• Phase 3: otel: Reactive Flush Instrumentation #2134
• Phase 4: otel: Reactive Execution Instrumentation #2135
• Phase 5: otel: Value Updates and Logging #2136
• Phase 6: otel: Error Handling and Sanitization #2137
• Phase 7: otel: User-Facing API #2138
• Phase 8: otel: Testing Infrastructure #2139
• Phase 9: otel: Documentation #2140
• Phase 10: otel: Follow-up Evaluation #2141

Next Steps

This PR will remain in DRAFT status until all 10 phases are completed and merged into the otel branch. Once all phases are complete, this PR will be marked as ready for review and merged into main.

---

🤖 Generated with Claude Code

Co-Authored-By: Claude noreply@anthropic.com