Skip to content

CI Fails Due to External Network Timeouts and Nil Pointer Panics in Tests #52

@picatz

Description

@picatz

The current CI runs are failing due to a combination of:

  1. Network timeouts when tests attempt live DNS-over-HTTPS queries to public resolvers (e.g., Quad9, Cloudflare), which are unreliable in CI environments.

  2. Nil pointer panics in test code where error handling does not return after encountering network errors, leading to dereferencing nil responses.

    Example errors:

    doh: failed HTTP request: ... context deadline exceeded
    dial tcp ...: i/o timeout
    panic: runtime error: invalid memory address or nil pointer dereference

    Files involved:

    • internal/cli/command_test.go
    • pkg/dj/dj_test.go
    • pkg/doh/doh_test.go

Possible solutions:

  1. Replace Live Queries with Local Test Servers
    • Refactor tests to use local HTTP(S) servers (like what’s done in TestCommand_Query_InsecureSkipVerify) instead of real DoH endpoints.
  2. Fix Error Handling in Tests
    • Ensure all test error checks are followed by return to avoid panics due to nil dereference.
  3. Optionally Skip Network-Dependent Tests in CI
    • Use testing.Short() and t.Skip() to skip these tests in CI, or guard them behind an environment variable.
  4. Increase Robustness with Retries and Timeouts
    • If live queries are necessary, increase timeouts and retries, though this is not ideal for CI stability.

Other thoughts:

  • Should all network-dependent tests be replaced with local mocks, or only skipped in CI?
  • Are there any other best practices we can adopt for testing CLI/network tools in CI environments?

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions