-
Notifications
You must be signed in to change notification settings - Fork 10
Open
Description
The current CI runs are failing due to a combination of:
-
Network timeouts when tests attempt live DNS-over-HTTPS queries to public resolvers (e.g., Quad9, Cloudflare), which are unreliable in CI environments.
-
Nil pointer panics in test code where error handling does not return after encountering network errors, leading to dereferencing nil responses.
Example errors:
doh: failed HTTP request: ... context deadline exceeded dial tcp ...: i/o timeout panic: runtime error: invalid memory address or nil pointer dereference
Files involved:
internal/cli/command_test.go
pkg/dj/dj_test.go
pkg/doh/doh_test.go
Possible solutions:
- Replace Live Queries with Local Test Servers
- Refactor tests to use local HTTP(S) servers (like what’s done in
TestCommand_Query_InsecureSkipVerify
) instead of real DoH endpoints.
- Refactor tests to use local HTTP(S) servers (like what’s done in
- Fix Error Handling in Tests
- Ensure all test error checks are followed by return to avoid
panic
s due tonil
dereference.
- Ensure all test error checks are followed by return to avoid
- Optionally Skip Network-Dependent Tests in CI
- Use
testing.Short()
andt.Skip()
to skip these tests in CI, or guard them behind an environment variable.
- Use
- Increase Robustness with Retries and Timeouts
- If live queries are necessary, increase timeouts and retries, though this is not ideal for CI stability.
Other thoughts:
- Should all network-dependent tests be replaced with local mocks, or only skipped in CI?
- Are there any other best practices we can adopt for testing CLI/network tools in CI environments?
Copilot
Metadata
Metadata
Assignees
Labels
No labels