Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: do not abort when unrelated connections are not yet loaded #4233

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

pdecat
Copy link
Contributor

@pdecat pdecat commented Apr 8, 2024

This PR resolves issues when connection configurations are created dynamically concurrently to executing queries.

Symptoms are that an unrelated connection that hasn't completed loading breaks other connections:

2024-04-05 12:05:19.955 UTC [INFO]  hub: goFdwBeginForeignScan, connection 'aws_000000000000', table 'aws_ec2_transit_gateway', explain: false
2024-04-05 12:05:19.957 UTC [WARN]  hub: pluginHub.GetIterator FAILED: failed to attach to plugin process for '[email protected]': no connection config loaded for 'aws_222222222222'
2024-04-05 12:05:19.957 UTC [299] ERROR:  failed to attach to plugin process for '[email protected]': no connection config loaded for 'aws_222222222222'
2024-04-05 12:05:19.957 UTC [299] STATEMENT:  SELECT 'APIGATEWAY' AS subtype, row_to_json(r) AS data FROM (SELECT
[...SNIP...]
2024-04-05 12:05:19.958 UTC [INFO]  hub: goFdwAbortCallback
2024-04-05 12:05:19.958 UTC [INFO]  hub: Hub Abort
2024-04-05 12:05:20.023 UTC [INFO]  hub: goFdwIterateForeignScan returned empty row - this scan complete (0xc00226f080)
2024-04-05 12:05:20.025 UTC [INFO]  hub: goFdwIterateForeignScan calling pluginHub.StartScan, table 'aws_account' (0xc00226f680)
2024-04-05 12:05:20.025 UTC [INFO]  hub: StartScan for table: aws_account, cache enabled: false, iterator 0xc00226f680, 0 quals (1712318705836)
2024-04-05 12:05:20.026 UTC [INFO]  hub: goFdwIterateForeignScan returned empty row - this scan complete (0xc00226f680)
2024-04-05 12:05:20.028 UTC [INFO]  hub: goFdwReScanForeignScan, connection '', table 'aws_ec2_transit_gateway'
2024-04-05 12:05:20.028 UTC [INFO]  hub: goFdwBeginForeignScan, connection 'aws_111111111111', table 'aws_ec2_transit_gateway', explain: false
2024-04-05 12:05:20.030 UTC [WARN]  hub: pluginHub.GetIterator FAILED: failed to attach to plugin process for '[email protected]': no connection config loaded for 'aws_222222222222'
2024-04-05 12:05:20.030 UTC [INFO]  hub: .
******************************************************

                steampipe postgres fdw shutdown

******************************************************

This problem seems to happen more frequently when connections are slower to load, most notably when limiters are configured.

See https://turbot-community.slack.com/archives/C01UECB59A7/p1712324097931309

For this change to be effective, the FDW has to be rebuilt with:

go mod edit -replace="github.com/turbot/steampipe=github.com/pdecat/steampipe @0a5b8a0928292f08ad842a63211188306e9998de"

pdecat added a commit to pdecat/steampipe-4233-repro that referenced this pull request Apr 8, 2024
@pdecat
Copy link
Contributor Author

pdecat commented Apr 8, 2024

Submitted a reproduction case: https://github.com/pdecat/steampipe-4233-repro

@pdecat pdecat force-pushed the fix/do_not_abort_when_unrelated_connection_not_loaded branch from 0a5b8a0 to 68c3fe7 Compare April 10, 2024 12:27
@pdecat pdecat force-pushed the fix/do_not_abort_when_unrelated_connection_not_loaded branch 2 times, most recently from 38b3446 to 3bcdc14 Compare April 26, 2024 14:54
@pdecat pdecat force-pushed the fix/do_not_abort_when_unrelated_connection_not_loaded branch from 3bcdc14 to 4b663e0 Compare May 3, 2024 13:31
@pdecat pdecat force-pushed the fix/do_not_abort_when_unrelated_connection_not_loaded branch from 4b663e0 to 359c16c Compare May 13, 2024 10:22
@pdecat pdecat force-pushed the fix/do_not_abort_when_unrelated_connection_not_loaded branch from 0e72e8c to fc45610 Compare May 17, 2024 13:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant