Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
engine: nested exec simplifications and service fixes (dagger#7213)
* engine: consolidate IDs and re-use servers for nested execs This is an internal only refactor, though it fixes a few bugs while also simplifying quite a bit and setting us up for more simplifications soon. The biggest change is that nested execs connect back to the same server as the main client caller rather than being completely independent. * This is required for the fix to services used in modules (separate PR) to fully work * It also should fix the lack of docker auth in many of our integ tests, specifically those that use nested execs, which leads to dockerhub rate limiting Along the way it also does some consolidation of IDs, removing ModuleCallerDigest and just exclusively using ClientID. This requires that we tell module functions and other nested execs which ID to use, but that itself is setup for even more simplifications in follow-ups (we can remove the need for the current DaggerServer construct entirely, among other things). Signed-off-by: Erik Sipsma <[email protected]> * namespace services by server, not by client Previously it was possible to start a dependent service in one module API call, and then use it again in a later call, only to have it fail because it cannot resolve the service address, even though it's still running. This happened because each invocation has its own client ID, and client IDs were used to build service addresses. This change brings service addresses into alignment with the recent change to uniq them by service ID instead of client ID. The overall effect is that services are deduped within a Dagger invocation, even across module calls. So with this change, the service will just stay running and be re-used by a later call, thanks to the grace period. Signed-off-by: Alex Suraci <[email protected]> * fix routing of host services to correct client Signed-off-by: Erik Sipsma <[email protected]> * deregister secret tokens once client disconnects We previously never explicitly removed client ID -> secret token mappings because it theoretically opened more possibilities for malicious attempts to register a client ID with a different token. However, we need to deregister these now since Client IDs are a content hash of the function call/nested exec definition, which means the same client ID can connect and disconnect multiple times per server. The security implications of this also end up being extremely minimal. Registering a client ID with a different secret token was and still is possible *before* a client fully connects. It is possible to after a client disconnects now but this would only amount to a DOS since the "real" client would just be unable to connect. No information would be leaked. It also would have to be in the same server (i.e. a module or nested exec called by the main client directly or transitively). This issue can also be squashed by not leaking the buildkit sock to nested execs/modules, which is possible now by migrating functionality from our shim to our custom executor. There's no immediate plans to do this but the possibility is open whenever needed (or when we make that change for other reasons). Signed-off-by: Erik Sipsma <[email protected]> * add integ test coverage Signed-off-by: Erik Sipsma <[email protected]> --------- Signed-off-by: Erik Sipsma <[email protected]> Signed-off-by: Alex Suraci <[email protected]> Signed-off-by: Erik Sipsma <[email protected]> Co-authored-by: Alex Suraci <[email protected]>
- Loading branch information