-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Description
Bug description
bindfilter.ApplyFileBinding (from github.com/Microsoft/go-winio/pkg/bindfilter) returns "Access is denied" for all calls made from a Kubernetes HostProcess container, even between two ordinary host directories. This prevents buildkitd from mounting containerd snapshots when running as a HostProcess container on Windows.
Reproduction
Deploy buildkitd as a HostProcess DaemonSet:
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: buildkitd-windows
namespace: buildkit
spec:
selector:
matchLabels:
app: buildkitd-windows
template:
metadata:
labels:
app: buildkitd-windows
spec:
nodeSelector:
kubernetes.io/os: windows
securityContext:
windowsOptions:
hostProcess: true
runAsUserName: "NT AUTHORITY\\SYSTEM"
hostNetwork: true
containers:
- name: buildkitd
image: <registry>/buildkitd:windows-0.28.0-servercore-ltsc2022
command:
- \$env:CONTAINER_SANDBOX_MOUNT_POINT/buildkit/buildkitd.exe
args:
- --addr
- tcp://0.0.0.0:1234
- --containerd-worker-namespace=k8s.io
- --root
- C:/buildkitdbuildkitd starts and finds the containerd worker:
level=info msg="found worker \"...\", labels=map[...worker.containerd.namespace:k8s.io ...worker.executor:containerd ...worker.snapshotter:windows], platforms=[windows/amd64]"
level=info msg="running server on [::]:1234"Any build fails at the first step:
#1 [internal] load build definition from Dockerfile
#1 ERROR: failed to mount {windows-layer ...\snapshots\120494 [rw]}:
failed to set volume mount path for layer ...\snapshots\120494:
failed to bind target "...\\snapshots\\120494\\Files"
to root "C:\\buildkitd\\tmp\\buildkit-mount1819228504": Access is denied.Root cause
To isolate the failure, we compiled a standalone test using go-winio/pkg/bindfilter and ran it inside the HostProcess container:
package main
import (
"fmt"
"os"
"github.com/Microsoft/go-winio/pkg/bindfilter"
)
func main() {
os.MkdirAll(os.Args[2], 0700)
err := bindfilter.ApplyFileBinding(os.Args[2], os.Args[1], true)
if err != nil {
fmt.Printf("FAILED: %v\n", err)
} else {
fmt.Println("OK")
bindfilter.RemoveFileBinding(os.Args[2])
}
}Test 1 — bind between two regular host directories (no containerd involvement):
```
C:> test-bind.exe C:\buildkitd\src-test C:\buildkitd\dst-test
FAILED: failed to bind target "C:\buildkitd\src-test" to root "C:\buildkitd\dst-test": Access is denied.
```
Test 2 — the HCS layer APIs succeed from the same process:
```
C:> test-mount.exe C:\ProgramData\containerd...\snapshots\117263 C:\buildkitd\tmp\test
ActivateLayer... OK
PrepareLayer... OK
GetLayerMountPath... OK: C:\ProgramData\containerd...\snapshots\117263
ApplyFileBinding failed: Access is denied.
```
The process has SeBackupPrivilege and SeRestorePrivilege enabled, all directories have SYSTEM:Full Control ACLs, and bindflt.sys is running. Since the bind filter fails even between two fresh directories created by the same process, the issue is not related to containerd snapshots, ACLs, or privileges.
containerd 1.7+ elevates HostProcess containers to partial silos (hcsshim PR #1344). The bind filter driver does not support attaching filesystem filters inside silos (microsoft/Windows-Containers#268, microsoft/hcsshim#1699).
Workaround
The HostProcess container installs buildkitd as a native Windows service via --register-service. The service process runs outside the silo context, so the bind filter works.
Version information
```
buildkitd v0.28.0
containerd 1.7.29
Windows Server 2022 build 20348.4773
Kubernetes v1.34.2+k0s
```