v7.10.0

🚨 Breaking

• cf resource is not included in Concourse binary anymore since its repo has been moved to cloudfoundry community and no longer being maintained by Concourse team.

✈️ Features

• Update base image of all built-in resource types:

  • The following resources now use concourse/resource-types-base-image-static, which is based on paketobuildpacks/run-jammy-static: time, bosh-io-release, bosh-io-stemcell, github-release, mock
  • The following resources now use paketobuildpacks/run-jammy-base: git, docker-image, registry-image, tracker, hg, semver, s3, pool

• Update references to use ginkgo/v2 (#8550) @dtimm _^🔗

• Support "raw" encoding for volume streaming. (#8706) @evanchaoli _^🔗
  Add a new compression method raw to CONCOURSE_STREAMING_ARTIFACTS_COMPRESSION. The new method will cost more network bandwidth of workers but save a lot of worker's CPU times, and make volume streaming dramatically faster. The bigger volume under streaming the more dramatic improvement on streaming speed.

• Add a drift based number of goroutines to component scheduler. (#8709) @evanchaoli _^🔗
  Add a new ATC option --num-goroutine-threshold to specify a threshold of goroutine count. If set, when a ATC's goroutine count reaches to threshold, then it will get less possibility to run workloads than other ATCs that have less goroutines. This option will help distribute workloads across ATCs evenly.

• Hermetic for task container (#8713) @xtremerui _^🔗

  • add Hermetic: bool to task step configuration. When set to true, the task container will be running without external network access. Only worker runtime containerd supports this feature. There will be a reminder as warning when setting a pipeline contains task step that sets hermetic: true.

• Optimize db notify. (#8736) @evanchaoli _^🔗
  Optimized the database notifications, which will reduce TPS/QPS in the database side. A new ATC option --db-notification-bus-queue-size is added, defaults to 10000. If the UI doesn't load logs of running builds in time, then consider to increase value of the option.

• Added a maximum volume size that can be streamed (#8756) @evanchaoli _^🔗
  Add a new ATC option `CONCOURSE_STREAMING_SIZE_LIMITATION" that restricts maximum size in MB of volumes can be streamed between workers. This is a mechanism to prevent rogue pipeline from hurting multiple workers.

🐞 Bug Fixes

• atc/db: fix dropped error (#8678) @alrs _^🔗

• Fix cf connector error during web node startup (#8699) @xtremerui _^🔗

  • Fix web node start up error when cf connector is configured

• Fixed a race condition in component factory. (#8746) @evanchaoli _^🔗

• Bump ifrit to fix ATC gracefully terminate issue. (#8751) @evanchaoli _^🔗

  • Fixed an ATC gracefully terminate issue described in #8747.

• Add reset character in WaitingForStreamedVolume event render (#8768) @selzoc _^🔗

• Unhide the --instance-var option in fly set-pipeline (#8778) @neilmayhew _^🔗

🤷 Miscellaneous

• Bump dex to latest (#8666) @xtremerui _^🔗

• Fix failed fly integration test in darwin (#8681) @xtremerui _^🔗

  • Bump Golang to v1.20

• Update module github.com/containerd/containerd to v1.6.18 [SECURITY] (#8688) @renovate _^🔗

• Ignore elm and client-go in renovate deps bump (#8704) @xtremerui _^🔗

• bump lager to v3 (#8707) @xtremerui _^🔗

  • bump code.cloudfoundry.org/lager, concourse/retryhttp and concourse/flag to latest to remove indirect import of ginkgo v1 in Concourse's go.mod file.

• fix(deps): update module github.com/opencontainers/runc to v1.1.5 [security] (#8718) @renovate _^🔗

• fix ginkgo warning and k8s topgun failure (#8723) @xtremerui _^🔗

• add events logging when pod is not running for k8s topgun (#8733) @xtremerui _^🔗

  • Add method in k8s topgun test to log pod events when it is being initialized.

• Increase timeout for bosh topgun (#8740) @xtremerui _^🔗

• Fix test failure due to mock resource that built with paketo jammy (#8760) @xtremerui _^🔗

• Remove btrfs baggageclaim test over COS image (#8766) @xtremerui _^🔗

• fix(deps): update module github.com/opencontainers/runc to v1.1.5 [security] (#8770) @renovate _^🔗

• bumping containerd runtime libs (#8771) @xtremerui _^🔗

• refactor: move from io/ioutil to io and os packages (#8774) @Juneezee _^🔗

• chore: unnecessary use of fmt.Sprintf or fmt.Sprint (#8786) @testwill _^🔗

📦 Bundled resource types

• bosh-io-release: v1.2.0
• bosh-io-stemcell: v1.2.0
• docker-image: v1.8.0
• git: v1.15.0
• github-release: v1.9.0
• hg: v1.3.0
• mock: v0.13.0
• pool: v1.4.0
• registry-image: v1.9.0
• s3: v1.3.0
• semver: v1.7.0
• time: v1.7.0
• tracker: v1.1.0