v2.18.0

Announcements

We are looking for maintainers, reach out in #5432.

Deprecation / Removal

• [Ambassador] Remove code, ci and ansible tags as it's no longer maintained and not working anymore. (#8086, @floryut)
• Drop support for Fedora 33 (#8246, @floryut)
• Remove ovn4nfv support (#8265, @floryut)
• Mitogen: support for the mitogen playbook accelerator is now deprecated in preparation of ansible upgrades, please clean up your playbooks that depend on it. (#8147, @cristicalin)
• Remove registry-proxy of container registry (#8327, @zhengtianbao)

Feature / Major changes

• Replace docker with containerd as the default container_manager (#8175, @cristicalin)
• Add ArgoCD as a kubernetes-app, using the new argocd_enabled variable (#7895, @atorrescogollo)
• Add ServiceTypes support to container registry (using new variables registry_service_type, registry_service_clusterIP, registry_service_loadBalancerIP, registry_service_annotations, registry_service_nodePort) (#8291, @zhengtianbao)
• Add TLS and authentication support to container registry (using new variables registry_tls_secret, registry_htpasswd, registry_config) (#8229, @zhengtianbao)
• Add a new option cert_manager_trusted_internal_ca to specify trusted internal ca of cert_manager. (#8135, @infra-monkey)
• Add a new option metrics_server_resizer (default to false) to control the addon-resizer container deployment in metrics-server pod (#8018, @oomichi)
• Add an optional fallback to node drain during cluster upgrades using --disable-eviction flag (#8094, @utkuozdemir)
• Add capability to use node swap with kubernetes 1.22+ (using new variable kubelet_fail_swap_on, default to true) (#8241, @cristicalin)
• Add possibility of automation creation of Load Balancers on Google Compute Engine (#8179, @lmercl)
• Add support for Fedora 35 (#8234, @floryut)
• Add support for Rocky Linux (#8095, @ooraini)
• Add support for cgroups v2 (no more reverting to cgroups v1 for Fedora) (#8237, @cristicalin)
• Add the ability to skip some phases in the kubeadm join_phase using kubeadm_join_phases_skip (#8067, @necatican)
• Added terraform support for Hetzner Cloud (#8053, @Xartos)
• Allow to scrape etcd metrics using a service (#8203, @sathieu)
• Default DNS replica count is now set to the minimum value between 2 and the length of k8s_cluster inventory group. (#8112, @smasset)
• Determine root filesistem device and partition before running growpart (allowing to not always be sda1) (#8024, @mlorenzo-stratio)
• Ensure apparmor is installed on Ubuntu (#8036, @rtsp)
• Fail metrics-server installation when addon-resizer is used on a platform different than amd64 (#8144, @zhengtianbao)
• Krew: upgrade to v0.4.2 (#8168, @zhengtianbao)
• Move deprecated kube_feature_gates from kebelet args to kubelet config (#8048, @fungusakafungus)
• Multiple Ansible versions are now supported (2.9/2.10/2.11) and tested by CI (#8172, @cristicalin)
• Prefer nodelocaldns as dns server over coredns when defined (#7731, @Alvaro-Campesino)
• Python 2.7: revive python2.7 support on EL7, note that this is not properly exercised in CI. (#8192, @cristicalin)
• Remove Terraform 0.14/0.15 support and CI -> Add TF 1.x (#8062, @floryut)
• Support Python 3.10 - ruamel.yaml.clib need to be updated to 0.2.4 (#8034, @olivierlemasle)
• Update Netchecker to v1.2.2 - now local etcd backend is needed to run (#8074, @cristicalin)
• Update registry template with additional options (security context and proves) and variables (registry_storage_access_mode to changes access mode, registry_replica_count for replicas) (#8198, @zhengtianbao)
• [nodelocaldns] add the capability to hot swap nodelocaldns without causing DNS blackholes during the swap (#8100, @cristicalin)
• Add Ingress support to container registry (using new variables registry_ingress_annotations, registry_ingress_host, registry_ingress_tls_secret) (#8311, @zhengtianbao)

Applications

• [cinder-csi] Add new variable cinder_csi_rescan_on_resize to control rescan-on-resize option (#8057, @reneluria)
• [cinder-csi] Added variable cinder_tolerations that sets tolerations for cinder-csi-nodeplugin DaemonSet (no tolerations by default) (#8137, @Ajarmar)
• [cinder-csi] Update version to support Kubernetes 1.22 and up (#8296, @StevenReitsma)
• [Metallb] Allow changing metallb default pool name (var metallb_pool_name) (#8111, @damjanek)
• [Metallb] Allow setting 'auto-assign' property to 'false' for default IP pool (var matallb_auto_assign) (#8193, @IKRozhkov)
• [Openstack] Fix a bug where Openstack cloud provider could not be used with username/password (#8021, @bl0m1)
• [Openstack] Replaces the global use_server_groups with the option to enable and set server group policy for each of the master, etcd, and node server groups respectively. (#8046, @OlleLarsson) (see Notes 2)
• [Openstack] Adds the option to set boot volume type for k8s nodes (using node_volume_type variable) (#8256, @robinAwallace)
• [Openstack] Use a pre-existing floating IP for bastion node, instead of creating a new one. (#8214, @feber)
• [nginx-ingress] Nginx controller now also watch kind:ingress without class (#8128, @LuckySB)
• [vSphere-CSI] Update to 2.4.0 (#8295, @cristicalin)
• [vSphere] Terraform code now documents and requires specification of the OVF template to use and separate specification of the netmask to use. (#8178, @llarsson)

Network

• [Calico] Add support for BGPPeer sourceAddress (#8306, @kakkotetsu)
• [Calico] Reduced calico bird route removal time on large ...

v2.17.1

Major changes

• Update kubernetes version to 1.21.6 (#8142, @oomichi)
• Add a new option metrics_server_resizer (default to false) to control the addon-resizer container deployment in metrics-server pod (#8018, @oomichi)
• Add an optional fallback to node drain during cluster upgrades using --disable-eviction flag (#8102, @utkuozdemir)
• Ensure apparmor is installed on Ubuntu (#8036, @rtsp)
• Default DNS replica count is now set to the minimum value between 2 and the length of k8s_cluster inventory group (#8109, @smasset)

Applications

• [Openstack] Fix a bug where Openstack cloud provider could not be used with username/password (#8021, @bl0m1)

Network

• [Calico] Check if 'plugins' key exists in calico_cni_config object allowing user to add nodes using both playbooks (#7717, @dlouks)
• [Calico] Fix typha prometheus causing a deployment error (#8005, @ericlake)
• [Calico] Increase node probe timeouts and add calico_node_readinessprobe_timeout/calico_node_livenessprobe_timeout to tune them (#7981, @cristicalin)
• [Cilium] Fix operator metrics activation (enable-metrics key missing) (#8000, @L3o-pold)

Bug or Regression

• Add missing proxy settings for subscription-manager in RHEL OS (if http_proxy is defined) (#8012, @oomichi)
• Fix CentOS7 issue with allowPrivilegeEscalation value from metrics-server (#8014, @oomichi)
• Fix k8s-certs-renew cp path wrongly using /usr/bin/ (#7992, @lazybetrayer)
• Fix containerd failed to start if apparmor is not installed (#8011, @rtsp)

Other note worthy changes

• Use kube_config_dir for kubeconfig instead of hard path in multiple plays (#7996, @oomichi)

v2.17.0

Announcements

We are looking for maintainers, reach out in #5432.

Deprecation / Removal

• Drop support for Fedora 32 (#7657)

Major changes

• Add support for Fedora 34 (#7657)
• Add Debian 11 (bullseye) support (#7853)
• Enable Graceful Node Shutdown for Kubernetes >= 1.21.0 (#7746)
• Move to Ansible 3.x by default (#7672) (see Notes 1)
• Set selinux type t_etc if selinux state is enforcing (#7791)
• Add Infomaniak to compatible public clouds list (#7910)
• During pre-upgrade add a flag to always cordon (#7892) (see Notes 2)
• Update Terraform 0.15 to tf validated and tested versions (#7927)
• Feature DynamicKubeletConfig is deprecated in 1.22 and will not move to GA (#7938) (see Notes 5)
• Inventory builder can now add IP to inventory (#7583) (see Notes 6)
• Add a new option kubeadm_upgrade_auto_cert_renewal to control certificates renewal during control plane upgrade (#7976)

Applications

• [Openstack] Openstack cloud config: store cloud.conf and API CA cert in k8s secret and avoid writing them to disk (#7603)
• [vSphere] vSphere credentials can now be passed as environment variables (#7646)
• [vSphere] Update vSphere CPI ClusterRole according to the latest official CPI manifests (#7838)
• [vSphere] Add suport of Vsphere CSI driver 2.2.X versions (#7848)
• [Cinder] Add cinder_csi_ignore_volume_az (#7624)
• [Cinder] Added support for application credentials for cinder-csi (#7799)
• [Cinder] Added support for sourcing application credentials from environment variables (#7799)
• [MetalLB] Update to v0.10.2 (#7925)
• [MetalLB] Update default variable: keep nodeSelector in one place (#7931)
• [CSI] Update CSI snapshotter and allow enabling it stand-alone (#7943)
• [nginx-ingress] Bump to 1.0.0 to support kube 1.22 (#7942) (see Notes 3) (see Notes 4)
• [UpCloud] Updated terraform script to use private network and dynamic additional disks (#7779)

Container managers

• [Kata-container] Replace deprecated 1.x version of Kata containers with the new 2.x (#7670)
• [gVisor] Add initial support for gVisor container runtime (#7661)
• [CRI-O] Allow cri-o offline install (#7777)
• [CRI-O] Add cri-o to support secure/insecure registry authentication (#7837)
• [Containerd] Enable containerd on Fedora CoreOS (#7794)
• [Containerd] Add containerd on Flatcar Container Linux (#7681)
• [Containerd] Add containerd secure/insecure registry authentication support (#7868)

Network

• [Calico] Add support for Calico 3.19.1 (#7630)
• [Calico] Add retries to 'Set label for route reflector' task (#7645)
• [Calico] Support enabling the eBPF dataplane for Calico (#7618)
• [Calico] Add Wireguard support (#7638)
• [Calico] Use --allow-version-mismatch in calicoctl.sh to allow upgrades (#7873)
• [Calico] kube_service_addresses_ipv6 is now added to serviceClusterIPs if enable_dual_stack_networks is true (#7944)
• [Cilium] Add cilium_operator_api_serve_addr to cilium operator config (#7901)

Other note worthy changes

• Add nodeSelector for other services and node labels before CNI setup (#7613)
• Allow deployers to limit the interface on which nodelocaldns exposes its prometheus listening port (#7748)
• Ubuntu changed package name python-apt to python3-apt (#7769)
• Retry to fetch binary if it fails first time (#7839)
• Remove environment variable in remove-node play (#7729)
• addons/cert_manager: Retries until webhook pods has been created (#7850)
• Add tags: always to all included service playbook (#7906)
• Use --no-cache-dir flag to pip in dockerfiles to save space (#7898)

Component versions:

• Kubernetes v1.21.5
• Etcd 3.4.13
• Docker 20.10
• Containerd 1.4.9
• CRI-O 1.21
• CNI-plugins v0.9.1
• Calico v3.19.2
• Cilium 1.9.10
• Flannel 0.14.0
• Kube-ovn 1.7.2
• Kube-Router 1.3.0
• Multus 3.7.2
• ovn4nfv v1.1.0
• Weave 2.8.1
• CoreDNS 1.8.0
• Nodelocaldns 1.17.1
• Helm 3.6.3
• ambassador: v1.5
• Nginx-ingress 1.0.0
• Cert-manager 1.0.4
• Kubernetes Dashboard v2.3.1

Known issues

• Ubuntu-16 won't work with default containerd version (1.4.9) as packages are not available, please use 1.4.6

Notes

1. Users need to uninstall ansible 2.9 to be able to install on top ansible 3.x which was split between ansible-base and ansible-collections.
2. Setting roles/upgrade/pre-upgrade/defaults/main.yml:upgrade_node_always_cordon to true causes a node to be drained before an upgrade and uncordoned after an upgrade even if the node is not cordoned when the upgrade begins.
3. Ingress-nginx: upgrade to 1.0.0 with stable ingress API, this version requires explicitly setting kubernetes.io/ingress.class: nginx on managed ingresses
4. ⚠️ nginx-ingress 1.0 does not support networking.k8s.io/v1beta
5. Flag --dynamic-config-dir has been deprecated, Feature DynamicKubeletConfig is deprecated in 1.22 and will not move to GA. It is planned to be removed from Kubernetes in the version 1.23. Please use alternative ways to update kubelet configuration.
6. The dynamic inventory builder will by default overwrite the inventory config. This was previously unintended behavior. In order to add new hosts into the already existing inventory config use the add command e.g. $ inventory.py add 10.0.1.8

v2.16.0

Announcements

We are looking for maintainers, reach out in #5432.

Deprecation / Removal

• Remove contrib/vault (Outdated since 2018) (#7400)
• Drop support for calico version 3.15.x (#7545)

Major changes

• Replace inventory group kube-master with kube_control_plane (#7256) (see Notes 5)
• Move kubernetes/master to kubernetes/control-plane (#7218) (see Notes 1)
• Move recover_control_plane/master to control-plane (#7236) (see Notes 2)
• Replace KUBE_MASTERS with KUBE_CONTROL_HOSTS (#7257) (see Notes 3)
• Rename ansible groups to use _ instead of - (#7552) (see Notes 7)
• Add AlmaLinux support (#7538)
• Add terraform support for Exoscale (#7141)
• Add terraform support for Vsphere (#7306)
• Add terraform support for UpCloud (#7360)
• Support for CentOS 8 and derivatives is considered stable (#7615)
• Support dual stack IPv4 & IPv6 networking (#6859)
• Auto renew control plane certificates (#7358) (see Notes 4)
• Add auto_renew_certificates_systemd_calendar to configure when K8S certificates renewal runs (#7490)
• Specify runAsGroup, allow safe sysctls by default (#7399)
• Add KubeSchedulerConfiguration for k8s 1.19 and up (#7351) (see Notes 6)
• Add script for generate download files and images list (#7561)
• Terraform 0.12+ is now required to run scripts under contrib/terraform/aws (#7576)
• Allow using ansible 2.10.x to deploy Kubespray (#7600)
• Add a contrib playbook (os-manage) to disable service firewall for Kubespray development and test (#7431)

Applications

• [Krew] Add krew support (#7464)
• [Openstack] Make sure worker rules is applied on workers (#7279)
• [Openstack] Write openstack controller manifests with correct perms (#7284)
• [Openstack] Allow users to set image_uuid instead of name, this allows the use of openstack community images (#7283)
• [Openstack] Use image id instad of name (#7293)
• [Openstack] Update Cinder CSI driver to v1.20.0 (#7280)
• [Openstack] Add most_recent = true while retrieving the latest image (#7376)
• [Openstack] Add external_openstack_enable_ingress_hostname option for external-openstack-cloud-controller-manager (#7572)
• [Metallb] Introduces optional tolerations and nodeSelector for metallb components (controller and speaker) (#7334)
• [CSI] Add suport of Vsphere CSI driver 2.X versions (#7480)
• [External-Provisioner] Add new variable "local_volume_provisioner_use_node_name_only" to configure local volume provisioner "useNodeNameOnly" option (#7421)

Container managers

• [CRI-O] Add experimental cri-o support for Amazon Linux 2 (#7353)
• [CRI-O] Add support for configuring cri-o pids_limit (#7525)
• [CRI-O] Fix support for cri-o on OracleLinux and add support for AlmaLinux (#7541)
• [Containerd] Fix reset.yml failing when using containerd (#7308)
• [Containerd] Add privileged_without_host_devices support (#7343)
• [Containerd] Update config.toml to V2 and set default runtime to io.containerd.runc.v2 and cgroup to systemd (#7398)
• [Containerd] Add containerd_extra_args (#7461)
• [Containerd] Add nerdctl cli tool for containerd users (#7500)
• [Containerd] Add support for Amazon Linux 2(#7595)
• [Docker] docker_dns_servers_strict had different default values, the default is now the same everywhere: false (#7499)
• [Docker] Add enablerepo: amzn2extra-docker to allow docker installation on Amazon linux (#7507)
• [crun] Update and changed the default crun version to v0.19 (#7433)
• [crictl] Change the owner of /etc/crictl.yaml to root (#7254)

Network

• [Calico] Fixup check when ipipMode / vxlanMode is not present (#7195)
• [Calico] Support for dual stack (IPv4 & IPv6) network deployment using Calico is introduced as an opt-in feature (#6859)
• [Calico] Add option to use calico with azure when using calico in vxlan (#7300)
• [Calico] Download Calico KDD CRDs (#7372)
• [Calico] Add the ability to customize calico's bird port, via calico_bird_listen_port variable (#7419)
• [Calico] Add new variable calico_node_startup_loglevel to configure CALICO_STARTUP_LOGLEVEL (Default to error) (#7530)
• [Calico] Allow specifying overriding BGP peer name (#7591)
• [Calico] Enables Calico serviceAccount token monitoring and update of /etc/cni/net.d/calico-kubeconfig if need be (#7586)
• [Calico] Add support to advertise MetalLB allocated IPs through Calico when using Calico 3.18 and greater (#7593)
• [Cilium] Allow cilium to be deployed with transparent encryption (#7342)
• [Cilium] Add cilium_ipam_mode variable (#7418)
• [Cilium] Move cilium kvstore settings to configmap (#7462)
• [Cilium] Update Cilium documentation and overall update of cilium role (#7521)
• [Ambassador] Add ingress_ambassador_multi_namespace setting, allows Ambassador operator to watch all namespaces for AmbassadorInstallation CRD resources (#7516)
• [Flannel] Add image_arch in image tag (#7560)

Other note worthy changes

• Added the ping_access_ip variable to enable(default)/disable ping test during preinstall (#7020)
• Rework proxy support (#7095)
• Remove ignore_errors from drain tasks and enable retires (#7151)
• Add other masters sequentially, not in parallel (#7166)
• Add 2 variables for upgrade, to prompt (upgrade_node_confirm, default false) and delay (upgrade_node_pause_seconds, default 0 seconds) (#7168)
• Change node-role.kubernetes.io from master to control-plane (#7183)
• Add retries to drain during upgrade. Allow leaving nodes cordoned after drain failure. Allow continuing upgrade if drain fails (#7227)
• Vagrantfile: always recreate inventory symlink (#7245)
• Updated etcd cert check tasks to detect when new cert gen is required (#7219)
• Only use stat get_checksum: yes when needed (#7270)
• Match on os-release ID / VARIANT_ID (#7269)
• Fix issue with kubeadm when *_PROXY variables are present in the environment (#7275)
• Kubespray now ignores *_PROXY vars found in your environment and only uses proxy configuration from the inventory (#7309)
• Facts.yaml: reduce the number of setup calls by ~7x (#7286)
• Fixup kubelet.conf to point to kubelet-client-current.pem (#7347)
• Check for dummy kernel module (#7348)
• Disable gather_facts for correctly work via bastion (#7265)
• Add etcd max snapshot and wals (#7382)
• Add cryptography module installation (#7404)
• Allow connecting to bastion via non-standard SSH port (#7396)
• Remove local lb privileged securityContext (#7437)
• Regenerate apiserver.crt on all controle-plane nodes when needed instead of just the first one (#7463)
• Check if python netaddr is installed and if Jinja is recent enough (#7486)
• Add ingress controller ingress-class var (#7522)
• Update Dockerfile to reduce Kubespray image size (#7556)
• Change kubeadm coredns addon images name to coredns/coredns (#7570)
• Allow usage of jinja2_native=True (#7612 / #7606)

Component versions:

• Kubernetes v1.20.7
• Etcd 3.4.13
• Docker 19.03
• Containerd 1.4.4
• CRI-O 1.20
• CNI-plugins v0.9.1
• Calico v3.17.4
• Cilium 1.8.9
• Flannel 0.13.0
• Kube-Router 1.2.2
• Multus 3.7
• Kube-ovn 1.6.2
• Weave 2.8.1
• CoreDNS 1.7.0
• Nodelocaldns 1.17.1
• Helm 3.5.4
• Nginx-ingress 0.43.0
• Cert-manager 1.0.4
• Kubernetes Dashboard v2.2.0

Known issues

• Ansible 2.11 is not supported and using it will results in errors
• Using Docker container engine could prompt "PLEG IS NOT HEALTHY" error, due to a runc bug, please see this issue for more information.

Notes

1. The role kubernetes/master has been renamed to kubernetes/control-plane, if using the role kubernetes/master solely on previous Kubespray, it is necessary to update the specified role.
2. The role recover_control_plane/master has been renamed to recover_control_plane/control-plane. If using the role recover_control_plane/master solely on previous Kubespray, it is necessary to update the specified role.
3. inventory_builder starts referring the environment variable KUBE_CONTROL_HOSTS to get the number of control-plane nodes, it still refers KUBE_MASTERS but it will be not referred after some deprecation cycles. Please specify KUBE_CONTROL_HOSTS if now specifying KUBE_MASTERS
4. You can enable control plane certificates automatic renewal using auto_renew_certificates, or manually use k8s-certs-renew.sh force_certificate_regeneration is removed as it was only renewing the api server certs and not all the other ones
5. The inventory group kube-master has been renamed to kube_control_plane. Please update your inventory file by replacing kube-master if continuing to use the existing inventory file.
6. New vars for configuring kube-scheduler were introduced (including extenders and profiles). Default vaules can be found at roles/kubernetes/control-plane/defaults/main/kube-scheduler.yml
7. Ansible groups were updated to be more consistent with dynamic inventory plugins: k8s-cluster -> k8s_cluster / kube-node -> kube_node / calico-rr -> calico_rr / no-floating -> no_floating

v2.15.1

This release includes the following changes (among other things):

• Set Kubernetes default version to v1.19.9
• Remove local lb privileged (#7454)
• Check kube-apiserver up on all masters before upgrade (#7217)
• Check for dummy kernel module (#7348)
• containerd,docker: stop installing extras repo on CentOS/RHEL
• Calico: fixup check when ipipMode / vxlanMode is not present
• Update azure cloud config (#7221)
• roles/docker: Make repokey fingerprint overrideable (#7263)
• Adding other masters sequentially, not in parallel (#7166)
• calico: fix NetworkManager check (#7169)
• Remove ignore_errors from drain tasks and enable retires (#7151)
• Correct Jinja Syntax for etcd-unsupported-arch (#6919)
• Fix unintended SIGPIPEs. (#7214)
• Fix: Bastion undefined variable (#7227)
• Ensure when use_oracle_public_repo is set to false the public Oracle
• Fix ansible calico route reflector tasks in calico role (#7224)
• Run containerd related tasks on OracleLinux. (#7250)
• Remove deletion of coredns deployment. (#7211)
• Fix Restart network doesn't work on Fedora CoreOS (#7271)
• Only use stat get_checksum: yes when needed (#7270)
• Fixup cri-o metacopy mount options (#7287)
• Ensure kubeadm doesn't use proxy (#7275)
• Ensure we gather IPv6 facts
• Add privileged_without_host_devices support (#7343)
• Auto renew control plane certificates (#7358)
• Fix k8s-certs-renew for k8s < 1.20 (#7410)
• Fixup kubelet.conf to point to kubelet-client-current.pem (#7347)
• Fix "api is up" check (#7295)
• Fix remove-node by removing jq usage (#7405)
• Fix reset when using containerd (#7308)
• Fix proxy usage when *_PROXY are present in environment (#7309)
• Fix the filename </etc/vault> is Duplicate in the reset role. (#7313)
• Fix recover-control-plane undefined 'proxy_disable_env' variable (#7326)
• Fix: added string to bool conversion for use_localhost_as_kube api load balancer (#7324)

v2.15.0

Announcements

We are looking for maintainers, reach out in #5432.

Deprecation / Removal

• Remove support for Fedora 31 (EOL)
• Remove support for Contiv CNI (#6964)
• Remove hyperkube support, no longer available in Kubernetes (#6965)
• Helm 2 can no longer be installed (#6846)

Major changes

• Add support for Fedora 33 (#7072) (see Notes)
• Add Kata Containers support to CRI-O runtime (#6830)
• Add RHEL support subscription registration (#6572)
• Add crun support (#6864)
• Add etcd tls cipher suites support (#7001)
• Add GCP terraform support (#6974)
• Allow airgapped CRI-O installation (#6927)
• Harden reset to work in more cases (#6781)
• Disable Kubernetes Dashboard by default (#6804) (see Notes)
• Add an option to force apiserver and respective client certificate to be regenerated without upgrading (#6403)
• Add a script to collect necessary container images and register the images to local registry (#7024)
• Major proxy rework on different playbooks (#7095)

Applications

• Allow configuration of nodelabels in local_volume_provisioner (#6620)
• [Openstack] Add external_openstack_lbaas_provider setting for occm (#6566)
• [Openstack] Add security groups not managed by terraform (#6865)
• [Openstack] Do not apply floating IP's before router port is created (#6887)
• [Openstack] Add cluster-name to external-openstack-cloud-controller-manager (#7055)
• [Azure / AWS] Added support for dynamic tags in AWS and Azure (#6752)

Container managers

• [All] Remove libseccomp install tasks (#7074)
• [Containerd] Add registry mirror support (#6962)
• [Containerd] Ensure libseccomp is installed before starting containerd on CentOS 8 (#6922)
• [Containerd] Add download run once feature (#6997)
• [Containerd] Allow root path and state path to be configured (#7098)
• [CRI-O] Use system default for storage driver by default (#6637)
• [CRI-O] Ensure service is started and enabled (#6753)
• [CRI-O] Reset is now working when CRI is set to CRI-O (#6812)
• [CRI-O] Avoid extra restart after install and upgrade (#6882)
• [CRI-O] Disable CRI-O restart by Multus (#6930)
• [CRI-O] Add registry mirror support (#6977)
• [CRI-O] Allow to enable download_run_once (#6998)
• [Docker] Add CentOS 8 and Fedora 32 docker repository (#6747)

Network

• [Weave] Add iptables_backend to weave options (#6639)
• [Calico] Add support for Calico CNI host-local IPAM plugin (#6580)
• [Calico] Added ability to set VXLAN vni and port. defaults to calico's documented default (#6678)
• [Calico] default to using kdd datastore (#6693)
• [Calico] Add retries to update calico-rr data in etcd through calicoctl (#6505)
• [Calico] Handle calico-rr nodes as workers so they get upgraded too (#6447)
• [Calico] Avoid POD restart during initial deploy (#6886)
• [Calico] Add serviceExternalIPs option for calico installation (#6928)
• [Calico] Update files to handle multi-asn bgp peering conditions (#6971)
• [Calico] Blacklist Calico's VXLAN interface from NetworkManager (#7037)
• [Calico] Check if inventory settings match cluster settings (#6969)
• [Flannel] Add multi architeture support to flannel (#6166)

Other note worthy changes

• Allow pre-existing floating IPs to be specified with k8s_master_fips (#6755)
• Set ansible_python_interpreter to python3 on debian (#6633)
• Allow resource management of metrics-server container (#6652)
• Use "kubeadm join" to join masters to control plane (#6661)
• Add new variable allowing additionnal audit webhook server configuration (#6726)
• Add leader election timeouts and durations to available parameters (#6691)
• Make sure node_ip is set if node is in etcd group (#6719)
• Install etcdctl to host when etcd deployment type is kubeadm (#6857)
• Chmod kubeconfig to avoid group-readable (#6800)
• Hold the docker-ce-cli upgrade in Debian (#6995)
• Removes apps tags from apps meta dependencies (#7041)
• Change owner to root for bin_dir directory (#6814)
• Add an option to disable globally applying a proxy to etc/yum.conf (#6828)
• Set feature gates in kube-proxy ConfigMap (#6851)
• Allow configuring container log limits for Kubelet (#6933) (see Notes)
• Remove executable bit from yaml and j2 files (#6894)
• Fails if kubeadm_version do not matches kubernetes version (#6302)
• Disable docker-ce yum repo by default (#7080)
• Improve reset with many tweak (#7094)
• Small Proxy fixes (add svc,svc.{{ dns_domain }} to no_proxy) (#7102)
• Restore ability to set pod eviction timer (#7114) (see Notes)
• Add ping_access_ip variable to enable/disable ping test during preinstall. Enabled by default (#7020)
• Remove unnecessary condition check when updating server field in kube-proxy kubeconfig (#7145)

Component versions:

• Kubernetes v1.19.7
• Etcd 3.4.13
• Docker 19.03
• Containerd 1.3.9
• CRI-O 1.19
• CNI-plugins v0.9.0
• Calico v3.16.5
• Cilium 1.8.6
• Flannel 0.13.0
• Kube-Router 1.1.1
• Multus 3.6
• Kube-ovn 1.5.2
• Weave 2.7.0
• CoreDNS 1.7.0
• Nodelocaldns 1.16.0
• Helm 3.3.4
• Nginx-ingress 0.41.2
• Cert-manager 1.0.4
• Kubernetes Dashboard v2.1.0

Known issues

• Ansible 2.10 is not supported and using it will results in errors (cf #7130)

Notes

• Kubernetes Dashboard deployment needs to be explicitly configured with dashboard_enabled: true
• Docker version for Fedora 33 needs to be set to 20.10 as they are the only packages available and validated
• Two new variables are used for this use case kube_apiserver_pod_eviction_not_ready_timeout_seconds and kube_apiserver_pod_eviction_unreachable_timeout_seconds
• Action required: users that relies on the default value of calico_datastore needs to explicitly configure their datastore choice.

v2.14.2

This release includes the following changes:

• Set ansible_python_interpreter to python3 on debian (#6744)
• Kubernetes v1.18.10 as default (#6842)
• Fix cinder & external_openstack cacert deployment (#6832)
• Fix unintended SIGPIPE (#6817)

v2.14.1

This release includes the following changes:

• NetworkManager lists must be separated by , (#6649)
• Move from widehat.opensuse to download.opensuse for crio centos (#6682)
• fix kubelet_flexvolumes_plugins_dir undefined (#6670)
• Add Kubernetes hashes 1.19.2/1.18.9/1.17.12 and set default (#6699)
• Make sure node_ip is set if node is in etcd group (#6720)
• properly generate extravolumes in kubeadmconfig for centos (#6707)

v2.13.4

This release includes the following changes:

• Add Kubernetes hashes 1.17.12 and set default (#6701)
• Add missing 'Set up proxy environment' tasks (#6591)
• Add Kubernetes 1.16.15 hashes (#6626)

v2.12.10

This release includes the following changes:

• Kubernetes 1.16.15 (#6583)