Release Notes v1.81
23KE release notes and upgrade guide
Due to the upgrade of cert-management resources, you need to set some annoations and a label after the upgrade:
kubectl annotate customresourcedefinitions certificaterevocations.cert.gardener.cloud meta.helm.sh/release-name=garden-cert-management meta.helm.sh/release-namespace=garden
kubectl label customresourcedefinitions certificaterevocations.cert.gardener.cloud app.kubernetes.io/managed-by=Helm
kubectl annotate customresourcedefinitions certificates.cert.gardener.cloud meta.helm.sh/release-name=garden-cert-management meta.helm.sh/release-namespace=garden
kubectl label customresourcedefinitions certificates.cert.gardener.cloud app.kubernetes.io/managed-by=Helm
kubectl annotate customresourcedefinitions issuers.cert.gardener.cloud meta.helm.sh/release-name=garden-cert-management meta.helm.sh/release-namespace=garden
kubectl label --overwrite customresourcedefinitions issuers.cert.gardener.cloud app.kubernetes.io/managed-by=Helm
Maybe (in case of bad timing) you need to trigger the reconciliation of the cert-management helm release manually afterwards:
flux suspend hr cert-management
flux resume hr cert-management
Related upstream release notes / changelogs
Update os-ubuntu to 1.23.0
[gardener/gardener-extension-os-ubuntu]
⚠️ Breaking Changes
[OPERATOR]extension-os-ubuntuno longer supports Shoots with Кubernetes version < 1.22. by @shafeeqes [#82]
Update gardener-controlplane to 1.80.1
[gardener/gardener]
🐛 Bug Fixes
[USER]The two additional labelsworker.gardener.cloud/image-nameandworker.gardener.cloud/image-versionthat were previously introduced and attached to worker nodes are removed again to fix a regression that causes thekubeletto restart on nodes that are due to be upgraded to a new OS but not rolled yet which causes theirPods to become temporarily unready. by @gardener-ci-robot [#8551]
Docker Images
admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.80.1
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.80.1
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.80.1
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.80.1
operator: eu.gcr.io/gardener-project/gardener/operator:v1.80.1
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.80.1
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.80.1
Update gardener-controlplane to 1.80.1
[gardener/gardener]
🐛 Bug Fixes
[USER]The two additional labelsworker.gardener.cloud/image-nameandworker.gardener.cloud/image-versionthat were previously introduced and attached to worker nodes are removed again to fix a regression that causes thekubeletto restart on nodes that are due to be upgraded to a new OS but not rolled yet which causes theirPods to become temporarily unready. by @gardener-ci-robot [#8551]
Docker Images
admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.80.1
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.80.1
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.80.1
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.80.1
operator: eu.gcr.io/gardener-project/gardener/operator:v1.80.1
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.80.1
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.80.1
Update gardenlet to 1.80.1
[gardener/gardener]
🐛 Bug Fixes
[USER]The two additional labelsworker.gardener.cloud/image-nameandworker.gardener.cloud/image-versionthat were previously introduced and attached to worker nodes are removed again to fix a regression that causes thekubeletto restart on nodes that are due to be upgraded to a new OS but not rolled yet which causes theirPods to become temporarily unready. by @gardener-ci-robot [#8551]
Docker Images
admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.80.1
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.80.1
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.80.1
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.80.1
operator: eu.gcr.io/gardener-project/gardener/operator:v1.80.1
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.80.1
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.80.1
Update gardener-controlplane to 1.80.2
[gardener/gardener]
🐛 Bug Fixes
[USER]A bug causing unnecessary reorder of extension inShootspec.extensionsis fixed. by @gardener-ci-robot [#8575][OPERATOR]Fixed a possibility for themigratephase of control plane migration to become permanently stuck if the shoot was created when theMachineControllerManagerDeploymentfeature gate is disabled, control plane migration is triggered for the shoot and the feature gate is enabled during the migration phase. by @gardener-ci-robot [#8570]
Update gardener-controlplane to 1.80.2
[gardener/gardener]
🐛 Bug Fixes
[USER]A bug causing unnecessary reorder of extension inShootspec.extensionsis fixed. by @gardener-ci-robot [#8575][OPERATOR]Fixed a possibility for themigratephase of control plane migration to become permanently stuck if the shoot was created when theMachineControllerManagerDeploymentfeature gate is disabled, control plane migration is triggered for the shoot and the feature gate is enabled during the migration phase. by @gardener-ci-robot [#8570]
Update gardenlet to 1.80.2
[gardener/gardener]
🐛 Bug Fixes
[USER]A bug causing unnecessary reorder of extension inShootspec.extensionsis fixed. by @gardener-ci-robot [#8575][OPERATOR]Fixed a possibility for themigratephase of control plane migration to become permanently stuck if the shoot was created when theMachineControllerManagerDeploymentfeature gate is disabled, control plane migration is triggered for the shoot and the feature gate is enabled during the migration phase. by @gardener-ci-robot [#8570]
Update gardener-controlplane to 1.80.3
[gardener/gardener]
🐛 Bug Fixes
[USER]A bug has been fixed that prevented users without permissions to listCustomResourceDefinitions from interacting with the Gardener APIs when using akubectlversion lower than1.27. by @gardener-ci-robot [#8580]
Update gardener-controlplane to 1.80.3
[gardener/gardener]
🐛 Bug Fixes
[USER]A bug has been fixed that prevented users without permissions to listCustomResourceDefinitions from interacting with the Gardener APIs when using akubectlversion lower than1.27. by @gardener-ci-robot [#8580]
Update gardenlet to 1.80.3
[gardener/gardener]
🐛 Bug Fixes
[USER]A bug has been fixed that prevented users without permissions to listCustomResourceDefinitions from interacting with the Gardener APIs when using akubectlversion lower than1.27. by @gardener-ci-robot [#8580]
Update provider-openstack to 1.37.0
[gardener/machine-controller-manager]
🐛 Bug Fixes
[OPERATOR]IncludedUnavailableReplicasin determining if a machine deployment status update is needed by @rishabh-11 [gardener/machine-controller-manager#833][OPERATOR]Force drain and delete volume attachments for nodes un-healthy due toReadOnlyFileSystemandNotReadyfor too long by @elankath [gardener/machine-controller-manager#839][OPERATOR]An issue causing nil pointer panic on scaleup of the machinedeployment along with trigger of rolling update, is fixed by @acumino [gardener/machine-controller-manager#814][USER]An edge case where outdated DesiredReplicas annotation blocked a rolling update is fixed. by @rishabh-11 [gardener/machine-controller-manager#821]
🏃 Others
[DEVELOPER]status.Status now captures underline cause, allowing consumers to introspect the error returned by the provider. WrapError() function could be used to wrap the provider error by @unmarshall [gardener/machine-controller-manager#842][DEVELOPER]Removed dead metrics code and refactored the remaining metrics code by @himanshu-kun [gardener/machine-controller-manager#823][DEVELOPER]A new make target is introduced to add license headers. by @unmarshall [gardener/machine-controller-manager#845][DEVELOPER]Bumpk8s.io/*deps to v0.27.2 by @afritzler [gardener/machine-controller-manager#820][OPERATOR]Added a new metric that will allow to get the number of stale (due to unhealthiness) machines that are getting terminated by @jguipi [gardener/machine-controller-manager#808][OPERATOR]Updated to go v1.20.5 by @rishabh-11 [gardener/machine-controller-manager#827][OPERATOR]Makefile targets have changed: Introduced gardener-setup, gardener-restore, gardener-local-mcm-up, non-gardener-setup, non-gardener-restore, non-gardener-local-mcm-up. Users can also directly use the scripts which are used by these makefile targets. by @unmarshall [gardener/machine-controller-manager#852][OPERATOR]AddederrorCodefield in theLastOperationstruct. This should be implemented only for theCreateMachinecall in thetriggerCreationFlow. This field will be utilized by Cluster autoscaler to do early backoff by @rishabh-11 [gardener/machine-controller-manager#851][OPERATOR]New metrics introduced:- api_request_duration_seconds -> tracks time taken for successful invocation of provider APIs. This metric can be filtered by provider and service.
- driver_request_duration_seconds -> tracks total time taken to successfully complete driver method invocation. This metric can be filtered by provider and operation.
- driver_requests_failed_total -> records total number of failed driver API requests. This metric can be filtered by provider, operations and error_code. by @unmarshall [gardener/machine-controller-manager#842]
[gardener/gardener-extension-provider-openstack]
⚠️ Breaking Changes
[OPERATOR]provider-openstackno longer supports Shoots or Seeds with Кubernetes version < 1.24. by @shafeeqes [#670]
🏃 Others
[OPERATOR]Enable propagating pod routes to nodes without overlay network by @ScheererJ [#654][OPERATOR]updated image csi-attacher ->v4.4.0by @kon-angelo [#675][OPERATOR]updated image manila-csi-pluginv1.27.1->v1.27.2by @kon-angelo [#675][OPERATOR]updated image snapshot-controller ->v6.3.0by @kon-angelo [#675][OPERATOR]updated image livenessprobe ->v2.11.0by @kon-angelo [#675][OPERATOR]Add manila topology labels to machines. This enables scaling from 0 for pods depending on manila volumes. by @kon-angelo [#674][OPERATOR]updated image cinder-csi-pluginv1.27.1->v1.27.2by @kon-angelo [#675][OPERATOR]updated image cloud-provider-openstackv1.27.1->v1.27.2by @kon-angelo [#675][OPERATOR]updated image csi-provisioner ->v3.6.0by @kon-angelo [#675][OPERATOR]updated image csi-resizer ->v1.9.0by @kon-angelo [#675][DEVELOPER]Added description to openstack security group rules. by @nschad [#666]
[gardener/machine-controller-manager-provider-openstack]
🏃 Others
[USER]Vendor gardenerv1.79.0by @kon-angelo [gardener/machine-controller-manager-provider-openstack#100][USER]ResourceExhaustederror code is returned when no valid host is found in the zone by @rishabh-11 [gardener/machine-controller-manager-provider-openstack#97][DEPENDENCY]The following dependency is updated:- github.com/gardener/machine-controller-manager v0.49.1 -> v0.50.0 by @rishabh-11 [gardener/machine-controller-manager-provider-openstack#98]
Update cert-management to 0.11.2
[gardener/cert-management]
🏃 Others
[OPERATOR]Update k8s dependencies by updating controller-manager-library by @MartinWeindel [#142][OPERATOR]Bumps golang from 1.21.1 to 1.21.2. by @MartinWeindel [#142]
Update external-dns-management to 0.15.9
[gardener/external-dns-management]
✨ New Features
[USER]Creating aDNSEntryfor the base domain of a hosted zone is now allowed for all providers butazure-dnsandazure-private-dns. by @MartinWeindel [#316]
🏃 Others
[OPERATOR]Update AWS canonical hosted zones from github.com/kubernetes-sigs/external-dns repository. by @MartinWeindel [#322][OPERATOR]Bumps golang from 1.21.1 to 1.21.2. by @MartinWeindel [#323][OPERATOR]Bumps golang from 1.20.7 to 1.21.1. by @dependabot[bot] [#318][OPERATOR]Update k8s dependencies by updating controller-manager-library by @MartinWeindel [#323][OPERATOR]Additional AWS regionsap-southeast-4andil-central-1with canonical hosted zones for ELBs by @MartinWeindel [#321][USER]Infoblox provider: support for extensible attributes by @nitrocb [#320]
Update shoot-cert-service to 1.38.0
[gardener/gardener-extension-shoot-cert-service]
🐛 Bug Fixes
[OPERATOR]TheCustomResourceDefinitions deployed to shoot clusters are now labelled withshoot.gardener.cloud/no-cleanup=trueto preventgardenletto deleting them during shoot deletion. by @MartinWeindel [#195][OPERATOR]TheCustomResourceDefinitions deployed to shoot clusters are now annotated withresources.gardener.cloud/skip-health-check=trueto preventgardener-resource-managerfrom recreating them too fast during shoot deletion. by @rfranzke [#194]
🏃 Others
[OPERATOR]Bumps golang from 1.21.0 to 1.21.1. by @dependabot[bot] [#193][OPERATOR]The following dependency is updated:- github.com/gardener/gardener: v1.77.0-> v1.80.1
- k8s.io/* : v0.26.3 -> v0.28.2
- sigs.k8s.io/controller-runtime: v0.14.6-> v0.16.2 by @acumino [#196]
[gardener/cert-management]
🐛 Bug Fixes
[OPERATOR]Fix edge case of inconsistent certificate/secret: request certificate in this case. by @MartinWeindel [gardener/cert-management#138][USER]Disable followCNAME by default again as it was activated implicitly by github.com/go-acme/lego version upgrade by @MartinWeindel [gardener/cert-management#140]
🏃 Others
[OPERATOR]Bumps golang from 1.21.1 to 1.21.2. by @MartinWeindel [gardener/cert-management#142][OPERATOR]Update k8s dependencies by updating controller-manager-library by @MartinWeindel [gardener/cert-management#142]
Update shoot-dns-service to 1.39.0
[gardener/external-dns-management]
✨ New Features
[USER]Creating aDNSEntryfor the base domain of a hosted zone is now allowed for all providers butazure-dnsandazure-private-dns. by @MartinWeindel [gardener/external-dns-management#316]
🏃 Others
[USER]Infoblox provider: support for extensible attributes by @nitrocb [gardener/external-dns-management#320][OPERATOR]Update AWS canonical hosted zones from github.com/kubernetes-sigs/external-dns repository. by @MartinWeindel [gardener/external-dns-management#322][OPERATOR]Bumps golang from 1.20.7 to 1.21.1. by @dependabot[bot] [gardener/external-dns-management#318][OPERATOR]Additional AWS regionsap-southeast-4andil-central-1with canonical hosted zones for ELBs by @MartinWeindel [gardener/external-dns-management#321][OPERATOR]Update k8s dependencies by updating controller-manager-library by @MartinWeindel [gardener/external-dns-management#323][OPERATOR]Bumps golang from 1.21.1 to 1.21.2. by @MartinWeindel [gardener/external-dns-management#323]
[gardener/gardener-extension-shoot-dns-service]
⚠️ Breaking Changes
[OPERATOR]extension-shoot-dns-serviceno longer supports Shoots with Кubernetes version < 1.24. by @shafeeqes [#241]
🐛 Bug Fixes
[OPERATOR]TheCustomResourceDefinitions deployed to shoot clusters are now annotated withresources.gardener.cloud/skip-health-check=trueto preventgardener-resource-managerfrom recreating them too fast during shoot deletion. by @rfranzke [#240][OPERATOR]TheCustomResourceDefinitions deployed to shoot clusters are now labelled withshoot.gardener.cloud/no-cleanup=trueto preventgardenletto deleting them during shoot deletion. by @MartinWeindel [#243]
🏃 Others
[OPERATOR]Add copy of images.yaml to charts dir to resolve installation issue for landscapes using RBSC by @MartinWeindel [#237][OPERATOR]The following dependency is updated:- github.com/gardener/gardener: v1.77.0-> v1.80.1
- k8s.io/* : v0.26.3 -> v0.28.2
- sigs.k8s.io/controller-runtime: v0.14.6-> v0.16.2 by @acumino [#244]
[OPERATOR]Bumps golang from 1.21.0 to 1.21.1. by @dependabot[bot] [#239]
Update gardener-controlplane to 1.81.0
[gardener/etcd-druid]
⚠️ Breaking Changes
[USER]Update etcd-custom-image tov3.4.26-2. by @shreyas-s-rao [gardener/etcd-druid#656][OPERATOR]Etcd druid will now not supportpolicy/v1beta1forPodDisruptionBudgets and will only usepolicy/v1forPodDisruptionBudgets by @aaronfern [gardener/etcd-druid#681]
📰 Noteworthy
[OPERATOR]custodian-sync-periodvalue is set to15sin the Helm chart for etcd-druid. by @shreyas-s-rao [gardener/etcd-druid#688][OPERATOR]Add new flagmetrics-scrape-wait-durationfor compaction controller to set a wait duration at the end of every compaction job, to allow for metrics to be scraped by a Prometheus instance. by @abdasgupta [gardener/etcd-druid#686][OPERATOR]Etcd snapshot compaction jobs will now be named<etcd-name>-compactorfor better readability for human operators. by @abdasgupta [gardener/etcd-druid#672]
✨ New Features
[OPERATOR]IntroduceSpec.Backup.DeltaSnapshotRetentionPeriodin theEtcdresource to allow configuring retention period for delta snapshots. by @seshachalam-yv [gardener/etcd-druid#651][DEVELOPER]Add support forLocalprovider for e2e tests. by @shreyas-s-rao [gardener/etcd-druid#668]
🐛 Bug Fixes
[OPERATOR]A bug causing incorrect volume mount path forEtcds andEtcdCopyBackupsTasks usingLocalsnapshot storage provider while using distroless etcd-backup-restore imagev0.25.xhas been resolved. by @aaronfern [gardener/etcd-druid#662][OPERATOR]Custodian controller no longer watches leases owned by the etcd resources, thus reducing frequency of etcd status updates and now honouringcustodian-sync-periodvalue. by @shreyas-s-rao [gardener/etcd-druid#688][OPERATOR]Resolved an issue where the Custodian Controller was not updating theReplicasfield in theetcdstatus to reflect theCurrentReplicasfrom the StatefulSet status. This fix ensures consistent behavior with theetcdController in Druid. by @seshachalam-yv [gardener/etcd-druid#701][OPERATOR]A bug causingEtcdCopyBackupsTaskjobs to fail to create temp snapshot directory while using distroless etcd-backup-restore imagev0.25.xhas been resolved. by @aaronfern [gardener/etcd-druid#662]
🏃 Others
[OPERATOR]Upgradedetcd-backup-restorefromv0.24.3tov0.24.6foretcd-custom-image, and fromv0.25.1tov0.26.0foretcd-wrapperby @gardener-robot-ci-3 [gardener/etcd-druid#687][OPERATOR]All default images are now present inimages.yamlby @aaronfern [gardener/etcd-druid#673]
📖 Documentation
[DEVELOPER]Introduce DEPs (Druid Enhancement Proposals) for proposing large design changes in etcd-druid. by @shreyas-s-rao [gardener/etcd-druid#659][OPERATOR]Introduce DEP-04 EtcdMember Custom Resource. by @shreyas-s-rao [gardener/etcd-druid#658]
[gardener/etcd-backup-restore]
📰 Noteworthy
[USER]Introduce flagmetrics-scrape-wait-durationtoetcdbrctl compactcommand, that specifies a wait duration at the end of a snapshot compaction, to allow Prometheus to scrape metrics related to compaction before theetcdbrctlprocess exits. by @abdasgupta [gardener/etcd-backup-restore#667][OPERATOR]Etcd-backup-restore now uses the user home directory to create files. by @aaronfern [gardener/etcd-backup-restore#637][OPERATOR]Etcd-backup-restore now uses a distroless image as its base image. It is no longer compatible with etcd-custom-image, and must be used with etcd-wrapper instead. by @aaronfern [gardener/etcd-backup-restore#637]
🏃 Others
[OPERATOR]Upgraded Ginkgo v1 to v2 and updated other dependencies by @seshachalam-yv [gardener/etcd-backup-restore#647][OPERATOR]While scaling up a non-HA etcd cluster to HA skipping the scale-up checks for first member of etcd cluster as first member can never be a part of scale-up scenarios. by @ishan16696 [gardener/etcd-backup-restore#649][OPERATOR]Bump alpine base version for Docker build to3.18.2. by @shreyas-s-rao [gardener/etcd-backup-restore#638][OPERATOR]Backup-restore waits for its etcd to be ready before attempting to update peerUrl by @aaronfern [gardener/etcd-backup-restore#628][OPERATOR]Introduceddelta-snapshot-retention-periodCLI flag to extend the configurable retention period for delta snapshots inetcd-backup-restore, enhancing flexibility for backup retention. by @seshachalam-yv [gardener/etcd-backup-restore#640][OPERATOR]Revendors the bbolt fromv1.3.6tov1.3.7by @ishan16696 [gardener/etcd-backup-restore#659][DEVELOPER]Add CVE categorization for etcd-backup-restore. by @shreyas-s-rao [gardener/etcd-backup-restore#644]
[gardener/machine-controller-manager]
🐛 Bug Fixes
[OPERATOR]Force drain and delete volume attachments for nodes un-healthy due toReadOnlyFileSystemandNotReadyfor too long by @elankath [gardener/machine-controller-manager#839][OPERATOR]IncludedUnavailableReplicasin determining if a machine deployment status update is needed by @rishabh-11 [gardener/machine-controller-manager#833][OPERATOR]An issue causing nil pointer panic on scaleup of the machinedeployment along with trigger of rolling update, is fixed by @acumino [gardener/machine-controller-manager#814][USER]An edge case where outdated DesiredReplicas annotation blocked a rolling update is fixed. by @rishabh-11 [gardener/machine-controller-manager#821]
🏃 Others
[DEVELOPER]status.Status now captures underline cause, allowing consumers to introspect the error returned by the provider. WrapError() function could be used to wrap the provider error by @unmarshall [gardener/machine-controller-manager#842][DEVELOPER]A new make target is introduced to add license headers. by @unmarshall [gardener/machine-controller-manager#845][DEVELOPER]Bumpk8s.io/*deps to v0.27.2 by @afritzler [gardener/machine-controller-manager#820][DEVELOPER]Removed dead metrics code and refactored the remaining metrics code by @himanshu-kun [gardener/machine-controller-manager#823][OPERATOR]New metrics introduced:- api_request_duration_seconds -> tracks time taken for successful invocation of provider APIs. This metric can be filtered by provider and service.
- driver_request_duration_seconds -> tracks total time taken to successfully complete driver method invocation. This metric can be filtered by provider and operation.
- driver_requests_failed_total -> records total number of failed driver API requests. This metric can be filtered by provider, operations and error_code. by @unmarshall [gardener/machine-controller-manager#842]
[OPERATOR]Updated to go v1.20.5 by @rishabh-11 [gardener/machine-controller-manager#827][OPERATOR]Added a new metric that will allow to get the number of stale (due to unhealthiness) machines that are getting terminated by @jguipi [gardener/machine-controller-manager#808][OPERATOR]AddederrorCodefield in theLastOperationstruct. This should be implemented only for theCreateMachinecall in thetriggerCreationFlow. This field will be utilized by Cluster autoscaler to do early backoff by @rishabh-11 [gardener/machine-controller-manager#851][OPERATOR]Makefile targets have changed: Introduced gardener-setup, gardener-restore, gardener-local-mcm-up, non-gardener-setup, non-gardener-restore, non-gardener-local-mcm-up. Users can also directly use the scripts which are used by these makefile targets. by @unmarshall [gardener/machine-controller-manager#852]
[gardener/gardener]
⚠️ Breaking Changes
[DEPENDENCY]Extensions have to implement theForceDeletefunction in the actuator with the logic of forcefully deleting all the resources deployed by them. by @shafeeqes [#8414][DEPENDENCY]Theextensions/pkg/controller.Use{TokenRequestor,ServiceAccountTokenVolumeProjection}functions have been removed since they always returntrue. by @rfranzke [#8582][OPERATOR]⚠️ Gardener does no longer support garden, seed, or shoot clusters with Kubernetes versions < 1.24. Make sure to upgrade all existing clusters before upgrading to this Gardener version. by @shafeeqes [#8487][DEVELOPER]Thepkg/utils/gardener.IntStrPtrFromIntfunction has been renamed toIntStrPtrFromInt32sinceintstr.FromIntis deprecated. by @rfranzke [#8579][USER]Thealpha.kube-apiserver.scaling.shoot.gardener.cloud/classannotation onShoots has no effect anymore and should be removed. by @rfranzke [#8526]
📰 Noteworthy
[USER]The two additional labelsworker.gardener.cloud/image-nameandworker.gardener.cloud/image-versionthat were previously introduced and attached to worker nodes are removed again to fix a regression that causes thekubeletto restart on nodes that are due to be upgraded to a new OS but not rolled yet which causes theirPods to become temporarily unready. by @MrBatschner [#8524][OPERATOR]TheMachineControllerManagerDeploymenthas been promoted to beta and is now enabled by default. Make sure that all registered provider extensions support this feature gate before upgrading to this version of Gardener. by @rfranzke [#8526][OPERATOR]TheDisableScalingClassesForShootsfeature gates has been promoted to GA (and is now always enabled). by @rfranzke [#8526]
✨ New Features
[USER]Thegardener-schedulernow populates scheduling failure reasons to theShoot's.status.lastOperation.descriptionfield. by @rfranzke [#8527][USER]When theShootForceDeletionfeaturegate in the apiserver is turned on, users will be able to force-delete the Shoot. You MUST ensure that all the resources created in the IaaS account are cleaned up to prevent orphaned resources. Gardener will NOT delete any resources in the Shoot cloud-provider account. See Shoot Force Deletion for more details. by @shafeeqes [#8414][USER]Multiple expanders forcluster-autoscalercan now be specified in theShootAPI via the.spec.kubernetes.clusterAutoscaler.expanderfield. by @aaronfern [#8573]
🐛 Bug Fixes
[OPERATOR]Fixed a possibility for themigratephase of control plane migration to become permanently stuck if the shoot was created when theMachineControllerManagerDeploymentfeature gate is disabled, control plane migration is triggered for the shoot and the feature gate is enabled during the migration phase. by @plkokanov [#8568][USER]Fix an issue, where DNS lookups for non-existing pods of a StatefulSet yielded one of the existing pods even when it should not have. by @axel7born [#8544][USER]A bug has been fixed that prevented users without permissions to listCustomResourceDefinitions from interacting with the Gardener APIs when using akubectlversion lower than1.27. by @rfranzke [#8577][USER]A bug causing unnecessary reorder of extension inShootspec.extensionsis fixed. by @acumino [#8569]
🏃 Others
[OPERATOR]The shoot namespace in seeds is redeployed during the shoot migration flow to update the zones in use. by @plkokanov [#8564][OPERATOR]nginx-ingress-controllerimage is updated tov1.9.0. by @shafeeqes [#8558][OPERATOR]Add an alert for VPNHAShootNoPods when shoot in HA (high availability) mode. by @tedteng [#8506][USER]Gardener refined the scope of the problematic webhook matcher forendpointobjects. Earlier, shoot clusters were assigned a constraint reporting a problem with afailurePolocy: Failwebhook acting on these objects. Now, onlyendpoints in thekube-systemanddefaultsnamespaces are considered for this check. by @acumino [#8521]
[gardener/autoscaler]
✨ New Features
[DEVELOPER]unit tests framework introduced to test implemented methods ofCloudproviderandNodegroupinterface by @rishabh-11 [gardener/autoscaler#215][USER]Gardener autoscaler now backs-off early from a node-group (i.e. machinedeployment) in case ofResourceExhaustederror. Refer docs athttps://github.com/gardener/autoscaler/blob/machine-controller-manager-provider/cluster-autoscaler/FAQ.md#when-does-autoscaler-backs-off-early-from-a-node-groupfor details. by @himanshu-kun [gardener/autoscaler#253]
🐛 Bug Fixes
[OPERATOR]A bug where MCM removed a machine other than the one , CA wanted , is resolved. by @rishabh-11 [gardener/autoscaler#215]
🏃 Others
[OPERATOR]Initial implementation forRefresh()method ofCloudProviderinterface done by @rishabh-11 [gardener/autoscaler#215][OPERATOR]machinepriority.machine.sapcloud.ioannotation on machine is now reset to 3 by autoscaler if the corresponding node doesn't haveToBeDeletedByClusterAutoscalertaint by @rishabh-11 [gardener/autoscaler#215]
[gardener/etcd-custom-image]
📰 Noteworthy
[OPERATOR]Update alpine base image version to 3.18.3. by @shreyas-s-rao [gardener/etcd-custom-image#40]
Update gardener-controlplane to 1.81.0
[gardener/etcd-druid]
⚠️ Breaking Changes
[USER]Update etcd-custom-image tov3.4.26-2. by @shreyas-s-rao [gardener/etcd-druid#656][OPERATOR]Etcd druid will now not supportpolicy/v1beta1forPodDisruptionBudgets and will only usepolicy/v1forPodDisruptionBudgets by @aaronfern [gardener/etcd-druid#681]
📰 Noteworthy
[OPERATOR]custodian-sync-periodvalue is set to15sin the Helm chart for etcd-druid. by @shreyas-s-rao [gardener/etcd-druid#688][OPERATOR]Add new flagmetrics-scrape-wait-durationfor compaction controller to set a wait duration at the end of every compaction job, to allow for metrics to be scraped by a Prometheus instance. by @abdasgupta [gardener/etcd-druid#686][OPERATOR]Etcd snapshot compaction jobs will now be named<etcd-name>-compactorfor better readability for human operators. by @abdasgupta [gardener/etcd-druid#672]
✨ New Features
[OPERATOR]IntroduceSpec.Backup.DeltaSnapshotRetentionPeriodin theEtcdresource to allow configuring retention period for delta snapshots. by @seshachalam-yv [gardener/etcd-druid#651][DEVELOPER]Add support forLocalprovider for e2e tests. by @shreyas-s-rao [gardener/etcd-druid#668]
🐛 Bug Fixes
[OPERATOR]A bug causing incorrect volume mount path forEtcds andEtcdCopyBackupsTasks usingLocalsnapshot storage provider while using distroless etcd-backup-restore imagev0.25.xhas been resolved. by @aaronfern [gardener/etcd-druid#662][OPERATOR]Custodian controller no longer watches leases owned by the etcd resources, thus reducing frequency of etcd status updates and now honouringcustodian-sync-periodvalue. by @shreyas-s-rao [gardener/etcd-druid#688][OPERATOR]Resolved an issue where the Custodian Controller was not updating theReplicasfield in theetcdstatus to reflect theCurrentReplicasfrom the StatefulSet status. This fix ensures consistent behavior with theetcdController in Druid. by @seshachalam-yv [gardener/etcd-druid#701][OPERATOR]A bug causingEtcdCopyBackupsTaskjobs to fail to create temp snapshot directory while using distroless etcd-backup-restore imagev0.25.xhas been resolved. by @aaronfern [gardener/etcd-druid#662]
🏃 Others
[OPERATOR]Upgradedetcd-backup-restorefromv0.24.3tov0.24.6foretcd-custom-image, and fromv0.25.1tov0.26.0foretcd-wrapperby @gardener-robot-ci-3 [gardener/etcd-druid#687][OPERATOR]All default images are now present inimages.yamlby @aaronfern [gardener/etcd-druid#673]
📖 Documentation
[DEVELOPER]Introduce DEPs (Druid Enhancement Proposals) for proposing large design changes in etcd-druid. by @shreyas-s-rao [gardener/etcd-druid#659][OPERATOR]Introduce DEP-04 EtcdMember Custom Resource. by @shreyas-s-rao [gardener/etcd-druid#658]
[gardener/etcd-backup-restore]
📰 Noteworthy
[USER]Introduce flagmetrics-scrape-wait-durationtoetcdbrctl compactcommand, that specifies a wait duration at the end of a snapshot compaction, to allow Prometheus to scrape metrics related to compaction before theetcdbrctlprocess exits. by @abdasgupta [gardener/etcd-backup-restore#667][OPERATOR]Etcd-backup-restore now uses the user home directory to create files. by @aaronfern [gardener/etcd-backup-restore#637][OPERATOR]Etcd-backup-restore now uses a distroless image as its base image. It is no longer compatible with etcd-custom-image, and must be used with etcd-wrapper instead. by @aaronfern [gardener/etcd-backup-restore#637]
🏃 Others
[OPERATOR]Upgraded Ginkgo v1 to v2 and updated other dependencies by @seshachalam-yv [gardener/etcd-backup-restore#647][OPERATOR]While scaling up a non-HA etcd cluster to HA skipping the scale-up checks for first member of etcd cluster as first member can never be a part of scale-up scenarios. by @ishan16696 [gardener/etcd-backup-restore#649][OPERATOR]Bump alpine base version for Docker build to3.18.2. by @shreyas-s-rao [gardener/etcd-backup-restore#638][OPERATOR]Backup-restore waits for its etcd to be ready before attempting to update peerUrl by @aaronfern [gardener/etcd-backup-restore#628][OPERATOR]Introduceddelta-snapshot-retention-periodCLI flag to extend the configurable retention period for delta snapshots inetcd-backup-restore, enhancing flexibility for backup retention. by @seshachalam-yv [gardener/etcd-backup-restore#640][OPERATOR]Revendors the bbolt fromv1.3.6tov1.3.7by @ishan16696 [gardener/etcd-backup-restore#659][DEVELOPER]Add CVE categorization for etcd-backup-restore. by @shreyas-s-rao [gardener/etcd-backup-restore#644]
[gardener/machine-controller-manager]
🐛 Bug Fixes
[OPERATOR]Force drain and delete volume attachments for nodes un-healthy due toReadOnlyFileSystemandNotReadyfor too long by @elankath [gardener/machine-controller-manager#839][OPERATOR]IncludedUnavailableReplicasin determining if a machine deployment status update is needed by @rishabh-11 [gardener/machine-controller-manager#833][OPERATOR]An issue causing nil pointer panic on scaleup of the machinedeployment along with trigger of rolling update, is fixed by @acumino [gardener/machine-controller-manager#814][USER]An edge case where outdated DesiredReplicas annotation blocked a rolling update is fixed. by @rishabh-11 [gardener/machine-controller-manager#821]
🏃 Others
[DEVELOPER]status.Status now captures underline cause, allowing consumers to introspect the error returned by the provider. WrapError() function could be used to wrap the provider error by @unmarshall [gardener/machine-controller-manager#842][DEVELOPER]A new make target is introduced to add license headers. by @unmarshall [gardener/machine-controller-manager#845][DEVELOPER]Bumpk8s.io/*deps to v0.27.2 by @afritzler [gardener/machine-controller-manager#820][DEVELOPER]Removed dead metrics code and refactored the remaining metrics code by @himanshu-kun [gardener/machine-controller-manager#823][OPERATOR]New metrics introduced:- api_request_duration_seconds -> tracks time taken for successful invocation of provider APIs. This metric can be filtered by provider and service.
- driver_request_duration_seconds -> tracks total time taken to successfully complete driver method invocation. This metric can be filtered by provider and operation.
- driver_requests_failed_total -> records total number of failed driver API requests. This metric can be filtered by provider, operations and error_code. by @unmarshall [gardener/machine-controller-manager#842]
[OPERATOR]Updated to go v1.20.5 by @rishabh-11 [gardener/machine-controller-manager#827][OPERATOR]Added a new metric that will allow to get the number of stale (due to unhealthiness) machines that are getting terminated by @jguipi [gardener/machine-controller-manager#808][OPERATOR]AddederrorCodefield in theLastOperationstruct. This should be implemented only for theCreateMachinecall in thetriggerCreationFlow. This field will be utilized by Cluster autoscaler to do early backoff by @rishabh-11 [gardener/machine-controller-manager#851][OPERATOR]Makefile targets have changed: Introduced gardener-setup, gardener-restore, gardener-local-mcm-up, non-gardener-setup, non-gardener-restore, non-gardener-local-mcm-up. Users can also directly use the scripts which are used by these makefile targets. by @unmarshall [gardener/machine-controller-manager#852]
[gardener/gardener]
⚠️ Breaking Changes
[DEPENDENCY]Extensions have to implement theForceDeletefunction in the actuator with the logic of forcefully deleting all the resources deployed by them. by @shafeeqes [#8414][DEPENDENCY]Theextensions/pkg/controller.Use{TokenRequestor,ServiceAccountTokenVolumeProjection}functions have been removed since they always returntrue. by @rfranzke [#8582][OPERATOR]⚠️ Gardener does no longer support garden, seed, or shoot clusters with Kubernetes versions < 1.24. Make sure to upgrade all existing clusters before upgrading to this Gardener version. by @shafeeqes [#8487][DEVELOPER]Thepkg/utils/gardener.IntStrPtrFromIntfunction has been renamed toIntStrPtrFromInt32sinceintstr.FromIntis deprecated. by @rfranzke [#8579][USER]Thealpha.kube-apiserver.scaling.shoot.gardener.cloud/classannotation onShoots has no effect anymore and should be removed. by @rfranzke [#8526]
📰 Noteworthy
[USER]The two additional labelsworker.gardener.cloud/image-nameandworker.gardener.cloud/image-versionthat were previously introduced and attached to worker nodes are removed again to fix a regression that causes thekubeletto restart on nodes that are due to be upgraded to a new OS but not rolled yet which causes theirPods to become temporarily unready. by @MrBatschner [#8524][OPERATOR]TheMachineControllerManagerDeploymenthas been promoted to beta and is now enabled by default. Make sure that all registered provider extensions support this feature gate before upgrading to this version of Gardener. by @rfranzke [#8526][OPERATOR]TheDisableScalingClassesForShootsfeature gates has been promoted to GA (and is now always enabled). by @rfranzke [#8526]
✨ New Features
[USER]Thegardener-schedulernow populates scheduling failure reasons to theShoot's.status.lastOperation.descriptionfield. by @rfranzke [#8527][USER]When theShootForceDeletionfeaturegate in the apiserver is turned on, users will be able to force-delete the Shoot. You MUST ensure that all the resources created in the IaaS account are cleaned up to prevent orphaned resources. Gardener will NOT delete any resources in the Shoot cloud-provider account. See Shoot Force Deletion for more details. by @shafeeqes [#8414][USER]Multiple expanders forcluster-autoscalercan now be specified in theShootAPI via the.spec.kubernetes.clusterAutoscaler.expanderfield. by @aaronfern [#8573]
🐛 Bug Fixes
[OPERATOR]Fixed a possibility for themigratephase of control plane migration to become permanently stuck if the shoot was created when theMachineControllerManagerDeploymentfeature gate is disabled, control plane migration is triggered for the shoot and the feature gate is enabled during the migration phase. by @plkokanov [#8568][USER]Fix an issue, where DNS lookups for non-existing pods of a StatefulSet yielded one of the existing pods even when it should not have. by @axel7born [#8544][USER]A bug has been fixed that prevented users without permissions to listCustomResourceDefinitions from interacting with the Gardener APIs when using akubectlversion lower than1.27. by @rfranzke [#8577][USER]A bug causing unnecessary reorder of extension inShootspec.extensionsis fixed. by @acumino [#8569]
🏃 Others
[OPERATOR]The shoot namespace in seeds is redeployed during the shoot migration flow to update the zones in use. by @plkokanov [#8564][OPERATOR]nginx-ingress-controllerimage is updated tov1.9.0. by @shafeeqes [#8558][OPERATOR]Add an alert for VPNHAShootNoPods when shoot in HA (high availability) mode. by @tedteng [#8506][USER]Gardener refined the scope of the problematic webhook matcher forendpointobjects. Earlier, shoot clusters were assigned a constraint reporting a problem with afailurePolocy: Failwebhook acting on these objects. Now, onlyendpoints in thekube-systemanddefaultsnamespaces are considered for this check. by @acumino [#8521]
[gardener/autoscaler]
✨ New Features
[DEVELOPER]unit tests framework introduced to test implemented methods ofCloudproviderandNodegroupinterface by @rishabh-11 [gardener/autoscaler#215][USER]Gardener autoscaler now backs-off early from a node-group (i.e. machinedeployment) in case ofResourceExhaustederror. Refer docs athttps://github.com/gardener/autoscaler/blob/machine-controller-manager-provider/cluster-autoscaler/FAQ.md#when-does-autoscaler-backs-off-early-from-a-node-groupfor details. by @himanshu-kun [gardener/autoscaler#253]
🐛 Bug Fixes
[OPERATOR]A bug where MCM removed a machine other than the one , CA wanted , is resolved. by @rishabh-11 [gardener/autoscaler#215]
🏃 Others
[OPERATOR]Initial implementation forRefresh()method ofCloudProviderinterface done by @rishabh-11 [gardener/autoscaler#215][OPERATOR]machinepriority.machine.sapcloud.ioannotation on machine is now reset to 3 by autoscaler if the corresponding node doesn't haveToBeDeletedByClusterAutoscalertaint by @rishabh-11 [gardener/autoscaler#215]
[gardener/etcd-custom-image]
📰 Noteworthy
[OPERATOR]Update alpine base image version to 3.18.3. by @shreyas-s-rao [gardener/etcd-custom-image#40]
Update gardenlet to 1.81.0
[gardener/etcd-druid]
⚠️ Breaking Changes
[USER]Update etcd-custom-image tov3.4.26-2. by @shreyas-s-rao [gardener/etcd-druid#656][OPERATOR]Etcd druid will now not supportpolicy/v1beta1forPodDisruptionBudgets and will only usepolicy/v1forPodDisruptionBudgets by @aaronfern [gardener/etcd-druid#681]
📰 Noteworthy
[OPERATOR]custodian-sync-periodvalue is set to15sin the Helm chart for etcd-druid. by @shreyas-s-rao [gardener/etcd-druid#688][OPERATOR]Add new flagmetrics-scrape-wait-durationfor compaction controller to set a wait duration at the end of every compaction job, to allow for metrics to be scraped by a Prometheus instance. by @abdasgupta [gardener/etcd-druid#686][OPERATOR]Etcd snapshot compaction jobs will now be named<etcd-name>-compactorfor better readability for human operators. by @abdasgupta [gardener/etcd-druid#672]
✨ New Features
[OPERATOR]IntroduceSpec.Backup.DeltaSnapshotRetentionPeriodin theEtcdresource to allow configuring retention period for delta snapshots. by @seshachalam-yv [gardener/etcd-druid#651][DEVELOPER]Add support forLocalprovider for e2e tests. by @shreyas-s-rao [gardener/etcd-druid#668]
🐛 Bug Fixes
[OPERATOR]A bug causing incorrect volume mount path forEtcds andEtcdCopyBackupsTasks usingLocalsnapshot storage provider while using distroless etcd-backup-restore imagev0.25.xhas been resolved. by @aaronfern [gardener/etcd-druid#662][OPERATOR]Custodian controller no longer watches leases owned by the etcd resources, thus reducing frequency of etcd status updates and now honouringcustodian-sync-periodvalue. by @shreyas-s-rao [gardener/etcd-druid#688][OPERATOR]Resolved an issue where the Custodian Controller was not updating theReplicasfield in theetcdstatus to reflect theCurrentReplicasfrom the StatefulSet status. This fix ensures consistent behavior with theetcdController in Druid. by @seshachalam-yv [gardener/etcd-druid#701][OPERATOR]A bug causingEtcdCopyBackupsTaskjobs to fail to create temp snapshot directory while using distroless etcd-backup-restore imagev0.25.xhas been resolved. by @aaronfern [gardener/etcd-druid#662]
🏃 Others
[OPERATOR]Upgradedetcd-backup-restorefromv0.24.3tov0.24.6foretcd-custom-image, and fromv0.25.1tov0.26.0foretcd-wrapperby @gardener-robot-ci-3 [gardener/etcd-druid#687][OPERATOR]All default images are now present inimages.yamlby @aaronfern [gardener/etcd-druid#673]
📖 Documentation
[DEVELOPER]Introduce DEPs (Druid Enhancement Proposals) for proposing large design changes in etcd-druid. by @shreyas-s-rao [gardener/etcd-druid#659][OPERATOR]Introduce DEP-04 EtcdMember Custom Resource. by @shreyas-s-rao [gardener/etcd-druid#658]
[gardener/etcd-backup-restore]
📰 Noteworthy
[USER]Introduce flagmetrics-scrape-wait-durationtoetcdbrctl compactcommand, that specifies a wait duration at the end of a snapshot compaction, to allow Prometheus to scrape metrics related to compaction before theetcdbrctlprocess exits. by @abdasgupta [gardener/etcd-backup-restore#667][OPERATOR]Etcd-backup-restore now uses the user home directory to create files. by @aaronfern [gardener/etcd-backup-restore#637][OPERATOR]Etcd-backup-restore now uses a distroless image as its base image. It is no longer compatible with etcd-custom-image, and must be used with etcd-wrapper instead. by @aaronfern [gardener/etcd-backup-restore#637]
🏃 Others
[OPERATOR]Upgraded Ginkgo v1 to v2 and updated other dependencies by @seshachalam-yv [gardener/etcd-backup-restore#647][OPERATOR]While scaling up a non-HA etcd cluster to HA skipping the scale-up checks for first member of etcd cluster as first member can never be a part of scale-up scenarios. by @ishan16696 [gardener/etcd-backup-restore#649][OPERATOR]Bump alpine base version for Docker build to3.18.2. by @shreyas-s-rao [gardener/etcd-backup-restore#638][OPERATOR]Backup-restore waits for its etcd to be ready before attempting to update peerUrl by @aaronfern [gardener/etcd-backup-restore#628][OPERATOR]Introduceddelta-snapshot-retention-periodCLI flag to extend the configurable retention period for delta snapshots inetcd-backup-restore, enhancing flexibility for backup retention. by @seshachalam-yv [gardener/etcd-backup-restore#640][OPERATOR]Revendors the bbolt fromv1.3.6tov1.3.7by @ishan16696 [gardener/etcd-backup-restore#659][DEVELOPER]Add CVE categorization for etcd-backup-restore. by @shreyas-s-rao [gardener/etcd-backup-restore#644]
[gardener/machine-controller-manager]
🐛 Bug Fixes
[OPERATOR]Force drain and delete volume attachments for nodes un-healthy due toReadOnlyFileSystemandNotReadyfor too long by @elankath [gardener/machine-controller-manager#839][OPERATOR]IncludedUnavailableReplicasin determining if a machine deployment status update is needed by @rishabh-11 [gardener/machine-controller-manager#833][OPERATOR]An issue causing nil pointer panic on scaleup of the machinedeployment along with trigger of rolling update, is fixed by @acumino [gardener/machine-controller-manager#814][USER]An edge case where outdated DesiredReplicas annotation blocked a rolling update is fixed. by @rishabh-11 [gardener/machine-controller-manager#821]
🏃 Others
[DEVELOPER]status.Status now captures underline cause, allowing consumers to introspect the error returned by the provider. WrapError() function could be used to wrap the provider error by @unmarshall [gardener/machine-controller-manager#842][DEVELOPER]A new make target is introduced to add license headers. by @unmarshall [gardener/machine-controller-manager#845][DEVELOPER]Bumpk8s.io/*deps to v0.27.2 by @afritzler [gardener/machine-controller-manager#820][DEVELOPER]Removed dead metrics code and refactored the remaining metrics code by @himanshu-kun [gardener/machine-controller-manager#823][OPERATOR]New metrics introduced:- api_request_duration_seconds -> tracks time taken for successful invocation of provider APIs. This metric can be filtered by provider and service.
- driver_request_duration_seconds -> tracks total time taken to successfully complete driver method invocation. This metric can be filtered by provider and operation.
- driver_requests_failed_total -> records total number of failed driver API requests. This metric can be filtered by provider, operations and error_code. by @unmarshall [gardener/machine-controller-manager#842]
[OPERATOR]Updated to go v1.20.5 by @rishabh-11 [gardener/machine-controller-manager#827][OPERATOR]Added a new metric that will allow to get the number of stale (due to unhealthiness) machines that are getting terminated by @jguipi [gardener/machine-controller-manager#808][OPERATOR]AddederrorCodefield in theLastOperationstruct. This should be implemented only for theCreateMachinecall in thetriggerCreationFlow. This field will be utilized by Cluster autoscaler to do early backoff by @rishabh-11 [gardener/machine-controller-manager#851][OPERATOR]Makefile targets have changed: Introduced gardener-setup, gardener-restore, gardener-local-mcm-up, non-gardener-setup, non-gardener-restore, non-gardener-local-mcm-up. Users can also directly use the scripts which are used by these makefile targets. by @unmarshall [gardener/machine-controller-manager#852]
[gardener/gardener]
⚠️ Breaking Changes
[DEPENDENCY]Extensions have to implement theForceDeletefunction in the actuator with the logic of forcefully deleting all the resources deployed by them. by @shafeeqes [#8414][DEPENDENCY]Theextensions/pkg/controller.Use{TokenRequestor,ServiceAccountTokenVolumeProjection}functions have been removed since they always returntrue. by @rfranzke [#8582][OPERATOR]⚠️ Gardener does no longer support garden, seed, or shoot clusters with Kubernetes versions < 1.24. Make sure to upgrade all existing clusters before upgrading to this Gardener version. by @shafeeqes [#8487][DEVELOPER]Thepkg/utils/gardener.IntStrPtrFromIntfunction has been renamed toIntStrPtrFromInt32sinceintstr.FromIntis deprecated. by @rfranzke [#8579][USER]Thealpha.kube-apiserver.scaling.shoot.gardener.cloud/classannotation onShoots has no effect anymore and should be removed. by @rfranzke [#8526]
📰 Noteworthy
[USER]The two additional labelsworker.gardener.cloud/image-nameandworker.gardener.cloud/image-versionthat were previously introduced and attached to worker nodes are removed again to fix a regression that causes thekubeletto restart on nodes that are due to be upgraded to a new OS but not rolled yet which causes theirPods to become temporarily unready. by @MrBatschner [#8524][OPERATOR]TheMachineControllerManagerDeploymenthas been promoted to beta and is now enabled by default. Make sure that all registered provider extensions support this feature gate before upgrading to this version of Gardener. by @rfranzke [#8526][OPERATOR]TheDisableScalingClassesForShootsfeature gates has been promoted to GA (and is now always enabled). by @rfranzke [#8526]
✨ New Features
[USER]Thegardener-schedulernow populates scheduling failure reasons to theShoot's.status.lastOperation.descriptionfield. by @rfranzke [#8527][USER]When theShootForceDeletionfeaturegate in the apiserver is turned on, users will be able to force-delete the Shoot. You MUST ensure that all the resources created in the IaaS account are cleaned up to prevent orphaned resources. Gardener will NOT delete any resources in the Shoot cloud-provider account. See Shoot Force Deletion for more details. by @shafeeqes [#8414][USER]Multiple expanders forcluster-autoscalercan now be specified in theShootAPI via the.spec.kubernetes.clusterAutoscaler.expanderfield. by @aaronfern [#8573]
🐛 Bug Fixes
[OPERATOR]Fixed a possibility for themigratephase of control plane migration to become permanently stuck if the shoot was created when theMachineControllerManagerDeploymentfeature gate is disabled, control plane migration is triggered for the shoot and the feature gate is enabled during the migration phase. by @plkokanov [#8568][USER]Fix an issue, where DNS lookups for non-existing pods of a StatefulSet yielded one of the existing pods even when it should not have. by @axel7born [#8544][USER]A bug has been fixed that prevented users without permissions to listCustomResourceDefinitions from interacting with the Gardener APIs when using akubectlversion lower than1.27. by @rfranzke [#8577][USER]A bug causing unnecessary reorder of extension inShootspec.extensionsis fixed. by @acumino [#8569]
🏃 Others
[OPERATOR]The shoot namespace in seeds is redeployed during the shoot migration flow to update the zones in use. by @plkokanov [#8564][OPERATOR]nginx-ingress-controllerimage is updated tov1.9.0. by @shafeeqes [#8558][OPERATOR]Add an alert for VPNHAShootNoPods when shoot in HA (high availability) mode. by @tedteng [#8506][USER]Gardener refined the scope of the problematic webhook matcher forendpointobjects. Earlier, shoot clusters were assigned a constraint reporting a problem with afailurePolocy: Failwebhook acting on these objects. Now, onlyendpoints in thekube-systemanddefaultsnamespaces are considered for this check. by @acumino [#8521]
[gardener/autoscaler]
✨ New Features
[DEVELOPER]unit tests framework introduced to test implemented methods ofCloudproviderandNodegroupinterface by @rishabh-11 [gardener/autoscaler#215][USER]Gardener autoscaler now backs-off early from a node-group (i.e. machinedeployment) in case ofResourceExhaustederror. Refer docs athttps://github.com/gardener/autoscaler/blob/machine-controller-manager-provider/cluster-autoscaler/FAQ.md#when-does-autoscaler-backs-off-early-from-a-node-groupfor details. by @himanshu-kun [gardener/autoscaler#253]
🐛 Bug Fixes
[OPERATOR]A bug where MCM removed a machine other than the one , CA wanted , is resolved. by @rishabh-11 [gardener/autoscaler#215]
🏃 Others
[OPERATOR]Initial implementation forRefresh()method ofCloudProviderinterface done by @rishabh-11 [gardener/autoscaler#215][OPERATOR]machinepriority.machine.sapcloud.ioannotation on machine is now reset to 3 by autoscaler if the corresponding node doesn't haveToBeDeletedByClusterAutoscalertaint by @rishabh-11 [gardener/autoscaler#215]
[gardener/etcd-custom-image]
📰 Noteworthy
[OPERATOR]Update alpine base image version to 3.18.3. by @shreyas-s-rao [gardener/etcd-custom-image#40]
Update dashboard to 1.70.0
[gardener/dashboard]
⚠️ Breaking Changes
[OPERATOR]Tickets are no longer automatically closed by the dashboard when a shoot is deleted. This should be handled by a different component instead, like a robot. by @petersutter [#1517][OPERATOR]Thegardener-dashboardDeploymentnow runs with high availability config (with labelhigh-availability-config.resources.gardener.cloud/type=server). For more information about the HA config see resource-manager.md#high-availability-config by @petersutter [#1504][OPERATOR]Terminals: TheconnectSrcdirective of the Content Security Policy no longer permits the use ofwss:by default. You need to set up the allowed hosts that the browser can connect to for the web terminal feature. This can be done usingValues.global.terminal.allowedHostSourceList. For more details see webterminals.md#allowlist-for-hosts by @petersutter [#1561][OPERATOR]Terminals: As gardener took over the responsibility of providing kube-apiserver endpoints with trusted certificates theterminal-bootstrapcomponent was dropped. Make sure to clean up the created resources of the terminal bootstrapper. See PR description for more details. by @petersutter [#1561]
✨ New Features
[USER]Added an input field to configure the maintenance time window duration. The textfield hints now show the UTC begin and end time by @grolu [#1587][USER]Enable focus mode for regular project lists. This feature used to be exclusive for operators. It can now be used by regular users on all cluster lists. This feature enables users to get a static overview of clusters with issues. In order to use this feature, you need to enable it on theSettingspage. by @grolu [#1557][USER]Implementation of lazy loading for copy to clipboard functionality by @holgerkoser [#1546][USER]Added functionality to create, view and manage workerless clusters by @grolu [#1531][OPERATOR]Allow customization of dashboard branding like product logo, name, title, slogan and custom html templates for teaser and footer. The documentation and a detailed example can be found in the dashboard operation guidelines https://github.com/gardener/dashboard/blob/master/docs/operations/customization.md by @holgerkoser [#1568][OPERATOR]Terminal: By default, the access service account within the garden terminal pod is bound togardener.cloud:system:administrators, notcluster-adminanymore. by @petersutter [#1541][OPERATOR]Github app authentication is now supported (in addition to token authentication) for the ticket feature by @petersutter [#1514][OPERATOR]Terminal: You can now configure the role bindings to which the access service account within thegardenterminal pod is bound (Values.global.terminal.garden.roleBindings). by @petersutter [#1541][OPERATOR]Thedashboardchart allows to optionally configure a projected volume based kubeconfig by @timuthy [#1598][OPERATOR]In addition to thedefaultNodesCIDRconfig (Values.global.dashboard.frontendConfig.defaultNodesCIDR) of thegardener-dashboardwhich applies for all new Shoots, you can now have a configuration per cloud profile, by setting.spec.providerConfig.defaultNodesCIDRon the respectiveCloudProfileby @petersutter [#1591][DEVELOPER]Migrated frontend code to Vue 3 and Vuetify 3 UI components. Vue 2 will reach End of Life (EOL) on December 31st, 2023 by @holgerkoser [#1510]
🐛 Bug Fixes
[USER]Fixed code completion and tooltips in cluster editor: Recent Gardener releases dropped support for OpenAPI v2. Dashboard now uses OpenAPI v3 to fetch shoot resource information by @grolu [#1600][USER]Users with permission tolistallprojectscan see them now in the dashboard. Previously the permission togetsecretsacross all namespaces was required. by @petersutter [#1518][USER]Fixed hibernation schedule time input for Safari browser by @grolu [#1536][USER]Fixed an issue where the filter on theSecrets,MembersandClusterspage was not reset when switching the Project by @petersutter [#1529]
🏃 Others
[OPERATOR]The default grant types for the garden cluster OIDC kubeconfig have changed toauto,authcodeanddevice-code.authcode-keyboardwas removed anddevice-codegrant type was added. The default grant types can be overridden by settingValues.global.dashboard.frontendConfig.grantTypesin thegardener-dashboardhelm chart by @petersutter [#1512][OPERATOR]An error will now be logged in case the OIDC issuer discovery fails by @petersutter [#1562]
Update dashboard to 1.70.0
[gardener/dashboard]
⚠️ Breaking Changes
[OPERATOR]Tickets are no longer automatically closed by the dashboard when a shoot is deleted. This should be handled by a different component instead, like a robot. by @petersutter [#1517][OPERATOR]Thegardener-dashboardDeploymentnow runs with high availability config (with labelhigh-availability-config.resources.gardener.cloud/type=server). For more information about the HA config see resource-manager.md#high-availability-config by @petersutter [#1504][OPERATOR]Terminals: TheconnectSrcdirective of the Content Security Policy no longer permits the use ofwss:by default. You need to set up the allowed hosts that the browser can connect to for the web terminal feature. This can be done usingValues.global.terminal.allowedHostSourceList. For more details see webterminals.md#allowlist-for-hosts by @petersutter [#1561][OPERATOR]Terminals: As gardener took over the responsibility of providing kube-apiserver endpoints with trusted certificates theterminal-bootstrapcomponent was dropped. Make sure to clean up the created resources of the terminal bootstrapper. See PR description for more details. by @petersutter [#1561]
✨ New Features
[USER]Added an input field to configure the maintenance time window duration. The textfield hints now show the UTC begin and end time by @grolu [#1587][USER]Enable focus mode for regular project lists. This feature used to be exclusive for operators. It can now be used by regular users on all cluster lists. This feature enables users to get a static overview of clusters with issues. In order to use this feature, you need to enable it on theSettingspage. by @grolu [#1557][USER]Implementation of lazy loading for copy to clipboard functionality by @holgerkoser [#1546][USER]Added functionality to create, view and manage workerless clusters by @grolu [#1531][OPERATOR]Allow customization of dashboard branding like product logo, name, title, slogan and custom html templates for teaser and footer. The documentation and a detailed example can be found in the dashboard operation guidelines https://github.com/gardener/dashboard/blob/master/docs/operations/customization.md by @holgerkoser [#1568][OPERATOR]Terminal: By default, the access service account within the garden terminal pod is bound togardener.cloud:system:administrators, notcluster-adminanymore. by @petersutter [#1541][OPERATOR]Github app authentication is now supported (in addition to token authentication) for the ticket feature by @petersutter [#1514][OPERATOR]Terminal: You can now configure the role bindings to which the access service account within thegardenterminal pod is bound (Values.global.terminal.garden.roleBindings). by @petersutter [#1541][OPERATOR]Thedashboardchart allows to optionally configure a projected volume based kubeconfig by @timuthy [#1598][OPERATOR]In addition to thedefaultNodesCIDRconfig (Values.global.dashboard.frontendConfig.defaultNodesCIDR) of thegardener-dashboardwhich applies for all new Shoots, you can now have a configuration per cloud profile, by setting.spec.providerConfig.defaultNodesCIDRon the respectiveCloudProfileby @petersutter [#1591][DEVELOPER]Migrated frontend code to Vue 3 and Vuetify 3 UI components. Vue 2 will reach End of Life (EOL) on December 31st, 2023 by @holgerkoser [#1510]
🐛 Bug Fixes
[USER]Fixed code completion and tooltips in cluster editor: Recent Gardener releases dropped support for OpenAPI v2. Dashboard now uses OpenAPI v3 to fetch shoot resource information by @grolu [#1600][USER]Users with permission tolistallprojectscan see them now in the dashboard. Previously the permission togetsecretsacross all namespaces was required. by @petersutter [#1518][USER]Fixed hibernation schedule time input for Safari browser by @grolu [#1536][USER]Fixed an issue where the filter on theSecrets,MembersandClusterspage was not reset when switching the Project by @petersutter [#1529]
🏃 Others
[OPERATOR]The default grant types for the garden cluster OIDC kubeconfig have changed toauto,authcodeanddevice-code.authcode-keyboardwas removed anddevice-codegrant type was added. The default grant types can be overridden by settingValues.global.dashboard.frontendConfig.grantTypesin thegardener-dashboardhelm chart by @petersutter [#1512][OPERATOR]An error will now be logged in case the OIDC issuer discovery fails by @petersutter [#1562]