<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
    <id>https://yakecloud.github.io/yake/blog</id>
    <title>Yet another Kubernetes Engine Blog</title>
    <updated>2023-06-06T00:00:00.000Z</updated>
    <generator>https://github.com/jpmonette/feed</generator>
    <link rel="alternate" href="https://yakecloud.github.io/yake/blog"/>
    <subtitle>Yet another Kubernetes Engine Blog</subtitle>
    <icon>https://yakecloud.github.io/yake/img/favicon.ico</icon>
    <entry>
        <title type="html"><![CDATA[Gardener Hackathon - May 2023]]></title>
        <id>https://yakecloud.github.io/yake/blog/Gardener Hackathon 2023</id>
        <link href="https://yakecloud.github.io/yake/blog/Gardener Hackathon 2023"/>
        <updated>2023-06-06T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[TLDR;]]></summary>
        <content type="html"><![CDATA[<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="tldr">TLDR;<a href="https://yakecloud.github.io/yake/blog/Gardener%20Hackathon%202023#tldr" class="hash-link" aria-label="Direct link to TLDR;" title="Direct link to TLDR;" translate="no">​</a></h2>
<p>From May 22th until 26th, the colleagues from SAP, StackIT, x-cellent, and 23 Technologies met for another Gardener hackathon. One output is another <a href="https://github.com/gardener-community/hackathon" target="_blank" rel="noopener noreferrer" class="">repository</a> collecting the hackathon outputs. Go ahead and checkout the repo, for a concise summary of all past hackathons and information on future hackathons.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="another-great-experience-with-great-achievements">Another great experience with great achievements<a href="https://yakecloud.github.io/yake/blog/Gardener%20Hackathon%202023#another-great-experience-with-great-achievements" class="hash-link" aria-label="Direct link to Another great experience with great achievements" title="Direct link to Another great experience with great achievements" translate="no">​</a></h2>
<p>When we met on Monday 22th, we synchronized our expectations for the week in the first place. Almost everyone agreed that having a good time together belongs to the main expectations. A few days after the hackathon, I can definitely state that we had a good and productive time together. From the social perspective, we enjoyed the fruitful discussions during lunch or dinner. From the hacking perspective, we were really fascinated by the progress made with respect to some topics which have made it to the agenda for several times before:</p>
<ul>
<li class="">Supporting pure IPv6 shoot clusters and</li>
<li class="">Replacing the bash scripts for node provisioning with a golang-based approach.</li>
</ul>
<p>Moreover, we were working on a more research oriented topic dealing with the deployment of "masterful clusters" aka. "autonomous shoots". Even though the final concept for "gardener-like initial clusters" was not developed during the hackathon, the collected experience with respect to this challenging task is crucial for further steps.
Besides the bigger topics mentioned above, we brought the following task close to (or even in) production:</p>
<ul>
<li class="">We moved the <code>machine-controller-manager</code> deployment responsibility to the <code>gardenlet</code></li>
<li class="">We introduced an <code>InternalSecret</code> resource in the Gardener API</li>
<li class="">We replaced the <code>ShootState</code>s With data in backup buckets</li>
<li class="">We found a concept for Garden cluster acccess for extensions in Seed clusters.</li>
</ul>
<p>Of course, there are still open questions and not every issue was solved during this short week. Therefore, we are happy that the colleagues from x-cellent opted for organizing the next Gardener hackathon in November/December 2023.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="conclusion">Conclusion<a href="https://yakecloud.github.io/yake/blog/Gardener%20Hackathon%202023#conclusion" class="hash-link" aria-label="Direct link to Conclusion" title="Direct link to Conclusion" translate="no">​</a></h2>
<p>Once again, the Gardener hackathon was a great experience with great achievements for the overall project. The community work towards a "managed Kubernetes done right" service is still gathering pace which is forms a great basis for all future development.</p>]]></content>
        <author>
            <name>Jens Schneider</name>
            <uri>https://github.com/jensac</uri>
        </author>
        <category label="gardener" term="gardener"/>
        <category label="hackathon" term="hackathon"/>
    </entry>
    <entry>
        <title type="html"><![CDATA[Build Cluster with CAPH - the challanges when installing 23KE]]></title>
        <id>https://yakecloud.github.io/yake/blog/build-cluster-with-caph</id>
        <link href="https://yakecloud.github.io/yake/blog/build-cluster-with-caph"/>
        <updated>2023-05-12T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[TLDR;]]></summary>
        <content type="html"><![CDATA[<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="tldr">TLDR;<a href="https://yakecloud.github.io/yake/blog/build-cluster-with-caph#tldr" class="hash-link" aria-label="Direct link to TLDR;" title="Direct link to TLDR;" translate="no">​</a></h2>
<p>We recently built new Kubernetes clusters on Hetzner Cloud. We had several challenges to get the cluster up and running.</p>
<p>This started with the selection of the correct Kubernetes version, the CNI solution and the actual deployment of 23KE. Spoiler: We had to change the CNI solution and reset containerd.</p>
<p>If these instructions in this blog post are followed, you can build a working Gardener cluster.</p>
<p><strong>Table of Contents</strong></p>
<ul>
<li class=""><a href="https://yakecloud.github.io/yake/blog/build-cluster-with-caph#tldr" class="">TLDR;</a></li>
<li class=""><a href="https://yakecloud.github.io/yake/blog/build-cluster-with-caph#introduction" class="">Introduction</a></li>
<li class=""><a href="https://yakecloud.github.io/yake/blog/build-cluster-with-caph#requirements" class="">Requirements</a></li>
<li class=""><a href="https://yakecloud.github.io/yake/blog/build-cluster-with-caph#setup-the-management-cluster" class="">Setup the management cluster</a>
<ul>
<li class=""><a href="https://yakecloud.github.io/yake/blog/build-cluster-with-caph#modifications" class="">Modifications</a></li>
</ul>
</li>
<li class=""><a href="https://yakecloud.github.io/yake/blog/build-cluster-with-caph#setup-the-worker-cluster" class="">Setup the worker cluster</a></li>
<li class=""><a href="https://yakecloud.github.io/yake/blog/build-cluster-with-caph#install-of-23ke" class="">Install of 23KE</a></li>
<li class=""><a href="https://yakecloud.github.io/yake/blog/build-cluster-with-caph#summary" class="">Summary</a></li>
</ul>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="introduction">Introduction<a href="https://yakecloud.github.io/yake/blog/build-cluster-with-caph#introduction" class="hash-link" aria-label="Direct link to Introduction" title="Direct link to Introduction" translate="no">​</a></h2>
<p>In times of rising costs and the reduction of the CO<sub>2</sub> footprint in all areas of life, we also wanted to reduce these sustainably in our day-to-day operations.</p>
<p>We had been running okeanos.dev on a managed kubernetes cluster on Azure. This is very expensive so we wanted to minimize these costs. In this case, the European cloud from Hetzner was the obvious choice. The following question was how we could best build a k8s cluster there.</p>
<p>After some research, ClusterAPI provider for Hetzner (CAPH) from <a href="https://syself.com/" target="_blank" rel="noopener noreferrer" class="">Syself</a> turned out to be the optimal solution.</p>
<p>When testing the provider, we had to overcome a few challenges, which we would like to discuss in the following.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="requirements">Requirements<a href="https://yakecloud.github.io/yake/blog/build-cluster-with-caph#requirements" class="hash-link" aria-label="Direct link to Requirements" title="Direct link to Requirements" translate="no">​</a></h2>
<p>You need to install some basic tools to work with CAPH and Gardener. It makes sense to set up a management vm (on Hetzner) running a kind cluster and on it the management cluster.</p>
<ul>
<li class=""><a href="https://docs.docker.com/engine/install/" target="_blank" rel="noopener noreferrer" class="">docker-engine</a></li>
<li class=""><a href="https://kind.sigs.k8s.io/docs/user/quick-start/#installing-from-release-binaries" target="_blank" rel="noopener noreferrer" class="">kind</a></li>
<li class=""><a href="https://cluster-api.sigs.k8s.io/user/quick-start.html#install-clusterctl" target="_blank" rel="noopener noreferrer" class="">clusterctl</a></li>
<li class=""><a href="https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/" target="_blank" rel="noopener noreferrer" class="">kubectl</a></li>
<li class=""><a href="https://helm.sh/docs/intro/install/" target="_blank" rel="noopener noreferrer" class="">helm</a></li>
<li class=""><a href="https://fluxcd.io/flux/installation/" target="_blank" rel="noopener noreferrer" class="">flux</a></li>
<li class=""><a href="https://k9scli.io/topics/install/" target="_blank" rel="noopener noreferrer" class="">k9s</a></li>
</ul>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="setup-the-management-cluster">Setup the management cluster<a href="https://yakecloud.github.io/yake/blog/build-cluster-with-caph#setup-the-management-cluster" class="hash-link" aria-label="Direct link to Setup the management cluster" title="Direct link to Setup the management cluster" translate="no">​</a></h2>
<p>To start, let's create a kind cluster with a customized Kubernetes version (Currently it is important that the kubernetes version is <code>&gt;1.26</code>)</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">kind create cluster -n my-cluster --image=kindest/node:v1.25.9</span><br></div></code></pre></div></div>
<p>When the cluster is up, <a href="https://github.com/syself/cluster-api-provider-hetzner/blob/main/docs/topics/preparation.md" target="_blank" rel="noopener noreferrer" class="">initialize the management cluster</a> with CAPH</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">clusterctl init --core cluster-api --bootstrap kubeadm --control-plane kubeadm --infrastructure hetzner</span><br></div></code></pre></div></div>
<p>and export your environment variables</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">export HCLOUD_SSH_KEY="MY_SSH_KEY" \</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">export CLUSTER_NAME="my-cluster" \</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">export HCLOUD_REGION="nbg1" \</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">export CONTROL_PLANE_MACHINE_COUNT=3 \</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">export WORKER_MACHINE_COUNT=3 \</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">export KUBERNETES_VERSION=1.25.9 \</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">export HCLOUD_CONTROL_PLANE_MACHINE_TYPE=cpx31 \</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">export HCLOUD_WORKER_MACHINE_TYPE=cpx41 \</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">export HCLOUD_TOKEN="YOUR_HCLOUD_TOKEN_HERE"</span><br></div></code></pre></div></div>
<p>The region can be for sure anything else like <code>hel1</code>.</p>
<p>To be able to build the machines a secret must be created:</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">kubectl create secret generic hetzner --from-literal=hcloud=$HCLOUD_TOKEN</span><br></div></code></pre></div></div>
<p>You can also still build yourself a customized node image, but we didn't do that. We used the Ubuntu 22.04 image, which is available from Hetzner.</p>
<p>Now let's create the <code>my-cluster.yaml</code> with the private network flavor as in the <a href="https://github.com/syself/cluster-api-provider-hetzner/blob/main/docs/topics/quickstart.md" target="_blank" rel="noopener noreferrer" class="">quickstart-guide</a>:</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">clusterctl generate cluster my-cluster --kubernetes-version v1.25.9 --control-plane-machine-count=3 --worker-machine-count=3 --flavor hcloud-network &gt; my-cluster.yaml</span><br></div></code></pre></div></div>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="modifications">Modifications<a href="https://yakecloud.github.io/yake/blog/build-cluster-with-caph#modifications" class="hash-link" aria-label="Direct link to Modifications" title="Direct link to Modifications" translate="no">​</a></h3>
<p>After creation you need to modify the <code>my-cluster.yaml</code>. Remove the following blocks (it gives two of them) because we need to reset the containerd config</p>
<div class="language-diff codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-diff codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">- - content: |</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">-     version = 2</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">-     [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">-       runtime_type = "io.containerd.runc.v2"</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">-     [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">-       SystemdCgroup = true</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">-     [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.crun]</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">-       runtime_type = "io.containerd.runc.v2"</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">-     [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.crun.options]</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">-       BinaryName = "crun"</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">-       Root = "/usr/local/sbin"</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">-       SystemdCgroup = true</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">-     [plugins."io.containerd.grpc.v1.cri".containerd]</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">-       default_runtime_name = "crun"</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">-     [plugins."io.containerd.runtime.v1.linux"]</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">-       runtime = "crun"</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">-       runtime_root = "/usr/local/sbin"</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">-   owner: root:root</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">-   path: /etc/containerd/config.toml</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">-   permissions: "0744"</span><br></div></code></pre></div></div>
<p>and add</p>
<div class="language-diff codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-diff codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">+ - mkdir /etc containerd</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">+ - containerd config default &gt; /etc/containerd/confi.toml</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">+ - sed -i 's/SystemdCgroup = false/SystemdCgroup = true/g' /etc/containerd/config.toml</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">  - systemctl daemon-reload &amp;&amp; systemctl enable containerd &amp;&amp; systemctl start containerd</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">+ - sysctl fs.inotify.max_user_instances=8192</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">+ - sysctl fs.inotify.max_user_watches=524288</span><br></div></code></pre></div></div>
<p>The containerd-config is missing some options. In addition, the <code>SystemdCgroup</code> must be set to <code>true</code> and the <code>inotify</code> settings needs to be increased. Only then the <code>vpn-seed-server</code> starts, which is created when a shoot is created.</p>
<p>When you have finished the modification, you can start building the worker cluster by appling the modified file on the management cluster</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">kubectl apply -f my-cluster.yaml</span><br></div></code></pre></div></div>
<p>With <code>watch clusterctl describe cluster my-cluster</code> you can watch one control-plane and three workers are being built.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="setup-the-worker-cluster">Setup the worker cluster<a href="https://yakecloud.github.io/yake/blog/build-cluster-with-caph#setup-the-worker-cluster" class="hash-link" aria-label="Direct link to Setup the worker cluster" title="Direct link to Setup the worker cluster" translate="no">​</a></h2>
<blockquote>
<p><strong><em>NOTE:</em></strong> From now on you should make sure that you are working on the newly built cluster. Make sure that you have exported the Kubeconfig for the worker cluster!</p>
</blockquote>
<p>Get the kubeconfig of the worker cluster</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">clusterctl get kubeconfig my-cluster &gt; /path/to/my/worker-cluster.kc</span><br></div></code></pre></div></div>
<p>and export it to work on the cluster (you can do this on your local machine. You don't need to work anymore on the management vm):</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">export KUBECONFIG=/path/to/my/worker-cluster.kc</span><br></div></code></pre></div></div>
<p>At the point of "Deploy a CNI solution" keep in mind not to install cilium like in the guide but <a href="https://docs.tigera.io/calico/latest/getting-started/kubernetes/helm" target="_blank" rel="noopener noreferrer" class="">calico</a>. We had a lot of trouble with cilium.</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">helm repo add projectcalico https://docs.tigera.io/calico/charts</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">kubectl create namespace tigera-operator</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">helm install calico projectcalico/tigera-operator --version v3.25.1 --namespace tigera-operator</span><br></div></code></pre></div></div>
<p>Wait till calico is ready.</p>
<p>Now deploy the CCM for hcloud only. Set <code>privateNetwork.enabled=true</code> because we need a private network to function properly:</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">helm repo add syself https://charts.syself.com</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">helm upgrade --install ccm syself/ccm-hcloud --version 1.0.11 \</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">  --namespace kube-system \</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">  --set secret.name=hetzner \</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">  --set secret.tokenKeyName=hcloud \</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">  --set privateNetwork.enabled=true</span><br></div></code></pre></div></div>
<p>At the end we need a CSI to build volumes on hcloud:</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">cat &lt;&lt; EOF &gt; csi-values.yaml</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">storageClasses:</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">  - name: hcloud-volumes</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    defaultStorageClass: true</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    reclaimPolicy: Retain</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">EOF</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">helm upgrade --install csi syself/csi-hcloud --version 0.2.0 \</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">  --namespace kube-system -f csi-values.yaml</span><br></div></code></pre></div></div>
<p>The remaining control planes should now be built and added to the cluster. Wait until the entire cluster has <code>Ready</code> status before proceeding.</p>
<blockquote>
<p><strong><em>NOTE:</em></strong> If you want to transform the workload cluster into a management cluster, you need to do the "move"-steps in the guide. But its recommended to have a management vm with kind installed and hold there the management cluster.</p>
</blockquote>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="install-of-23ke">Install of 23KE<a href="https://yakecloud.github.io/yake/blog/build-cluster-with-caph#install-of-23ke" class="hash-link" aria-label="Direct link to Install of 23KE" title="Direct link to Install of 23KE" translate="no">​</a></h2>
<p>If you plan to install 23KE, you need to keep in mind that there are a few customizations that need to be made for hcloud.</p>
<p>If you have installed 23KE and the data from the installation is in a repository, the following file need to be adjusted.</p>
<p>Add the following lines in <code>gardenlet-values.yaml</code></p>
<div class="language-diff codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-diff codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">settings:</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">+ loadBalancerServices:</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">+   annotations:</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">+     load-balancer.hetzner.cloud/location: nbg1</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">+     load-balancer.hetzner.cloud/ipv6-disabled: "true"</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">+     load-balancer.hetzner.cloud/disable-private-ingress: "true"</span><br></div></code></pre></div></div>
<p>Commit and push your changes. You can execute a <code>flux reconcile source git 23ke-config</code> to speed up the things.</p>
<p>If any ingress won't get a public IP, you can add some annotations manualy e.g. for <code>nginx-ingress-controller</code>:</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">kubectl annotate svc -n garden nginx-ingress-controller load-balancer.hetzner.cloud/location=nbg1 \</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">  load-balancer.hetzner.cloud/ipv6-disabled=true \</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">  load-balancer.hetzner.cloud/disable-private-ingress=true</span><br></div></code></pre></div></div>
<p>Now all things should come up as well as e.g. the dashboard should be accessible. If something is not running smoothly, the cluster can be inspected with k9s and you have an easy overview of what the problem is.</p>
<p>In the next steps, secrets can be added to connect to a public or private cloud. With these and a cloudprofile it is then possible to build and run shoots (k8s clusters).</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="summary">Summary<a href="https://yakecloud.github.io/yake/blog/build-cluster-with-caph#summary" class="hash-link" aria-label="Direct link to Summary" title="Direct link to Summary" translate="no">​</a></h2>
<p>With all these steps, it is possible for you to build a functional Gardener cluster on Hetzner Cloud. With this you can run a low cost K8s cluster and run what you want on it. Whereas 23KE is already very cool.</p>
<p>This setup has allowed us to drastically reduce the cost of a fully functional Gardener. As a pleasant side effect, we are no longer dependent on a cloud in the US, but now run Gardener in a data center in Germany with a German operator that is GDPR compliant and uses green electricity (good for our CO<sub>2</sub> footprint).</p>]]></content>
        <author>
            <name>Marius Wernicke</name>
            <uri>https://github.com/rhizoet</uri>
        </author>
        <category label="gardener" term="gardener"/>
        <category label="caph" term="caph"/>
        <category label="k8s" term="k8s"/>
    </entry>
    <entry>
        <title type="html"><![CDATA[Hack-The-Garden - the Remote Hackathon Experience]]></title>
        <id>https://yakecloud.github.io/yake/blog/hack-the-garden</id>
        <link href="https://yakecloud.github.io/yake/blog/hack-the-garden"/>
        <updated>2022-10-06T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[Thanks to everyone]]></summary>
        <content type="html"><![CDATA[<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="thanks-to-everyone">Thanks to everyone<a href="https://yakecloud.github.io/yake/blog/hack-the-garden#thanks-to-everyone" class="hash-link" aria-label="Direct link to Thanks to everyone" title="Direct link to Thanks to everyone" translate="no">​</a></h2>
<p>In the last week of September 2022, I participated in the <a href="https://metal-stack.io/blog/2022/10/hack-the-garden/" target="_blank" rel="noopener noreferrer" class="">Gardener Hackathon</a>. Unfortunately, I need to attend the event remotely, as I caught Covid the week before the event. Therefore, I want to thank everyone making this experience possible for me, anyway.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="even-remote-hacking-together-is-great">Even remote hacking together is great<a href="https://yakecloud.github.io/yake/blog/hack-the-garden#even-remote-hacking-together-is-great" class="hash-link" aria-label="Direct link to Even remote hacking together is great" title="Direct link to Even remote hacking together is great" translate="no">​</a></h2>
<p>As <a href="https://github.com/timebertt/timebertt" target="_blank" rel="noopener noreferrer" class="">Tim</a> also participated remotely, we formed a remote hacking team contributing to the development of the Gardener <a href="https://github.com/gardener/gardener-extension-registry-cache" target="_blank" rel="noopener noreferrer" class="">extension-registry-cache</a>. Of course, we were in touch with the on-site contributors via video calls which laid the foundation for three highly productive hacking days. Right after tying down a common todo list, we distributed the workload to the on-site and remote teams, and started hacking. From time to time, we held synchronization meetings, so that everyone was up-to-date and the current state of work was not only reflected in commits, branches, and pull request but also communicated to everyone in the team.</p>
<p>Beyond the internal communication within the team, Tim and <a href="https://github.com/rfranzke" target="_blank" rel="noopener noreferrer" class="">Rafael</a> organized a demo session on day 2. It was really amazing to see the progress made with respect to the various topics covered by the hackathon.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="conclusion">Conclusion<a href="https://yakecloud.github.io/yake/blog/hack-the-garden#conclusion" class="hash-link" aria-label="Direct link to Conclusion" title="Direct link to Conclusion" translate="no">​</a></h2>
<p>Clearly, the hackathon was a great event, and even hacking together remotely was a great experience. Of course, the social aspect of working together on-site cannot be mimicked. Therefore, I am already looking forward to the next Gardener hackathon which I hopefully can attend on-site.</p>]]></content>
        <author>
            <name>Jens Schneider</name>
            <uri>https://github.com/jensac</uri>
        </author>
        <category label="gardener" term="gardener"/>
        <category label="gardener-extensions" term="gardener-extensions"/>
    </entry>
    <entry>
        <title type="html"><![CDATA[A Single Point of Truth for Gardener Helm Charts]]></title>
        <id>https://yakecloud.github.io/yake/blog/gardener-chart-releaser</id>
        <link href="https://yakecloud.github.io/yake/blog/gardener-chart-releaser"/>
        <updated>2022-09-19T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[TLDR;]]></summary>
        <content type="html"><![CDATA[<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="tldr">TLDR;<a href="https://yakecloud.github.io/yake/blog/gardener-chart-releaser#tldr" class="hash-link" aria-label="Direct link to TLDR;" title="Direct link to TLDR;" translate="no">​</a></h2>
<p>Recently, we consolidated Gardener related Helm charts in a helm repository hosted on GitHub pages of the <a href="https://github.com/gardener-community/gardener-charts" target="_blank" rel="noopener noreferrer" class="">gardener-charts</a> repository.
For this purpose we implemented a custom chart release bot - the <a href="https://github.com/gardener-community/gardener-chart-releaser" target="_blank" rel="noopener noreferrer" class="">gardener-chart-releaser</a>.
Keep on reading to find out more.</p>
<p><strong>Table of Contents</strong></p>
<ul>
<li class=""><a href="https://yakecloud.github.io/yake/blog/gardener-chart-releaser#tldr" class="">TLDR;</a></li>
<li class=""><a href="https://yakecloud.github.io/yake/blog/gardener-chart-releaser#introduction" class="">Introduction</a></li>
<li class=""><a href="https://yakecloud.github.io/yake/blog/gardener-chart-releaser#the-gardener-chart-releaser" class="">The Gardener Chart Releaser</a>
<ul>
<li class=""><a href="https://yakecloud.github.io/yake/blog/gardener-chart-releaser#export-the-charts-to-a-local-directory" class="">Export the charts to a local directory</a></li>
<li class=""><a href="https://yakecloud.github.io/yake/blog/gardener-chart-releaser#update-all-version-field-to-the-latest-version" class="">Update all version field to the latest version</a></li>
</ul>
</li>
<li class=""><a href="https://yakecloud.github.io/yake/blog/gardener-chart-releaser#handling-gardener-extensions" class="">Handling Gardener extensions</a></li>
<li class=""><a href="https://yakecloud.github.io/yake/blog/gardener-chart-releaser#running-the-release-process-nightly" class="">Running the release process nightly</a></li>
<li class=""><a href="https://yakecloud.github.io/yake/blog/gardener-chart-releaser#summary" class="">Summary</a></li>
</ul>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="introduction">Introduction<a href="https://yakecloud.github.io/yake/blog/gardener-chart-releaser#introduction" class="hash-link" aria-label="Direct link to Introduction" title="Direct link to Introduction" translate="no">​</a></h2>
<p>Installing <a href="https://github.com/gardener/gardener" target="_blank" rel="noopener noreferrer" class="">Gardener</a> is a complicated process, even though the <a href="https://github.com/gardener/garden-setup" target="_blank" rel="noopener noreferrer" class="">garden-setup</a> installer is provided in the same GitHub organization space. One of the reasons is that the Gardener related Helm charts are spread over multiple repositories.
Consequently, other Helm chart-based installers like <a href="https://github.com/schrodit/gardener-installation" target="_blank" rel="noopener noreferrer" class="">Schrodit's</a> <a href="https://github.com/schrodit/gardener-installation" target="_blank" rel="noopener noreferrer" class="">gardener-installation</a> popped up.
Moreover, we consolidated gardener related Helm charts in the <a href="https://github.com/23technologies/23ke-charts" target="_blank" rel="noopener noreferrer" class="">23ke-charts</a> repository by a simple Python script, and developed a very <a href="https://github.com/23technologies/23ke-charts/tree/main/hack/gardener-from-helmcharts" target="_blank" rel="noopener noreferrer" class="">basic installation approach</a> based on these charts.
The chart collection was released in a Helm repository hosted on GitHub pages by helm's <a href="https://github.com/helm/chart-releaser" target="_blank" rel="noopener noreferrer" class="">chart-releaser</a>.
With the consolidation of the Gardener charts, we faced the issue that the collected charts need to be kept up to date somehow.
There our journey begins, and we started to keep track of the upstream charts with the help of <a href="https://github.com/renovatebot/renovate" target="_blank" rel="noopener noreferrer" class="">renovate</a>.
However, renovate is designed to keep dependencies up to date and finds its limitations, when it comes to tracking multiple versions of the same piece of software.
First, we tried to find a workaround by tracking three branches for the last three minor versions and shifted the latest branch, as soon as a new minor release appeared.
Even though this approach could potentially succeed, we were running into issues from time to time due to missing automatic merges, or failures in the branch shift routine.
As a consequence, we decided to build our own tracking tool, the <a href="https://github.com/gardener-community/gardener-chart-releaser" target="_blank" rel="noopener noreferrer" class="">gardener-chart-releaser</a>.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="the-gardener-chart-releaser">The Gardener Chart Releaser<a href="https://yakecloud.github.io/yake/blog/gardener-chart-releaser#the-gardener-chart-releaser" class="hash-link" aria-label="Direct link to The Gardener Chart Releaser" title="Direct link to The Gardener Chart Releaser" translate="no">​</a></h2>
<p>As already stated above, we wanted to keep track of the last three minor versions of all Gardener related Helm charts, and release these charts in a single helm repository.
In order to achieve this, we needed to make some decisions.
First, we needed to drop our old Python-based Helm chart import script, as working with helm charts in code is way easier, when using the Go-based <a href="https://github.com/helm/helm/tree/main/pkg" target="_blank" rel="noopener noreferrer" class="">helm packages</a> directly. Further, helm's <a href="https://github.com/helm/chart-releaser" target="_blank" rel="noopener noreferrer" class="">chart-releaser</a> is written in Go and there exists a solid implementation of a git and GitHub module in Go.
So, we reimplemented our chart import and release functionality in Go with a view to tracking the last three minor releases.
Another design goal was to keep the tool simple, especially from the user point of view.
As of now, the user only needs to worry about a configuration file in yaml format.
Consider the following example:</p>
<div class="language-text codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-text codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain"># contents of config.yaml</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">destination:</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">	owner: gardener-community</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">	repo: gardener-charts</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">sources:</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">	- name: gardener-controlplane</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">	  version: v1.53.0</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">	  repo: gardener/gardener</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">	  charts:</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">		- charts/gardener/controlplane</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">  - ...</span><br></div></code></pre></div></div>
<p>The <code>destination</code> map defines a GitHub owner and repository, where the collected charts are released.
Under <code>sources</code> a list of source Helm charts is provided by an <code>owner/repo</code> entry and a list of paths pointing to the charts to be released.
With a valid <code>config.yaml</code> a user can simply run</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">export GITHUB_TOKEN=....</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">gardener-chart-releaser update</span><br></div></code></pre></div></div>
<p>and the configured charts will be collected and released.
Note that the <code>version</code> field is ignored for the actual release process, as we want to track several versions.
However, the <code>version</code> field has its own reasoning. Keep on reading ;-)</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="export-the-charts-to-a-local-directory">Export the charts to a local directory<a href="https://yakecloud.github.io/yake/blog/gardener-chart-releaser#export-the-charts-to-a-local-directory" class="hash-link" aria-label="Direct link to Export the charts to a local directory" title="Direct link to Export the charts to a local directory" translate="no">​</a></h3>
<p>Just by collecting and releasing charts to a GitHub repository you won't get to see the charts' contents at all.
But what if you want to work with the charts itself in a local development scenario.
For this purpose, you can export the charts to a local directory instead of releasing them to a remote repo.
Just call</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">gardener-chart-releaser export</span><br></div></code></pre></div></div>
<p>and the charts will be exported to a local <code>./charts</code> directory.
In this case, the <code>version</code> field in <code>config.yaml</code> defines the version to be exported.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="update-all-version-field-to-the-latest-version">Update all version field to the latest version<a href="https://yakecloud.github.io/yake/blog/gardener-chart-releaser#update-all-version-field-to-the-latest-version" class="hash-link" aria-label="Direct link to Update all version field to the latest version" title="Direct link to Update all version field to the latest version" translate="no">​</a></h3>
<p>As the entire Gardener ecosystem is moving quickly, your <code>config.yaml</code> will be outdated soon.
In order to avoid manual updates of the <code>version</code> fields, we introduced another command called <code>fetchLatestVersions</code>.
If you run</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">gardener-chart-releaser fetchLatestVersions</span><br></div></code></pre></div></div>
<p>your <code>config.yaml</code> will be updated, so that you will find the versions of the latest upstream releases in the file.
Of course, it only makes sense to run this before a local export to make sure that the most recent versions of charts are exported.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="handling-gardener-extensions">Handling Gardener extensions<a href="https://yakecloud.github.io/yake/blog/gardener-chart-releaser#handling-gardener-extensions" class="hash-link" aria-label="Direct link to Handling Gardener extensions" title="Direct link to Handling Gardener extensions" translate="no">​</a></h2>
<p>You might be wondering how Gardener extensions are managed in this approach, as these are not provided as Helm charts upstream.
Remember that we wanted to build a single point of truth for a Gardener provisioning, and consequently the gardener-chart-releaser also packages Gardener extensions as charts.
For each entry like e.g.</p>
<div class="language-yaml codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-yaml codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token key atrule" style="color:#00a4db">sources</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">  </span><span class="token punctuation" style="color:#393A34">-</span><span class="token plain"> </span><span class="token key atrule" style="color:#00a4db">name</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> runtime</span><span class="token punctuation" style="color:#393A34">-</span><span class="token plain">gvisor</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">	</span><span class="token key atrule" style="color:#00a4db">version</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> v0.5.1</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">	</span><span class="token key atrule" style="color:#00a4db">repo</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> gardener/gardener</span><span class="token punctuation" style="color:#393A34">-</span><span class="token plain">extension</span><span class="token punctuation" style="color:#393A34">-</span><span class="token plain">runtime</span><span class="token punctuation" style="color:#393A34">-</span><span class="token plain">gvisor</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">	</span><span class="token key atrule" style="color:#00a4db">charts</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">	  </span><span class="token punctuation" style="color:#393A34">-</span><span class="token plain"> controller</span><span class="token punctuation" style="color:#393A34">-</span><span class="token plain">registration</span><br></div></code></pre></div></div>
<p>in the configuration file, it will generate a Helm chart for the specified extension and release it the same way as the Gardener core charts.
Furthermore, this approach provides the opportunity to release charts for the extension itself (i.e. controllerRegistration and controllerDeployment) and the charts for the admission controllers as sub-charts.
For instance</p>
<div class="language-yaml codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-yaml codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token key atrule" style="color:#00a4db">sources</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">  </span><span class="token punctuation" style="color:#393A34">-</span><span class="token plain"> </span><span class="token key atrule" style="color:#00a4db">name</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> provider</span><span class="token punctuation" style="color:#393A34">-</span><span class="token plain">azure</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">	</span><span class="token key atrule" style="color:#00a4db">version</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> v1.29.0</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">	</span><span class="token key atrule" style="color:#00a4db">repo</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> gardener/gardener</span><span class="token punctuation" style="color:#393A34">-</span><span class="token plain">extension</span><span class="token punctuation" style="color:#393A34">-</span><span class="token plain">provider</span><span class="token punctuation" style="color:#393A34">-</span><span class="token plain">azure</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">	</span><span class="token key atrule" style="color:#00a4db">charts</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">	  </span><span class="token punctuation" style="color:#393A34">-</span><span class="token plain"> controller</span><span class="token punctuation" style="color:#393A34">-</span><span class="token plain">registration</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">	  </span><span class="token punctuation" style="color:#393A34">-</span><span class="token plain"> charts/gardener</span><span class="token punctuation" style="color:#393A34">-</span><span class="token plain">extension</span><span class="token punctuation" style="color:#393A34">-</span><span class="token plain">admission</span><span class="token punctuation" style="color:#393A34">-</span><span class="token plain">azure</span><br></div></code></pre></div></div>
<p>will package a top-level chart called <code>provider-azure</code> with sub-charts for the extension controller and admission controller, respectively.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="running-the-release-process-nightly">Running the release process nightly<a href="https://yakecloud.github.io/yake/blog/gardener-chart-releaser#running-the-release-process-nightly" class="hash-link" aria-label="Direct link to Running the release process nightly" title="Direct link to Running the release process nightly" translate="no">​</a></h2>
<p>As we want to be as transparent as possible, we set up a GitHub <a href="https://github.com/gardener-community/gardener-charts/actions" target="_blank" rel="noopener noreferrer" class="">action</a>, so that the chart-releaser is run nightly.
This will ensure that we do not miss any important upstream change and the Helm repository is always up to date.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="summary">Summary<a href="https://yakecloud.github.io/yake/blog/gardener-chart-releaser#summary" class="hash-link" aria-label="Direct link to Summary" title="Direct link to Summary" translate="no">​</a></h2>
<p>The gardener-chart-releaser enables a single point of truth for Gardener related Helm charts and could be a good starting point for custom Gardener installation routines.</p>]]></content>
        <author>
            <name>Jens Schneider</name>
            <uri>https://github.com/jensac</uri>
        </author>
        <author>
            <name>Malte Münch</name>
            <uri>https://github.com/mxmxchere</uri>
        </author>
        <category label="gardener" term="gardener"/>
        <category label="gardener-community" term="gardener-community"/>
    </entry>
    <entry>
        <title type="html"><![CDATA[Getting started with Gardener extension development]]></title>
        <id>https://yakecloud.github.io/yake/blog/gardener-ext-dev</id>
        <link href="https://yakecloud.github.io/yake/blog/gardener-ext-dev"/>
        <updated>2022-06-08T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[TLDR;]]></summary>
        <content type="html"><![CDATA[<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="tldr">TLDR;<a href="https://yakecloud.github.io/yake/blog/gardener-ext-dev#tldr" class="hash-link" aria-label="Direct link to TLDR;" title="Direct link to TLDR;" translate="no">​</a></h2>
<p>Recently, we developed the <a href="https://github.com/23technologies/gardener-extension-mwe" target="_blank" rel="noopener noreferrer" class="">gardener-extension-mwe</a>, which serves as a minimal working example for Gardener extensions.
If you are only interested in the code, go and checkout the repository on github.
If you want to learn more, keep on reading.</p>
<p><strong>Table of Contents</strong></p>
<ul>
<li class=""><a href="https://yakecloud.github.io/yake/blog/gardener-ext-dev#tldr" class="">TLDR;</a></li>
<li class=""><a href="https://yakecloud.github.io/yake/blog/gardener-ext-dev#introduction" class="">Introduction</a></li>
<li class=""><a href="https://yakecloud.github.io/yake/blog/gardener-ext-dev#basic-workflows" class="">Basic workflows</a></li>
<li class=""><a href="https://yakecloud.github.io/yake/blog/gardener-ext-dev#the-mininal-working-example" class="">The mininal working example</a>
<ul>
<li class=""><a href="https://yakecloud.github.io/yake/blog/gardener-ext-dev#rapid-prototyping-on-a-kubernetes-cluster-tested-with-version-1226" class="">Rapid prototyping on a Kubernetes cluster (tested with version 1.22.6)</a></li>
<li class=""><a href="https://yakecloud.github.io/yake/blog/gardener-ext-dev#development-in-a-real-gardener-environment" class="">Development in a real Gardener environment</a></li>
</ul>
</li>
<li class=""><a href="https://yakecloud.github.io/yake/blog/gardener-ext-dev#last-words" class="">Last words</a></li>
</ul>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="introduction">Introduction<a href="https://yakecloud.github.io/yake/blog/gardener-ext-dev#introduction" class="hash-link" aria-label="Direct link to Introduction" title="Direct link to Introduction" translate="no">​</a></h2>
<p>Starting the development of new <a href="https://gardener.cloud/" target="_blank" rel="noopener noreferrer" class="">Gardener</a> extensions can be challenging.
As the <a href="https://gardener.cloud/docs/" target="_blank" rel="noopener noreferrer" class="">Gardener documentation</a> is fairly complex and driven by the history of the project, getting into the overall concepts is not easy.
Moreover, code examples for Gardener extensions reside in separate Git repositories lacking documentation.
However, early in March 2022 the <a href="https://github.com/gardener/gardener-extension-shoot-networking-filter" target="_blank" rel="noopener noreferrer" class="">gardener-extension-shoot-networking-filter</a> was published, which comes at a more beginner friendly level than the e.g. the cloud-provider extensions.
In particular, it extends <code>Shoot</code> clusters by the use of <a href="https://gardener.cloud/docs/gardener/concepts/resource-manager/#managedresource-controller" target="_blank" rel="noopener noreferrer" class="">managed resources</a>, which might be more straight-forward than the interaction with a cloud service provider as performed by e.g. the <a href="https://github.com/gardener/gardener-extension-provider-aws" target="_blank" rel="noopener noreferrer" class="">gardener-extension-provider-aws</a>.
Thus, <a href="https://github.com/gardener/gardener-extension-shoot-networking-filter" target="_blank" rel="noopener noreferrer" class="">gardener-extension-shoot-networking-filter</a> provides a reasonable starting point for new developments, which target at automated deployments into <code>Shoot</code> clusters.</p>
<p>However, going beyond the identification of a starting point, it makes sense to take a closer look at the concepts for extension development.
In the <a href="https://github.com/gardener/gardener/tree/master/extensions" target="_blank" rel="noopener noreferrer" class="">extension directory</a> of the Gardener Git repository, we find several Go-packages defining interfaces, which can be implemented by extension controllers.
Put simply, we can search for files matching <code>pkg/controller/*/actuator.go</code>, in order to find interfaces for controllers acting on the corresponding resources.
E.g., if our controller defines a (golang)-type -- called <code>actuator</code> -- implementing the interface defined in <code>pkg/controller/extension/actuator.go</code>, the controller will reconcile resources of type <code>*extensionsv1alpha1.Extension</code>.
Consequently, the controller will take care for all the steps we define in the <code>Reconcile</code> method of the <code>actuator</code>, when <code>Extension</code> resources are updated.
Of course, there might be more complex scenarios where reconciling <code>Extension</code> resources would be insufficient.
In these cases, other interfaces like e.g. defined in <code>pkg/controller/infrastructure/actuator.go</code> would need to be implemented.
However, these cases lie beyond the scope of this blog post.</p>
<p>Next, we will dive into some basic workflows for Gardener extension development.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="basic-workflows">Basic workflows<a href="https://yakecloud.github.io/yake/blog/gardener-ext-dev#basic-workflows" class="hash-link" aria-label="Direct link to Basic workflows" title="Direct link to Basic workflows" translate="no">​</a></h2>
<p>In software engineering, it is reasonable to develop on a local machine with a controllable toolchain.
As already mentioned above, Gardener extensions are implemented in <a href="https://go.dev/" target="_blank" rel="noopener noreferrer" class="">Go</a>.
Therefore, let's identify a few requirements for the development:</p>
<ul>
<li class="">An installation of Go</li>
<li class="">A text editor, which (optionally) supports <a href="https://pkg.go.dev/golang.org/x/tools/gopls" target="_blank" rel="noopener noreferrer" class="">gopls</a></li>
<li class="">A Go debugger, which is most likely to be <a href="https://github.com/go-delve/delve" target="_blank" rel="noopener noreferrer" class="">Delve</a></li>
<li class="">A Gardener development environment. This can be setup by
<ul>
<li class=""><a href="https://gardener.cloud/docs/gardener/development/getting_started_locally/" target="_blank" rel="noopener noreferrer" class="">Running Gardener locally</a> (also checkout <a href="https://github.com/gardener/gardener/issues/5548" target="_blank" rel="noopener noreferrer" class="">#5548</a>, if you are running on Linux)</li>
<li class="">Setting up a development Gardener on some cloud infrastructure. This definitely comes closer to the real world scenario your extension will eventually live in.
The block diagram below depicts the overall setup including the requirements from above.</li>
</ul>
</li>
</ul>
<div class="language-text codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-text codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">┌──────────────────────────────────────────────┐</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">│    Development Computer                      │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">├──────────────────────────────────────────────┤</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">│                                              │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">│ ┌──────────────────────────────────────────┐ │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">│ │             - Your toolchain             │ │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">│ └──────────────────────────────────────────┘ │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">│                                              │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">│     ┌────────────┐        ┌──────────────┐   │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">│     │Kubeconfigs │        │Your code     │   │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">│     └──┬──────┬──┘        └────────┬─────┘   │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">│        │      │                    │         │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">└────────┼──────┼────────────────────┼─────────┘</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">         │      │                    │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">         │      │apply               │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">  apply  │      │resources           │reconcile</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">resources│      │                    │resources</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">         │      └──────────────────┐ │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">         │                         │ │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">         │                         │ │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">         ▼                         ▼ ▼</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">   ┌─────────────────┐       ┌─────────────────┐</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">   │ Garden Cluster  │       │ Seed Cluster    │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">   ├─────────────────┤       ├─────────────────┤</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">   │- Project        │       │- Extension      │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">   │- Seed           │       │- Controller     │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">   │- Shoot          │       │- ...            │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">   │- ...            │       │                 │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">   └─────────────────┘       └─────────────────┘</span><br></div></code></pre></div></div>
<p>As you can see, the code for the extension controller is running on your local development machine, and reconciles resources, such as <code>Extension</code> resources, in the <code>Seed</code> cluster.
Of course, you will not have dedicated clusters for the <code>Garden</code> cluster and <code>Seed</code> cluster, when running Gardener locally.
However, the overall buidling blocks stay conceptually the same.
Once these requirements are met, you are good to go for your first steps with Gardener extensions.
Wait! I have to setup an entire Gardener, if I want to rapidly prototype an extension controller?
Yes and No. Depending on your development/test case, it might be reasonable to "fake" a Gardener environment on a vanilla Kubernetes cluster. We will get to this development case below.
For rock solid testing, however, you most probably need a real world Gardener environment.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="the-mininal-working-example">The mininal working example<a href="https://yakecloud.github.io/yake/blog/gardener-ext-dev#the-mininal-working-example" class="hash-link" aria-label="Direct link to The mininal working example" title="Direct link to The mininal working example" translate="no">​</a></h2>
<p>As of May 2022, we provide a Minimal Working Example (MWE) for gardener extensions.
Of course, this example did not come out of nowhere.
Therefore, we review the development process and break the example down to its components in the following:
Taking a look at other extensions, it is observed that we need some boiler plate code for running our controller, so that it works together with all the other Gardener components.
For the MWE, we collected the relevant files and adjusted them to our needs.
Thus, we can have a look at the <code>cmd</code> directory of the <a href="https://github.com/23technologies/gardener-extension-mwe" target="_blank" rel="noopener noreferrer" class="">gardener-extension-mwe</a> and find a simple structure with 3 files, which are responsible for starting our controller and ensuring that it acts on the defined resources.</p>
<div class="language-text codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-text codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">cmd</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">└── gardener-extension-mwe</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    ├── app</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    │&nbsp;&nbsp; ├── app.go</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    │&nbsp;&nbsp; └── options.go</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    └── main.go</span><br></div></code></pre></div></div>
<p>If you want to start the development of a new extension, you do not need to worry too much about these files.
Most probably, you can simple copy them over, and adjust some variables to your needs.
Actually, we also copied these files from the <a href="https://github.com/gardener/gardener-extension-shoot-networking-filter" target="_blank" rel="noopener noreferrer" class="">gardener-extension-shoot-networking-filter</a> and adjusted them to the needs of the MWE.
Given that we have the boilerplate code in the <code>cmd</code> directory available now, we can go ahead and define a type which implements an <code>actuator</code> interface.
For this, we need the files in the <code>pkg</code> directory.
Lets take a look at the structure:</p>
<div class="language-text codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-text codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">pkg</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">└── controller</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">	└── lifecycle</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">		├── actuator.go</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">		└── add.go</span><br></div></code></pre></div></div>
<p>Also here, we find only two files, and the implementation of the interface is located in <a href="https://github.com/23technologies/gardener-extension-mwe/blob/main/pkg/controller/lifecycle/actuator.go" target="_blank" rel="noopener noreferrer" class=""><code>actuator.go</code></a>.
This is the place where most of the magic of your new controller happens.
In the case of the MWE, the actuator will only output logs, when <code>Extension</code> resources are reconciled.
Obviously, all code is written in Go and consequently we will also need to pull in some dependencies.
For this, we need the files <code>go.mod</code> and <code>go.sum</code>.
Typically, the source code of the dependencies is also committed to the repository, which comes at advantages and downsides.
The main advantage is that all code needed for building an application is available in the repository.
On the other hand, committing several 1000s lines of code during vendoring clutters the commit history of the repository.
Therefore, we only provide the files mentioned above in the <a href="https://github.com/23technologies/gardener-extension-mwe/commit/455c9c76876161bf8d5197e1330a9fc28f825baa" target="_blank" rel="noopener noreferrer" class="">initial commit</a> of the MWE and perform the vendoring (by running <code>go mod vendor</code>) in <a href="https://github.com/23technologies/gardener-extension-mwe/commit/3c238bdcc5697392d567d5b6e5f2cf6126b3c756" target="_blank" rel="noopener noreferrer" class="">another commit</a>.
At this state of the repository, we can already start and go the first steps with our new controller in a vanilla Kubernetes cluster.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="rapid-prototyping-on-a-kubernetes-cluster-tested-with-version-1226">Rapid prototyping on a Kubernetes cluster (tested with version 1.22.6)<a href="https://yakecloud.github.io/yake/blog/gardener-ext-dev#rapid-prototyping-on-a-kubernetes-cluster-tested-with-version-1226" class="hash-link" aria-label="Direct link to Rapid prototyping on a Kubernetes cluster (tested with version 1.22.6)" title="Direct link to Rapid prototyping on a Kubernetes cluster (tested with version 1.22.6)" translate="no">​</a></h3>
<p>Assuming you have read the <a href="https://yakecloud.github.io/yake/blog/gardener-ext-dev#basic-workflows" class="">basic workflows</a> section, we are ready to dive into the exemplary development techniques.
So let's fetch the code and setup the repository:</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">git clone https://github.com/23technologies/gardener-extension-mwe.git</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">cd gardener-extension-mwe</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">git checkout 3c238bd  # checkout the commit containing first vendoring</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">mkdir dev</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">cp PATH-TO/KUBECONFIG.yaml dev/kubeconfig.yaml</span><br></div></code></pre></div></div>
<p>Now, we can already start our controller and should get some output showing that it was started:</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">go run ./cmd/gardener-extension-mwe --kubeconfig=dev/kubeconfig.yaml  --ignore-operation-annotation=true --leader-election=false</span><br></div></code></pre></div></div>
<p>However, we will not observe any other output, since the controller is still freewheeling.
Remember, reconciliation will be triggered on <code>Extension</code> resources.
As our vanilla Kubernetes cluster does not know anything about <code>Extension</code> resources yet, we will have to "fake" the Gardener environment.
In other Gardener extensions, we find resources for a "fake" Gardener setup in the <code>example</code> directory.
Therefore, we prepared the <code>example</code> directory in <a href="https://github.com/23technologies/gardener-extension-mwe/commit/50f7136330e114ec2f795f3e30a756381dd4cbc6" target="_blank" rel="noopener noreferrer" class="">another commit</a>.
Let's check it out:
Open a new terminal pane and navigate to your repository and go for</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">git checkout 50f7136 # this commit entails the example directory</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">export KUBECONFIG=dev/kubeconfig.yaml</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">kubectl apply -f example/10-fake-shoot-controlplane.yaml</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">kubectl apply -f example/20-crd-cluster.yaml</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">kubectl apply -f example/20-crd-extension.yaml</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">kubectl apply -f example/30-cluster.yaml</span><br></div></code></pre></div></div>
<p>Now, the cluster simulates a Gardener environment and we can apply an <code>Extension</code> resource:</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">kubectl apply -f example/40-extension.yaml</span><br></div></code></pre></div></div>
<p>Take another look at the terminal running our controller now.
It should have logged a "Hello World" message.
Of course, we can also delete the <code>Extension</code> resource again and the controller will tell us that the <code>Delete</code> method was called.</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">kubectl delete -f example/40-extension.yaml</span><br></div></code></pre></div></div>
<p>As we have the code and a method to trigger its execution available now, we can go ahead for a more interactive approach based on the <a href="https://github.com/go-delve/delve" target="_blank" rel="noopener noreferrer" class="">Delve debugger</a>.
Let's start all over again and run our controller using Delve</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">dlv debug ./cmd/gardener-extension-mwe -- --kubeconfig=dev/kubeconfig.yaml  --ignore-operation-annotation=true --leader-election=false</span><br></div></code></pre></div></div>
<p>and we will end up in a commandline with a <code>(dlv)</code> prompt.
Next, we ask <code>dlv</code> to break in the <code>Reconcile</code> method</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">(dlv) b github.com/23technologies/gardener-extension-mwe/pkg/controller/lifecycle.(*actuator).Reconcile</span><br></div></code></pre></div></div>
<p>and continue the execution of the controller</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">(dlv) c</span><br></div></code></pre></div></div>
<p>Afterwards, you should observe some output of the controller, again.
However, Delve will not break the execution until the <code>Reconcile</code> method is called.
Thus, we apply the <code>Extension</code> resource once again</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">kubectl apply -f example/40-extension.yaml</span><br></div></code></pre></div></div>
<p>and Delve will stop in the <code>Reconcile</code> method.
Now, you can step through the code, see where it enters code paths pointing into the vendor directory, and inspect the values of certain variables.
Obviously, the amount of variables you can inspect is limited in the MWE, but e.g. we can have a look at the <code>*extensionsv1alpha1.Extension</code> passed to the <code>Reconcile</code> method</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">(dlv) p ex.ObjectMeta.Name</span><br></div></code></pre></div></div>
<p>which should print <code>"mwe"</code>.
Generally, this is a great way to approach unknown software, since you will quickly get a feeling for the different components.
Thus, we expect that you can benefit from this workflow, when developing your own extensions.
Even though this approach offers capabilities for rapid prototyping, it is still limited, since we cannot act e.g. on <code>Shoot</code> clusters as available in a real world Gardener.
Therefore, we step into the development in a Gardener environment in the next section.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="development-in-a-real-gardener-environment">Development in a real Gardener environment<a href="https://yakecloud.github.io/yake/blog/gardener-ext-dev#development-in-a-real-gardener-environment" class="hash-link" aria-label="Direct link to Development in a real Gardener environment" title="Direct link to Development in a real Gardener environment" translate="no">​</a></h3>
<p>Developing and testing our extension in a real world Gardener requires a <code>ControllerRegistration</code> resource in the <code>Garden</code> cluster causing the installation of the controller in <code>Seed</code> clusters.
Generally, the installation performed by Helm charts and consequently, we need to provide these charts in the repository.
Also for the MWE, we prepared the <code>charts</code> directory containing all relevant Helmcharts for the deployment of the controller.
Note that this set of charts is very limited and in production scenarios you might want to add something like a <code>VerticalPodAutoscaler</code> as done e.g. in the <a href="https://github.com/gardener/gardener-extension-shoot-networking-filter" target="_blank" rel="noopener noreferrer" class="">gardener-extension-shoot-networking-filter</a>.
However, implementing production ready charts goes beyond the scope of this post, and consequently the MWE charts were added in <a href="https://github.com/23technologies/gardener-extension-mwe/commit/beee274314ac628c65a3dcb4846dad577744b36a" target="_blank" rel="noopener noreferrer" class="">another commit</a>.
These charts target at running the controller in <code>Seed</code> clusters.
Thus, in <code>charts/gardener-extension-mwe/values.yaml</code>, the image for the deployment is defined.
However, we do not want to push that image to a public container registry for each and every change we make to our code.
Moreover, we want to run the controller on our local machine for development purposes.
Therefore, we need to tweak the values before generating the <code>controller-registration.yaml</code>.
Let's go through it step by step:</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">git clone https://github.com/23technologies/gardener-extension-mwe.git</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">cd gardener-extension-mwe</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">mkdir dev</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">cp PATH-TO/KUBECONFIG-FOR-SEED.yaml dev/kubeconfig.yaml</span><br></div></code></pre></div></div>
<p>Next, we generate the <code>controller-registration.yaml</code>, such that the controller is not deployed to the seed cluster and we can hook-in the controller running locally.
In particular, we set <code>replicaCount=0</code> and <code>ignoreResources=true</code> in <code>./charts/gardener-extension-mwe/values.yaml</code>, before generating the <code>controller-registration.yaml</code>:</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">yq eval -i '.replicaCount=0 | .ignoreResources=true' charts/gardener-extension-mwe/values.yaml</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">./vendor/github.com/gardener/gardener/hack/generate-controller-registration.sh mwe charts/gardener-extension-mwe v0.0.1 example/controller-registration.yaml Extension:mwe</span><br></div></code></pre></div></div>
<p>Now, let's deploy the generated <code>controller-registration.yaml</code> into the <code>Garden</code> cluster:</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">export KUBECONFIG=PATH-TO/GARDEN-CLUSTER-KUBECONFIG.yaml</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">kubectl apply -f example/controller-registration.yaml</span><br></div></code></pre></div></div>
<p>From now on, <code>Extension</code> resources of the type <code>mwe</code> will be deployed to <code>Seed</code> clusters when new <code>Shoot</code> clusters with</p>
<div class="language-yaml codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-yaml codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token punctuation" style="color:#393A34">---</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token key atrule" style="color:#00a4db">apiVersion</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> core.gardener.cloud/v1beta1</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token key atrule" style="color:#00a4db">kind</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> Shoot</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token key atrule" style="color:#00a4db">metadata</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">  </span><span class="token key atrule" style="color:#00a4db">name</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> bar</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">  </span><span class="token key atrule" style="color:#00a4db">namespace</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> garden</span><span class="token punctuation" style="color:#393A34">-</span><span class="token plain">foo</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token key atrule" style="color:#00a4db">spec</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">  </span><span class="token key atrule" style="color:#00a4db">extensions</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">  </span><span class="token punctuation" style="color:#393A34">-</span><span class="token plain"> </span><span class="token key atrule" style="color:#00a4db">type</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> mwe</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token punctuation" style="color:#393A34">...</span><br></div></code></pre></div></div>
<p>are created.
In our controller, the <code>Reconcile</code> method will be triggered, when these <code>Extension</code> resources are reconciled.
Therefore, we can run the extension controller with Delve now</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">dlv debug ./cmd/gardener-extension-mwe -- --kubeconfig=dev/kubeconfig.yaml  --ignore-operation-annotation=true --leader-election=false --gardener-version="v1.44.4"</span><br></div></code></pre></div></div>
<p>and we can perform debugging operations as explained above.
Remember, Delve will not break the execution until the <code>Reconcile</code> method is called.
Now, Gardener will create <code>Extension</code> resources for <code>Shoot</code>s which will trigger the <code>Reconcile</code> method of our controller.
Consequently, we need a new terminal pane in the repository root and execute</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">export KUBECONFIG=PATH-TO/GARDEN-CLUSTER-KUBECONFIG.yaml</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">kubectl apply -f example/50-shoot.yaml</span><br></div></code></pre></div></div>
<p>Note that it will take some time until the corresponding <code>Extension</code> resource will be created in the <code>Seed</code> cluster.
Hang on tight and wait for the <code>Reconcile</code> method being executed.
You can start investigating where your code goes using Delve now.
Happy hacking!</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="last-words">Last words<a href="https://yakecloud.github.io/yake/blog/gardener-ext-dev#last-words" class="hash-link" aria-label="Direct link to Last words" title="Direct link to Last words" translate="no">​</a></h2>
<p>This blog post shares our experience, when getting started with Gardener extension development.
We hope that this contribution helps you get started more quickly than us.
If you have any comment or ideas for improvements, do not hesitate to contact us.
We are always willing to improve our work.</p>]]></content>
        <author>
            <name>Jens Schneider</name>
            <uri>https://github.com/jensac</uri>
        </author>
        <category label="gardener" term="gardener"/>
        <category label="gardener-extensions" term="gardener-extensions"/>
    </entry>
    <entry>
        <title type="html"><![CDATA[A Gardener Extension for universal Shoot Configuration]]></title>
        <id>https://yakecloud.github.io/yake/blog/gardener-ext-shoot-flux</id>
        <link href="https://yakecloud.github.io/yake/blog/gardener-ext-shoot-flux"/>
        <updated>2022-06-08T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[TLDR;]]></summary>
        <content type="html"><![CDATA[<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="tldr">TLDR;<a href="https://yakecloud.github.io/yake/blog/gardener-ext-shoot-flux#tldr" class="hash-link" aria-label="Direct link to TLDR;" title="Direct link to TLDR;" translate="no">​</a></h2>
<p>Recently, we developed the <a href="https://github.com/23technologies/gardener-extension-shoot-flux" target="_blank" rel="noopener noreferrer" class="">gardener-extension-shoot-flux</a>, which enables preconfiguring <code>Shoot</code> clusters.
If you want to give it a try, go and checkout the repository on Github.
If you want to learn more, keep on reading.</p>
<p><strong>Table of Contents</strong></p>
<ul>
<li class=""><a href="https://yakecloud.github.io/yake/blog/gardener-ext-shoot-flux#tldr" class="">TLDR;</a></li>
<li class=""><a href="https://yakecloud.github.io/yake/blog/gardener-ext-shoot-flux#introduction" class="">Introduction</a></li>
<li class=""><a href="https://yakecloud.github.io/yake/blog/gardener-ext-shoot-flux#example-use-cases" class="">Example use cases</a>
<ul>
<li class=""><a href="https://yakecloud.github.io/yake/blog/gardener-ext-shoot-flux#development" class="">Development</a></li>
<li class=""><a href="https://yakecloud.github.io/yake/blog/gardener-ext-shoot-flux#cicd" class="">CI/CD</a></li>
</ul>
</li>
<li class=""><a href="https://yakecloud.github.io/yake/blog/gardener-ext-shoot-flux#general-concept" class="">General concept</a></li>
<li class=""><a href="https://yakecloud.github.io/yake/blog/gardener-ext-shoot-flux#example-usage" class="">Example Usage</a></li>
</ul>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="introduction">Introduction<a href="https://yakecloud.github.io/yake/blog/gardener-ext-shoot-flux#introduction" class="hash-link" aria-label="Direct link to Introduction" title="Direct link to Introduction" translate="no">​</a></h2>
<p><a href="https://fluxcd.io/" target="_blank" rel="noopener noreferrer" class="">Flux</a> offers a set of controllers allowing for reconciling a Kubernetes cluster with a declarative state defined in e.g. a Git repository.
Thus it enables GitOps workflows for Kubernetes clusters.
Moreover, it provides a general approach of deploying software components into Kubernetes clusters.
<a href="https://gardener.cloud/" target="_blank" rel="noopener noreferrer" class="">Gardener</a> is a multi cloud managed Kubernetes service allowing end users to create clusters with a few clicks in its dashboard.
However, the user will obtain a vanilla Kubernetes cluster and has to take care for all the components to be deployed into it.
Of course, the deployment can be performed manually by applying Kubernetes manifests to the cluster.
On the other hand, tools like Flux can help to keep track of the deployments and automate the overall process.
Thus, the combination of Gardener and Flux features the potential of creating new Kubernetes clusters in a pre-defined state.
For the end users, this results in the seamless creation of clusters with all components on their wish list installed.
The <a href="https://github.com/23technologies/gardener-extension-shoot-flux" target="_blank" rel="noopener noreferrer" class="">gardener-extension-shoot-flux</a> bridges the gap between Gardener and Flux and allows for reconciliation of <code>Shoot</code> clusters to resources defined in a Git repository.
By concept, the extension operates on a per-project basis so that clusters in different projects can be reconciled to different repositories.</p>
<p>The rest of this post is organized as follows:
First, we will review a few use cases for this extension.
Further, the general concept of the extension is outlined, and finally we provide an example on how to use the extension.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="example-use-cases">Example use cases<a href="https://yakecloud.github.io/yake/blog/gardener-ext-shoot-flux#example-use-cases" class="hash-link" aria-label="Direct link to Example use cases" title="Direct link to Example use cases" translate="no">​</a></h2>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="development">Development<a href="https://yakecloud.github.io/yake/blog/gardener-ext-shoot-flux#development" class="hash-link" aria-label="Direct link to Development" title="Direct link to Development" translate="no">​</a></h3>
<p>Imagine you are developing software which will eventually run on a Kubernetes cluster in the public cloud.
Moreover, you and your colleagues want to be able to perform some end-to-end tests besides running your local test suite.
For these end-to-end test, an environment mimicking the final production environment is required.
Therefore, you might need tools like <a href="https://cert-manager.io/" target="_blank" rel="noopener noreferrer" class="">cert-manager</a> or <a href="https://min.io/" target="_blank" rel="noopener noreferrer" class="">MinIO</a>.
However, you do not want keep several testing clusters in the public cloud available for economic reasons and, in consequence, you need to create new clusters on demand.
In this case, the <a href="https://github.com/23technologies/gardener-extension-shoot-flux" target="_blank" rel="noopener noreferrer" class="">gardener-extension-shoot-flux</a> comes handy, since it allows to configure the cluster asynchronously.
Put simply, you can define the desired state of your cluster in a Git repository, and the new clusters will be reconciled to this state automatically.
Eventually, this will save the effort to configure the clusters each and every time manually.
Of course, you could achieve something similar by hibernation of the development clusters.
However, in that case you are less flexible, since throwing away the cluster in case you lost track of your clusters state comes at the price of reconfiguring the entire cluster.</p>
<h3 class="anchor anchorTargetStickyNavbar_Vzrq" id="cicd">CI/CD<a href="https://yakecloud.github.io/yake/blog/gardener-ext-shoot-flux#cicd" class="hash-link" aria-label="Direct link to CI/CD" title="Direct link to CI/CD" translate="no">​</a></h3>
<p>Similar to the development use case above, you might want to run your CI/CD pipeline in Kubernetes clusters coming with a few components already installed.
As your pipeline runs frequently, you want to create clusters on the fly or maybe pre-spawn just a few of them.
In order to keep your pipeline simple, you can use the <a href="https://github.com/23technologies/gardener-extension-shoot-flux" target="_blank" rel="noopener noreferrer" class="">gardener-extension-shoot-flux</a> for the configuration of your CI/CD clusters.
This way your pipeline can focus on the actual action and does not have to perform the cluster configuration beforehand.
This most probably results in cleaner and more stable CI/CD pipelines.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="general-concept">General concept<a href="https://yakecloud.github.io/yake/blog/gardener-ext-shoot-flux#general-concept" class="hash-link" aria-label="Direct link to General concept" title="Direct link to General concept" translate="no">​</a></h2>
<p>The general concept of this extension is visualized in the block diagram below.</p>
<div class="language-text codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-text codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">                 ┌─────────────────────────────────────────────────────────┐</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                 │ Gardener operator                                       │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                 ├─────────────────────────────────────────────────────────┤</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                 │ - A human being                                         │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                 │                                                         ├────────────┐</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                 │                                                         │            │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                 │                                                         │            │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                 └────────┬────────────────────────────────────────────────┘            │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                          │                           ▲                                 │configures</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                          │deploys                    │                                 │SSH-key</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                          │Configmap                  │read SSH-key                     │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                          │                           │                                 │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                          ▼                           │                                 │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                 ┌────────────────────────────────────┴───────────────────┐             │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                 │ Garden cluster                                         │             │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                 ├────────────────────────┬─────────────────────────┬─────┤             │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                 │ Projetct 1             │ Project 2               │ ... │             ▼</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                 ├────────────────────────┼─────────────────────────┼─────┤  ┌─────────────────────┐</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                 │- Configmap containing  │- Configmap containing   │     │  │ Git repository      │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                 │  flux configuration    │  flux configuration     │     │  ├─────────────────────┤</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                 │                        │                         │     │  │ - Configuration for │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">            ┌───►│- ControllerRegistration│- ControllerRegistration │ ... │  │   shoot clusters    │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">            │    │                        │                         │     │  └─────────────────────┘</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">            │    │- Shoot with extension  │- Shoot with extension   │     │             ▲</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">            │    │  enabled               │  enabled                │     │             │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">            │    │                        │                         │     │             │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">read config │    │                        │                         │     │             │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">and generate│    └────────────────────────┴─────────────────────────┴─────┘             │reconcile</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">SSH-keys    │                                                                           │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">            │    ┌────────────────────────┐     ┌────────────────────────┐              │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">            │    │ Seed cluster           │     │ Shoot cluster          │              │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">            │    ├────────────────────────┤     ├────────────────────────┤              │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">            │    │- Controller watching   │     │                        │              │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">            └────┼─ extension resource    │     │- Flux controllers  ────┼──────────────┘</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                 │     │                  │     │                        │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                 │     │deploys           │     │- GitRepository resource│</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                 │     │                  │     │                        │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                 │     ▼                  │     │- A main kustomization  │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                 │- Managed resources     │     │                        │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                 │  for flux controllers  │     │                        │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                 │  and flux config       │     │                        │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                 │                        │     │                        │</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">                 └────────────────────────┘     └────────────────────────┘</span><br></div></code></pre></div></div>
<p>As depicted, the Gardener operator needs to deploy a <code>ConfigMap</code> into the Garden cluster.
This <code>ConfigMap</code> holds some configuration parameters for the extension controller.
Moreover, the Gardener operator needs to configure an SSH-key for the Git repository in case of a private repository.
This key can be read from the <code>Secret</code> called <code>flux-source</code> in the Garden cluster which is created by the extension controller.
Of course, the process of adding the SSH-key to the repository depends on the repository host.
E.g. for repositories hosted on Github, the key can simply be added as "Deploy key" in the web-interface.</p>
<p>The extension controller is running in <code>Seed</code> clusters.
Besides generating <code>Secret</code>s containing SSH-keys, it reads the configuration from the Garden cluster and creates <code>Managedresources</code> to be processed by the <a href="https://gardener.cloud/docs/gardener/concepts/resource-manager/#managedresource-controller" target="_blank" rel="noopener noreferrer" class="">Gardener Resource Manager</a>.
These <code>Managedresources</code> entail the resources for the Flux controllers, a <a href="https://fluxcd.io/docs/components/source/gitrepositories/" target="_blank" rel="noopener noreferrer" class="">GitRepository</a> resource matching the configuration, and a main <a href="https://fluxcd.io/docs/components/kustomize/kustomization/" target="_blank" rel="noopener noreferrer" class="">Kustomization</a> resource.
Once the Gardener Resource Manager has deployed these resources to the <code>Shoot</code> cluster, the Flux controllers will reconcile the cluster to the state defined in the Git repository.</p>
<p>You might wonder how the communication between <code>Seed</code> clusters and Garden cluster is established.
This is achieved by making use of the <code>Secret</code> containing the <code>gardenlet-kubeconfig</code> which should be available, when the gardenlet is run inside the <code>Seed</code> cluster.
Most probably, this is not the most elegant solution, but it resulted in a quick first working solution.</p>
<h2 class="anchor anchorTargetStickyNavbar_Vzrq" id="example-usage">Example Usage<a href="https://yakecloud.github.io/yake/blog/gardener-ext-shoot-flux#example-usage" class="hash-link" aria-label="Direct link to Example Usage" title="Direct link to Example Usage" translate="no">​</a></h2>
<p>Of course, you need to install the extension before you can use it.
You can find <code>ControllerRegistration</code>s on our <a href="https://github.com/23technologies/gardener-extension-shoot-flux/releases" target="_blank" rel="noopener noreferrer" class="">Github release page</a>.
So, you can simply go for</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">export KUBECONFIG=KUBECONFIG-FOR-GARDEN-CLUSTER</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">kubectl -f https://github.com/23technologies/gardener-extension-shoot-flux/releases/download/v0.1.2/controller-registration.yaml</span><br></div></code></pre></div></div>
<p>in order to install the extension.</p>
<p>For an exemplary use of the extension, we prepared a public repository containing manifest for the installation of <a href="https://github.com/stefanprodan/podinfo" target="_blank" rel="noopener noreferrer" class="">Podinfo</a>.
As a Gardener operator you can apply the following <code>ConfigMap</code> to your Garden cluster</p>
<div class="language-yaml codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-yaml codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token key atrule" style="color:#00a4db">apiVersion</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> v1</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token key atrule" style="color:#00a4db">kind</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> ConfigMap</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token key atrule" style="color:#00a4db">metadata</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">  </span><span class="token key atrule" style="color:#00a4db">name</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> flux</span><span class="token punctuation" style="color:#393A34">-</span><span class="token plain">config</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">  </span><span class="token key atrule" style="color:#00a4db">namespace</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> YOUR</span><span class="token punctuation" style="color:#393A34">-</span><span class="token plain">PROJECT</span><span class="token punctuation" style="color:#393A34">-</span><span class="token plain">NAMESPACE</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token key atrule" style="color:#00a4db">data</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">  </span><span class="token key atrule" style="color:#00a4db">fluxVersion</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> v0.29.5 </span><span class="token comment" style="color:#999988;font-style:italic"># optional, if not defined the latest release will be used</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">  </span><span class="token key atrule" style="color:#00a4db">repositoryUrl</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> https</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain">//github.com/23technologies/shootflux.git</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">  </span><span class="token key atrule" style="color:#00a4db">repositoryBranch</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> main</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">  </span><span class="token key atrule" style="color:#00a4db">repositoryType</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> public</span><br></div></code></pre></div></div>
<p>As the repository is public you can create a new <code>Shoot</code> now and enable the extension for this <code>Shoot</code>.
Take the snipped below as an example.</p>
<div class="language-yaml codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-yaml codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token key atrule" style="color:#00a4db">apiVersion</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> core.gardener.cloud/v1beta1</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token key atrule" style="color:#00a4db">kind</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> Shoot</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token key atrule" style="color:#00a4db">metadata</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">  </span><span class="token key atrule" style="color:#00a4db">name</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> bar</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">  </span><span class="token key atrule" style="color:#00a4db">namespace</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> garden</span><span class="token punctuation" style="color:#393A34">-</span><span class="token plain">foo</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token key atrule" style="color:#00a4db">spec</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">  </span><span class="token key atrule" style="color:#00a4db">extensions</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">  </span><span class="token punctuation" style="color:#393A34">-</span><span class="token plain"> </span><span class="token key atrule" style="color:#00a4db">type</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> shoot</span><span class="token punctuation" style="color:#393A34">-</span><span class="token plain">flux</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token punctuation" style="color:#393A34">...</span><br></div></code></pre></div></div>
<p>Gardener will take care for the <code>Shoot</code> creation process.
As soon as you can, you can fetch the <code>kubeconfig.yaml</code> for your new <code>Shoot</code> from e.g. the Gardener dashboard.
Now, you can watch this cluster by</p>
<div class="language-shell codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-shell codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">export KUBECONFIG=KUBECONFIG-FOR-SHOOT</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">k9s</span><br></div></code></pre></div></div>
<p>and you should see that a <code>podinfo</code> deployment should come up.
Great! You successfully created a <code>Shoot</code> with the <a href="https://github.com/23technologies/gardener-extension-shoot-flux" target="_blank" rel="noopener noreferrer" class="">gardener-extension-shoot-flux</a>.</p>]]></content>
        <author>
            <name>Jens Schneider</name>
            <uri>https://github.com/jensac</uri>
        </author>
        <category label="gardener" term="gardener"/>
        <category label="gardener-extensions" term="gardener-extensions"/>
    </entry>
</feed>