Basics
------
helm | - client side CLI that installs and manages kubernetes packages - manages charts |
tiller | - installed on k8 cluster - maintains releases - manages releeases [tiller] 2018/08/30 01:02:33 executing 0 crd-install hooks for sample-chart-74027 [tiller] 2018/08/30 01:02:33 hooks complete for crd-install sample-chart-74027 [tiller] 2018/08/30 01:02:33 executing 0 pre-install hooks for sample-chart-74027 [tiller] 2018/08/30 01:02:33 hooks complete for pre-install sample-chart-74027 [storage] 2018/08/30 01:02:33 getting release history for "sample-chart-74027" [storage] 2018/08/30 01:02:33 creating release "sample-chart-74027.v1" [kube] 2018/08/30 01:02:33 building resources from manifest [kube] 2018/08/30 01:02:33 creating 2 resource(s) [tiller] 2018/08/30 01:02:33 executing 0 post-install hooks for sample-chart-74027 [tiller] 2018/08/30 01:02:33 hooks complete for post-install sample-chart-74027 [storage] 2018/08/30 01:02:33 updating release "sample-chart-74027.v1" [storage] 2018/08/30 01:02:33 getting last revision of "sample-chart-74027" [storage] 2018/08/30 01:02:33 getting release history for "sample-chart-74027" [kube] 2018/08/30 01:02:33 Doing get for Service: "sample-chart-74027-tomcat" [kube] 2018/08/30 01:02:33 get relation pod of object: kafkastreams-dev/Service/sample-chart-74027-tomcat [kube] 2018/08/30 01:02:33 Doing get for Deployment: "sample-chart-74027-tomcat" [kube] 2018/08/30 01:02:33 get relation pod of object: kafkastreams-dev/Deployment/sample-chart-74027-tomcat [storage] 2018/08/30 01:02:40 listing all releases with filter |
chart | - helm package - collection of k8s manifests that is used to deploy an application
~$ helm search NAME CHART VERSION APP VERSION DESCRIPTION local/sample 0.1.0 1.0 A Helm chart for Kubernetes rancher-stable/rancher 2.2.2 v2.2.2 Install Rancher Server to manage stable/acs-engine-autoscaler 2.2.2 2.1.1 DEPRECATED Scales worker nodes ... [...] |
releases | specific configuration of a particular chart . . |
revisions | specific configuration of a particular release NAME REVISION UPDATED STATUS CHART APP VERSION NAMESPACE tomcat 3 Wed May 8 13:01:02 2019 DEPLOYED tomcat-0.2.0 7 default |
Permissions needed for "helm install":
- view pods in the namespace "kube-system"
- create pods/portforward in the namespace "kube-system"
Value Precedence
----------------
which ever comes last will win | Both .yaml files contain "favoritefood" variable. |
Tutorials
---------
Installing HELM | curl https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get > get_helm.sh chmod 700 get_helm.sh ./get_helm.sh |
|
Managing plugins | # installing helm plugin install https://github.com/futuresimple/helm-secrets helm plugin install https://github.com/databus23/helm-diff --version master helm plugin install https://github.com/databus23/helm-diff --version v2.10.0 |
|
Listing charts | # all versions |
|
Installing charts | helm install stable/mariadb --> "stable" is the repo name (see mapping below) helm upgrade --install -f values.yml --namespace <namespace> <release-name> charts/ # Chart.yaml must be present in charts/ directory NAME URL |
|
Deleting charts | helm delete <release name> |
|
Ways of adding charts | helm repo add sata-charts 'myrepo.com/charts' helm repo add --ca-file=/tmp/myrepo.crt sata-charts 'myrepo.com/charts' |
|
Deletes tiller | helm reset |
|
Configure tiller (w/ RBAC) | 1. Use this manifest to create RBAC for tiller and run "kubectl create -f <file name>" apiVersion: v1 kind: ServiceAccount metadata: name: tiller namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding metadata: name: tiller roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cluster-admin subjects: - kind: ServiceAccount name: tiller namespace: kube-system
or you may use these 2 line-command:
kubectl -n kube-system create serviceaccount tiller kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller
4. Check roll out status
kubectl -n kube-system rollout status deploy/tiller-deploy 3. Tiller must now be running as a deployment "tiller-deploy" under kube-system namespace
|
|
Securing tiller |
| |
Creating a chart layout | helm create mychart |
|
Check what will be deployed | helm install --dry-run --debug ./mychart |
|
Deploying multiple releases (different configurations) of a chart | helm install --name web-base-app ./mychart helm install --name web-bug-fix ./mychart helm install --name web-final-version ./mychart |
|
Overwrides a setting | helm install --name web1 ./mychart/ --set service.type=NodePort |
|
Linter | helm lint ./mychart |
|
Packages a chart | helm package mychart/ |
|
Creating a local repo | 1. Open up local repo relative to charts directory |
|
Adding dependencies | 1. Create requirements.yml (same level as values.yml) dependencies: - name: mariadb version: 0.6.0 repository: https://kubernetes-charts.storage.googleapis.com |
|
Creating secrets.yaml using Helm secrets | This assumes that you already have gpg and helm secrets plugin installed.
1. Create a simple gpg kay pair that never expires.
workstation ~$ gpg --gen-key gpg (GnuPG) 1.4.23; Copyright (C) 2015 Free Software Foundation, Inc. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.
gpg: directory `/home/merrell/.gnupg' created gpg: new configuration file `/home/merrell/.gnupg/gpg.conf' created gpg: WARNING: options in `/home/merrell/.gnupg/gpg.conf' are not yet active during this run gpg: keyring `/home/merrell/.gnupg/secring.gpg' created gpg: keyring `/home/merrell/.gnupg/pubring.gpg' created Please select what kind of key you want: (1) RSA and RSA (default) (2) DSA and Elgamal (3) DSA (sign only) (4) RSA (sign only) Your selection? RSA keys may be between 1024 and 4096 bits long. What keysize do you want? (2048) Requested keysize is 2048 bits Please specify how long the key should be valid. 0 = key does not expire <n> = key expires in n days <n>w = key expires in n weeks <n>m = key expires in n months <n>y = key expires in n years Key is valid for? (0) Key does not expire at all Is this correct? (y/N) y
You need a user ID to identify your key; the software constructs the user ID from the Real Name, Comment and Email Address in this form: "Heinrich Heine (Der Dichter) <heinrichh@duesseldorf.de>"
Real name: helmsecrets Email address: helmsecrets@nxdomain.com Comment: You selected this USER-ID: "helmsecrets <helmsecrets@nxdomain.com>"
Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O You need a Passphrase to protect your secret key.
You don't want a passphrase - this is probably a *bad* idea! I will do it anyway. You can change your passphrase at any time, using this program with the option "--edit-key".
We need to generate a lot of random bytes. It is a good idea to perform some other action (type on the keyboard, move the mouse, utilize the disks) during the prime generation; this gives the random number generator a better chance to gain enough entropy. .....+++++ ...+++++ We need to generate a lot of random bytes. It is a good idea to perform some other action (type on the keyboard, move the mouse, utilize the disks) during the prime generation; this gives the random number generator a better chance to gain enough entropy. ....+++++ ......+++++ gpg: /home/merrell/.gnupg/trustdb.gpg: trustdb created gpg: key 22CA508A marked as ultimately trusted public and secret key created and signed.
gpg: checking the trustdb gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u pub 2048R/22CA508A 2019-06-26 Key fingerprint = 5C81 EB08 BFCA 45C2 2491 4F6C A8D9 3ECA 22CA 508A uid helmsecrets <helmsecrets@nxdomain.com> sub 2048R/96A3E515 2019-06-26
workstation ~$
2. Have a look at the key pair generated.
workstation ~$ gpg --list-keys /home/merrell/.gnupg/pubring.gpg -------------------------------- pub 2048R/22CA508A 2019-06-26 uid helmsecrets <helmsecrets@nxdomain.com> sub 2048R/96A3E515 2019-06-26 workstation ~$ workstation ~$ gpg --list-secret-keys /home/merrell/.gnupg/secring.gpg -------------------------------- sec 2048R/22CA508A 2019-06-26 uid helmsecrets <helmsecrets@nxdomain.com> ssb 2048R/96A3E515 2019-06-26
workstation ~$
3. Get the secret key fingerprint.
workstation ~$ gpg --fingerprint /home/merrell/.gnupg/pubring.gpg -------------------------------- pub 2048R/22CA508A 2019-06-26 Key fingerprint = 5C81 EC08 BFDA 45C2 2491 4E6C A8D9 3FCA 22CA 508A uid helmsecrets <helmsecrets@nxdomain.com> sub 2048R/96A3E515 2019-06-26
workstation ~$
4. Create .sops.yaml in the current directory and put the fingerprint there. Helm will use that fingerprint in encrypting the file we will create later.
workstation ~$ cat .sops.yaml creation_rules: - pgp: '5C81 EC08 BFDA 45C2 2491 4E6C A8D9 3FCA 22CA 508A' workstation ~$
5. Create the file you want to encrypt.
workstation ~$ cat secrets.yaml foo: bar hello: world workstation ~$
6. Encrypt the file using helm secrets. Now on the fun part, since helm secrets needs a tiller server, there are 2 methods on how to do this. Either of the methods below will work.
a. Point your ~/.kubeconfig to a cluster with running tiller server and encrypt the file.
workstation ~$ export KUBECONFIG=~/.kube-configs/innove-lowerenv-merrell workstation ~$ helm secrets enc secrets.yaml Encrypting secrets.yaml Encrypted secrets.yaml workstation ~$
b. Use helm tiller plugin so that you don't need to connect to a remote cluster.
workstation ~$ helm tiller run -- helm secrets enc secrets.yaml Installed Helm version v2.10.0 Installed Tiller version v2.10.0 Helm and Tiller are the same version! Starting Tiller... Tiller namespace: kube-system Running: helm secrets enc secrets.yaml
Encrypting secrets.yaml Encrypted secrets.yaml Stopping Tiller... workstation ~$
7. The file is now encrypted and looks like this.
workstation ~$ cat secrets.yaml foo: ENC[AES256_GCM,data:Ngwp,iv:rkCxM1Qfob5mFppMsHPfF+ujInore9xyoF72D4PIwqA=,tag:QtBbsjEOXSK6hR3pYhzFkg==,type:str] hello: ENC[AES256_GCM,data:jxOHyvg=,iv:LMcyz4rgh0Qzp7PInA3kyjJ7fTQM82Yx0Tsn99XVaE0=,tag:89FpMGZ9o8FFuJzp+ulqqQ==,type:str] sops: kms: [] gcp_kms: [] lastmodified: '2019-06-26T02:47:57Z' mac: ENC[AES256_GCM,data:jg1b7RayAzFn2RDEpCJsoGAXZh8DdE6vKhw2FLmmrlMa/OWe1X6FPADR9ltK3zK6GM5blUsYZfWMaldGNj6KiV1CUIwBsLFiyaSQ1IZ293PZnFPJ1r28NhMBdseqS/fXzOAlVV0fQEUOmRJbFrIag4V9P8HBnUNnLfhnicx73Og=,iv:yH4WCqasJUDa6J/bajIpwu4MQgb0Hz7bYnOkLEkNgKw=,tag:lqL9gywxlMrO6qX+Lb+f4w==,type:str] pgp: - created_at: '2019-06-26T02:47:57Z' enc: |- -----BEGIN PGP MESSAGE-----
wcBMA2HaF/mWo+UVAQgAwu9zc7+PM/fg3isBIpJWHLGvuGLqpaurluTZKqJCY/mr p8b/CUYtIxJqnrn94jOiZ39Zrqqz53RpCr6oHpk91t0mXH5N5GPP6PKdj9tgUvc/ e8ny2RsdTkh7WbH8W3TnxeSEU3nTg/aZUzwBRQJNGlLogcnnPhvWhdwWbhDPCpzb mMZaEymOhfgLUivrJQ68zPkCd00JMaHjGT9qSwx3iE9SxnJXh71T7wD3jjGagEDe rNjRF92LpMByJNQ/0zGrwztxzrPQqrU1GfllXSWXpZfPtICQjrPH2qn4q+MIuibc VLX5ga2L6cr7D+VWXV4ci08ETwBLGzxPrBJ5k5sc3dLgAeSGlPYqxfXMYa2l8dLn W6Ob4aLd4DngVuG8l+B/4p4c8FjgqOXN2xtXTXxDDMYIClKfWQhiedYKkOq618x1 Z+ZFDVyTSuAM5P/XMarYdQiRTDD6mkZnZwziFaS2yeEepwA= =wlEH -----END PGP MESSAGE----- fp: 5C81EB08BFCA45C224914F6CA8D93ECA22CA508A unencrypted_suffix: _unencrypted version: 3.0.3 workstation ~$ |
|
Editing secrets.yaml using Helm secrets | 1. Make sure you have the correct gpg key pair. If someone created the ecnrypted file, ask him to export his gpg keys (gpg --export-secret-keys) and import it to your workstation (gpg --import).
gpg --list-keys gpg --list-secret-keys
2. Edit the file (you may also use helm tiller plugin alongside w/ helm secrets)
helm secrets edit secrets.yaml |
|
Plugins
-------
Helm Secrets |
# Encrypts a file (you need to first put the fingerprint of private key you will use to encrypt inside .sops.yaml) helm secrets enc secrets.yaml
# Installing helm plugin install https://github.com/futuresimple/helm-secrets
# Decrypts secret first and upgrade a release helm secrets upgrade --install --namespace=nexus-stg -f values.yaml -f secrets.yaml nexus-stg . |
|
Helm Tiller | Tillerless Helm
helm tiller run -- helm ls helm tiller run -- bash -c "helm upgrade --install sample stable/tomcat" |
|
Troubleshooting
---------------
No permission | Issue: This appears during "helm ls":
kubectl create serviceaccount --namespace kube-system tiller kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kube-system:tiller kubectl patch deploy --namespace kube-system tiller-deploy -p '{"spec":{"template":{"spec":{"serviceAccount":"tiller"}}}}' helm init --service-account tiller --upgrade |
Sources
-------
Helm tutorials | |
Creating charts | |
Helm 3 | |
Helm Security | |
Helm template function |
No comments:
Post a Comment