ksonnetでkubernetesのmanifestを環境ごとに生成/applyする
kubernetesksonnetはJSONのテンプレートエンジンjsonnetからk8sのmanifestを環境ごとに生成してapplyするツール。kubeflowでも使われている。
追記(2020-11-22): 既に開発が終了しリポジトリもアーカイブされている。 kubeflowはkustomizeに移行した。
$ brew install ksonnet/tap/ks
$ ks version
ksonnet version: 0.11.0
jsonnet version: v0.10.0
client-go version:
init
まずks initしてディレクトリを作成する。
$ kubectl config current-context
minikube
$ ks init kstest
$ cd kstest
$ ls
app.yaml components environments lib vendor
$ cat app.yaml
apiVersion: 0.1.0
environments:
default:
destination:
namespace: default
server: https://192.168.99.100:8443
k8sVersion: v1.10.0
path: default
kind: ksonnet.io/app
name: kstest
registries:
incubator:
gitVersion:
commitSha: 40285d8a14f1ac5787e405e1023cf0c07f6aa28c
refSpec: master
protocol: github
uri: github.com/ksonnet/parts/tree/master/incubator
version: 0.0.1
Prototypeのインストール
app.yamlにあるように最初はincubatorというregistryが登録されている。追加する場合はks registry add。
ks pkg listでregistryのpackage一覧を見て、ks pkg installするとpackageに含まれるいくつかのprototypeを持ってくることができる。
prototypeというのはparameterを足してcomponentになるもと。
# ks registry add *** github.com/***
$ ks pkg list
REGISTRY NAME INSTALLED
======== ==== =========
incubator apache
incubator efk
incubator mariadb
incubator memcached
incubator mongodb
incubator mysql
incubator nginx
incubator node
incubator postgres
incubator redis
incubator tomcat
$ ks pkg install incubator/redis@master
$ ks prototype list
NAME DESCRIPTION
==== ===========
...
io.ksonnet.pkg.redis-all-features A Redis deployment with metrics, ingress, and persistent storage.
io.ksonnet.pkg.redis-persistent A simple Redis deployment, backed by persistent storage.
io.ksonnet.pkg.redis-stateless A simple, stateless Redis deployment,
$ ls vendor/incubator/redis/
README.md examples parts.yaml prototypes redis.libsonnet
Componentの生成
ks generateでprototypeからcomponentを生成する。deployed-serviceは最初から入っているprototype。
$ ks generate deployed-service myapp --image gcr.io/google-samples/hello-app:2.0 --type ClusterIP
$ ks generate redis-stateless redis
$ ls components/
myapp.jsonnet params.libsonnet redis.jsonnet
jsonnetではparamsを参照しているが、environmentごとの設定がなければcomponentsのparams.libsonnetの値になる。
$ cat components/myapp.jsonnet
local env = std.extVar("__ksonnet/environments");
local params = std.extVar("__ksonnet/params").components.myapp;
[
{
"apiVersion": "v1",
"kind": "Service",
"metadata": {
"name": params.name
},
"spec": {
"ports": [
{
"port": params.servicePort,
"targetPort": params.containerPort
}
],
"selector": {
"app": params.name
},
"type": params.type
}
},
{
"apiVersion": "apps/v1beta2",
"kind": "Deployment",
"metadata": {
"name": params.name
},
"spec": {
"replicas": params.replicas,
"selector": {
"matchLabels": {
"app": params.name
},
},
"template": {
"metadata": {
"labels": {
"app": params.name
}
},
"spec": {
"containers": [
{
"image": params.image,
"name": params.name,
"ports": [
{
"containerPort": params.containerPort
}
]
}
]
}
}
}
}
]
$ cat components/params.libsonnet
{
global: {
// User-defined global parameters; accessible to all component and environments, Ex:
// replicas: 4,
},
components: {
// Component-level parameters, defined initially from 'ks prototype use ...'
// Each object below should correspond to a component in the components/ directory
redis: {
name: "redis",
redisPassword: "null",
},
myapp: {
containerPort: 80,
image: "gcr.io/google-samples/hello-app:2.0",
name: "myapp",
replicas: 1,
servicePort: 80,
type: "ClusterIP",
},
},
}
Environmentの追加
Environmentごとに向き先を変えて、異なるパラメータでapplyできる。
$ ks env add prd -n prd
$ ks env list
NAME OVERRIDE KUBERNETES-VERSION NAMESPACE SERVER
==== ======== ================== ========= ======
default v1.10.0 default https://192.168.99.100:8443
prd v1.10.0 prd https://192.168.99.100:8443
--serverフラグでSERVERを指定できる。対象クラスタのIPはkubectl config viewで分かる。
ks param setでパラメータを設定できる。これがcomponentsのparams.libsonnetのものより優先される。
$ ks param set myapp replicas 2 --env prd
$ cat environments/prd/params.libsonnet
local params = std.extVar('__ksonnet/params');
local globals = import 'globals.libsonnet';
local envParams = params + {
components+: {
myapp+: {
replicas: 2,
},
},
};
{
components: {
[x]: envParams.components[x] + globals
for x in std.objectFields(envParams.components)
},
}
yamlマニフェストを表示
$ ks show prd
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
labels:
app: redis
name: redis
spec:
template:
metadata:
labels:
app: redis
spec:
containers:
- env:
- name: ALLOW_EMPTY_PASSWORD
value: "yes"
image: bitnami/redis:3.2.9-r2
imagePullPolicy: IfNotPresent
livenessProbe:
exec:
command:
- redis-cli
- ping
initialDelaySeconds: 30
timeoutSeconds: 5
name: redis
ports:
- containerPort: 6379
name: redis
readinessProbe:
exec:
command:
- redis-cli
- ping
initialDelaySeconds: 5
timeoutSeconds: 1
resources:
requests:
cpu: 100m
memory: 256Mi
volumeMounts:
- mountPath: /bitnami/redis
name: redis-data
volumes:
- name: redis-data
---
apiVersion: v1
kind: Service
metadata:
labels:
app: redis
name: redis
spec:
ports:
- name: redis
port: 6379
targetPort: redis
selector:
app: redis
---
apiVersion: v1
kind: Service
metadata:
name: myapp
spec:
ports:
- port: 80
targetPort: 80
selector:
app: myapp
type: ClusterIP
---
apiVersion: apps/v1beta2
kind: Deployment
metadata:
name: myapp
spec:
replicas: 2
selector:
matchLabels:
app: myapp
template:
metadata:
labels:
app: myapp
spec:
containers:
- image: gcr.io/google-samples/hello-app:2.0
name: myapp
ports:
- containerPort: 80
apply
$ ks apply default -c myapp