基于 Go 的 Kubernetes Operator 入门（一）：环境配置

1、准备一台全新的 Linux 服务器

这台 Linux 服务器用来配置基于 Go 的 Kubernetes Operator 所需要的依赖环境。这里用的是一台云服务商的 Ubuntu 18.04 轻量服务器，配置为 CPU/内存/磁盘：2 核/2 GB/80 GB，地域为中国香港（避免安装依赖环境时出现网络超时问题，否则的话要配置代理科学上网）。

机器配置信息

2、安装 Git 、Go 、Docker 环境

2.1 安装 Git

# apt-get update
# apt-get install git

2.2 安装 Go，这里安装 Go 1.15 版本

# wget https://golang.org/dl/go1.15.11.linux-amd64.tar.gz
# tar xvf go1.15.11.linux-amd64.tar.gz -C /usr/local/
# cat >> /etc/profile << EOF
### go env
export GO111MODULE=on
export GOROOT=/usr/local/go
export GOPROXY=https://goproxy.cn,direct
export PATH=\$PATH:\$GOROOT/bin
EOF
# source /etc/profile
# go version
go version go1.15.11 linux/amd64

2.3 安装 Docker

# apt-get update
# apt-get install apt-transport-https ca-certificates curl gnupg lsb-release
# curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
# echo "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null
# apt-get update
# apt-get install docker-ce docker-ce-cli containerd.io
// 检查 Docker 是否运行了。
# systemctl status docker

3、使用 minikube 安装单机版的 Kubernetes 环境

# curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
# install minikube-linux-amd64 /usr/local/bin/minikube
// 启动一个 单机版的 Kubernetes 环境
# minikube start --force --driver=docker
# minikube kubectl -- get pods -A
NAMESPACE     NAME                               READY   STATUS    RESTARTS   AGE
kube-system   coredns-74ff55c5b-wfwq2            1/1     Running   0          113s
kube-system   etcd-minikube                      1/1     Running   0          2m1s
kube-system   kube-apiserver-minikube            1/1     Running   0          2m1s
kube-system   kube-controller-manager-minikube   1/1     Running   0          2m1s
kube-system   kube-proxy-6lbzr                   1/1     Running   0          113s
kube-system   kube-scheduler-minikube            1/1     Running   0          2m1s
kube-system   storage-provisioner                1/1     Running   0          2m6s

4、安装 kubectl 来连接 Kubernetes

# curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
# install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
# kubectl cluster-info
Kubernetes control plane is running at https://192.168.49.2:8443
KubeDNS is running at https://192.168.49.2:8443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
# kubectl get pods -A
NAMESPACE     NAME                               READY   STATUS    RESTARTS   AGE
kube-system   coredns-74ff55c5b-wfwq2            1/1     Running   0          5m40s
kube-system   etcd-minikube                      1/1     Running   0          5m48s
kube-system   kube-apiserver-minikube            1/1     Running   0          5m48s
kube-system   kube-controller-manager-minikube   1/1     Running   0          5m48s
kube-system   kube-proxy-6lbzr                   1/1     Running   0          5m40s
kube-system   kube-scheduler-minikube            1/1     Running   0          5m48s
kube-system   storage-provisioner                1/1     Running   0          5m53s

5. 安装 operator-sdk 来开发基于 Go 的 Kubernetes Operator

# export ARCH=$(case $(arch) in x86_64) echo -n amd64 ;; aarch64) echo -n arm64 ;; *) echo -n $(arch) ;; esac)
# export OS=$(uname | awk '{print tolower($0)}')
# export OPERATOR_SDK_DL_URL=https://github.com/operator-framework/operator-sdk/releases/latest/download
# curl -LO ${OPERATOR_SDK_DL_URL}/operator-sdk_${OS}_${ARCH}
# chmod +x operator-sdk_${OS}_${ARCH} && mv operator-sdk_${OS}_${ARCH} /usr/local/bin/operator-sdk
# operator-sdk version
operator-sdk version: "v1.5.0", commit: "98f30d59ade2d911a7a8c76f0169a7de0dec37a0", kubernetes version: "1.19.4", go version: "go1.15.5", GOOS: "linux", GOARCH: "amd64"

6. 准备一个 Docker Registry 镜像仓库（这里使用阿里云 Docker Registry），用来推送 operator 到镜像仓库

// 要先在阿里云 Docker Registry 创建命名空间 operator-dev，仓库为 memcached-operator
// 创建 Docker 访问镜像仓库的凭据
# docker login --username=xxx registry.cn-hongkong.aliyuncs.com
# cat /root/.docker/config.json 
{
    "auths": {
        "registry.cn-hongkong.aliyuncs.com": {
            "auth": "xxxxxx"
        }
    }
// 创建 Kubernetes default 命名空间访问镜像仓库的凭据
// 注意：等下 memcached-operator-controller 是在不同的命名空间运行的，也要单独创建访问镜像仓库的凭据
# kubectl create secret docker-registry aliyun-registry-secret --from-file=.dockerconfigjson=/root/.docker/config.json
# kubectl get serviceaccounts
NAME      SECRETS   AGE
default   1         63m
// 删除了 metadata 的 resourceVersion 字段。添加了 imagePullSecrets 字段，name 为上述添加的 secret aliyun-registry-secret
# kubectl edit serviceaccounts default
apiVersion: v1
kind: ServiceAccount
metadata:
  creationTimestamp: "2021-04-10T07:20:19Z"
  name: default
  namespace: default
  uid: a4ce6cdf-cb9f-4bf8-9fc7-704d796ec95d
secrets:
- name: default-token-76z56
imagePullSecrets:
- name: aliyun-registry-secret

7、使用 operator-sdk 构建一个 memcached-operator 示例

# mkdir memcached-operator
# cd memcached-operator
// 使用 operator-sdk 初始化一个 operator 项目（repo 可以指定为自己的仓库地址）
# operator-sdk init --domain github.com --repo github.com/ackfin/memcached-operator
// 创建一个 Memcached API
# operator-sdk create api --group cache --version v1alpha1 --kind Memcached --resource --controller
// 把项目工程制作成镜像，并推送到镜像仓库
# export OPERATOR_IMG="registry.cn-hongkong.aliyuncs.com/operator-dev/memcached-operator:v0.0.1"
// Ubuntu 系统需要编辑下 Makefile，添加 SHELL := /bin/bash 参数到文件中
# vim Makefile
SHELL := /bin/bash
# make docker-build docker-push IMG=$OPERATOR_IMG
// 可以看到构建后的镜像
# docker images | grep memcached-operator
registry.cn-hongkong.aliyuncs.com/operator-dev/memcached-operator   v0.0.1    cc6acc83f0d9   46 seconds ago   46.5MB

8、安装 memcached-operator 到 Kubernetes 中

# cd memcached-operator
# export OPERATOR_IMG="registry.cn-hongkong.aliyuncs.com/operator-dev/memcached-operator:v0.0.1"
# make deploy IMG=$OPERATOR_IMG
// 看到拉取 memcached-operator 镜像失败了
# kubectl get pods -A | grep memcached-operator
memcached-operator-system   memcached-operator-controller-manager-6b687c6579-bmwkr   1/2     ImagePullBackOff   0          81s
// 创建 Kubernetes memcached-operator-system 命名空间访问镜像仓库的凭据
# kubectl create secret docker-registry aliyun-registry-secret --from-file=.dockerconfigjson=/root/.docker/config.json -n memcached-operator-system
// 删除了 metadata 的 resourceVersion 字段。添加了 imagePullSecrets 字段，name 为上述添加的 secret aliyun-registry-secret
# kubectl edit serviceaccounts default -n memcached-operator-system
apiVersion: v1
kind: ServiceAccount
metadata:
  creationTimestamp: "2021-04-10T09:05:00Z"
  name: default
  namespace: memcached-operator-system
  uid: 5dce023b-2aa9-497e-81d5-d348978ff68e
secrets:
- name: default-token-9kg8j
imagePullSecrets:
- name: aliyun-registry-secret
// 删除原来的 memcached-operator ，会自动创建新 memcached-operator
# kubectl delete pods memcached-operator-controller-manager-6b687c6579-bmwkr -n memcached-operator-system
# kubectl get pods -n memcached-operator-system
NAME                                                     READY   STATUS    RESTARTS   AGE
memcached-operator-controller-manager-6b687c6579-j84cc   2/2     Running   0          15s

9、测试 memcached-operator 能否使用

# cd memcached-operator
# cat config/samples/cache_v1alpha1_memcached.yaml
apiVersion: cache.github.com/v1alpha1
kind: Memcached
metadata:
  name: memcached-sample
spec:
  # Add fields here
  foo: bar

// 可以看到能正常调用 
# kubectl apply -f config/samples/cache_v1alpha1_memcached.yaml
memcached.cache.github.com/memcached-sample created

参考：

1. operator-sdk 依赖要求：https://sdk.operatorframework.io/docs/building-operators/golang/installation/
2. Go 安装：https://golang.org/doc/install
3. Ubantu Docker 安装：https://docs.docker.com/engine/install/ubuntu/
4. minikube 安装：https://minikube.sigs.k8s.io/docs/start/
5. kubectl 安装：https://kubernetes.io/zh/docs/tasks/tools/install-kubectl-linux/
6. operator-sdk 安装： https://sdk.operatorframework.io/docs/installation/
7. operator-sdk 入门示例：https://sdk.operatorframework.io/docs/building-operators/golang/quickstart/
8. Kubernetes 从私有仓库拉取镜像：https://kubernetes.io/zh/docs/tasks/configure-pod-container/pull-image-private-registry/

有疑问加站长微信联系（非本文作者）