软创互联

Gitlab-runner对接k8s


#1

准备gitlab-runner的token

  • 启动一个runner
docker run -it --entrypoint /bin/bash gitlab/gitlab-runner:latest
  • 运行注册命令
gitlab-runner register
Runtime platform                                    arch=amd64 os=linux pid=26 revision=4745a6f3 version=11.8.0
Running in system-mode.

Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/):
https://gitlab.gugud.com/
Please enter the gitlab-ci token for this runner:
[YOUR-TOKEN]
Please enter the gitlab-ci description for this runner:
[gitlab-runner-5c57b85dcf-mvzh9]: k8s-runner
Please enter the gitlab-ci tags for this runner (comma separated):
k8s-runner-test
Registering runner... succeeded                     runner=9upxqvem
Please enter the executor: docker-ssh, shell, ssh, virtualbox, docker+machine, docker-ssh+machine, docker, parallels, kubernetes:
kubernetes
Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!

[YOUR-TOKEN]请找gitlab的管理员获取

  • 获取token
cat /etc/gitlab-runner/config.toml

concurrent = 1
check_interval = 0

[[runners]]
  name = "temp runner"
  url = "https://gitlab.gugud.com/"
  token = [TOKEN]
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "busybox:latest"
    privileged = false
    disable_cache = false
    volumes = ["/cache"]
    shm_size = 0
  [runners.cache]

出于安全考虑,此处我用[token]替代了我实际的token, 请保留你的token

在Kubernetes创建runner

  • 创建新的namespace

gitlab-runner会创建很多的ci容器, 需与其他的环境区分开来, 我们新建一个namespace单独用于ci

kubectl create namespace gitlab-runner
  • 创建具备pods等权限的ServiceAccunt
[root@ip-172-31-39-134 [master] gitlab-runner]# cat gitlab-runner-service-account.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: gitlab-admin
  namespace: gitlab-runner
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: gitlab-runner
  name: gitlab-admin
rules:
  - apiGroups: [""]
    resources: ["*"]
    verbs: ["*"]

---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: gitlab-admin
  namespace: gitlab-runner
subjects:
  - kind: ServiceAccount
    name: gitlab-admin
    namespace: gitlab-runner
roleRef:
  kind: Role
  name: gitlab-admin
  apiGroup: rbac.authorization.k8s.io
  • 创建支持dind(docker-in-docker)的runner config文件
[root@ip-172-31-39-134 [master] gitlab-runner]# cat gitlab-runner-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: gitlab-runner-config
  namespace: gitlab-runner
data:
  config.toml: |-
    concurrent = 4
    [[runners]]
      name = "Kubernetes Demo Runner"
      url = "https://gitlab.gugud.com/"
      token = "547d288cce510fdcc0e54519e6317b"
      executor = "kubernetes"
      [runners.kubernetes]
        namespace = "gitlab-runner"
        poll_timeout = 600
        cpu_request = "1"
        service_cpu_request = "200m"
        [[runners.kubernetes.volumes.host_path]]
            name = "docker"
            mount_path = "/var/run/docker.sock"
            host_path = "/var/run/docker.sock"
  • 创建runner的deploy文件
[root@ip-172-31-39-134 [master] gitlab-runner]# cat gitlab-runner-deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: gitlab-runner
  namespace: gitlab-runner
spec:
  replicas: 1
  selector:
    matchLabels:
      name: gitlab-runner
  template:
    metadata:
      labels:
        name: gitlab-runner
    spec:
      serviceAccountName: gitlab-admin
      containers:
        - args:
          - run
          image: gitlab/gitlab-runner:latest
          imagePullPolicy: Always
          name: gitlab-runner
          resources:
            requests:
              cpu: "100m"
            limits:
              cpu: "100m"
          volumeMounts:
            - name: config
              mountPath: /etc/gitlab-runner/config.toml
              readOnly: true
              subPath: config.toml
      volumes:
        - name: config
          configMap:
            name: gitlab-runner-config
      restartPolicy: Always
  • 启动
kubectl apply -f gitlab-runner-service-account.yaml
kubectl apply -f gitlab-runner-config.yaml
kubectl apply -f gitlab-runner-deployment.yaml

测试流程

  • 在gitlab的runner管理页面中将新注册的runner打一个特定tag
  • 准备测试代码
  • 准备.gitlab-ci.yml

类似如下:

image: docker.io/docker:dind

stages:
  - build
  - test

release_job:
  stage: test
  tags:
    - [TAG]
  script:
    - echo 'ho ho ho'

[TAG]请用你实际的tag替换

  • 在代码库中创建一个push操作
  • gitlab代码库中查看ci的pipeline信息,查看是否正常, 正常的结果类似如下:
Running with gitlab-runner 11.8.0 (4745a6f3)
  on Kubernetes Demo Runner 547d288c
Using Kubernetes namespace: gitlab-runner
Using Kubernetes executor with image docker.io/docker:dind ...
Waiting for pod gitlab-runner/runner-547d288c-project-264-concurrent-06mmrw to be running, status is Pending
Waiting for pod gitlab-runner/runner-547d288c-project-264-concurrent-06mmrw to be running, status is Pending
Waiting for pod gitlab-runner/runner-547d288c-project-264-concurrent-06mmrw to be running, status is Pending
Waiting for pod gitlab-runner/runner-547d288c-project-264-concurrent-06mmrw to be running, status is Pending
Waiting for pod gitlab-runner/runner-547d288c-project-264-concurrent-06mmrw to be running, status is Pending
Waiting for pod gitlab-runner/runner-547d288c-project-264-concurrent-06mmrw to be running, status is Pending
Waiting for pod gitlab-runner/runner-547d288c-project-264-concurrent-06mmrw to be running, status is Pending
Waiting for pod gitlab-runner/runner-547d288c-project-264-concurrent-06mmrw to be running, status is Pending
Waiting for pod gitlab-runner/runner-547d288c-project-264-concurrent-06mmrw to be running, status is Pending
Waiting for pod gitlab-runner/runner-547d288c-project-264-concurrent-06mmrw to be running, status is Pending
Waiting for pod gitlab-runner/runner-547d288c-project-264-concurrent-06mmrw to be running, status is Pending
Waiting for pod gitlab-runner/runner-547d288c-project-264-concurrent-06mmrw to be running, status is Pending
Waiting for pod gitlab-runner/runner-547d288c-project-264-concurrent-06mmrw to be running, status is Pending
Running on runner-547d288c-project-264-concurrent-06mmrw via gitlab-runner-5c57b85dcf-mvzh9...
Cloning repository...
Cloning into '/lafrinte/exporter'...
Checking out da10eb32 as test-drone...
Skipping Git submodules setup
$ echo 'ho ho ho'
ho ho ho
Job succeeded