如何在kubernete集群上部署springboot应用

作者 |  朱晋君
来源 | 君哥聊技术（ID：gh_1f109b82d301）

在之前的文章中，我讲了使用kubeadm从0到1搭建kubernete集群，今天我们来聊一下如何在这套k8s集群上部署springboot应用。首先说明一下，我们这次集群的网络使用flannel网络，master节点启动命令如下:

kubeadm init --image-repository registry.aliyuncs.com/google_containers --kubernetes-version=v1.17.3 --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address 192.168.59.132

上面的参数 --pod-network-cidr=10.244.0.0/16

就是指明集群使用flannel网络，切记，不然部署springboot后，容器访问外面网络会有问题。

再来看一下这次实验使用的集群环境，如下图，我在个人笔记本上安装了mysql数据库，ip地址是192.168.59.1，同时安装了2个vmware虚拟机，ip地址分别是192.168.59.132和192.168.59.138，kubernete集群就部署在这2个虚机上，192.168.59.132部署着master节点，192.168.59.138部署着worker1节点

这次使用的springboot源码地址：

https://github.com/jinjunzhu/spring-boot-mybatis

1.打包springboot镜像

这儿遇到问题可以查看我之前的文章Linux下docker制作springboot应用镜像

编译jar包

mvn clean -Dmaven.test.skip=true install

编译后上传到虚机，编写DockerFile文件

FROM openjdk:8-jdk-alpine
VOLUME /tmp
ADD spring-boot-mybatis-1.0-SNAPSHOT.jar springboot-mybatis.jar
ENTRYPOINT ["java","-jar","springboot-mybatis.jar"]

执行打包镜像命令，这里我打包的版本号是1.2，每次修改代码后都要升级版本号，不然上传dockerhub不会更新

docker build -t zjj2006forever/springboot-mybatis:1.2 .

之后下面2个命令登录dockerhub并且上传

docker login
docker push zjj2006forever/springboot-mybatis:1.2
[root@master k8s]# docker push zjj2006forever/springboot-mybatis:1.2
The push refers to a repository [docker.io/zjj2006forever/springboot-mybatis]
a3ce9cf52624: Pushed 
ceaf9e1ebef5: Pushed 
9b9b7f3d56a0: Pushed 
f1b5933fe4b5: Pushed 
1.0: digest: sha256:8d37516e910515b0e336ecc29befaa0f9240ebd92a3c22af8b90846476644420 size: 1159

2.在kubernete上发布镜像

编写yaml文件，如下：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: springboot-mybatis-deployment
spec:
  selector:
    matchLabels:
      app: springboot-mybatis
  replicas: 2
  template:
    metadata:
      labels:
        app: springboot-mybatis
    spec:
      containers:
      - name: spingboot-mybatis
        imagePullPolicy: IfNotPresent
        #imagePullPolicy: Never
        image: zjj2006forever/springboot-mybatis:1.2
        ports:
        - containerPort: 8300

上面的yaml文件中，我指定了镜像名称是我刚刚打包的镜像：image: zjj2006forever/springboot-mybatis:1.2，副本数量是2(replicas: 2)，

这样部署的时候会生成2个pod，如果提供服务的时候如果一个pod挂了，集群会再创建一个新的出来，一直保持有2个pod。

kubectl apply -f springboot-mybatis.yaml

输出下面提示后我们看一下pod状态

deployment.apps/springboot-mybatis-deployment created

[root@master k8s]# kubectl get pods --all-namespaces
NAMESPACE     NAME                                            READY   STATUS    RESTARTS   AGE
default       springboot-mybatis-deployment-9f5b78b76-2qtfg   1/1     Running   0          9s
default       springboot-mybatis-deployment-9f5b78b76-tzf4x   1/1     Running   0          9s
kube-system   coredns-9d85f5447-7k44c                         1/1     Running   0          30m
kube-system   coredns-9d85f5447-sh24j                         1/1     Running   0          30m
kube-system   etcd-master                                     1/1     Running   0          30m
kube-system   kube-apiserver-master                           1/1     Running   0          30m
kube-system   kube-controller-manager-master                  1/1     Running   0          30m
kube-system   kube-flannel-ds-amd64-rd842                     1/1     Running   0          16m
kube-system   kube-flannel-ds-amd64-tdkwx                     1/1     Running   0          16m
kube-system   kube-proxy-d6qqg                                1/1     Running   0          25m
kube-system   kube-proxy-pm7cc                                1/1     Running   0          30m
kube-system   kube-scheduler-master                           1/1     Running   0          30m

再看一下pod启动日志

[root@master k8s]# kubectl logs -f springboot-mybatis-deployment-9f5b78b76-2qtfg

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.1.6.RELEASE)

2020-06-13 08:25:15,002 [main] [INFO] boot.Application - Starting Application v1.0-SNAPSHOT on springboot-mybatis-deployment-9f5b78b76-2qtfg with PID 1 (/springboot-mybatis.jar started by root in /)
2020-06-13 08:25:15,006 [main] [INFO] boot.Application - The following profiles are active: dev
2020-06-13 08:25:20,545 [main] [INFO] org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker - Bean 'org.springframework.transaction.annotation.ProxyTransactionManagementConfiguration' of type [org.springframework.transaction.annotation.ProxyTransactionManagementConfiguration$$EnhancerBySpringCGLIB$$c636a19f] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-06-13 08:25:21,688 [main] [INFO] org.springframework.boot.web.embedded.tomcat.TomcatWebServer - Tomcat initialized with port(s): 8083 (http)
2020-06-13 08:25:21,723 [main] [INFO] org.apache.coyote.http11.Http11NioProtocol - Initializing ProtocolHandler ["http-nio-8083"]
2020-06-13 08:25:21,839 [main] [INFO] org.apache.catalina.core.StandardService - Starting service [Tomcat]
2020-06-13 08:25:21,840 [main] [INFO] org.apache.catalina.core.StandardEngine - Starting Servlet engine: [Apache Tomcat/9.0.21]
2020-06-13 08:25:22,260 [main] [INFO] org.apache.catalina.core.ContainerBase.[Tomcat].[localhost].[/] - Initializing Spring embedded WebApplicationContext
2020-06-13 08:25:22,261 [main] [INFO] org.springframework.web.context.ContextLoader - Root WebApplicationContext: initialization completed in 6985 ms
2020-06-13 08:25:25,220 [main] [INFO] springfox.documentation.spring.web.PropertySourcedRequestMappingHandlerMapping - Mapped URL path [/v2/api-docs] onto method [public org.springframework.http.ResponseEntity<springfox.documentation.spring.web.json.Json> springfox.documentation.swagger2.web.Swagger2Controller.getDocumentation(java.lang.String,javax.servlet.http.HttpServletRequest)]
2020-06-13 08:25:25,631 [main] [INFO] org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor - Initializing ExecutorService 'applicationTaskExecutor'
2020-06-13 08:25:27,595 [main] [INFO] springfox.documentation.spring.web.plugins.DocumentationPluginsBootstrapper - Context refreshed
2020-06-13 08:25:27,710 [main] [INFO] springfox.documentation.spring.web.plugins.DocumentationPluginsBootstrapper - Found 1 custom documentation plugin(s)
2020-06-13 08:25:27,857 [main] [INFO] springfox.documentation.spring.web.scanners.ApiListingReferenceScanner - Scanning for api listing references
2020-06-13 08:25:28,755 [main] [INFO] org.apache.coyote.http11.Http11NioProtocol - Starting ProtocolHandler ["http-nio-8083"]
2020-06-13 08:25:28,956 [main] [INFO] org.springframework.boot.web.embedded.tomcat.TomcatWebServer - Tomcat started on port(s): 8083 (http) with context path ''
2020-06-13 08:25:29,253 [main] [INFO] org.springframework.boot.web.embedded.tomcat.TomcatWebServer - Tomcat initialized with port(s): 18082 (http)
2020-06-13 08:25:29,255 [main] [INFO] org.apache.coyote.http11.Http11NioProtocol - Initializing ProtocolHandler ["http-nio-127.0.0.1-18082"]
2020-06-13 08:25:29,289 [main] [INFO] org.apache.catalina.core.StandardService - Starting service [Tomcat]
2020-06-13 08:25:29,289 [main] [INFO] org.apache.catalina.core.StandardEngine - Starting Servlet engine: [Apache Tomcat/9.0.21]
2020-06-13 08:25:29,304 [main] [INFO] org.apache.catalina.core.ContainerBase.[Tomcat-1].[localhost].[/] - Initializing Spring embedded WebApplicationContext
2020-06-13 08:25:29,305 [main] [INFO] org.springframework.web.context.ContextLoader - Root WebApplicationContext: initialization completed in 336 ms
2020-06-13 08:25:29,389 [main] [INFO] org.springframework.boot.actuate.endpoint.web.EndpointLinksResolver - Exposing 16 endpoint(s) beneath base path '/actuator'
2020-06-13 08:25:29,659 [main] [INFO] org.apache.coyote.http11.Http11NioProtocol - Starting ProtocolHandler ["http-nio-127.0.0.1-18082"]
2020-06-13 08:25:29,692 [main] [INFO] org.springframework.boot.web.embedded.tomcat.TomcatWebServer - Tomcat started on port(s): 18082 (http) with context path ''
2020-06-13 08:25:29,696 [main] [INFO] boot.Application - Started Application in 17.382 seconds (JVM running for 21.29)

上面的结果可以看到我刚刚发布的2个springboot的pod启动成功了，我们再看下其中一个pod的状态，执行如下命令

kubectl describe pod springboot-mybatis-deployment-9f5b78b76-2qtfg

[root@master k8s]# kubectl describe pod springboot-mybatis-deployment-9f5b78b76-2qtfg
Name:         springboot-mybatis-deployment-9f5b78b76-2qtfg
Namespace:    default
Priority:     0
Node:         worker1/192.168.59.138
Start Time:   Sat, 13 Jun 2020 04:25:04 -0400
Labels:       app=springboot-mybatis
              pod-template-hash=9f5b78b76
Annotations:  <none>
Status:       Running
IP:           10.244.1.2
IPs:
  IP:           10.244.1.2
Controlled By:  ReplicaSet/springboot-mybatis-deployment-9f5b78b76
Containers:
  spingboot-mybatis:
    Container ID:   docker://f3b0b7d1e2b9da0970e23134bbece26e4eb4fb0416e303119a0c4a34b954b666
    Image:          zjj2006forever/springboot-mybatis:1.2
    Image ID:       docker-pullable://zjj2006forever/springboot-mybatis@sha256:0e6f78256ab70ed5244042abc2a5bd5154a193fb9f97f05a3061e8334d387986
    Port:           8300/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Sat, 13 Jun 2020 04:25:08 -0400
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-cjp27 (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  default-token-cjp27:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-cjp27
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason     Age    From               Message
  ----    ------     ----   ----               -------
  Normal  Scheduled  3m53s  default-scheduler  Successfully assigned default/springboot-mybatis-deployment-9f5b78b76-2qtfg to worker1
  Normal  Pulled     3m50s  kubelet, worker1   Container image "zjj2006forever/springboot-mybatis:1.2" already present on machine
  Normal  Created    3m50s  kubelet, worker1   Created container spingboot-mybatis
  Normal  Started    3m49s  kubelet, worker1   Started container spingboot-mybatis

上面的输出看到这个pod被调度到了worker1节点上，同时从上面的Events可以看到pod的启动过程。

3.测试

先让我们看一下我们刚刚部署的2个pod

[root@master k8s]# kubectl get pod -l app -o wide
NAME                                             READY   STATUS    RESTARTS   AGE     IP           NODE      NOMINATED NODE   READINESS GATES
springboot-mybatis-deployment-5b78f66997-2hhdq   1/1     Running   0          7m23s   10.244.1.4   worker1   <none>           <none>
springboot-mybatis-deployment-5b78f66997-k87dx   1/1     Running   0          6m42s   10.244.1.5   worker1   <none>           <none>

上面的输出结果中可以看到，这2个pod的ip地址是10.244.1.4和10.244.1.5，这是由flannel网络插件分配的。

接着我们进入容器内部看一下网络连通性

kubectl exec -it springboot-mybatis-deployment-5b78f66997-2hhdq -- /bin/sh

我们分别ping集群的主192.168.59.132，从节点192.168.59.138和mysql数据库地址192.168.59.1，发现都是正常返回数据包的

我们用curl看一下健康检查的地址，返回如下

[root@worker1 ~]# curl http://10.244.1.4:18082/actuator/health
{"status":"UP","details":{"db":{"status":"UP","details":{"firstDataSource":{"status":"UP","details":{"database":"MySQL","hello":1}},"secondDataSource":{"status":"UP","details":{"database":"MySQL","hello":1}}}},"diskSpace":{"status":"UP","details":{"total":18238930944,"free":10361712640,"threshold":10485760}}}}

我们用浏览器输入地址http://10.244.1.4:8083/employee/jinjunzhu，返回505

这时在容器里面查看日志,输出见下面截图

cd /app/jar/logs/springboot-mybatis-deployment-5b78f66997-2hhdq/applog
tail -f catalina.out

这个原因很明显，mysql没有对这个ip访问授权，执行一下授权命令：

//对ip地址192.168.59.132授权
grant all privileges on *.* to 'root'@'192.168.59.132' identified by '123456';
//是授权立即生效
flush privileges;

再次访问上面的url，成功，见下图

pod中的日志如下：