

Kubernetes使用Keda进行弹性伸缩,更合理利用资源
source link: https://www.pkslow.com/archives/keda
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

Kubernetes自带的HPA是只支持CPU/MEM的,很多时候我们并不根据这两项指标来进行伸缩资源。比如消费者不断处理MQ的消息,我们希望MQ如果堆积过多,就启动更多的消费者来处理任务。而Keda
给了我们很多选择。
KEDA 是 Kubernetes 基于事件驱动的自动伸缩工具,通过 KEDA 我们可以根据需要处理的事件数量来驱动 Kubernetes 中任何容器的扩展。KEDA 可以直接部署到任何 Kubernetes 集群中和标准的组件一起工作。
Keda所支持的事件源非常丰富,本文我们以RabbitMQ为例进行演示。
2 安装Keda
安装的方法很多,我们直接通过yaml文件来安装,这样还可以修改镜像地址等。先从( https://github.com/kedacore/keda/releases/download/v2.2.0/keda-2.2.0.yaml )下载yaml文件,然后执行:
$ kubectl apply -f ~/Downloads/keda-2.2.0.yaml
namespace/keda created
customresourcedefinition.apiextensions.k8s.io/clustertriggerauthentications.keda.sh created
customresourcedefinition.apiextensions.k8s.io/scaledjobs.keda.sh created
customresourcedefinition.apiextensions.k8s.io/scaledobjects.keda.sh created
customresourcedefinition.apiextensions.k8s.io/triggerauthentications.keda.sh created
serviceaccount/keda-operator created
clusterrole.rbac.authorization.k8s.io/keda-external-metrics-reader created
clusterrole.rbac.authorization.k8s.io/keda-operator created
rolebinding.rbac.authorization.k8s.io/keda-auth-reader created
clusterrolebinding.rbac.authorization.k8s.io/keda-hpa-controller-external-metrics created
clusterrolebinding.rbac.authorization.k8s.io/keda-operator created
clusterrolebinding.rbac.authorization.k8s.io/keda:system:auth-delegator created
service/keda-metrics-apiserver created
deployment.apps/keda-metrics-apiserver created
deployment.apps/keda-operator created
apiservice.apiregistration.k8s.io/v1beta1.external.metrics.k8s.io created
检查一下是否都已经启动完成:
$ kubectl get all -n keda
NAME READY STATUS RESTARTS AGE
pod/keda-metrics-apiserver-55dc9f9498-smc2d 1/1 Running 0 2m41s
pod/keda-operator-59dcf989d6-pxcbb 1/1 Running 0 2m41s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/keda-metrics-apiserver ClusterIP 10.104.255.44 <none> 443/TCP,80/TCP 2m41s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/keda-metrics-apiserver 1/1 1 1 2m42s
deployment.apps/keda-operator 1/1 1 1 2m42s
NAME DESIRED CURRENT READY AGE
replicaset.apps/keda-metrics-apiserver-55dc9f9498 1 1 1 2m42s
replicaset.apps/keda-operator-59dcf989d6 1 1 1 2m42s
也可以看到镜像多了:
$ docker images | grep keda
ghcr.io/kedacore/keda-metrics-apiserver 2.2.0 a43d40453368 6 weeks ago 95.3MB
ghcr.io/kedacore/keda 2.2.0 42b88f042914 6 weeks ago 83MB
如果要卸载请执行:
$ kubectl delete -f ~/Downloads/keda-2.2.0.yaml
3 安装RabbitMQ
为了快速安装,也方便日后删除,我们通过Helm
来安装RabbitMQ。
查看可用的chart:
$ helm search repo rabbit
执行安装:
$ helm install azure-rabbitmq azure/rabbitmq
检查一下:
$ helm list
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
azure-ingress default 1 2021-02-14 01:21:07.212107 +0800 CST deployed nginx-ingress-1.41.3 v0.34.1
azure-rabbitmq default 1 2021-05-05 11:29:06.979437 +0800 CST deployed rabbitmq-6.18.2 3.8.2
用户名为user
,密码获取如下:
$ echo "Password : $(kubectl get secret --namespace default azure-rabbitmq -o jsonpath="{.data.rabbitmq-password}" | base64 --decode)"
Password : YNsEayx8w2
部署消费者,注意这里有个MQ连接信息和加密,要根据自己情况修改。
$ kubectl apply -f src/main/kubernetes/deploy-consumer.yaml
secret/rabbitmq-consumer-secret created
deployment.apps/rabbitmq-consumer created
scaledobject.keda.sh/rabbitmq-consumer created
triggerauthentication.keda.sh/rabbitmq-consumer-trigger created
查看deployment,发现是没有Pod创建,因为还不需要处理,MQ现在的队列为0。
$ kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
azure-ingress-nginx-ingress-controller 1/1 1 1 80d
azure-ingress-nginx-ingress-default-backend 1/1 1 1 80d
rabbitmq-consumer 0/0 0 0 131m
部署生产者,往MQ发送消息:
$ kubectl apply -f src/main/kubernetes/deploy-publisher-job.yaml
job.batch/rabbitmq-publish created
可以看到,慢慢消费者就起来了,并且创建了越来越多的Pod来处理MQ:
$ kubectl get deployments rabbitmq-consumer
NAME READY UP-TO-DATE AVAILABLE AGE
rabbitmq-consumer 1/1 1 1 167m
$ kubectl get deployments rabbitmq-consumer
NAME READY UP-TO-DATE AVAILABLE AGE
rabbitmq-consumer 3/4 4 3 168m
$ kubectl get deployments rabbitmq-consumer
NAME READY UP-TO-DATE AVAILABLE AGE
rabbitmq-consumer 4/8 8 4 168m
$ kubectl get deployments rabbitmq-consumer
NAME READY UP-TO-DATE AVAILABLE AGE
rabbitmq-consumer 6/8 8 6 169m
$ kubectl get deployments rabbitmq-consumer
NAME READY UP-TO-DATE AVAILABLE AGE
rabbitmq-consumer 0/0 0 0 171m
查看Deployment的Event也可以看到结果:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ScalingReplicaSet 5m55s (x2 over 172m) deployment-controller Scaled up replica set rabbitmq-consumer-7b477f78b4 to 1
Normal ScalingReplicaSet 5m6s deployment-controller Scaled up replica set rabbitmq-consumer-7b477f78b4 to 4
Normal ScalingReplicaSet 4m6s deployment-controller Scaled up replica set rabbitmq-consumer-7b477f78b4 to 8
Normal ScalingReplicaSet 3m5s deployment-controller Scaled up replica set rabbitmq-consumer-7b477f78b4 to 16
Normal ScalingReplicaSet 3m3s (x2 over 172m) deployment-controller Scaled down replica set rabbitmq-consumer-7b477f78b4 to 0
处理完成后,又会回到0了。
代码请查看:https://github.com/LarryDpk/pkslow-samples
欢迎关注微信公众号<南瓜慢说>,将持续为你更新...
推荐阅读:
如何制定切实可行的计划并好好执行
容器技术(Docker-Kubernetes)
SpringBoot-Cloud相关
Https专题
Recommend
-
94
今年一直在做的事情就是成本优化,今天分享的是如何打造一个弹性可伸缩服务。 why? 为什么需要弹性伸缩? 一个网站,通常流量大小不是每时每刻都一样,有高峰,有低谷,如果每时每刻都要保持能够扛住高峰流量的机器数...
-
40
5 月 30 日, 字节跳动技术沙龙 | 基础架构专场 进行了在线直播。我们邀请到了字节跳动基础架构团队资深研发工程师邵伟...
-
26
Rainbond 5.1.9发布,新增实例弹性伸缩、OAuth代码仓库互联功能 16 December 2019Rainbond 5.1.9发布,新增实例弹性伸缩、OAuth代码仓库互联功能 Rainbond 5.1.9发布,新增实例弹性伸缩、OAuth代码仓库互联功能
-
12
文章中使用的是keda 1.5版本,2.0还未release 1.5版本支持deployment,job两种资源。而在2.0增加了StatefulSet以及自定义资源 keda 是一个支持多种事件源来对应用进行弹性伸缩的控制器。 我...
-
17
美团弹性伸缩系统的技术演进与落地实践2021年04月01日 作者: tuyang 文章...
-
6
详细介绍CSS3弹性伸缩box布局爱前端不爱恋爱关注微信公众号:web前端学习圈,领取85G前端全套系统教程...
-
5
Fluid 给数据弹性一双隐形的翅膀 仅限深圳|现场揭秘:腾讯云原生数据库架构探索与实践 >>
-
9
联合作者 | Yan Xun,阿里云 EDAS 团队高级工程师 Andy Shi,阿里云开发者倡导者 Tom Kerkhove,Codit 容器化业务负责人兼 Azure 架构师、KEDA 维护者、CNCF 大使 来源 |
-
13
kubernetes pod的弹性伸缩(三)(基于事件驱动伸缩) 软件 版本 keda 2.3.0 Kubernetes 1.20.7 目前在Kubernetes中做POD弹性伸缩...
-
3
云端干货|降本必备—弹性伸缩的基本原理 - SpotMax云上说禅的个人空间 - OSCHINA - 中文开源技术交流社区 弹性伸缩(Auto Scaling)是云商提供的、根据自身业务需求自动调整计算能力(即实例数量)的服务。使用该服务时,实例数量可根据...
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK