论文标题

Kubeadaptor:kubernetes上工作流容器化的对接框架

KubeAdaptor: A Docking Framework for Workflow Containerization on Kubernetes

论文作者

Shan, Chenggang, Wang, Guan, Xia, Yuanqing, Zhan, Yufeng, Zhang, Jinhui

论文摘要

随着Kubernetes成为云本地时代的基础架构,工作流系统与Kubernetes的整合变得越来越受欢迎。据我们所知,工作流系统采用调度算法来优化工作流程执行顺序以提高性能和执行效率。但是,由于其固有的调度机制,Kubernetes并未执行在迁移工作流程系统到Kubernetes平台的优化任务执行顺序之后的容器调度。在任务调度顺序中的这种不一致会严重降低工作流执行的效率,并为Kubernetes上工作流程系统的容器化过程带来了许多挑战。在本文中,我们提出了一个云本地工作流引擎,也称为Kubeadaptor,该引擎能够在Kubernetes上实现工作流集装,将工作流程系统与Kubernetes集成在一起,从而确保任务调度顺序的一致性。我们介绍了Kubeadaptor的设计和架构,并详细介绍了Kubeadaptor中的功能实现和事件触发机构。关于四个现实世界工作流的实验结果表明,kubeadaptor确保了工作流程度和库伯纳列在任务调度顺序中的一致性。与基线ARGO工作流引擎相比,Kubeadaptor在任务吊舱,平均工作流程生命周期和资源使用率的平均执行时间方面取得了更好的性能。

As Kubernetes becomes the infrastructure of the cloud-native era, the integration of workflow systems with Kubernetes is gaining more and more popularity. To our knowledge, workflow systems employ scheduling algorithms that optimize task execution order of workflow to improve performance and execution efficiency. However, due to its inherent scheduling mechanism, Kubernetes does not execute containerized scheduling following the optimized task execution order of workflow amid migrating workflow systems to the Kubernetes platform. This inconsistency in task scheduling order seriously degrades the efficiency of workflow execution and brings numerous challenges to the containerized process of workflow systems on Kubernetes. In this paper, we propose a cloud-native workflow engine, also known as KubeAdaptor, a docking framework able to implement workflow containerization on Kubernetes, integrate workflow systems with Kubernetes, ensuring the consistency of task scheduling order. We introduce the design and architecture of the KubeAdaptor, elaborate on the functionality implementation and the event-trigger mechanism within the KubeAdaptor. Experimental results about four real-world workflows show that the KubeAdaptor ensures the consistency of the workflow systems and Kubernetes in the task scheduling order. Compared with the baseline Argo workflow engine, the KubeAdaptor achieves better performance in terms of the average execution time of task pod, average workflow lifecycle, and resource usage rate.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源