Container-driven Scheduling Strategy for Scientific Workflows in Multi-vCPU Environments

XIANG Peng; LIN Bing; YU Hongjie; LIU Dui

doi:10.6054/j.jscnun.2023010

Journal of South China Normal University (Natural Science Edition) > 2023 > 55(1): 102-112. > DOI: 10.6054/j.jscnun.2023010

XIANG Peng, LIN Bing, YU Hongjie, LIU Dui. Container-driven Scheduling Strategy for Scientific Workflows in Multi-vCPU Environments[J]. Journal of South China Normal University (Natural Science Edition), 2023, 55(1): 102-112. DOI: 10.6054/j.jscnun.2023010

Citation:

PDF (1606 KB)

Container-driven Scheduling Strategy for Scientific Workflows in Multi-vCPU Environments

College of Physics and Energy, Fujian Normal University, Fuzhou 350118, China

More Information

Received Date: December 02, 2022
Available Online: April 11, 2023

Graphical Abstract

Abstract

Abstract

Existing scientific workflow scheduling studies barely consider the multi-channel programming of computational resources, which makes it hard to simultaneously achieve effective container sharing and optimize task parallelism and resource utilization. In order to solve the above problems, a segmented workflow scheduling strategy based on container technology in multi-vCPU environment is proposed. It reduces the solution space size of the heuristic algorithm through a segmented scheduling approach. And it uses an adaptive discrete particle swarm optimization algorithm with genetic operators (ADPSOGA) to optimize the completion time of each workflow under the constraint of device rental cost. In addition, a dynamic scaling scheme between containers and devices is proposed to reuse containers and solve the problems related to resource contention when tasks are parallel in a device. The results indicate that ADPSOGA outperforms other similar heuristics algorithms, and the segmented scheduling approach with container scaling scheme shows good performance in workflow scheduling and can adapt well to the impact of increasing task parallelism.
- scientific workflow scheduling,
- container,
- vCPUs,
- particle swarm optimization

FullText(HTML)

References (23)

References

[1]	GAO Y, ZHANG S, ZHOU J. A hybrid algorithm for multi-objective scientific workflow scheduling in IaaS cloud[J]. IEEE Access, 2019, 7: 125783-125795. doi: 10.1109/ACCESS.2019.2939294
[2]	PAHL C. Containerization and the paas cloud[J]. IEEE Cloud Computing, 2015, 2(3): 24-31. doi: 10.1109/MCC.2015.51
[3]	NARDELLI M, HOCHREINER C, SCHULTE S. Elastic provisioning of virtual machines for container deployment[C]//Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering Companion. L'Aquila: Association for Computing Machinery, 2017: 5-10.
[4]	WU Q, ISHIKAWA F, ZHU Q, et al. Deadline-constrained cost optimization approaches for workflow scheduling in clouds[J]. IEEE Transactions on Parallel and Distributed Systems, 2017, 28(12): 3401-3412. doi: 10.1109/TPDS.2017.2735400
[5]	陈刚, 徐胜超. 基于蚁群算法的容器云任务低能耗调度方法[J]. 计算机与数字工程, 2022, 50(11): 2467-2472;2501. doi: 10.3969/j.issn.1672-9722.2022.11.021 CHEN G, XU S C. Low energy consumption scheduling method for container cloud tasks based on ant colony algorithm[J]. Computer & Digital Engineering, 2022, 50(11): 2467-2472;2501. doi: 10.3969/j.issn.1672-9722.2022.11.021
[6]	TAN B, MA H, MEI Y. Novel genetic algorithm with dual chromosome representation for resource allocation in container-based clouds[C]//Proceedings of 2019 IEEE 12th International Conference on Cloud Computing (CLOUD). Milan: IEEE, 2019: 452-456.
[7]	ZHU L, HUANG K, HU Y, et al. A self-adapting task scheduling algorithm for container cloud using learning automata[J]. IEEE Access, 2021, 9: 81236-81252. doi: 10.1109/ACCESS.2021.3078773
[8]	TAN B, MA H, MEI Y. A group genetic algorithm for resource allocation in container-based clouds[C]//Proceedings of Evolutionary Computation in Combinatorial Optimization. Cham: Springer International Publishing, 2020: 180-196.
[9]	ZHENG C, THAIN D. Integrating containers into workflows: a case study using makeflow, work queue, and docker[C]//Proceedings of the 8th International Workshop on Virtualization Technologies in Distributed Computing. Portland: Association for Computing Machinery, 2015: 31-38.
[10]	ZHANG W, LIU Y, WANG L, et al. Cost-efficient and latency-aware workflow scheduling policy for container-based systems[C]//Proceedings of 2018 IEEE 24th International Conference on Parallel and Distributed Systems(ICPADS). Singapore: IEEE, 2018: 763-770.
[11]	ZHANG J, ZHOU X, GE T, et al. Joint task scheduling and containerizing for efficient edge computing[J]. IEEE Transactions on Parallel and Distributed Systems, 2021, 32(8): 2086-2100. doi: 10.1109/TPDS.2021.3059447
[12]	TAGHINEZHAD-NIAR A, PASHAZADEH S, TAHERI J. Workflow scheduling of scientific workflows under simultaneous deadline and budget constraints[J]. Cluster Computing, 2021, 24(4): 3449-3467. doi: 10.1007/s10586-021-03314-3
[13]	LIU L, ZHANG M, BUYYA R, et al. Deadline-constrained coevolutionary genetic algorithm for scientific workflow scheduling in cloud computing[J]. Concurrency and Computation: Practice and Experience, 2017, 29(5): e3942/1-12.
[14]	RAJASEKAR P, PALANICHAMY Y. Scheduling multiple scientific workflows using containers on IaaS cloud[J]. Journal of Ambient Intelligence and Humanized Computing, 2021, 12(7): 7621-7636. doi: 10.1007/s12652-020-02483-0
[15]	PANG S, LI W, HE H, et al. An eda-ga hybrid algorithm for multi-objective task scheduling in cloud computing[J]. IEEE Access, 2019, 7: 146379-146389. doi: 10.1109/ACCESS.2019.2946216
[16]	RODRIGUEZ M A, BUYYA R. Deadline based resource provisioningand scheduling algorithm for scientific workflows on clouds[J]. IEEE Transactions on Cloud Computing, 2014, 2(2): 222-235. doi: 10.1109/TCC.2014.2314655
[17]	WU H, CHEN X, SONG X, et al. Scheduling large-scale scientific workflow on virtual machines with different numbers of vcpus[J]. The Journal of Supercomputing, 2021, 77(1): 679-710. doi: 10.1007/s11227-020-03273-3
[18]	TOPCUOGLU H, HARIRI S, MIN Y W. Performance-effective and low-complexity task scheduling for hetero-geneous computing[J]. IEEE Transactions on Parallel and Distributed Systems, 2002, 13(3): 260-274. doi: 10.1109/71.993206
[19]	KENNEDY J, EBERHART R. Particle swarm optimization[C]//Proceedings of ICNN'95-International Conference on Neural Networks. Perth: IEEE, 1995: 1942-1948.
[20]	MCCALL J. Genetic algorithms for modelling and optimisation[J]. Journal of Computational and Applied Mathematics, 2005, 184(1): 205-222. doi: 10.1016/j.cam.2004.07.034
[21]	WU Z, NI Z, GU L, et al. A revised discrete particle swarm optimization for cloud workflow scheduling[C]//Proceedings of 2010 International Conference on Computational Intelligence and Security. Nanning: IEEE, 2010: 184-188.
[22]	BHARATHI S, CHERVENAK A, DEELMAN E, et al. Characterization of scientific workflows[C]//Proceedings of 2008 Third Workshop on Workflows in Support of Large- Scale Science. Austin: IEEE, 2008: 1-10.
[23]	GUO W, LIN B, CHEN G, et al. Cost-driven scheduling for deadline-based workflow across multiple clouds[J]. IEEE Transactions on Network and Service Management, 2018, 15(4): 1571-1585. doi: 10.1109/TNSM.2018.2872066

Cited By

Cited by

Periodical cited type(7)

1.	洪森荣，朱盈盈，李紫莹，胡明艳，欧阳克蕙. 盐胁迫下金花菜和紫花苜蓿试管苗的转录组分析及其耐盐基因筛选. 中国农学通报. 2023(03): 111-118 .
2.	王星哲，武悦，王艺煊，王瑞莲，周文新，李瑞莲，陈阳. 玉米发芽期响应盐胁迫的转录组分析. 分子植物育种. 2023(02): 370-378 .
3.	董亚茹，聂玉霞，李云芝，赵东晓，耿兵，王照红. 瞬时过表达MnERF2基因对桑树耐盐性的影响. 山东农业科学. 2022(04): 9-16 .
4.	辛建攀，李燕，赵楚，田如男. 镉胁迫下梭鱼草叶片转录组测序及苯丙烷代谢途径相关基因挖掘. 生物技术通报. 2022(06): 198-210 .
5.	张超，马晓丽，卢晓峰，李刚，耿怡爽，孙云保，杨修一，耿计彪. 盐分胁迫下土施甲哌■对棉苗叶片生理和根系形态的影响. 江苏农业科学. 2022(22): 81-86 .
6.	陈静，陈芸，热依麦阿依·阿布都艾尼，方志刚，凯迪日耶·玉苏普，马刘峰. 月季插穗不定根起始的转录组分析和关键基因筛选. 华南师范大学学报(自然科学版). 2021(03): 54-63 .
7.	洪森荣，陈轩宇，李文丽，张座栋，刘军，刘佳，蔡红，陈荣华. 盐胁迫对怀玉山三叶青2个栽培种试管苗生理特性和次生代谢产物的影响. 山东农业科学. 2021(09): 38-45 .

Other cited types(5)

Get Citation

PDF

XML

Article views (139) PDF downloads (38) Cited by(12)

Turn off MathJax

Article Contents

Abstract

References

Container-driven Scheduling Strategy for Scientific Workflows in Multi-vCPU Environments

Abstract

References

Cited by

Periodical cited type(7)

Other cited types(5)

Catalog

Export File

Citation

Format

Content