Container-driven Scheduling Strategy for Scientific Workflows in Multi-vCPU Environments

Existing scientific workflow scheduling studies barely consider the multi-channel programming of computational resources, which makes it hard to simultaneously achieve effective container sharing and optimize task parallelism and resource utilization. In order to solve the above problems, a segmented workflow scheduling strategy based on container technology in multi-vCPU environment is proposed. It reduces the solution space size of the heuristic algorithm through a segmented scheduling approach. And it uses an adaptive discrete particle swarm optimization algorithm with genetic operators (ADPSOGA) to optimize the completion time of each workflow under the constraint of device rental cost. In addition, a dynamic scaling scheme between containers and devices is proposed to reuse containers and solve the problems related to resource contention when tasks are parallel in a device. The results indicate that ADPSOGA outperforms other similar heuristics algorithms, and the segmented scheduling approach with container scaling scheme shows good performance in workflow scheduling and can adapt well to the impact of increasing task parallelism.