Refactoring the Monolith Workflow into Independent Micro-Workflows to Support Stream Processing

In modern scientific computing, the scientific workflow (SWF) is considered an essential tool for the description and implementation of complex applications. The workflow application is described as a directed acyclic graph (DAG) of tasks (vertices) with I/O data flow (edges) between them. However, this approach does not support the ability to handle data streams from different IoT sources, nor does it support independent deployment and scaling of individual computing tasks. One approach to optimizing a SWF is to partition it into multiple stages, but the implementation is complicated by the tight coupling relationship between the vertices. In the article, we propose Micro-Workflows (MWF) algorithms which automatically separate the partitioned monolith workflow into a set of independent smaller workflows called MWF by refactoring the edges between the vertices.