A testing engine for high-performance and cost-effective workflow execution in the cloud

Document Type

Conference Presentation

Department

Electrical and Computer Engineering

Conference Title

Proceedings of the International Conference on Parallel Processing

Date of Presentation

12-8-2015

Abstract

While pursuing high performance and cost effectiveness for directed acyclic graph (DAG)-structured scientific workflow executions in the cloud, it is critical to identify appropriate resource instances and their quantity. This paper presents a testing engine that employs a resource-selection heuristic, which statically analyzes the DAG structure to guide the selection of resource instances, how many and which ones. The testing engine combines the heuristic with two platform-independent DAG-scheduling policies, the Area-oriented DAG-scheduling heuristic (AO) and the Locally-Optimal heuristic (L-OPT), to perform extensive validation assessments. The testing engine ensures the realism of these assessments by modeling the performance variability of the cloud platform using real traces. The testing engine also enables cost-effectiveness analysis that guides users to select a small set of instance candidates that provide performance-cost trade off. Our empirical results show that the pairing of the resource-selection heuristic with AO scheduling policy is a powerful method for cost-effective DAG-structured workflow execution in the cloud.

ISSN

01903918

Volume

2015-December

First Page

849

Last Page

858

DOI

10.1109/ICPP.2015.94

Link to Full Text

Share

COinS