A2Cloud-RF: A Random Forest Based Statistical Framework to Guide Resource Selection for High-performance Scientific Computing on the Cloud

Department

Electrical and Computer Engineering

Document Type

Article

Publication Title

Concurrency Computat. Pract. Exper.

ISSN

1532-0634

Volume

32

Issue

2

DOI

10.1002/cpe.5942

Publication Date

Summer 1-1-2020

Abstract

This article proposes a random-forest based A2Cloud framework to match scientific applications with Cloud providers and their instances for high performance. The framework leverages four engines for this task: PERF engine, Cloud trace engine, A2Cloud-ext engine, and the random forest classifier (RFC) engine. The PERF engine profiles the application to obtain performance characteristics, including the number of single-precision (SP) floating-point operations (FLOPs), double-precision (DP) FLOPs, x87 operations, memory accesses, and disk accesses. The Cloud trace engine obtains the corresponding performance characteristics of the selected Cloud instances including: SP floating point operations per second (FLOPS), DP FLOPS, x87 operations per second, memory bandwidth, and disk bandwidth. The A2Cloud-ext engine uses the application and Cloud instance characteristics to generate objective scores that represent the application-to-Cloud match. The RFC engine uses these objective scores to generate two types of random forests to assist users with rapid analysis: application-specific random forests (ARF) and application-class based random forests. The ARF consider only the input application's characteristics to generate a random forest and provide numerical ratings to the selected Cloud instances. To generate the application-class based random forests, the RFC engine downloads the application profiles and scores of previously tested applications that perform similar to the input application. Using these data, the RFC engine creates a random forest for instance recommendation. We exhaustively test this framework using eight real-world applications across 12 instances from different Cloud providers. Our tests show significant statistical agreement between the instance ratings given by the framework and the ratings obtained via actual Cloud executions.

Link to Full Text

Share

COinS