Home > Archive > 2020 > Volume 10 Number 1 (Jan. 2020) >
IJMLC 2020 Vol.10(1): 140-147 ISSN: 2010-3700
DOI: 10.18178/ijmlc.2020.10.1.911

Workflow Scheduling with Amazon EC2 Spot Instances: Building Reliable Compute Environments

Altino M. Sampaio and Jorge G. Barbosa

Abstract—Amazon Elastic Compute Cloud (EC2) gives access to resources in the form of instances. EC2 Spot Instances (SIs) offer spare compute capacity at steep discounts compared to reliable and fixed price on-demand instances. However, SIs are unreliable since they can be reclaimed by EC2 at any given time, with a two-minute interruption notice. In this paper, we propose a container migration-based solution to build reliable compute environments on top of unreliable EC2 instances. Our solution leverages recent findings on performance and behavior characteristics of EC2 SIs. We compare the performance of our algorithm to that of state-of-the-art algorithms, by running a real-life workflow application constrained by user-defined deadline and budget parameters. The results show that our solution is able to build reliable virtual compute environments on top of EC2 on-demand-, spot block, and SI purchasing models, and successfully conclude submitted workflow applications with budget and deadline constraints, for a worse-case scenario.

Index Terms—Amazon EC2, cloud computing, reliability, migration of containers.

Altino M. Sampaio is with the CIICESI, Escola Superior de Tecnologia e Gestão, Instituto Politécnico do Porto, Felgueiras, Portugal (e-mail: ams@estg.ipp.pt).
Jorge G. Barbosa is with LIACC, Departamento de Engenharia Informática, Faculdade de Engenharia, Universidade do Porto, Porto, Portugal (e-mail: jbarbosa@fe.up.pt).

[PDF]

Cite: Altino M. Sampaio and Jorge G. Barbosa, "Workflow Scheduling with Amazon EC2 Spot Instances: Building Reliable Compute Environments," International Journal of Machine Learning and Computing vol. 10, no. 1, pp. 140-147, 2020.

Copyright © 2020 by the authors. This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).

 

General Information

  • E-ISSN: 2972-368X
  • Abbreviated Title: Int. J. Mach. Learn.
  • Frequency: Quaterly
  • DOI: 10.18178/IJML
  • Editor-in-Chief: Dr. Lin Huang
  • Executive Editor:  Ms. Cherry L. Chen
  • Abstracing/Indexing: Inspec (IET), Google Scholar, Crossref, ProQuest, Electronic Journals LibraryCNKI.
  • E-mail: ijml@ejournal.net


Article Metrics in Dimensions