Spotsy is a newly open-sourced Play! Framework web application for analyzing Amazon EC2 Spot prices. It has been developed independently, but is currently used at Kifi to reduce infrastructure costs. We have been able to cut our EC2 monthly bill by over 60% by using a spot-first environment.
Some EC2 Spot instances are chronically underpriced for a variety of reasons. Amazon, likely for good business reasons, doesn't make it simple to identify where signifiant discounts are available. Spotsy downloads spot price history, along with instance information, to easily compare which instances are underprices, relative to their CPU, memory, etc.
Currently, Spotsy is just the backend API — a featured frontend is currently under development.
This has also been a chance for me to experiment with Scala libraries that I've wanted to use. They include: cats for functional programming constructs, scalaz for
Task, macwire for compile-time dependency injection, coursier as an Ivy replacement, and the latest version of Play. As such, a large part of this project is exploratory.
Spotsy efficiently stores and can generate statistical information about historical spot price information, in the context of instance size / ability. This allows you to make easy cost-per-compute-ability decisions. Here is small sample of what can be generated by Spotsy:
||$ / week
As you can see, significant discounts are available on the spot market.
% Off is based on the regular demand price for that instance, compared to the previous week's realized spot price, given an infinite bid.
The only external dependency Spotsy has is
sbt. Once installed, running
sbt run will build and run the Play! web service on port 9000.
You will need to configure AWS keys by creating a file at
conf/credentials. Here is an example stub:
accessKey = "AAAAAAAA"
secretKey = "BBBBBB+J/zhpX"
play.crypto.secret = "010101010101"
Upon first request, if you have never run Spotsy before, spot price data will be downloaded and saved to disk. This may take quite some time due to AWS rate limits. If you're downloading for several zones and for a large time window (see
HistoricalSpotPrice.scala to configure
startBackdate), the process may take several hours.
Feedback is greatly appreciated. The code is available here: https://github.com/andrewconner/spotsy