As an unplanned follow up to my previous post I wanted to reply to some of the feedback I received and take another run at this little IO test. The feedback was generally around what I tested rather than the why and how. I had no doubt the why was super clear and I was not interested in debating the how because as I said before this is a very informal test so I am glad I did not receive either of those remarks. As for the what feedback it boiled down as:
- You get what you pay for at $9 a month for Azure and free for EC2 what did you expect?
- Try testing on a more realistic platform, one that someone may actually expect decent IO.
- How about a newer Cloud Ready OS bro?
- Hey buddy, we are friends and I work for Rackspace, so why didn’t you include them in the mix?
All are fair comments so lets take another stab at this and see what paying a bit more money can get us.
For those that did not read the previous post, the reason I am doing this testing is because the general feeling from a few of us using VMs running in the cloud is the IO seems or feels pretty slow. While Amazon, Windows Azure, and RS give you options when it comes to the number CPUs, network speed, disk space, and RAM it seems when it comes to disk IO you get what you get. While Amazon EC2 does give you designators such as low or high IO for some of their instances, there is no real indication of what that actually means or how it compares to other providers.
I decided to dump Windows 2008R2 and go with Windows Server 2012. Looking at the configuration options available between what Windows Azure, Amazon EC2, and Rackspace offer I decided to go with the following configurations:
- Windows Azure: Large 4 x 1.6 7GB RAM
- Amazon EC2: M1 Large 2 x 7.5GB RAM
- Rackspace 8192 4x 8GB RAM
You sharp ones out there will notice the EC2 configuration has two less CPUs than Windows Azure and Rackspace. While true, EC2 is a bit short, I could not see how I could get the three configurations to align with both CPU and memory so I decided to go with what I felt was the closest with regards to specs, plus I was comparing IO and not CPU so hopefully the number of processors would not play that big of a role.
I setup each test instance using Windows Server 2012 (x64) and used the configuration template each service offered. For the test analyzers I used ATTO Disk Benchmark v2.47 and Crystal Disk Mark 3.0.2. And finally, all tests were conducted on December 5th 2012.
For Windows Azure I could not get either test analyzers to work without hanging when analyzing the OS disk (C: drive). With both analyzers hanging I decided to rebuild the Azure image using different storage services however after 4 rebuilds I was never able to get either tool to run. I was told by someone at Microsoft that the OS drive was optimized for booting much more than general application data so I may want to use a data drive. Armed with that bit of knowledge I added a 10GB data drive (F:) to my Azure VM and was able to execute the test analyzers. So it is Azure’s data disk and not OS disk whose numbers appear below while the EC2 and Rackspace tests were done off their standard OS drives.
The table below takes 10 measures which were produced by the Crystal Disk Mark and ATTO Disk Benchmark tools. The ATTO Disk Benchmark measures are the max values seen for both Read and Write regardless of the buffer size used.
Rackspace won 6 of the 10 measures while EC2 won the other 4. Unfortunately Azure did not perform very well even though I followed the suggestion of Microsoft and used a dedicated data disk. EC2 and Rackspace too have the option of using dedicated data disks which are supposed to be a faster option than using the OS disks but as previously mentioned they were not used for this test as the number of options and costs are many.
|Azure (MB/sec)||EC2 (MB/sec)||Rackspace (MB/sec)|
|Random 512K Read||11.26||67.26||66.06|
|Random 512K Write||17.44||40.47||162.90|
|Random 4K Read||0.162||6.593||1.064|
|Random 4K Write||0.492||4.133||13.47|
|Random 4K with Queue Depth = 32 Read||2.626||61.03||5.846|
|Random 4K with Queue Depth = 32 Write||4.098||14.60||9.119|
|Max ATTO Benchmark Read||59||111||349|
|Max ATTO Benchmark Write||65||45||371|
Below are some screen shots of the analyzers after each completed their respective tests on each environment. The numbers visible in these screenshots are what was used to populated the table above.
While Azure did not fair as well as I would have imagined (or hoped), of the 3 services it is the only one which is currently still in Beta/Preview. EC2 did fall short in a number of measures as compared to Rackspace however its service is a bit cheaper in some cases. Rackspace appears to have a clear winner when it comes to IO performance and in some cases blows the competition away, especially if you look at the ATTO Disk Benchmark numbers. One constant with cloud services is they are constantly changing and evolving their technology, solutions, and prices so I expect these numbers to be different in 6 months and very different in a year.
I am a former Microsoft employee and I do consulting work for Microsoft on occasion. The nature of that work does not have any relation to Windows Azure. I am not affiliated with Microsoft, Amazon, or Rackspace and I do not make any direct or indirect income from selling any of the products tested, competing products, or any products in general.