Old McDonald Had a SharePoint Farm E-I-E…IO – Part 2

As an unplanned follow up to my previous post I wanted to reply to some of the feedback I received and take another run at this little IO test. The feedback was generally around “what” I tested rather than the “why” and “how”. I had no doubt the “why” was super clear and I was not interested in debating the “how” because as I said before this is a very informal test so I am glad I did not receive either of those remarks. As for the “what” feedback it boiled down as:

  • You get what you pay for – at $9 a month for Azure and free for EC2 what did you expect?
  • Try testing on a more realistic platform, one that someone may actually expect decent IO.
  • How about a newer “Cloud Ready” OS bro?
  • Hey buddy, we are friends and I work for Rackspace, so why didn’t you include them in the mix?

All are fair comments – so lets take another stab at this and see what paying a bit more money can get us.

Rational

For those that did not read the previous post, the reason I am doing this testing is because the general feeling from a few of us using VMs running in the cloud is the IO seems or feels pretty slow. While Amazon, Windows Azure, and RS give you options when it comes to the number CPUs, network speed, disk space, and RAM it seems when it comes to disk IO you get what you get. While Amazon EC2 does give you designators such as “low” or “high” IO for some of their instances, there is no real indication of what that actually means or how it compares to other providers.

Configuration

I decided to dump Windows 2008R2 and go with Windows Server 2012. Looking at the configuration options available between what Windows Azure, Amazon EC2, and Rackspace offer I decided to go with the following configurations:

  • Windows Azure: Large 4 x 1.6 7GB RAM
  • Amazon EC2: M1 Large 2 x 7.5GB RAM
  • Rackspace 8192 4x 8GB RAM

You sharp ones out there will notice the EC2 configuration has two less CPUs than Windows Azure and Rackspace. While true, EC2 is a bit short, I could not see how I could get the three configurations to align with both CPU and memory so I decided to go with what I felt was the closest with regards to specs, plus I was comparing IO and not CPU so hopefully the number of processors would not play that big of a role.

I setup each test instance using Windows Server 2012 (x64) and used the configuration template each service offered. For the test analyzers I used ATTO Disk Benchmark v2.47 and Crystal Disk Mark 3.0.2. And finally, all tests were conducted on December 5th 2012.

For Windows Azure I could not get either test analyzers to work without hanging when analyzing the OS disk (C: drive). With both analyzers hanging I decided to rebuild the Azure image using different storage services however after 4 rebuilds I was never able to get either tool to run. I was told by someone at Microsoft that the OS drive was optimized for booting much more than general application data so I may want to use a data drive. Armed with that bit of knowledge I added a 10GB data drive (F:) to my Azure VM and was able to execute the test analyzers. So it is Azure’s data disk and not OS disk whose numbers appear below while the EC2 and Rackspace tests were done off their standard OS drives.

The Results

The table below takes 10 measures which were produced by the Crystal Disk Mark and ATTO Disk Benchmark tools. The ATTO Disk Benchmark measures are the max values seen for both Read and Write regardless of the buffer size used.

Rackspace won 6 of the 10 measures while EC2 won the other 4. Unfortunately Azure did not perform very well even though I followed the suggestion of Microsoft and used a dedicated data disk. EC2 and Rackspace too have the option of using dedicated data disks which are supposed to be a faster option than using the OS disks but as previously mentioned they were not used for this test as the number of options and costs are many.

Azure (MB/sec) EC2 (MB/sec) Rackspace (MB/sec)
Sequential Read 25.27 60.00 165.7
Sequential Write 21.56 43.03  271.3
Random 512K Read 11.26 67.26 66.06
Random 512K Write 17.44 40.47  162.90
Random 4K Read 0.162 6.593 1.064
Random 4K Write 0.492 4.133  13.47
Random 4K with Queue Depth = 32 Read 2.626 61.03 5.846
Random 4K with Queue Depth = 32 Write 4.098 14.60 9.119
Max ATTO Benchmark Read 59 111  349
Max ATTO Benchmark Write 65 45  371

* Winner

Below are some screen shots of the analyzers after each completed their respective tests on each environment. The numbers visible in these screenshots are what was used to populated the table above.

Windows Azure

azure-largeazure-large-crystal

Amazon EC2

ec2-largeec2-large-crystal

Rackspace

rs8192rs8192-crystal

Conclusion

While Azure did not fair as well as I would have imagined (or hoped), of the 3 services it is the only one which is currently still in Beta/Preview. EC2 did fall short in a number of measures as compared to Rackspace however its service is a bit cheaper in some cases. Rackspace appears to have a clear winner when it comes to IO performance and in some cases blows the competition away, especially if you look at the ATTO Disk Benchmark numbers. One constant with cloud services is they are constantly changing and evolving their technology, solutions, and prices so I expect these numbers to be different in 6 months and very different in a year.

Disclaimer

I am a former Microsoft employee and I do consulting work for Microsoft on occasion. The nature of that work does not have any relation to Windows Azure. I am not affiliated with Microsoft, Amazon, or Rackspace and I do not make any direct or indirect income from selling any of the products tested, competing products, or any products in general.

Post to Twitter Post to Facebook Post to LinkedIn Post to Delicious Post to Digg

3 thoughts on “Old McDonald Had a SharePoint Farm E-I-E…IO – Part 2”

  1. Thanks for putting together this useful post comparing our options.

    Thanks too for this clarification – “I am not affiliated with Microsoft, Amazon, or Rackspace and I do not make any direct or indirect income from selling any of the products tested, competing products, or any products in general.”

    Pity that your self-appointed “friend at Rackspace” felt compelled to post such a lame comment, though. It would be more interesting if a “thank you for including us” comment came from someone from one of the other two companies (especially Azure!) and better still if nobody so publically connected with one of those companies did so.

  2. Thanks for the write-up Todd, it helps to validate my experiences so far.

    I’m glad to see Rackspace listed here – they are another major player in this space and more competition means a better product for all of us to use. And the data here certainly justifies them having a spot at the table when considering cloud VM hosting.

    And I don’t see any reason why someone affiliated with the organizations reviewed shouldn’t post here. If there is some other cloud provider that offers comparable or better performance I’d want them to post here also.

Leave a Reply

Your email address will not be published. Required fields are marked *