Scale out vs. Scale up (on VM guest)

We are doing a grid computing environment, do we need to create some large VMs (with 8 vCPU, 8GB Mem) or lot's of small VMs (1 vCPU, 1GB MEM)?

I've been asked this question a couple of times and always replied with: "well... that depends...". However, most of the time scaling larger VMs vs. multiple small VMs, the results are better with multiple small VMs. And yes, multiple VMs might be more expensive (more OS licenses, more mem overhead, little more cpu overhead for every single OS) but if you need the highest calculating needs, scaling out is your game.

I've done some (non scientific) tests with my cpubusiest tool. It can do multi threading and it shows your calculation results, so using it should give some nice proofs. So I started doing performance calculations with 1 VM and added 1 vCPU every time and measured the results. And I did the same test with 1 VM and 1vCPU, 2VMs with 1 vCPU and so on. Here are the results:

# expected scale up scale out
1 5,52 5,52 5,52
2 2,76 2,76 2,76
3 1,84 2 1,84
4 1,38 1,58 1,38
5 1,10 1,38 1,1
6 0,92 1,38 0,88
7 0,79 1,17 0,78
8 0,69 0,78 0,68

Results are in seconds. Running with 1 vcpu in 1 vm is the same result for every situations since there is no scaling here. The 'expected' is calculated based on that. All tests where done with 100 Million Sines and 8 threads:


To clarify the results, I've translated them into percentages to show the 'loss' in scale up:

# expected scale up scale out
1 100% 100% 100%
2 100% 100% 100%
3 100% 92% 100%
4 100% 87% 100%
5 100% 80% 100%
6 100% 67% 105%
7 100% 67% 101%
8 100% 88% 101%


Quite cool. These tests were done on an empty ESXi host (vSphere 4.1.0 build 260247), HP ProLiant BL460c G7 blade (2x X5650 @ 2.67GHz) 12 cores in total (24 logical processors). Guest OS is Windows 2008 R2, up-to-date.

Remember I told you: 'It depends...' in the beginning of this short article? Well, looking at the last test, with 8 vCPUs, the results went from 67% (with 7 vCPUs) to 88%. After doing some testing, I can conclude that if your application automatically changes the amount of threads to the exact same amount of vCPUs, scaling up would have a less overhead impact (but it is still slower than scaling out). So application devellopers: keep this in mind - make sure your program is able to scale up together with more CPUs - and also make sure not to run in kernel mode please!

In the end it will all come down to this: "Generally speaking: scale out gives the best performance, but TEST, TEST, TEST, TEST..."

( 2 Votes ) 
Comments (4)
  • Shemeka Truka  - Scale out vs. Scale up (on VM guest)

    You have brought up a very fantastic details , thanks for the post.

  • Deon Kwapniewski  - Scale out vs. Scale up (on VM guest)

    Hello, you used to write fantastic, but the last few posts have been kinda boringK I miss your tremendous writings. Past few posts are just a bit out of track! come on!

  • borvestinkral  - Scale out vs. Scale up (on VM guest)

    Some really marvellous work on behalf of the owner of this internet site, dead great content material.

  • borvestinkral  - Scale out vs. Scale up (on VM guest)

    Together with every thing that seems to be building throughout this particular area, all your points of view happen to be relatively stimulating. Nevertheless, I appologize, but I do not give credence to your entire idea, all be it exciting none the less. It would seem to us that your comments are not totally rationalized and in simple fact you are your self not thoroughly convinced of your argument. In any event I did take pleasure in looking at it.

Write comment
Your Contact Details:
Gravatar enabled
[b] [i] [u] [s] [url] [quote] [code] [img]