Many Magic Online bots: Virtualization on Windows Comparative Benchmark

We host an article by Enrique taken from his blog: http://www.mtgeconomist.com/en
You can read the original article here: http://www.mtgeconomist.com/en/instalar-bots-de-magic-online-virtualizacion-en-windows-parte-8-2/

Let me say that the main reason for writing this series is the enormous difficulty I have found when trying to search for articles that compare the reliability of different virtualization utilities (in fact I have not found anyone good enough). Therefore, I decided to write my own experience and test the main existing applications so you can decide the program that best suits your needs for your Magic Online stores machine.

What is virtualization?

Virtualization consists of emulating an Operating System (from now OS) inside another one which is the hosting OS. This software runs a full OS inside a window, as if it was another computer. The window that contains that emulated OS is also called a virtual OS or virtual machine (VM in advance).

If you remember from previous articles, the bot gets the resources of the mouse and keyboard to interact with the Magic Online client, which prevents you to use the computer where it is installed. With this software you will not have that problem if you install both Magic Online and the bot in a VM, since all the commands in these input devices are virtually processed.

A VM consumes resources like any computer. You can configure these resources with the only limitation of your hosting OS, considering that it also requires a minimal amount of resources. Some virtualization software can overcome these limits but such software emulation greatly reduces the VM performance.

Of course, nothing but the limits of the hardware prevents you from running more than one VM on the same computer. Thus, you could have several stores in Magic Online using a single computer.

Minimum requirements for a virtual machine

Since each VM will consume a significant part of our computer resources, I recommend that you set the minimum requirements in order to run the Magic Online client. The general requirements are (this is valid for the current version 3.0 not for the beta):

• Operating System: Windows XP Professional SP2 “Lite”
• CPU: 1 kernel
• RAM memory: 1.5 GB
• Hard Disk: 20 GB
• Video Resolution: 1440×900

The OS is a very small version of the original one. You can create an installation disk from the original image and remove all non-required extra tools with a program like nLite. In my case, my version of Windows XP consumes about 60-70 MB of RAM.

The CPU can be configured with more cores, but I think 1 is more than enough to run the bot with the Magic Online client.

The minimum RAM memory needed is about 800-1000 MB. However, with this amount of memory the OS can remap the loaded programs to the Windows swap file, leaving the VM unusable. An amount of 1.5 GB is the minimum size I have set on my VMs with no problems.

The hard disk space of the VM does not have to be 20 GB, in fact all installed applications and the trades backup files will not take you more than 6-8 GB. Anyway it is always better to leave more capacity, because if you create a virtual hard disk file with dynamic space assignation it will not use more space on your physical drive than required.

About the last feature, the minimum resolution is 1280×800 but it is too limited and the bot may have problems. It is a good idea to set a higher size (1440×900 or 1280×1024 for example), and if you have a lower resolution on your hosting OS the VM will add navigation bars so you can see its entire desktop.

You must also consider that the VMs will have more or less available resources depending on the hosting OS. For example, it is not the same that the hosting OS consumes 1 GB of RAM rather than another one that only uses 100 MB. However, you should also keep in mind that maybe the first hosting OS runs special services for the use of hardware virtualization or other useful features for VMs that are not present for the second one.

The ideal hosting OS would be the vSphere Hypervisor ESXi platform by VMware, which has been specifically designed to manage VMs consuming as little as 32 MB of RAM. However, installation is complex and is not compatible with any PC. If you would like more information about this topic, you can have a look at this series of articles written by Thrillski.

Virtualization with Windows hosting OS: benchmark setup

Once you have viewed the settings for each VM, we will create them with the Windows virtualization software selected for this comparison. Two of the applications I have chosen are completely free and unrestricted: VirtualBox 4.3.4 by Oracle and VMware Player 6.0.1. I have also considered a paid tool, VMware Workstation 10.0.1, in order to see if there are relevant differences that justify its price for its use with a Magic Online bot.

I have compared the performance by measuring the average times of a sequence of tests that any VM will run frequently. The three applications have been tested on two computers with the following configurations:

	Computer 1	Computer 2
Hosting OS	Windows 7 Home Premium SP1 64bit	Windows 7 Professional SP1 64bit
Processor	Intel Core Quad Q8200 2.33 GHz	Intel Core i7-2600 3.4 GHz
Hardware Virtualization	No	Yes
RAM memory	8 GB	4 GB
Hard Disk	500 GB 7200rpm	500 GB 7200rpm
Graphics Card	nVidia GTX 285 1GB	nVidia GeForce GT 220 1GB

Although it is interesting to see the performance differences between these computers, the rigurous comparison is done for the virtualization software running on the same machine. The tests involve measuring the time taken for the VM to perform the following tasks:

• VM first boot. Measures the performance without using the cache memory, useful if virtualization software starts upon booting the hosting OS.
• VM second boot. While you don’t run other programs, if you reboot the VM you will see how its performance is improved.
• Simultaneous boot of 2 VMs (same software). The aim consists of testing how virtualization software manages computer resources among multiple VMs. This test is useful for measuring the performance if using more than one store.
• Simultaneous boot of 2 VMs (other software). This test is used to test how an virtualization tool manages the system resources without caring about other VMs from other software.
• Running Magic Online update tool (Kicker). As this application mainly depends on the communication with the server of Magic Online, I wanted to use it to get an idea of ​​how virtualization software manages the network connection.
• Running Magic Online client. This test allows to compare the performance of data loading of Magic Online as well as the management of the CPU computation performed by the virtualization software.
• Load the price list of the MTGO Library bot. This phase of the bot setup makes heavy use of memory, which is important during the execution of the store.

Results and discussion

Firstly I will detail the conditions under which I have done every test to ensure repeatability and a good comparison between platforms:

• The first 4 time trials start right after clicking the start button of the VM and end when the desktop is fully loaded (icons and Start bar are shown on the window).
• Kicker starts right after clicking the icon to run it, and ends just after the check for new updates is completed. Magic Online client has been previously updated in all tests.
• The Magic Online client starts right when the Launch button is pressed on the kicker tool, and ends just when the login screen appears.
• Finally, the test of the MTGO Library bot starts when pressing the Launch button, and ends just after loading the OCR and before launching the Kicker. For this test the buying list has been configured with 4 copies of all cards available up to date in both, regular and foil versions.

Each test was repeated 5 times and the result is the average time obtained from these repetitions. The virtual OS has just the required applications pre-installed (Magic Online 3.0, .NET Framework and MTGO Library), and its hard disk has been defragmented and optimized. The bot version used is 6.31. In the following graphic you can see the results (the lower the better value).

As conclusions, firstly it can be seen that the overall performance of VMware products is higher than the provided by VirtualBox. Oracle software only seems to highlight on the bot load list test with the computer that has hardware virtualization, something that shall be considered. But in general terms, VirtualBox does not seem to be a suitable product for bot virtualization in Windows.

Perhaps the only exception is when you want to run more than one VM. As you can see, the test for 2 VMs with VirtualBox and VMware show a signifficant improvement on performance over the use of only one software. This is an interesting result because one could expect that one virtualization program should better manage the computer resources, but this has been proven to be false at least on the computers of this benchmark.

If we focus on the free and paid versions of VMware desktop, we can see that the time differences between the two solutions are not very noticeable. At least they are not for the cost savings of using the free version. Therefore, in my opinion I do not see it necessary to use the payment application to manage Magic Online stores.

On the other hand, it is also interesting to compare the difference in terms of performance between both computers. As you can see from the results, the improvement of speed for the second machine processors is very relevant. Although the memory of the first machine is twice the second one, this fact does not affect the load times for 2 VMs even with the exact required amount of memory. Therefore, it seems clear that the processor will be a critical element for the proper functioning of our stores.

In conclusion, for computers with Windows as the hosting OS I would personally use the VMware Player application. If you have a more powerful computer with more RAM you can try these tests with more than 2 VMs, compare the use of this application together with VirtualBox and comment your results.

In the following article I will describe the tests I have done to test virtualization in Linux with the most relevant software I could find for this task.