Building a Magento Testing and Development Environment – Creating the virtual machines

Part 2, Creating the virtual machines in Proxmox

In part 2 I am going to cover the setup of the initial virtual machines for the testing environment. Only the basic virtual machine creation will be covered in this post. Actual operating system installation and configuration will begin to be covered in the next part but will probably take 2-3 posts to cover everything.

The virtual environment I am creating is the Test Environment. It is supposed to be as close to production as possible so they can test new code. For development there will be a separate environment, virtualized on the same Proxmox servers, that the developers can destroy at will. But for true testing, you need a static system that is essentially a mirror of production. Otherwise you cannot truly know how code changes will affect the production environment and you could cause yourself a lot of problems.

So, for this environment I am going to setup 2 Magento servers, two NFS servers, two Redis servers and two MySQL servers. The Magento servers are running Centos 6.8 and the rest are running Ubuntu 14.04.4 server. I am also going to setup the auxiliary support servers for apt and yum repositories, and an ansible server.  These aren’t entirely necessary but I believe it is something you need for a production environment.

The Magento Servers

The first virtual machine we are going to create is the second Magento server.  I could have picked any server, there is no particular order at this point. The following is the process for creating the virtual machine.  Each of the virtual machines will be created in a similar manner, with the only difference being the amount of memory, storage, cpu cores and the boot source.

Magento VM Setup
Step 1 Naming the new virtual machine

First we have to click the icon in the upper left corner labelled “Create VM”.  We will not be using containers for this environment so make sure to click “Create VM” and not “Create CT”.  When prompted I changed the VM ID to 500 and the machine name to magento2.  The VM ID was changes so that in error reports posted to log servers we know which server the machine is running on, VM5 (see previous post).  The machine name I am entering will match the actual hostname of the system. Click Next when you have done this.

Magento VM Setup
Step 2 choosing the OS Type

Next we select the type of operating system we will be installing.  Click Next when you have done this.

Magento VM Setup
Step 3 picking the initial boot source

We have to tell the Proxmox server what we are going to initially boot from.  Think of this as inserting the CD or flash drive you will install the OS from.  In this example we are installing from the Centos 6.8 x64 ISO file.  Once you have chosen your boot source click Next.

Magento VM Setup
Step 4 Creating the storage

In the next step we create the storage for the virtual machine.  You may hear this referred to as the container, vdisk or virtual disk.  Essentially you are creating a large file that contains the entire system. Because the operating systems I am using support VirtIO interfaces I am choosing those because in my experience I get the best performance from this.  If you are using an older version of Linux, or Windows, you may need to select IDE, SATA or SCSI. For the storage location I chose the default local-lvm which is the RAID5 SSD array we created in the previous post. I set the disk size to 100GB which is much more than we need, but this will cover any future growth.  I also selected the Discard option which increases disk performance slightly. After you have made your choices click Next.

Magento VM Setup
Step 5 choosing the number of CPU cores

On the production Magento servers they will be using we’re expecting heavy CPU utilization.  Because we don’t want our test environment to give false performance data we’re making the CPU availability as high as production will be – 16 cores, or about half the system has available. These are virtual cores, the system will take your existing number of cores and creates a virtual pool of resources that are available to the virtual machines running on the system.  Remember, most of your systems are idle a considerable amount of time. So, the available resources can be easily shared by many systems while still achieving near bare-metal performance. If you are unsure how many to choose I would suggest choosing a high number and decreasing from there until you achieve the performance you require. Once you have chosen your settings click Next.

Magento VM Setup
Step 6 Choosing the available memory

Each of their virtual machine hosts has 128GB of RAM.  So, since their production Magento web servers will have 32GB each we are giving the virtual servers the same amount.  However, since they may be idle at times we are using the “balloon” memory so the system will always have 8GB of RAM dedicated but it can balloon up to 32GB when necessary. For our MySQL servers we will use a fixed amount of memory because of the way MySQL performs with balloon memory, but for the rest of the systems this technique will work just fine. Please make sure you don’t choose more memory than you actually have available or else you could be running off of swap which will severely affect performance.  For my configurations I usually reserve about 8GB for the host, and make the rest available the virtual machines. When you have chosen your settings click Next.

Magento VM Setup
Step 7 Choosing the network settings

In the final step we will choose our network settings.  There are many options available here, but for this setup we are simply using a bridge that was created by the default Proxmox installation.  The default Model is the Intel E1000 which is probably what you should choose if you are installing Windows or an older version of Linux.  Otherwise for best performance of your virtual machine and the others running on the host you should choose VirtIO. If you are using a separate VLAN for your development environment or whatever servers you are installing, you can enter that information here as well.  Limiting the transfer rate is another common option and something I often choose if installing something like a proxy server. Once you have made your selection click Next.

Magento VM Setup
Step 8 Settings Review

Review the settings you have chosen.  If anything is incorrect click the Back button and change it.  Otherwise, click Finish. Once you have done this the new virtual machines should be displayed on the left in the Proxmox virtual environment console.  At first you will just see the VM ID number, but after the storage creation has finished you will see the Name you chose as well.  Click on that, then click Hardware.

Magento VM Setup
Step 9 Reviewing the settings before first start

You should be able to see all of the hardware configuration options you chose here.  If you decide you need to change something you can select it here and click Edit.  If the system is running you’ll typically have to restart it before your changes go into effect.  If you are ready to begin the operating system installation simply click Start in the upper right, and then click Console to launch the virtual console.

Magento VM Setup
Virtual Machine Settings

The above table shows the settings I chose for the virtual machines I created.  The servers magento1 and magento2 are the actual Magento EE 1.4.x web servers.  The servers magentodb1 and magentodb2 are the MySQL servers.  The servers magentoredis1 and magentoredis2 are the Redis indexing servers. The servers magentoutility1 and magentoutility2 are the NFS file servers that host the images for Magento.  The serer magentoapt1 is a local Ubuntu APT repository. The server magentoyum1 is a local Yum repository.  The server magentoansible1 is an Ansible configuration management servers.  And finally, the system magentopc2 is an Xubuntu 16.04 desktop.  This desktop is something the developers can use to do quick edits if they are away from their normal system with their git credentials, or for testing how the website looks on the local (fast) network – more about this in part 3, maybe.

So now you have your virtual machines, but what do you do with them? In part 3 I will cover the basic operating system installation and in part 4 I will most likely cover the Magento installation and configuration. If you have any questions feel free to ask.

Intro – Magento Development Environment
Part 1 – Installing Proxmox
Part 2 – Creating the Virtual Machines
Part 3 – Setup MySQL Master-Master
Part 4 – Setup Ansible
Part 5 – Setup APT Proxy

Leave a Reply