Several years ago I was hired as a Linux System Administrator to consolidate several network services into a Xen virtual environment. Even though Xen was already the incorporated virtual machine manager (VMM) distributed with Redhat Enterprise Linux 5 (rhel5), I could not find any quick guide to help me tackle the task. To this day, there are very few clear how-tos when it comes to virtualization using Xen. Therefore, I am writing this quick guide hoping that it’s going to be a valuable reference for anyone who opts to use Xen rather than any other VMM out there.
In case you are not familiar with Xen, it was a fruit of a research project conducted at the University of Cambridge back in the early 2000s. The first version came out in 2003. In 2007 Citrix Systems bought its source code and later the same year made the source public through Xen Project.
When one has decided to consolidate servers or services into a Xen’s virtual environment, one is faced with a couple of choices to make: 1) Select Virtual Machine Type; and 2) Select Migration Method.
Virtual Machine Types
Xen offers a two VM types: Para-virtualized and Fully-virtualized.
- Para-Virtualized VM means that xen (as VMM) alters the kernel of a guest operating system making it totally dependent on VMM. In other words, VMM mediates communications between guest OS and hardware devices by presenting the hardware devices to the guest OS. This type of virtualization does not require that the hardware being used supports virtualization technology.
- Full-Virtualized VM on the other hand does not require alteration of the guest OS because CPU traps all the privileged instructions and sends them to the VMM to emulate. A full-virtualized VM requires you to have hardware or processor that supports virtualization technology.
- Install From Installation Source: This method consists of installing the same version of the system being migrated from its installation source or media and then migrate the data from the stand-alone system into the newly created VM (domU). This migration method is the easiest approach to migrating any given system into a Xen’s virtual environment because it minimizes the complexity of manually setting up the system being migrated as domU in the virtual environment. In other words, by installing the OS being migrated, all the complexity of setting up a new virtual machine is watered down. In fact, the installation can even be unattended by using kickstart.
- Manually Migrate An Existing System: This method is tedious in nature since everything needs to be done manually using the same utilities that a normal installation environment uses to achieve the same tasks.
As you may already imagine, this article is going to focus on the easy and straight forward approach, so there is not need to panic at all.
Base on the Redhat Documentation, the following are the hardware requirements to effectively run virtualization using rhel5:
Minimum system requirements
- 6GB free disk space
- 2GB of RAM
Recommended system requirements
- 6GB plus the required disk space recommended by the guest operating system per guest.
- One processor core or hyper-thread for each virtualized CPU and one for the hypervisor.
- 2GB of RAM plus additional RAM for virtualized guests.
Hardware Being Used
For the purpose of this lab, I am using a HP Proliant GL360 G5 Sever with the following hardware characteristics:
- Processor 2x Intel Xeon 2.66GHz Duel Core
- Memory 16GB (DDR2 SDRAM)
- Hard Disk 3x 73GB SAS
Fell free to use anything that you have. I am using this server because it’s what I have available and my project is actually more extensive than the scope of this article. The project entails the deployment of 5 domUs (VMs) within the virtual environment. Yet, because the scope o this article is basically showing an easy way of creating a domU on CentOS VMM, I am going to illustrate that specific part of the project here.
On this server, I am going to install base CentOS – virtual machine manager (dom0). Since I am using CentOS 5.8 rather than Redhat and Fedora, this is going to be a CentOS Xen’s kernel over-which all domUs (VMs) are going to run.
- Put CentOS DVD in and boot from it.
- Type “linux text” and press [ENTER]
- Chose “Skip” and go to the next step
- Now proceed normally with installation making selection as you prefer until it gets to where you need to choose what type of system to install.
- Now, deselect the box besides ‘desktop’, select the box besides ‘server’, and at the bottom select ‘Customize now’. Then, proceed by clicking or pressing ‘Next’.
- On the left-side, select ‘Servers’; On the right-side deselect everything except for ‘Web Server’.
- Still on the same screen, select ‘Virtualization’ on the left-side; and then, select ‘Xen’ on the right-side. And proceed with installation by clicking ‘Next’.
- Then, click ‘Next’ again to install the base system.
What you have just done is simply starting a base installation of CentOS as the VMM. At this point, you may take a quick break and get a cup of coffee while the installation is proceeding. Once all the packages are installed, the system is going to reboot and come up as firstboot: giving you the opportunity to configure network, firewall, etc. Setup whatever you need as you prefer.
The following shows how I configured the network on the dom0 (the system just installed):
/etc/sysconfig/network: NETWORKING=yes NETWORKING_IPV6=no HOSTNAME=jcrealm.gl-studio.me GATEWAY=10.20.10.1 GATEWAYDEV=eth0
/etc/sysconfig/network-scripts/brdg0 DEVICE=brdg0 TYPE=bridge ONBOOT=yes DELAY=0 BOOTPROTO=static IPADDR=10.20.10.10 NETMASK=255.255.0.0 GATEWAY=10.20.10.5 /etc/host.conf # Lookup nameds via /etc/hosts firts # then, falls back to DNS resolver. order hosts,bind # I have machines with multiple addresses. multi on /etc/sysconfig/network-scripts/eth0 DEVICE=eth0 ONBOOT=yes BRIDGE=brdg0 HWADDR=00:19:EE:4C:61:FE /etc/sysconfig/network-scripts/route-brdg0 10.0.0.0/0 via 10.20.10.10 dev brdg0 /etc/resolv.conf domain gl-studio.me search gl-studio.me belkin.gl-studio.me nameserver 10.20.10.1 /etc/hosts 10.20.10.1 belkin.gl-studio.me belkin 10.20.10.20 jcreigns.gl-studio.me jcreigns 10.20.10.10 jcrealm.gl-studio.me jcrealm 10.20.10.21 jcweb.gl-studio.me jcweb 10.20.10.23 jcdstore.gl-studio.me jcdstore 10.20.10.25 jcmail.gl-studio.me jcmail 10.20.10.27 jcsec.gl-studio.me jcsec 10.20.10.29 jcman.gl-studio.me jcman
Once you have your network configured as needed you may proceed to the core of this demonstration, which is setting up the environment for the domUs installations. For clarification purposes: my dom0, the host server (VMM) is called jcrealm and its main ip address is 10.20.10.10. On this server will be hosted 5 domUs, which are:
For the purpose of this illustration I am going to demonstrate the installation of the host
jcweb, which is going to be the webserver for my environment.
1) Check the volume group
[root@jcrealm ~]# vgdisplay --- Volume group --- VG Name vg00 System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 13 VG Access read/write VG Status resizable MAX LV 0 Cur LV 12 Open LV 12 Max PV 0 Cur PV 1 Act PV 1 VG Size 204.88 GB PE Size 32.00 MB Total PE 6556 Alloc PE / Size 1275 / 202.03 GB Free PE / Size 5281 / 175 GB VG UUID jzgGPI-U3Li-myZE-EGh1-0tMN-ORNv-2w6ewn
From this output I picked up the volume group’s name and free space.
2) Create a logical volume to store the disk image for the domU to be created:
[root@jcrealm ~]# lvcreate -n /dev/vg00/lv_asher --size 20G vg00 Logical volume "lv_asher" created
3) Format the newly created logical volume as ext3 fs:
[root@jcrealm ~]# mke2fs -j /dev/vg00/lv_asher
4) Create a directory to mount the newly created fs on:
[root@jcrealm ~]# mkdir -p /xen/asher
Now we need to check the SELinux security context on the newly created directory. Basically, the default security context that SELinux assigned to ‘/xen/asher’ is not adequate for the Xen processes. This means that Xen processes will not have access to that directory, so we need to change that. Xen processes are going to need access to that directory in order to install and amange domU within it. As displayed below, ‘default_t’ is the default security context assigned to the ‘/xen/asher’ directory.
5) Verify the security context:
[root@jcrealm ~]# ls -dZ /xen/asher/ drwxr-xr-x root root user_u:object_r:default_t /xen/asher/
To give Xen adequate access to ‘/xen/asher’ directory we need to change ‘defualt_t’ to ‘xen_image_t’ security context.
6) Change the security context:
[root@jcrealm ~]# semanage fcontext -a \ -t xen_image_t "/xen/asher(/.*)?"
That command appends a line to ‘/etc/selinux/targeted/contexts/files/file_contexts.local’. A tail of that file shall reveal what I am talking about.
[root@jcrealm ~]# tail /etc/selinux/targeted/contexts/files/file_contexts.local /xen/asher(/.*)? system_u:object_r:xen_image_t:s0
7) Where the change was made or written.
[root@jcrealm ~]# tail /etc/selinux/targeted/contexts/files/file_contexts.logal /xen/asher(/*)? system_u:object_r:xen_image_t:s0
8 Use ‘restorecon’ utility to recursively give the directory the right context as following:
[root@jcrealm ~]# restorecon -R /xen/asher
Now, an ls of the same directory shall display the new security context and approve that the change has taken affect.
[root@jcrealm ~]# ls -dZ /xen/asher/ drwxr-xr-x root root system_u:object_r:xen_image_t /xen/asher/
Now, an ls of the same directory shall display the new security context and approve that the change has taken affect.
9) Verify the security context again:
[root@jcrealm ~]# ls -dZ /xen/asher drwxr-xr-x root root system_u:object_r:xen_image_t /xen/asher/
10) Mount the created fs on /xen/asher.
[root@jcrealm ~]# mount -t ext3 /dev/vg00/lv_asher /xen/asher/
In order for this mount to persist reboots, we need to add the new fs to ‘/etc/fstab’.
11) Add the new fs to ‘/etc/fstab’:
[root@jcrealm ~]# echo -e "/dev/vg00/lv_asher\t/xen/asher\text3\tdefaults,rw,nosuid\t1 2" >> /etc/fstab
12) Create relevant image file for the domU to be installed.
[root@jcrealm ~]# dd if=/dev/zero of=/xen/asher/jcweb.img oflag=direct bs=1M count=2028
13) Create a kickstart file and serve it via http:
[root@jcrealm ~]# mkdir /var/www/html/ks [root@jcrealm ~]# touch /var/www/html/ks/jcweb-ks.cfg [root@jcrealm ~]# vi /var/www/html/ks/jcweb-ks.cfg
Add the following content to jdweb-ks.cfg file and save it.
install url --url http://10.20.10.10/ lang en_US.UTF-8 network --device eth0 --bootproto static --ip 10.20.10.21 \ --netmask 255.255.0.0 --gateway 10.20.10.1 \ --nameserver 10.20.10.1,10.20.10.10 --hostname jcweb -–noipv6 # Change with your hashed password rootpw --iscrypted firewall --enabled --port=22:tcp authconfig --enableshadow --enablemd5 selinux --enforcing timezone --utc America/Chicago bootloader --location=mbr --driveorder=xvda --append="console=xvc0" reboot skipx firstboot --disable # Partitioning clearpart --all --initlabel --drives=xvda part /boot --fstype ext3 --size=100 --ondisk=xvda part pv.2 --size=0 --grow --ondisk=xvda volgroup vg01 --pesize=32768 pv.2 logvol / --fstype ext3 --name=lv_root --vgname=vg01 --size=1024 --grow --maxsize=18000 logvol /chroot --fstype ext3 --name=lv_chroot --vgname=vg01 --size=224 --grow --maxsize=524 logvol swap --fstype swap --name=lv_swap --vgname=vg01 --size=256 --grow --maxsize=512 %packages @core --nobase %post ( echo -e "10.20.10.1\t\tbelkin.gl-studio.me\t\tbelkin" >>/etc/hosts echo -e "10.20.10.10\t\tjcrealm.gl-studio.me\t\tjcrealm" >>/etc/hosts echo -e "10.20.10.20\t\tjcreigns.gl-studio.me\t\tjcreigns" >>/etc/hosts echo -e "10.20.10.21\t\tjcweb.gl-studio.me\t\tjcweb" >>/etc/hosts echo -e "10.20.10.23\t\tjcdstore.gl-studio.me\t\tjcdstore" >>/etc/hosts echo -e "10.20.10.25\t\tjcmail.gl-studio.me\t\tjcmail" >>/etc/hosts echo -e "10.20.10.27\t\tjcsec.gl-studio.me\t\tjcsec" >>/etc/hosts echo -e "10.20.10.29\t\tjcman.gl-studio.me\t\tjcman" >>/etc/hosts #/etc/sysconfig/network echo -e "NETWORKING=yes\nNETWORKING_IPV6=no\nHOSTNAME="`hostname`".gt-studio.me\nGATEWAY=10.20.10.1\nGATEWAYDEV=eth0" >/etc/sysconfig/network #/etc/resolv.conf echo -e "domain gl-studio.me\nsearch gl-studio.me belkin.gl-studio.me 10.20.10.1\nnameserver 10.20.10.1" >/etc/resolv.conf #/etc/host.conf echo -e "# Lookup nameds via /etc/hosts firts\n# then, falls back to DNS resolver.\norder hosts,bind\n" >/etc/host.conf echo -e "# I have machines with multiple addresses.\nmulti on" >>/etc/host.conf ) 1>/root/post_install.log 2>&1
Note: notice that the ‘%post’ section of the kickstart file is essentially made up of some commands to configure network on the domU right after the OS’ installation. Therefore, you probably need to make appropriate changes to it to make it suite your network configuration, which may differ from mine. Also, on the top of the kickstart file, the lines starting with: url, network, and rootpw; need to be changed to suit your scenario. In fact, I advise you to customize the kickstart file the way you need it to be because a lot of things such as timezone shall be adjusted base on where in the world you’re located. The kickstart file I have used in for my environment is more complex. I share a simple version here to make things comprehensible.
14) Start http service and verify if the kickstart file (jcweb-ks.cfg) is being served correctly.
[root@jcrealm ~]# service httpd start [root@jcrealm ~]# lynx http://localhost/ks/jcweb-ks.cfg
If you don’t have lynx installed, you can use wget. You simply need to make sure that the kickstart file is being served without problems. In the event that the file is not being served correctly, you need to troubleshoot the issue. A good good place to start is the the error and access logs in /var/log/httpd/.
Supposing the kickstart is being served correctly, now put the DVD of the OS you would like to install in, and manually mount it as followowing.
15) Mount the installation media:
[root@jcrealm ~]# mount /dev/dvd /var/www/html/
This means that all the contents of the DVD is being served on the root directory of the local webserver. Make sure that you do not have any index.html in that directory and go forward.
At this point all you need to do is to launch the final command and let ‘virt-install’ create the new domU (in my case jcweb) and perform an unattended install of the OS.
16) Launch the domU installation command:
[root@jcrealm ~]# virt-install -n jcdweb -r 2048 --vcpus=1 --os-type=linux \ --accelerate -w bridge:br0 --disk "path=/xen/asher/jcdweb.img,size=20" \ -l http://10.20.10.10/ --nographics -x "ks=http://10.20.10.10/ks/jcweb-ks.cfg"
Stick around for for a second to make sure that everything takes off well. Supposing that everything is setup correctly, the installation shall roll smoorthly without any problems. At the end of installation, the newly created domU is going to reboot and your shell prompt will automatically attach to domU’s shell to allow you to login. From this point on, you may loging and set up the new VM or domU system as prefer.
When ‘virt-install’ creates a new domU it also creates its configuration file with the same name as the name of domU given at the moment of installation. This configuration file can be found in ‘/etc/xen/’ directory. Once your domU is up and adequately setup for deployment, you shall move its configuration file from ‘/etc/xen/’ to ‘/etc/xen/auto/’ directory, so that it can be automatically booted whenever VMM ( the host system) is rebooted.
Finally, Xen comes with a very useful CLI utility called ‘xm’. After having installed your domU, issuing a ‘xm list’ will display the newly created domU in addition to your dom0. On part-2 of this virtualization series, I am going to talk about data migratioin, which shall be much shorter because the basics are already covered here.
All rights reserved ©