FatMin

XenServer Switch Ports Configuration Best Practices

Finally I have found it! Citrix's XenServer switch configuration best practices document.

While everyone in the world has blog posts and documentation regarding how to set up and configure bridged networks in Xen, they hardly ever go into the physical switch configuration required.

This is the document that you will need to pass along to your friendly Network Administrator, as they will more than likely not be familar with networking for Xen as its much different from networking for Vmware ESX.

http://support.citrix.com/article/CTX123158

The contents of the document above are outlined below.

Change the following options on the switches for XenServer ports:

Enable PortFast on XenServer connected ports.
PortFast allows a switch port running Spanning Tree Protocol (STP) to go directly from blocking to forwarding mode by skipping the learning and listening modes. PortFast should only be enabled on ports connected to a single host. Port must be an 802.1q trunk port if you are using VLANS and the port must be in access mode.
Ports used for storage should have PortFast enabled.
Note: It is important that you enable PortFast with caution, and only on ports that do not connect to multi-homed devices such as hubs or switches.
Disable Port Security on XenServer connected ports.
Port security prevents multiple MAC addresses from being presented to the same port. In a virtual environment, you see multiple MAC addresses presented from Virtual Machines to the same port. If you have enabled Port Security, it shuts down the port.
Disable Spanning Tree Protocol on XenServer connected ports.
Spanning Tree Protocol must be disabled if you are using Bonded or teamed NICs in a virtual environment. Spanning Tree Protocol should be disabled because of the nature of Bonds and NIC teaming, to avoid failover delay issues when using bonding.
Disable BPDU guard on XenServer connected ports.
BPDU is a protection setting part of the STP that prevents you from attaching a network device to a switch port. When you attach a network device, the port shuts down and has to be enabled by an administrator.
A PortFast port should never receive configuration BPDUs.
Note: When BPDUs are received by a PortFast port, it indicates another bridge is connected to the port, and it indicates that there is a possibility of a bridging loop formation during the Listening and Learning phases. In a valid PortFast configuration, configuration BPDUs should never be received, so Cisco switches support a feature called PortFast BPDU Guard, which is a feature that shuts down a PortFast-enabled port in the event a BPDU is received. This feature ensures that a bridging loop is not formed, because the switch's shutting down the port removes the possibility of a loop forming.

Forcibly Remove Storage Devices From XenCenter via CLI

This is my second post in focused on forcibily something from XenCenter, as apparently XenCenter has a hard time knowing when to "forget" a device or host on its own. Now while this is annoying, it can be resolved rather easily from the command line.

Now, lets take a step back and reflect for a moment on the first post in this series... you can find it here. Anyway this first post showed us how to remove the orphaned device via the Windows CLI, however I find the process of using the CLI in Windows to be painful, so this post is going to outline a similar process but via the CLI on the XenServer host itself.

Anyway in the previous post I had to force remove a host from XenCenter via the CLI, and in this post I need to remove the "Local Storage", "Removable Storage" and "DVD Drive" that were associated with the orphaned host. These are the items in red below.

By clicking on each one of the items in XenCenter I can see that they are all in a Detached State, plus i can see the UUID of each item. You will need the UUID for the next step.

Now ssh into your master node and run the following

#xe sr-list

Look for the entry with the matching UUID

uuid ( RO)                : 64e69b7d-ae97-4d42-c4d8-c260553b31d1
name-label ( RW): Local storage
name-description ( RW):
host ( RO): <not in database>
type ( RO): lvm
content-type ( RO): user

Then run the following command to remove it from XenCenter Inventory

# xe sr-forget uuid=64e69b7d-ae97-4d42-c4d8-c260553b31d1

Now rinse and repeat for any remaining orphaned items.

 

Forcibly Remove a Host From XenCenter via CLI

Welcome back to another XenServer adventure. Today we are going to review the process of removing an orphaned XenServer host from XenCenter via the command line.

A few days ago one of my Xen hosts died due to an issue with the raid controller. In order to revive it I had to rebuild it, and add it back into the pool. However there was a problem; the original entry for this host in XenCenter appeared in red, which indicated that it was disconnected, and another entry for the same exact host with the same exact hostname appeared next to it in the host list for this particular cluster. I was unable to remove the orphaned host via XenCenter as there was no remove option.

I was in a pickle.

However I was able to find a solution.

First and formost it appears that there are actually two ways to do this; the first one being from the Windows command line where you are running Xencenter, or on the master node in the cluster.

This example is from the Windows CLI. Note that the UUID that is output to the command line is the UUID of the Host that is unreachable -- the orphaned host.


C:\Program Files\Citrix\XenCenter>xe -s <Hostname_of_master> -u root -pw <root_password> pool-sync-database
You attempted an operation which involves a host which could not be contacted.
host: 560e233b-0e8b-4c2f-a641-cf1876630a6b (virt04.atlc1)

C:\Program Files\Citrix\XenCenter>xe -s <Hostname_of_master> -u root -pw <root_password> host-foget uuid=560e233b-0e8b-4c2f-a641-cf1876630a6b
WARNING: A host should only be forgotten if it is physically unrecoverable;
WARNING: if possible, Hosts should be 'ejected' from the Pool instead.
WARNING: Once a host has been forgotten it will have to be re-installed.
WARNING: This operation is irreversible.
Type 'yes' to continue
yes

Returning to XenCenter I found that the orphaned host entry had been removed.

 

Xenserver: How To Create A Custom Kickstart Template via the CLI

If you are reading this post, then you should know that I have been spending a lot of time as of late trying to learn XenServer, and I am doing my best to get Xenserver to do my evil bidding.

When I first took a look at XenServer I was dissapointed to find that you cannot PXE boot a VM unless you use the "Other Install Media" Template. However, when you use this template you are not building a fully paravirtualized vm, and you loose some functionality on your vm (like hot adding a virtual disk).

So lets say you want to kickstart a Centos 5 64-bit vm. Traditionally in XenServer,  you need to create a new vm based on the "Centos 5 (64-bit)" template and then point your vm to your kickstart media and ks config file. Being that this is a manual process, and I am trying to automate building virtual machines, I started searching for a better way to make Xenserver do what I wanted... I think I may have accomplished my goal.

So the first thing I did was create a new vm via cli. This step spits out a UUID for your new vm.

#xe vm-install template=CentOS\ 5\ \(64-bit\) new-name-label=Centos5.4_Kickstart

Now setup your boot params to point your new vm to your kickstart config file

#xe vm-param-set uuid=0415bc6c-6129-9bc2-26d7-e15625cf85a1 PV-args="ks=http://<my_kickstart_server>/kickstart/ks/centos5-u4_x86_64.cfg ksdevice=eth0"

Then tell your new vm where to find its install DVD.

#xe vm-param-set uuid=6aaf7e10-59e4-9895-9c7b-6eee32eab9f1 other-config:install-repository=http://<my_kickstart_server>/centos5-u4-x86_64/disc1/

Now figure out the UUID of your Kickstart VLAN

#xe network-list

Create a VIF (virtual interface) on your kickstart vlan.

#xe vif-create vm-uuid=0415bc6c-6129-9bc2-26d7-e15625cf85a1 network-uuid=f5a61f5b-f17c-ac40-0995-c41c3a5a3ea3 device=0

Now on the next step i cheated - I used XenCenter to quick create a vm based on my new template.

Now when I create a new vm from this template, it startes to kickstart on boot. My next steps are to create multiple templates, each based on my different kickstart images/configs. Then figure out how to set their ips and hostnames.

Hopefully that post is coming soon.

 

How to Create A Dedicated Storage NIC in XenServer

In Vmware ESX, when using NFS storage, you are required to create a separate and additional vmkernel portgroup to access your NFS storage. This way ESX Management traffic travels over one port group and NFS traffic travels over another.

In Xenserver the concept is similar; however executed much differently.

First and foremost you need to be aware of a few limitations in XenServer. While XenServer does allow you to create a dedicated nic (or bond) for NFS traffic, this nic must remain unmanaged by Xenserver. In contrast, a XenServer's management traffic travels over its "Management interface", which obviously has to be an interface that is managed via XenServer.

Allow me sum this up. You cannot share a nic or nics between management traffic and NFS traffic. This means that you are going want to create a bond for management traffic and a bond for NFS traffic you are going to need 4 free interfaces on your Xenserver box.

Now it is possible to use VLAN tagging and route your Management traffic and virtual machine traffic over the same physical interfaces. However its important to know that XenServer does not support VLAN tagging on the Management interface, so whatever VLAN you use for management, it must be the native vlan on the ports configuration.

The information below is from the Admin Guide for XenServer 5.5

Switch ports configured to perform 802.1Q VLAN tagging/untagging, commonly referred to as ports with a native VLAN or as access mode ports, can be used with XenServer management interfaces to place managementtraffic on a desired VLAN. In this case the XenServer host is unaware of any VLAN configuration.

XenServer management interfaces cannot be assigned to a XenServer VLAN via a trunk port.

Bottom line; its probably best to have seperate physical connections for your management traffic and NFS traffic.

Also, according to the Admin Guide, your NFS network should not be routed. See the words below and read them thusly.

"Before dedicating a network interface as a storage interface for use with iSCSI or NFS SRs, you must ensure that the dedicated interface uses a separate IP subnet which is not routable from the main management interface. If this is not enforced, then storage traffic may be directed via the main management interface after a host reboot, due to the order in which network interfaces are initialized."

Ok now that we got all that out of the way, lets actually create our dedicated storage nic.

First we need to get the uuid of the pif (physical interface) that we want to use. Note that this is just an example using a standalone interface.

#xe pif-list host-name-label=<"your_hostname>

Next we reconfigure our pif

#xe pif-reconfigure-ip mode=statc IP=<your-ip-on-nfs-vlan> netmask=<your-mask> uuid=<pif-uuid>

#xe pif-param-set disallow-unplug=true uuid=<pif-uuid>

#xe pif-param-set other-config:ManagementPurpose="Storage" uuid=<pif-uuid>

Alternatively you can use xe pif-forget to remote the interface from Xencenter database and configure it manually via the XenServer Kernel like you would any other interface in Linux, however this could be more confusing in the long run.

XenServer 6.0 Quick Start Guide

XenServer 6.0 Quick Start Guide

http://support.citrix.com/article/CTX130419

XenServer 6.0 Release Notes & Administrator Guide

Hello World!

XenServer 6.0 was released in late September, 2011. I am currently working on building out my test cluster, as I expect to role out our first XenServer Server (hey that's redundant) in the next few weeks. I expect to be posting a lot more on the subject as I become more familiar with the product.

 

Release notes below.. includes new features:

http://support.citrix.com/article/CTX130418

Xenserver 6.0 Administrators Guide below for those of you who can read.

http://support.citrix.com/article/CTX130421

 

Citrix XenServer Tech Specs - Minimums & Maximums

The link below is basically the minimums and maximums document from Xenserver 5.6. Have been looking for this document for days and now I have it, and its all mine.

XenServer Tech Specs -- Link

Below are the specs for a physical host for those of you who do not care to click on the link above. Look 512GB memory ... nice!

 

• 64-bit x86 server-class system 
• CPU: 1.5 GHz minimum, 2 GHz or faster multi-core recommended
• Intel® VT or AMD-V™ required for support of Windows guests
• 2GB to 512GB physical memory
• Up to 64 logical processors
• 100Mb/s or faster NIC
• Up to 16 physical NICs
• Local, Fibre Channel or SAN boot disk with 16 GB of space minimum, 60 GB or more recommended

Getting Started with XenServer: Features and Licensing

This post is pretty much a bunch of links that I have found that I am using to get up to speed with ZenServer. I've got a new virtualization project just around the corner and Vmware is no longer a good fit for us.

The link below is from Citrix.com and compares the various features in the different versions of XenServer.

^{http://www.citrix.com/English/ps2/products/subfeature.asp?contentID=2300456}

And the link below explains XenServer licensing. There are three licensed versions of XenServer, and of course a free version. The specific version that I am looking at is Enterprise, which only lacks a few features available in Platinum. Street price for Enterprise is $2500 per server.

^{http://www.citrix.com/English/ps2/products/subfeature.asp?contentID=2313292}

If you are interested in test driving a licensed version of XenServer, you can use the link below to register for a 45 day trial.

^{http://deliver.citrix.com/go/citrix/ceXenServerTrial}

Last but not least, the link below is to Citrix's documentation portal.

^{http://support.citrix.com/productdocs/}

Now just need to find the minimum and maximums docs and I should be good to go.

 

Considering Alternatives to VMware

Interested in doing some serious virtualization on Cisco's new UCS C-Series line of servers? Seriously, why would'nt you be.  Cisco's UCS C460 M2 server supports up to 4 intel E7-4800 processors -- these are 10 core procs people -- and up to 1tb of memory. Thats 40 cores in 4 RU!

I am eager to get one of these servers, or even one or the two proc models into our lab and test them out. I have been hearing great things about them and think they would be perfect virtualization servers for our non-production infrastructure where we seem to have at least 6 or 7 different QA environments. Plus Vmware and Cicso partnered together on these things.

But slow your roll cowboy, with Vmware's recent licensing announcement regarding Vsphere 5 and vRam limitations, it would take 11 Enterprise plus licenses to utilize 1tb of memory. At 3 to 4k per license you are talking about 40k in licensing for one server. Nasty!

Enter the alternatives in the virtualization world. Citrix ZenServer and Redhat (RHEV-M). Hell I'll even mention Microsoft's Hyper-V.  However, to move to a different technology you need to be able to get your Vmware vms off of ESX and onto your new virtualization flavor of choice.

Here is how to do it in XenServer. Note that I have not tried this in the lab yet, but trust me i will. I will also do the same with RHEV-M, its its only $750 per socket per year.

http://support.citrix.com/article/CTX116603

Sure the alertnatives are not as advanced as Vmware ESX right now, but they making improvements slowly but surely, and I am sure that the recent Vmware licensing announcements are only going to bring more potential customers to other virtualization vendors which will only help to further mature their products.

Find out more about the Cicso USC C-Series via the link below

http://www.cisco.com/en/US/prod/ps10265/ps10493/c_series_comparison.html