How to run a network monitoring station on a XenServer VM

I spent way too much time on this seemingly simple task, I hope someone will find this guide useful.

Network monitoring applications require a promiscous mode on a network interface. Easy on a physical machine, didn't work on XenServer. Interface inside a VM switched to promiscous mode, but the traffic was filtered before it reached VM. Only for a brief period I could see all the packets, then only broadcasts got through. Just like on the switch that's learning - that should have pointed me to the right answer.

Open VSwitch

I googled for a solution and found some (e.g. this one from Citrix), but nothing worked. Until I remembered that XenServer 6 uses Open VSwitch by default and all the answers I found were for earlier versions using bridging backend. Now it should be easy. VSwitch works like a high-end, configurable switch so I'll just put the right port into traffic mirroring mode, right? Wrong.

To configure Open VSwitch in XenServer you use a VSwitch Controller virtual appliance. Installation is simple - just download the file, open it with XenCenter and select a VM. Then configure an IP address if you don't use DHCP, point the web browser at it and enter a pool to use. At this point it complained it can't work with free XenServer license. Which Citrix failed to mention on their website.

The other solution

It's always good to have a backup plan. Mine was simple: set XenServer to use the old bridging code. I rejected it at first for two reasons:
- it requires a reboot,
- VSwitch has more features (which I didn't need at the moment, but I might one day) and should perform better.

After scheduling a downtime for VMs, I typed one simple command into a CLI:

xe-switch-network-backend bridge

Reboot and it's done. You can check with 'cat /etc/xensource/network.conf' to be sure.

Forcing promiscous mode - the old way

Now all you have to do is put the right interfaces into promiscous mode. Interfaces - because you need to do it on both PIF and VIF (physical and virtual interface, one that connects to the real network and one that connects to VM). First, find the UUIDs of the interfaces with:

xe vif-list
xe pif-list

If you've got many VMs and NICs, you'll get a long list. You can try filtering, e.g. xe vif-list vm-name-label="My monitoring VM", see xe help vif-list for details. However, some filters didn't work for me. Once you have UUIDs, do:

xe pif-param-set uuid= other-config:promiscuous="true"
xe vif-param-set uuid= other-config:promiscuous="true"

Check if it worked:

xe pif-param-list uuid=
xe vif-param-list uuid=

Disconnect VIF from VM and connect it back:

xe vif-unplug uuid=
xe vif-plug uuid=

Note: unplug/plug only works if you have XenServer Tools installed. Otherwise reboot the VM.

Other possible ways

Some howtos for older XenServers used: brctl setageing xenbr0 0 (or whatever Xen bridging interface you use). I didn't need it, but you can try it if xe ...param-set fails.

I could probably configure VSwitch without Citrix controller appliance. I might one day.