Solaris has a nifty utility named "dladm" which creates "dynamic" links.It can be used to:
- work with virtual network interfaces
- work with wireless interfaces (WiFi)
- work with virtual switches (named etherstubs and bridges)
For our Xen-based xVM environment, a virtual switch to connect DomU's to an internal network is a common configuration. This virtual switch may even have an interface to the Dom0 system - the path to the outside world.
Look at the configuration example on the picture above (click to enlarge it).
This configuration would appear like this on our Solaris host:
pascal@teroknor:~# dladm show-linkFirst some Solaris definitions:
LINK CLASS MTU STATE BRIDGE OVER
nge0 phys 1500 up -- --
nge1 phys 1500 up -- --
e1000g0 phys 1500 up mybridge --
e1000g1 phys 1500 up mybridge --
xenswitch1 etherstub 1500 unknown -- --
xenswitch2 etherstub 1500 unknown mybridge --
loveit0 vnic 1500 up -- xenswitch1
xvm17_1 vnic 1500 up -- xenswitch1
xvm19_1 vnic 1500 up -- xenswitch2
xvm14_1 vnic 1500 up -- nge0
- A physical interface (phys) is an interface controlled by a hardware driver. These interfaces are physically present. Examples include an ethernet interface or a fiber channel ip interface.
- A virtual interface (vnic) is a network interface only known by the kernel which can be used in software like a real interface. You may use them in Dom0 or in any DomU you want (the latter by assignment via xm or virsh).
- An etherstub is like an unmanaged dumb ethernet switch for virtual interfaces, traffic originating from connecting virtual links to the etherstub is directed to the right direction (layer 2) to other virtual interfaces "connected" to this etherstub. A virtual interface MUST be connected to an etherstub OR a physical hardware interface otherwise it will not work - you may even not define it, the dladm command will abort with an error.
- A bridge is a 802.1d instance
making a real bridge instance out of an etherstub. Physical interfaces
can be "connected" directly to a bridge while virtual ones must be
connected to an etherstub which itself is member of the bridge.
The first example on the graphic is easy: A virtual interface is directly connected to a physical one. The kernel acts like an unmanaged switch between these two interfaces. You may "connect" as many virtual interfaces as you want to a real interface.
Our vnic "xvm14" was easy to accomplish:
And voilà, our xvm14 network interface is ready. In fact, this is exactly what the xen daemon does when starting a virtual machine - it hooks dynamically virtual interfaces to the appopriate "bridges" (be careful: In Xen terminology a "bridge" is not a "bridge" in the Solaris terms: For Xen it just means a virtual switch - so remember: a Xen bridge is a Solaris etherstub).
The second example has been configured like this:
The vnic xvm_17 was created automatically by the xen daemon.
The "dladm set-linkprop -p mtu=1500" is IMPORTANT because etherstubs are created with a default MTU of 9000 bytes - OpenSolaris just crashes badly when you try to use a vnic with this MTU as a Xen interface...
In the Dom0 Solaris the new interface "loveit0" can be used just like any other one:
This interface can be used to route/forward, including NAT (refer to routeadm and ipnat/ipf).
The third example includes a real bridge between vnic(s) and physical interfaces:
If you want to install a DomU using "virt-install" just use the "--bridge" (Xen-bridge -> etherstub) commandline option to select the right etherstub to connect to.
Example:
This installation will connect a Xen interface to the "xenswitch1" etherstub. If this Windows DomU has the IP address 192.168.200.2/24 it will be able to talk to the Dom0 by using 192.168.200.1 as a target. To connect this Windows host to the outer network you will have to turn IP forwarding on the OpenSolaris Dom0 and - depending on your network configuration - you'll have to turn on NAT (via /etc/ipf/ipnat.conf and svcadm enable ipfilter).
If you omit the "--bridge" parameter virt-install will just define a vnic connected directly to the first configured hardware interface in your machine - if that's your network link to your LAN the virtual machine will be part of that LAN.
Our vnic "xvm14" was easy to accomplish:
# dladm create-vnic -l nge0 xvm14And voilà, our xvm14 network interface is ready. In fact, this is exactly what the xen daemon does when starting a virtual machine - it hooks dynamically virtual interfaces to the appopriate "bridges" (be careful: In Xen terminology a "bridge" is not a "bridge" in the Solaris terms: For Xen it just means a virtual switch - so remember: a Xen bridge is a Solaris etherstub).
The second example has been configured like this:
# dladm create-etherstub xenswitch1
# dladm set-linkprop -p mtu=1500 xenswitch1
# dladm create-vnic -l xenswitch1 loveit0The vnic xvm_17 was created automatically by the xen daemon.
The "dladm set-linkprop -p mtu=1500" is IMPORTANT because etherstubs are created with a default MTU of 9000 bytes - OpenSolaris just crashes badly when you try to use a vnic with this MTU as a Xen interface...
In the Dom0 Solaris the new interface "loveit0" can be used just like any other one:
# ifconfig loveit0 plumb
# ifconfig loveit0 192.168.200.1 netmask 0xffffff00 broadcast 255.255.255.0 up
# ifconfig loveit0
loveit0: flags=1100843<UP,BROADCAST,RUNNING,MULTICAST,ROUTER,IPv4> mtu 1500 index 4
inet 192.168.200.1 netmask ffffff00 broadcast 192.168.200.255
ether 2:8:20:8a:35:b5This interface can be used to route/forward, including NAT (refer to routeadm and ipnat/ipf).
The third example includes a real bridge between vnic(s) and physical interfaces:
# dladm create-bridge mybridge
# dladm create-etherstub xenswitch2
# dladm set-linkprop -p mtu=1500 xenswitch2
# dladm add-bridge -l xenswitch2 mybridge
# dladm add-bridge -l e1000g0 mybridge
# dladm add-bridge -l e1000g1 mybridgeIf you want to install a DomU using "virt-install" just use the "--bridge" (Xen-bridge -> etherstub) commandline option to select the right etherstub to connect to.
Example:
# virt-install --hvm --cdrom=/rpool/ISO/windows2008r2.iso --ram 2048 --disk path=.... --bridge xenswitch1 --vnc --os-type windows --os-variant windows --name "testdomu"This installation will connect a Xen interface to the "xenswitch1" etherstub. If this Windows DomU has the IP address 192.168.200.2/24 it will be able to talk to the Dom0 by using 192.168.200.1 as a target. To connect this Windows host to the outer network you will have to turn IP forwarding on the OpenSolaris Dom0 and - depending on your network configuration - you'll have to turn on NAT (via /etc/ipf/ipnat.conf and svcadm enable ipfilter).
If you omit the "--bridge" parameter virt-install will just define a vnic connected directly to the first configured hardware interface in your machine - if that's your network link to your LAN the virtual machine will be part of that LAN.
Hi,
perfect howto! :-) Good Work.
But what will happen if vlans ar added in this configuration?
should i configure vlan interface on e1000g's but i can't connect a vlan interface to mybridge?
I need help!
Look at
http://southbrain.com/south/2009/12/dladm-vlan-solaris-xvm.html
:-)