[OpenIndiana-discuss] CIFS performance issues

[Robin Axelsson]

On 2012-01-25 21:50, James Carlson wrote:
> Robin Axelsson wrote:
>>> I'm confused.  If VirtualBox is just going to talk to the physical
>>> interface itself, why is plumbing IP necessary at all?  It shouldn't be
>>> needed.
>> Maybe I'm the one being confused here. I just believed that the IP must
>> be visible to the host for VirtualBox to be able to find the interface
>> in first place but maybe that is not the case. When choosing an adapter
>> for bridged networking on my system, the drop-down menu will give me the
>> options e1000g1, e1000g2 and rge0. So I'm not sure how or what part of
>> the system that gives the physical interfaces those names. I mean if the
>> host can't see those interfaces how will VirtualBox be able to see them?
>> At least that was my reasoning behind it.
> The names come from the datalink layer.  It has nothing whatsoever to do
> with IP.  IP (like many other network layers) can open and use datalink
> layer interfaces if desired.
>
> They're quite distinct in terms of implementation, though the user
> interfaces (and documentation :-<) tend to blur the lines.  The datalink
> layer object is managed by dladm.  It has a name that defaults to the
> driver's name plus an instance number, but that can be configured by the
> administrator if desired.  There are also virtual interfaces at this
> level for various purposes.  You can think of it as being the Ethernet
> "port," assuming no VLANs are involved.
>
> The IP layer object is managed by ifconfig.  It's used only by IP.
> Other protocols don't (or at least _shouldn't_) use the IP objects.  In
> general terms, these objects each contain an IP address, a subnet mask,
> and a set of IFF_* flags.
>
> By default, the first IP layer object created on a given datalink layer
> object has the same name as that datalink layer object -- even though
> they're distinct ideas.  The second and subsequent such objects created
> get the somewhat-familiar ":N" addition to the name.  It's sometimes
> helpful to think of that first object as being really named "e1000g1:0"
> at the IP layer, in order to keep it distinct from the "e1000g1"
> datalink layer item.

This sounds like an implementation of the OSI model which separates the 
datalink layer from the network layer.  When speaking of blurred lines, 
it seems that the line between the network layer, transport layer and 
session layer (as specified in the OSI model) is also quite blurry.

> Since VirtualBox is providing datalink layer objects to the guest
> operating system (through a simulated driver), it needs access to the
> datalink layer on the host system.  That means "e1000g1" or something
> like that.
>
> It doesn't -- and can't -- use the IP layer objects.  Those allow you to
> send only IP datagrams.
>
> If VirtualBox used the IP objects from the host operating system, what
> would happen when the guest attempts to send an ARP message or (heaven
> help us) IPX?  Those work at a layer below IP.

There are different ways to virtualize networking in VirtualBox. Apart 
from bridged networking, which is also used by default by hypervisors 
such as Xen (don't know about KVM), you can also use NAT.

In NAT mode the VirtualBox hypervisor acts as a virtual router that sits 
between the IP stack of the host and the VMs. It has its obvious 
limitations as you have pointed out which is why I don't use this mode. 
This mode also has (or at least had in prior versions of VirtualBox) 
stability/performance issues.

>
>>> There's probably a way to do this with ipadm, but I'm too lazy to read
>>> the man page for it.  I suggest it, though, as a worthwhile thing to do
>>> on a lazy Sunday afternoon.
>> I'll look into it if "all else fails". I see that the manual entry for
>> ipadm is missing in OI. I will also see if there is more up-to-date
>> documentation on the ipmp. I assume that when a "ClientID" value is
>> generated a MAC address also comes with it, at least when it negotiates
>> with the DHCP server.
> Nope.  See section 9.14 of RFC 2132 for a description of the DHCP
> "Client-identifier" option, and the numerous references to client
> identifiers in RFC 2131.
>
> Back in the bad old days of BOOTP, clients were in fact forced to use
> hardware (MAC) addresses for identification.  That's still the default
> for DHCP, just to make things easy, but a fancy client (such as the one
> in OpenIndiana) can create client identifiers from all sorts of sources,
> including thin air.

I guess that this ClientID feature is only used by more advanced 
routers, because all my machines (virtual and physical) are identified 
by a given MAC address in the router. Or maybe it is capable of ClientID 
and I'm not aware of it, the router documentation is mum about this 
feature though.

>>> Try "man ifconfig", "man ipmpstat", "man if_mpadm".  Those should be
>>> reasonable starting points.
>> Thanks, these man pages exists. I saw in the ifconfig that there is some
>> info about ipmp although it is brief.
> It's possible that some of meem's "Clearview" project documentation that
> revamped large parts of IPMP are available on line as well.

I'll look into it, I see that it is on one of those Solaris/Oracle blogs.

>
>>> Or perhaps you're running NWAM, and that daemon is undoing your work
>>> behind your back.  You probably don't want to use NWAM with a reasonably
>>> complex system configuration like this.
>> I think it is a bit strange that the changes only apply to the IPv4
>> settings but maybe it doesn't matter as the router only uses IPv4 (I
>> think).
> You have to tell ifconfig what you want to do.  If you want to modify
> the separate IPv6 interfaces, then specify "inet6", as in:
>
> 	ifconfig e1000g0 inet6 unplumb

Ok. When I plumb/unplumb a connection with ifconfig, will this 
(un)plumbing be permanent or only be in effect until next reboot or 
power cycle? I would expect that this plumbing configuration is in some 
/etc/<config> file somewhere...

>> Hmm, I'm starting to wonder how netmasks and subnets work in
>> IPv6 as none appears to be specified in ifconfig -a.... I'm starting to
>> realize that you don't need nwam for dhcp.
> The prefix certainly should be there as "/n".  It's basically the same
> as in modern IPv4, except with 128 bits instead of 32.
>
> In modern -- CIDR -- IPv4, you don't normally refer to a subnet mask as
> something like 255.255.255.128, but rather as a prefix length, like
> "/25".  Besides being more compact, the prefix length notation avoids
> dumb (and pointless) mistakes that occur when someone accidentally
> specifies non-contiguous mask bits.

Ok, so I assume that the /n means that the /n/ least significant 
consecutive bits are 0 in the network mask. So that means I'm no longer 
able to specify network masks such as 170.85.144.70 where for example 
144 in binary form is 10010000, i.e. that the zeroes in the mask are not 
consecutive/contiguous as there are ones in between.

So I understand this restriction is enforced so as to prevent the 
accidental creation of subnets that not mutually exclusive, i.e that 
overlap each other.

>
>> "dladm" you say. I trust that VirtualBox does what it needs in that
>> regard and that I have to worry about it, for now.
> I should hope so.
>
>>> ifconfig(1M), unfortunately, comes from BSD which blurred the lines
>>> between datalink and network layers.  They're really quite distinct in
>>> OpenIndiana (and Solaris before it), but the confused user interface in
>>> ifconfig sadly leads users astray.
>> I see... I read on a forum that when unplumb fails it sometimes helps to
>> issue the 'ifconfig<IFP>  down' command before unplumbing the
>> connection. But that was a BSD (don't know which flavor) forum.
> At least in terms of Solaris/OpenSolaris/OpenIndiana, I can't make heads
> or tails out of that suggestion.
>
>>> That seems likely.  I think the whole thing was set in doubt when two
>>> separate interfaces were configured up on the same subnet.  Everything
>>> past that point sounds like the experience of a test pilot.  Some of the
>>> control surfaces departed the craft in flight, but, hey, if that weren't
>>> one of the possibilities, then we wouldn't need these brave souls.
>>>
>> I tried for a change to look into the Graphical Network Adminstration
>> tool in Gnome (/usr/lib/nwam-manager-properties) and the e1000g2
>> connection was disabled there even though 'ifconfig -a' showed that it
>> wasn't. I also verified with ping that this connection was not reponding
>> (ping 10.40.137.171) but that could be because I have done 'ifconfig ...
>> down' on it.
> Right.  Marking it "down" means that it is no longer administratively
> available.
>
>> When I shut down the virtual machine, the e1000g2 connection disappeared
>> from 'ifconfig -a'. So it seems that the VM somehow blocked/prevented
>> ifconfig to apply the unplumbing and once the VM was shut down the
>> unplumbing kicked in first then.
> That's weird.  I don't know enough about VirtualBox, though, to know how
> or why that would happen.  I don't think it should.  It kinda sounds
> like a bug.

Yes, there is a bug somewhere, the question is which part of the system 
is at fault here; VirtualBox and the vboxflt driver or OpenIndiana and 
its IP stack?

>> I have now managed to disable the e1000g2 and the rge0 interface
>> permanently by the nwam-manager-properties program (by 'Edit"ing  the
>> Network Profile). Both connections have now disappeared from 'ifconfig
>> -a' (also the IPv6 part is gone as well), even after a reboot, so I
>> consider them permanently unplumbed. It looks like this when issuing
>> 'ifconfig -a':
>>
>> lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL>  mtu
>> 8232 index 1
>>          inet 127.0.0.1 netmask ff000000
>> e1000g1: flags=1004843<UP,BROADCAST,RUNNING,MULTICAST,DHCP,IPv4>  mtu
>> 1500 index 2
>>          inet 10.40.137.185 netmask ffffff00 broadcast 10.40.137.255
>> lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL>  mtu
>> 8252 index 1
>>          inet6 ::1/128
>> e1000g1: flags=20002004841<UP,RUNNING,MULTICAST,DHCP,IPv6>  mtu 1500 index 2
>>          inet6 fe80::6a05:caff:fe01:da8e/10
>>
>> The VirtualBox Qt GUI still recognizes all physical interfaces (e1000g1,
>> e1000g2 and rge0) even though they are disabled from the host which is
>> good.
>>
>> So I guess that once again the bets are on and all is good then...
> That sounds much better to me.
>

I have now tortured my system for a while and I have not yet been able 
to reproduce these freeze-ups that I used to have so it seems that 
everything is fine now. I will continue to keep an eye on this for a few 
weeks though.

I'm happy to have received responses from someone as knowledgeable such 
as yourself and I appreciate the time you have taken to help me (and 
hopefully others in this mailing list) with this issue!

Robin.