[OpenIndiana-discuss] multiple IP addresses, same NIC

[Doug Hughes]

On 3/5/2013 7:38 PM, Edward Ned Harvey (openindiana) wrote:
> All subnets have been renamed to protect the innocent.  ;-)
>
> At home, I use 192.168.1.x /24, and unfortunately, I need to VPN to work where they use both 192.168.1.x /24 and 192.168.10.x /24.  Fortunately, I don't need to access any of the 192.168.1.x machines at work; I only need to access 192.168.10.x.  So I invented a new network at home, 192.168.2.x /24.  My firewall has 192.168.1.1 and also 192.168.2.1 on the same LAN NIC.  My OI box has 192.168.1.100 and also 192.168.2.100 (again, two separate subnets on the same NIC).  My Windows VM has 192.168.1.101 and also 192.168.2.101.  For both my OI and Windows boxes, the default GW is 192.168.1.1.  And for both machines, I added a static route, 192.168.10.x via 192.168.2.1.  Both OI and Windows can ping 192.168.2.1, and 192.168.1.1, and everything on both the local LANs, and the internet.
>
> On my windows box, ping 192.168.10.x is also fine, meaning on windows, everything is fine.
>
> On my OI box...  Despite successfully doing everything else, I still can't reach 192.168.10.x.  Thanks to wireshark, I can see what's happening:
>
> When I ping 192.168.2.1, I look at the packet capture, and I can see source IP address 192.168.2.100, destination MAC address belongs to 192.168.2.1.  This is perfect.  ping 192.168.2.1 works perfectly.  And also ping internet, and LAN, but the problem is ...
>
> When I try to ping 192.168.10.x, I see the packet go out my OI network interface, source IP address 192.168.1.100 (which is wrong) route via the MAC address of 192.168.2.1 (which is right).  The source IP and the router IP are not in the same subnet with each other, but because they're on the same physical wire with no VLAN separation, the packet actually reaches the router.  Which is then, of course, confused and unable to route the packet.  The desired behavior is to send *from* 192.168.2.100 and route via 192.168.2.1.  Apparently, the OI system gets confused because both subnets are on the same NIC.
>
> I can't see any reason why the system would correctly use 192.168.2.100 as the source IP when I'm pinging 192.168.2.1, but then use a different source IP when I'm pinging something that gets routed via 192.168.2.1
>
> Anybody have any pearls of wisdom for me here?  Any other info I could provide, such as the routing tables, or the commands I used to configure the IP's and routes?
>
> Thanks, if you know anything....
> _______________________________________________
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss at openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss
>
it sounds like your default route is via 192.168.1.X (confirmed with 
netstat -nr)

If that's the case, and you want to change the route, you have a couple 
of choices.

1) you can change the default router to 192.168.2.1 (delete current, add 
new with route add default 192.168.2.1)
2) explicitly set the route for 192.168.10.x :
route add 192.168.10.0/<mask> 192.168.2.1

another solution you could use is bidirectional NAT. But it may be more 
complication than its worth since you appear to be so close.