[kwlug-disc] Web references on issues of router behind router?

[unsolicited]

OK, let's pursue this further, it is illustrative to the list.

The situation is probably not atypical, you buy a new router, and want
to play with it on your home net / have confidence in it before
replacing your main gateway.

The scenario is keeping NAT, on both, for the moment.

[ISP(Rogers) 1.2.3.4]
^
|
v
[Router1(PIX)] <-> [Rest of outer LAN 192.168.123.0/24]
^
|
v
[Router2(TrendNET)] <-> [Rest of inner LAN 192.168.234.0/24]

Configuring both routers as though the other weren't there, i.e. as 
though a typical home setup. Inner LAN gets out to internet just fine.

The outer LAN is not successful in talking (initiating) to the inner
LAN, which makes sense as Router2 has built no translation from
outside to inside (by default).

Realistic scenario: Providing a public wi-fi facility, say, to your
neighbours, at a LAN party, a meeting, or whatever. However, in that
case, the inner and outer LANs not being able to talk to each other is
a good thing.

If the inner LAN tries to talk to the outer LAN, Router2 building a
translation in the process, is not successful in talking to the outer
LAN. (See John's excellent explanation.) Note: This is a theoretical
exercise, arguably, one would either not use the WAN port on Router2,
or turn off NAT on Router2. So this is a networking theoretical exercise.

John Van Ostrand wrote, On 05/27/2010 2:50 PM:
> ----- "unsolicited" <unsolicited at swiz.ca> wrote:
>> John Van Ostrand wrote, On 05/26/2010 10:05 PM:
>>> 1. Make sure the subnets are different. e.g. use 192.168.1.x on
>>> one router and 192.168.2.x on the other.
>> Yes, that's taken care of.
>> 
>>> 2. If #1 is good, then you should be able to have the inner
>>> subnet access the outer subnet's systems.
>> Not happening, which is baffling me. It's a Cisco PIX off Roger's
>>  modem, and debug packet inside <subnet>/24 sees nothing.
>> Wireshark on
>> 
>> another machine never sees incoming packets from the WAN IP.

John, given your explanation below, this machine should have seen the
incoming ping (ICMP) and replied. (Not suggesting the reply would get
where desired.) It did not see the packet. [PEBKAC is possible, but I
don't think so.]

>> Darned strange. I expect the inner WAN to MAC find the outer one.
>> Otherwise it should go to the gateway and be routed back
>> internally. [Seems to me coming in and going back out the in is
>> verboten, but I also seem to remember that had to do with
>> routing protocols.] If nothing else, after the first 'go here'
>> the remaining packets would go directly.
> 
> Routing back is generally not done. It's also called hairpinning,
> because of the hairpin one would have to draw to show packet
> travel. I've done it on routers here at the office for convenience,
> now I use split DNS.

Thank you, 'split DNS' was the verboten term I couldn't remember.

Seems 'DNS' in this context is a bit of a misnomer, as we're talking
IP addresses and/or MAC addresses, at this point.

> Hair-pinning doesn't work because of this:
> 
> 1. Device "A" 192.168.1.100 sends packet to device "B" 1.2.3.4 (an
> address on your firewall). 2. Your firewall receives the packet,
> looks to it's NAT table and realizes this packet is for an inside
> system, say 192.168.1.200. It rewrites the destination address from
> 1.2.3.4 to 192.168.1.200 and passes the packet along. 3. Device
> 'B', the destination, gets the packet and sends a reply. The source
> IP for the packet is 192.168.1.200 and the destination is
> 192.168.1.100. The packet is sent directly back to device 'A'
> instead of the firewall since they share a LAN. 4. When Device 'A'
> receives the reply packet it looks to see what connection it
> relates to. It looks in it's table and finds a connection to
> 1.2.3.4, but that can't be it since this packet is from
> 192.168.1.200. It assumes the packet is erroneous and drops it.

Summarizing: The outgoing packet from Router2 built a translation to
Router1, but the incoming packet is from a local (outer LAN) machine,
and Router2 has no such translation, so drops the packet.

[Interesting / makes sense.]

Actually: if the outer LAN machine were in the ARP table, such as by a
ping from Router2, I would have guessed the ping from the Inner LAN to
that same Outer LAN machine would get a direct translation built. The 
MAC redirect from Router1 should have had Router2 update its ARP 
table, certainly in time for the next ping / packet.

Or, even, subsequent pings from inner to outer would start to succeed.
Perhaps the translation would have to expire first, which may have a
timeout longer than the ARP table timeout.

Hold on - isn't this a 'bug'? You would think a router would recognize
the packet is destined for the WAN net, do an arp whois broadcast, and
_then_ build the translation entry. Perhaps another case of chicken
and egg - I've seen where some expected functionality isn't possible
due to the order of processing. Last time I remember when this really
hit was wanting to do both source and address ip and port
translations, it could only do one at a time, and by the time the one
was done, the second couldn't be. Or something. (By the time the 'if
it came from here' rule kicked in, it was no longer 'from here' - half
had been translated.) [Was doing a site to site VPN at the time, and 
neither side wanted the other to know anything at all about the other 
- not IP's, not ports, not nothing.]

> One way to get hair-pinning to work is to perform SNAT on the
> packet on the firewall. That way Device 'B' will send the packet
> back to the firewall and it will be de-SNATed.

Sadly, a capability beyond most residential (read: 'cheap') routers.

> The idea that your inner router (PIX) will perform ARP on it's WAN
> to route to your outer LAN.

Unfortunately, my devices are reversed. PIX is to the ISP. I don't
expect that to change any time soon - it's a much more capable /
functional device than any home router. Some day I'll replace it with
mythbuntu or something, but not any time soon.

> Like this:
> 
> [Inner LAN] ---- [PIX] ---- [Outer LAN] ---- [Linksys?] ----
> [Internet]
> 
> Structurally the Inner LAN to Outer LAN is identical to a typical
> home Internet connection, just change the labels. 192.168 addresses
> are called non-routable and some routers are configured so as not
> to route those addresses.

Sadly, from what I've seen, very few. Best practices seems to indicate
one should block them oneself. Let alone bogons, or martians (which
should be a superset of both.)

Note: From the 2 or 3 Rogers internet shutdown of it's clients I've
seen, it appears they were shutdown because of malware scanning out to
non-existent addresses, starting at 0.0.0.0. Infuriatingly, the shut
down didn't happen until a week later, when virus software had long
killed the malware, so there was nothing to find to determine what had
caused Rogers to shut things down. When I ultimately got to a deep
enough level of Rogers tech to have them say that, he indicated one
can associate an e-mail address with them to receive logs when they do
shut people down - but the e-mail address must be in place before the
situation occurs. I've yet to see such a log, probably because I have
yet to see anyone else shut down since. Never been shut down, myself.
Note this is residential. Paul's experience, Roger's business, is that
he can get zero out of Rogers to explain his frequent shutdowns.

Readers - See:
- http://en.wikipedia.org/wiki/IP_address#IPv4_private_addresses
- http://en.wikipedia.org/wiki/Bogon_filtering
- http://en.wikipedia.org/wiki/Martian_packet
- http://www.team-cymru.org/Services/Bogons/
- http://www.team-cymru.org/Services/Bogons/http.html (Lists)

> It's possible that the PIX has some setting for that.

Probably, but, sadly, not the TrendNet.

>> Problem is I don't have a dual NIC <something> around to put 
>> transparently between the WAN and the rest of the network to see
>> if it's actually sending something out or not. Or the router is
>> just blocking everything to it's WAN subnet. The router can ping
>>  everything, but not something behind the router. And, of course,
>> the outer can't ping the inner - no outgoing nat translation is
>> present.
> 
> How about a linksys with OpenWRT? I bet you could set up port
> mirroring, or use the openwrt packet inspector.

Sadly don't have one of those, either. Thus my comment about being
reminded of Richard's comment of OpenWRT being something to stuff in
the kit bag. Some day, perhaps.

>>> 3. VPNs like openvpn should work fine. IPSec can work as well,
>>> but it's trickier.
>> I'm actually thinking coming in from outside, not the reverse.
>> But it's reassuring to read your point.
> 
> So you have to orchestrate some firewall rules. In the process it
> can be confusing, certainly at first, to keep IP addresses correct.

Situation normal to track this stuff, for me in my network work. It's
just been a while.

>>> 4. PPTP (is anyone still using this?) can have issues with
>>> multiple VPNS sharing the same IP address.
>> Why would you have multiple VPNs? Assuming machine or net to net
>>  connections. After that, it should all just be routing.
> 
> Multiple PCs going out to the same VPN server. I see it
> occasionally with customers.

Yes, as have I. An exercise in education when that happens, and
usually frustration. I can see how it happens though - someone finds
something useful, then others like it too. And when the situation
happens, to go back and correct it with a site to site VPN would mean
an investment of some $$$, and there's no interest. Let alone the
security kerfluffle, at both ends - even when you explain you're
already doing it, just via software instead of hardware.

Sadly, how this all started: Only the WAN port gets ntp, apparently,
on today's residential routers. Certainly on my two. Log times with
the wrong timestamps are really annoying. So, although one might
normally not use the WAN port, one has to to get the time. Without the
WAN port you have a simple (wi-fi) bridge / AP. With it ... 
complexity. The Netgear even outsmarts you - if you set both LAN and 
WAN on the same subnet, it automatically shifts the LAN subnet. Argh.

Thanks for the post John. Hope people find the information here useful.

Still expected the inner and outer lans to see each other. I actually 
expected to have to put in blocks to prevent it. Curious, although the 
explanation makes sense.