[PLUG] Routing problem...

Wed Dec 23 05:56:55 UTC 2009

On Tue, Dec 22, 2009 at 12:50 PM, Michael Robinson <plug_1 at robinson-west.com
> wrote:

> > what DOES happen when the destination is 192.168.1.0/24? Can you provide
> a
> > traceroute?
> >
> > I don't know what "table 3" means, but I'm pretty sure that if it doesn't
> > show up in "ip route list" it's not going to be effective.
> >
> > >From the ip(8) man page:
> >
> > ...
> > Route tables: Linux-2.x can pack routes  into  several  routing  tables
> > identified  by  a number in the range from 1 to 255 or by name from the
> > file /etc/iproute2/rt_tables main table (ID 254) and  the  kernel  only
> > uses this table when calculating routes.
> > ...
> >
> > On my system, ip route list gives the same output as ip route list table
> > 254. I would expect the same result on your system.
> >
> > -wes
>
> Dodo is a Fedora Core 1 network root system:
> Linux dodo.w2.robinson-west.pri 2.4.22-1.2115.nptlcustom2 #1 Sun Jun 29
> 20:59:36 PDT 2008 i686 i686 i386 GNU/Linux
>
> [root at dodo firewall]# ping -c3 192.168.1.1
> connect: Network is unreachable
> [root at dodo firewall]#
>
> [root at dodo firewall]# ip rule show
> 0:      from all lookup local
> 32764:  from all fwmark 0x3 lookup 3
> 32765:  from all fwmark 0x2 lookup 2
> 32766:  from all lookup main
> 32767:  from all lookup 253
> [root at dodo firewall]#
>
> [root at dodo firewall]# ip route show table local
> local 192.168.3.1 dev eth0  proto kernel  scope host  src 192.168.3.1
> local 192.168.3.17 dev eth0  proto kernel  scope host  src 192.168.3.17
> broadcast 192.168.3.0 dev eth0  proto kernel  scope link  src
> 192.168.3.1
> broadcast 192.168.3.16 dev eth0  proto kernel  scope link  src
> 192.168.3.17
> broadcast 192.168.0.255 dev eth1  proto kernel  scope link  src
> 192.168.0.2
> broadcast 127.255.255.255 dev lo  proto kernel  scope link  src
> 127.0.0.1
> local 192.168.5.2 dev eth1  proto kernel  scope host  src 192.168.5.2
> local 192.168.5.3 dev eth1  proto kernel  scope host  src 192.168.5.2
> local 192.168.5.4 dev eth1  proto kernel  scope host  src 192.168.5.2
> broadcast 192.168.4.15 dev eth2  proto kernel  scope link  src
> 192.168.4.1
> broadcast 192.168.4.0 dev eth2  proto kernel  scope link  src
> 192.168.4.1
> broadcast 192.168.0.0 dev eth1  proto kernel  scope link  src
> 192.168.0.2
> local 192.168.4.1 dev eth2  proto kernel  scope host  src 192.168.4.1
> local 192.168.0.2 dev eth1  proto kernel  scope host  src 192.168.0.2
> broadcast 127.0.0.0 dev lo  proto kernel  scope link  src 127.0.0.1
> broadcast 192.168.3.15 dev eth0  proto kernel  scope link  src
> 192.168.3.1
> broadcast 192.168.3.31 dev eth0  proto kernel  scope link  src
> 192.168.3.17
> local 127.0.0.1 dev lo  proto kernel  scope host  src 127.0.0.1
> broadcast 192.168.5.15 dev eth1  proto kernel  scope link  src
> 192.168.5.2
> local 127.0.0.0/8 dev lo  proto kernel  scope host  src 127.0.0.1
> [root at dodo firewall]#
>
> [root at dodo firewall]# ip route show table 3
> 192.168.1.0/24 via 192.168.3.2 dev eth0
> [root at dodo firewall]#
>
> [root at dodo firewall]# ip route show table 2
> 192.168.1.0/24 via 192.168.3.18 dev eth0
> [root at dodo firewall]#
>
> [root at dodo firewall]# ip route show table main
> 192.168.4.16/28 via 192.168.4.2 dev eth2
> 192.168.4.0/28 dev eth2  scope link
> 192.168.5.0/28 dev eth1  proto kernel  scope link  src 192.168.5.2
> 192.168.3.0/28 dev eth0  proto kernel  scope link  src 192.168.3.1
> 192.168.4.48/28 via 192.168.4.2 dev eth2
> 192.168.3.16/28 dev eth0  scope link
> 192.168.4.32/28 via 192.168.4.2 dev eth2
> 192.168.0.0/24 dev eth1  scope link
> 127.0.0.0/8 dev lo  scope link
> [root at dodo firewall]#
>
> [root at dodo firewall]# ip route show table 253
> [root at dodo firewall]#
>
> [root at dodo firewall]# iptables -nvL -t mangle
> Chain PREROUTING (policy ACCEPT 576K packets, 127M bytes)
>  pkts bytes target     prot opt in     out     source
> destination
>   879 66553 MARK       all  --  *      *       0.0.0.0/0
> 0.0.0.0/0          MAC 00:02:E3:02:C8:8F MARK set 0x3
>  144 10713 MARK       all  --  *      *       0.0.0.0/0
> 0.0.0.0/0          MAC 00:40:F4:2D:AF:5C MARK set 0x2
>   95 29259 MARK       all  --  *      *       0.0.0.0/0
> 0.0.0.0/0          MARK set 0x3
>
> Chain INPUT (policy ACCEPT 576K packets, 127M bytes)
>  pkts bytes target     prot opt in     out     source
> destination
>
> Chain FORWARD (policy ACCEPT 178 packets, 52674 bytes)
>  pkts bytes target     prot opt in     out     source
> destination
>
> Chain OUTPUT (policy ACCEPT 552K packets, 151M bytes)
>  pkts bytes target     prot opt in     out     source
> destination
>
> Chain POSTROUTING (policy ACCEPT 552K packets, 151M bytes)
>  pkts bytes target     prot opt in     out     source
> destination
> [root at dodo firewall]#
>
> If I understand things correctly, when fwmark 0x3 is seen
> routing table 3 should be used and when fwmark 0x2 is seen
> routing table 2 should be used.
>
> If I do:
>
> ip rule add table 2
> or
> ip rule add table 3
>
> then this table will get used and ping works.
>
> I can't hard wire the route, sometimes packets will come from
> web and sometimes they will come from xerxes.  Which
> is why I have mac_route added to the firewall:
>
> [root at dodo firewall]# cat mac_route
> iptables -t mangle -A PREROUTING -m mac --mac-source 00:02:E3:02:C8:8F \
>                   -j MARK --set-mark 3
>
> iptables -t mangle -A PREROUTING -m mac --mac-source 00:40:F4:2D:AF:5C \
>                   -j MARK --set-mark 2
> [root at dodo firewall]#
>
> The following is route_web.bash:
>
> #!/bin/bash
> #
> PATH=/sbin:/usr/bin
>
> # Get line count PREROUTING -t mangle...
> line_count=`iptables -nvL PREROUTING -t mangle|wc -l|tr -d ' '`
> let line_count-=2
>
> if [ "$line_count" == "2" ]
> then
>      iptables -t mangle -I PREROUTING 3 -j MARK --set-mark 2
> else
>      iptables -t mangle -R PREROUTING 3 -j MARK --set-mark 2
> fi
>
> The following is route_xerxes.bash:
>
> #!/bin/bash
> #
> PATH=/sbin:/usr/bin
>
> # Get line count PREROUTING -t mangle...
> line_count=`iptables -nvL PREROUTING -t mangle|wc -l|tr -d ' '`
> let line_count-=2
>
> if [ "$line_count" == "2" ]
> then
>      iptables -t mangle -I PREROUTING 3 -j MARK --set-mark 3
> else
>      iptables -t mangle -R PREROUTING 3 -j MARK --set-mark 3
> fi
>
> The following is the portion of the firewall that manipulates the
> routing tables:
>                  ...
>     export lan_net="192.168.1.0/24"
>                  ...
>             export w1nweb="192.168.3.18"
>             export w1nxer="192.168.3.2"
>
> ip route flush table 2
> ip route flush table 3
>
> ip route add $lan_net dev eth0 via $w1nweb table 2
> ip route add $lan_net dev eth0 via $w1nxer table 3
>
> ip rule add fwmark 2 table 2
> ip rule add fwmark 3 table 3
>
> As far as traceroute, Network is unreachable doesn't seem traceable.
>
> Another routing table can get used, the problem is that the MARK
> applied to packets in the PREROUTING chain of the mangle table
> never seems to trigger use of the appropriate table.  I'm beginning
> to wonder if there is some sysctl option breaking this or something
> similar.
>
> _______________________________________________
> PLUG mailing list
> PLUG at lists.pdxlinux.org
> http://lists.pdxlinux.org/mailman/listinfo/plug
>

well, I'm glad you included more information, but like Mike, your situation
is now way over my head. I agree with him that it would be in your best
interests to attempt to simplify your network configuration. but, if this is
the way you need it to be, then you should be prepared for this kind of
consequence that occurs naturally as a result of complexity.

-wes