[PLUG] alien ping response

[Alan]

> Hi Michael,
>
> On Thu, Feb 28, 2008 at 02:17:31PM -0800, Michael Rasmussen wrote:
>> We've been tracking down an intermittent, odd problem. One of the
>> symptoms is
>> ping response behavior:
>>
>>
>> hostname % ping -I 1 -n 10.11.22.122
>>  ...
>> 64 bytes from 10.11.22.122: icmp_seq=1436. time=1. ms
>> 64 bytes from 10.11.22.122: icmp_seq=1437. time=0. ms
>> 64 bytes from 10.11.44.212: icmp_seq=1283. time=445990198. ms
>> 64 bytes from 10.11.22.122: icmp_seq=1438. time=1. ms
>> 64 bytes from 10.11.22.122: icmp_seq=1439. time=1. ms
>>
>> What?  Why is the response from another device on another subnet
>> arriving
>> here?
>>
>> Have you seen behavior like this?
>> What was the cause?
>
> Once upon a time in a college network I was working in, I saw this kind
> of traffic: echo replies being received from random sites on the
> Internet when no outbound echo was sent.  As we were doing security
> research at the time with IDSes, we spend some time looking into it.
>
> We reported it to the network operations guys, but they really weren't
> interested.  We had suspected hanky-panky with ping spoofing, but they
> didn't want to deal with it and told us no routing/firewall rules were
> in place to prevent spoofing like this, so the real source of the echo
> requests could be anywhere within the (rather large) campus network.
>
> Then a couple of months later, the main college routers to the Internet
> went down.  The network guys were obviously interested in that.  The
> investigated and found out that it died because of an enormous amount of
> ICMP traffic coming to/from the campus.  They finally traced it back to
> a few Solaris workstations that had been compromised and were used for
> DoSing poor bastards on the Internet.
>
> So, what is your network's source IP restriction policy?  In my case,
> the echo replies were coming from hosts on the Internet because the
> attackers were spoofing my IPs when sending out.  In your case, why
> would an attacker try to bounce traffic off of an internal host?  It may
> not be malicious. Could it be the result of some broken NAT rule or
> something?  Just guessing.  To really find out the answer, you'd
> probably have to sit down on 10.11.44.212's local switch and follow MAC
> addresses one router at a time to figure out where the echo requests are
> coming from.

I have seen this sort of weirdness caused by something completly different.

Check to see if you have a router loop.  If you have a nest of switches
plugged into switches sometimes you can get switch A plugged into switch B
plugged into Switch C which is plugged into switch A.  This causes all
sorts of fun on the network.

Another possibility is that you have a failing switch that is flaking and
throwing in bogus packets or delaying things randomly.