Re: rtfree: 0xc741ee88 has 1 refs

Previous thread: PCMCIA RS232 card by Bachilo Dmitry on Friday, November 2, 2007 - 2:51 am. (1 message)

Next thread: [head tinderbox] failure on i386/i386 by FreeBSD Tinderbox on Friday, November 2, 2007 - 7:48 am. (1 message)
From: Stefan Lambrev
Date: Friday, November 2, 2007 - 5:56 am

Hi,

I see rtfree: 0xc741ee88 has 1 refs with freebsd releng_7 (i386) from today.

I think it's easy reproducible. What I have is:

releng_7 (10.1.1.2) -> default GW (10.1.1.1)
on default GW I have route to 10.10.1.1/24 -> 10.1.1.3

so everytime when 10.1.1.2 try to contact someone from 10.10.1.1/24 I see:
rtfree: 0xc741ee88 has 1 refs

if I add direct route on 10.1.1.2 to 10.10.1.1/24 through 10.1.1.3 the 
message will go away.

Should I ignore this msg for now, or should I expect kernel panic soon? :)

_______________________________________________
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
From: Stefan Lambrev
Date: Tuesday, November 6, 2007 - 2:35 am

Hi,

Sorry to reply to myself, but I found that the problem exist only if the 
GW is carp interface, e.g. 10.1.1.1 sits on carp0 on default GW.
I'm still testing how to reproduce this in my test lab and will fill a PR.


-- 

Best Wishes,
Stefan Lambrev
ICQ# 24134177

_______________________________________________
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
From: Per olof Ljungmark
Date: Tuesday, November 6, 2007 - 10:53 am

Just FYI, I see this on a few boxes including the 7-BETA2 I'm writing 
this on. None of them has a carp interface though. What I find 
interesting here is that none of them are able to run a SMP kernel 
without crashing (no panic, they're just frozen completely).

Perhaps it is a coincidence, I don't know, but I am very interested in 
your findings and have testbeds if you need.

--per
_______________________________________________
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
From: Kip Macy
Date: Tuesday, November 6, 2007 - 11:15 am

The warning is mostly harmless. It means that rtfree is being called
where RTFREE_LOCKED should be used. Adding a kdb_backtrace() should
track it down for you.

 -Kip
_______________________________________________
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
From: Per olof Ljungmark
Date: Tuesday, November 6, 2007 - 12:51 pm

Kip Macy wrote:

"Mostly", meaning that in some cases it actually does harm? I cannot 
help noting the fact I decribed above but as I have not investigated it 
it may as well be just a coincidence.

I hope I can set aside some time to investigate it
  further.

--per
_______________________________________________
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
From: Stefan Lambrev
Date: Wednesday, November 7, 2007 - 1:01 am

Hi,

In the begging I thought this is very easy reproducible, but it isn't :)
In my situation the IP of the default GW sits on carp interface (the 
host have nothing to do with carp)
and when the host receive icmp redirect messages they actually are send 
from the IP of the netwrok card of the GW (not the IP that sits on carp0)
then the host things that those icmp type 5 are spoofed and just ignore 
them and do not add shorter route.
In this situation after reboot the hosts start to moan about rtfree, so 
I played with routes and settings, added manually routes and now when I 
restore everything to previous state
I do not see anymore rtfree warnings. So may be once the route is 
created manually and then removed, wrong call of rtfree is not triggered 
anymore.
I'll reboot the host latter to see if this easy reproducible at least on 
reboot, and if it is, I'll compile debug kernel and will run backtrace.

The other thing that bother me is that icmp redirects are not send from 
the carpIP, but from the real interface's IP? Isn't this a bug, or it is 
normal behavior?

-- 

Best Wishes,
Stefan Lambrev
ICQ# 24134177

_______________________________________________
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
From: Stefan Lambrev
Date: Wednesday, November 7, 2007 - 1:52 am

Hi,

Just to confirm that after reboot I'm seeing imediately rtfree: %P has 1 
refs.

I'll recompile kernel with debug and will try to find out more.


-- 

Best Wishes,
Stefan Lambrev
ICQ# 24134177

_______________________________________________
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
From: Stefan Lambrev
Date: Wednesday, November 7, 2007 - 3:55 am

Hi,

I compiled kernel with all debug stuff in it, but I'm not sure what to 
do next :)
I have serial console to the server so I boot with -dDh and enter debig 
mode.
I can enter at debug mode at any time now, so any idea how to catch what 
trigger this rtfree ? :)
I'm following steps from 
http://www.freebsd.org/doc/en/books/developers-handbook/kerneldebug-online-ddb.html
What info should I provide?


-- 

Best Wishes,
Stefan Lambrev
ICQ# 24134177

_______________________________________________
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
From: Yuri Pankov
Date: Wednesday, November 7, 2007 - 5:24 am

This thread might be useful:
http://lists.freebsd.org/pipermail/freebsd-current/2007-May/072619.html


HTH,
Yuri
_______________________________________________
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
From: Stefan Lambrev
Date: Wednesday, November 7, 2007 - 6:48 am

Hi,

I hope the attached trace will help.

BTW :

rtfree: 0xc74e6258 has 1 refs
KDB: stack backtrace:
db_trace_self_wrapper(c070e5d9,e59d1af0,c05efdce,c0718d8f,c06e7f6d,...) 
at db_trace_self_wrapper+0x26
kdb_backtrace(c0718d8f,c06e7f6d,c74e6258,1,c74e6264,...) apanic: 
sched_priority: invalid priority 224: nice 0, ticks 21273056 ftick 15171 
ltick 26026 tick pri 44
cpuid = 2
KDB: enter: panic
[thread pid 712 tid 100074 ]
Stopped at      kdb_enter+0x32: leave

Is this expected?

-- 

Best Wishes,
Stefan Lambrev
ICQ# 24134177

From: Stefan Lambrev
Date: Wednesday, November 7, 2007 - 6:55 am

Hm attachment truncated :)
Here is the result of including kdb_backtrace() in route.c at line 248:

Starting syslogd.
rtfree: 0xc74e6258 has 1 refs
KDB: stack backtrace:
db_trace_self_wrapper(c070e5d9,e59d1af0,c05efdce,c0718d8f,c06e7f6d,...) 
at db_trace_self_wrapper+0x26
kdb_backtrace(c0718d8f,c06e7f6d,c74e6258,1,c74e6264,...) at 
kdb_backtrace+0x29
rtfree(c74e6258,c6f4d570,10,0,c070a4f4,...) at rtfree+0xd5
rtredirect(e59d1bb8,e59d1ba8,0,6,e59d1b98,...) at rtredirect+0x1a3
icmp_input(c743da00,14,c0775620,e59d1c04,c054fe54,...) at icmp_input+0x4ee
ip_input(c743da00,c743da00,800,c6ef0c00,800,...) at ip_input+0x64a
netisr_dispatch(2,c743da00,10,3,0,...) at netisr_dispatch+0x6c
ether_demux(c6ef0c00,c743da00,3,0,3,...) at ether_demux+0x1d4
ether_input(c6ef0c00,c743da00,c06fbb7b,bc4,e59d1cc4,...) at 
ether_input+0x387
bge_intr(c6f3a000,0,c0708331,471,c6e30164,...) at bge_intr+0x7c5
ithread_loop(c6f05be0,e59d1d38,c07080a0,314,c6eea2a8,...) at 
ithread_loop+0x1a5
fork_exit(c0540fa0,c6f05be0,e59d1d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe59d1d70, ebp = 0 ---
rtfree: 0xc74e6258 has 1 refs
KDB: stack backtrace:
db_trace_self_wrapper(c070e5d9,e59d1af0,c05efdce,c0718d8f,c06e7f6d,...) 
at db_trace_self_wrapper+0x26
kdb_backtrace(c0718d8f,c06e7f6d,c74e6258,1,c74e6264,...) at 
kdb_backtrace+0x29
rtfree(c74e6258,c6f4d570,10,0,c070a4f4,...) at rtfree+0xd5
rtredirect(e59d1bb8,e59d1ba8,0,6,e59d1b98,...) at rtredirect+0x1a3
icmp_input(c743c500,14,c0775620,e59d1c04,c054fe54,...) at icmp_input+0x4ee
ip_input(c743c500,c743c500,800,c6ef0c00,800,...) at ip_input+0x64a
netisr_dispatch(2,c743c500,10,3,0,...) at netisr_dispatch+0x6c
ether_demux(c6ef0c00,c743c500,3,0,3,...) at ether_demux+0x1d4
ether_input(c6ef0c00,c743c500,c06fbb7b,bc4,e59d1cc4,...) at 
ether_input+0x387
bge_intr(c6f3a000,0,c0708331,471,c6e30164,...) at bge_intr+0x7c5
ithread_loop(c6f05be0,e59d1d38,c07080a0,314,c6eea2a8,...) at ...
Previous thread: PCMCIA RS232 card by Bachilo Dmitry on Friday, November 2, 2007 - 2:51 am. (1 message)

Next thread: [head tinderbox] failure on i386/i386 by FreeBSD Tinderbox on Friday, November 2, 2007 - 7:48 am. (1 message)