На 24.8.2010 г. 22:19, Eric Dumazet написа:
quoted text > Le mardi 24 août 2010 à 20:25 +0300, Plamen Petrov a écrit :
>> Above patch applied, and happy to report the machine now spits data
>> in the logs instead of oopsing. Here is what we have now:
>> [ 707.276981] ---[ end trace 75e4f8534893c910 ]---
>> [ 707.376998] 100: csum_start 306, offset 16, headroom 390, headlen 70,
>> len 70
>> [ 707.477015] nr_frags=0 gso_size=0
>> [ 707.577031]
>> [ 1021.032794] ---[ end trace 75e4f8534893c911 ]---
>> [ 1021.132812] 100: csum_start 306, offset 16, headroom 390, headlen 153,
>> len 153
>> [ 1021.232828] nr_frags=0 gso_size=0
>> [ 1021.332844]
>>
>
> Thanks !
>
> csum_offset = 16.
>
> so its offsetof(struct tcphdr, check)
>
> maybe a bug in net/ipv4/netfilter/nf_nat_helper.c ?
>
> We should trace all spots where we set csum_start/csum_offset
>
> Or/And trace the skb content.
>
> Please add a :
>
> print_hex_dump(KERN_ERR, "skb data:", DUMP_PREFIX_OFFSET,
> 16, 1, skb->head, skb_end_pointer(skb)-skb->head,true);
>
Done! See the results below.
quoted text >
> call in skb_csum_start_bug(), right after the pr_err("\n") and before
> the "return 1;"
>
>
> int skb_csum_start_bug(const struct sk_buff *skb, int pos)
> {
>
> if (skb->ip_summed == CHECKSUM_PARTIAL) {
> long csstart;
>
> csstart = skb->csum_start - skb_headroom(skb);
> if (WARN_ON(csstart> skb_headlen(skb))) {
> int i;
>
> pr_err("%d: csum_start %u, offset %u, headroom %d, headlen %d, len %d\n",
> pos, skb->csum_start, skb->csum_offset, skb_headroom(skb),
> skb_headlen(skb), skb->len);
> pr_err("nr_frags=%u gso_size=%u ",
> skb_shinfo(skb)->nr_frags,
> skb_shinfo(skb)->gso_size);
> for (i = 0; i< skb_shinfo(skb)->nr_frags; i++) {
> pr_err("frag_size=%u ", skb_shinfo(skb)->frags[i].size);
> }
> pr_err("\n");
> print_hex_dump(KERN_ERR, "skb data:", DUMP_PREFIX_OFFSET,
> 16, 1, skb->head, skb_end_pointer(skb) - skb->head, true);
> return 1;
> }
> }
> return 0;
> }
>
>
I see you liked the previous one, here's some more. ;)
This one is based on Linus' latest tree,
hence the ID "2.6.36-rc2-FS-00210-geedff42".
quoted text > [ 10.510191] XFS mounting filesystem md12
> [ 10.693540] Ending clean XFS mount for filesystem: md12
> [ 11.592737] IPv4 FIB: Using LC-trie version 0.409
> [ 11.592827] eth2: link up, 100Mbps, full-duplex, lpa 0x45E1
> [ 11.677311] eth0: link up, 100Mbps, full-duplex, lpa 0x41E1
> [ 11.687604] tg3 0000:04:00.0: irq 44 for MSI/MSI-X
> [ 11.719166] ADDRCONF(NETDEV_UP): eth1: link is not ready
> [ 11.845858] sixxs_t: Disabled Privacy Extensions
> [ 14.815688] tg3 0000:04:00.0: eth1: Link is up at 1000 Mbps, full duplex
> [ 14.815693] tg3 0000:04:00.0: eth1: Flow control is on for TX and on for RX
> [ 14.815740] ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
> [ 15.470040] tun0: Disabled Privacy Extensions
> [ 310.470021] ------------[ cut here ]------------
> [ 310.570041] WARNING: at net/core/dev.c:1945 skb_csum_start_bug+0x46/0x133()
> [ 310.670050] Hardware name: PowerEdge SC440
> [ 310.770060] Pid: 2960, comm: FahCore_78.exe Not tainted 2.6.36-rc2-FS-00210-geedff42 #1
> [ 310.870069] Call Trace:
> [ 311.070087] [<c102d87c>] ? warn_slowpath_common+0x67/0x8c
> [ 311.270103] [<c12adca9>] ? skb_csum_start_bug+0x46/0x133
> [ 311.470126] [<c12adca9>] ? skb_csum_start_bug+0x46/0x133
> [ 311.670144] [<c102d8bc>] ? warn_slowpath_null+0x1b/0x1f
> [ 311.870167] [<c12adca9>] ? skb_csum_start_bug+0x46/0x133
> [ 312.070191] [<c102456f>] ? __wake_up_sync_key+0x3c/0x52
> [ 312.270207] [<c12a7bbb>] ? skb_copy_and_csum_dev+0x2a/0xaf
> [ 312.470224] [<c122484b>] ? rtl8139_start_xmit+0x4a/0x13a
> [ 312.670238] [<c12ae2ee>] ? dev_hard_start_xmit+0x220/0x4cc
> [ 312.870253] [<c12bfc3d>] ? sch_direct_xmit+0xac/0x174
> [ 313.070268] [<c12c3fb9>] ? nf_iterate+0x69/0x7c
> [ 313.270283] [<c12e89c6>] ? ip_finish_output+0x0/0x2b6
> [ 313.470297] [<c12b013c>] ? dev_queue_xmit+0xc7/0x354
> [ 313.670312] [<c12e89c6>] ? ip_finish_output+0x0/0x2b6
> [ 313.870326] [<c12e8ae2>] ? ip_finish_output+0x11c/0x2b6
> [ 314.070341] [<c12e8f61>] ? ip_output+0xa4/0xc3
> [ 314.270355] [<c12e89c6>] ? ip_finish_output+0x0/0x2b6
> [ 314.470370] [<c12e5049>] ? ip_forward_finish+0x39/0x44
> [ 314.670385] [<c12e3a88>] ? ip_rcv_finish+0xe8/0x39f
> [ 314.870399] [<c12ad01d>] ? __netif_receive_skb+0x237/0x2b3
> [ 315.070413] [<c12ad62b>] ? netif_receive_skb+0x5f/0x64
> [ 315.270427] [<c12ad67e>] ? napi_gro_complete+0x4e/0x94
> [ 315.470440] [<c12ad9ba>] ? dev_gro_receive+0x158/0x1f5
> [ 315.670454] [<c12adba4>] ? napi_gro_receive+0x16/0x1f
> [ 315.870468] [<c1217f0b>] ? tg3_poll_work+0x5bc/0xbfb
> [ 316.070483] [<c1006e50>] ? nommu_sync_single_for_device+0x0/0x1
> [ 316.270498] [<c121ce78>] ? tg3_poll+0x43/0x194
> [ 316.470512] [<c12ad7d3>] ? net_rx_action+0xcc/0x15b
> [ 316.670526] [<c1031cbd>] ? __do_softirq+0x7f/0xfa
> [ 316.870541] [<c1053dd9>] ? handle_IRQ_event+0x48/0xa6
> [ 317.070555] [<c10568ab>] ? move_native_irq+0x9/0x3e
> [ 317.270569] [<c1031d5f>] ? do_softirq+0x27/0x2a
> [ 317.470582] [<c1031ead>] ? irq_exit+0x63/0x68
> [ 317.670596] [<c1003dda>] ? do_IRQ+0x44/0xa1
> [ 317.870610] [<c10035c3>] ? do_device_not_available+0x0/0x49
> [ 318.070624] [<c1002d29>] ? common_interrupt+0x29/0x30
> [ 318.270639] [<c1390000>] ? quirk_ati_exploding_mce+0x46/0x7a
> [ 318.370647] ---[ end trace df8deff2ad2a9760 ]---
> [ 318.470656] 100: csum_start 306, offset 16, headroom 390, headlen 151, len 151
> [ 318.570664] nr_frags=0 gso_size=0
> [ 318.670671]
> [ 318.770680] skb data:00000000: 00 a4 27 cc 17 5e ef ec 00 1a a0 38 8a 1b 08 00 ..'..^.....8....
> [ 318.870688] skb data:00000010: 45 00 00 b7 00 00 40 00 40 11 a4 62 c0 a8 0a 01 E.....@.@..b....
> [ 318.970697] skb data:00000020: c0 a8 0a 82 00 35 f2 69 00 a3 96 88 d3 4e 81 80 .....5.i.....N..
> [ 319.070706] skb data:00000030: 00 01 00 04 00 00 00 00 06 61 6b 61 6d 61 69 0d .........akamai.
> [ 319.170714] skb data:00000040: 73 6d 61 72 74 61 64 73 65 72 76 65 72 03 63 6f smartadserver.co
> [ 319.270723] skb data:00000050: 6d 00 00 01 00 01 c0 0c 00 05 00 01 00 00 81 0a m...............
> [ 319.370731] skb data:00000060: 00 28 06 61 6b 61 6d 61 69 0d 73 6d 61 72 74 61 .(.akamai.smarta
> [ 319.470739] skb data:00000070: 64 73 65 72 76 65 72 03 63 6f 6d 09 65 64 67 65 dserver.com.edge
> [ 319.570747] skb data:00000080: 73 75 69 74 65 03 6e 65 74 00 c0 36 00 00 00 00 suite.net..6....
> [ 319.670756] skb data:00000090: 00 00 00 00 00 00 00 00 00 00 08 00 45 00 00 34 ............E..4
> [ 319.770764] skb data:000000a0: 0d 39 40 00 40 06 2f 89 7f 00 00 01 7f 00 00 01 .9@.@./.........
> [ 319.870772] skb data:000000b0: be 75 19 4e 14 43 0f 38 14 5d 49 65 00 00 00 00 .u.N.C.8.]Ie....
> [ 319.970780] skb data:000000c0: 00 00 00 00 00 00 00 00 00 00 08 00 45 00 00 57 ............E..W
> [ 320.070789] skb data:000000d0: 4b 40 40 00 40 06 f1 5e 7f 00 00 01 7f 00 00 01 K@@.@..^........
> [ 320.170797] skb data:000000e0: 19 4e be 75 14 5d 49 65 14 43 0f 38 80 18 04 00 .N.u.]Ie.C.8....
> [ 320.270805] skb data:000000f0: fe 4b 00 00 01 01 08 0a 00 00 02 03 00 00 02 03 .K..............
> [ 320.370813] skb data:00000100: 00 a8 27 cc 00 00 00 00 00 00 00 00 00 00 00 00 ..'.............
> [ 320.470821] skb data:00000110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> [ 320.570829] skb data:00000120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> [ 320.670837] skb data:00000130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> [ 320.770845] skb data:00000140: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> [ 320.870853] skb data:00000150: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> [ 320.970862] skb data:00000160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> [ 321.070870] skb data:00000170: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> [ 321.170878] skb data:00000180: 00 0a e6 ac 07 db 00 0e 2e 5c 27 b2 00 0e 2e 5c .........\'....\
> [ 321.270887] skb data:00000190: 27 ef 08 00 45 00 00 89 09 27 40 00 7f 06 46 74 '...E....'@...Ft
> [ 321.370895] skb data:000001a0: c0 a8 01 02 5b 67 8e c2 04 8f 00 50 98 49 d7 bc ....[g.....P.I..
> [ 321.470903] skb data:000001b0: ff 40 e9 4a 50 18 ff ff ac 4f 00 00 33 42 25 32 .@.JP....O..3B%2
> [ 321.570911] skb data:000001c0: 34 73 68 25 33 44 33 25 33 42 25 32 34 73 77 25 4sh%3D3%3B%24sw%
> [ 321.670919] skb data:000001d0: 33 44 33 3b 20 70 69 64 3d 35 30 32 31 37 34 33 3D3; pid=5021743
> [ 321.770927] skb data:000001e0: 34 32 30 33 31 30 32 39 39 37 38 33 0d 0a 43 6f 420310299783..Co
> [ 321.870936] skb data:000001f0: 6f 6b 69 65 32 3a 20 24 56 65 72 73 69 6f 6e 3d okie2: $Version=
> [ 321.970944] skb data:00000200: 31 0d 0a 43 6f 6e 6e 65 63 74 69 6f 6e 3a 20 4b 1..Connection: K
> [ 322.070952] skb data:00000210: 65 65 70 2d 41 6c 69 76 65 0d 0a 0d 0a 00 00 00 eep-Alive.......
> [ 322.170960] skb data:00000220: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> [ 322.270968] skb data:00000230: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> [ 322.370977] skb data:00000240: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> [ 322.470985] skb data:00000250: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> [ 322.570993] skb data:00000260: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> [ 322.671004] skb data:00000270: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
The rest is in the attached file, in case you need to run it trough some
debugging app...
Thanks,
Plamen
