From shin-5@jp-c.ne.jp Wed Dec 1 02:02:00 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 01 Dec 2004 02:02:06 -0800 (PST) Received: from MIN14 ([221.212.226.3]) by oss.sgi.com (8.13.0/8.13.0) with SMTP id iB1A1wn1011500 for ; Wed, 1 Dec 2004 02:01:59 -0800 Date: Wed, 1 Dec 2004 02:01:58 -0800 Message-Id: <200412011001.iB1A1wn1011500@oss.sgi.com> From: "=?iso-2022-jp?B?c2pkazE0QHlhaG9vLmNvLmpw?=" To: "netdev@oss.sgi.com" X-mailer: Super Mailer 9 [en][outlook] Subject: =?iso-2022-jp?B?GyRCISE0KCQkJEckOSQrISkbKEI=?= MIME-Version: 1.0 Content-Type: text/plain; charset="iso-2022-jp" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4927.1200 X-archive-position: 12362 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: alkjd@yahoo.co.jp Precedence: bulk X-list: netdev $B$$$$$M!A:#=5$b:G9b$G$9!*(B http://iidote.info/kinjyo ****$B%a%k%^%,2r=|(B/$BLd$$9g$o$;(B**** $B9-El>J!!HM!!9@(B yotsuba_kouyou@yahoo.co.jp ******************************* From hadi@cyberus.ca Wed Dec 1 05:23:12 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 01 Dec 2004 05:23:19 -0800 (PST) Received: from mx01.cybersurf.com (mx01.cybersurf.com [209.197.145.104]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB1DNB1C021461 for ; Wed, 1 Dec 2004 05:23:12 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx01.cybersurf.com with esmtp (Exim 4.30) id 1CZURT-0003Du-Es for netdev@oss.sgi.com; Wed, 01 Dec 2004 08:22:47 -0500 Received: from cpe0030ab124d2f-cm014500000962.cpe.net.cable.rogers.com ([24.103.99.32] helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1CZURP-0008Vw-8l; Wed, 01 Dec 2004 08:22:43 -0500 Subject: Re: [E1000-devel] Transmission limit From: jamal Reply-To: hadi@cyberus.ca To: Lennert Buytenhek Cc: Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com In-Reply-To: <20041201001107.GE4203@xi.wantstofly.org> References: <1101467291.24742.70.camel@mellia.lipar.polito.it> <41A73826.3000109@draigBrady.com> <16807.20052.569125.686158@robur.slu.se> <1101484740.24742.213.camel@mellia.lipar.polito.it> <41A76085.7000105@draigBrady.com> <1101499285.1079.45.camel@jzny.localdomain> <16811.8052.678955.795327@robur.slu.se> <1101821501.1043.43.camel@jzny.localdomain> <20041130134600.GA31515@xi.wantstofly.org> <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> Content-Type: text/plain Organization: jamalopolous Message-Id: <1101902900.1041.9.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 01 Dec 2004 07:08:20 -0500 Content-Transfer-Encoding: 7bit X-archive-position: 12363 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev On Tue, 2004-11-30 at 19:11, Lennert Buytenhek wrote: > On Tue, Nov 30, 2004 at 09:25:54AM -0500, jamal wrote: > > > > > > Also from what I understand new HW and MSI can help in the case where > > > > > pass objects between CPU. Did I dream or did someone tell me that S2IO > > > > > could have several TX ring that could via MSI be routed to proper cpu? > > > > > > > > I am wondering if the per CPU tx/rx irqs are valuable at all. They sound > > > > like more hell to maintain. > > > > > > On the TX path you'd have qdiscs to deal with as well, no? > > > > I think management of it would be non-trivial in SMP. Youd have to start > > playing stupid loadbalancing tricks which would reduce the value of > > existence of tx irqs to begin with. > > You mean the management of qdiscs would be non-trivial? I mean it is useful in only the most ideal cases and if you want to actually do something useful in most cases with it you will have to muck around. Take the case of forwarding (maybe with a little or almost no localhost generated traffic) - then you end allocating in CPUA, processing and queueing on egress. Tx softirq, which is what stashes the packet on tx DMA eventually, is not guaranteed to run on the same CPU. Now add a little latency between ingress and egress .. The ideal case is where you end up processing to completion from ingress to egress (which is known to happen in Linux when theres no congestion). > Probably the idea of these kinds of tricks is to skip the qdisc step > altogether. > Which is preached by the BSD folks - bogus in my opinion. If you want to do something as bland/boring as that you can probably afford a $500 DLINK router which can do it at wire rate with (with cost you being locked in whatever features they have). cheers, jamal From hadi@cyberus.ca Wed Dec 1 05:40:41 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 01 Dec 2004 05:40:46 -0800 (PST) Received: from mx02.cybersurf.com (mx02.cybersurf.com [209.197.145.105]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB1Debev022056 for ; Wed, 1 Dec 2004 05:40:41 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx02.cybersurf.com with esmtp (Exim 4.30) id 1CZUiL-0006Ct-BZ for netdev@oss.sgi.com; Wed, 01 Dec 2004 08:40:13 -0500 Received: from cpe0030ab124d2f-cm014500000962.cpe.net.cable.rogers.com ([24.103.99.32] helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1CZUiH-000245-JX; Wed, 01 Dec 2004 08:40:09 -0500 Subject: Re: [E1000-devel] Transmission limit From: jamal Reply-To: hadi@cyberus.ca To: mellia@prezzemolo.polito.it Cc: Lennert Buytenhek , Harald Welte , P@draigBrady.com, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com In-Reply-To: <1101804146.11111.23.camel@mellia.lipar.polito.it> References: <1101467291.24742.70.camel@mellia.lipar.polito.it> <41A73826.3000109@draigBrady.com> <1101483081.24742.174.camel@mellia.lipar.polito.it> <20041127092503.GA12592@sunbeam.de.gnumonks.org> <1101718412.14930.46.camel@verza.polito.it> <20041129145028.GC18788@xi.wantstofly.org> <1101804146.11111.23.camel@mellia.lipar.polito.it> Content-Type: text/plain Organization: jamalopolous Message-Id: <1101903944.1042.29.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 01 Dec 2004 07:25:44 -0500 Content-Transfer-Encoding: 7bit X-archive-position: 12364 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev On Tue, 2004-11-30 at 03:42, Marco Mellia wrote: > On Mon, 2004-11-29 at 15:50, Lennert Buytenhek wrote: > > On Mon, Nov 29, 2004 at 09:53:33AM +0100, Marco Mellia wrote: > > > > > Th's our intuition too. > > > Notice that we get the same results with 3com (broadcom based) gigabit > > > cards. > > > We are thinking of sending packet in "bursts" instead of single > > > transfers. The only problem is to let the NIC know that there are more > > > than a packet in a burst... > > > > Jamal implemented exactly this for e1000 already, he might be persuaded > > into posting his patch here. Jamal? :) > > I guess that saying that we are _very_ interested in this might help. > :-) > We can offer as "beta-testers" as well... Sorry missed this (I wasnt CCed so it went to a low priority queue which i read on a best effort basis). Let me clean up the patches a little bit this weekend. The patch is at least 4 months old; latest reincarnation was due to issue1 on my SUCON presentation. Would a patch against latest 2.6.x bitkeeper (whatever it is this weekend) be fine? If you are in a rush and dont mind a little ugliness then i will pass them as is. BTW, Scott posted a interesting patch yesterday, you may wanna give that a shot as well. cheers, jamal From buytenh@wantstofly.org Wed Dec 1 07:24:56 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 01 Dec 2004 07:25:08 -0800 (PST) Received: from xi.wantstofly.org (alephnull.demon.nl [212.238.201.82]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB1FOtOe011798 for ; Wed, 1 Dec 2004 07:24:56 -0800 Received: by xi.wantstofly.org (Postfix, from userid 500) id 09F592B0ED; Wed, 1 Dec 2004 16:24:33 +0100 (MET) Date: Wed, 1 Dec 2004 16:24:33 +0100 From: Lennert Buytenhek To: jamal Cc: Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: [E1000-devel] Transmission limit Message-ID: <20041201152433.GA12558@xi.wantstofly.org> References: <16807.20052.569125.686158@robur.slu.se> <1101484740.24742.213.camel@mellia.lipar.polito.it> <41A76085.7000105@draigBrady.com> <1101499285.1079.45.camel@jzny.localdomain> <16811.8052.678955.795327@robur.slu.se> <1101821501.1043.43.camel@jzny.localdomain> <20041130134600.GA31515@xi.wantstofly.org> <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101902900.1041.9.camel@jzny.localdomain> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1101902900.1041.9.camel@jzny.localdomain> User-Agent: Mutt/1.4.1i X-archive-position: 12365 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: buytenh@wantstofly.org Precedence: bulk X-list: netdev On Wed, Dec 01, 2004 at 07:08:20AM -0500, jamal wrote: [ per-CPU TX/RX rings ] > > You mean the management of qdiscs would be non-trivial? > > I mean it is useful in only the most ideal cases and if you want to > actually do something useful in most cases with it you will have to > muck around. > Take the case of forwarding (maybe with a little or almost no localhost > generated traffic) - then you end allocating in CPUA, processing and > queueing on egress. Tx softirq, which is what stashes the packet on tx > DMA eventually, is not guaranteed to run on the same CPU. Now add a > little latency between ingress and egress .. > The ideal case is where you end up processing to completion from ingress > to egress (which is known to happen in Linux when theres no congestion). We disagreed on this topic at SUCON and I'm afraid we'll be disagreeing on it forever :) IMHO, on 10GbE any kind of qdisc is a waste of cycles. I don't think it's very likely that you'll be using that single 10GbE NIC for forwarding packets, doing that with a PC at this point in the history of PCs is just silly. If you do use it for forwarding, how likely is it that you'll be able to process an incoming burst of packets fast enough to require queueing on the egress interface? You have to be able to send a burst of packets bigger than the NIC's TX FIFO at >10GbE in the first place for queueing to be effective/useful at all. (Leaving the question of whether or not there'll be some room in the TX FIFO at TX time unanswered, what you're doing with per-CPU TX rings is basically just simulating the "N individual NICs each bound to its own CPU" case with a single NIC.) > > Probably the idea of these kinds of tricks is to skip the qdisc step > > altogether. > > Which is preached by the BSD folks - bogus in my opinion. If you want to > do something as bland/boring as that you can probably afford a $500 > DLINK router which can do it at wire rate with (with cost you being > locked in whatever features they have). That's an unfair comparison. Just because I don't need CBQ doesn't mean my $500 DLINK router does everything I'd want it to -- advanced firewalling is one thing that comes to mind. Last time I looked I couldn't load my own kernel modules on my DLINK router either. --L From Robert.Olsson@data.slu.se Wed Dec 1 07:34:53 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 01 Dec 2004 07:34:58 -0800 (PST) Received: from mail1.slu.se (mail1.slu.se [130.238.96.11]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB1FYmVv012294 for ; Wed, 1 Dec 2004 07:34:53 -0800 Received: from robur.slu.se (robur.slu.se [130.238.98.12]) by mail1.slu.se (8.12.10/8.12.10) with ESMTP id iB1FYCKO029728; Wed, 1 Dec 2004 16:34:12 +0100 Received: by robur.slu.se (Postfix, from userid 1000) id 5C1C6EC001; Wed, 1 Dec 2004 16:34:12 +0100 (CET) From: Robert Olsson MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <16813.58484.343629.570703@robur.slu.se> Date: Wed, 1 Dec 2004 16:34:12 +0100 To: sfeldma@pobox.com Cc: Lennert Buytenhek , jamal , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: [E1000-devel] Transmission limit In-Reply-To: <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> References: <1101467291.24742.70.camel@mellia.lipar.polito.it> <41A73826.3000109@draigBrady.com> <16807.20052.569125.686158@robur.slu.se> <1101484740.24742.213.camel@mellia.lipar.polito.it> <41A76085.7000105@draigBrady.com> <1101499285.1079.45.camel@jzny.localdomain> <16811.8052.678955.795327@robur.slu.se> <1101821501.1043.43.camel@jzny.localdomain> <20041130134600.GA31515@xi.wantstofly.org> <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> X-Mailer: VM 7.18 under Emacs 21.3.1 X-archive-position: 12366 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: Robert.Olsson@data.slu.se Precedence: bulk X-list: netdev Scott Feldman writes: > Hey, turns out, I know some e1000 tricks that might help get the kpps > numbers up. > > My problem is I only have a P4 desktop system with a 82544 nic running > at PCI 32/33Mhz, so I can't play with the big boys. But, attached is a > rework of the Tx path to eliminate 1) Tx interrupts, and 2) Tx > descriptor write-backs. For me, I see a nice jump in kpps, but I'd like > others to try with their setups. We should be able to get to wire speed > with 60-byte packets. > > System: Intel 865 (HT 2.6Ghz) > Nic: 82544 PCI 32-bit/33Mhz > Driver: linux-2.6.9 e1000 (5.3.19-k2-NAPI), no Interrupt Delays > 4096 descs > pkt_size = 60: 541618pps 277Mb/sec errors: 914 Hello! Nice but I no improvements w. 82546GB @ 133 MHz on 1.6 GHz Opteron it seems. SMP kernel linux-2.6.9-rc2 Vanilla. 801077pps 410Mb/sec (410151424bps) errors: 95596 Patch TXD=4096 608690pps 311Mb/sec (311649280bps) errors: 0 Patch TXD=2048 624103pps 319Mb/sec (319540736bps) errors: 0 Patch TXD=1024 551289pps 282Mb/sec (282259968bps) errors: 4506 Error count is a bit confusing... --ro From sfeldma@pobox.com Wed Dec 1 08:47:41 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 01 Dec 2004 08:47:45 -0800 (PST) Received: from orb.pobox.com (orb.pobox.com [207.8.226.5]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB1GlafG018005 for ; Wed, 1 Dec 2004 08:47:41 -0800 Received: from orb (localhost [127.0.0.1]) by orb.pobox.com (Postfix) with ESMTP id EAD952F9553; Wed, 1 Dec 2004 11:47:14 -0500 (EST) Received: from [172.20.3.21] (66.239.228.194.ptr.us.xo.net [66.239.228.194]) by orb.sasl.smtp.pobox.com (Postfix) with ESMTP id D00302F84CC; Wed, 1 Dec 2004 11:47:04 -0500 (EST) Subject: Re: [E1000-devel] Transmission limit From: Scott Feldman Reply-To: sfeldma@pobox.com To: Robert Olsson Cc: Lennert Buytenhek , jamal , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com In-Reply-To: <16813.58484.343629.570703@robur.slu.se> References: <1101467291.24742.70.camel@mellia.lipar.polito.it> <41A73826.3000109@draigBrady.com> <16807.20052.569125.686158@robur.slu.se> <1101484740.24742.213.camel@mellia.lipar.polito.it> <41A76085.7000105@draigBrady.com> <1101499285.1079.45.camel@jzny.localdomain> <16811.8052.678955.795327@robur.slu.se> <1101821501.1043.43.camel@jzny.localdomain> <20041130134600.GA31515@xi.wantstofly.org> <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <16813.58484.343629.570703@robur.slu.se> Content-Type: text/plain Message-Id: <1101919791.5198.15.camel@localhost.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 (1.4.6-2) Date: Wed, 01 Dec 2004 08:49:51 -0800 Content-Transfer-Encoding: 7bit X-archive-position: 12367 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: sfeldma@pobox.com Precedence: bulk X-list: netdev On Wed, 2004-12-01 at 07:34, Robert Olsson wrote: > Nice but I no improvements w. 82546GB @ 133 MHz on 1.6 GHz Opteron it seems. > SMP kernel linux-2.6.9-rc2 > > Vanilla. > 801077pps 410Mb/sec (410151424bps) errors: 95596 > > Patch TXD=4096 > 608690pps 311Mb/sec (311649280bps) errors: 0 Thank you Robert for trying it out. Well those results are counter-intuitive! We remove Tx interrupts and Tx descriptor DMA write-backs and get no re-tries, and performance drops? The only bus activities left are the DMA of buffers to device and the register writes to increment tail. I'm stumped. I'll need to get my hands on a faster system. Maybe there is a bus analyzer under the tree. :-) -scott From Robert.Olsson@data.slu.se Wed Dec 1 08:48:14 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 01 Dec 2004 08:48:19 -0800 (PST) Received: from mail1.slu.se (mail1.slu.se [130.238.96.11]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB1Gm7iB018063 for ; Wed, 1 Dec 2004 08:48:14 -0800 Received: from robur.slu.se (robur.slu.se [130.238.98.12]) by mail1.slu.se (8.12.10/8.12.10) with ESMTP id iB1GlOKO024025; Wed, 1 Dec 2004 17:47:24 +0100 Received: by robur.slu.se (Postfix, from userid 1000) id 34CC8EC001; Wed, 1 Dec 2004 17:47:24 +0100 (CET) From: Robert Olsson MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <16813.62876.180869.404122@robur.slu.se> Date: Wed, 1 Dec 2004 17:47:24 +0100 To: "David S. Miller" Cc: Robert Olsson , hadi@cyberus.ca, P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, jorge.finochietto@polito.it, galante@polito.it, netdev@oss.sgi.com Subject: Re: [E1000-devel] Transmission limit In-Reply-To: <20041129121604.47eb6593.davem@davemloft.net> References: <1101467291.24742.70.camel@mellia.lipar.polito.it> <41A73826.3000109@draigBrady.com> <16807.20052.569125.686158@robur.slu.se> <1101484740.24742.213.camel@mellia.lipar.polito.it> <41A76085.7000105@draigBrady.com> <1101499285.1079.45.camel@jzny.localdomain> <16811.8052.678955.795327@robur.slu.se> <20041129121604.47eb6593.davem@davemloft.net> X-Mailer: VM 7.18 under Emacs 21.3.1 X-archive-position: 12368 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: Robert.Olsson@data.slu.se Precedence: bulk X-list: netdev David S. Miller writes: > > Did I dream or did someone tell me that S2IO > > could have several TX ring that could via MSI be routed to proper cpu? > > One of Sun's gigabit chips can do this too, except it isn't > via MSI, the driver has to read the descriptor to figure out > which cpu gets the software interrupt to process the packet. > > SGI had hardware which allowed you to do this kind of stuff too. > > Obviously the MSI version works much better. > > It is important, the cpu selection process. First of all, it must > be calculated such that flows always go through the same cpu. > Otherwise TCP sockets bounce between the cpus for a streaming > transfer. > > And even this doesn't avoid all such problems, TCP LISTEN state > sockets will still thrash between the cpus with such a "pick > a cpu based upon" flow scheme. > > Anyways, just some thoughts. Thanks for the the info. Well we'll be forced to get into those problems when the HW is capable. I'll guess it will be w. the 10 GIGE cards. --ro From wensong@linux-vs.org Wed Dec 1 08:49:54 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 01 Dec 2004 08:50:02 -0800 (PST) Received: from lb1.ctrip.com ([218.244.111.2]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB1Gnqeo018487 for ; Wed, 1 Dec 2004 08:49:53 -0800 Received: from penguin.linux-vs.org ([61.149.145.226]) by lb1.ctrip.com (8.12.10/8.12.10) with ESMTP id iB1GmjMh028406 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Thu, 2 Dec 2004 00:48:49 +0800 Received: from penguin.linux-vs.org (localhost.localdomain [127.0.0.1]) by penguin.linux-vs.org (8.12.8/8.12.8) with ESMTP id iB1GmZ9A001083; Thu, 2 Dec 2004 00:48:36 +0800 Received: from localhost (wensong@localhost) by penguin.linux-vs.org (8.12.8/8.12.8/Submit) with ESMTP id iB1GmRZa001079; Thu, 2 Dec 2004 00:48:31 +0800 X-Authentication-Warning: penguin.linux-vs.org: wensong owned process doing -bs Date: Thu, 2 Dec 2004 00:48:26 +0800 (CST) From: Wensong Zhang To: "David S. Miller" , netdev@oss.sgi.com Subject: [PATCH] [IPVS] add a sysctl variable to expire quiescent template Message-ID: MIME-Version: 1.0 Content-Type: MULTIPART/Mixed; BOUNDARY="-1463811584-572891765-1097000265=:2451" Content-ID: X-archive-position: 12369 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: wensong@linux-vs.org Precedence: bulk X-list: netdev This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. ---1463811584-572891765-1097000265=:2451 Content-Type: TEXT/PLAIN; CHARSET=US-ASCII; format=flowed Content-ID: Hi Dave, Here is the patch from Horms to add a sysctl variable to expire quiescent templat. Please check and apply them to kernel 2.4 and 2.6 respectively. Thanks, Wensong ---1463811584-572891765-1097000265=:2451 Content-Type: TEXT/PLAIN; charset=US-ASCII; name="linux-2.4-ipvs-quiescent_template.patch" Content-Transfer-Encoding: BASE64 Content-ID: Content-Description: Content-Disposition: attachment; filename="linux-2.4-ipvs-quiescent_template.patch" IyBUaGlzIGlzIGEgQml0S2VlcGVyIGdlbmVyYXRlZCBkaWZmIC1OcnUgc3R5 bGUgcGF0Y2guDQojDQojIENoYW5nZVNldA0KIyAgIDIwMDQvMTIvMDIgMDA6 MDI6NDgrMDg6MDAgd2Vuc29uZ0BsaW51eC12cy5vcmcgDQojICAgW0lQVlNd IGFkZCBhIHN5c2N0bCB2YXJpYWJsZSB0byBleHBpcmUgcXVpZXNjZW50IHRl bXBsYXRlDQojICAgDQojICAgVGhlIHBhdGNoIGlzIGZyb20gSG9ybXMgPGhv cm1zQHZlcmdlbmV0Lm5ldD4NCiMgDQojIG5ldC9pcHY0L2lwdnMvaXBfdnNf Y3RsLmMNCiMgICAyMDA0LzEyLzAyIDAwOjAyOjM4KzA4OjAwIHdlbnNvbmdA bGludXgtdnMub3JnICs0IC0wDQojICAgc2V0IHN5c2N0bF9pcF92c19leHBp cmVfcXVpZXNjZW50X3RlbXBsYXRlDQojIA0KIyBuZXQvaXB2NC9pcHZzL2lw X3ZzX2Nvbm4uYw0KIyAgIDIwMDQvMTIvMDIgMDA6MDI6MzcrMDg6MDAgd2Vu c29uZ0BsaW51eC12cy5vcmcgKzMgLTENCiMgICBkb24ndCB1c2UgcXVpZXNj ZW50IHRlbXBsYXRlIGlmIHRoZSBleHBpcmVfcXVpZXNjZW50X3RlbXBsYXRl IGlzIGVuYWJsZWQNCiMgDQojIGluY2x1ZGUvbmV0L2lwX3ZzLmgNCiMgICAy MDA0LzEyLzAyIDAwOjAyOjM3KzA4OjAwIHdlbnNvbmdAbGludXgtdnMub3Jn ICsyIC0wDQojICAgYWRkIHRoZSBzeXNjdGxfaXBfdnNfZXhwaXJlX3F1aWVz Y2VudF90ZW1wbGF0ZQ0KIyANCmRpZmYgLU5ydSBhL2luY2x1ZGUvbmV0L2lw X3ZzLmggYi9pbmNsdWRlL25ldC9pcF92cy5oDQotLS0gYS9pbmNsdWRlL25l dC9pcF92cy5oCTIwMDQtMTItMDIgMDA6MTY6MzYgKzA4OjAwDQorKysgYi9p bmNsdWRlL25ldC9pcF92cy5oCTIwMDQtMTItMDIgMDA6MTY6MzYgKzA4OjAw DQpAQCAtMzE3LDYgKzMxNyw3IEBADQogCU5FVF9JUFY0X1ZTX0VYUElSRV9O T0RFU1RfQ09OTj0yMywNCiAJTkVUX0lQVjRfVlNfU1lOQ19USFJFU0hPTEQ9 MjQsDQogCU5FVF9JUFY0X1ZTX05BVF9JQ01QX1NFTkQ9MjUsDQorCU5FVF9J UFY0X1ZTX0VYUElSRV9RVUlFU0NFTlRfVEVNUExBVEU9MjYsDQogCU5FVF9J UFY0X1ZTX0xBU1QNCiB9Ow0KIA0KQEAgLTcwMCw2ICs3MDEsNyBAQA0KICAq Lw0KIGV4dGVybiBpbnQgc3lzY3RsX2lwX3ZzX2NhY2hlX2J5cGFzczsNCiBl eHRlcm4gaW50IHN5c2N0bF9pcF92c19leHBpcmVfbm9kZXN0X2Nvbm47DQor ZXh0ZXJuIGludCBzeXNjdGxfaXBfdnNfZXhwaXJlX3F1aWVzY2VudF90ZW1w bGF0ZTsNCiBleHRlcm4gaW50IHN5c2N0bF9pcF92c19zeW5jX3RocmVzaG9s ZDsNCiBleHRlcm4gaW50IHN5c2N0bF9pcF92c19uYXRfaWNtcF9zZW5kOw0K IGV4dGVybiBzdHJ1Y3QgaXBfdnNfc3RhdHMgaXBfdnNfc3RhdHM7DQpkaWZm IC1OcnUgYS9uZXQvaXB2NC9pcHZzL2lwX3ZzX2Nvbm4uYyBiL25ldC9pcHY0 L2lwdnMvaXBfdnNfY29ubi5jDQotLS0gYS9uZXQvaXB2NC9pcHZzL2lwX3Zz X2Nvbm4uYwkyMDA0LTEyLTAyIDAwOjE2OjM2ICswODowMA0KKysrIGIvbmV0 L2lwdjQvaXB2cy9pcF92c19jb25uLmMJMjAwNC0xMi0wMiAwMDoxNjozNiAr MDg6MDANCkBAIC0xMTMxLDcgKzExMzEsOSBAQA0KIAkgKiBDaGVja2luZyB0 aGUgZGVzdCBzZXJ2ZXIgc3RhdHVzLg0KIAkgKi8NCiAJaWYgKChkZXN0ID09 IE5VTEwpIHx8DQotCSAgICAhKGRlc3QtPmZsYWdzICYgSVBfVlNfREVTVF9G X0FWQUlMQUJMRSkpIHsNCisJICAgICEoZGVzdC0+ZmxhZ3MgJiBJUF9WU19E RVNUX0ZfQVZBSUxBQkxFKSB8fCANCisJICAgIChzeXNjdGxfaXBfdnNfZXhw aXJlX3F1aWVzY2VudF90ZW1wbGF0ZSAmJiANCisJICAgICAoYXRvbWljX3Jl YWQoJmRlc3QtPndlaWdodCkgPT0gMCkpKSB7DQogCQlJUF9WU19EQkcoOSwg ImNoZWNrX3RlbXBsYXRlOiBkZXN0IG5vdCBhdmFpbGFibGUgZm9yICINCiAJ CQkgICJwcm90b2NvbCAlcyBzOiV1LiV1LiV1LiV1OiVkIHY6JXUuJXUuJXUu JXU6JWQgIg0KIAkJCSAgIi0+IGQ6JXUuJXUuJXUuJXU6JWRcbiIsDQpkaWZm IC1OcnUgYS9uZXQvaXB2NC9pcHZzL2lwX3ZzX2N0bC5jIGIvbmV0L2lwdjQv aXB2cy9pcF92c19jdGwuYw0KLS0tIGEvbmV0L2lwdjQvaXB2cy9pcF92c19j dGwuYwkyMDA0LTEyLTAyIDAwOjE2OjM2ICswODowMA0KKysrIGIvbmV0L2lw djQvaXB2cy9pcF92c19jdGwuYwkyMDA0LTEyLTAyIDAwOjE2OjM2ICswODow MA0KQEAgLTc0LDYgKzc0LDcgQEANCiBzdGF0aWMgaW50IHN5c2N0bF9pcF92 c19hbV9kcm9wcmF0ZSA9IDEwOw0KIGludCBzeXNjdGxfaXBfdnNfY2FjaGVf YnlwYXNzID0gMDsNCiBpbnQgc3lzY3RsX2lwX3ZzX2V4cGlyZV9ub2Rlc3Rf Y29ubiA9IDA7DQoraW50IHN5c2N0bF9pcF92c19leHBpcmVfcXVpZXNjZW50 X3RlbXBsYXRlID0gMDsNCiBpbnQgc3lzY3RsX2lwX3ZzX3N5bmNfdGhyZXNo b2xkID0gMzsNCiBpbnQgc3lzY3RsX2lwX3ZzX25hdF9pY21wX3NlbmQgPSAw Ow0KIA0KQEAgLTE0MzksNiArMTQ0MCw5IEBADQogCSAgJnByb2NfZG9pbnR2 ZWN9LA0KIAkge05FVF9JUFY0X1ZTX05BVF9JQ01QX1NFTkQsICJuYXRfaWNt cF9zZW5kIiwNCiAJICAmc3lzY3RsX2lwX3ZzX25hdF9pY21wX3NlbmQsIHNp emVvZihpbnQpLCAwNjQ0LCBOVUxMLA0KKwkgICZwcm9jX2RvaW50dmVjfSwN CisJIHtORVRfSVBWNF9WU19FWFBJUkVfUVVJRVNDRU5UX1RFTVBMQVRFLCAi ZXhwaXJlX3F1aWVzY2VudF90ZW1wbGF0ZSIsDQorCSAgJnN5c2N0bF9pcF92 c19leHBpcmVfcXVpZXNjZW50X3RlbXBsYXRlLCBzaXplb2YoaW50KSwgMDY0 NCwgTlVMTCwNCiAJICAmcHJvY19kb2ludHZlY30sDQogCSB7MH19LA0KIAl7 e05FVF9JUFY0X1ZTLCAidnMiLCBOVUxMLCAwLCAwNTU1LCBpcHY0X3ZzX3Rh YmxlLnZzX3ZhcnN9LA0K ---1463811584-572891765-1097000265=:2451 Content-Type: TEXT/PLAIN; charset=US-ASCII; name="linux-2.6-ipvs-quiescent_template.patch" Content-Transfer-Encoding: BASE64 Content-ID: Content-Description: Content-Disposition: attachment; filename="linux-2.6-ipvs-quiescent_template.patch" IyBUaGlzIGlzIGEgQml0S2VlcGVyIGdlbmVyYXRlZCBkaWZmIC1OcnUgc3R5 bGUgcGF0Y2guDQojDQojIENoYW5nZVNldA0KIyAgIDIwMDQvMTIvMDIgMDA6 NDI6MTUrMDg6MDAgd2Vuc29uZ0BsaW51eC12cy5vcmcgDQojICAgW0lQVlNd IGFkZCBhIHN5c2N0bCB2YXJpYWJsZSB0byBleHBpcmUgcXVpZXNjZW50IHRl bXBsYXRlDQojICAgDQojICAgVGhlIHBhdGNoIGlzIGZyb20gSG9ybXMgPGhv cm1zQHZlcmdlbmV0Lm5ldD4NCiMgDQojIG5ldC9pcHY0L2lwdnMvaXBfdnNf Y3RsLmMNCiMgICAyMDA0LzEyLzAyIDAwOjQxOjU2KzA4OjAwIHdlbnNvbmdA bGludXgtdnMub3JnICsyMCAtMTENCiMgICBzZXQgdGhlIHN5c2N0bF9pcF92 c19leHBpcmVfcXVpZXNjZW50X3RlbXBsYXRlDQojIA0KIyBuZXQvaXB2NC9p cHZzL2lwX3ZzX2Nvbm4uYw0KIyAgIDIwMDQvMTIvMDIgMDA6NDE6NTYrMDg6 MDAgd2Vuc29uZ0BsaW51eC12cy5vcmcgKzMgLTENCiMgICBkb24ndCB1c2Ug cXVpZXNjZW50IHRlbXBsYXRlIGlmIHRoZSBleHBpcmVfcXVpZXNjZW50X3Rl bXBsYXRlIGlzIGVuYWJsZWQNCiMgDQojIGluY2x1ZGUvbmV0L2lwX3ZzLmgN CiMgICAyMDA0LzEyLzAyIDAwOjQxOjU2KzA4OjAwIHdlbnNvbmdAbGludXgt dnMub3JnICsyIC0wDQojICAgYWRkIHRoZSBzeXNjdGxfaXBfdnNfZXhwaXJl X3F1aWVzY2VudF90ZW1wbGF0ZSBwcm90b3R5cGUNCiMgDQpkaWZmIC1OcnUg YS9pbmNsdWRlL25ldC9pcF92cy5oIGIvaW5jbHVkZS9uZXQvaXBfdnMuaA0K LS0tIGEvaW5jbHVkZS9uZXQvaXBfdnMuaAkyMDA0LTEyLTAyIDAwOjQzOjE0 ICswODowMA0KKysrIGIvaW5jbHVkZS9uZXQvaXBfdnMuaAkyMDA0LTEyLTAy IDAwOjQzOjE0ICswODowMA0KQEAgLTM1OCw2ICszNTgsNyBAQA0KIAlORVRf SVBWNF9WU19FWFBJUkVfTk9ERVNUX0NPTk49MjMsDQogCU5FVF9JUFY0X1ZT X1NZTkNfVEhSRVNIT0xEPTI0LA0KIAlORVRfSVBWNF9WU19OQVRfSUNNUF9T RU5EPTI1LA0KKwlORVRfSVBWNF9WU19FWFBJUkVfUVVJRVNDRU5UX1RFTVBM QVRFPTI2LA0KIAlORVRfSVBWNF9WU19MQVNUDQogfTsNCiANCkBAIC04Nzks NiArODgwLDcgQEANCiAgKi8NCiBleHRlcm4gaW50IHN5c2N0bF9pcF92c19j YWNoZV9ieXBhc3M7DQogZXh0ZXJuIGludCBzeXNjdGxfaXBfdnNfZXhwaXJl X25vZGVzdF9jb25uOw0KK2V4dGVybiBpbnQgc3lzY3RsX2lwX3ZzX2V4cGly ZV9xdWllc2NlbnRfdGVtcGxhdGU7DQogZXh0ZXJuIGludCBzeXNjdGxfaXBf dnNfc3luY190aHJlc2hvbGRbMl07DQogZXh0ZXJuIGludCBzeXNjdGxfaXBf dnNfbmF0X2ljbXBfc2VuZDsNCiBleHRlcm4gc3RydWN0IGlwX3ZzX3N0YXRz IGlwX3ZzX3N0YXRzOw0KZGlmZiAtTnJ1IGEvbmV0L2lwdjQvaXB2cy9pcF92 c19jb25uLmMgYi9uZXQvaXB2NC9pcHZzL2lwX3ZzX2Nvbm4uYw0KLS0tIGEv bmV0L2lwdjQvaXB2cy9pcF92c19jb25uLmMJMjAwNC0xMi0wMiAwMDo0Mzox NCArMDg6MDANCisrKyBiL25ldC9pcHY0L2lwdnMvaXBfdnNfY29ubi5jCTIw MDQtMTItMDIgMDA6NDM6MTQgKzA4OjAwDQpAQCAtNDUzLDcgKzQ1Myw5IEBA DQogCSAqIENoZWNraW5nIHRoZSBkZXN0IHNlcnZlciBzdGF0dXMuDQogCSAq Lw0KIAlpZiAoKGRlc3QgPT0gTlVMTCkgfHwNCi0JICAgICEoZGVzdC0+Zmxh Z3MgJiBJUF9WU19ERVNUX0ZfQVZBSUxBQkxFKSkgew0KKwkgICAgIShkZXN0 LT5mbGFncyAmIElQX1ZTX0RFU1RfRl9BVkFJTEFCTEUpIHx8IA0KKwkgICAg KHN5c2N0bF9pcF92c19leHBpcmVfcXVpZXNjZW50X3RlbXBsYXRlICYmIA0K KwkgICAgIChhdG9taWNfcmVhZCgmZGVzdC0+d2VpZ2h0KSA9PSAwKSkpIHsN CiAJCUlQX1ZTX0RCRyg5LCAiY2hlY2tfdGVtcGxhdGU6IGRlc3Qgbm90IGF2 YWlsYWJsZSBmb3IgIg0KIAkJCSAgInByb3RvY29sICVzIHM6JXUuJXUuJXUu JXU6JWQgdjoldS4ldS4ldS4ldTolZCAiDQogCQkJICAiLT4gZDoldS4ldS4l dS4ldTolZFxuIiwNCmRpZmYgLU5ydSBhL25ldC9pcHY0L2lwdnMvaXBfdnNf Y3RsLmMgYi9uZXQvaXB2NC9pcHZzL2lwX3ZzX2N0bC5jDQotLS0gYS9uZXQv aXB2NC9pcHZzL2lwX3ZzX2N0bC5jCTIwMDQtMTItMDIgMDA6NDM6MTQgKzA4 OjAwDQorKysgYi9uZXQvaXB2NC9pcHZzL2lwX3ZzX2N0bC5jCTIwMDQtMTIt MDIgMDA6NDM6MTQgKzA4OjAwDQpAQCAtNzUsNiArNzUsNyBAQA0KIHN0YXRp YyBpbnQgc3lzY3RsX2lwX3ZzX2FtX2Ryb3ByYXRlID0gMTA7DQogaW50IHN5 c2N0bF9pcF92c19jYWNoZV9ieXBhc3MgPSAwOw0KIGludCBzeXNjdGxfaXBf dnNfZXhwaXJlX25vZGVzdF9jb25uID0gMDsNCitpbnQgc3lzY3RsX2lwX3Zz X2V4cGlyZV9xdWllc2NlbnRfdGVtcGxhdGUgPSAwOw0KIGludCBzeXNjdGxf aXBfdnNfc3luY190aHJlc2hvbGRbMl0gPSB7IDMsIDUwIH07DQogaW50IHN5 c2N0bF9pcF92c19uYXRfaWNtcF9zZW5kID0gMDsNCiANCkBAIC0xNDQ3LDkg KzE0NDgsOSBAQA0KIAl7DQogCQkuY3RsX25hbWUJPSBORVRfSVBWNF9WU19U T19FUywNCiAJCS5wcm9jbmFtZQk9ICJ0aW1lb3V0X2VzdGFibGlzaGVkIiwN Ci0JICAJLmRhdGEJPSAmdnNfdGltZW91dF90YWJsZV9kb3MudGltZW91dFtJ UF9WU19TX0VTVEFCTElTSEVEXSwNCisJCS5kYXRhCT0gJnZzX3RpbWVvdXRf dGFibGVfZG9zLnRpbWVvdXRbSVBfVlNfU19FU1RBQkxJU0hFRF0sDQogCQku bWF4bGVuCQk9IHNpemVvZihpbnQpLA0KLQkJLm1vZGUJCT0gMDY0NCwgDQor CQkubW9kZQkJPSAwNjQ0LA0KIAkJLnByb2NfaGFuZGxlcgk9ICZwcm9jX2Rv aW50dmVjX2ppZmZpZXMsDQogCX0sDQogCXsNCkBAIC0xNDU3LDcgKzE0NTgs NyBAQA0KIAkJLnByb2NuYW1lCT0gInRpbWVvdXRfc3luc2VudCIsDQogCQku ZGF0YQk9ICZ2c190aW1lb3V0X3RhYmxlX2Rvcy50aW1lb3V0W0lQX1ZTX1Nf U1lOX1NFTlRdLA0KIAkJLm1heGxlbgkJPSBzaXplb2YoaW50KSwNCi0JCS5t b2RlCQk9IDA2NDQsIA0KKwkJLm1vZGUJCT0gMDY0NCwNCiAJCS5wcm9jX2hh bmRsZXIJPSAmcHJvY19kb2ludHZlY19qaWZmaWVzLA0KIAl9LA0KIAl7DQpA QCAtMTQ2NSw3ICsxNDY2LDcgQEANCiAJCS5wcm9jbmFtZQk9ICJ0aW1lb3V0 X3N5bnJlY3YiLA0KIAkJLmRhdGEJPSAmdnNfdGltZW91dF90YWJsZV9kb3Mu dGltZW91dFtJUF9WU19TX1NZTl9SRUNWXSwNCiAJCS5tYXhsZW4JCT0gc2l6 ZW9mKGludCksDQotCQkubW9kZQkJPSAwNjQ0LCANCisJCS5tb2RlCQk9IDA2 NDQsDQogCQkucHJvY19oYW5kbGVyCT0gJnByb2NfZG9pbnR2ZWNfamlmZmll cywNCiAJfSwNCiAJew0KQEAgLTE0NzMsNyArMTQ3NCw3IEBADQogCQkucHJv Y25hbWUJPSAidGltZW91dF9maW53YWl0IiwNCiAJCS5kYXRhCT0gJnZzX3Rp bWVvdXRfdGFibGVfZG9zLnRpbWVvdXRbSVBfVlNfU19GSU5fV0FJVF0sDQog CQkubWF4bGVuCQk9IHNpemVvZihpbnQpLA0KLQkJLm1vZGUJCT0gMDY0NCwg DQorCQkubW9kZQkJPSAwNjQ0LA0KIAkJLnByb2NfaGFuZGxlcgk9ICZwcm9j X2RvaW50dmVjX2ppZmZpZXMsDQogCX0sDQogCXsNCkBAIC0xNDg5LDcgKzE0 OTAsNyBAQA0KIAkJLnByb2NuYW1lCT0gInRpbWVvdXRfY2xvc2UiLA0KIAkJ LmRhdGEJPSAmdnNfdGltZW91dF90YWJsZV9kb3MudGltZW91dFtJUF9WU19T X0NMT1NFXSwNCiAJCS5tYXhsZW4JCT0gc2l6ZW9mKGludCksDQotCQkubW9k ZQkJPSAwNjQ0LCANCisJCS5tb2RlCQk9IDA2NDQsDQogCQkucHJvY19oYW5k bGVyCT0gJnByb2NfZG9pbnR2ZWNfamlmZmllcywNCiAJfSwNCiAJew0KQEAg LTE0OTcsNyArMTQ5OCw3IEBADQogCQkucHJvY25hbWUJPSAidGltZW91dF9j bG9zZXdhaXQiLA0KIAkJLmRhdGEJPSAmdnNfdGltZW91dF90YWJsZV9kb3Mu dGltZW91dFtJUF9WU19TX0NMT1NFX1dBSVRdLA0KIAkJLm1heGxlbgkJPSBz aXplb2YoaW50KSwNCi0JCS5tb2RlCQk9IDA2NDQsIA0KKwkJLm1vZGUJCT0g MDY0NCwNCiAJCS5wcm9jX2hhbmRsZXIJPSAmcHJvY19kb2ludHZlY19qaWZm aWVzLA0KIAl9LA0KIAl7DQpAQCAtMTUwNSw3ICsxNTA2LDcgQEANCiAJCS5w cm9jbmFtZQk9ICJ0aW1lb3V0X2xhc3RhY2siLA0KIAkJLmRhdGEJPSAmdnNf dGltZW91dF90YWJsZV9kb3MudGltZW91dFtJUF9WU19TX0xBU1RfQUNLXSwN CiAJCS5tYXhsZW4JCT0gc2l6ZW9mKGludCksDQotCQkubW9kZQkJPSAwNjQ0 LCANCisJCS5tb2RlCQk9IDA2NDQsDQogCQkucHJvY19oYW5kbGVyCT0gJnBy b2NfZG9pbnR2ZWNfamlmZmllcywNCiAJfSwNCiAJew0KQEAgLTE1MTMsNyAr MTUxNCw3IEBADQogCQkucHJvY25hbWUJPSAidGltZW91dF9saXN0ZW4iLA0K IAkJLmRhdGEJPSAmdnNfdGltZW91dF90YWJsZV9kb3MudGltZW91dFtJUF9W U19TX0xJU1RFTl0sDQogCQkubWF4bGVuCQk9IHNpemVvZihpbnQpLA0KLQkJ Lm1vZGUJCT0gMDY0NCwgDQorCQkubW9kZQkJPSAwNjQ0LA0KIAkJLnByb2Nf aGFuZGxlcgk9ICZwcm9jX2RvaW50dmVjX2ppZmZpZXMsDQogCX0sDQogCXsN CkBAIC0xNTIxLDcgKzE1MjIsNyBAQA0KIAkJLnByb2NuYW1lCT0gInRpbWVv dXRfc3luYWNrIiwNCiAJCS5kYXRhCT0gJnZzX3RpbWVvdXRfdGFibGVfZG9z LnRpbWVvdXRbSVBfVlNfU19TWU5BQ0tdLA0KIAkJLm1heGxlbgkJPSBzaXpl b2YoaW50KSwNCi0JCS5tb2RlCQk9IDA2NDQsIA0KKwkJLm1vZGUJCT0gMDY0 NCwNCiAJCS5wcm9jX2hhbmRsZXIJPSAmcHJvY19kb2ludHZlY19qaWZmaWVz LA0KIAl9LA0KIAl7DQpAQCAtMTUyOSw3ICsxNTMwLDcgQEANCiAJCS5wcm9j bmFtZQk9ICJ0aW1lb3V0X3VkcCIsDQogCQkuZGF0YQk9ICZ2c190aW1lb3V0 X3RhYmxlX2Rvcy50aW1lb3V0W0lQX1ZTX1NfVURQXSwNCiAJCS5tYXhsZW4J CT0gc2l6ZW9mKGludCksDQotCQkubW9kZQkJPSAwNjQ0LCANCisJCS5tb2Rl CQk9IDA2NDQsDQogCQkucHJvY19oYW5kbGVyCT0gJnByb2NfZG9pbnR2ZWNf amlmZmllcywNCiAJfSwNCiAJew0KQEAgLTE1NTMsNiArMTU1NCwxNCBAQA0K IAkJLmN0bF9uYW1lCT0gTkVUX0lQVjRfVlNfRVhQSVJFX05PREVTVF9DT05O LA0KIAkJLnByb2NuYW1lCT0gImV4cGlyZV9ub2Rlc3RfY29ubiIsDQogCQku ZGF0YQkJPSAmc3lzY3RsX2lwX3ZzX2V4cGlyZV9ub2Rlc3RfY29ubiwNCisJ CS5tYXhsZW4JCT0gc2l6ZW9mKGludCksDQorCQkubW9kZQkJPSAwNjQ0LA0K KwkJLnByb2NfaGFuZGxlcgk9ICZwcm9jX2RvaW50dmVjLA0KKwl9LA0KKwl7 DQorCQkuY3RsX25hbWUJPSBORVRfSVBWNF9WU19FWFBJUkVfUVVJRVNDRU5U X1RFTVBMQVRFLA0KKwkJLnByb2NuYW1lCT0gImV4cGlyZV9xdWllc2NlbnRf dGVtcGxhdGUiLA0KKwkJLmRhdGEJCT0gJnN5c2N0bF9pcF92c19leHBpcmVf cXVpZXNjZW50X3RlbXBsYXRlLA0KIAkJLm1heGxlbgkJPSBzaXplb2YoaW50 KSwNCiAJCS5tb2RlCQk9IDA2NDQsDQogCQkucHJvY19oYW5kbGVyCT0gJnBy b2NfZG9pbnR2ZWMsDQo= ---1463811584-572891765-1097000265=:2451-- From Robert.Olsson@data.slu.se Wed Dec 1 09:38:06 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 01 Dec 2004 09:38:10 -0800 (PST) Received: from mail1.slu.se (mail1.slu.se [130.238.96.11]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB1Hc487020583 for ; Wed, 1 Dec 2004 09:38:06 -0800 Received: from robur.slu.se (robur.slu.se [130.238.98.12]) by mail1.slu.se (8.12.10/8.12.10) with ESMTP id iB1HbaKO016341; Wed, 1 Dec 2004 18:37:37 +0100 Received: by robur.slu.se (Postfix, from userid 1000) id 0CEF4EC001; Wed, 1 Dec 2004 18:37:37 +0100 (CET) From: Robert Olsson MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <16814.353.15446.489489@robur.slu.se> Date: Wed, 1 Dec 2004 18:37:37 +0100 To: sfeldma@pobox.com Cc: Robert Olsson , Lennert Buytenhek , jamal , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: [E1000-devel] Transmission limit In-Reply-To: <1101919791.5198.15.camel@localhost.localdomain> References: <1101467291.24742.70.camel@mellia.lipar.polito.it> <41A73826.3000109@draigBrady.com> <16807.20052.569125.686158@robur.slu.se> <1101484740.24742.213.camel@mellia.lipar.polito.it> <41A76085.7000105@draigBrady.com> <1101499285.1079.45.camel@jzny.localdomain> <16811.8052.678955.795327@robur.slu.se> <1101821501.1043.43.camel@jzny.localdomain> <20041130134600.GA31515@xi.wantstofly.org> <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <16813.58484.343629.570703@robur.slu.se> <1101919791.5198.15.camel@localhost.localdomain> X-Mailer: VM 7.18 under Emacs 21.3.1 X-archive-position: 12370 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: Robert.Olsson@data.slu.se Precedence: bulk X-list: netdev Scott Feldman writes: > Thank you Robert for trying it out. > > Well those results are counter-intuitive! We remove Tx interrupts and > Tx descriptor DMA write-backs and get no re-tries, and performance > drops? The only bus activities left are the DMA of buffers to device > and the register writes to increment tail. I'm stumped. I'll need to > get my hands on a faster system. Maybe there is a bus analyzer under > the tree. :-) Huh. I've got a deja-vu feeling. What will happen if we remove almost all events (interrupts) and just have the timer waking up once-in-a-while? --ro From buytenh@wantstofly.org Wed Dec 1 10:30:10 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 01 Dec 2004 10:30:14 -0800 (PST) Received: from xi.wantstofly.org (alephnull.demon.nl [212.238.201.82]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB1IU5tj022456 for ; Wed, 1 Dec 2004 10:30:09 -0800 Received: by xi.wantstofly.org (Postfix, from userid 500) id 80D2B2B0ED; Wed, 1 Dec 2004 19:29:43 +0100 (MET) Date: Wed, 1 Dec 2004 19:29:43 +0100 From: Lennert Buytenhek To: Scott Feldman Cc: jamal , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: [E1000-devel] Transmission limit Message-ID: <20041201182943.GA14470@xi.wantstofly.org> References: <16807.20052.569125.686158@robur.slu.se> <1101484740.24742.213.camel@mellia.lipar.polito.it> <41A76085.7000105@draigBrady.com> <1101499285.1079.45.camel@jzny.localdomain> <16811.8052.678955.795327@robur.slu.se> <1101821501.1043.43.camel@jzny.localdomain> <20041130134600.GA31515@xi.wantstofly.org> <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> User-Agent: Mutt/1.4.1i X-archive-position: 12371 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: buytenh@wantstofly.org Precedence: bulk X-list: netdev On Tue, Nov 30, 2004 at 05:09:59PM -0800, Scott Feldman wrote: > This doubles the kpps numbers for 60-byte packets. I'd like to see what > happens on higher bus bandwidth systems. Anyone? Dual Xeon 2.4GHz, a 82540EM and a 82541GI both on 32/66 on separate PCI buses. BEFORE performance is approx the same for both, ~620kpps. AFTER performance is ~730kpps, also approx the same for both. (Note: only sending with one NIC at a time.) Once or twice it went into a state where it started spitting out these kinds of messages and never recovered: Dec 1 19:13:18 phi kernel: NETDEV WATCHDOG: eth1: transmit timed out [...] Dec 1 19:13:31 phi kernel: NETDEV WATCHDOG: eth1: transmit timed out [...] Dec 1 19:13:43 phi kernel: NETDEV WATCHDOG: eth1: transmit timed out But overall, looks good. Strange thing that Robert's numbers didn't improve. Doing some more measurements right now. --L From buytenh@wantstofly.org Wed Dec 1 11:56:12 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 01 Dec 2004 11:56:17 -0800 (PST) Received: from xi.wantstofly.org (alephnull.demon.nl [212.238.201.82]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB1JuBrN024519 for ; Wed, 1 Dec 2004 11:56:12 -0800 Received: by xi.wantstofly.org (Postfix, from userid 500) id 5BE772B100; Wed, 1 Dec 2004 20:55:50 +0100 (MET) Date: Wed, 1 Dec 2004 20:55:50 +0100 From: Lennert Buytenhek To: Ben Greear Cc: netdev@oss.sgi.com Subject: Re: via-rhine unable to send back-to-back packets? Message-ID: <20041201195550.GC14470@xi.wantstofly.org> References: <20041129222700.GA22918@xi.wantstofly.org> <20041129172540.6b959858.davem@davemloft.net> <20041130064823.GA27872@xi.wantstofly.org> <41ACB0C3.5020408@candelatech.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <41ACB0C3.5020408@candelatech.com> User-Agent: Mutt/1.4.1i X-archive-position: 12372 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: buytenh@wantstofly.org Precedence: bulk X-list: netdev On Tue, Nov 30, 2004 at 09:41:23AM -0800, Ben Greear wrote: > >Yeah, preamble (8 bytes), CRC (4 bytes), inter-packet gap (12 bytes). > > > >Perhaps the via-rhine simply can't send out packets back-to-back and > >needs 14 byte times of inter-packet gap. I couldn't find any stray +2 > >in the driver anywhere but I'm just checking. > > Couldn't you just sniff the resulting traffic to see if it is too big? Sorry, forgot to mention: I did check that, and the data portion of the packet doesn't appear to be too big. --L From buytenh@wantstofly.org Wed Dec 1 12:02:10 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 01 Dec 2004 12:02:15 -0800 (PST) Received: from xi.wantstofly.org (alephnull.demon.nl [212.238.201.82]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB1K2AuT024983 for ; Wed, 1 Dec 2004 12:02:10 -0800 Received: by xi.wantstofly.org (Postfix, from userid 500) id 088982B100; Wed, 1 Dec 2004 21:01:49 +0100 (MET) Date: Wed, 1 Dec 2004 21:01:48 +0100 From: Lennert Buytenhek To: Roger Luethi Cc: "David S. Miller" , netdev@oss.sgi.com Subject: Re: via-rhine unable to send back-to-back packets? Message-ID: <20041201200148.GE14470@xi.wantstofly.org> References: <20041129222700.GA22918@xi.wantstofly.org> <20041129172540.6b959858.davem@davemloft.net> <20041130064823.GA27872@xi.wantstofly.org> <20041130122503.0adac947.davem@davemloft.net> <20041130220644.GC29947@k3.hellgate.ch> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20041130220644.GC29947@k3.hellgate.ch> User-Agent: Mutt/1.4.1i X-archive-position: 12373 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: buytenh@wantstofly.org Precedence: bulk X-list: netdev On Tue, Nov 30, 2004 at 11:06:44PM +0100, Roger Luethi wrote: > > > Perhaps the via-rhine simply can't send out packets back-to-back and > > > needs 14 byte times of inter-packet gap. I couldn't find any stray +2 > > > in the driver anywhere but I'm just checking. > > > > Or the via-rhine driver is not programming one of the registers > > proper to get optimal spacing. > > > > As with most Donald Becker drivers, many of the register layouts > > are not documented in the sources so it's not possible to just > > scan the driver looking for potential problems like this. > > For example, maybe the TxConfig register has an "IPG" field but > > we'll never know by reading anything in the driver source. > > Presumably Donald Becker had only access to the publicly available > documentation at the time which is very incomplete and buggy. What > little time my day job leaves for hacking via-rhine is consumed by the > WOL issues that have come up with 2.6.9+, but if you have a specific > question that can be answered by someone who knows the chip but not > necessarily Linux I can try and poke my contacts. "Is the hardware capable of sending back-to-back packets (i.e. with an inter-packet gap of no more than 96 bit times)?" "Can misprogramming the chip lead to the effect that the inter-packet gap is never less than 112 bit times?" Thanks in advance. > Of course, you can always check if VIA's driver has the same issue. If > it doesn't, chances are we can borrow the fix. Hmm, didn't know they had such a driver. Where can I find it? cheers, Lennert From buytenh@wantstofly.org Wed Dec 1 13:36:15 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 01 Dec 2004 13:36:20 -0800 (PST) Received: from xi.wantstofly.org (alephnull.demon.nl [212.238.201.82]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB1LaBxK001792 for ; Wed, 1 Dec 2004 13:36:14 -0800 Received: by xi.wantstofly.org (Postfix, from userid 500) id 549B22B100; Wed, 1 Dec 2004 22:35:50 +0100 (MET) Date: Wed, 1 Dec 2004 22:35:50 +0100 From: Lennert Buytenhek To: Scott Feldman Cc: jamal , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: [E1000-devel] Transmission limit Message-ID: <20041201213550.GF14470@xi.wantstofly.org> References: <1101484740.24742.213.camel@mellia.lipar.polito.it> <41A76085.7000105@draigBrady.com> <1101499285.1079.45.camel@jzny.localdomain> <16811.8052.678955.795327@robur.slu.se> <1101821501.1043.43.camel@jzny.localdomain> <20041130134600.GA31515@xi.wantstofly.org> <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <20041201182943.GA14470@xi.wantstofly.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="uZ3hkaAS1mZxFaxD" Content-Disposition: inline In-Reply-To: <20041201182943.GA14470@xi.wantstofly.org> User-Agent: Mutt/1.4.1i X-archive-position: 12374 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: buytenh@wantstofly.org Precedence: bulk X-list: netdev --uZ3hkaAS1mZxFaxD Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Wed, Dec 01, 2004 at 07:29:43PM +0100, Lennert Buytenhek wrote: > > This doubles the kpps numbers for 60-byte packets. I'd like to see what > > happens on higher bus bandwidth systems. Anyone? > > Dual Xeon 2.4GHz, a 82540EM and a 82541GI both on 32/66 on separate > PCI buses. > > BEFORE performance is approx the same for both, ~620kpps. > AFTER performance is ~730kpps, also approx the same for both. Pretty graph attached. From ~220B packets or so it does wire speed, but there's still an odd drop in performance around 256B packets (which is also there without your patch.) From 350B packets or so, performance is identical with or without your patch (wire speed.) So. Do you have any other good plans perhaps? :) > Once or twice it went into a state where it started spitting out these > kinds of messages and never recovered: > > Dec 1 19:13:18 phi kernel: NETDEV WATCHDOG: eth1: transmit timed out > [...] > Dec 1 19:13:31 phi kernel: NETDEV WATCHDOG: eth1: transmit timed out > [...] > Dec 1 19:13:43 phi kernel: NETDEV WATCHDOG: eth1: transmit timed out Didn't see this happen anymore. (ifconfig down and then up recovered it both times I saw it happen.) thanks, Lennert --uZ3hkaAS1mZxFaxD Content-Type: image/png Content-Disposition: attachment; filename="feldman.png" Content-Transfer-Encoding: base64 iVBORw0KGgoAAAANSUhEUgAAAoAAAAHPCAIAAABMSfxQAAAACXBIWXMAAAsTAAALEwEAmpwY AAAAB3RJTUUH1AwBFR0dvrAfowAAHrZJREFUeNrt3e1yqjoYBtB4Zt9Euf9ro5fh+UFLYxJC VJCvtWbPHkqVirY+vklIbn3fBwDgs/7zFACAAAYAAdym67rZGwzePPLLxwGAvfn3gfQdu5nj 7ZYj930/3uXl4wDA2SrgZ4OwfuM4y8cjDxn81HEA4OQV8MtBOGaqKAVAAK8lidu8MXm8wbAx lcqzsa17GIDPeL+A/PfJRxl36OY3mG3QrvcBS18APub90Uj/Nnnc67U8X6RNu95U4EydrJN1 pk72A2f6pv+2fWTxLePXbGyXNuAZgFN6qwIu9t0mwRl38U7tmapl4/RtvBcAHMJtjTDbpGy9 VDsPABtaJHFWaYKWgsAn3wfXGIO5q3n3Vn0ke5thMJ/0cL1XeVvmggZoKiTOeqlFY8m01OnX jzM0oA5Of22LAAZgsZz+2HFO4J+nADh6KhTn9ol3Jm/9s7MDTdVts0euFHzxMNVkovv2cEoe an4WyZDYlmejvWyN79LyrObnNXuc4qOqfOspX133vaf4F8DAqSQruBSn3nthZZfkjqE6qV/x Lsm1IfF3Z48z9ROLp1zZaP+okYdfcbr+5Lxa9swe59mXRgUMsKOauFiZxbk4bLzzFp8fJz/a az9rKheT/ZWFavKNyr1afnp7lfz+C7dG+n79PrZhYyd1sAAGrhLJyZ733+VbjrPgz0oO8toB Vyori23j7x9qKUPi7q0J2iAs4CqSq1nyxt7G5Jg9Tn7A+pje8cKb2fnwiwd5LfBWGmOctDC3 nFfyrF5nlJYKGDh5+Ts7GV/cUJxM8Fecj699Ur+p2yS3f6p1erxQ54XhVMV7tSRf+xSH9Qc2 e5zw5Ki0F+rg/bid5rOGmbCAg753nfKN69y17H5nwgIABDDAfp21TNQeKYABQAADB2fxg5Od 6csPsnKD9y9DshgDwOdY/GCl03/hOPlA65fP4uUfbTEGAHad06seRz/uqlwHDDz9jmzxg3CW xQ+eLaNbnvnkRz/1HK60GEP31fXffb4tgIEDs/hBOPLiB1MTZecfLGbPopiy9We+uPHsr0rT r9N3P+TuftJXAAPL1MTB4gcTh9r54gdTz+HsuS/1mBuf/AUzeD9/OwIYWCuSkz0WP1ij4SF8 ZLTaImfR+JhXmkJrhxWwQVjAWix+sEYRWTnsC4sfbHIWW6XvWAergIFzlr8WP6g/nr0tfrDI WUx9WElehfwx1+N5saHgUdW7nwrYYgzA5epyix+wh8TRBA0AGxDAwLVY/AABDMDZrDRPuAAG YJskS5Yo6B5Vsmq9Pfn+F/qh81PID3vWeaEFMMAB0jdfoqCPxDfLE3GNPUud5nWWXhDAyWv/ 5e8c2GH6vtChm18RtN6e4kNd8Prj4oRZliMEYF1TU4kVg2q3sVRvGK+c13Wup/rv2r/l34pg 4KDFccuqvRtW6nkBPX5cyD83XPMiZjNhARwvfTePq5bITObTnpqm+7X1sh6PsHwp1fffAhiA C03gFY/0bl688vuI564PGODAKZWUm+Ejo7GK0Tg1RGtqueXiceJx3estD7wTV6+Ah27gg356 As4at/HGmHlJSuULJBSXUFxpT/uHhvp911h64SgsxhAEMMB6xfo+j7lV4sQ0QQOwmDWS8qyV sQB2MRIAAhgABDAAIIBXpBUaAAEMAAIYABDAACCAT0U3MAACGAAEMAAggNejFRoAAQwAArgq X5bynZtV7tL9WvUZUQQDcIAAXi8Ok5Wchy+TVTAXZ11CAA4QwO1rNLZEZnybZCXnLCZ7rxwA 1w3g9vRNbvmZxuSXi2Ct0ACs7d/nf2Scx3k7cyXXx5tVgr/lNgDwWn4dKYCLzdTJOQw3mG3Q TmLbrwIAKuCmjwxxz+5KP2uRIw+t0AZkATBVB75/tFWuA44f2Th0uZiOyS3j7eFbil0AVMC1 0jZELcn14Iw7fSvhOtxsvEHjvRb9GKEIBmAttzXCbJOytT6ASwADsKvEWaUJWqMxAGwQwOfg gmAABDAACGAAQACvRys0AAIYAAQwACCA16MVGgABDAACGAAQwOvRCg2AAAYAAQwACOD1aIUG QAADgAAGAATwerRCAyCAAUAAAwACeD1aoQEQwAAggAEAAbwerdAACGAAEMBXoggGQAB/Wt9/ exIAEMAAIIAvUwRrhQZAAAOAAAYABPBKtEIDIIABQABfiSIYAAH8aS4IBkAAA4AAvlIRrBUa AAEMAAJYEQwAAhgABLAiGAABDAAIYEUwAAIYABDA+6YIBkAAf5qZKQEQwAAggK9UBGuFBkAA A4AAVgQDgAAGgGMHcNd1szcYvHnkl4/zeYpgAOr+fSB9+77Pt1uO3Pf9eJeXj/N5WqEBWLcC XjYI4ywfjzxkcJZwvVcOgOtWwC1BWLzNmKlnjdKhCDY1BwCrBPBr5XLemDzm8bAxlcotsX36 aAdgK8sOQvr3sQcdJ2JyDnkw14vpnfcBK4IB2EUA53m5XnwqfAFYO2IWKYVXuQ64OJxq9pbx zcZ26UMUu9WnwnBoAApu78TbVEty8cvKbRpL5/q96v3HGwawVmiAk1kkcW5rJNYmZes+A1gG AwjgolWaoHXEAsAGAczjxxETYwEggAFAACuCARDAAIAAVgQDIIBZhAwGQABvUAR7EgAQwIpg AASwIhgAAQwACOCzFcFaoQEQwAAggBXBAAhgAEAAK4IBEMAAgAA+KEUwgADm00zKASCAUQQD IIAVwQAIYBTBAAhgRTAAAhhFMAACWBEMgABGEQyAAFYEAyCAUQQDCGAUwQAI4GtnsCIYQAAD AAJYEQyAAEYGAyCAAUAAowgGQADLYAAEMAAggBXBAAhgimQwgABmgyLYkwAggNkmgxXBAAIY ABDAimAABDAAIIAVwQAIYJYggwEEMBsUwZ4EAAHMNhmsCAYQwADAvgO467r379X9UgT7xQUQ wCumb9/3432HLwcXz+BgNBaAAG7M0WdzerxXMW4bD3jiItgvLsDR/Vs/LfpK1l48St/J4K77 ksQAAvjFmjhvZ66kcktsi3YA1suvYwdwfg5DWM42ViexrQhWBAOogJ8Nj/5wR5bBAJevfPoF S+GNrwOOzyHOzrFdWrHb8BwaEQ2gAp6O2LF/N+70rYTrcLPxBo33ut7HMZcFAwjgasE+u3P2 ZnK3ksEaogGOxVSU6mAABDBvkMEAApgNimBPAoAAZpsMVgQDCGBkMAACGAAEMIpgAAHMCclg AAHMBkWwJwFAALNNBiuCAQQwACCAFcEACGBkMIAARgYDIIBZiAwGEMBsUATLYAABzGYZDIAA ZoMMVgQDCGBkMIAARgYDIICRwQACmHOSwQACmA2KYE8CgABmmwxWBAMIYGQwgABGBgMggFmb DAYQwGxQBHsSAAQw22SwIhhAALMNGQwggNmgCPYkAAhgtslgRTCAAEYGAwhgZDAAAhgZDCCA kcEACGBkMIAARgYDIICRwQACGBkMIIChTgYDCGA2KII9CQACmG0yWBEMIICRwQACGBkMgADm AxkshgEEMBtkcDAuGkAAs0kGa44GEMBsFsMyGGAvAdz9SnbmN1tkDzIYQACHruv6X2NYDjvj 7FxqDzIYQADXIjmEMGbnUnvYFRkMsK8A5iJFsCcB4Cn/ln4j/itPh4L143XYlj/94hncdV+S GDixZdtf/y3+4Mbki7eRwQCsGMA7yACRL4MB1o2YRUrh/z7wcJcde6WwPkQGex4A6m6Lh1mx FzZPzaX2JD9XNu+EOhg49VvcAolzO01iCWAZDHCgxHEZEmvRFg0ggJHBAAKY62WwGAYQwGyQ wcFclQACmE0yWHM0gABmsxiWwQACGBkMIICRwQACGGQwgABGBgMIYJDBAAKYE2SwGAYEMGyQ wcE0HYAAvqbuq+u+unwbGQywkn+XTdwQQv/dx1k7bvff1jTcLIOtYAiogE/9dv/dTyWu9N02 hnUJAwL4EhlcqYPZKoONjgYE8JnFnb5jEieVMUphAAG8fPomLc/j8KsxhiWxUhhgJRcdhBW3 Pyc749zVH7yTUtjILEAAXyKbn+oSHm4vqmUwwFNcB5x66sIk6fvJDPY8AAL45Ok7jsYqpmz8 rXH/2HA9bMTbnlUZDJDTBJ1WtMVUjtulk0yNq+T4lvk4L5bKYM3RgAr4/KVwMXSLyT2mb7FW ZqkMDiG4Qgk4gVvfnyQkuq4LIXz4dOKSN6mAZfDKL/dXHMkAh0scFfC76VsM45DNdsnipbAl HIBDE8BvZMBvxA4bcftzXBYHo7FWjmEt0oAAvm4MT/2fzK6V19DFbZ6NYaUwcDhGQa+bzcV1 D4sZnNTTPJvBQx2sSxhQAfOw3kNlKo/49tL3nQx2rTCgAibkfcB5QRwPnJ66zjhkPcrMlsLB 6GhABXzdJChNp1WsjGd7gqXvsxkcdAkDApg8ZfNwTQK7soEMBs7BRBz7iup6Se339fnfCsOy gJ0mjgp4X+lbTFlzerxTCrtKGNgng7B2ExXV1mbl7zsZrBQGdkgFfJjiOP8fpTAggFm3OI6X WppaD5HZDHahMLAfmqCPlMEh6ipO1iEu3ljD9VQpHFwoDKiAaVFceSmpgJM98RwgGrGLpbBq GBDAzKdvJYPD41pM4w2SKrnYlH3ZJBbDwLY0QR8hKqLllZI0zUO6ePdi+rZkcJ7r54vhMYM1 SgMqYOZL4amW55B1/VbSN7ll/mV9Gq8zlcIhBNUw8ElmwjpnTods+o7ZOrj4ZUt1fqYSWSkM fCxxNEGfsaSLauJkeyp9Q9aI3RLG52ugjieRFsPAqjRBXyKJw2NHcuMlxWO4ximbd0Wf8Hmz lgOwPk3QTDyfUfpWAv7EQ7TUwcCqiaMCplY91/N1HESdbBTr6SPWwWbOAtajD5j5GJ7K4GKP cn1ikNlQ32EMK4WBNSxfAXe/8mp9jT3vGCaJGjbiL2mP56nrmuKLjOOm7LhWTv7ttmg2ZQdw gADuuq7/NYblsDPOzqX2sK3GK5qG7XqH8f67k8UwsPcKOHrD6sfUHL4csnOpPYvUvnH5G39L Kfxy+obq3JlHH1Btyg7gAAG8Z999//07em3YSL5kPoqyK5riDuNk51ROJ9l8jBM3cxawhOUH YY3l6SZXBL3/05PKeIjkr64b/h+/jPdL4jA9YXVeAU/V0+FxscWdF8RxBhufBRexbB/o8gE8 Jt/YaLznOrhYBCdB2xjbcRKPR7jiL2hUCrePf65PUr3/GA6GSQObB/DWb4iLvXFXOomL+y+e u0lBnG+H6cuZQmlyrmQW6/xm+4lqMQxXeX+LKsz3j/bfBx7usmOvPlBYDz3Es53ESdAWE9pg rqfq5uL0W2MZnQyrztN322lATNwBbFkBx0OUx5gcdsapudSebc2Gq+7hF9J3ahGI4rTVyc49 VMYm7gAamQt6gRiuj9KSxC/kcS3h5pZm2km7tBiGM79NmQt6P5KIjVuweSF9Kx3JxXyduiJ5 QybuAFTA28gbqOslsiuaWiK5ONNWnsFJrbx5QTxmsIIYJI4K+BM18QuTe+RTdIWGzuZzG4Zf xdtTixkn7c9Jck8N0fpAufw7qk9BDKiAt6uDG8NbL/IidXMS5FOXM21SEKuG4eKJYznC1evg MYlnm6Dz2E42xPDL6RuyC4srQ6zzzF42pF03DARN0PtMa95P37jhOs/U4jXH+e1X7ULWKA0C mM+F69gxXNzIkzjfSNYwjutjqxqPOZqM2JpalClfwLi4UlO9Sl4whiUxCGC2zOni0K08dKd2 UkniYgYnO5O4TebeWimJjdKCCzII6zAWGc/leqeH35lsvq08wvMLn4oTdU1tJ4dqbM3WPQxX SByDsA5WH9djeGpUF1P5mmxUYjifibp4LdPsNcotGWyUFlyBJugjpW++/dTKEEE/8fMl8mx2 JpN2VXqXn73kSaM0CGD2FcNTg7biUV1TCR1/KYNbSuRiBlfm9Kj0Lie9yI19yWIYBDA7rYnf 6dBNCuL8SyE9JG6xfi1OWD07S9f71bAkhnPQB3zyeJ6qes1z+VoSJ1NdTo2sTsrlJMXjvuQw Nw3IQ8z/9g3rHoYTMAr6il4IXQtILPArWl2v6emdt3ucysDhEkcTtPp4spM42WDB9M1bs4vT gNQuZ7rfkoIYOBYBfN30bUzWZLqPfFi1RuwWYyt0sa4tjttK7hvf8e/uQwzfb2IYDkcTNE0Z bDHjZavhOFmfmlqreIVxyNqlw/0Wnpz9A/hw4hiEBZ9O3+KKxVPfmq2V46Dt7rffj9b3EEL3 +wax6qoSgApYBfzpsri4v1Iiq6GXTfGpybYekjsaq5VP1+WZhA0TRx8wL8qnBHkzQa0t8Wz6 Fld5Sovp+23413VfYxjPXgEFfIAAZrFSeHa4VnH/1MbUrCByOh7PFaoDux7uNiTx7f7zb/0l ngABzOp1cDIXZlIQP3uBU2O41qfxulopHE/3ES+BnCyHHI+aDrf71DSZgADmKilejOT2Ylop nJTFobocclwQTyUxsDaDsNiLeLni+nCtZ3Odet1cuX7JWC1YL3FUwOyrDm5JzcZW7tle5Gt2 J5fnrP5tmh56iLvuS9M0rE0As9MwjldXDNmg63pOP9tGPdWLfNZszheHiJN4CGNN07A2TdCc 0MvBWWz0vsIzll+YlDdNF3uU4Zo0QcNkjtY3Zgdg5xdEndvUyolx03TXfcVFsKZpeJMA5uQZ /FRUv3ll1DliuNI0PV5G7PolEMAwE66VXuRK6BavjKoM4zrZeK60Dh79zqtVuX5JEoMAhqej Og7mpPZtD9fKNF4HTeKx3/cvjB+bpiUxvMBqSFBL5Tx6G2fcPJNkvaaH/++3cbjWsPjSmL4G bYEKGJbM48aZNesTXB86iStXEv8s+fA713ScvuOXgACG1uhtvxhpdgB2Ui5PXXy885CeTOL+ +6d1+nHVB0kMAhgWSOL68K4pLRNZ5zt3vhjUVBLHw7XyJB7oKubi9AHD8jk9tVEP3UN3JxdX gwghdPfbz9fDnB6luabNO40KGNgynuulcyhNmbnPkM4XYkquJC4OnDZ8GgEMrJu+U4k7G8nv rDmxlyQeLl5KLiZ+XPtBEiOAgdWTOL/4eGrPVEEc2hqrt10MKk7iv8bq34uJ/9Z++E3i8Hgh kyTmlPQBw+6CebbqDW3dyTtc1ml6+PTvdvcVwk9vcbwyhN8NVMDAlnlc/G77mhPFYN6kO7n1 QibpiwAGdpjNb645MdWdHKoTai470ebMhUzDKkzjP23RnIgmaDhVEjeu7xRXwGGhxuqvrntn +eTihUx/VzGF8DOCevxu/+3V59Bup1nBfpHlkeGClm18/u77N5M4/dOOxkU/ZHP3JYY5dOJo ggaemOl66gbFqnq2EbuxMk7m6PiplfvvYfj0+M/ryLFoggbp24+V64er6qFWHm6cbIzRnlfA cR7HFXCcwSpj9k8TNLBk6L4c5MUALi5r2LLE4RjGkpjdJo4KGFiskh6jd6qurSR08SLmewhD zuZN0HVj7iqL2a21+oC7xz+wLvt7W2oPsKsMDm0XR71wEfNrhq7ivMNYnzGb+0QF3HVd3/fD /8vuAXYexvWIrTRWJwXxInmcVMCKY7a1Sh9w3Di+Ru4WM1gfMJzMItHb2Hmsz5iXY25HFfCY l14hYHPDCOrZaaX1GfN5ZxuENQa/UhiOrlj7FuflqAfwc0e4R/NiZv3EIlnhu98A1kELfKCi nc3gqUWF81WHp44fSn3G6mN2XQGPHxA2CWPxD1fL4KmNOGLjPuA8g/M7Try9FMZwieFr/e5F Q5HeP9paE3GsN/ZqKtcNwoKzSi4gvt1DewUcB/CzFXDbe52a+IoOMxFHnppL7QGupiV9Q9Ty PA7CKqx4OHHHpzJ4ai5MkcxmFfBBP48AO6x9i569PCnO1Kdq3DfelOSxCnjrChjgNUPK5pNZ vnCoxsksk17h5Agt01BX4jZpr57qqP7AhwP2QAADpClbbKB+dkGIeiR33VcI927oXHscsJ30 XotkAQywZR0clpsguiWDixtJOrZfjlwox++3nyN3XyHc47I4Sf2kSpbHp/GfpwAgr3SnStKH EJ0K17Yf8ZOm/Xe438L9NiwXEW73v3+PPz0ZQVa80BkBDLCiccnCykb85bPpWwzRZGdcicYb U/unbhnH/BjGP//iML7du+4rH8s9NeUI+6cJGrhKYLe0YA8B+dV192wN49u9C2FoOf6pTou1 crG9unLLZJKQh0cz/LDR7T72HMf9x8UM1litAgZYOErjGreyke+vlMj5YXP32088D9ci12vo uiFYx4D86rr+ux+K3ngjTeH7bSiRx/+HJuu/hussfcflj4cfp0oWwAAv+u77cTnhcU9xI693 G5ujZ6P9fgtfXRc3Fc/+S/J1yMhhoyXyh1veb+lMXvdwC/fb8H/ff8eN1WOTdfyxII9kv1EC GGDFzC5GbCVo69H+WuHeWKZP3WDM4DGJhz7i4f+fVujfwB8iOQwXO8W9yI9DuOWxAAZ4JVPj ajjfiL+sl8izpXPlATx15JaDN9bxYxIPGZxk57DzL4mnR3WN6R43U4tkAQywWGA/VbzWo70S 6vXKuyXCnw3msY16/BcH89gv/NORnJXIQxLfwl0v8ueZCxpgG19dN4yvbvxwEA/Jjv+vZPZ4 myFb40iO/QV2uCffKK4r9dpEYGdiLmiAY9fl4Zl+5WIRPE6XnVw0lUfsd9+H0A0bQ4KOeRyV y7c4ksPt55Knv+CJroxKquHxaK59EsAAVw/1qYbuOI+TxurB787bw7cem6l/DhR98zeGTZzZ RBM0AH+G9upiL3IevmmTdUgnDyk2ep8gmDVBA7C8/vuvsfp+S3uRx2I3brKeKpFvyWReUYk8 XkkVrtpqLYAB+DO2WicdyWOr9d9lx4VkvaW172Me38MtLqzHb16zO1kTNABLvAk/DuyayJxC k/XMXXaZx5qgAdiLIR2/+9B/T+fx/faw8+cq5LQlO7l9Prwr/qHH7U4WwACsnsdjldx/92OO Di3SNdG3f3I6beXuiks0CmAASFN5DMgxPvO69yd577VBXrcQ7rfb7T4M5uri4V1TB9xVNgtg AHaRx1HOPgTzVB7/XAd1+ymOJ0rkv0I6bsTeQyQLYAD2XiiHMDVbyO3hquVwL7Rax1/+fr2H AdgCGIAD5PFUd3L8/5DHeZo+hHbisaqeGvC1RjC7DAmA40fAV7k7eTKJ2/I4uWM0fnv4+q08 VgEDcJIqeSoO80bsn+1soq50Jq8w2Yj9fjUsgAG4RDzHjdiTA7BDtRH7bxj2LfwuQiWAAeDd ijmplScr5lsIzywlKYAB4LlauRjMgzcn/RDAAPBcMHehC2/3Af/naQWAzxPAACCAAUAAAwAC GAAEMAAggAFAAAMAAhgABDAAIIABQAADgAAGAAQwAAhgAEAAA4AABgAEMAAIYABgrQDufiU7 85stsgcABHDouq7/NYblsDPOzqX2XFb+EceZOlkn60yd7NUr4GIkhxDG7FxqDwAI4B9DRgIA df/WLnw/7FLF8aXatbysTtaZOlkV8H7TFwCuWwFvlb4iH4DrVsB5+i4+9kp5DcAJ3JYNs6Tp fzx4nppL7QEAAQwANDEVJQAIYAAQwADASv6d4BzaR36d4EwvMk4tPotzv77j2cXncuKXNT7Z s76y74xFPcHJnv4NOTmRd17WMwRwKF0EPK7ccJoMLq4NlZzjOc46P9Ozvr75a3fulzU/2VO+ ssm787n/Wosne4U35EVe1nM2QZ9v5YbKh6yTrVfR8ot74pU5Lr4MyclONvlQde6XtfKXe6aT zRfoe+dl/e9Mf7onfnu6ztVilb/h872+l7oI8FKvbLjYpAXFCuF8L+vir+l/Z3perFR47j/v E7++l32zPusrO9ZDF3lBi6WhN+RZZ+gDNpfINSsn6Xv0kz3xWZ94KOjsyZ71fNd4KV2GBNLX yULTL/CybRv/neNJyT+RnX7lhuusV3Hi1/e1xUtOc7JnfWVbzutML+tF/mD7X2PR//7LejvN hVlJY8jJcug6VxYWz/Ssr+/FLxi9wit7wcu7T/+GvOB1wBZjAIAN6AMGAAEMAAIYABDAACCA AQABDAACGAAQwAAggAEAAQwAAhgABDAAsJ5/U99IlrNoX8lk6miVhTLajzOrZRmK+uIVjQvU vPyYlzoOACcM4CSQ4hUQ67d5Kn6G7deOU//Q8NRtkodRTMfkXi8/5uRnLXjuABzLJ5qgW3Jx kZ/yTu07LqScf+udaIyPWflZKmAAAVwLg67rnk3TjxV2LT/ltUfSXty/8PwAcE3/2uMzqQjj Mq492PJ7FcvBzdU/N1TK5RC1M8dFcOUzjQoYQABPhlAeDy39l7PDnZJu1wOVj/ljnhqnVk9W fcAAAviJEvCpUvJwAfPaQxWfALT7rzGB2mvT+Jb9r9DQBpvvX6Qgrh9kbCt+djBz+zPw5s8C 4JRuU03HeW3Xcv3uVI/mC9cBP5tSLY+5/Rrf4sVIzz4bLR9u9AEDCOBlLFLeLVgjKjcB2KH/ AdjzOoqApIL+AAAAAElFTkSuQmCC --uZ3hkaAS1mZxFaxD-- From buytenh@wantstofly.org Wed Dec 1 13:59:45 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 01 Dec 2004 13:59:50 -0800 (PST) Received: from xi.wantstofly.org (alephnull.demon.nl [212.238.201.82]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB1Lxirr002754 for ; Wed, 1 Dec 2004 13:59:45 -0800 Received: by xi.wantstofly.org (Postfix, from userid 500) id 7BA6F2B101; Wed, 1 Dec 2004 22:59:23 +0100 (MET) Date: Wed, 1 Dec 2004 22:59:23 +0100 From: Lennert Buytenhek To: Robert Olsson Cc: netdev@oss.sgi.com Subject: Re: [PATCH,pktgen] account for preamble and inter-packet gap Message-ID: <20041201215923.GH14470@xi.wantstofly.org> References: <20041128213251.GA9330@xi.wantstofly.org> <16811.19277.594137.939120@robur.slu.se> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <16811.19277.594137.939120@robur.slu.se> User-Agent: Mutt/1.4.1i X-archive-position: 12375 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: buytenh@wantstofly.org Precedence: bulk X-list: netdev On Mon, Nov 29, 2004 at 05:16:13PM +0100, Robert Olsson wrote: > I've heard some boxes that didn't do the ipg correctly. Don't know what NIC > this was. We can only estimate stats at the point where we are delivering > packets. Other devices has "true" L2 stats. Just found out the other day that via-rhine appears to do this. > OK. Adding FCS yes kinda compromise and maybe was wrong in this aspect. Let's remove it then? > The only solution I can think of is of having a selectable option in > the config for output stats. To support different L2 layers and with > warning if we are "predicting" L2 statistics. Feel free to extend your > patch. At that point it might be easier to just do it in userspace, no? --L From sjackman@gmail.com Wed Dec 1 14:03:42 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 01 Dec 2004 14:03:48 -0800 (PST) Received: from mproxy.gmail.com (mproxy.gmail.com [216.239.56.249]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB1M3fSQ003184 for ; Wed, 1 Dec 2004 14:03:42 -0800 Received: by mproxy.gmail.com with SMTP id w41so88283cwb for ; Wed, 01 Dec 2004 14:03:18 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=SdwSV4xL+mxR4tLQ/vd+ZennOM9ZCci6fHRQLD19YMy5YRmYsoPTXwFxMUPmx/1CHUh6LVIUJa766uCakASbKSy1Mowr7dtF5GwaWDRgn69iGZX/sxowkM1Cwo5+7wKsGrfa0IWh0aVFhGeKsPma6K4e/su00/CSvlw4eqleM5M= Received: by 10.11.116.64 with SMTP id o64mr326004cwc; Wed, 01 Dec 2004 14:03:17 -0800 (PST) Received: by 10.11.99.50 with HTTP; Wed, 1 Dec 2004 14:03:17 -0800 (PST) Message-ID: <7f45d939041201140329d0273f@mail.gmail.com> Date: Wed, 1 Dec 2004 14:03:17 -0800 From: Shaun Jackman Reply-To: Shaun Jackman To: netdev@oss.sgi.com, Andrew Morton Subject: Re: Multicast filtering for tun.c [PATCH] In-Reply-To: <20041127011006.03232bb6.akpm@osdl.org> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <7f45d9390411251715138b35d0@mail.gmail.com> <20041127011006.03232bb6.akpm@osdl.org> X-archive-position: 12376 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: sjackman@gmail.com Precedence: bulk X-list: netdev This patch adds multicast filtering to the TUN network driver, for packets being sent from the network device to the character device. It applies against the 2.6.8.1 kernel tree. Cheers, Shaun On Sat, 27 Nov 2004 01:10:06 -0800, Andrew Morton wrote: > Shaun Jackman wrote: > > > > This patch adds multicast filtering to the TUN network driver, > > You should cc netdev@oss.sgi.com on networking stuff. > > You may not get any feedback on this work. Persist. If you get nowhere, > ping me and I'll help push things along. > > > This is my first attempt at sending a patch using gmail. > > Send the patch to yourself first, check that the result applies OK. 2004-11-25 Shaun Jackman * drivers/net/tun.c: Add multicast filtering for packets travelling from the network device to the character device. * include/linux/if_tun.h (tun_struct): Add interface flags, a hardware device addres, and a multicast filter. diff -ur linux-2.6.8.1.orig/drivers/net/tun.c linux-2.6.8.1/drivers/net/tun.c --- linux-2.6.8.1.orig/drivers/net/tun.c 2004-08-14 03:55:23.000000000 -0700 +++ linux-2.6.8.1/drivers/net/tun.c 2004-11-25 17:00:22.000000000 -0800 @@ -41,6 +41,7 @@ #include #include #include +#include #include #include @@ -104,11 +105,42 @@ return 0; } -static void tun_net_mclist(struct net_device *dev) +/** Add the specified Ethernet address to this multicast filter. */ +static void +add_multi(u32* filter, const u8* addr) { - /* Nothing to do for multicast filters. - * We always accept all frames. */ - return; + int bit_nr = ether_crc(ETH_ALEN, addr) >> 26; + filter[bit_nr >> 5] |= 1 << (bit_nr & 31); +} + +/** Remove the specified Ethernet addres from this multicast filter. */ +static void +del_multi(u32* filter, const u8* addr) +{ + int bit_nr = ether_crc(ETH_ALEN, addr) >> 26; + filter[bit_nr >> 5] &= ~(1 << (bit_nr & 31)); +} + +/** Update the list of multicast groups to which the network device belongs. + * This list is used to filter packets being sent from the character device to + * the network device. */ +static void +tun_net_mclist(struct net_device *dev) +{ + struct tun_struct *tun = netdev_priv(dev); + const struct dev_mc_list *mclist; + int i; + DBG(KERN_DEBUG "%s: tun_net_mclist: mc_count %d\n", + dev->name, dev->mc_count); + memset(tun->chr_filter, 0, sizeof tun->chr_filter); + for (i = 0, mclist = dev->mc_list; i < dev->mc_count && mclist != NULL; + i++, mclist = mclist->next) { + add_multi(tun->net_filter, mclist->dmi_addr); + DBG(KERN_DEBUG "%s: tun_net_mclist: %x:%x:%x:%x:%x:%x\n", + dev->name, + mclist->dmi_addr[0], mclist->dmi_addr[1], mclist->dmi_addr[2], + mclist->dmi_addr[3], mclist->dmi_addr[4], mclist->dmi_addr[5]); + } } static struct net_device_stats *tun_net_stats(struct net_device *dev) @@ -301,6 +333,10 @@ add_wait_queue(&tun->read_wait, &wait); while (len) { + const u8 ones[ ETH_ALEN] = { 0xff, 0xff, 0xff, 0xff, 0xff, 0xff }; + u8 addr[ ETH_ALEN]; + int bit_nr; + current->state = TASK_INTERRUPTIBLE; /* Read frames from the queue */ @@ -320,10 +356,37 @@ } netif_start_queue(tun->dev); - ret = tun_put_user(tun, skb, (struct iovec *) iv, len); - - kfree_skb(skb); - break; + /** Decide whether to accept this packet. This code is designed to + * behave identically to an Ethernet interface. Accept the packet if + * - we are promiscuous. + * - the packet is addressed to us. + * - the packet is broadcast. + * - the packet is multicast and + * - we are multicast promiscous. + * - we belong to the multicast group. + */ + memcpy(addr, skb->data, min(sizeof addr, skb->len)); + bit_nr = ether_crc(sizeof addr, addr) >> 26; + if ((tun->if_flags & IFF_PROMISC) || + memcmp(addr, tun->dev_addr, sizeof addr) == 0 || + memcmp(addr, ones, sizeof addr) == 0 || + (((addr[0] == 1 && addr[1] == 0 && addr[2] == 0x5e) || + (addr[0] == 0x33 && addr[1] == 0x33)) && + ((tun->if_flags & IFF_ALLMULTI) || + (tun->chr_filter[bit_nr >> 5] & (1 << (bit_nr & 31)))))) { + DBG(KERN_DEBUG "%s: tun_chr_readv: accepted: %x:%x:%x:%x:%x:%x\n", + tun->dev->name, addr[0], addr[1], addr[2], + addr[3], addr[4], addr[5]); + ret = tun_put_user(tun, skb, (struct iovec *) iv, len); + kfree_skb(skb); + break; + } else { + DBG(KERN_DEBUG "%s: tun_chr_readv: rejected: %x:%x:%x:%x:%x:%x\n", + tun->dev->name, addr[0], addr[1], addr[2], + addr[3], addr[4], addr[5]); + kfree_skb(skb); + continue; + } } current->state = TASK_RUNNING; @@ -417,6 +480,12 @@ tun = netdev_priv(dev); tun->dev = dev; tun->flags = flags; + /* Be promiscuous by default to maintain previous behaviour. */ + tun->if_flags = IFF_PROMISC; + /* Generate random Ethernet address. */ + *(u16 *)tun->dev_addr = htons(0x00FF); + get_random_bytes(tun->dev_addr + sizeof(u16), 4); + memset(tun->chr_filter, 0, sizeof tun->chr_filter); tun_net_init(dev); @@ -457,13 +526,16 @@ unsigned int cmd, unsigned long arg) { struct tun_struct *tun = file->private_data; + void __user* argp = (void __user*)arg; + struct ifreq ifr; + + if (cmd == TUNSETIFF || _IOC_TYPE(cmd) == 0x89) + if (copy_from_user(&ifr, argp, sizeof ifr)) + return -EFAULT; if (cmd == TUNSETIFF && !tun) { - struct ifreq ifr; int err; - if (copy_from_user(&ifr, (void __user *)arg, sizeof(ifr))) - return -EFAULT; ifr.ifr_name[IFNAMSIZ-1] = '\0'; rtnl_lock(); @@ -473,7 +545,7 @@ if (err) return err; - if (copy_to_user((void __user *)arg, &ifr, sizeof(ifr))) + if (copy_to_user(argp, &ifr, sizeof(ifr))) return -EFAULT; return 0; } @@ -519,6 +591,61 @@ break; #endif + case SIOCGIFFLAGS: + ifr.ifr_flags = tun->if_flags; + if (copy_to_user( argp, &ifr, sizeof ifr)) + return -EFAULT; + return 0; + + case SIOCSIFFLAGS: + /** Set the character device's interface flags. Currently only + * IFF_PROMISC and IFF_ALLMULTI are used. */ + tun->if_flags = ifr.ifr_flags; + DBG(KERN_INFO "%s: interface flags 0x%lx\n", + tun->dev->name, tun->if_flags); + return 0; + + case SIOCGIFHWADDR: + memcpy(ifr.ifr_hwaddr.sa_data, tun->dev_addr, + min(sizeof ifr.ifr_hwaddr.sa_data, sizeof tun->dev_addr)); + if (copy_to_user( argp, &ifr, sizeof ifr)) + return -EFAULT; + return 0; + + case SIOCSIFHWADDR: + /** Set the character device's hardware address. This is used when + * filtering packets being sent from the network device to the character + * device. */ + memcpy(tun->dev_addr, ifr.ifr_hwaddr.sa_data, + min(sizeof ifr.ifr_hwaddr.sa_data, sizeof tun->dev_addr)); + DBG(KERN_DEBUG "%s: set hardware address: %x:%x:%x:%x:%x:%x\n", + tun->dev->name, + tun->dev_addr[0], tun->dev_addr[1], tun->dev_addr[2], + tun->dev_addr[3], tun->dev_addr[4], tun->dev_addr[5]); + return 0; + + case SIOCADDMULTI: + /** Add the specified group to the character device's multicast filter + * list. */ + add_multi(tun->chr_filter, ifr.ifr_hwaddr.sa_data); + DBG(KERN_DEBUG "%s: add multi: %x:%x:%x:%x:%x:%x\n", + tun->dev->name, + (u8)ifr.ifr_hwaddr.sa_data[0], (u8)ifr.ifr_hwaddr.sa_data[1], + (u8)ifr.ifr_hwaddr.sa_data[2], (u8)ifr.ifr_hwaddr.sa_data[3], + (u8)ifr.ifr_hwaddr.sa_data[4], (u8)ifr.ifr_hwaddr.sa_data[5]); + return 0; + + case SIOCDELMULTI: + /** Remove the specified group from the character device's multicast + * filter list. */ + del_multi(tun->chr_filter, ifr.ifr_hwaddr.sa_data); + DBG(KERN_DEBUG "%s: del multi: %x:%x:%x:%x:%x:%x\n", + tun->dev->name, + (u8)ifr.ifr_hwaddr.sa_data[0], (u8)ifr.ifr_hwaddr.sa_data[1], + (u8)ifr.ifr_hwaddr.sa_data[2], (u8)ifr.ifr_hwaddr.sa_data[3], + (u8)ifr.ifr_hwaddr.sa_data[4], (u8)ifr.ifr_hwaddr.sa_data[5]); + return 0; + default: return -EINVAL; }; diff -ur linux-2.6.8.1.orig/include/linux/if_tun.h linux-2.6.8.1/include/linux/if_tun.h --- linux-2.6.8.1.orig/include/linux/if_tun.h 2004-08-14 03:55:09.000000000 -0700 +++ linux-2.6.8.1/include/linux/if_tun.h 2004-11-25 16:47:31.000000000 -0800 @@ -45,6 +45,11 @@ struct fasync_struct *fasync; + unsigned long if_flags; + u8 dev_addr[ETH_ALEN]; + u32 chr_filter[2]; + u32 net_filter[2]; + #ifdef TUN_DEBUG int debug; #endif From jketreno@linux.intel.com Wed Dec 1 17:35:30 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 01 Dec 2004 17:35:45 -0800 (PST) Received: from orsfmr003.jf.intel.com (fmr18.intel.com [134.134.136.17]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB21ZQbP012623 for ; Wed, 1 Dec 2004 17:35:30 -0800 Received: from talaria.jf.intel.com (talaria.jf.intel.com [10.7.209.7]) by orsfmr003.jf.intel.com (8.12.10/8.12.10/d: major-outer.mc,v 1.1 2004/09/17 17:50:56 root Exp $) with ESMTP id iB21Z0xK001499 for ; Thu, 2 Dec 2004 01:35:00 GMT Received: from linux.intel.com (vpnfm001-139-dhcp-client.fm.intel.com [10.19.13.139]) by talaria.jf.intel.com (8.12.9-20030918-01/8.12.9/d: major-inner.mc,v 1.11 2004/07/29 22:51:53 root Exp $) with ESMTP id iB21Q9uc018759 for ; Thu, 2 Dec 2004 01:26:09 GMT Message-ID: <41AE7143.80505@linux.intel.com> Date: Wed, 01 Dec 2004 19:34:59 -0600 From: James Ketrenos User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6b) Gecko/20031205 Thunderbird/0.4 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Netdev Subject: Steps for netdev-2.6 inclusion? Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.31 (www . roaringpenguin . com / mimedefang) X-archive-position: 12377 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jketreno@linux.intel.com Precedence: bulk X-list: netdev Ok, its been a long time coming, but it appears the ipw* wireless drivers are to the point where being more proactive at getting them into the kernel is appropriate (at least based on the frequency of emails I'm getting of 'why isn't this in mainline?') So, what would be the set of steps required to get a version in the queue for inclusion? The drivers we have are the ipw2100 (supporting the Intel PRO/Wireless 2100 Network Connection adapter) and ipw2200 (supporting the Intel PRO/Wireless 2200BG and 2915ABG Network Connection adapters). There is also a shared generic ieee 802.11 stack (ieee80211*.ko) supporting 802.11b, g, and a for BSS and IBSS modes. The ipw2100 recently was stamped 1.0.0, which means we've put it through a validation and stabalization phase. As we want to try and ensure end users don't have to spend all night fighting with their wireless connection just because they've updated the kernel, we've adopted the following version numbering scheme for the ipw* drivers in the form of of x.y.z, where: .z increases from snapshot to snapshot (pushed out as tarballs on SourceForge) .y increases (and sets .z to 0) when a snapshot has gone through a regression validation cycle. .x increases if there are significant functionality changes to the driver. The idea is to then only have x.y.0 (stable/tested) versions go out for wider distribution (kernel inclusion). For those that are curious, the tip of development for the ipw2100, ipw2200, and ieee80211 stack is available at bk://ipw.bkbits.net/ipw-2.6. Given what I described above, would it be most appropriate to create a ipw-2.6-stable bk tree with the parent as netdev-2.6 that we put the stable versions of the ipw* drivers into, and then request that be pulled? The ipw-2.6 tree could then continue to represent the development tip. I've searched around for some BKMs on this, but haven't found a whole lot. The ipw2100 1.0.0 snapshot (and newer versions) can be found at http://ipw2100.sf.net/#downloads. The latest ipw2200 snapshot (0.15) is available at http://ipw2200.sf.net/#downloads. Also, we have a Bugzilla database setup for the above drivers at http://bughost.org for those that are curious about current remaining issues, etc. Thanks, James From horms@koto.vergenet.net Wed Dec 1 20:31:46 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 01 Dec 2004 20:31:54 -0800 (PST) Received: from koto.vergenet.net (koto.vergenet.net [210.128.90.7]) by oss.sgi.com (8.13.0/8.13.0) with SMTP id iB24VjpX020259 for ; Wed, 1 Dec 2004 20:31:45 -0800 Received: (qmail 26081 invoked by uid 7100); 2 Dec 2004 04:15:29 -0000 Date: Thu, 2 Dec 2004 13:07:20 +0900 From: Horms To: Wensong Zhang Cc: "David S. Miller" , netdev@oss.sgi.com Subject: Re: [PATCH] [IPVS] add a sysctl variable to expire quiescent template Message-ID: <20041202040717.GF32190@verge.net.au> Mail-Followup-To: Wensong Zhang , "David S. Miller" , netdev@oss.sgi.com References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Cluestick: seven User-Agent: Mutt/1.5.6+20040907i X-archive-position: 12378 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: horms@verge.net.au Precedence: bulk X-list: netdev On Thu, Dec 02, 2004 at 12:48:26AM +0800, Wensong Zhang wrote: > > > Hi Dave, > > Here is the patch from Horms to add a sysctl > variable to expire quiescent templat. Please check and apply them to > kernel 2.4 and 2.6 respectively. > > Thanks, > > Wensong I can do this too, just in case you need it. Signed-off-by: Horms > # This is a BitKeeper generated diff -Nru style patch. > # > # ChangeSet > # 2004/12/02 00:02:48+08:00 wensong@linux-vs.org > # [IPVS] add a sysctl variable to expire quiescent template > # > # The patch is from Horms > # > # net/ipv4/ipvs/ip_vs_ctl.c > # 2004/12/02 00:02:38+08:00 wensong@linux-vs.org +4 -0 > # set sysctl_ip_vs_expire_quiescent_template > # > # net/ipv4/ipvs/ip_vs_conn.c > # 2004/12/02 00:02:37+08:00 wensong@linux-vs.org +3 -1 > # don't use quiescent template if the expire_quiescent_template is enabled > # > # include/net/ip_vs.h > # 2004/12/02 00:02:37+08:00 wensong@linux-vs.org +2 -0 > # add the sysctl_ip_vs_expire_quiescent_template > # > diff -Nru a/include/net/ip_vs.h b/include/net/ip_vs.h > --- a/include/net/ip_vs.h 2004-12-02 00:16:36 +08:00 > +++ b/include/net/ip_vs.h 2004-12-02 00:16:36 +08:00 > @@ -317,6 +317,7 @@ > NET_IPV4_VS_EXPIRE_NODEST_CONN=23, > NET_IPV4_VS_SYNC_THRESHOLD=24, > NET_IPV4_VS_NAT_ICMP_SEND=25, > + NET_IPV4_VS_EXPIRE_QUIESCENT_TEMPLATE=26, > NET_IPV4_VS_LAST > }; > > @@ -700,6 +701,7 @@ > */ > extern int sysctl_ip_vs_cache_bypass; > extern int sysctl_ip_vs_expire_nodest_conn; > +extern int sysctl_ip_vs_expire_quiescent_template; > extern int sysctl_ip_vs_sync_threshold; > extern int sysctl_ip_vs_nat_icmp_send; > extern struct ip_vs_stats ip_vs_stats; > diff -Nru a/net/ipv4/ipvs/ip_vs_conn.c b/net/ipv4/ipvs/ip_vs_conn.c > --- a/net/ipv4/ipvs/ip_vs_conn.c 2004-12-02 00:16:36 +08:00 > +++ b/net/ipv4/ipvs/ip_vs_conn.c 2004-12-02 00:16:36 +08:00 > @@ -1131,7 +1131,9 @@ > * Checking the dest server status. > */ > if ((dest == NULL) || > - !(dest->flags & IP_VS_DEST_F_AVAILABLE)) { > + !(dest->flags & IP_VS_DEST_F_AVAILABLE) || > + (sysctl_ip_vs_expire_quiescent_template && > + (atomic_read(&dest->weight) == 0))) { > IP_VS_DBG(9, "check_template: dest not available for " > "protocol %s s:%u.%u.%u.%u:%d v:%u.%u.%u.%u:%d " > "-> d:%u.%u.%u.%u:%d\n", > diff -Nru a/net/ipv4/ipvs/ip_vs_ctl.c b/net/ipv4/ipvs/ip_vs_ctl.c > --- a/net/ipv4/ipvs/ip_vs_ctl.c 2004-12-02 00:16:36 +08:00 > +++ b/net/ipv4/ipvs/ip_vs_ctl.c 2004-12-02 00:16:36 +08:00 > @@ -74,6 +74,7 @@ > static int sysctl_ip_vs_am_droprate = 10; > int sysctl_ip_vs_cache_bypass = 0; > int sysctl_ip_vs_expire_nodest_conn = 0; > +int sysctl_ip_vs_expire_quiescent_template = 0; > int sysctl_ip_vs_sync_threshold = 3; > int sysctl_ip_vs_nat_icmp_send = 0; > > @@ -1439,6 +1440,9 @@ > &proc_dointvec}, > {NET_IPV4_VS_NAT_ICMP_SEND, "nat_icmp_send", > &sysctl_ip_vs_nat_icmp_send, sizeof(int), 0644, NULL, > + &proc_dointvec}, > + {NET_IPV4_VS_EXPIRE_QUIESCENT_TEMPLATE, "expire_quiescent_template", > + &sysctl_ip_vs_expire_quiescent_template, sizeof(int), 0644, NULL, > &proc_dointvec}, > {0}}, > {{NET_IPV4_VS, "vs", NULL, 0, 0555, ipv4_vs_table.vs_vars}, > # This is a BitKeeper generated diff -Nru style patch. > # > # ChangeSet > # 2004/12/02 00:42:15+08:00 wensong@linux-vs.org > # [IPVS] add a sysctl variable to expire quiescent template > # > # The patch is from Horms > # > # net/ipv4/ipvs/ip_vs_ctl.c > # 2004/12/02 00:41:56+08:00 wensong@linux-vs.org +20 -11 > # set the sysctl_ip_vs_expire_quiescent_template > # > # net/ipv4/ipvs/ip_vs_conn.c > # 2004/12/02 00:41:56+08:00 wensong@linux-vs.org +3 -1 > # don't use quiescent template if the expire_quiescent_template is enabled > # > # include/net/ip_vs.h > # 2004/12/02 00:41:56+08:00 wensong@linux-vs.org +2 -0 > # add the sysctl_ip_vs_expire_quiescent_template prototype > # > diff -Nru a/include/net/ip_vs.h b/include/net/ip_vs.h > --- a/include/net/ip_vs.h 2004-12-02 00:43:14 +08:00 > +++ b/include/net/ip_vs.h 2004-12-02 00:43:14 +08:00 > @@ -358,6 +358,7 @@ > NET_IPV4_VS_EXPIRE_NODEST_CONN=23, > NET_IPV4_VS_SYNC_THRESHOLD=24, > NET_IPV4_VS_NAT_ICMP_SEND=25, > + NET_IPV4_VS_EXPIRE_QUIESCENT_TEMPLATE=26, > NET_IPV4_VS_LAST > }; > > @@ -879,6 +880,7 @@ > */ > extern int sysctl_ip_vs_cache_bypass; > extern int sysctl_ip_vs_expire_nodest_conn; > +extern int sysctl_ip_vs_expire_quiescent_template; > extern int sysctl_ip_vs_sync_threshold[2]; > extern int sysctl_ip_vs_nat_icmp_send; > extern struct ip_vs_stats ip_vs_stats; > diff -Nru a/net/ipv4/ipvs/ip_vs_conn.c b/net/ipv4/ipvs/ip_vs_conn.c > --- a/net/ipv4/ipvs/ip_vs_conn.c 2004-12-02 00:43:14 +08:00 > +++ b/net/ipv4/ipvs/ip_vs_conn.c 2004-12-02 00:43:14 +08:00 > @@ -453,7 +453,9 @@ > * Checking the dest server status. > */ > if ((dest == NULL) || > - !(dest->flags & IP_VS_DEST_F_AVAILABLE)) { > + !(dest->flags & IP_VS_DEST_F_AVAILABLE) || > + (sysctl_ip_vs_expire_quiescent_template && > + (atomic_read(&dest->weight) == 0))) { > IP_VS_DBG(9, "check_template: dest not available for " > "protocol %s s:%u.%u.%u.%u:%d v:%u.%u.%u.%u:%d " > "-> d:%u.%u.%u.%u:%d\n", > diff -Nru a/net/ipv4/ipvs/ip_vs_ctl.c b/net/ipv4/ipvs/ip_vs_ctl.c > --- a/net/ipv4/ipvs/ip_vs_ctl.c 2004-12-02 00:43:14 +08:00 > +++ b/net/ipv4/ipvs/ip_vs_ctl.c 2004-12-02 00:43:14 +08:00 > @@ -75,6 +75,7 @@ > static int sysctl_ip_vs_am_droprate = 10; > int sysctl_ip_vs_cache_bypass = 0; > int sysctl_ip_vs_expire_nodest_conn = 0; > +int sysctl_ip_vs_expire_quiescent_template = 0; > int sysctl_ip_vs_sync_threshold[2] = { 3, 50 }; > int sysctl_ip_vs_nat_icmp_send = 0; > > @@ -1447,9 +1448,9 @@ > { > .ctl_name = NET_IPV4_VS_TO_ES, > .procname = "timeout_established", > - .data = &vs_timeout_table_dos.timeout[IP_VS_S_ESTABLISHED], > + .data = &vs_timeout_table_dos.timeout[IP_VS_S_ESTABLISHED], > .maxlen = sizeof(int), > - .mode = 0644, > + .mode = 0644, > .proc_handler = &proc_dointvec_jiffies, > }, > { > @@ -1457,7 +1458,7 @@ > .procname = "timeout_synsent", > .data = &vs_timeout_table_dos.timeout[IP_VS_S_SYN_SENT], > .maxlen = sizeof(int), > - .mode = 0644, > + .mode = 0644, > .proc_handler = &proc_dointvec_jiffies, > }, > { > @@ -1465,7 +1466,7 @@ > .procname = "timeout_synrecv", > .data = &vs_timeout_table_dos.timeout[IP_VS_S_SYN_RECV], > .maxlen = sizeof(int), > - .mode = 0644, > + .mode = 0644, > .proc_handler = &proc_dointvec_jiffies, > }, > { > @@ -1473,7 +1474,7 @@ > .procname = "timeout_finwait", > .data = &vs_timeout_table_dos.timeout[IP_VS_S_FIN_WAIT], > .maxlen = sizeof(int), > - .mode = 0644, > + .mode = 0644, > .proc_handler = &proc_dointvec_jiffies, > }, > { > @@ -1489,7 +1490,7 @@ > .procname = "timeout_close", > .data = &vs_timeout_table_dos.timeout[IP_VS_S_CLOSE], > .maxlen = sizeof(int), > - .mode = 0644, > + .mode = 0644, > .proc_handler = &proc_dointvec_jiffies, > }, > { > @@ -1497,7 +1498,7 @@ > .procname = "timeout_closewait", > .data = &vs_timeout_table_dos.timeout[IP_VS_S_CLOSE_WAIT], > .maxlen = sizeof(int), > - .mode = 0644, > + .mode = 0644, > .proc_handler = &proc_dointvec_jiffies, > }, > { > @@ -1505,7 +1506,7 @@ > .procname = "timeout_lastack", > .data = &vs_timeout_table_dos.timeout[IP_VS_S_LAST_ACK], > .maxlen = sizeof(int), > - .mode = 0644, > + .mode = 0644, > .proc_handler = &proc_dointvec_jiffies, > }, > { > @@ -1513,7 +1514,7 @@ > .procname = "timeout_listen", > .data = &vs_timeout_table_dos.timeout[IP_VS_S_LISTEN], > .maxlen = sizeof(int), > - .mode = 0644, > + .mode = 0644, > .proc_handler = &proc_dointvec_jiffies, > }, > { > @@ -1521,7 +1522,7 @@ > .procname = "timeout_synack", > .data = &vs_timeout_table_dos.timeout[IP_VS_S_SYNACK], > .maxlen = sizeof(int), > - .mode = 0644, > + .mode = 0644, > .proc_handler = &proc_dointvec_jiffies, > }, > { > @@ -1529,7 +1530,7 @@ > .procname = "timeout_udp", > .data = &vs_timeout_table_dos.timeout[IP_VS_S_UDP], > .maxlen = sizeof(int), > - .mode = 0644, > + .mode = 0644, > .proc_handler = &proc_dointvec_jiffies, > }, > { > @@ -1553,6 +1554,14 @@ > .ctl_name = NET_IPV4_VS_EXPIRE_NODEST_CONN, > .procname = "expire_nodest_conn", > .data = &sysctl_ip_vs_expire_nodest_conn, > + .maxlen = sizeof(int), > + .mode = 0644, > + .proc_handler = &proc_dointvec, > + }, > + { > + .ctl_name = NET_IPV4_VS_EXPIRE_QUIESCENT_TEMPLATE, > + .procname = "expire_quiescent_template", > + .data = &sysctl_ip_vs_expire_quiescent_template, > .maxlen = sizeof(int), > .mode = 0644, > .proc_handler = &proc_dointvec, -- Horms From sfeldma@pobox.com Wed Dec 1 22:11:17 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 01 Dec 2004 22:11:23 -0800 (PST) Received: from orb.pobox.com (orb.pobox.com [207.8.226.5]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB26BGK4022289 for ; Wed, 1 Dec 2004 22:11:17 -0800 Received: from orb (localhost [127.0.0.1]) by orb.pobox.com (Postfix) with ESMTP id 2F5622FA235; Thu, 2 Dec 2004 01:10:55 -0500 (EST) Received: from [192.168.2.72] (adsl-68-127-20-190.dsl.pltn13.pacbell.net [68.127.20.190]) by orb.sasl.smtp.pobox.com (Postfix) with ESMTP id 585752F9F7E; Thu, 2 Dec 2004 01:10:45 -0500 (EST) Subject: Re: [E1000-devel] Transmission limit From: Scott Feldman Reply-To: sfeldma@pobox.com To: Lennert Buytenhek Cc: jamal , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com In-Reply-To: <20041201213550.GF14470@xi.wantstofly.org> References: <1101484740.24742.213.camel@mellia.lipar.polito.it> <41A76085.7000105@draigBrady.com> <1101499285.1079.45.camel@jzny.localdomain> <16811.8052.678955.795327@robur.slu.se> <1101821501.1043.43.camel@jzny.localdomain> <20041130134600.GA31515@xi.wantstofly.org> <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <20041201182943.GA14470@xi.wantstofly.org> <20041201213550.GF14470@xi.wantstofly.org> Content-Type: text/plain Message-Id: <1101967983.4782.9.camel@localhost.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 (1.4.6-2) Date: Wed, 01 Dec 2004 22:13:33 -0800 Content-Transfer-Encoding: 7bit X-archive-position: 12379 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: sfeldma@pobox.com Precedence: bulk X-list: netdev On Wed, 2004-12-01 at 13:35, Lennert Buytenhek wrote: > Pretty graph attached. From ~220B packets or so it does wire speed, but > there's still an odd drop in performance around 256B packets (which is > also there without your patch.) From 350B packets or so, performance is > identical with or without your patch (wire speed.) Seems this is helping PCI nics but not PCI-X. I was using PCI 32/33. Can't explain the dip around 256B. > So. Do you have any other good plans perhaps? :) Idea#1 Is the write of TDT causing interference with DMA transactions? In addition to my patch, what happens if you bump the Tx tail every n packets, where n is like 16 or 32 or 64? if((i % 16) == 0) E1000_REG_WRITE(&adapter->hw, TDT, i); This might piss the NETDEV timer off if the send count isn't a multiple of n, so you might want to disable netdev->tx_timeout. Idea#2 The Ultimate: queue up 4096 packets and then write TDT once to send all 4096 in one shot. Well, maybe a few less that 4096 so we don't wrap the ring. How about pkt_size = 4000? Take my patch and change the timer call in e1000_xmit_frame from jiffies + 1 to jiffies + HZ This will schedule the cleanup of the skbs 1 second after the first queue, so we shouldn't be doing any cleanup while the 4000 packets are DMA'ed. Oh, and change the tail write to if((i % 4000) == 0) E1000_REG_WRITE(&adapter->hw, TDT, i); Of course you'll need to close/open the driver after each run. Idea#3 http://www.mail-archive.com/freebsd-net@freebsd.org/msg10826.html Set TXDMAC to 0 in e1000_configure_tx. > > Once or twice it went into a state where it started spitting out these > > kinds of messages and never recovered: > > > > Dec 1 19:13:18 phi kernel: NETDEV WATCHDOG: eth1: transmit timed out > > [...] > > Dec 1 19:13:31 phi kernel: NETDEV WATCHDOG: eth1: transmit timed out > > [...] > > Dec 1 19:13:43 phi kernel: NETDEV WATCHDOG: eth1: transmit timed out > > Didn't see this happen anymore. (ifconfig down and then up recovered it > both times I saw it happen.) Well, it's probably not a HW bug that's causing the reset; it's probably some bug with my patch. -scott From jgarzik@pobox.com Wed Dec 1 22:19:07 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 01 Dec 2004 22:19:11 -0800 (PST) Received: from www.linux.org.uk (IDENT:93@parcelfarce.linux.theplanet.co.uk [195.92.249.252]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB26J6O8022740 for ; Wed, 1 Dec 2004 22:19:06 -0800 Received: from rdu74-155-169.nc.rr.com ([24.74.155.169] helo=[10.10.10.88]) by www.linux.org.uk with asmtp (TLSv1:AES256-SHA:256) (Exim 4.33) id 1CZkId-00016k-W9; Thu, 02 Dec 2004 06:18:44 +0000 Message-ID: <41AEB3B8.2000406@pobox.com> Date: Thu, 02 Dec 2004 01:18:32 -0500 From: Jeff Garzik User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.3) Gecko/20040922 X-Accept-Language: en-us, en MIME-Version: 1.0 To: James Ketrenos CC: Netdev Subject: Re: Steps for netdev-2.6 inclusion? References: <41AE7143.80505@linux.intel.com> In-Reply-To: <41AE7143.80505@linux.intel.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 12380 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jgarzik@pobox.com Precedence: bulk X-list: netdev It's fairly easy, just email me and netdev the patch for inclusion, and it'll get reviewed. Once review issues are addressed, I'll merge it immediately, which causes it to be automatically propagated to Andrew Morton's -mm tree for testing. Once consensus agrees that we can push this + HostAP upstream, that's an easy 10-minute task. One potential showstopper is firmware crapola: I'm concerned about a situation where we have drivers in the kernel, but the firmware must be downloaded from SourceForge or somesuch. IOW, the kernel driver as-is is useless without a differently-licensed firmware. Comments? Jeff From akpm@osdl.org Wed Dec 1 23:34:51 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 01 Dec 2004 23:34:55 -0800 (PST) Received: from mail.osdl.org (fw.osdl.org [65.172.181.6]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB27YpGI024362 for ; Wed, 1 Dec 2004 23:34:51 -0800 Received: from bix (build.pdx.osdl.net [172.20.1.2]) by mail.osdl.org (8.11.6/8.11.6) with SMTP id iB27YM912409; Wed, 1 Dec 2004 23:34:22 -0800 Date: Wed, 1 Dec 2004 23:34:00 -0800 From: Andrew Morton To: Shaun Jackman Cc: netdev@oss.sgi.com Subject: Re: Multicast filtering for tun.c [PATCH] Message-Id: <20041201233400.45078efe.akpm@osdl.org> In-Reply-To: <7f45d939041201140329d0273f@mail.gmail.com> References: <7f45d9390411251715138b35d0@mail.gmail.com> <20041127011006.03232bb6.akpm@osdl.org> <7f45d939041201140329d0273f@mail.gmail.com> X-Mailer: Sylpheed version 0.9.7 (GTK+ 1.2.10; i386-redhat-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-archive-position: 12381 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: akpm@osdl.org Precedence: bulk X-list: netdev Shaun Jackman wrote: > > This patch adds multicast filtering to the TUN network driver, for > packets being sent from the network device to the character device. It > applies against the 2.6.8.1 kernel tree. > > ... Minor points: > diff -ur linux-2.6.8.1.orig/drivers/net/tun.c linux-2.6.8.1/drivers/net/tun.c > --- linux-2.6.8.1.orig/drivers/net/tun.c 2004-08-14 03:55:23.000000000 -0700 > +++ linux-2.6.8.1/drivers/net/tun.c 2004-11-25 17:00:22.000000000 -0800 > @@ -41,6 +41,7 @@ > #include > #include > #include > +#include You're sure this shouldn't be using crc-ccitt? > +del_multi(u32* filter, const u8* addr) del_multi(u32 *filter, const u8 *addr) would be a more typical layout. > static struct net_device_stats *tun_net_stats(struct net_device *dev) > @@ -301,6 +333,10 @@ > > add_wait_queue(&tun->read_wait, &wait); > while (len) { > + const u8 ones[ ETH_ALEN] = { 0xff, 0xff, 0xff, 0xff, 0xff, 0xff }; The space after the `[' is gratuitous. This array will be allocated onthe stack and populated by hand each time this function is called. It should be made static. > + u8 addr[ ETH_ALEN]; extraneous space. > + memcpy(addr, skb->data, min(sizeof addr, skb->len)); We normally put parentheses around the argument of the sizeof operator, for no particular reason ;) > + if (copy_from_user(&ifr, argp, sizeof ifr)) Ditto > + case SIOCGIFFLAGS: > + ifr.ifr_flags = tun->if_flags; > + if (copy_to_user( argp, &ifr, sizeof ifr)) extraneous space. > + if (copy_to_user( argp, &ifr, sizeof ifr)) ditto > + DBG(KERN_DEBUG "%s: add multi: %x:%x:%x:%x:%x:%x\n", > + tun->dev->name, > + (u8)ifr.ifr_hwaddr.sa_data[0], (u8)ifr.ifr_hwaddr.sa_data[1], > + (u8)ifr.ifr_hwaddr.sa_data[2], (u8)ifr.ifr_hwaddr.sa_data[3], > + (u8)ifr.ifr_hwaddr.sa_data[4], (u8)ifr.ifr_hwaddr.sa_data[5]); Why all the typecasts? > + case SIOCDELMULTI: > + /** Remove the specified group from the character device's multicast > + * filter list. */ > + del_multi(tun->chr_filter, ifr.ifr_hwaddr.sa_data); > + DBG(KERN_DEBUG "%s: del multi: %x:%x:%x:%x:%x:%x\n", > + tun->dev->name, > + (u8)ifr.ifr_hwaddr.sa_data[0], (u8)ifr.ifr_hwaddr.sa_data[1], > + (u8)ifr.ifr_hwaddr.sa_data[2], (u8)ifr.ifr_hwaddr.sa_data[3], > + (u8)ifr.ifr_hwaddr.sa_data[4], (u8)ifr.ifr_hwaddr.sa_data[5]); ditto. From cranium2003@yahoo.com Thu Dec 2 01:44:59 2004 Received: with ECARTIS (v1.0.0; list netdev); Thu, 02 Dec 2004 01:45:11 -0800 (PST) Received: from web41411.mail.yahoo.com (web41411.mail.yahoo.com [66.218.93.77]) by oss.sgi.com (8.13.0/8.13.0) with SMTP id iB29ix1n000503 for ; Thu, 2 Dec 2004 01:44:59 -0800 Received: (qmail 39134 invoked by uid 60001); 2 Dec 2004 09:44:33 -0000 Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; b=k92BT0y43RKYSYKMggOHP5GR4uBRRgWyRcOAX5JmqyP/9qc7ObXyWrE7P9ymSzL+66OC8Ent972edeO7XCBfxbGmHyDKsnRKOhZZP1xOKL0zYIpVe3+rRwO6a4AMd3PpgkzKY/dpAceYFejgq5HEi0RIiLQRnYBcF1ypdL5sjWo= ; Message-ID: <20041202094433.39132.qmail@web41411.mail.yahoo.com> Received: from [202.56.231.117] by web41411.mail.yahoo.com via HTTP; Thu, 02 Dec 2004 01:44:33 PST Date: Thu, 2 Dec 2004 01:44:33 -0800 (PST) From: cranium2003 Subject: Maximum packet size in kernel To: kernerl mail Cc: net dev MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-archive-position: 12382 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: cranium2003@yahoo.com Precedence: bulk X-list: netdev hello, I want to know how much maximum packet size is assigned by alloc_skb in kernel? and is there any extra space remains in packet allocated memory? Does it dependent on CPU cache? regards, cranium. __________________________________ Do you Yahoo!? The all-new My Yahoo! - What will yours do? http://my.yahoo.com From rl@hellgate.ch Thu Dec 2 03:02:30 2004 Received: with ECARTIS (v1.0.0; list netdev); Thu, 02 Dec 2004 03:02:36 -0800 (PST) Received: from mail2.bluewin.ch (mail2.bluewin.ch [195.186.4.73]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB2B2TXn004351 for ; Thu, 2 Dec 2004 03:02:30 -0800 Received: from k3.hellgate.ch (83.77.143.24) by mail2.bluewin.ch (Bluewin AG 7.0.031.3) id 41888440002B727B; Thu, 2 Dec 2004 11:01:17 +0000 Received: by k3.hellgate.ch (Postfix, from userid 1000) id 8BCF691F5FB; Thu, 2 Dec 2004 12:01:20 +0100 (CET) Date: Thu, 2 Dec 2004 12:01:20 +0100 From: Roger Luethi To: Lennert Buytenhek Cc: "David S. Miller" , netdev@oss.sgi.com Subject: Re: via-rhine unable to send back-to-back packets? Message-ID: <20041202110120.GA16597@k3.hellgate.ch> References: <20041129222700.GA22918@xi.wantstofly.org> <20041129172540.6b959858.davem@davemloft.net> <20041130064823.GA27872@xi.wantstofly.org> <20041130122503.0adac947.davem@davemloft.net> <20041130220644.GC29947@k3.hellgate.ch> <20041201200148.GE14470@xi.wantstofly.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20041201200148.GE14470@xi.wantstofly.org> X-Operating-System: Linux 2.6.10-rc2-bk11 on i686 X-GPG-Fingerprint: 92 F4 DC 20 57 46 7B 95 24 4E 9E E7 5A 54 DC 1B X-GPG: 1024/80E744BD wwwkeys.ch.pgp.net User-Agent: Mutt/1.5.6i X-archive-position: 12383 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: rl@hellgate.ch Precedence: bulk X-list: netdev [Thanks to A.J. from VNT for the answer which I am relaying below.] On Wed, 01 Dec 2004 21:01:48 +0100, Lennert Buytenhek wrote: > "Is the hardware capable of sending back-to-back packets (i.e. with > an inter-packet gap of no more than 96 bit times)?" => YES. All the VIA Rhine Fast Ethernet controllers follows the IEEE standard (inter-frame gap is 96 bit time) and it is hardware fixed and NOT programmable. > "Can misprogramming the chip lead to the effect that the inter-packet > gap is never less than 112 bit times?" => NO, it is fixed and neither software nor board level change could modify the inter-frame gap of VIA Rhine Fast Ethernet controller. > > Of course, you can always check if VIA's driver has the same issue. If > > it doesn't, chances are we can borrow the fix. > > Hmm, didn't know they had such a driver. Where can I find it? => in http://www.viaarena.com/ , choose "downloads" => "Drivers" => choose target OS (e.g. Fedora Core Linux) => "Ethernet (Networking/LAN)" => choose the type of your Ethernet controller => click to download the combo driver software package. From buytenh@wantstofly.org Thu Dec 2 03:09:59 2004 Received: with ECARTIS (v1.0.0; list netdev); Thu, 02 Dec 2004 03:10:04 -0800 (PST) Received: from xi.wantstofly.org (alephnull.demon.nl [212.238.201.82]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB2B9wjF004946 for ; Thu, 2 Dec 2004 03:09:59 -0800 Received: by xi.wantstofly.org (Postfix, from userid 500) id 17EA82B0ED; Thu, 2 Dec 2004 12:09:37 +0100 (MET) Date: Thu, 2 Dec 2004 12:09:37 +0100 From: Lennert Buytenhek To: Roger Luethi Cc: "David S. Miller" , netdev@oss.sgi.com Subject: Re: via-rhine unable to send back-to-back packets? Message-ID: <20041202110937.GC24069@xi.wantstofly.org> References: <20041129222700.GA22918@xi.wantstofly.org> <20041129172540.6b959858.davem@davemloft.net> <20041130064823.GA27872@xi.wantstofly.org> <20041130122503.0adac947.davem@davemloft.net> <20041130220644.GC29947@k3.hellgate.ch> <20041201200148.GE14470@xi.wantstofly.org> <20041202110120.GA16597@k3.hellgate.ch> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20041202110120.GA16597@k3.hellgate.ch> User-Agent: Mutt/1.4.1i X-archive-position: 12384 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: buytenh@wantstofly.org Precedence: bulk X-list: netdev On Thu, Dec 02, 2004 at 12:01:20PM +0100, Roger Luethi wrote: > On Wed, 01 Dec 2004 21:01:48 +0100, Lennert Buytenhek wrote: > > "Is the hardware capable of sending back-to-back packets (i.e. with > > an inter-packet gap of no more than 96 bit times)?" > > => YES. All the VIA Rhine Fast Ethernet controllers follows the IEEE > standard (inter-frame gap is 96 bit time) and it is hardware fixed and > NOT programmable. As far as I know, the IEEE standard only mandates a certain minimum inter-frame gap. Thanks for the information, I'll have a look at the VIA driver. --L From jgarzik@pobox.com Thu Dec 2 03:25:43 2004 Received: with ECARTIS (v1.0.0; list netdev); Thu, 02 Dec 2004 03:25:49 -0800 (PST) Received: from www.linux.org.uk (IDENT:93@parcelfarce.linux.theplanet.co.uk [195.92.249.252]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB2BPgmF005571 for ; Thu, 2 Dec 2004 03:25:43 -0800 Received: from rdu74-155-169.nc.rr.com ([24.74.155.169] helo=[10.10.10.88]) by www.linux.org.uk with asmtp (TLSv1:AES256-SHA:256) (Exim 4.33) id 1CZp5N-00027Y-1w; Thu, 02 Dec 2004 11:25:21 +0000 Message-ID: <41AEFB95.8000100@pobox.com> Date: Thu, 02 Dec 2004 06:25:09 -0500 From: Jeff Garzik User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.3) Gecko/20040922 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Stephen Hemminger CC: "David S. Miller" , netdev@oss.sgi.com Subject: Re: [PATCH] b44: allow ethtool get_settings when down References: <20041129094523.3185c64c@zqx3.pdx.osdl.net> In-Reply-To: <20041129094523.3185c64c@zqx3.pdx.osdl.net> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 12385 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jgarzik@pobox.com Precedence: bulk X-list: netdev Stephen Hemminger wrote: > The FC and Suse startup scripts use ethtool to check for link present. This has > problems on my laptop with Broadcom because it quieries settings before > bringing link up. The problem is driver returns EAGAIN when queried for > settings but not up. Just go ahead and return values anyway, the supported and link > state values will be correct, speed will end up being 10BaseT/Half which is a > reasonable default. > > Signed-off-by: Stephen Hemminger > > diff -Nru a/drivers/net/b44.c b/drivers/net/b44.c > --- a/drivers/net/b44.c 2004-11-29 09:41:27 -08:00 > +++ b/drivers/net/b44.c 2004-11-29 09:41:27 -08:00 > @@ -1487,8 +1487,6 @@ > { > struct b44 *bp = netdev_priv(dev); > > - if (!(bp->flags & B44_FLAG_INIT_COMPLETE)) > - return -EAGAIN; > cmd->supported = (SUPPORTED_Autoneg); > cmd->supported |= (SUPPORTED_100baseT_Half | > SUPPORTED_100baseT_Full | I'm not so sure about this one... This sounds like working around stupid userland in the kernel? Jeff From mellia@prezzemolo.polito.it Thu Dec 2 05:40:57 2004 Received: with ECARTIS (v1.0.0; list netdev); Thu, 02 Dec 2004 05:41:04 -0800 (PST) Received: from prezzemolo.polito.it (IDENT:root@prezzemolo.polito.it [130.192.9.131]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB2DeuAB012945 for ; Thu, 2 Dec 2004 05:40:57 -0800 Received: from mellia.lipar.polito.it ([192.168.85.105]) by prezzemolo.polito.it (8.12.10/8.12.10) with ESMTP id iB2DdWdW015837; Thu, 2 Dec 2004 14:39:35 +0100 Subject: Re: [E1000-devel] Transmission limit From: Marco Mellia Reply-To: mellia@prezzemolo.polito.it To: hadi@cyberus.ca Cc: mellia@prezzemolo.polito.it, Lennert Buytenhek , Harald Welte , P@draigBrady.com, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com In-Reply-To: <1101903944.1042.29.camel@jzny.localdomain> References: <1101467291.24742.70.camel@mellia.lipar.polito.it> <41A73826.3000109@draigBrady.com> <1101483081.24742.174.camel@mellia.lipar.polito.it> <20041127092503.GA12592@sunbeam.de.gnumonks.org> <1101718412.14930.46.camel@verza.polito.it> <20041129145028.GC18788@xi.wantstofly.org> <1101804146.11111.23.camel@mellia.lipar.polito.it> <1101903944.1042.29.camel@jzny.localdomain> Content-Type: text/plain Organization: Message-Id: <1101994772.18491.16.camel@mellia.lipar.polito.it> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 (1.2.2-5) Date: 02 Dec 2004 14:39:32 +0100 Content-Transfer-Encoding: 7bit X-TLC-MailScanner-Information: Please contact the ISP for more information X-TLC-MailScanner: Found to be clean X-TLC-MailScanner-SpamCheck: not spam, SpamAssassin (score=-4.785, required 5.5, AWL 0.12, BAYES_00 -4.90) X-archive-position: 12386 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: mellia@prezzemolo.polito.it Precedence: bulk X-list: netdev > > > > We are thinking of sending packet in "bursts" instead of single > > > > transfers. The only problem is to let the NIC know that there are more > > > > than a packet in a burst... > > > > > > Jamal implemented exactly this for e1000 already, he might be persuaded > > > into posting his patch here. Jamal? :) > > > > I guess that saying that we are _very_ interested in this might help. > > :-) > > We can offer as "beta-testers" as well... > > Sorry missed this (I wasnt CCed so it went to a low priority queue which > i read on a best effort basis). > Let me clean up the patches a little bit this weekend. The patch is at > least 4 months old; latest reincarnation was due to issue1 on my SUCON > presentation. Would a patch against latest 2.6.x bitkeeper (whatever it > is this weekend) be fine? If you are in a rush and dont mind a little > ugliness then i will pass them as is. > We'll be glad to spend some time trying this out. Please, we are not very confortable with the linux bitkeeper maintenance method. Can we ask you to provide us a patch to a standard kernel/driver (whatever you prefer...)? Also a complete source sub-tree would be ok ;-) > BTW, Scott posted a interesting patch yesterday, you may wanna give that > a shot as well. We're trying that out right now... (which means, that in a couple of days, we'll try it ;-)) Thanks a lot. -- Ciao, /\/\/\rco +-----------------------------------+ | Marco Mellia - Assistant Professor| | Tel: 39-011-2276-608 | | Tel: 39-011-564-4173 | | Cel: 39-340-9674888 | /"\ .. . . . . . . . . . . . . | Politecnico di Torino | \ / . ASCII Ribbon Campaign . | Corso Duca degli Abruzzi 24 | X .- NO HTML/RTF in e-mail . | Torino - 10129 - Italy | / \ .- NO Word docs in e-mail. | http://www1.tlc.polito.it/mellia | .. . . . . . . . . . . . . +-----------------------------------+ The box said "Requires Windows 95 or Better." So I installed Linux. From wensong@linux-vs.org Thu Dec 2 06:53:15 2004 Received: with ECARTIS (v1.0.0; list netdev); Thu, 02 Dec 2004 06:53:22 -0800 (PST) Received: from lb1.ctrip.com ([218.244.111.2]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB2ErDuh020118 for ; Thu, 2 Dec 2004 06:53:15 -0800 Received: from penguin.linux-vs.org ([61.149.157.27]) by lb1.ctrip.com (8.12.10/8.12.10) with ESMTP id iB2EpwMh015802 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Thu, 2 Dec 2004 22:52:06 +0800 Received: from penguin.linux-vs.org (localhost.localdomain [127.0.0.1]) by penguin.linux-vs.org (8.12.8/8.12.8) with ESMTP id iB2EpiPX001140; Thu, 2 Dec 2004 22:51:44 +0800 Received: from localhost (wensong@localhost) by penguin.linux-vs.org (8.12.8/8.12.8/Submit) with ESMTP id iB2EpZmc001136; Thu, 2 Dec 2004 22:51:40 +0800 X-Authentication-Warning: penguin.linux-vs.org: wensong owned process doing -bs Date: Thu, 2 Dec 2004 22:51:34 +0800 (CST) From: Wensong Zhang To: Horms cc: "David S. Miller" , netdev@oss.sgi.com Subject: Re: [PATCH] [IPVS] add a sysctl variable to expire quiescent template In-Reply-To: <20041202040717.GF32190@verge.net.au> Message-ID: References: <20041202040717.GF32190@verge.net.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-archive-position: 12387 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: wensong@linux-vs.org Precedence: bulk X-list: netdev On Thu, 2 Dec 2004, Horms wrote: > On Thu, Dec 02, 2004 at 12:48:26AM +0800, Wensong Zhang wrote: >> >> >> Hi Dave, >> >> Here is the patch from Horms to add a sysctl >> variable to expire quiescent templat. Please check and apply them to >> kernel 2.4 and 2.6 respectively. >> >> Thanks, >> >> Wensong > > I can do this too, just in case you need it. > > Signed-off-by: Horms > Sure. :) I will ask you to make the patches for kernel 2.4 and 2.6 next time, and I just need to verify and test them. :) Cheers, Wensong From mellia@prezzemolo.polito.it Thu Dec 2 09:26:09 2004 Received: with ECARTIS (v1.0.0; list netdev); Thu, 02 Dec 2004 09:26:16 -0800 (PST) Received: from prezzemolo.polito.it (IDENT:root@prezzemolo.polito.it [130.192.9.131]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB2HQ8oi027479 for ; Thu, 2 Dec 2004 09:26:09 -0800 Received: from mellia.lipar.polito.it ([192.168.85.105]) by prezzemolo.polito.it (8.12.10/8.12.10) with ESMTP id iB2HOYdW032567; Thu, 2 Dec 2004 18:24:49 +0100 Subject: Re: [E1000-devel] Transmission limit From: Marco Mellia Reply-To: mellia@prezzemolo.polito.it To: hadi@cyberus.ca Cc: mellia@prezzemolo.polito.it, P@draigBrady.com, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com In-Reply-To: <1101822364.1044.60.camel@jzny.localdomain> References: <1101467291.24742.70.camel@mellia.lipar.polito.it> <41A73826.3000109@draigBrady.com> <1101483081.24742.174.camel@mellia.lipar.polito.it> <1101498963.1076.39.camel@jzny.localdomain> <1101738118.14930.142.camel@verza.polito.it> <1101822364.1044.60.camel@jzny.localdomain> Content-Type: text/plain Organization: Message-Id: <1102008274.19646.14.camel@mellia.lipar.polito.it> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 (1.2.2-5) Date: 02 Dec 2004 18:24:34 +0100 Content-Transfer-Encoding: 7bit X-TLC-MailScanner-Information: Please contact the ISP for more information X-TLC-MailScanner: Found to be clean X-TLC-MailScanner-SpamCheck: not spam, SpamAssassin (score=-4.788, required 5.5, AWL 0.11, BAYES_00 -4.90) X-archive-position: 12388 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: mellia@prezzemolo.polito.it Precedence: bulk X-list: netdev > > In our experiments, we modified the kernel to drop packets just after > > receiving them. skb are just deallocated (using standerd kernel > > routines, i.e., no recycling is used). Logically, that happen when the > > netif_rx() is called. > > > > Now, we have three cases > > 1) just mofify the netif_rx() to drop packets. > > 2) as in one, plus remove the protocol check in the driver > > (i.e., comment the line > > skb->protocol = eth_type_trans(skb, netdev); > > ) to avoid to access the real packet data. > > 3) as in 2, but dealloc is performed at the driver level, instead of > > calling the netif_rx() > > > > In the first case, we can receive about 1.1Mpps (~80% of packets) > > Possible. I was able to receive 900Kpps or so in my experiments with > gact drop which is slightly above this with a 2.4 Ghz machine with IRQ > affinity. I double checked with the people that actually did the job. They indeed tested both cases, i.e., dropping packets either using IRQ (therefore using netif_rx()) or using NAPI (therefore using netif_receive_skb()). In both cases, disabling the eth_type_trans() check, we receive 100% of packets... > > In the third case, we can NOT receive 100% of packets! > > The only difference is that we actually _REMOVED_ a funcion call. This > > reduces the overhead, and the compiler/cpu/whatever can not optimize the > > data path to access to the skb which must be freed. > > It doesnt seem like you were runing NAPI if you depended on calling > netif_rx > In that case, #3 would be freeing in hard IRQ context while #2 is > softIRQ. Again, it was my mistake. Case #3 was performed using the NAPI stack, i.e., freeing up skb instead of calling the netif_receive_skb(). Doing that, we observed a performance drop, that we hint to some caching isses. Indeed, investigating with a Oprofile, in case #3 it registers about twice the number of cache miss than in case #2. Again, we do not have any plain explanation, but our intuition is that adding a function call with pointer as argument might allow the compiler/cpu to prefecth the skb and speed up the memory release... > > Our guess is that by freeing up the skb in the netif_rx() function > > actually allows the compiler/cpu to prefetch the skb itself, and > > therefore keep the pipeline working... > > > > My guess is that if you change compiler, cpu, memory subsystem, you may > > get very counterintuitive results... > > Refer to my comment above. > Repeat tests with NAPI and see if you get same results. We were using NAPI. Sorry for the misunderstanding. Hope this helps. -- Ciao, /\/\/\rco +--+ | Marco Mellia - Assistant Professor| | Tel: 39-011-2276-608 | | Tel: 39-011-564-4173 | | Cel: 39-340-9674888 | /"\ .. . . . . . . . . . . . . | Politecnico di Torino | \ / . ASCII Ribbon Campaign . | Corso Duca degli Abruzzi 24 | X .- NO HTML/RTF in e-mail . | Torino - 10129 - Italy | / \ .- NO Word docs in e-mail. | http://www1.tlc.polito.it/mellia | .. . . . . . . . . . . . . +--+ The box said "Requires Windows 95 or Better." So I installed Linux. From sjackman@gmail.com Thu Dec 2 09:28:59 2004 Received: with ECARTIS (v1.0.0; list netdev); Thu, 02 Dec 2004 09:29:04 -0800 (PST) Received: from mproxy.gmail.com (mproxy.gmail.com [216.239.56.244]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB2HSxL0027742 for ; Thu, 2 Dec 2004 09:28:59 -0800 Received: by mproxy.gmail.com with SMTP id w41so155372cwb for ; Thu, 02 Dec 2004 09:28:36 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=tt+V2mOZVtqNGj2lUTzutEa6y3bD2GO0hWS5b6Xur4NSQ2logiGPSVmg9tnuVxTgrY0xztUkGWxBv96sLTt11zWL2ksmpN/pd54vyP9sNEko8JgvzIXwJaePCalzBBPVOwGV8Yhvuj/nO6Li+oeAl6NTpGyqMEDf2axP1uCdbWQ= Received: by 10.11.116.64 with SMTP id o64mr418223cwc; Thu, 02 Dec 2004 09:28:36 -0800 (PST) Received: by 10.11.99.50 with HTTP; Thu, 2 Dec 2004 09:28:36 -0800 (PST) Message-ID: <7f45d9390412020928f298944@mail.gmail.com> Date: Thu, 2 Dec 2004 09:28:36 -0800 From: Shaun Jackman Reply-To: Shaun Jackman To: netdev@oss.sgi.com Subject: Re: Multicast filtering for tun.c [PATCH] In-Reply-To: <20041201233400.45078efe.akpm@osdl.org> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <7f45d9390411251715138b35d0@mail.gmail.com> <20041127011006.03232bb6.akpm@osdl.org> <7f45d939041201140329d0273f@mail.gmail.com> <20041201233400.45078efe.akpm@osdl.org> X-archive-position: 12389 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: sjackman@gmail.com Precedence: bulk X-list: netdev Thanks for your careful review, Andrew. I appreciate your help. > You're sure this shouldn't be using crc-ccitt? I used ether_crc because every driver in drivers/net except ppp_async.c does. Either would work as long as the chosen hash function is used consistently. > del_multi(u32 *filter, const u8 *addr) > > would be a more typical layout. I prefer the asterisk with the rest of the type information, but I know the latter is more typical. Fixed. > > + const u8 ones[ ETH_ALEN] = { 0xff, 0xff, 0xff, 0xff, 0xff, 0xff }; > > The space after the `[' is gratuitous. > > This array will be allocated onthe stack and populated by hand each time > this function is called. It should be made static. > > > + u8 addr[ ETH_ALEN]; > > extraneous space. Fixed. > > + memcpy(addr, skb->data, min(sizeof addr, skb->len)); > > We normally put parentheses around the argument of the sizeof operator, for > no particular reason ;) > > > + if (copy_from_user(&ifr, argp, sizeof ifr)) > > Ditto I prefer not to add the extraneous punctuation. I find it makes it harder to read. > > + case SIOCGIFFLAGS: > > + ifr.ifr_flags = tun->if_flags; > > + if (copy_to_user( argp, &ifr, sizeof ifr)) > > extraneous space. > > > + if (copy_to_user( argp, &ifr, sizeof ifr)) > > ditto Fixed. > > + DBG(KERN_DEBUG "%s: add multi: %x:%x:%x:%x:%x:%x\n", > > + tun->dev->name, > > + (u8)ifr.ifr_hwaddr.sa_data[0], (u8)ifr.ifr_hwaddr.sa_data[1], > > + (u8)ifr.ifr_hwaddr.sa_data[2], (u8)ifr.ifr_hwaddr.sa_data[3], > > + (u8)ifr.ifr_hwaddr.sa_data[4], (u8)ifr.ifr_hwaddr.sa_data[5]); > > Why all the typecasts? [clip] > ditto. sa_data is signed for some reason, which causes the signed byte to be sign extended resulting in the mac address being printed as 0:c:6e:14:ffffffa0:5a, for example. Here's a short diff of the changes to my patch. The full patch follows. 27c27 < +add_multi(u32* filter, const u8* addr) --- > +add_multi(u32 *filter, const u8 *addr) 38c38 < +del_multi(u32* filter, const u8* addr) --- > +del_multi(u32 *filter, const u8 *addr) 71,72c71,72 < + const u8 ones[ ETH_ALEN] = { 0xff, 0xff, 0xff, 0xff, 0xff, 0xff }; < + u8 addr[ ETH_ALEN]; --- > + static const u8 ones[ETH_ALEN] = { 0xff, 0xff, 0xff, 0xff, 0xff, 0xff }; > + u8 addr[ETH_ALEN]; 137c137 < + void __user* argp = (void __user*)arg; --- > + void __user *argp = (void __user*)arg; 168c168 < + if (copy_to_user( argp, &ifr, sizeof ifr)) --- > + if (copy_to_user(argp, &ifr, sizeof ifr)) 183c183 < + if (copy_to_user( argp, &ifr, sizeof ifr)) --- > + if (copy_to_user(argp, &ifr, sizeof ifr)) Cheers, Shaun 2004-11-25 Shaun Jackman * drivers/net/tun.c: Add multicast filtering for packets travelling from the network device to the character device. * include/linux/if_tun.h (tun_struct): Add interface flags, a hardware device addres, and a multicast filter. diff -ur linux-2.6.8.1.orig/drivers/net/tun.c linux-2.6.8.1/drivers/net/tun.c --- linux-2.6.8.1.orig/drivers/net/tun.c 2004-08-14 03:55:23.000000000 -0700 +++ linux-2.6.8.1/drivers/net/tun.c 2004-11-25 17:00:22.000000000 -0800 @@ -41,6 +41,7 @@ #include #include #include +#include #include #include @@ -104,11 +105,42 @@ return 0; } -static void tun_net_mclist(struct net_device *dev) +/** Add the specified Ethernet address to this multicast filter. */ +static void +add_multi(u32 *filter, const u8 *addr) { - /* Nothing to do for multicast filters. - * We always accept all frames. */ - return; + int bit_nr = ether_crc(ETH_ALEN, addr) >> 26; + filter[bit_nr >> 5] |= 1 << (bit_nr & 31); +} + +/** Remove the specified Ethernet addres from this multicast filter. */ +static void +del_multi(u32 *filter, const u8 *addr) +{ + int bit_nr = ether_crc(ETH_ALEN, addr) >> 26; + filter[bit_nr >> 5] &= ~(1 << (bit_nr & 31)); +} + +/** Update the list of multicast groups to which the network device belongs. + * This list is used to filter packets being sent from the character device to + * the network device. */ +static void +tun_net_mclist(struct net_device *dev) +{ + struct tun_struct *tun = netdev_priv(dev); + const struct dev_mc_list *mclist; + int i; + DBG(KERN_DEBUG "%s: tun_net_mclist: mc_count %d\n", + dev->name, dev->mc_count); + memset(tun->chr_filter, 0, sizeof tun->chr_filter); + for (i = 0, mclist = dev->mc_list; i < dev->mc_count && mclist != NULL; + i++, mclist = mclist->next) { + add_multi(tun->net_filter, mclist->dmi_addr); + DBG(KERN_DEBUG "%s: tun_net_mclist: %x:%x:%x:%x:%x:%x\n", + dev->name, + mclist->dmi_addr[0], mclist->dmi_addr[1], mclist->dmi_addr[2], + mclist->dmi_addr[3], mclist->dmi_addr[4], mclist->dmi_addr[5]); + } } static struct net_device_stats *tun_net_stats(struct net_device *dev) @@ -301,6 +333,10 @@ add_wait_queue(&tun->read_wait, &wait); while (len) { + static const u8 ones[ETH_ALEN] = { 0xff, 0xff, 0xff, 0xff, 0xff, 0xff }; + u8 addr[ETH_ALEN]; + int bit_nr; + current->state = TASK_INTERRUPTIBLE; /* Read frames from the queue */ @@ -320,10 +356,37 @@ } netif_start_queue(tun->dev); - ret = tun_put_user(tun, skb, (struct iovec *) iv, len); - - kfree_skb(skb); - break; + /** Decide whether to accept this packet. This code is designed to + * behave identically to an Ethernet interface. Accept the packet if + * - we are promiscuous. + * - the packet is addressed to us. + * - the packet is broadcast. + * - the packet is multicast and + * - we are multicast promiscous. + * - we belong to the multicast group. + */ + memcpy(addr, skb->data, min(sizeof addr, skb->len)); + bit_nr = ether_crc(sizeof addr, addr) >> 26; + if ((tun->if_flags & IFF_PROMISC) || + memcmp(addr, tun->dev_addr, sizeof addr) == 0 || + memcmp(addr, ones, sizeof addr) == 0 || + (((addr[0] == 1 && addr[1] == 0 && addr[2] == 0x5e) || + (addr[0] == 0x33 && addr[1] == 0x33)) && + ((tun->if_flags & IFF_ALLMULTI) || + (tun->chr_filter[bit_nr >> 5] & (1 << (bit_nr & 31)))))) { + DBG(KERN_DEBUG "%s: tun_chr_readv: accepted: %x:%x:%x:%x:%x:%x\n", + tun->dev->name, addr[0], addr[1], addr[2], + addr[3], addr[4], addr[5]); + ret = tun_put_user(tun, skb, (struct iovec *) iv, len); + kfree_skb(skb); + break; + } else { + DBG(KERN_DEBUG "%s: tun_chr_readv: rejected: %x:%x:%x:%x:%x:%x\n", + tun->dev->name, addr[0], addr[1], addr[2], + addr[3], addr[4], addr[5]); + kfree_skb(skb); + continue; + } } current->state = TASK_RUNNING; @@ -417,6 +480,12 @@ tun = netdev_priv(dev); tun->dev = dev; tun->flags = flags; + /* Be promiscuous by default to maintain previous behaviour. */ + tun->if_flags = IFF_PROMISC; + /* Generate random Ethernet address. */ + *(u16 *)tun->dev_addr = htons(0x00FF); + get_random_bytes(tun->dev_addr + sizeof(u16), 4); + memset(tun->chr_filter, 0, sizeof tun->chr_filter); tun_net_init(dev); @@ -457,13 +526,16 @@ unsigned int cmd, unsigned long arg) { struct tun_struct *tun = file->private_data; + void __user *argp = (void __user*)arg; + struct ifreq ifr; + + if (cmd == TUNSETIFF || _IOC_TYPE(cmd) == 0x89) + if (copy_from_user(&ifr, argp, sizeof ifr)) + return -EFAULT; if (cmd == TUNSETIFF && !tun) { - struct ifreq ifr; int err; - if (copy_from_user(&ifr, (void __user *)arg, sizeof(ifr))) - return -EFAULT; ifr.ifr_name[IFNAMSIZ-1] = '\0'; rtnl_lock(); @@ -473,7 +545,7 @@ if (err) return err; - if (copy_to_user((void __user *)arg, &ifr, sizeof(ifr))) + if (copy_to_user(argp, &ifr, sizeof(ifr))) return -EFAULT; return 0; } @@ -519,6 +591,61 @@ break; #endif + case SIOCGIFFLAGS: + ifr.ifr_flags = tun->if_flags; + if (copy_to_user(argp, &ifr, sizeof ifr)) + return -EFAULT; + return 0; + + case SIOCSIFFLAGS: + /** Set the character device's interface flags. Currently only + * IFF_PROMISC and IFF_ALLMULTI are used. */ + tun->if_flags = ifr.ifr_flags; + DBG(KERN_INFO "%s: interface flags 0x%lx\n", + tun->dev->name, tun->if_flags); + return 0; + + case SIOCGIFHWADDR: + memcpy(ifr.ifr_hwaddr.sa_data, tun->dev_addr, + min(sizeof ifr.ifr_hwaddr.sa_data, sizeof tun->dev_addr)); + if (copy_to_user(argp, &ifr, sizeof ifr)) + return -EFAULT; + return 0; + + case SIOCSIFHWADDR: + /** Set the character device's hardware address. This is used when + * filtering packets being sent from the network device to the character + * device. */ + memcpy(tun->dev_addr, ifr.ifr_hwaddr.sa_data, + min(sizeof ifr.ifr_hwaddr.sa_data, sizeof tun->dev_addr)); + DBG(KERN_DEBUG "%s: set hardware address: %x:%x:%x:%x:%x:%x\n", + tun->dev->name, + tun->dev_addr[0], tun->dev_addr[1], tun->dev_addr[2], + tun->dev_addr[3], tun->dev_addr[4], tun->dev_addr[5]); + return 0; + + case SIOCADDMULTI: + /** Add the specified group to the character device's multicast filter + * list. */ + add_multi(tun->chr_filter, ifr.ifr_hwaddr.sa_data); + DBG(KERN_DEBUG "%s: add multi: %x:%x:%x:%x:%x:%x\n", + tun->dev->name, + (u8)ifr.ifr_hwaddr.sa_data[0], (u8)ifr.ifr_hwaddr.sa_data[1], + (u8)ifr.ifr_hwaddr.sa_data[2], (u8)ifr.ifr_hwaddr.sa_data[3], + (u8)ifr.ifr_hwaddr.sa_data[4], (u8)ifr.ifr_hwaddr.sa_data[5]); + return 0; + + case SIOCDELMULTI: + /** Remove the specified group from the character device's multicast + * filter list. */ + del_multi(tun->chr_filter, ifr.ifr_hwaddr.sa_data); + DBG(KERN_DEBUG "%s: del multi: %x:%x:%x:%x:%x:%x\n", + tun->dev->name, + (u8)ifr.ifr_hwaddr.sa_data[0], (u8)ifr.ifr_hwaddr.sa_data[1], + (u8)ifr.ifr_hwaddr.sa_data[2], (u8)ifr.ifr_hwaddr.sa_data[3], + (u8)ifr.ifr_hwaddr.sa_data[4], (u8)ifr.ifr_hwaddr.sa_data[5]); + return 0; + default: return -EINVAL; }; diff -ur linux-2.6.8.1.orig/include/linux/if_tun.h linux-2.6.8.1/include/linux/if_tun.h --- linux-2.6.8.1.orig/include/linux/if_tun.h 2004-08-14 03:55:09.000000000 -0700 +++ linux-2.6.8.1/include/linux/if_tun.h 2004-11-25 16:47:31.000000000 -0800 @@ -45,6 +45,11 @@ struct fasync_struct *fasync; + unsigned long if_flags; + u8 dev_addr[ETH_ALEN]; + u32 chr_filter[2]; + u32 net_filter[2]; + #ifdef TUN_DEBUG int debug; #endif From mellia@prezzemolo.polito.it Thu Dec 2 09:33:29 2004 Received: with ECARTIS (v1.0.0; list netdev); Thu, 02 Dec 2004 09:33:34 -0800 (PST) Received: from prezzemolo.polito.it (IDENT:root@prezzemolo.polito.it [130.192.9.131]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB2HXSrW028218 for ; Thu, 2 Dec 2004 09:33:28 -0800 Received: from mellia.lipar.polito.it ([192.168.85.105]) by prezzemolo.polito.it (8.12.10/8.12.10) with ESMTP id iB2HVUdW000522; Thu, 2 Dec 2004 18:31:30 +0100 Subject: Re: [E1000-devel] Transmission limit From: Marco Mellia Reply-To: mellia@prezzemolo.polito.it To: sfeldma@pobox.com Cc: birke@serveliper.polito.it, Lennert Buytenhek , jamal , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com In-Reply-To: <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> References: <1101467291.24742.70.camel@mellia.lipar.polito.it> <41A73826.3000109@draigBrady.com> <16807.20052.569125.686158@robur.slu.se> <1101484740.24742.213.camel@mellia.lipar.polito.it> <41A76085.7000105@draigBrady.com> <1101499285.1079.45.camel@jzny.localdomain> <16811.8052.678955.795327@robur.slu.se> <1101821501.1043.43.camel@jzny.localdomain> <20041130134600.GA31515@xi.wantstofly.org> <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> Content-Type: text/plain Organization: Message-Id: <1102008690.19646.22.camel@mellia.lipar.polito.it> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 (1.2.2-5) Date: 02 Dec 2004 18:31:30 +0100 Content-Transfer-Encoding: 7bit X-TLC-MailScanner-Information: Please contact the ISP for more information X-TLC-MailScanner: Found to be clean X-TLC-MailScanner-SpamCheck: not spam, SpamAssassin (score=-4.788, required 5.5, AWL 0.11, BAYES_00 -4.90) X-archive-position: 12390 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: mellia@prezzemolo.polito.it Precedence: bulk X-list: netdev On Wed, 2004-12-01 at 02:09, Scott Feldman wrote: > Hey, turns out, I know some e1000 tricks that might help get the kpps > numbers up. > > My problem is I only have a P4 desktop system with a 82544 nic running > at PCI 32/33Mhz, so I can't play with the big boys. But, attached is a > rework of the Tx path to eliminate 1) Tx interrupts, and 2) Tx > descriptor write-backs. For me, I see a nice jump in kpps, but I'd like > others to try with their setups. We should be able to get to wire speed > with 60-byte packets. > Here are the numbers in our setup: vanilla kernel [2.4.20 + packetgen + driver e1000 5.4.11] 4096 Descr => 356 Mbps (60 bytes long frames) => 941Mbps (1500 bytes lonf frames) 256 Descr => 354 Mbps (60 bytes long frames) => 941Mbps (1500 bytes lonf frames) Patched driver [2.4.20 + packetgen + driver e1000 5.4.11 patched] 4096 Descr => 357 Mbps (60 bytes long frames) => 941Mbps (1500 bytes lonf frames) I guess that was _not_ the bottleneck sigh... at least with a PCI-X bus. Again, latency issue of the DMA transfer from RAM to NIC? -- Ciao, /\/\/\rco +-----------------------------------+ | Marco Mellia - Assistant Professor| | Tel: 39-011-2276-608 | | Tel: 39-011-564-4173 | | Cel: 39-340-9674888 | /"\ .. . . . . . . . . . . . . | Politecnico di Torino | \ / . ASCII Ribbon Campaign . | Corso Duca degli Abruzzi 24 | X .- NO HTML/RTF in e-mail . | Torino - 10129 - Italy | / \ .- NO Word docs in e-mail. | http://www1.tlc.polito.it/mellia | .. . . . . . . . . . . . . +-----------------------------------+ The box said "Requires Windows 95 or Better." So I installed Linux. From shemminger@osdl.org Thu Dec 2 09:52:28 2004 Received: with ECARTIS (v1.0.0; list netdev); Thu, 02 Dec 2004 09:52:34 -0800 (PST) Received: from mail.osdl.org (fw.osdl.org [65.172.181.6]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB2HqRXO028837 for ; Thu, 2 Dec 2004 09:52:28 -0800 Received: from dxpl.pdx.osdl.net (dxpl.pdx.osdl.net [172.20.1.103]) by mail.osdl.org (8.11.6/8.11.6) with ESMTP id iB2Hpw908926; Thu, 2 Dec 2004 09:51:58 -0800 Date: Thu, 2 Dec 2004 09:51:58 -0800 From: Stephen Hemminger To: Jeff Garzik Cc: "David S. Miller" , netdev@oss.sgi.com Subject: Re: [PATCH] b44: allow ethtool get_settings when down Message-Id: <20041202095158.0722d176@dxpl.pdx.osdl.net> In-Reply-To: <41AEFB95.8000100@pobox.com> References: <20041129094523.3185c64c@zqx3.pdx.osdl.net> <41AEFB95.8000100@pobox.com> Organization: Open Source Development Lab X-Mailer: Sylpheed version 0.9.10claws (GTK+ 1.2.10; x86_64-suse-linux) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-archive-position: 12391 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: shemminger@osdl.org Precedence: bulk X-list: netdev On Thu, 02 Dec 2004 06:25:09 -0500 Jeff Garzik wrote: > Stephen Hemminger wrote: > > The FC and Suse startup scripts use ethtool to check for link present. This has > > problems on my laptop with Broadcom because it quieries settings before > > bringing link up. The problem is driver returns EAGAIN when queried for > > settings but not up. Just go ahead and return values anyway, the supported and link > > state values will be correct, speed will end up being 10BaseT/Half which is a > > reasonable default. > > > > Signed-off-by: Stephen Hemminger > > > > diff -Nru a/drivers/net/b44.c b/drivers/net/b44.c > > --- a/drivers/net/b44.c 2004-11-29 09:41:27 -08:00 > > +++ b/drivers/net/b44.c 2004-11-29 09:41:27 -08:00 > > @@ -1487,8 +1487,6 @@ > > { > > struct b44 *bp = netdev_priv(dev); > > > > - if (!(bp->flags & B44_FLAG_INIT_COMPLETE)) > > - return -EAGAIN; > > cmd->supported = (SUPPORTED_Autoneg); > > cmd->supported |= (SUPPORTED_100baseT_Half | > > SUPPORTED_100baseT_Full | > > I'm not so sure about this one... > > This sounds like working around stupid userland in the kernel? > > Jeff Don't bother with the patch, if I use smart user land code like NetworkManager then there is no problem. Although EAGAIN seems like a poor choice for errno how about ENETDOWN or ENONET From Robert.Olsson@data.slu.se Thu Dec 2 09:55:08 2004 Received: with ECARTIS (v1.0.0; list netdev); Thu, 02 Dec 2004 09:55:12 -0800 (PST) Received: from mail1.slu.se (mail1.slu.se [130.238.96.11]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB2Ht7ex029235 for ; Thu, 2 Dec 2004 09:55:07 -0800 Received: from robur.slu.se (robur.slu.se [130.238.98.12]) by mail1.slu.se (8.12.10/8.12.10) with ESMTP id iB2HsVKO021575; Thu, 2 Dec 2004 18:54:32 +0100 Received: by robur.slu.se (Postfix, from userid 1000) id 36228EC001; Thu, 2 Dec 2004 18:54:32 +0100 (CET) From: Robert Olsson MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <16815.22232.190929.306486@robur.slu.se> Date: Thu, 2 Dec 2004 18:54:32 +0100 To: sfeldma@pobox.com Cc: Robert Olsson , Lennert Buytenhek , jamal , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: [E1000-devel] Transmission limit In-Reply-To: <1101919791.5198.15.camel@localhost.localdomain> References: <1101467291.24742.70.camel@mellia.lipar.polito.it> <41A73826.3000109@draigBrady.com> <16807.20052.569125.686158@robur.slu.se> <1101484740.24742.213.camel@mellia.lipar.polito.it> <41A76085.7000105@draigBrady.com> <1101499285.1079.45.camel@jzny.localdomain> <16811.8052.678955.795327@robur.slu.se> <1101821501.1043.43.camel@jzny.localdomain> <20041130134600.GA31515@xi.wantstofly.org> <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <16813.58484.343629.570703@robur.slu.se> <1101919791.5198.15.camel@localhost.localdomain> X-Mailer: VM 7.18 under Emacs 21.3.1 X-archive-position: 12392 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: Robert.Olsson@data.slu.se Precedence: bulk X-list: netdev Scott Feldman writes: > Thank you Robert for trying it out. Scott! I've rerun some of the tests. I've set maxcpus=1 make sure all things happens on one CPU. Some HW as yesterday. I see a now lot variation in the results from your patch. vanilla 804353pps 411Mb/sec (411828736bps) errors: 98877 patch TXD=4096 Sometimes: 882362pps 451Mb/sec (451769344bps) errors: 0 patch TXD=2048 Sometimes: 943007pps 482Mb/sec (482819584bps) errors: 0 But very often runs around 500 kpps with patch. This smells scheduling to me as smaller rings use to mean higher performance but ring need to big enough to hide latencies. See also my next mail... --ro From Robert.Olsson@data.slu.se Thu Dec 2 10:23:55 2004 Received: with ECARTIS (v1.0.0; list netdev); Thu, 02 Dec 2004 10:24:00 -0800 (PST) Received: from mail1.slu.se (mail1.slu.se [130.238.96.11]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB2INsbO030225 for ; Thu, 2 Dec 2004 10:23:55 -0800 Received: from robur.slu.se (robur.slu.se [130.238.98.12]) by mail1.slu.se (8.12.10/8.12.10) with ESMTP id iB2INOKO003502; Thu, 2 Dec 2004 19:23:24 +0100 Received: by robur.slu.se (Postfix, from userid 1000) id 1F693EC001; Thu, 2 Dec 2004 19:23:24 +0100 (CET) From: Robert Olsson MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <16815.23964.93437.411404@robur.slu.se> Date: Thu, 2 Dec 2004 19:23:24 +0100 To: sfeldma@pobox.com Cc: Robert Olsson , Lennert Buytenhek , jamal , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: [E1000-devel] Transmission limit In-Reply-To: <1101919791.5198.15.camel@localhost.localdomain> References: <1101467291.24742.70.camel@mellia.lipar.polito.it> <41A73826.3000109@draigBrady.com> <16807.20052.569125.686158@robur.slu.se> <1101484740.24742.213.camel@mellia.lipar.polito.it> <41A76085.7000105@draigBrady.com> <1101499285.1079.45.camel@jzny.localdomain> <16811.8052.678955.795327@robur.slu.se> <1101821501.1043.43.camel@jzny.localdomain> <20041130134600.GA31515@xi.wantstofly.org> <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <16813.58484.343629.570703@robur.slu.se> <1101919791.5198.15.camel@localhost.localdomain> X-Mailer: VM 7.18 under Emacs 21.3.1 X-archive-position: 12393 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: Robert.Olsson@data.slu.se Precedence: bulk X-list: netdev Hello! Below is little patch to clean skb at xmit. It's old jungle trick Jamal and I used w. tulip. Note we can now even decrease the size of TX ring. It can increase TX performance from 800 kpps to 1125128pps 576Mb/sec (576065536bps) errors: 0 1124946pps 575Mb/sec (575972352bps) errors: 0 But suffers from scheduling problems as the previous patch. Often we just get 582108pps 298Mb/sec (298039296bps) errors: 0 When the sender CPU free (it's) skb's. we might get some "TX free affinity" which are unrelated to irq affinity of course not 100% perfect. And some of Scotts may still be used. --- drivers/net/e1000/e1000.h.orig 2004-12-01 13:59:36.000000000 +0100 +++ drivers/net/e1000/e1000.h 2004-12-02 20:11:31.000000000 +0100 @@ -103,7 +103,7 @@ #define E1000_MAX_INTR 10 /* TX/RX descriptor defines */ -#define E1000_DEFAULT_TXD 256 +#define E1000_DEFAULT_TXD 128 #define E1000_MAX_TXD 256 #define E1000_MIN_TXD 80 #define E1000_MAX_82544_TXD 4096 --- drivers/net/e1000/e1000_main.c.orig 2004-12-01 13:59:36.000000000 +0100 +++ drivers/net/e1000/e1000_main.c 2004-12-02 20:37:40.000000000 +0100 @@ -1820,6 +1820,10 @@ return NETDEV_TX_LOCKED; } + + if( adapter->tx_ring.next_to_use - adapter->tx_ring.next_to_clean > 80 ) + e1000_clean_tx_ring(adapter); + /* need: count + 2 desc gap to keep tail from touching * head, otherwise try next time */ if(E1000_DESC_UNUSED(&adapter->tx_ring) < count + 2) { --ro From jason.mcmullan@timesys.com Thu Dec 2 10:29:54 2004 Received: with ECARTIS (v1.0.0; list netdev); Thu, 02 Dec 2004 10:30:05 -0800 (PST) Received: from exchange.timesys.com (mail.timesys.com [65.117.135.102]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB2ITouC030675 for ; Thu, 2 Dec 2004 10:29:54 -0800 Received: from jmcmullan by owa.timesys.com; 02 Dec 2004 13:29:23 -0500 Subject: [PATCH] MII bus API for PHY devices Rev 2.0 From: Jason McMullan To: Andy Fleming Cc: Benjamin Herrenschmidt , Netdev In-Reply-To: References: <069B6F33-341C-11D9-9652-000393DBC2E8@freescale.com> <9B0D9272-398A-11D9-96F6-000393C30512@freescale.com> <1100820391.25521.14.camel@gaston> <97DA0EF0-3A70-11D9-B023-000393C30512@freescale.com> <1100904184.3856.46.camel@gaston> Content-Type: multipart/mixed; boundary="=-UCEP7LL7eTaCc7pqknBv" Date: Thu, 02 Dec 2004 13:29:22 -0500 Message-Id: <1102012163.6056.39.camel@jmcmullan> Mime-Version: 1.0 X-Mailer: Evolution 2.0.1-1mdk X-archive-position: 12394 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jason.mcmullan@timesys.com Precedence: bulk X-list: netdev --=-UCEP7LL7eTaCc7pqknBv Content-Type: text/plain Content-Transfer-Encoding: 7bit Ok, given previous input, I now release mii-bus 2.0 * CIS8201 interrupt support * The PHY device api is now similar to 'sungem' * Set up your PHY as autoneg or forced speeds. -- Jason McMullan --=-UCEP7LL7eTaCc7pqknBv Content-Disposition: attachment; filename=driver-mii-bus.patch Content-Type: text/x-patch; name=driver-mii-bus.patch; charset=ISO-8859-1 Content-Transfer-Encoding: base64 IyMjIyBBdXRvLWdlbmVyYXRlZCBwYXRjaCAjIyMjDQpTaWduZWQtb2ZmLWJ5OiBKYXNvbiBNY011 bGxhbiA8am1jbXVsbGFuQHRpbWVzeXMuY29tPg0KRGF0ZTogICAgICAgICAgVGh1LCAwMiBEZWMg MjAwNCAxMzoyMDo0OSAtMDUwMA0KRGVzY3JpcHRpb246ICAgTUlJIEJ1cyBpbnRlcmZhY2UNCkRl cGVuZHM6DQoNCiMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMNCg0KSW5kZXggb2YgY2hh bmdlczoNCg0KIGRyaXZlcnMvbmV0L01ha2VmaWxlICAgICAgICAgICAgfCAgICA0IA0KIGxpbnV4 L2RyaXZlcnMvbmV0L21paV9iaXRiYW5nLmMgfCAgMTM0ICsrKysrKysrDQogbGludXgvZHJpdmVy cy9uZXQvbWlpX2JpdGJhbmcuaCB8ICAgNDAgKysNCiBsaW51eC9kcml2ZXJzL25ldC9taWlfYnVz LmMgICAgIHwgIDYzOSArKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrDQog bGludXgvZHJpdmVycy9uZXQvcGh5X2NpY2FkYS5jICB8ICAxNzcgKysrKysrKysrKysNCiBsaW51 eC9kcml2ZXJzL25ldC9waHlfZGF2aWNvbS5jIHwgIDE0MCArKysrKysrKw0KIGxpbnV4L2RyaXZl cnMvbmV0L3BoeV9seHQ5N3guYyAgfCAgMjEwICsrKysrKysrKysrKysNCiBsaW51eC9kcml2ZXJz L25ldC9waHlfbWFydmVsbC5jIHwgIDEyNSArKysrKysrDQogbGludXgvaW5jbHVkZS9saW51eC9t aWlfYnVzLmggICB8ICAxOTEgKysrKysrKysrKysNCiA5IGZpbGVzIGNoYW5nZWQsIDE2NTkgaW5z ZXJ0aW9ucygrKSwgMSBkZWxldGlvbigtKQ0KDQoNCi0tLSBsaW51eC1vcmlnL2RyaXZlcnMvbmV0 L01ha2VmaWxlDQorKysgbGludXgvZHJpdmVycy9uZXQvTWFrZWZpbGUNCkBAIC02Miw3ICs2Miw5 IEBADQogIyBlbmQgbGluayBvcmRlciBzZWN0aW9uDQogIw0KIA0KLW9iai0kKENPTkZJR19NSUkp ICs9IG1paS5vDQorb2JqLSQoQ09ORklHX01JSSkgKz0gbWlpLm8gbWlpX2J1cy5vIG1paV9iaXRi YW5nLm8gXA0KKwkJICAgICBwaHlfZGF2aWNvbS5vIHBoeV9tYXJ2ZWxsLm8gcGh5X2NpY2FkYS5v IFwNCisJCSAgICAgcGh5X2x4dDk3eC5vDQogDQogb2JqLSQoQ09ORklHX1NVTkRBTkNFKSArPSBz dW5kYW5jZS5vDQogb2JqLSQoQ09ORklHX0hBTUFDSEkpICs9IGhhbWFjaGkubw0KLS0tIC9kZXYv bnVsbA0KKysrIGxpbnV4L2RyaXZlcnMvbmV0L21paV9iaXRiYW5nLmMNCkBAIC0wLDAgKzEsMTM0 IEBADQorLyogDQorICogZHJpdmVycy9uZXQvbWlpX2JpdGJhbmcuYw0KKyAqDQorICogQXV0aG9y OiBKYXNvbiBNY011bGxhbg0KKyAqDQorICogQ29weXJpZ2h0IChjKSAyMDA0IFRpbWVzeXMgQ29y cC4NCisgKg0KKyAqIFRoaXMgcHJvZ3JhbSBpcyBmcmVlIHNvZnR3YXJlOyB5b3UgY2FuIHJlZGlz dHJpYnV0ZSAgaXQgYW5kL29yIG1vZGlmeSBpdA0KKyAqIHVuZGVyICB0aGUgdGVybXMgb2YgIHRo ZSBHTlUgR2VuZXJhbCAgUHVibGljIExpY2Vuc2UgYXMgcHVibGlzaGVkIGJ5IHRoZQ0KKyAqIEZy ZWUgU29mdHdhcmUgRm91bmRhdGlvbjsgIGVpdGhlciB2ZXJzaW9uIDIgb2YgdGhlICBMaWNlbnNl LCBvciAoYXQgeW91cg0KKyAqIG9wdGlvbikgYW55IGxhdGVyIHZlcnNpb24uDQorICoNCisgKi8N CisNCisjaW5jbHVkZSA8bGludXgva2VybmVsLmg+DQorI2luY2x1ZGUgPGFzbS9zdHJpbmcuaD4N CisNCisjaW5jbHVkZSAibWlpX2JpdGJhbmcuaCINCisNCisjdW5kZWYgUEhZX1JFQUQNCisjdW5k ZWYgUEhZX1dSSVRFDQorI2RlZmluZSBQSFlfUkVBRAkwDQorI2RlZmluZSBQSFlfV1JJVEUJMQ0K Kw0KKy8qIA0KKyAqIDFzdCBieXRlOiAwMTAxUFBQUCBvbiB3cml0ZXMsIHdoZXJlIFBQUFAgaXMg dGhlIE1TQiBvZiB0aGUgcGh5LWlkDQorICogICAgICAgICAgIDAxMTBQUFBQIG9uIHJlYWRzLCB3 aGVyZSAgUFBQUCBpcyB0aGUgTVNCIG9mIHRoZSBwaHktaWQNCisgKiAybmQgYnl0ZTogUFJSUlJS MTAsIFAgaXMgdGhlIExTQiBvZiB0aGUgcGh5LWlkLCBSIGlzIHRoZSByZWdpc3Rlcg0KKyAqIDNy ZCw0dGggYnl0ZXM6IGNvbnRyb2wgb24gd3JpdGVzLCB2YWx1ZXMgb24gcmVhZHMNCisgKi8NCisN CitzdGF0aWMgaW5saW5lIHZvaWQgbWlpX2JpdGJhbmdfbWFyayh2b2lkICpwcml2KQ0KK3sNCisJ c3RydWN0IG1paV9iaXRiYW5nICppbmZvID0gcHJpdjsNCisJaW50IGk7DQorDQorCS8qIFdyaXRl IHByZWFtYmxlICovDQorCWZvciAoaSA9IDA7IGkgPCAzMjsgaSsrKQ0KKwkJaW5mby0+c2VuZChp bmZvLT5wcml2LCAxKTsNCit9DQorDQorc3RhdGljIGlubGluZSB2b2lkIG1paV9iaXRiYW5nX3Bo eV9pZCh2b2lkICpwcml2LCBpbnQgcGh5X2lkLCBpbnQgcmVnLCBpbnQgaXNfd3JpdGUpDQorew0K KwlzdHJ1Y3QgbWlpX2JpdGJhbmcgKmluZm8gPSBwcml2Ow0KKwlpbnQgaTsNCisNCisJLyogUHJl YW1ibGUgKi8NCisJaW5mby0+c2VuZChpbmZvLT5wcml2LDApOw0KKwlpbmZvLT5zZW5kKGluZm8t PnByaXYsMSk7DQorCWlmIChpc193cml0ZSkgew0KKwkJaW5mby0+c2VuZChpbmZvLT5wcml2LDAp Ow0KKwkJaW5mby0+c2VuZChpbmZvLT5wcml2LDEpOw0KKwl9IGVsc2Ugew0KKwkJaW5mby0+c2Vu ZChpbmZvLT5wcml2LDEpOw0KKwkJaW5mby0+c2VuZChpbmZvLT5wcml2LDApOw0KKwl9DQorDQor CS8qIFdyaXRlIFBIWSBhZGRyICovDQorCWZvciAoaSA9IDA7IGkgPCA1OyBpKyspDQorCQlpbmZv LT5zZW5kKGluZm8tPnByaXYsIChwaHlfaWQgPj4gKDQtaSkpICYgMSk7DQorDQorCS8qIFdyaXRl IHRoZSByZWdpc3RlciAqLw0KKwlmb3IgKGkgPSAwOyBpIDwgNTsgaSsrKQ0KKwkJaW5mby0+c2Vu ZChpbmZvLT5wcml2LCAocmVnID4+ICg0LWkpKSAmIDEpOw0KKw0KKwlpbmZvLT5zZW5kKGluZm8t PnByaXYsMSk7DQorCWluZm8tPnNlbmQoaW5mby0+cHJpdiwwKTsNCit9DQorDQorc3RhdGljIGlu dCBtaWlfYml0YmFuZ19yZWFkKHZvaWQgKnByaXYsIGludCBwaHlfaWQsIGludCByZWcpDQorew0K KwlzdHJ1Y3QgbWlpX2JpdGJhbmcgKmluZm8gPSBwcml2Ow0KKwlpbnQgaTsNCisJaW50IHJldHZh bD0wOw0KKw0KKwltaWlfYml0YmFuZ19tYXJrKHByaXYpOw0KKwltaWlfYml0YmFuZ19waHlfaWQo cHJpdiwgcGh5X2lkLCByZWcsIFBIWV9SRUFEKTsNCisNCisJZm9yIChpID0gMDsgaSA8IDE2OyBp KyspDQorCQlyZXR2YWwgPSAocmV0dmFsIDw8IDEpIHwgKGluZm8tPnJlY3YoaW5mby0+cHJpdikg JiAxKTsNCisNCisJbWlpX2JpdGJhbmdfbWFyayhwcml2KTsNCisNCisJcmV0dXJuIHJldHZhbDsN Cit9DQorDQorc3RhdGljIGludCBtaWlfYml0YmFuZ193cml0ZSh2b2lkICpwcml2LCBpbnQgcGh5 X2lkLCBpbnQgcmVnLCB1aW50MTZfdCB2YWwpDQorew0KKwlzdHJ1Y3QgbWlpX2JpdGJhbmcgKmlu Zm8gPSBwcml2Ow0KKwlpbnQgaTsNCisNCisJbWlpX2JpdGJhbmdfbWFyayhwcml2KTsNCisJbWlp X2JpdGJhbmdfcGh5X2lkKHByaXYsIHBoeV9pZCwgcmVnLCBQSFlfV1JJVEUpOw0KKw0KKwlmb3Ig KGk9MDsgaSA8IDE2OyBpKyspDQorCQlpbmZvLT5zZW5kKGluZm8tPnByaXYsICh2YWwgPj4gKDE1 LWkpKSAmIDEpOw0KKw0KKwltaWlfYml0YmFuZ19tYXJrKHByaXYpOw0KKw0KKwlyZXR1cm4gMDsN Cit9DQorDQorc3RhdGljIHZvaWQgbWlpX2JpdGJhbmdfcmVzZXQodm9pZCAqcHJpdikNCit7DQor CXN0cnVjdCBtaWlfYml0YmFuZyAqaW5mbyA9IHByaXY7DQorDQorCWluZm8tPnJlc2V0KGluZm8t PnByaXYpOw0KK30NCisNCisvKiBDcmVhdGVzIGEgYml0YmFuZyBNSUkgYnVzDQorICogUmV0dXJu cyA8IDAgb24gZXJyb3IsIG90aGVyd2lzZSBhIGJ1cyBJRA0KKyAqLw0KK2ludCBtaWlfYml0YmFu Z19yZWdpc3RlcihzdHJ1Y3QgbWlpX2JpdGJhbmcgKmluZm8pDQorew0KKwltZW1zZXQoJmluZm8t PmJ1cywgMCwgc2l6ZW9mKGluZm8tPmJ1cykpOw0KKw0KKwlpbmZvLT5idXMubmFtZSA9IGluZm8t Pm5hbWU7DQorCWluZm8tPmJ1cy5wcml2ID0gaW5mbzsNCisJaW5mby0+YnVzLnJlYWQgPSBtaWlf Yml0YmFuZ19yZWFkOw0KKwlpbmZvLT5idXMud3JpdGUgPSBtaWlfYml0YmFuZ193cml0ZTsNCisJ aW5mby0+YnVzLnJlc2V0ID0gbWlpX2JpdGJhbmdfcmVzZXQ7DQorDQorCXJldHVybiBtaWlfYnVz X3JlZ2lzdGVyKCZpbmZvLT5idXMpOw0KK30NCisNCisNCisvKiBVbnJlZ2lzdGVycyBhIGJpdGJh bmcgTUlJIGJ1cw0KKyAqLw0KK3ZvaWQgbWlpX2JpdGJhbmdfdW5yZWdpc3RlcihzdHJ1Y3QgbWlp X2JpdGJhbmcgKmluZm8pDQorew0KKwltaWlfYnVzX3VucmVnaXN0ZXIoJmluZm8tPmJ1cyk7DQor fQ0KKw0KKw0KDQotLS0gL2Rldi9udWxsDQorKysgbGludXgvZHJpdmVycy9uZXQvbWlpX2JpdGJh bmcuaA0KQEAgLTAsMCArMSw0MCBAQA0KKy8qIA0KKyAqIGRyaXZlcnMvbmV0L21paV9iaXRiYW5n LmgNCisgKg0KKyAqIEF1dGhvcjogSmFzb24gTWNNdWxsYW4NCisgKg0KKyAqIENvcHlyaWdodCAo YykgMjAwNCBUaW1lc3lzIENvcnAuDQorICoNCisgKiBUaGlzIHByb2dyYW0gaXMgZnJlZSBzb2Z0 d2FyZTsgeW91IGNhbiByZWRpc3RyaWJ1dGUgIGl0IGFuZC9vciBtb2RpZnkgaXQNCisgKiB1bmRl ciAgdGhlIHRlcm1zIG9mICB0aGUgR05VIEdlbmVyYWwgIFB1YmxpYyBMaWNlbnNlIGFzIHB1Ymxp c2hlZCBieSB0aGUNCisgKiBGcmVlIFNvZnR3YXJlIEZvdW5kYXRpb247ICBlaXRoZXIgdmVyc2lv biAyIG9mIHRoZSAgTGljZW5zZSwgb3IgKGF0IHlvdXINCisgKiBvcHRpb24pIGFueSBsYXRlciB2 ZXJzaW9uLg0KKyAqDQorICovDQorI2lmbmRlZiBfTkVUX01JSV9CSVRCQU5HX0gNCisjZGVmaW5l IF9ORVRfTUlJX0JJVEJBTkdfSA0KKw0KKyNpbmNsdWRlIDxsaW51eC9taWlfYnVzLmg+DQorDQor c3RydWN0IG1paV9iaXRiYW5nIHsNCisJY29uc3QgY2hhciAqbmFtZTsJLyogTmFtZSBvZiBkZXZp Y2UgKi8NCisJdm9pZCAqcHJpdjsJCS8qIFByaXZhdGUgZGF0YSAqLw0KKw0KKwl2b2lkICgqc2Vu ZCkodm9pZCAqcHJpdiwgaW50IGJpdCk7CS8qIFNlbmQgb25lIGJpdCAqLw0KKwlpbnQgKCpyZWN2 KSh2b2lkICpwcml2KTsJCS8qIFJlY3Ygb25lIGJpdCAqLw0KKwl2b2lkICgqcmVzZXQpKHZvaWQg KnByaXYpOw0KKw0KKwkvKiBBdXRvLWZpbGxlZC1pbiBpbmZvcm1hdGlvbiAqLw0KKwlzdHJ1Y3Qg bWlpX2J1cyBidXM7DQorfTsNCisNCisvKiBDcmVhdGVzIGEgYml0YmFuZyBNSUkgYnVzDQorICog UmV0dXJucyA8IDAgb24gZXJyb3IsIG90aGVyd2lzZSBhIGJ1cyBJRA0KKyAqLw0KK2V4dGVybiBp bnQgbWlpX2JpdGJhbmdfcmVnaXN0ZXIoc3RydWN0IG1paV9iaXRiYW5nICppbmZvKTsNCisNCisv KiBVbnJlZ2lzdGVycyBhIGJpdGJhbmcgTUlJIGJ1cw0KKyAqLw0KK2V4dGVybiB2b2lkIG1paV9i aXRiYW5nX3VucmVnaXN0ZXIoc3RydWN0IG1paV9iaXRiYW5nICppbmZvKTsNCisNCisjZW5kaWYg LyogX05FVF9NSUlfQklUQkFOR19IICovDQoNCi0tLSAvZGV2L251bGwNCisrKyBsaW51eC9kcml2 ZXJzL25ldC9taWlfYnVzLmMNCkBAIC0wLDAgKzEsNjM5IEBADQorLyogDQorICogZHJpdmVycy9u ZXQvbWlpX2J1cy5jDQorICoNCisgKiBBZGFwZXRlZCBmcm9tIGRyaXZlcnMvbmV0L2dpYW5mYXJf bWlpLmMsIGJ5IEFuZHkgRmxlbWluZw0KKyAqDQorICogQXV0aG9yOiBKYXNvbiBNY011bGxhbiAo amFzb24ubWNtdWxsYW5AdGltZXN5cy5jb20pIHRvIA0KKyAqIAkgICBiZSBhIGdlbmVyaWMgbWlp IGludGVyZmFjZQ0KKyAqDQorICogQ29weXJpZ2h0IChjKSAyMDA0IFRpbWVzeXMgSW5jDQorICoN CisgKiBUaGlzIHByb2dyYW0gaXMgZnJlZSBzb2Z0d2FyZTsgeW91IGNhbiByZWRpc3RyaWJ1dGUg IGl0IGFuZC9vciBtb2RpZnkgaXQNCisgKiB1bmRlciAgdGhlIHRlcm1zIG9mICB0aGUgR05VIEdl bmVyYWwgIFB1YmxpYyBMaWNlbnNlIGFzIHB1Ymxpc2hlZCBieSB0aGUNCisgKiBGcmVlIFNvZnR3 YXJlIEZvdW5kYXRpb247ICBlaXRoZXIgdmVyc2lvbiAyIG9mIHRoZSAgTGljZW5zZSwgb3IgKGF0 IHlvdXINCisgKiBvcHRpb24pIGFueSBsYXRlciB2ZXJzaW9uLg0KKyAqDQorICovDQorI2luY2x1 ZGUgPGxpbnV4L2NvbmZpZy5oPg0KKyNpbmNsdWRlIDxsaW51eC9rZXJuZWwuaD4NCisjaW5jbHVk ZSA8bGludXgvc3RyaW5nLmg+DQorI2luY2x1ZGUgPGxpbnV4L2Vycm5vLmg+DQorI2luY2x1ZGUg PGxpbnV4L3NsYWIuaD4NCisjaW5jbHVkZSA8bGludXgvaW50ZXJydXB0Lmg+DQorI2luY2x1ZGUg PGxpbnV4L2luaXQuaD4NCisjaW5jbHVkZSA8bGludXgvZGVsYXkuaD4NCisjaW5jbHVkZSA8bGlu dXgvbmV0ZGV2aWNlLmg+DQorI2luY2x1ZGUgPGxpbnV4L2V0aGVyZGV2aWNlLmg+DQorI2luY2x1 ZGUgPGxpbnV4L3NrYnVmZi5oPg0KKyNpbmNsdWRlIDxsaW51eC9zcGlubG9jay5oPg0KKyNpbmNs dWRlIDxsaW51eC9tbS5oPg0KKyNpbmNsdWRlIDxsaW51eC9taWlfYnVzLmg+DQorDQorI2luY2x1 ZGUgPGFzbS9pby5oPg0KKyNpbmNsdWRlIDxhc20vaXJxLmg+DQorI2luY2x1ZGUgPGFzbS91YWNj ZXNzLmg+DQorI2luY2x1ZGUgPGxpbnV4L21vZHVsZS5oPg0KKw0KKyN1bmRlZiBERUJVRw0KKw0K K3N0YXRpYyBzdHJ1Y3QgbWlpX2J1cyAqbWlpX2J1c1s4XTsNCisNCisjZGVmaW5lIE1JSV9CVVNf TUFYCShzaXplb2YobWlpX2J1cykvc2l6ZW9mKHN0cnVjdCBtaWlfYnVzICopKQ0KKw0KK3N0YXRp YyBpbmxpbmUgc3RydWN0IHBoeV9pbmZvICptaWlfcGh5X29mKHN0cnVjdCBtaWlfaWZfaW5mbyAq bWlpKQ0KK3sNCisJaWYgKG1paSAhPSBOVUxMKSB7DQorCQlpbnQgYnVzLGlkOw0KKwkJYnVzID0g TUlJX0JVUyhtaWktPnBoeV9pZCk7DQorCQlpZCAgPSBNSUlfSUQobWlpLT5waHlfaWQpOw0KKwkJ cmV0dXJuIG1paV9idXNbYnVzXS0+cGh5W2lkXTsNCisJfQ0KKw0KKwlyZXR1cm4gTlVMTDsNCit9 DQorDQorLyogV3JpdGUgdmFsdWUgdG8gdGhlIFBIWSBmb3IgdGhpcyBkZXZpY2UgdG8gdGhlIHJl Z2lzdGVyIGF0IHJlZ251bSwNCisgKiB3YWl0aW5nIHVudGlsIHRoZSB3cml0ZSBpcyBkb25lIGJl Zm9yZSBpdCByZXR1cm5zLiAgQWxsIFBIWSANCisgKiBjb25maWd1cmF0aW9uIGhhcyB0byBiZSBk b25lIHRocm91Z2ggdGhlIFRTRUMxIE1JSU0gcmVncyAqLw0KK0VYUE9SVF9TWU1CT0wobWlpX2J1 c193cml0ZSk7DQoraW50IG1paV9idXNfd3JpdGUoaW50IGJ1cywgaW50IGlkLCBpbnQgcmVnbnVt LCB1aW50MTZfdCB2YWx1ZSkNCit7DQorCWlmIChtaWlfYnVzW2J1c10gPT0gTlVMTCkNCisJCXJl dHVybiAtRUlOVkFMOw0KKw0KKyNpZmRlZiBERUJVRw0KKwlwcmludGsoS0VSTl9JTkZPICJwaHkl ZC4lZDogV3JpdGUgMHglLjJ4IDwtIDB4JS40eFxuIiwgYnVzLCBpZCwgcmVnbnVtLCB2YWx1ZSk7 DQorI2VuZGlmDQorCW1paV9idXNbYnVzXS0+d3JpdGUobWlpX2J1c1tidXNdLT5wcml2LCBpZCwg cmVnbnVtLCB2YWx1ZSk7DQorCXJldHVybiAwOw0KK30NCisNCisvKiBSZWFkcyBmcm9tIHJlZ2lz dGVyIHJlZ251bSBpbiB0aGUgUEhZIGZvciBkZXZpY2UgZGV2LA0KKyAqIHJldHVybmluZyB0aGUg dmFsdWUuICBDbGVhcnMgbWlpbWNvbSBmaXJzdC4gIEFsbCBQSFkNCisgKiBjb25maWd1cmF0aW9u IGhhcyB0byBiZSBkb25lIHRocm91Z2ggdGhlIFRTRUMxIE1JSU0gcmVncyAqLw0KK0VYUE9SVF9T WU1CT0wobWlpX2J1c19yZWFkKTsNCitpbnQgbWlpX2J1c19yZWFkKGludCBidXMsIGludCBpZCwg aW50IHJlZ251bSkNCit7DQorCWlmIChtaWlfYnVzW2J1c10gPT0gTlVMTCkNCisJCXJldHVybiAt RUlOVkFMOw0KKw0KKyNpZmRlZiBERUJVRw0KKwl7DQorCQlpbnQgcnY7DQorCQlydiA9IG1paV9i dXNbYnVzXS0+cmVhZChtaWlfYnVzW2J1c10tPnByaXYsIGlkLCByZWdudW0pOw0KKwkJcHJpbnRr KEtFUk5fSU5GTyAicGh5JWQuJWQ6IFJlYWQgIDB4JS4yeCAtPiAweCUuNHhcbiIsIGJ1cywgaWQs IHJlZ251bSwgcnYpOw0KKwkJcmV0dXJuIHJ2Ow0KKwl9DQorI2Vsc2UNCisJcmV0dXJuIG1paV9i dXNbYnVzXS0+cmVhZChtaWlfYnVzW2J1c10tPnByaXYsIGlkLCByZWdudW0pOw0KKyNlbmRpZg0K K30NCisNCisvKiBIZWxwZXIgZnVuY3Rpb24gKi8NCitzdGF0aWMgaW50IHBoeV9zZXRfYXV0b25l ZyhzdHJ1Y3QgcGh5X2luZm8gKnBoeSwgdWludDMyX3QgYWR2ZXJ0aXNlKQ0KK3sNCisJaW50IGVy cjsNCisNCisJZXJyID0gcGh5LT5vcHMtPnNldF9hdXRvbmVnKHBoeSwgYWR2ZXJ0aXNlKTsNCisJ aWYgKGVyciA8IDApDQorCQlyZXR1cm4gZXJyOw0KKw0KKwlwaHktPm5lZ290aWF0ZS5hZHZlcnRp c2UgPSBhZHZlcnRpc2U7DQorCXBoeS0+bmVnb3RpYXRlLmF1dG9uZWcgPSBBVVRPTkVHX0VOQUJM RTsNCisJcGh5LT5uZWdvdGlhdGUudGltZW91dCA9IGppZmZpZXMgKyBNSUlfVElNRU9VVDsNCisJ cGh5LT5zdGF0ZS5hdXRvbmVnID0gMTsNCisNCisJcmV0dXJuIDA7DQorfQ0KKw0KKyNkZWZpbmUg QlJJRUZfTUlJX0VSUk9SUw0KK0VYUE9SVF9TWU1CT0wocGh5X2dlbl9wb2xsKTsNCisvKiBXYWl0 IGZvciBhdXRvLW5lZ290aWF0aW9uIHRvIGNvbXBsZXRlICovDQoraW50IHBoeV9nZW5fcG9sbChz dHJ1Y3QgcGh5X2luZm8gKnBoeSkNCit7DQorCXN0cnVjdCBwaHlfc3RhdGUgKnBzdGF0ZTsNCisJ dWludDE2X3QgdmFsOw0KKw0KKwlwc3RhdGUgPSAmcGh5LT5zdGF0ZTsNCisNCisJcGh5X3JlYWQo cGh5LCBNSUlfQk1TUik7CS8qIER1bW15IHJlYWQgKi8NCisJdmFsID0gcGh5X3JlYWQocGh5LCBN SUlfQk1TUik7DQorDQorCS8qIElmIHRoZSBsaW5rIGp1c3QgY2FtZSB1cCwgcmVzdGFydCB0aGUg YXV0by1uZWcgcHJvY2VkdXJlDQorCSAqLw0KKwlpZiAodmFsICYgQk1TUl9MU1RBVFVTKSB7DQor CQlpZiAocHN0YXRlLT5saW5rID09IDAgJiYgDQorCQkgICAgcHN0YXRlLT5hdXRvbmVnID09IDAg JiYgcGh5LT5uZWdvdGlhdGUuYXV0b25lZykgew0KKwkJCXBoeV9zZXRfYXV0b25lZyhwaHksIHBo eS0+bmVnb3RpYXRlLmFkdmVydGlzZSk7DQorCQkJcmV0dXJuIC1FQUdBSU47DQorCQl9DQorCQlw c3RhdGUtPmxpbmsgPSAxOw0KKwl9IGVsc2Ugew0KKwkJcHN0YXRlLT5saW5rID0gMDsNCisJfQ0K Kw0KKwkvKiBPbmx5IGF1dG8tbmVnb3RpYXRlIGlmIHRoZSBsaW5rIGhhcyBqdXN0IGdvbmUgdXAg Ki8NCisJaWYgKHBzdGF0ZS0+bGluayAmJiBwc3RhdGUtPmF1dG9uZWcgJiYgDQorCSAgICAodGlt ZV9hZnRlcihqaWZmaWVzLHBoeS0+bmVnb3RpYXRlLnRpbWVvdXQpIHx8IA0KKwkgICAgICh2YWwg JiBCTVNSX0FORUdDT01QTEVURSkpKSB7DQorI2lmZGVmIEJSSUVGX01JSV9FUlJPUlMNCisJCWlm ICh2YWwgJiBCTVNSX0FORUdDT01QTEVURSkgew0KKwkJCXByaW50ayhLRVJOX0lORk8gIiVzOiBB dXRvLW5lZ290aWF0aW9uIGRvbmVcbiIsDQorCQkJICAgICAgIHBoeS0+bmFtZSk7DQorCQl9IGVs c2Ugew0KKwkJCXByaW50ayhLRVJOX0lORk8NCisJCQkgICAgICAgIiVzOiBBdXRvLW5lZ290aWF0 aW9uIHRpbWVkIG91dFxuIiwNCisJCQkgICAgICAgcGh5LT5uYW1lKTsNCisJCX0NCisjZW5kaWYN CisNCisJCXBzdGF0ZS0+YXV0b25lZyA9IDA7DQorDQorCQlpZiAodmFsICYgQk1TUl9BTkVHQ09N UExFVEUpIHsNCisJCQl2YWwgPSBwaHlfcmVhZChwaHksIE1JSV9MUEEpOw0KKwkJCXZhbCAmPSBw aHlfcmVhZChwaHksIE1JSV9BRFZFUlRJU0UpOw0KKw0KKwkJCS8qIEFjY29yZGluZyB0byBJRUVF IDgwMi4zLCBMUEEgZGVjaXNpb25zDQorCQkJICogbXVzdCBiZSBkb25lIGluIHRoaXMgb3JkZXIN CisJCQkgKi8NCisJCQlpZiAodmFsICYgTFBBXzEwMEZVTEwpIHsNCisJCQkJcHN0YXRlLT5zcGVl ZCA9IFNQRUVEXzEwMDsNCisJCQkJcHN0YXRlLT5kdXBsZXggPSBEVVBMRVhfRlVMTDsNCisJCQl9 IGVsc2UgaWYgKHZhbCAmIExQQV8xMDBIQUxGKSB7DQorCQkJCXBzdGF0ZS0+c3BlZWQgPSBTUEVF RF8xMDA7DQorCQkJCXBzdGF0ZS0+ZHVwbGV4ID0gRFVQTEVYX0hBTEY7DQorCQkJfSBlbHNlIGlm ICh2YWwgJiBMUEFfMTBGVUxMKSB7DQorCQkJCXBzdGF0ZS0+c3BlZWQgPSBTUEVFRF8xMDsNCisJ CQkJcHN0YXRlLT5kdXBsZXggPSBEVVBMRVhfRlVMTDsNCisJCQl9IGVsc2UgaWYgKHZhbCAmIExQ QV8xMEhBTEYpIHsNCisJCQkJcHN0YXRlLT5zcGVlZCA9IFNQRUVEXzEwOw0KKwkJCQlwc3RhdGUt PmR1cGxleCA9IERVUExFWF9IQUxGOw0KKwkJCX0gZWxzZSB7DQorCQkJCXBzdGF0ZS0+c3BlZWQg PSBTUEVFRF8xMDsNCisJCQkJcHN0YXRlLT5kdXBsZXggPSBEVVBMRVhfSEFMRjsNCisJCQl9DQor CQl9DQorCX0NCisNCisJcmV0dXJuIChwc3RhdGUtPmF1dG9uZWcgPyAtRUFHQUlOIDogMCk7DQor fQ0KKw0KK3N0YXRpYyBzdHJ1Y3QgcGh5X29wcyBnZW5fb3BzID0gew0KKwkuc2V0X2F1dG9uZWcg PSBwaHlfZ2VuX3NldF9hdXRvbmVnLA0KKwkucG9sbCA9IHBoeV9nZW5fcG9sbA0KK307DQorDQor c3RhdGljIHN0cnVjdCBwaHlfaW5mbyBwaHlfaW5mb19nZW5lcmljID0gew0KKwkuaWQgPSAweDAs DQorCS5uYW1lID0gIkdlbmVyaWMgUEhZIiwNCisJLnNoaWZ0ID0gMzIsDQorCS5vcHMgPSAmZ2Vu X29wcw0KK307DQorDQorc3RhdGljIExJU1RfSEVBRChwaHlfbGlzdCk7DQorDQorLyogVXNlIHRo ZSBQSFkgSUQgcmVnaXN0ZXJzIHRvIGRldGVybWluZSB3aGF0IHR5cGUgb2YgUEhZIGlzIGF0dGFj aGVkDQorICogdG8gZGV2aWNlIGRldi4gIHJldHVybiBhIHN0cnVjdCBwaHlfaW5mbyBzdHJ1Y3R1 cmUgZGVzY3JpYmluZyB0aGF0IFBIWQ0KKyAqLw0KK3N0cnVjdCBwaHlfaW5mbyAqbWlpX3BoeV9n ZXRfaW5mbyhpbnQgYnVzLCBpbnQgaWQpDQorew0KKwlzdHJ1Y3QgbGlzdF9oZWFkICpscDsNCisJ dWludDE2X3QgcGh5X3JlZzsNCisJdWludDMyX3QgcGh5X2lkOw0KKwlzdHJ1Y3QgcGh5X2luZm8g KmluZm8gPSBOVUxMOw0KKw0KKwlpZiAobWlpX2J1c1tidXNdID09IE5VTEwpDQorCQlyZXR1cm4g TlVMTDsNCisNCisJLyogR3JhYiB0aGUgYml0cyBmcm9tIFBIWUlSMSwgYW5kIHB1dCB0aGVtIGlu IHRoZSB1cHBlciBoYWxmICovDQorCXBoeV9yZWcgPSBtaWlfYnVzX3JlYWQoYnVzLCBpZCwgTUlJ X1BIWVNJRDEpOw0KKwlwaHlfaWQgPSAocGh5X3JlZyAmIDB4ZmZmZikgPDwgMTY7DQorDQorCS8q IEdyYWIgdGhlIGJpdHMgZnJvbSBQSFlJUjIsIGFuZCBwdXQgdGhlbSBpbiB0aGUgbG93ZXIgaGFs ZiAqLw0KKwlwaHlfcmVnID0gbWlpX2J1c19yZWFkKGJ1cywgaWQsIE1JSV9QSFlTSUQyKTsNCisJ cGh5X2lkIHw9IChwaHlfcmVnICYgMHhmZmZmKTsNCisNCisJLyogbG9vcCB0aHJvdWdoIGFsbCB0 aGUga25vd24gUEhZIHR5cGVzLCBhbmQgZmluZCBvbmUgdGhhdA0KKwkgKiBtYXRjaGVzIHRoZSBJ RCB3ZSByZWFkIGZyb20gdGhlIFBIWS4gKi8NCisJbGlzdF9mb3JfZWFjaChscCwgJnBoeV9saXN0 KSB7DQorCQlzdHJ1Y3QgcGh5X2luZm8gKnBoeSA9IGxpc3RfZW50cnkobHAsIHN0cnVjdCBwaHlf aW5mbywgbGlzdCk7DQorCQlpZiAoKHBoeS0+aWQgPj4gcGh5LT5zaGlmdCkgPT0gKHBoeV9pZCA+ PiBwaHktPnNoaWZ0KSkgew0KKwkJCWluZm8gPSBwaHk7DQorCQkJYnJlYWs7DQorCQl9DQorCX0N CisNCisJaWYgKGluZm8gPT0gTlVMTCkgew0KKwkJcHJpbnRrKEtFUk5fV0FSTklORw0KKwkJICAg ICAgICJwaHklZC4lZDogUEhZIGlkIDB4JXggaXMgbm90IHN1cHBvcnRlZCFcbiIsIGJ1cywgaWQs DQorCQkgICAgICAgcGh5X2lkKTsNCisJfSBlbHNlIHsNCisJCXByaW50ayhLRVJOX0lORk8gInBo eSVkLiVkOiBQSFkgaXMgJXMgKCV4KVxuIiwgYnVzLCBpZCwNCisJCSAgICAgICBpbmZvLT5uYW1l LCBwaHlfaWQpOw0KKwl9DQorDQorCXJldHVybiBpbmZvOw0KK30NCisNCitzdGF0aWMgaW50IG1k aW9fcmVhZChzdHJ1Y3QgbmV0X2RldmljZSAqZGV2LCBpbnQgcGh5X2lkLCBpbnQgcmVnKQ0KK3sN CisJcmV0dXJuIG1paV9idXNfcmVhZChNSUlfQlVTKHBoeV9pZCksIE1JSV9JRChwaHlfaWQpLCBy ZWcpOw0KK30NCisNCitzdGF0aWMgdm9pZCBtZGlvX3dyaXRlKHN0cnVjdCBuZXRfZGV2aWNlICpk ZXYsIGludCBwaHlfaWQsIGludCByZWcsIGludCB2YWwpDQorew0KKwltaWlfYnVzX3dyaXRlKE1J SV9CVVMocGh5X2lkKSwgTUlJX0lEKHBoeV9pZCksIHJlZywgdmFsICYgMHhmZmZmKTsNCit9DQor DQorc3RhdGljIGlubGluZSB2b2lkIG1paV9waHlfaXJxX2FjayhzdHJ1Y3QgbWlpX2lmX2luZm8g Km1paSkNCit7DQorCXN0cnVjdCBwaHlfaW5mbyAqcGh5ID0gbWlpX3BoeV9vZihtaWkpOw0KKw0K KwlwaHktPm9wcy0+aW50X2FjayhwaHkpOw0KK30NCisNCitzdGF0aWMgaXJxcmV0dXJuX3QgbWlp X3BoeV9pcnEoaW50IGlycSwgdm9pZCAqZGF0YSwgc3RydWN0IHB0X3JlZ3MgKnJlZ3MpDQorew0K KwlzdHJ1Y3QgbWlpX2lmX2luZm8gKm1paSA9ICh2b2lkICopZGF0YTsNCisJc3RydWN0IHBoeV9p bmZvICpwaHkgPSBtaWlfcGh5X29mKG1paSk7DQorDQorCW1paV9waHlfaXJxX2FjayhtaWkpOw0K Kw0KKwkvKiBTY2hlZHVsZSB0aGUgYm90dG9tIGhhbGYgKi8NCisJc2NoZWR1bGVfd29yaygmcGh5 LT5kZWx0YS50cSk7DQorDQorCXJldHVybiBJUlFfSEFORExFRDsNCit9DQorDQorRVhQT1JUX1NZ TUJPTChtaWlfcGh5X2lycV9lbmFibGUpOw0KK2ludCBtaWlfcGh5X2lycV9lbmFibGUoc3RydWN0 IG1paV9pZl9pbmZvICptaWksIGludCBpcnEsIHZvaWQgKCpmdW5jKSAodm9pZCAqKSwNCisJCSAg ICAgICB2b2lkICpkYXRhKQ0KK3sNCisJc3RydWN0IHBoeV9pbmZvICpwaHkgPSBtaWlfcGh5X29m KG1paSk7DQorCWludCBlcnI7DQorDQorCWlmIChwaHkgPT0gTlVMTCkNCisJCXJldHVybiAtRUlO VkFMOw0KKw0KKwlpZiAocGh5LT5kZWx0YS5kYXRhICE9IE5VTEwpDQorCQlyZXR1cm4gLUVCVVNZ Ow0KKw0KKwlpZiAocGh5LT5vcHMtPmludF9hY2sgPT0gTlVMTCB8fA0KKwkgICAgcGh5LT5vcHMt PmludF9lbmFibGUgPT0gTlVMTCB8fA0KKwkgICAgcGh5LT5vcHMtPmludF9kaXNhYmxlID09IE5V TEwpDQorCQlyZXR1cm4gLUVJTlZBTDsNCisNCisJcGh5LT5kZWx0YS5pcnEgPSBpcnE7DQorCXBo eS0+ZGVsdGEuZnVuYyA9IGZ1bmM7DQorCXBoeS0+ZGVsdGEuZGF0YSA9IGRhdGE7DQorDQorCWVy ciA9IHJlcXVlc3RfaXJxKGlycSwgbWlpX3BoeV9pcnEsIFNBX1NISVJRLCBwaHktPm5hbWUsIG1p aSk7DQorCWlmIChlcnIgPCAwKSB7DQorCQlwaHktPmRlbHRhLmlycSA9IC0xOw0KKwkJcGh5LT5k ZWx0YS5mdW5jID0gTlVMTDsNCisJCXBoeS0+ZGVsdGEuZGF0YSA9IE5VTEw7DQorCQlyZXR1cm4g ZXJyOw0KKwl9DQorDQorCXBoeS0+b3BzLT5pbnRfZW5hYmxlKHBoeSk7DQorCXJldHVybiAwOw0K K30NCisNCitFWFBPUlRfU1lNQk9MKG1paV9waHlfaXJxX2Rpc2FibGUpOw0KK3ZvaWQgbWlpX3Bo eV9pcnFfZGlzYWJsZShzdHJ1Y3QgbWlpX2lmX2luZm8gKm1paSwgdm9pZCAqZGF0YSkNCit7DQor CXN0cnVjdCBwaHlfaW5mbyAqcGh5ID0gbWlpX3BoeV9vZihtaWkpOw0KKw0KKwlpZiAocGh5ID09 IE5VTEwgfHwgcGh5LT5kZWx0YS5kYXRhICE9IGRhdGEpDQorCQlyZXR1cm47DQorDQorCXBoeS0+ b3BzLT5pbnRfZGlzYWJsZShwaHkpOw0KKw0KKwlmcmVlX2lycShwaHktPmRlbHRhLmlycSwgbWlp KTsNCisJcGh5LT5kZWx0YS5pcnEgPSAtMTsNCisJcGh5LT5kZWx0YS5mdW5jID0gTlVMTDsNCisJ cGh5LT5kZWx0YS5kYXRhID0gTlVMTDsNCit9DQorDQorLyogU2NoZWR1bGVkIGJ5IHRoZSB0YXNr IHF1ZXVlICovDQorc3RhdGljIHZvaWQgbWlpX3BoeV9kZWx0YSh2b2lkICpkYXRhKQ0KK3sNCisJ c3RydWN0IG1paV9pZl9pbmZvICptaWkgPSAodm9pZCAqKWRhdGE7DQorCXN0cnVjdCBwaHlfaW5m byAqcGh5ID0gbWlpX3BoeV9vZihtaWkpOw0KKwlzdHJ1Y3QgcGh5X3N0YXRlIG9sZDsNCisNCisJ b2xkPXBoeS0+c3RhdGU7DQorDQorCXBoeS0+b3BzLT5wb2xsKHBoeSk7DQorDQorCWlmIChtZW1j bXAoJm9sZCwmcGh5LT5zdGF0ZSxzaXplb2Yob2xkKSkgIT0gMCAmJiBwaHktPmRlbHRhLmZ1bmMp DQorCQlwaHktPmRlbHRhLmZ1bmMocGh5LT5kZWx0YS5kYXRhKTsNCit9DQorDQorc3RhdGljIHZv aWQgbWlpX3BoeV9wb2xsKHVuc2lnbmVkIGxvbmcgZGF0YSkNCit7DQorCXN0cnVjdCBtaWlfaWZf aW5mbyAqbWlpID0gKHZvaWQgKilkYXRhOw0KKwlzdHJ1Y3QgcGh5X2luZm8gKnBoeSA9IG1paV9w aHlfb2YobWlpKTsNCisNCisJc2NoZWR1bGVfd29yaygmcGh5LT5kZWx0YS50cSk7DQorDQorCW1v ZF90aW1lcigmcGh5LT5kZWx0YS50aW1lciwgamlmZmllcyArIEhaICogcGh5LT5kZWx0YS5tc2Vj cyAvIDEwMDApOw0KK30NCisNCitFWFBPUlRfU1lNQk9MKG1paV9waHlfcG9sbF9lbmFibGUpOw0K K2ludCBtaWlfcGh5X3BvbGxfZW5hYmxlKHN0cnVjdCBtaWlfaWZfaW5mbyAqbWlpLCB1bnNpZ25l ZCBsb25nIG1zZWNzLA0KKwkJCXZvaWQgKCpmdW5jKSAodm9pZCAqKSwgdm9pZCAqZGF0YSkNCit7 DQorCXN0cnVjdCBwaHlfaW5mbyAqcGh5ID0gbWlpX3BoeV9vZihtaWkpOw0KKw0KKwlpZiAocGh5 ID09IE5VTEwpDQorCQlyZXR1cm4gLUVJTlZBTDsNCisNCisJaWYgKEhaICogbXNlY3MgLyAxMDAw IDw9IDAgfHwgZnVuYyA9PSBOVUxMKQ0KKwkJcmV0dXJuIC1FSU5WQUw7DQorDQorCWlmIChwaHkt PmRlbHRhLmRhdGEgIT0gTlVMTCkNCisJCXJldHVybiAtRUlOVkFMOw0KKw0KKwlpbml0X3RpbWVy KCZwaHktPmRlbHRhLnRpbWVyKTsNCisJcGh5LT5kZWx0YS50aW1lci5mdW5jdGlvbiA9IG1paV9w aHlfcG9sbDsNCisJcGh5LT5kZWx0YS50aW1lci5kYXRhID0gKHVuc2lnbmVkIGxvbmcpbWlpOw0K KwlwaHktPmRlbHRhLmRhdGEgPSBkYXRhOw0KKwlwaHktPmRlbHRhLmZ1bmMgPSBmdW5jOw0KKwlw aHktPmRlbHRhLm1zZWNzID0gbXNlY3M7DQorCW1vZF90aW1lcigmcGh5LT5kZWx0YS50aW1lciwg amlmZmllcyArIEhaICogbXNlY3MgLyAxMDAwKTsNCisJc2NoZWR1bGVfd29yaygmcGh5LT5kZWx0 YS50cSk7DQorDQorCXJldHVybiAwOw0KK30NCisNCitFWFBPUlRfU1lNQk9MKG1paV9waHlfcG9s bF9kaXNhYmxlKTsNCit2b2lkIG1paV9waHlfcG9sbF9kaXNhYmxlKHN0cnVjdCBtaWlfaWZfaW5m byAqbWlpLCB2b2lkICpkYXRhKQ0KK3sNCisJc3RydWN0IHBoeV9pbmZvICpwaHkgPSBtaWlfcGh5 X29mKG1paSk7DQorDQorCWlmIChwaHkgPT0gTlVMTCB8fCBwaHktPmRlbHRhLmRhdGEgPT0gTlVM TCkNCisJCXJldHVybjsNCisNCisJZGVsX3RpbWVyX3N5bmMoJnBoeS0+ZGVsdGEudGltZXIpOw0K KwlwaHktPmRlbHRhLmZ1bmMgPSBOVUxMOw0KKwlwaHktPmRlbHRhLmRhdGEgPSBOVUxMOw0KK30N CisNCitFWFBPUlRfU1lNQk9MKHBoeV9nZW5fc2V0X2F1dG9uZWcpOw0KK2ludCBwaHlfZ2VuX3Nl dF9hdXRvbmVnKHN0cnVjdCBwaHlfaW5mbyAqcGh5LCB1aW50MzJfdCBhZHZlcnRpc2UpDQorew0K Kwl1aW50MTZfdCBhZHYsIGN0bDsNCisNCisJYWR2ID0gcGh5X3JlYWQocGh5LCBNSUlfQURWRVJU SVNFKTsNCisJYWR2ICY9IH4oQURWRVJUSVNFX0FMTCB8IEFEVkVSVElTRV8xMDBCQVNFNCk7DQor CWlmIChhZHZlcnRpc2UgJiBBRFZFUlRJU0VEXzEwYmFzZVRfSGFsZikNCisJCWFkdiB8PSBBRFZF UlRJU0VfMTBIQUxGOw0KKwlpZiAoYWR2ZXJ0aXNlICYgQURWRVJUSVNFRF8xMGJhc2VUX0Z1bGwp DQorCQlhZHYgfD0gQURWRVJUSVNFXzEwRlVMTDsNCisJaWYgKGFkdmVydGlzZSAmIEFEVkVSVElT RURfMTAwYmFzZVRfSGFsZikNCisJCWFkdiB8PSBBRFZFUlRJU0VfMTAwSEFMRjsNCisJaWYgKGFk dmVydGlzZSAmIEFEVkVSVElTRURfMTAwYmFzZVRfRnVsbCkNCisJCWFkdiB8PSBBRFZFUlRJU0Vf MTAwRlVMTDsNCisNCisJLyogQ29uZmlndXJlIHNvbWUgYmFzaWMgc3R1ZmYgKi8NCisJcGh5X3dy aXRlKHBoeSwgTUlJX0FEVkVSVElTRSwgYWR2KTsNCisNCisJLyogU3RhcnQgYXV0byBuZWdvdGlh dGlvbiAqLw0KKwljdGwgPSBwaHlfcmVhZChwaHksIE1JSV9CTUNSKTsNCisJY3RsIHw9IChCTUNS X0FORU5BQkxFIHwgQk1DUl9BTlJFU1RBUlQpOw0KKwlwaHlfd3JpdGUocGh5LCBNSUlfQk1DUiwg Y3RsKTsNCisNCisJcmV0dXJuIDA7DQorfQ0KKw0KKw0KK0VYUE9SVF9TWU1CT0wobWlpX3BoeV9h dHRhY2gpOw0KK2ludCBtaWlfcGh5X2F0dGFjaChzdHJ1Y3QgbWlpX2lmX2luZm8gKm1paSwgc3Ry dWN0IG5ldF9kZXZpY2UgKmRldiwgaW50IGJ1cywNCisJCSAgIGludCBpZCkNCit7DQorCXN0cnVj dCBwaHlfaW5mbyAqcGh5LCAqaW5mbzsNCisNCisJaWYgKG1paV9idXNbYnVzXSA9PSBOVUxMKSB7 DQorCQlwcmludGsoS0VSTl9FUlINCisJCSAgICAgICAibWlpX3BoeV9hdHRhY2g6IENhbid0IGF0 dGFjaCAlcywgbm8gTUlJIGJ1cyAlZCBwcmVzZW50XG4iLA0KKwkJICAgICAgIGRldi0+bmFtZSwg YnVzKTsNCisJCXJldHVybiAtRU5PREVWOw0KKwl9DQorDQorCWlmIChtaWlfYnVzW2J1c10tPnBo eVtpZF0gIT0gTlVMTCkgew0KKwkJcHJpbnRrKEtFUk5fRVJSDQorCQkgICAgICAgIm1paV9waHlf YXR0YWNoOiBwaHklZC4lZCBpcyBhbHJlYWR5IGF0dGFjaGVkIHRvICVzXG4iLA0KKwkJICAgICAg IGJ1cywgaWQsIGRldi0+bmFtZSk7DQorCQlyZXR1cm4gLUVCVVNZOw0KKwl9DQorDQorCWluZm8g PSBtaWlfcGh5X2dldF9pbmZvKGJ1cywgaWQpOw0KKwlpZiAoaW5mbyA9PSBOVUxMKQ0KKwkJcmV0 dXJuIC1FTk9ERVY7DQorDQorCXBoeSA9IGttYWxsb2Moc2l6ZW9mKCpwaHkpLCBHRlBfS0VSTkVM KTsNCisJaWYgKHBoeSA9PSBOVUxMKQ0KKwkJcmV0dXJuIC1FTk9NRU07DQorDQorCW1lbWNweShw aHksIGluZm8sIHNpemVvZigqcGh5KSk7DQorDQorCUlOSVRfV09SSygmcGh5LT5kZWx0YS50cSwg bWlpX3BoeV9kZWx0YSwgbWlpKTsNCisJc25wcmludGYoJnBoeS0+bmFtZVswXSxzaXplb2YocGh5 LT5uYW1lKSwicGh5JWQuJWQiLGJ1cyxpZCk7DQorCXBoeS0+cGh5X2lkID0gTUlJX1BIWV9JRChi dXMsIGlkKTsNCisJcGh5LT5kZWx0YS5mdW5jID0gTlVMTDsNCisJcGh5LT5kZWx0YS5kYXRhID0g TlVMTDsNCisJcGh5LT5kZWx0YS5pcnEgPSAtMTsNCisJcGh5LT5zdGF0ZS5saW5rID0gMDsNCisJ cGh5LT5zdGF0ZS5kdXBsZXggPSBEVVBMRVhfSEFMRjsNCisJcGh5LT5zdGF0ZS5zcGVlZCA9IFNQ RUVEXzEwOw0KKwlwaHktPm5lZ290aWF0ZS5hdXRvbmVnID0gMDsNCisJcGh5LT5uZWdvdGlhdGUu YWR2ZXJ0aXNlID0gMDsNCisNCisJbWVtc2V0KG1paSwgMCwgc2l6ZW9mKCptaWkpKTsNCisJbWlp LT5waHlfaWQgPSAoYnVzIDw8IDUpIHwgaWQ7DQorCW1paS0+cGh5X2lkX21hc2sgPSAweGZmOw0K KwltaWktPnJlZ19udW1fbWFzayA9IDB4MWY7DQorCW1paS0+ZGV2ID0gZGV2Ow0KKwltaWktPm1k aW9fcmVhZCA9IG1kaW9fcmVhZDsNCisJbWlpLT5tZGlvX3dyaXRlID0gbWRpb193cml0ZTsNCisN CisJbWlpX2J1c1tidXNdLT5waHlbaWRdID0gcGh5Ow0KKw0KKwlpZiAocGh5LT5vcHMtPmluaXQg IT0gTlVMTCkNCisJCXBoeS0+b3BzLT5pbml0KHBoeSk7DQorCXJldHVybiAwOw0KK30NCisNCitF WFBPUlRfU1lNQk9MKG1paV9waHlfZGV0YWNoKTsNCit2b2lkIG1paV9waHlfZGV0YWNoKHN0cnVj dCBtaWlfaWZfaW5mbyAqbWlpKQ0KK3sNCisJc3RydWN0IHBoeV9pbmZvICpwaHkgPSBtaWlfcGh5 X29mKG1paSk7DQorCXN0cnVjdCBtaWlfYnVzICpwYnVzOw0KKw0KKwlpZiAocGh5ID09IE5VTEwp DQorCQlyZXR1cm47DQorDQorCXBidXMgPSBtaWlfYnVzW01JSV9CVVMocGh5LT5waHlfaWQpXTsN CisNCisJaWYgKHBoeS0+ZGVsdGEuZGF0YSAhPSBOVUxMKSB7DQorCQlpZiAocGh5LT5kZWx0YS5p cnEgPCAwKQ0KKwkJCW1paV9waHlfcG9sbF9kaXNhYmxlKG1paSwgcGh5LT5kZWx0YS5kYXRhKTsN CisJCWVsc2UNCisJCQltaWlfcGh5X2lycV9kaXNhYmxlKG1paSwgcGh5LT5kZWx0YS5kYXRhKTsN CisJfQ0KKw0KKwlwYnVzLT5waHlbTUlJX0lEKHBoeS0+cGh5X2lkKV0gPSBOVUxMOw0KKwlrZnJl ZShwaHkpOw0KK30NCisNCitFWFBPUlRfU1lNQk9MKG1paV9waHlfc3RhdGUpOw0KK2ludCBtaWlf cGh5X3N0YXRlKHN0cnVjdCBtaWlfaWZfaW5mbyAqbWlpLCBzdHJ1Y3QgcGh5X3N0YXRlICpzdGF0 ZSkNCit7DQorCXN0cnVjdCBwaHlfaW5mbyAqcGh5ID0gbWlpX3BoeV9vZihtaWkpOw0KKwlpbnQg ZXJyID0gMDsNCisNCisJaWYgKHBoeSA9PSBOVUxMKQ0KKwkJcmV0dXJuIC1FSU5WQUw7DQorDQor CWlmIChwaHktPmRlbHRhLmZ1bmMgPT0gTlVMTCkNCisJCWVyciA9IHBoeS0+b3BzLT5wb2xsKHBo eSk7DQorDQorCW1lbWNweShzdGF0ZSwgJnBoeS0+c3RhdGUsIHNpemVvZigqc3RhdGUpKTsNCisN CisJcmV0dXJuIGVycjsNCit9DQorDQorRVhQT1JUX1NZTUJPTChtaWlfcGh5X3NldF9hdXRvbmVn KTsNCitpbnQgbWlpX3BoeV9zZXRfYXV0b25lZyhzdHJ1Y3QgbWlpX2lmX2luZm8gKm1paSwgdWlu dDMyX3QgYWR2ZXJ0aXNlKQ0KK3sNCisJc3RydWN0IHBoeV9pbmZvICpwaHkgPSBtaWlfcGh5X29m KG1paSk7DQorDQorCWlmIChwaHkgPT0gTlVMTCB8fCBwaHktPm9wcy0+c2V0X2F1dG9uZWcgPT0g TlVMTCkNCisJCXJldHVybiAtRUlOVkFMOw0KKw0KKwlyZXR1cm4gcGh5X3NldF9hdXRvbmVnKHBo eSwgYWR2ZXJ0aXNlKTsNCit9DQorDQorRVhQT1JUX1NZTUJPTChtaWlfcGh5X3NldF9mb3JjZWQp Ow0KK2ludCBtaWlfcGh5X3NldF9mb3JjZWQoc3RydWN0IG1paV9pZl9pbmZvICptaWksIGludCBz cGVlZCwgaW50IGR1cGxleCkNCit7DQorCXN0cnVjdCBwaHlfaW5mbyAqcGh5ID0gbWlpX3BoeV9v ZihtaWkpOw0KKwlpbnQgZXJyID0gMDsNCisNCisJaWYgKHBoeSA9PSBOVUxMKQ0KKwkJcmV0dXJu IC1FSU5WQUw7DQorDQorCWlmIChwaHktPm9wcy0+c2V0X2ZvcmNlZCkNCisJCWVyciA9IHBoeS0+ b3BzLT5zZXRfZm9yY2VkKHBoeSwgc3BlZWQsIGR1cGxleCk7DQorDQorCWlmIChlcnIgPCAwKQ0K KwkJcmV0dXJuIGVycjsNCisNCisJcGh5LT5uZWdvdGlhdGUuYXV0b25lZyA9IEFVVE9ORUdfRElT QUJMRTsNCisJcGh5LT5zdGF0ZS5zcGVlZCA9IHNwZWVkOw0KKwlwaHktPnN0YXRlLmR1cGxleCA9 IGR1cGxleDsNCisJcGh5LT5zdGF0ZS5hdXRvbmVnID0gMDsNCisNCisJcmV0dXJuIDA7DQorfQ0K Kw0KK3N0YXRpYyBERUNMQVJFX01VVEVYKG1paV9idXNfbG9jayk7DQorDQorRVhQT1JUX1NZTUJP TChtaWlfYnVzX3JlZ2lzdGVyKTsNCitpbnQgbWlpX2J1c19yZWdpc3RlcihzdHJ1Y3QgbWlpX2J1 cyAqYnVzKQ0KK3sNCisJaW50IGJ1c19pZDsNCisNCisJaWYgKGJ1cyA9PSBOVUxMIHx8IGJ1cy0+ bmFtZSA9PSBOVUxMIHx8IGJ1cy0+cmVhZCA9PSBOVUxMIHx8DQorCSAgICBidXMtPndyaXRlID09 IE5VTEwpDQorCQlyZXR1cm4gLUVJTlZBTDsNCisNCisJZG93bigmbWlpX2J1c19sb2NrKTsNCisN CisJZm9yIChidXNfaWQgPSAwOyBidXNfaWQgPCBNSUlfQlVTX01BWDsgYnVzX2lkKyspIHsNCisJ CWlmIChtaWlfYnVzW2J1c19pZF0gPT0gTlVMTCkNCisJCQlicmVhazsNCisJfQ0KKw0KKwlpZiAo YnVzX2lkID49IE1JSV9CVVNfTUFYKSB7DQorCQlidXNfaWQgPSAtRU5PTUVNOw0KKwkJZ290byBl bmQ7DQorCX0NCisNCisJbWlpX2J1c1tidXNfaWRdID0gYnVzOw0KKw0KKwlpZiAoYnVzLT5yZXNl dCkNCisJCWJ1cy0+cmVzZXQoYnVzLT5wcml2KTsNCisNCisJcHJpbnRrKEtFUk5fSU5GTyAiJXM6 IHJlZ2lzdGVyZWQgYXMgUEhZIGJ1cyAlZFxuIiwgYnVzLT5uYW1lLCBidXNfaWQpOw0KKw0KKyAg ICAgIGVuZDoNCisJdXAoJm1paV9idXNfbG9jayk7DQorDQorCXJldHVybiBidXNfaWQ7DQorfQ0K Kw0KK0VYUE9SVF9TWU1CT0wobWlpX2J1c191bnJlZ2lzdGVyKTsNCit2b2lkIG1paV9idXNfdW5y ZWdpc3RlcihzdHJ1Y3QgbWlpX2J1cyAqYnVzKQ0KK3sNCisJaW50IGk7DQorDQorCWRvd24oJm1p aV9idXNfbG9jayk7DQorDQorCWZvciAoaSA9IDA7IGkgPCBNSUlfQlVTX01BWDsgaSsrKSB7DQor CQlpZiAobWlpX2J1c1tpXSA9PSBidXMpIHsNCisJCQltaWlfYnVzW2ldID0gTlVMTDsNCisJCQli cmVhazsNCisJCX0NCisJfQ0KKw0KKwl1cCgmbWlpX2J1c19sb2NrKTsNCit9DQorDQorLyogSW5z ZXJ0IGludG8gJ3BoeV9saXN0JyBzb3J0ZWQgYnkNCisgKiBzaGlmdCAoc21hbGxlc3QgdG8gbGFy Z2VzdCkNCisgKi8NCitFWFBPUlRfU1lNQk9MKHBoeV9yZWdpc3Rlcik7DQoraW50IHBoeV9yZWdp c3RlcihzdHJ1Y3QgcGh5X2luZm8gKmluZm8pDQorew0KKwlzdHJ1Y3QgbGlzdF9oZWFkICpscDsN CisNCisJaWYgKGluZm89PU5VTEwgfHwgaW5mby0+b3BzID09IE5VTEwgfHwgaW5mby0+b3BzLT5w b2xsID09IE5VTEwpDQorCQlyZXR1cm4gLUVJTlZBTDsNCisNCisJbGlzdF9mb3JfZWFjaChscCwg JnBoeV9saXN0KSB7DQorCQlzdHJ1Y3QgcGh5X2luZm8gKnBoeSA9IGxpc3RfZW50cnkobHAsIHN0 cnVjdCBwaHlfaW5mbywgbGlzdCk7DQorCQlpZiAocGh5LT5zaGlmdCA+IGluZm8tPnNoaWZ0KQ0K KwkJCWJyZWFrOw0KKw0KKwkJLyogQ2hlY2sgZm9yIGR1cGxpY2F0ZXMgKi8NCisJCWlmICgocGh5 LT5zaGlmdD09aW5mby0+c2hpZnQpICYmIChpbmZvLT5pZCA9PSBwaHktPmlkKSkNCisJCQlyZXR1 cm4gLUVCVVNZOw0KKwl9DQorDQorCS8qIFRoaXMgZG9lcyB0aGUgJ3JpZ2h0IHRoaW5nJyBldmVu IGlmIGxwID09ICZwaHlfbGlzdA0KKwkgKi8NCisJbGlzdF9hZGRfdGFpbCgmaW5mby0+bGlzdCwg bHApOw0KKw0KKwlyZXR1cm4gMDsNCit9DQorDQorRVhQT1JUX1NZTUJPTChwaHlfdW5yZWdpc3Rl cik7DQordm9pZCBwaHlfdW5yZWdpc3RlcihzdHJ1Y3QgcGh5X2luZm8gKmluZm8pDQorew0KKwls aXN0X2RlbF9pbml0KCZpbmZvLT5saXN0KTsNCit9DQorDQorc3RhdGljIGludCBtaWlfYnVzX2lu aXQodm9pZCkNCit7DQorCXJldHVybiBwaHlfcmVnaXN0ZXIoJnBoeV9pbmZvX2dlbmVyaWMpOw0K K30NCisNCitzdGF0aWMgdm9pZCBtaWlfYnVzX2V4aXQodm9pZCkNCit7DQorCXBoeV91bnJlZ2lz dGVyKCZwaHlfaW5mb19nZW5lcmljKTsNCit9DQorDQorbW9kdWxlX2luaXQobWlpX2J1c19pbml0 KTsNCittb2R1bGVfZXhpdChtaWlfYnVzX2V4aXQpOw0KDQotLS0gL2Rldi9udWxsDQorKysgbGlu dXgvZHJpdmVycy9uZXQvcGh5X2NpY2FkYS5jDQpAQCAtMCwwICsxLDE3NyBAQA0KKy8qIA0KKyAq IGRyaXZlcnMvbmV0L3BoeV9jaWNhZGEuYw0KKyAqDQorICogQXV0aG9yOiBKYXNvbiBNY011bGxh bg0KKyAqDQorICogQ29weXJpZ2h0IChjKSAyMDA0IFRpbWVzeXMgQ29ycC4NCisgKg0KKyAqIFRo aXMgcHJvZ3JhbSBpcyBmcmVlIHNvZnR3YXJlOyB5b3UgY2FuIHJlZGlzdHJpYnV0ZSAgaXQgYW5k L29yIG1vZGlmeSBpdA0KKyAqIHVuZGVyICB0aGUgdGVybXMgb2YgIHRoZSBHTlUgR2VuZXJhbCAg UHVibGljIExpY2Vuc2UgYXMgcHVibGlzaGVkIGJ5IHRoZQ0KKyAqIEZyZWUgU29mdHdhcmUgRm91 bmRhdGlvbjsgIGVpdGhlciB2ZXJzaW9uIDIgb2YgdGhlICBMaWNlbnNlLCBvciAoYXQgeW91cg0K KyAqIG9wdGlvbikgYW55IGxhdGVyIHZlcnNpb24uDQorICoNCisgKi8NCisjaW5jbHVkZSA8bGlu dXgvbW9kdWxlLmg+DQorI2luY2x1ZGUgPGxpbnV4L2luaXQuaD4NCisjaW5jbHVkZSA8bGludXgv bWlpX2J1cy5oPg0KKw0KKy8qIENpY2FkYSBBdXhpbGlhcnkgQ29udHJvbC9TdGF0dXMgUmVnaXN0 ZXIgKi8NCisjZGVmaW5lIE1JSU1fQ0lTODIwMV9BVVhfQ09OU1RBVCAgICAgICAgMHgxYw0KKyNk ZWZpbmUgTUlJTV9DSVM4MjAxX0FVWENPTlNUQVRfSU5JVCAgICAweDAwMDQNCisjZGVmaW5lIE1J SU1fQ0lTODIwMV9BVVhDT05TVEFUX0RVUExFWCAgMHgwMDIwDQorI2RlZmluZSBNSUlNX0NJUzgy MDFfQVVYQ09OU1RBVF9TUEVFRCAgIDB4MDAxOA0KKyNkZWZpbmUgTUlJTV9DSVM4MjAxX0FVWENP TlNUQVRfR0JJVCAgICAweDAwMTANCisjZGVmaW5lIE1JSU1fQ0lTODIwMV9BVVhDT05TVEFUXzEw MCAgICAgMHgwMDA4DQorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICANCisvKiBDaWNhZGEgRXh0ZW5k ZWQgQ29udHJvbCBSZWdpc3RlciAxICovDQorI2RlZmluZSBNSUlNX0NJUzgyMDFfRVhUX0NPTjEg ICAgICAgICAgIDB4MTcNCisjZGVmaW5lIE1JSU1fQ0lTODIwMV9FWFRDT04xX0lOSVQgICAgICAg MHgwMDAwDQorDQorLyogQ0lTODIwMSAqLw0KKyNkZWZpbmUgTUlJX0NJUzgyMDFfRVBDUiAgICAg ICAgMHgxNw0KKyNkZWZpbmUgRVBDUl9NT0RFX01BU0sgICAgICAgICAgMHgzMDAwDQorI2RlZmlu ZSBFUENSX0dNSUlfTU9ERSAgICAgICAgICAweDAwMDANCisjZGVmaW5lIEVQQ1JfUkdNSUlfTU9E RSAgICAgICAgIDB4MTAwMA0KKyNkZWZpbmUgRVBDUl9UQklfTU9ERSAgICAgICAgICAgMHgyMDAw DQorI2RlZmluZSBFUENSX1JUQklfTU9ERSAgICAgICAgICAweDMwMDANCisNCitzdGF0aWMgaW50 IGNpczgyMDFfaW5pdChzdHJ1Y3QgcGh5X2luZm8gKnBoeSkNCit7DQorCXVpbnQxNl90IGVwY3I7 DQorCWNvbnN0IGNoYXIgKm1vZGUgPSAiVW5rbm93biI7DQorDQorCWVwY3IgPSBwaHlfcmVhZChw aHksIE1JSV9DSVM4MjAxX0VQQ1IpOw0KKw0KKwlzd2l0Y2ggKGVwY3IgJiBFUENSX01PREVfTUFT Sykgew0KKwkJY2FzZSBFUENSX0dNSUlfTU9ERTogbW9kZSA9ICJHTUlJIjsgYnJlYWs7DQorCQlj YXNlIEVQQ1JfUkdNSUlfTU9ERTogbW9kZSA9ICJSR01JSSI7IGJyZWFrOw0KKwkJY2FzZSBFUENS X1RCSV9NT0RFOiBtb2RlID0gIlRCSSI7IGJyZWFrOw0KKwkJY2FzZSBFUENSX1JUQklfTU9ERTog bW9kZSA9ICJSVEJJIjsgYnJlYWs7DQorCX0NCisNCisJcHJpbnRrKEtFUk5fSU5GTyAiJXM6ICVz IG1vZGVcbiIscGh5LT5uYW1lLCBtb2RlKTsNCisNCisJcmV0dXJuIDA7DQorfQ0KKw0KKyNkZWZp bmUgTUlJX0NJUzgyMDFfSU5UUl9DVFJMCTB4MTkNCisjZGVmaW5lIE1JSV9DSVM4MjAxX0lOVFJf U1RBVAkweDFhDQorDQorI2RlZmluZSBNSUlfQ0lTODIwMV9JTlRSX0VOQUJMRQkweDgwMDANCisj ZGVmaW5lIE1JSV9DSVM4MjAxX0lOVFJfU1BFRUQJMHg0MDAwDQorI2RlZmluZSBNSUlfQ0lTODIw MV9JTlRSX0xJTksJMHgyMDAwDQorI2RlZmluZSBNSUlfQ0lTODIwMV9JTlRSX0RVUExFWAkweDEw MDANCisjZGVmaW5lIE1JSV9DSVM4MjAxX0lOVFJfQU5fRVJSCTB4MDgwMA0KKyNkZWZpbmUgTUlJ X0NJUzgyMDFfSU5UUl9BTl9ET04JMHgwNDAwDQorI2RlZmluZSBNSUlfQ0lTODIwMV9JTlRSX0FM TAkweDdjMDANCisNCitzdGF0aWMgaW50IGNpczgyMDFfaW50X2VuYWJsZShzdHJ1Y3QgcGh5X2lu Zm8gKnBoeSkNCit7DQorCXBoeV93cml0ZShwaHksIE1JSV9DSVM4MjAxX0lOVFJfQ1RSTCwgTUlJ X0NJUzgyMDFfSU5UUl9FTkFCTEUgfCBNSUlfQ0lTODIwMV9JTlRSX0FMTCk7DQorDQorCXJldHVy biAwOw0KK30NCisNCitzdGF0aWMgaW50IGNpczgyMDFfaW50X2Rpc2FibGUoc3RydWN0IHBoeV9p bmZvICpwaHkpDQorew0KKwlwaHlfd3JpdGUocGh5LCBNSUlfQ0lTODIwMV9JTlRSX0NUUkwsIDAp Ow0KKw0KKwlyZXR1cm4gMDsNCit9DQorDQorc3RhdGljIGludCBjaXM4MjAxX2ludF9hY2soc3Ry dWN0IHBoeV9pbmZvICpwaHkpDQorew0KKwlwaHlfcmVhZChwaHksIE1JSV9DSVM4MjAxX0lOVFJf U1RBVCk7DQorDQorCXJldHVybiAwOw0KK30NCisNCisjZGVmaW5lCU1JSV9DSVM4MjAxX0FDU1IJ MHgxYw0KKyNkZWZpbmUgIEFDU1JfRU5BQkxFXzEwMDBCQVNFVAkweDAwMDQNCisjZGVmaW5lICBB Q1NSX0RVUExFWF9TVEFUVVMJMHgwMDIwDQorI2RlZmluZSAgQUNTUl9TUEVFRF8xMDAwQkFTRVQJ MHgwMDEwDQorI2RlZmluZSAgQUNTUl9TUEVFRF8xMDBCQVNFVAkweDAwMDgNCisNCitzdGF0aWMg aW50IGNpczgyMDFfcG9sbChzdHJ1Y3QgcGh5X2luZm8gKnBoeSkNCit7DQorCXVpbnQxNl90IGFj c3I7DQorCXN0cnVjdCBwaHlfc3RhdGUgKnBzdGF0ZSA9ICZwaHktPnN0YXRlOw0KKwlpbnQgYXV0 b25lZyA9IHBoeS0+c3RhdGUuYXV0b25lZzsNCisJaW50IGVycjsNCisNCisJZXJyID0gcGh5X2dl bl9wb2xsKHBoeSk7DQorCWlmIChlcnIgPCAwKQ0KKwkJcmV0dXJuIGVycjsNCisNCisJaWYgKHBz dGF0ZS0+bGluayA9PSAwKQ0KKwkJcmV0dXJuIDA7DQorDQorCS8qIFdlIHVzZSB0aGUgb2xkIGNv cHkgb2YgJ3BoeS0+c3RhdGUuYXV0b25lZycNCisJICogYXMgcGh5X2dlbl9wb2xsIHdpbGwgaGF2 ZSBzZXQgaXQgdG8gMA0KKwkgKi8NCisJaWYgKGF1dG9uZWcpIHsNCisJCWFjc3IgPSBwaHlfcmVh ZChwaHksIE1JSV9DSVM4MjAxX0FDU1IpOw0KKw0KKwkJaWYgKGFjc3IgJiBBQ1NSX0RVUExFWF9T VEFUVVMpDQorCQkJcHN0YXRlLT5kdXBsZXggPSBEVVBMRVhfRlVMTDsNCisJCWVsc2UNCisJCQlw c3RhdGUtPmR1cGxleCA9IERVUExFWF9IQUxGOw0KKwkJaWYgKGFjc3IgJiBBQ1NSX1NQRUVEXzEw MDBCQVNFVCkgew0KKwkJCXBzdGF0ZS0+c3BlZWQgPSBTUEVFRF8xMDAwOw0KKwkJfSBlbHNlIGlm IChhY3NyICYgQUNTUl9TUEVFRF8xMDBCQVNFVCkNCisJCQlwc3RhdGUtPnNwZWVkID0gU1BFRURf MTAwOw0KKwkJZWxzZQ0KKwkJCXBzdGF0ZS0+c3BlZWQgPSBTUEVFRF8xMDsNCisJfQ0KKw0KKwkv KiBPbiBub24tYW5lZywgd2UgYXNzdW1lIHdoYXQgd2UgcHV0IGluIEJNQ1IgaXMgdGhlIHNwZWVk LA0KKwkgKiB0aG91Z2ggbWFnaWMtYW5lZyBzaG91bGRuJ3QgcHJldmVudCB0aGlzIGNhc2UgZnJv bSBvY2N1cnJpbmcNCisJICovDQorDQorCXJldHVybiAwOw0KK30NCisNCitzdGF0aWMgaW50IGNp czgyMDFfc2V0X2F1dG9uZWcoc3RydWN0IHBoeV9pbmZvICpwaHksIHVpbnQzMl90IGFkdmVydGlz ZSkNCit7DQorCXVpbnQxNl90IHZhbDsNCisNCisJLyogRG8gdGhlIDEwMDBCVCBzZXR1cCBoZXJl LiAqLw0KKwl2YWwgPSBwaHlfcmVhZChwaHksIE1JSV9DSVM4MjAxX0FDU1IpOw0KKwlpZiAoYWR2 ZXJ0aXNlICYgQURWRVJUSVNFRF8xMDAwYmFzZVRfRnVsbCkNCisJCXBoeV93cml0ZShwaHksIE1J SV9DSVM4MjAxX0FDU1IsIHZhbCB8IEFDU1JfRU5BQkxFXzEwMDBCQVNFVCk7DQorCWVsc2UNCisJ CXBoeV93cml0ZShwaHksIE1JSV9DSVM4MjAxX0FDU1IsIHZhbCAmIH5BQ1NSX0VOQUJMRV8xMDAw QkFTRVQpOw0KKw0KKwlyZXR1cm4gcGh5X2dlbl9zZXRfYXV0b25lZyhwaHksIGFkdmVydGlzZSk7 DQorfQ0KKw0KKw0KK3N0cnVjdCBwaHlfb3BzIHBoeV9vcHNfY2lzODIwMSA9IHsNCisJLmluaXQg CQk9IGNpczgyMDFfaW5pdCwNCisJLnNldF9hdXRvbmVnIAk9IGNpczgyMDFfc2V0X2F1dG9uZWcs DQorCS5wb2xsCQk9IGNpczgyMDFfcG9sbCwNCisJLmludF9lbmFibGUJPSBjaXM4MjAxX2ludF9l bmFibGUsDQorCS5pbnRfZGlzYWJsZQk9IGNpczgyMDFfaW50X2Rpc2FibGUsDQorCS5pbnRfYWNr CT0gY2lzODIwMV9pbnRfYWNrDQorfTsNCisNCisvKiBDaWNhZGEgODIwMSAqLw0KK3N0YXRpYyBz dHJ1Y3QgcGh5X2luZm8gcGh5X2luZm9fY2lzODIwMSA9IHsNCisJLmlkID0gMHgwMDBmYzQ0MCwN CisJLm5hbWUgPSAiQ0lTODIwMSIsDQorCS5zaGlmdCA9IDQsDQorCS5vcHMgPSAmcGh5X29wc19j aXM4MjAxDQorfTsNCisNCitzdGF0aWMgaW50IHBoeV9jaWNhZGFfaW5pdCh2b2lkKQ0KK3sNCisJ cmV0dXJuIHBoeV9yZWdpc3RlcigmcGh5X2luZm9fY2lzODIwMSk7DQorfQ0KKw0KK3N0YXRpYyB2 b2lkIHBoeV9jaWNhZGFfZXhpdCh2b2lkKQ0KK3sNCisJcGh5X3VucmVnaXN0ZXIoJnBoeV9pbmZv X2NpczgyMDEpOw0KK30NCisNCittb2R1bGVfaW5pdChwaHlfY2ljYWRhX2luaXQpOw0KK21vZHVs ZV9leGl0KHBoeV9jaWNhZGFfZXhpdCk7DQoNCi0tLSAvZGV2L251bGwNCisrKyBsaW51eC9kcml2 ZXJzL25ldC9waHlfZGF2aWNvbS5jDQpAQCAtMCwwICsxLDE0MCBAQA0KKy8qIA0KKyAqIGRyaXZl cnMvbmV0L3BoeV9kYXZpY29tLmMNCisgKg0KKyAqIEF1dGhvcjogSmFzb24gTWNNdWxsYW4NCisg Kg0KKyAqIENvcHlyaWdodCAoYykgMjAwNCBUaW1lc3lzIENvcnAuDQorICoNCisgKiBUaGlzIHBy b2dyYW0gaXMgZnJlZSBzb2Z0d2FyZTsgeW91IGNhbiByZWRpc3RyaWJ1dGUgIGl0IGFuZC9vciBt b2RpZnkgaXQNCisgKiB1bmRlciAgdGhlIHRlcm1zIG9mICB0aGUgR05VIEdlbmVyYWwgIFB1Ymxp YyBMaWNlbnNlIGFzIHB1Ymxpc2hlZCBieSB0aGUNCisgKiBGcmVlIFNvZnR3YXJlIEZvdW5kYXRp b247ICBlaXRoZXIgdmVyc2lvbiAyIG9mIHRoZSAgTGljZW5zZSwgb3IgKGF0IHlvdXINCisgKiBv cHRpb24pIGFueSBsYXRlciB2ZXJzaW9uLg0KKyAqDQorICovDQorI2luY2x1ZGUgPGxpbnV4L21v ZHVsZS5oPg0KKyNpbmNsdWRlIDxsaW51eC9pbml0Lmg+DQorI2luY2x1ZGUgPGxpbnV4L2RlbGF5 Lmg+DQorI2luY2x1ZGUgPGxpbnV4L21paV9idXMuaD4NCisNCisvKiBETTkxNjEgQ29udHJvbCBy ZWdpc3RlciB2YWx1ZXMgKi8NCisjZGVmaW5lIE1JSU1fRE05MTYxX0NSX1NUT1AJMHgwNDAwDQor I2RlZmluZSBNSUlNX0RNOTE2MV9DUl9SU1RBTgkweDEyMDANCisNCisjZGVmaW5lIE1JSU1fRE05 MTYxX1NDUgkJMHgxMA0KKyNkZWZpbmUgTUlJTV9ETTkxNjFfU0NSX0lOSVQJMHgwNjEwDQorDQor LyogRE05MTYxIFNwZWNpZmllZCBDb25maWd1cmF0aW9uIGFuZCBTdGF0dXMgUmVnaXN0ZXIgKi8N CisjZGVmaW5lIE1JSU1fRE05MTYxX1NDU1IJMHgxMQ0KKyNkZWZpbmUgTUlJTV9ETTkxNjFfU0NT Ul8xMDBGCTB4ODAwMA0KKyNkZWZpbmUgTUlJTV9ETTkxNjFfU0NTUl8xMDBICTB4NDAwMA0KKyNk ZWZpbmUgTUlJTV9ETTkxNjFfU0NTUl8xMEYJMHgyMDAwDQorI2RlZmluZSBNSUlNX0RNOTE2MV9T Q1NSXzEwSAkweDEwMDANCisNCisvKiBETTkxNjEgSW50ZXJydXB0IFJlZ2lzdGVyICovDQorI2Rl ZmluZSBNSUlNX0RNOTE2MV9JTlRSCTB4MTUNCisjZGVmaW5lIE1JSU1fRE05MTYxX0lOVFJfUEVO RAkJMHg4MDAwDQorI2RlZmluZSBNSUlNX0RNOTE2MV9JTlRSX0RQTFhfTUFTSwkweDA4MDANCisj ZGVmaW5lIE1JSU1fRE05MTYxX0lOVFJfU1BEX01BU0sJMHgwNDAwDQorI2RlZmluZSBNSUlNX0RN OTE2MV9JTlRSX0xJTktfTUFTSwkweDAyMDANCisjZGVmaW5lIE1JSU1fRE05MTYxX0lOVFJfTUFT SwkJMHgwMTAwDQorI2RlZmluZSBNSUlNX0RNOTE2MV9JTlRSX0RQTFhfQ0hBTkdFCTB4MDAxMA0K KyNkZWZpbmUgTUlJTV9ETTkxNjFfSU5UUl9TUERfQ0hBTkdFCTB4MDAwOA0KKyNkZWZpbmUgTUlJ TV9ETTkxNjFfSU5UUl9MSU5LX0NIQU5HRQkweDAwMDQNCisjZGVmaW5lIE1JSU1fRE05MTYxX0lO VFJfSU5JVCAJCTB4MDAwMA0KKyNkZWZpbmUgTUlJTV9ETTkxNjFfSU5UUl9TVE9QCVwNCisoTUlJ TV9ETTkxNjFfSU5UUl9EUExYX01BU0sgfCBNSUlNX0RNOTE2MV9JTlRSX1NQRF9NQVNLIFwNCisg fCBNSUlNX0RNOTE2MV9JTlRSX0xJTktfTUFTSyB8IE1JSU1fRE05MTYxX0lOVFJfTUFTSykNCisN CisvKiBETTkxNjEgMTBCVCBDb25maWd1cmF0aW9uL1N0YXR1cyAqLw0KKyNkZWZpbmUgTUlJTV9E TTkxNjFfMTBCVENTUgkweDEyDQorI2RlZmluZSBNSUlNX0RNOTE2MV8xMEJUQ1NSX0lOSVQJMHg3 ODAwDQorDQorc3RhdGljIGludCBkbTkxNjFfaW5pdChzdHJ1Y3QgcGh5X2luZm8gKnBoeSkNCit7 DQorCW1kZWxheSgyMDAwKTsNCisNCisJcGh5X3dyaXRlKHBoeSwgTUlJX0JNQ1IsIE1JSU1fRE05 MTYxX0NSX1NUT1ApOw0KKwlwaHlfd3JpdGUocGh5LCBNSUlNX0RNOTE2MV9TQ1IsIE1JSU1fRE05 MTYxX1NDUl9JTklUKTsNCisJcGh5X3dyaXRlKHBoeSwgTUlJTV9ETTkxNjFfMTBCVENTUiwgTUlJ TV9ETTkxNjFfMTBCVENTUl9JTklUKTsNCisNCisJcmV0dXJuIDA7DQorfQ0KKw0KK3N0YXRpYyBp bnQgZG05MTYxX2ludF9lbmFibGUoc3RydWN0IHBoeV9pbmZvICpwaHkpDQorew0KKwkvKiBDbGVh ciBhbnkgcGVuZGluZyBpbnRlcnJ1cHRzICovDQorCXBoeV9yZWFkKHBoeSwgTUlJTV9ETTkxNjFf SU5UUik7DQorDQorCXJldHVybiAwOw0KK30NCisNCitzdGF0aWMgaW50IGRtOTE2MV9pbnRfYWNr KHN0cnVjdCBwaHlfaW5mbyAqcGh5KQ0KK3sNCisJLyogQ2xlYXIgYW55IHBlbmRpbmcgaW50ZXJy dXB0cyAqLw0KKwlwaHlfcmVhZChwaHksIE1JSU1fRE05MTYxX0lOVFIpOw0KKw0KKwlyZXR1cm4g MDsNCit9DQorDQorc3RhdGljIGludCBkbTkxNjFfaW50X2Rpc2FibGUoc3RydWN0IHBoeV9pbmZv ICpwaHkpDQorew0KKwkvKiBDbGVhciBhbnkgcGVuZGluZyBpbnRlcnJ1cHRzICovDQorCXBoeV9y ZWFkKHBoeSwgTUlJTV9ETTkxNjFfSU5UUik7DQorDQorCXJldHVybiAwOw0KK30NCisNCitzdGF0 aWMgaW50IGRtOTE2MV9wb2xsKHN0cnVjdCBwaHlfaW5mbyAqcGh5KQ0KK3sNCisJaW50IGF1dG9u ZWcgPSBwaHktPnN0YXRlLmF1dG9uZWc7DQorCWludCBlcnI7DQorCXVpbnQxNl90IHZhbDsNCisN CisJZXJyID0gcGh5X2dlbl9wb2xsKHBoeSk7DQorCWlmIChlcnIgPCAwKQ0KKwkJcmV0dXJuIGVy cjsNCisNCisJaWYgKHBoeS0+c3RhdGUubGluayAmJiBhdXRvbmVnKSB7DQorCQl2YWwgPSBwaHlf cmVhZChwaHksIE1JSU1fRE05MTYxX1NDU1IpOw0KKw0KKwkJaWYgKHZhbCAmIChNSUlNX0RNOTE2 MV9TQ1NSXzEwMEYgfCBNSUlNX0RNOTE2MV9TQ1NSXzEwMEgpKQ0KKwkJCXBoeS0+c3RhdGUuc3Bl ZWQgPSAxMDA7DQorCQllbHNlDQorCQkJcGh5LT5zdGF0ZS5zcGVlZCA9IDEwOw0KKw0KKwkJaWYg KHZhbCAmIChNSUlNX0RNOTE2MV9TQ1NSXzEwMEYgfCBNSUlNX0RNOTE2MV9TQ1NSXzEwRikpDQor CQkJcGh5LT5zdGF0ZS5kdXBsZXggPSAxOw0KKwkJZWxzZQ0KKwkJCXBoeS0+c3RhdGUuZHVwbGV4 ID0gMDsNCisJfQ0KKw0KKwlyZXR1cm4gMDsNCit9DQorDQorc3RhdGljIHN0cnVjdCBwaHlfb3Bz IHBoeV9vcHNfZG05MTYxID0gew0KKwkuaW5pdCA9IGRtOTE2MV9pbml0LA0KKwkucG9sbCA9IGRt OTE2MV9wb2xsLA0KKwkuaW50X2VuYWJsZSA9IGRtOTE2MV9pbnRfZW5hYmxlLA0KKwkuaW50X2Fj ayA9IGRtOTE2MV9pbnRfYWNrLA0KKwkuaW50X2Rpc2FibGUgPSBkbTkxNjFfaW50X2Rpc2FibGUs DQorfTsNCisNCitzdGF0aWMgc3RydWN0IHBoeV9pbmZvIHBoeV9pbmZvX2RtOTE2MSA9IHsNCisJ LmlkID0gMHgwMTgxYjg4MCwNCisJLm5hbWUgPSAiRGF2aWNvbSBETTkxNjFFIiwNCisJLnNoaWZ0 ID0gNCwNCisJLm9wcyA9ICZwaHlfb3BzX2RtOTE2MSwNCit9Ow0KKw0KK3N0YXRpYyBpbnQgcGh5 X2Rhdmljb21faW5pdCh2b2lkKQ0KK3sNCisJcmV0dXJuIHBoeV9yZWdpc3RlcigmcGh5X2luZm9f ZG05MTYxKTsNCit9DQorDQorc3RhdGljIHZvaWQgcGh5X2Rhdmljb21fZXhpdCh2b2lkKQ0KK3sN CisJcGh5X3VucmVnaXN0ZXIoJnBoeV9pbmZvX2RtOTE2MSk7DQorfQ0KKw0KK21vZHVsZV9pbml0 KHBoeV9kYXZpY29tX2luaXQpOw0KK21vZHVsZV9leGl0KHBoeV9kYXZpY29tX2V4aXQpOw0KDQot LS0gL2Rldi9udWxsDQorKysgbGludXgvZHJpdmVycy9uZXQvcGh5X2x4dDk3eC5jDQpAQCAtMCww ICsxLDIxMCBAQA0KKy8qIA0KKyAqIGRyaXZlcnMvbmV0L3BoeV9seHQ5N3guYw0KKyAqDQorICog QXV0aG9yOiBKYXNvbiBNY011bGxhbg0KKyAqDQorICogQ29weXJpZ2h0IChjKSAyMDA0IFRpbWVz eXMgQ29ycC4NCisgKg0KKyAqIFRoaXMgcHJvZ3JhbSBpcyBmcmVlIHNvZnR3YXJlOyB5b3UgY2Fu IHJlZGlzdHJpYnV0ZSAgaXQgYW5kL29yIG1vZGlmeSBpdA0KKyAqIHVuZGVyICB0aGUgdGVybXMg b2YgIHRoZSBHTlUgR2VuZXJhbCAgUHVibGljIExpY2Vuc2UgYXMgcHVibGlzaGVkIGJ5IHRoZQ0K KyAqIEZyZWUgU29mdHdhcmUgRm91bmRhdGlvbjsgIGVpdGhlciB2ZXJzaW9uIDIgb2YgdGhlICBM aWNlbnNlLCBvciAoYXQgeW91cg0KKyAqIG9wdGlvbikgYW55IGxhdGVyIHZlcnNpb24uDQorICoN CisgKi8NCisjaW5jbHVkZSA8bGludXgvbW9kdWxlLmg+DQorI2luY2x1ZGUgPGxpbnV4L2luaXQu aD4NCisjaW5jbHVkZSA8bGludXgvbWlpX2J1cy5oPg0KKw0KKy8qIC0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0g Ki8NCisvKiBUaGUgTGV2ZWwgb25lIExYVDk3MCBpcyB1c2VkIGJ5IG1hbnkgYm9hcmRzCQkJCSAg ICAgKi8NCisNCisjZGVmaW5lIE1JSV9MWFQ5NzBfTUlSUk9SICAgIDE2ICAvKiBNaXJyb3IgcmVn aXN0ZXIgICAgICAgICAgICovDQorI2RlZmluZSBNSUlfTFhUOTcwX0lFUiAgICAgICAxNyAgLyog SW50ZXJydXB0IEVuYWJsZSBSZWdpc3RlciAqLw0KKyNkZWZpbmUgTUlJX0xYVDk3MF9JU1IgICAg ICAgMTggIC8qIEludGVycnVwdCBTdGF0dXMgUmVnaXN0ZXIgKi8NCisjZGVmaW5lIE1JSV9MWFQ5 NzBfQ09ORklHICAgIDE5ICAvKiBDb25maWd1cmF0aW9uIFJlZ2lzdGVyICAgICovDQorI2RlZmlu ZSBNSUlfTFhUOTcwX0NTUiAgICAgICAyMCAgLyogQ2hpcCBTdGF0dXMgUmVnaXN0ZXIgICAgICAq Lw0KKw0KK3N0YXRpYyBpbnQgbHh0OTcwX2ludF9lbmFibGUoc3RydWN0IHBoeV9pbmZvICpwaHkp DQorew0KKwlwaHlfd3JpdGUocGh5LCBNSUlfTFhUOTcwX0lFUiwgMHgwMDAyKTsNCisNCisJcmV0 dXJuIDA7DQorfTsNCisNCitzdGF0aWMgaW50IGx4dDk3MF9pbnRfYWNrKHN0cnVjdCBwaHlfaW5m byAqcGh5KQ0KK3sNCisJcGh5X3JlYWQocGh5LCBNSUlfTFhUOTcwX0lTUik7DQorDQorCXJldHVy biAwOw0KK30NCisNCitzdGF0aWMgaW50IGx4dDk3MF9pbnRfZGlzYWJsZShzdHJ1Y3QgcGh5X2lu Zm8gKnBoeSkNCit7DQorCXBoeV93cml0ZShwaHksIE1JSV9MWFQ5NzBfSUVSLCAweDAwMDApOw0K Kw0KKwlyZXR1cm4gMDsNCit9Ow0KKw0KK3N0YXRpYyBpbnQgbHh0OTcwX3BvbGwoc3RydWN0IHBo eV9pbmZvICpwaHkpDQorew0KKwlpbnQgYXV0b25lZyA9IHBoeS0+c3RhdGUuYXV0b25lZzsNCisJ aW50IGVycjsNCisNCisJZXJyID0gcGh5X2dlbl9wb2xsKHBoeSk7DQorCWlmIChlcnIgPCAwKQ0K KwkJcmV0dXJuIGVycjsNCisNCisJaWYgKHBoeS0+c3RhdGUubGluayAmJiBhdXRvbmVnKSB7DQor CQkvKiBmaW5kIG91dCB0aGUgY3VycmVudCBzdGF0ZSAqLw0KKwkJdWludDE2X3QgdmFsOw0KKw0K KwkJdmFsID0gcGh5X3JlYWQocGh5LCBNSUlfTFhUOTcwX0NTUik7DQorCQlpZiAodmFsICYgMHgx MDAwKQ0KKwkJCXBoeS0+c3RhdGUuZHVwbGV4ID0gMTsNCisJCWVsc2UNCisJCQlwaHktPnN0YXRl LmR1cGxleCA9IDA7DQorDQorCQlpZiAodmFsICYgMHgwODAwKSB7DQorCQkJcGh5LT5zdGF0ZS5z cGVlZCA9IDEwMDsNCisJCX0gZWxzZSB7DQorCQkJcGh5LT5zdGF0ZS5zcGVlZCA9IDEwOw0KKwkJ fQ0KKwl9DQorDQorCXJldHVybiAwOw0KK30NCisNCisNCitzdGF0aWMgc3RydWN0IHBoeV9vcHMg cGh5X29wc19seHQ5NzAgPSB7DQorCS5wb2xsIAkJPSBseHQ5NzBfcG9sbCwNCisJLmludF9lbmFi bGUJPSBseHQ5NzBfaW50X2VuYWJsZSwNCisJLmludF9hY2sJPSBseHQ5NzBfaW50X2FjaywNCisJ LmludF9kaXNhYmxlCT0gbHh0OTcwX2ludF9kaXNhYmxlDQorfTsNCisNCitzdGF0aWMgc3RydWN0 IHBoeV9pbmZvIHBoeV9pbmZvX2x4dDk3MCA9IHsNCisJLmlkID0gMHgwNzgxMDAwMCwNCisJLnNo aWZ0ID0gNCwNCisJLm5hbWUgPSAiTFhUOTcwIiwNCisJLm9wcyA9ICZwaHlfb3BzX2x4dDk3MCwN Cit9Ow0KKw0KKy8qIFNhbWUgYXMgdGhlIExYVDk3MCwgYnV0IGRpZmZlcmVudCBJRA0KKyAqLw0K K3N0YXRpYyBzdHJ1Y3QgcGh5X2luZm8gcGh5X2luZm9fbHh0OTcwYSA9IHsNCisJLmlkID0gMHgw MDgxMDAwMCwNCisJLnNoaWZ0ID0gNCwNCisJLm5hbWUgPSAiTFhUOTcwQSIsDQorCS5vcHMgPSAm cGh5X29wc19seHQ5NzAsDQorfTsNCisNCisvKiByZWdpc3RlciBkZWZpbml0aW9ucyBmb3IgdGhl IDk3MSAqLw0KKw0KKyNkZWZpbmUgTUlJX0xYVDk3MV9QQ1IgICAgICAgMTYgIC8qIFBvcnQgQ29u dHJvbCBSZWdpc3RlciAgICAgKi8NCisjZGVmaW5lIE1JSV9MWFQ5NzFfU1IyICAgICAgIDE3ICAv KiBTdGF0dXMgUmVnaXN0ZXIgMiAgICAgICAgICovDQorI2RlZmluZSBNSUlfTFhUOTcxX0lFUiAg ICAgICAxOCAgLyogSW50ZXJydXB0IEVuYWJsZSBSZWdpc3RlciAqLw0KKyNkZWZpbmUgTUlJX0xY VDk3MV9JU1IgICAgICAgMTkgIC8qIEludGVycnVwdCBTdGF0dXMgUmVnaXN0ZXIgKi8NCisjZGVm aW5lIE1JSV9MWFQ5NzFfTENSICAgICAgIDIwICAvKiBMRUQgQ29udHJvbCBSZWdpc3RlciAgICAg ICovDQorI2RlZmluZSBNSUlfTFhUOTcxX1RDUiAgICAgICAzMCAgLyogVHJhbnNtaXQgQ29udHJv bCBSZWdpc3RlciAqLw0KKw0KK3N0YXRpYyBpbnQgbHh0OTcxX2ludF9lbmFibGUoc3RydWN0IHBo eV9pbmZvICpwaHkpDQorew0KKwlwaHlfd3JpdGUocGh5LCBNSUlfTFhUOTcxX0lFUiwgMHgwMGYy KTsNCisNCisJcmV0dXJuIDA7DQorfQ0KKw0KK3N0YXRpYyBpbnQgbHh0OTcxX3BvbGwoc3RydWN0 IHBoeV9pbmZvICpwaHkpDQorew0KKwlpbnQgYXV0b25lZyA9IHBoeS0+c3RhdGUuYXV0b25lZzsN CisJaW50IGVycjsNCisNCisJZXJyID0gcGh5X2dlbl9wb2xsKHBoeSk7DQorCWlmIChlcnIgPCAw KQ0KKwkJcmV0dXJuIGVycjsNCisNCisJaWYgKHBoeS0+c3RhdGUubGluayAmJiBhdXRvbmVnKSB7 DQorCQkvKiBmaW5kIG91dCB0aGUgY3VycmVudCBzdGF0ZSAqLw0KKwkJdWludDE2X3QgdmFsOw0K Kw0KKwkJdmFsID0gcGh5X3JlYWQocGh5LCBNSUlfTFhUOTcxX1NSMik7DQorDQorCQlpZiAodmFs ICYgMHg0MDAwKSB7DQorCQkJcGh5LT5zdGF0ZS5zcGVlZCA9IDEwMDsNCisJCX0gZWxzZSB7DQor CQkJcGh5LT5zdGF0ZS5zcGVlZCA9IDEwOw0KKwkJfQ0KKw0KKwkJaWYgKHZhbCAmIDB4MDIwMCkg ew0KKwkJCXBoeS0+c3RhdGUuZHVwbGV4ID0gMTsNCisJCX0gZWxzZSB7DQorCQkJcGh5LT5zdGF0 ZS5kdXBsZXggPSAwOw0KKwkJfQ0KKw0KKwkJaWYgKHZhbCAmIDB4MDAwOCkNCisJCQlwcmludGso S0VSTl9ERUJVRyAiJXM6IEZhdWx0IGRldGVjdGVkXG4iLHBoeS0+bmFtZSk7DQorCX0NCisNCisJ cmV0dXJuIDA7DQorfQ0KKw0KK3N0YXRpYyBpbnQgbHh0OTcxX2ludF9hY2soc3RydWN0IHBoeV9p bmZvICpwaHkpDQorew0KKwlwaHlfcmVhZChwaHksIE1JSV9MWFQ5NzFfSVNSKTsNCisNCisJcmV0 dXJuIDA7DQorfQ0KKw0KK3N0YXRpYyBpbnQgbHh0OTcxX2ludF9kaXNhYmxlKHN0cnVjdCBwaHlf aW5mbyAqcGh5KQ0KK3sNCisJcGh5X3dyaXRlKHBoeSwgTUlJX0xYVDk3MV9JRVIsIDB4MDAwMCk7 DQorDQorCXJldHVybiAwOw0KK307DQorDQorc3RhdGljIHN0cnVjdCBwaHlfb3BzIHBoeV9vcHNf bHh0OTcxID0gew0KKwkucG9sbCAJCT0gbHh0OTcxX3BvbGwsDQorCS5pbnRfZW5hYmxlCT0gbHh0 OTcxX2ludF9lbmFibGUsDQorCS5pbnRfYWNrCT0gbHh0OTcxX2ludF9hY2ssDQorCS5pbnRfZGlz YWJsZQk9IGx4dDk3MV9pbnRfZGlzYWJsZSwNCit9Ow0KKw0KK3N0YXRpYyBzdHJ1Y3QgcGh5X2lu Zm8gcGh5X2luZm9fbHh0OTcxID0gew0KKwkuaWQgPSAweDAwMTM3OGUwLA0KKwkuc2hpZnQgPSA0 LA0KKwkubmFtZSA9ICJMWFQ5NzEiLA0KKwkub3BzID0gJnBoeV9vcHNfbHh0OTcxLA0KK307DQor DQorc3RhdGljIGludCBwaHlfbHh0OTd4X2luaXQodm9pZCkNCit7DQorCWludCBlcnI7DQorDQor CWVycj1waHlfcmVnaXN0ZXIoJnBoeV9pbmZvX2x4dDk3MCk7DQorCWlmIChlcnIpDQorCQlyZXR1 cm4gZXJyOw0KKw0KKwllcnI9cGh5X3JlZ2lzdGVyKCZwaHlfaW5mb19seHQ5NzBhKTsNCisJaWYg KGVycikgew0KKwkJcGh5X3VucmVnaXN0ZXIoJnBoeV9pbmZvX2x4dDk3MCk7DQorCQlyZXR1cm4g ZXJyOw0KKwl9DQorDQorCWVycj1waHlfcmVnaXN0ZXIoJnBoeV9pbmZvX2x4dDk3MSk7DQorCWlm IChlcnIpIHsNCisJCXBoeV91bnJlZ2lzdGVyKCZwaHlfaW5mb19seHQ5NzApOw0KKwkJcGh5X3Vu cmVnaXN0ZXIoJnBoeV9pbmZvX2x4dDk3MGEpOw0KKwl9DQorDQorCXJldHVybiBlcnI7DQorfQ0K Kw0KK3N0YXRpYyB2b2lkIHBoeV9seHQ5N3hfZXhpdCh2b2lkKQ0KK3sNCisJcGh5X3VucmVnaXN0 ZXIoJnBoeV9pbmZvX2x4dDk3MSk7DQorCXBoeV91bnJlZ2lzdGVyKCZwaHlfaW5mb19seHQ5NzBh KTsNCisJcGh5X3VucmVnaXN0ZXIoJnBoeV9pbmZvX2x4dDk3MCk7DQorfQ0KKw0KK21vZHVsZV9p bml0KHBoeV9seHQ5N3hfaW5pdCk7DQorbW9kdWxlX2V4aXQocGh5X2x4dDk3eF9leGl0KTsNCg0K LS0tIC9kZXYvbnVsbA0KKysrIGxpbnV4L2RyaXZlcnMvbmV0L3BoeV9tYXJ2ZWxsLmMNCkBAIC0w LDAgKzEsMTI1IEBADQorLyogDQorICogZHJpdmVycy9uZXQvcGh5X21hcnZlbGwuYw0KKyAqDQor ICogQXV0aG9yOiBKYXNvbiBNY011bGxhbg0KKyAqDQorICogQ29weXJpZ2h0IChjKSAyMDA0IFRp bWVzeXMgQ29ycC4NCisgKg0KKyAqIFRoaXMgcHJvZ3JhbSBpcyBmcmVlIHNvZnR3YXJlOyB5b3Ug Y2FuIHJlZGlzdHJpYnV0ZSAgaXQgYW5kL29yIG1vZGlmeSBpdA0KKyAqIHVuZGVyICB0aGUgdGVy bXMgb2YgIHRoZSBHTlUgR2VuZXJhbCAgUHVibGljIExpY2Vuc2UgYXMgcHVibGlzaGVkIGJ5IHRo ZQ0KKyAqIEZyZWUgU29mdHdhcmUgRm91bmRhdGlvbjsgIGVpdGhlciB2ZXJzaW9uIDIgb2YgdGhl ICBMaWNlbnNlLCBvciAoYXQgeW91cg0KKyAqIG9wdGlvbikgYW55IGxhdGVyIHZlcnNpb24uDQor ICoNCisgKi8NCisjaW5jbHVkZSA8bGludXgvbW9kdWxlLmg+DQorI2luY2x1ZGUgPGxpbnV4L2lu aXQuaD4NCisjaW5jbHVkZSA8bGludXgvbWlpX2J1cy5oPg0KKw0KKy8qIDg4RTEwMTEgUEhZIFN0 YXR1cyBSZWdpc3RlciAqLw0KKyNkZWZpbmUgTUlJTV84OEUxMDExX1BIWV9TVEFUVVMgICAgICAg ICAweDExDQorI2RlZmluZSBNSUlNXzg4RTEwMTFfUEhZU1RBVF9TUEVFRCAgICAgIDB4YzAwMA0K KyNkZWZpbmUgTUlJTV84OEUxMDExX1BIWVNUQVRfR0JJVCAgICAgICAweDgwMDANCisjZGVmaW5l IE1JSU1fODhFMTAxMV9QSFlTVEFUXzEwMCAgICAgICAgMHg0MDAwDQorI2RlZmluZSBNSUlNXzg4 RTEwMTFfUEhZU1RBVF9EVVBMRVggICAgIDB4MjAwMA0KKyNkZWZpbmUgTUlJTV84OEUxMDExX1BI WVNUQVRfTElOSwkweDA0MDANCisNCisjZGVmaW5lIE1JSU1fODhFMTAxMV9JRVZFTlQJCTB4MTMN CisjZGVmaW5lIE1JSU1fODhFMTAxMV9JRVZFTlRfQ0xFQVIJMHgwMDAwDQorDQorI2RlZmluZSBN SUlNXzg4RTEwMTFfSU1BU0sJCTB4MTINCisjZGVmaW5lIE1JSU1fODhFMTAxMV9JTUFTS19JTklU CQkweDY0MDANCisjZGVmaW5lIE1JSU1fODhFMTAxMV9JTUFTS19DTEVBUgkweDAwMDANCisNCitz dGF0aWMgaW50IG1hcnZlbGxfaW50X2VuYWJsZShzdHJ1Y3QgcGh5X2luZm8gKnBoeSkNCit7DQor CS8qIENsZWFyIHRoZSBJRVZFTlQgcmVnaXN0ZXIgKi8NCisJcGh5X3JlYWQocGh5LCBNSUlNXzg4 RTEwMTFfSUVWRU5UKTsNCisNCisJLyogU2V0IHVwIHRoZSBtYXNrICovDQorCXBoeV93cml0ZShw aHksIE1JSU1fODhFMTAxMV9JTUFTSywgTUlJTV84OEUxMDExX0lNQVNLX0lOSVQpOw0KKw0KKwly ZXR1cm4gMDsNCit9DQorDQorc3RhdGljIGludCBtYXJ2ZWxsX2ludF9hY2soc3RydWN0IHBoeV9p bmZvICpwaHkpDQorew0KKwkvKiBDbGVhciB0aGUgaW50ZXJydXB0ICovDQorCXBoeV9yZWFkKHBo eSwgTUlJTV84OEUxMDExX0lFVkVOVCk7DQorDQorCXJldHVybiAwOw0KK30NCisNCitzdGF0aWMg aW50IG1hcnZlbGxfaW50X2Rpc2FibGUoc3RydWN0IHBoeV9pbmZvICpwaHkpDQorew0KKwkvKiBD bGVhciB0aGUgaW50ZXJydXB0ICovDQorCXBoeV9yZWFkKHBoeSwgTUlJTV84OEUxMDExX0lFVkVO VCk7DQorCS8qIERpc2FibGUgSW50ZXJydXB0cyAqLw0KKwlwaHlfd3JpdGUocGh5LCBNSUlNXzg4 RTEwMTFfSU1BU0ssIE1JSU1fODhFMTAxMV9JTUFTS19DTEVBUik7DQorDQorCXJldHVybiAwOw0K K30NCisNCitzdGF0aWMgaW50IG1hcnZlbGxfcG9sbChzdHJ1Y3QgcGh5X2luZm8gKnBoeSkNCit7 DQorCWludCBhdXRvbmVnID0gcGh5LT5zdGF0ZS5hdXRvbmVnOw0KKwlpbnQgZXJyOw0KKw0KKwll cnIgPSBwaHlfZ2VuX3BvbGwocGh5KTsNCisJaWYgKGVyciA8IDApDQorCQlyZXR1cm4gZXJyOw0K Kw0KKwlpZiAocGh5LT5zdGF0ZS5saW5rICYmIGF1dG9uZWcpIHsNCisJCXVpbnQxNl90IHZhbDsN CisJCXVuc2lnbmVkIGludCBzcGVlZDsNCisNCisJCXZhbCA9IHBoeV9yZWFkKHBoeSwgTUlJTV84 OEUxMDExX1BIWV9TVEFUVVMpOw0KKw0KKwkJaWYgKHZhbCAmIE1JSU1fODhFMTAxMV9QSFlTVEFU X0RVUExFWCkNCisJCQlwaHktPnN0YXRlLmR1cGxleCA9IDE7DQorCQllbHNlDQorCQkJcGh5LT5z dGF0ZS5kdXBsZXggPSAwOw0KKw0KKwkJc3BlZWQgPSAodmFsICYgTUlJTV84OEUxMDExX1BIWVNU QVRfU1BFRUQpOw0KKw0KKwkJc3dpdGNoIChzcGVlZCkgew0KKwkJCWNhc2UgTUlJTV84OEUxMDEx X1BIWVNUQVRfR0JJVDoNCisJCQkJcGh5LT5zdGF0ZS5zcGVlZCA9IDEwMDA7DQorCQkJCWJyZWFr Ow0KKwkJCWNhc2UgTUlJTV84OEUxMDExX1BIWVNUQVRfMTAwOg0KKwkJCQlwaHktPnN0YXRlLnNw ZWVkID0gMTAwOw0KKwkJCQlicmVhazsNCisJCQlkZWZhdWx0Og0KKwkJCQlwaHktPnN0YXRlLnNw ZWVkID0gMTA7DQorCQkJCWJyZWFrOw0KKwkJfQ0KKwl9DQorDQorCXJldHVybiAwOw0KK30NCisN CitzdGF0aWMgc3RydWN0IHBoeV9vcHMgcGh5X29wc19tYXJ2ZWxsID0gew0KKwkucG9sbAkJPSBt YXJ2ZWxsX3BvbGwsDQorCS5pbnRfZW5hYmxlCT0gbWFydmVsbF9pbnRfZW5hYmxlLA0KKwkuaW50 X2Fjawk9IG1hcnZlbGxfaW50X2FjaywNCisJLmludF9kaXNhYmxlCT0gbWFydmVsbF9pbnRfZGlz YWJsZSwNCit9Ow0KKw0KK3N0YXRpYyBzdHJ1Y3QgcGh5X2luZm8gcGh5X2luZm9fTTg4RTEwMTFT ID0gew0KKwkuaWQgPSAweDAxNDEwYzYwLA0KKwkubmFtZSA9ICJNYXJ2ZWxsIDg4RTEwMTFTIiwN CisJLnNoaWZ0ID0gNCwNCisJLm9wcyA9ICZwaHlfb3BzX21hcnZlbGwsDQorfTsNCisNCitzdGF0 aWMgaW50IHBoeV9tYXJ2ZWxsX2luaXQodm9pZCkNCit7DQorCXJldHVybiBwaHlfcmVnaXN0ZXIo JnBoeV9pbmZvX004OEUxMDExUyk7DQorfQ0KKw0KK3N0YXRpYyB2b2lkIHBoeV9tYXJ2ZWxsX2V4 aXQodm9pZCkNCit7DQorCXBoeV91bnJlZ2lzdGVyKCZwaHlfaW5mb19NODhFMTAxMVMpOw0KK30N CisNCittb2R1bGVfaW5pdChwaHlfbWFydmVsbF9pbml0KTsNCittb2R1bGVfZXhpdChwaHlfbWFy dmVsbF9leGl0KTsNCg0KLS0tIC9kZXYvbnVsbA0KKysrIGxpbnV4L2luY2x1ZGUvbGludXgvbWlp X2J1cy5oDQpAQCAtMCwwICsxLDE5MSBAQA0KKy8qIA0KKyAqIGluY2x1ZGUvbGludXgvbWlpX2J1 cy5oDQorICoNCisgKiBBdXRob3I6IEphc29uIE1jTXVsbGFuDQorICoNCisgKiBDb3B5cmlnaHQg KGMpIDIwMDQgVGltZXN5cyBDb3JwLg0KKyAqDQorICogVGhpcyBwcm9ncmFtIGlzIGZyZWUgc29m dHdhcmU7IHlvdSBjYW4gcmVkaXN0cmlidXRlICBpdCBhbmQvb3IgbW9kaWZ5IGl0DQorICogdW5k ZXIgIHRoZSB0ZXJtcyBvZiAgdGhlIEdOVSBHZW5lcmFsICBQdWJsaWMgTGljZW5zZSBhcyBwdWJs aXNoZWQgYnkgdGhlDQorICogRnJlZSBTb2Z0d2FyZSBGb3VuZGF0aW9uOyAgZWl0aGVyIHZlcnNp b24gMiBvZiB0aGUgIExpY2Vuc2UsIG9yIChhdCB5b3VyDQorICogb3B0aW9uKSBhbnkgbGF0ZXIg dmVyc2lvbi4NCisgKg0KKyAqLw0KKyNpZm5kZWYgX19NSUlfQlVTX0gNCisjZGVmaW5lIF9fTUlJ X0JVU19IDQorDQorI2lmZGVmIF9fS0VSTkVMX18NCisNCisjaW5jbHVkZSA8bGludXgvc2NoZWQu aD4NCisjaW5jbHVkZSA8bGludXgvbGlzdC5oPg0KKyNpbmNsdWRlIDxsaW51eC9taWkuaD4NCisj aW5jbHVkZSA8bGludXgvZXRodG9vbC5oPg0KKw0KKyNkZWZpbmUgTUlJX1RJTUVPVVQgCSgyKkha KQ0KKw0KKyNkZWZpbmUgbWlpbV9lbmQgKC0yKQ0KKyNkZWZpbmUgbWlpbV9yZWFkICgtMSkNCisN CisvKiBNYWNyb3MgZm9yICdwaHlfaWQncyB1c2VkIGVsc2V3aGVyZS4NCisgKiBBIFBIWSBJRCBp cyAzIGJpdHMgb2YgYnVzLCBmb2xsb3dlZCBieSA1IGJpdHMgb2YgaWQNCisgKi8NCisjZGVmaW5l IE1JSV9CVVMocGh5X2lkKQkJKCgocGh5X2lkKSA+PiA1KSAmIDB4NykNCisjZGVmaW5lIE1JSV9J RChwaHlfaWQpCQkoKHBoeV9pZCkgJiAweDFmKQ0KKyNkZWZpbmUgTUlJX1BIWV9JRChidXMsIGlk KQkoKCgoYnVzKSAmIDB4NykgPDwgNSkgfCAoKGlkKSAmIDB4MWYpKQ0KKw0KKy8qDQorICogQ3Vy cmVudCBQSFkgc3RhdGUNCisgKi8NCitzdHJ1Y3QgcGh5X3N0YXRlIHsNCisJdW5zaWduZWQgaW50 IGxpbms6MTsNCisJdW5zaWduZWQgaW50IGR1cGxleDoxOw0KKwl1bnNpZ25lZCBpbnQgYXV0b25l ZzoxOw0KKwl1bnNpZ25lZCBpbnQgbG9vcGJhY2s6MTsNCisJdW5zaWduZWQgaW50IHNwZWVkOjI4 Ow0KK307DQorDQorLyogUEhZIG9wZXJhdGlvbnMgLSBib3Jyb3dlZCBmcm9tIHN1bmdlbV9waHku aA0KKyAqDQorICogd29sX29wdGlvbnM6CVNlZSBXQUtFXyogaW4gaW5jbHVkZS9saW51eC9ldGh0 b29sLmgNCisgKiBhZHZlcnRpc2UgIDogU2VlIEFEVkVSVElTRURfKiBpbiBpbmNsdWRlL2xpbnV4 L2V0aHRvb2wuaA0KKyAqLw0KK3N0cnVjdCBwaHlfaW5mbzsNCisNCitzdHJ1Y3QgcGh5X29wcyB7 DQorCWludAkoKmluaXQpKHN0cnVjdCBwaHlfaW5mbyAqcGh5KTsNCisJaW50CSgqc3VzcGVuZCko c3RydWN0IHBoeV9pbmZvICpwaHksIHVpbnQzMl90IHdvbF9vcHRpb25zKTsNCisJaW50CSgqc2V0 X2F1dG9uZWcpKHN0cnVjdCBwaHlfaW5mbyAqcGh5LCB1aW50MzJfdCBhZHZlcnRpc2UpOw0KKwlp bnQJKCpzZXRfZm9yY2VkKShzdHJ1Y3QgcGh5X2luZm8gKnBoeSwgaW50IHNwZWVkLCBpbnQgZHVw bGV4KTsNCisNCisJLyogUG9sbGluZyAqLw0KKwlpbnQJKCpwb2xsKShzdHJ1Y3QgcGh5X2luZm8g KnBoeSk7DQorDQorCS8qIEludGVycnVwdC1iYXNlZCAqLw0KKwlpbnQJKCppbnRfZW5hYmxlKShz dHJ1Y3QgcGh5X2luZm8gKnBoeSk7DQorCWludAkoKmludF9hY2spKHN0cnVjdCBwaHlfaW5mbyAq cGh5KTsNCisJaW50CSgqaW50X2Rpc2FibGUpKHN0cnVjdCBwaHlfaW5mbyAqcGh5KTsNCit9Ow0K Kw0KKy8qIHN0cnVjdCBwaHlfaW5mbzogYSBzdHJ1Y3R1cmUgd2hpY2ggZGVmaW5lcyBhdHRyaWJ1 dGVzIGZvciBhIFBIWQ0KKyAqDQorICogaWQgd2lsbCBjb250YWluIGEgbnVtYmVyIHdoaWNoIHJl cHJlc2VudHMgdGhlIFBIWS4gIER1cmluZw0KKyAqIHN0YXJ0dXAsIHRoZSBkcml2ZXIgd2lsbCBw b2xsIHRoZSBQSFkgdG8gZmluZCBvdXQgd2hhdCBpdHMNCisgKiBVSUQtLWFzIGRlZmluZWQgYnkg cmVnaXN0ZXJzIDIgYW5kIDMtLWlzLiAgVGhlIDMyLWJpdCByZXN1bHQNCisgKiBnb3R0ZW4gZnJv bSB0aGUgUEhZIHdpbGwgYmUgc2hpZnRlZCByaWdodCBieSAic2hpZnQiIGJpdHMgdG8NCisgKiBk aXNjYXJkIGFueSBiaXRzIHdoaWNoIG1heSBjaGFuZ2UgYmFzZWQgb24gcmV2aXNpb24gbnVtYmVy cw0KKyAqIHVuaW1wb3J0YW50IHRvIGZ1bmN0aW9uYWxpdHkNCisgKg0KKyAqIFRoZSBzdHJ1Y3Qg cGh5X2NtZCBlbnRyaWVzIHJlcHJlc2VudCBwb2ludGVycyB0byBhbiBhcnJheXMgb2YNCisgKiBj b21tYW5kcyB3aGljaCB0ZWxsIHRoZSBkcml2ZXIgd2hhdCB0byBkbyB0byB0aGUgUEhZLg0KKyAq Lw0KK3N0cnVjdCBwaHlfaW5mbyB7DQorCXN0cnVjdCBsaXN0X2hlYWQgbGlzdDsNCisNCisJdWlu dDMyX3QgaWQ7DQorCWNoYXIgbmFtZVszMl07DQorCXVuc2lnbmVkIGludCBzaGlmdDsNCisNCisJ c3RydWN0IHBoeV9vcHMgKm9wczsNCisNCisJLyogUGVyLVBIWSBkcml2ZXIgZGF0YSBnb2VzIGhl cmUgKi8NCisJdm9pZCAqcHJpdjsNCisNCisJLyogWW91ciBwb2xsKCkgcm91dGluZSBzaG91bGQg bW9kaWZ5IHRoaXMuDQorCSAqLw0KKwlzdHJ1Y3QgcGh5X3N0YXRlIHN0YXRlOw0KKw0KKwkvKiBF dmVyeXRoaW5nIGZyb20gaGVyZSBvbiBkb3duIHdpbGwNCisJICogYmUgZmlsbGVkIGluIGR1cmlu ZyByZWdpc3RyYXRpb24NCisJICovDQorCWludCBwaHlfaWQ7DQorDQorCXN0cnVjdCB7DQorCQlp bnQgaXJxOw0KKwkJdW5zaWduZWQgbG9uZyBtc2VjczsNCisJCXZvaWQgKCpmdW5jKSAodm9pZCAq ZGF0YSk7DQorCQl2b2lkICpkYXRhOw0KKwkJc3RydWN0IHdvcmtfc3RydWN0IHRxOw0KKwkJc3Ry dWN0IHRpbWVyX2xpc3QgdGltZXI7DQorCX0gZGVsdGE7DQorDQorCXN0cnVjdCB7DQorCQlpbnQg YXV0b25lZzsJCS8qIDE9YXV0bywgMD1mb3JjZWQgKi8NCisJCXVpbnQzMl90IGFkdmVydGlzZTsJ LyogbWFzayB0byBhbGxvdyAqLw0KKwkJdW5zaWduZWQgbG9uZyB0aW1lb3V0OwkvKiBqaWZmaWUg c3RhbXAgKi8NCisJfSBuZWdvdGlhdGU7DQorDQorfTsNCisNCitzdHJ1Y3QgbWlpX2J1cyB7DQor CWNvbnN0IGNoYXIgKm5hbWU7DQorCXZvaWQgKnByaXY7DQorCWludCAoKnJlYWQpICh2b2lkICpw cml2LCBpbnQgcGh5X2lkLCBpbnQgbG9jYXRpb24pOw0KKwlpbnQgKCp3cml0ZSkgKHZvaWQgKnBy aXYsIGludCBwaHlfaWQsIGludCBsb2NhdGlvbiwgdWludDE2X3QgdmFsKTsNCisJdm9pZCAoKnJl c2V0KSAodm9pZCAqcHJpdik7DQorDQorCS8qIEF1dG8tZmlsbGVkIGluIHZhbHVlcyAqLw0KKwlz dHJ1Y3QgcGh5X2luZm8gKnBoeVszMl07DQorfTsNCisNCisvKiBNSUkgYnVzIHJlZ2lzdHJhdGlv bg0KKyAqLw0KK2V4dGVybiBpbnQgbWlpX2J1c19yZWdpc3RlcihzdHJ1Y3QgbWlpX2J1cyAqYnVz KTsNCitleHRlcm4gdm9pZCBtaWlfYnVzX3VucmVnaXN0ZXIoc3RydWN0IG1paV9idXMgKmJ1cyk7 DQorDQorLyogUmF3IHJlYWQvd3JpdGUgcm91dGluZXMNCisgKiBSZXR1cm5zIGEgMTYtYml0IHJl Z2lzdGVyIHZhbHVlLCBvciA8IDAgZXJyb3IgY29kZQ0KKyAqLw0KK2V4dGVybiBpbnQgbWlpX2J1 c19yZWFkKGludCBidXNfaWQsIGludCBwaHlfaWQsIGludCByZWcpOw0KK2V4dGVybiBpbnQgbWlp X2J1c193cml0ZShpbnQgYnVzX2lkLCBpbnQgcGh5X2lkLCBpbnQgcmVnLCB1aW50MTZfdCB2YWwp Ow0KKw0KKy8qIFJvdXRpbmVzIHVzZWQgYnkgbmV0d29yayBkZXZpY2VzIHRoYXQgdXNlIHRoZSBN SUkgYnVzDQorICovDQorZXh0ZXJuIGludCBtaWlfcGh5X2F0dGFjaChzdHJ1Y3QgbWlpX2lmX2lu Zm8gKm1paSwgc3RydWN0IG5ldF9kZXZpY2UgKmRldiwNCisJCQkgIGludCBwaHlfYnVzLCBpbnQg cGh5X2lkKTsNCitleHRlcm4gdm9pZCBtaWlfcGh5X2RldGFjaChzdHJ1Y3QgbWlpX2lmX2luZm8g Km1paSk7DQorDQorLyogUmVhZCBjdXJyZW50IHBoeSBzdGF0ZQ0KKyAqLw0KK2V4dGVybiBpbnQg bWlpX3BoeV9zdGF0ZShzdHJ1Y3QgbWlpX2lmX2luZm8gKm1paSwgc3RydWN0IHBoeV9zdGF0ZSAq c3RhdGUpOw0KKw0KKy8qIFJlc2V0IE1JSSwgcmVuZWdvdGlhdGUgbGluaw0KKyAqLw0KK2V4dGVy biBpbnQgbWlpX3BoeV9zZXRfYXV0b25lZyhzdHJ1Y3QgbWlpX2lmX2luZm8gKm1paSwgdWludDMy X3QgYWR2ZXJ0aXNlKTsNCitleHRlcm4gaW50IG1paV9waHlfc2V0X2ZvcmNlZChzdHJ1Y3QgbWlp X2lmX2luZm8gKm1paSwgaW50IHNwZWVkLCBpbnQgZHVwbGV4KTsNCitleHRlcm4gaW50IG1paV9w aHlfc3VzcGVuZChzdHJ1Y3QgbWlpX2lmX2luZm8gKm1paSwgdWludDMyX3Qgd29sX29wdGlvbnMp Ow0KKw0KKy8qIFVzZSBhbiBJUlEgdG8gZGV0ZXJtaW5lIHdoZW4gdGhlIFBIWSBjaGFuZ2VzDQor ICovDQorZXh0ZXJuIGludCBtaWlfcGh5X2lycV9lbmFibGUoc3RydWN0IG1paV9pZl9pbmZvICpt aWksIGludCBpcnEsDQorCQkJICAgICAgdm9pZCAoKmZ1bmMpICh2b2lkICopLCB2b2lkICpkYXRh KTsNCitleHRlcm4gdm9pZCBtaWlfcGh5X2lycV9kaXNhYmxlKHN0cnVjdCBtaWlfaWZfaW5mbyAq bWlpLCB2b2lkICpkYXRhKTsNCisNCisvKiBQb2xsIHRoZSBQSFkNCisgKi8NCitleHRlcm4gaW50 IG1paV9waHlfcG9sbF9lbmFibGUoc3RydWN0IG1paV9pZl9pbmZvICptaWksIHVuc2lnbmVkIGxv bmcgbXNlY3MsDQorCQkJICAgICAgIHZvaWQgKCpmdW5jKSAodm9pZCAqKSwgdm9pZCAqZGF0YSk7 DQorZXh0ZXJuIHZvaWQgbWlpX3BoeV9wb2xsX2Rpc2FibGUoc3RydWN0IG1paV9pZl9pbmZvICpt aWksIHZvaWQgKmRhdGEpOw0KKw0KKy8qDQorICogUEhZIGRldmljZSByZWdpc3RyYXRpb24NCisg Ki8NCitleHRlcm4gaW50IHBoeV9yZWdpc3RlcihzdHJ1Y3QgcGh5X2luZm8gKnBoeSk7DQorZXh0 ZXJuIHZvaWQgcGh5X3VucmVnaXN0ZXIoc3RydWN0IHBoeV9pbmZvICpwaHkpOw0KKw0KK3N0YXRp YyBpbmxpbmUgaW50IHBoeV9yZWFkKHN0cnVjdCBwaHlfaW5mbyAqcGh5LCBpbnQgcmVnbnVtKQ0K K3sNCisJcmV0dXJuIG1paV9idXNfcmVhZChNSUlfQlVTKHBoeS0+cGh5X2lkKSwgTUlJX0lEKHBo eS0+cGh5X2lkKSwgcmVnbnVtKTsNCit9DQorDQorc3RhdGljIGlubGluZSBpbnQgcGh5X3dyaXRl KHN0cnVjdCBwaHlfaW5mbyAqcGh5LCBpbnQgcmVnLCB1aW50MTZfdCB2YWwpDQorew0KKwlyZXR1 cm4gbWlpX2J1c193cml0ZShNSUlfQlVTKHBoeS0+cGh5X2lkKSwgTUlJX0lEKHBoeS0+cGh5X2lk KSwgcmVnLCB2YWwpOwkNCit9DQorDQorLyogR2VuZXJpYyAnc3RydWN0IHBoeV9vcHMnIGRldmlj ZSByb3V0aW5lcyAqLw0KK2V4dGVybiBpbnQgcGh5X2dlbl9zZXRfYXV0b25lZyhzdHJ1Y3QgcGh5 X2luZm8gKnBoeSwgdTMyIGFkdmVydGlzZSk7DQorZXh0ZXJuIGludCBwaHlfZ2VuX3BvbGwoc3Ry dWN0IHBoeV9pbmZvICpwaHkpOw0KKw0KKyNlbmRpZiAvKiBfX0tFUk5FTF9fICovDQorDQorI2Vu ZGlmIC8qIF9fTUlJX0JVU19IICovDQoNCm== --=-UCEP7LL7eTaCc7pqknBv-- From jketreno@linux.intel.com Thu Dec 2 12:11:13 2004 Received: with ECARTIS (v1.0.0; list netdev); Thu, 02 Dec 2004 12:11:19 -0800 (PST) Received: from orsfmr002.jf.intel.com (fmr17.intel.com [134.134.136.16]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB2KBCrH007164 for ; Thu, 2 Dec 2004 12:11:13 -0800 Received: from petasus.jf.intel.com (petasus.jf.intel.com [10.7.209.6]) by orsfmr002.jf.intel.com (8.12.10/8.12.10/d: major-outer.mc,v 1.1 2004/09/17 17:50:56 root Exp $) with ESMTP id iB2KAkUu024758; Thu, 2 Dec 2004 20:10:46 GMT Received: from [127.0.0.1] (vpnfm001-139-dhcp-client.fm.intel.com [10.19.13.139]) by petasus.jf.intel.com (8.12.9-20030918-01/8.12.9/d: major-inner.mc,v 1.11 2004/07/29 22:51:53 root Exp $) with ESMTP id iB2KF9h9028646; Thu, 2 Dec 2004 20:15:11 GMT Message-ID: <41AF7708.3030804@linux.intel.com> Date: Thu, 02 Dec 2004 14:11:52 -0600 From: James Ketrenos User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.3) Gecko/20040922 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jeff Garzik CC: Netdev Subject: Re: Steps for netdev-2.6 inclusion? References: <41AE7143.80505@linux.intel.com> <41AEB3B8.2000406@pobox.com> In-Reply-To: <41AEB3B8.2000406@pobox.com> X-Enigmail-Version: 0.86.0.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.31 (www . roaringpenguin . com / mimedefang) X-archive-position: 12395 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jketreno@linux.intel.com Precedence: bulk X-list: netdev Jeff Garzik wrote: > It's fairly easy, just email me and netdev the patch for inclusion, > and it'll get reviewed. Should I break the patch into the three components (ipw2100, ipw2200, and ieee80211) ? or just one huge patch? Not sure what you and others would prefer. > Once review issues are addressed, I'll merge it immediately, which > causes it to be automatically propagated to Andrew Morton's -mm tree > for testing. > > Once consensus agrees that we can push this + HostAP upstream, that's > an easy 10-minute task. > > One potential showstopper is firmware crapola: I'm concerned about a > situation where we have drivers in the kernel, but the firmware must > be downloaded from SourceForge or somesuch. The firmware doesn't have to be downloaded from Sourceforge, but it does need to exist on the system, just as iwconfig needs to exist if you want to be able to configure your wireless card. The user can get the firmware from Sourceforge, or have it installed by their distribution or package management system, have it on their Knoppix CD, etc. Loading the driver without the firmware (or hotplug being enabled) won't take down the machine -- it will just print a kernel log message saying the firmware can't be found. For some common questions and answers on the redistribution of the license, see http://intel.com/support/wireless/wlan/sb/cs-016675.htm Regarding the topic of loading firmware from disk... the firmware_class subsystem was designed for this purpose. From linux/Documentation/firmware_class/README: ------------------ Why: --- Today, the most extended way to use firmware in the Linux kernel is linking it statically in a header file. Which has political and technical issues: 1) Some firmware is not legal to redistribute. 2) The firmware occupies memory permanently, even though it often is just used once. 3) Some people, like the Debian crowd, don't consider some firmware free enough and remove entire drivers (e.g.: keyspan). ------------------- Point one is partially applicable -- redistribution of the ipw firmware *is* legal, but due to the terms of the GPL, inclusion in the kernel is not possible (the firmware can't be licensed GPL, and so static inclusion in the driver as a header binary is impossible.) So, the firmware must be loaded onto the NIC from some other storage medium. The second point from the firmware_class's README is also valid (although the current state of suspend/resume necessitates a pre-allocated memory buffer in the driver. Once that issue is remedied, the host will be able to free up a reasonable amount of memory that is otherwise unnecessary.) > IOW, the kernel driver as-is is useless without a differently-licensed > firmware. Wireless (and many other kernel components) are arguably useless without user space utilities that must be downloaded, built, and installed. So the issue of having to download something, or have some other pre-requisite met before the driver is useful/functional is not unique to this driver. James From shemminger@osdl.org Thu Dec 2 13:50:00 2004 Received: with ECARTIS (v1.0.0; list netdev); Thu, 02 Dec 2004 13:50:07 -0800 (PST) Received: from mail.osdl.org (fw.osdl.org [65.172.181.6]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB2Lo00x012926 for ; Thu, 2 Dec 2004 13:50:00 -0800 Received: from dxpl.pdx.osdl.net (dxpl.pdx.osdl.net [172.20.1.103]) by mail.osdl.org (8.11.6/8.11.6) with ESMTP id iB2LnU925168; Thu, 2 Dec 2004 13:49:30 -0800 Date: Thu, 2 Dec 2004 13:49:30 -0800 From: Stephen Hemminger To: "David S. Miller" Cc: michael.vittrup.larsen@ericsson.com, netdev@oss.sgi.com Subject: Re: [PATCH] tcp: efficient port randomisation (revised) Message-Id: <20041202134930.132d7bd8@dxpl.pdx.osdl.net> In-Reply-To: <20041201204622.7b760400.davem@davemloft.net> References: <20041027092531.78fe438c@guest-251-240.pdx.osdl.net> <200411020854.44745.michael.vittrup.larsen@ericsson.com> <20041104100104.570e67cd@dxpl.pdx.osdl.net> <200411051103.59032.michael.vittrup.larsen@ericsson.com> <20041117153025.160eaa04@zqx3.pdx.osdl.net> <20041130214643.7b72300e.davem@davemloft.net> <20041201152446.3a0d5ce3@dxpl.pdx.osdl.net> <20041201204622.7b760400.davem@davemloft.net> Organization: Open Source Development Lab X-Mailer: Sylpheed version 0.9.10claws (GTK+ 1.2.10; x86_64-suse-linux) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-archive-position: 12396 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: shemminger@osdl.org Precedence: bulk X-list: netdev On Wed, 1 Dec 2004 20:46:22 -0800 "David S. Miller" wrote: > I'm more interested in simply things like "lat_connect" > from lmbench run over loopback. Oh, that was easy using OSDL PLM/STP which gives an easy way to run local tests. Baseline... [STP 299123] lmbench_long results Kernel: patch-2.6.10-rc2 PLM # 3869 *Local* Communication latencies in microseconds - smaller is better ------------------------------------------------------------------- Host OS 2p/0K Pipe AF UDP RPC/ TCP RPC/ TCP ctxsw UNIX UDP TCP conn --------- ------------- ----- ----- ---- ----- ----- ----- ----- ---- stp2-001 Linux 2.6.10- 8.270 38.6 24.3 61.6 48.5 45.9 76.6 74.6 stp2-001 Linux 2.6.10- 8.170 43.5 24.5 58.0 54.8 45.6 63.4 74.7 stp2-001 Linux 2.6.10- 2.740 50.6 29.9 40.3 48.3 59.8 75.1 101. stp2-001 Linux 2.6.10- 8.140 46.6 29.7 57.6 48.8 45.5 72.0 74.4 stp2-001 Linux 2.6.10- 2.690 47.1 26.3 40.8 48.9 45.5 75.4 74.8 ----------------- Patched... [STP 299118] lmbench_long results Kernel: tcp-port-randomization-2 PLM # 3907 *Local* Communication latencies in microseconds - smaller is better ------------------------------------------------------------------- Host OS 2p/0K Pipe AF UDP RPC/ TCP RPC/ TCP ctxsw UNIX UDP TCP conn --------- ------------- ----- ----- ---- ----- ----- ----- ----- ---- stp2-001 Linux 2.6.10- 2.770 46.3 25.0 64.4 49.5 44.3 75.2 75.6 stp2-001 Linux 2.6.10- 2.780 44.1 21.2 63.5 55.6 45.3 63.5 75.2 stp2-001 Linux 2.6.10- 2.790 47.5 24.8 40.4 48.5 45.5 63.7 76.9 stp2-001 Linux 2.6.10- 8.330 47.5 24.8 40.7 55.6 44.6 63.8 75.1 stp2-001 Linux 2.6.10- 8.150 47.9 25.7 41.2 49.6 45.2 72.7 75.1 These are run on a relatively slow machine (2 way Pentium III 850Mhz) and it looks like the results are no change (in the noise). From davem@davemloft.net Thu Dec 2 13:57:03 2004 Received: with ECARTIS (v1.0.0; list netdev); Thu, 02 Dec 2004 13:57:08 -0800 (PST) Received: from cheetah.davemloft.net (mail@adsl-63-197-226-105.dsl.snfc21.pacbell.net [63.197.226.105]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB2Lv3ua013621 for ; Thu, 2 Dec 2004 13:57:03 -0800 Received: from localhost ([127.0.0.1] helo=cheetah.davemloft.net ident=davem) by cheetah.davemloft.net with smtp (Exim 3.36 #1 (Debian)) id 1CZyse-0003MU-00; Thu, 02 Dec 2004 13:52:52 -0800 Date: Thu, 2 Dec 2004 13:52:52 -0800 From: "David S. Miller" To: Stephen Hemminger Cc: michael.vittrup.larsen@ericsson.com, netdev@oss.sgi.com Subject: Re: [PATCH] tcp: efficient port randomisation (revised) Message-Id: <20041202135252.04e64f51.davem@davemloft.net> In-Reply-To: <20041202134930.132d7bd8@dxpl.pdx.osdl.net> References: <20041027092531.78fe438c@guest-251-240.pdx.osdl.net> <200411020854.44745.michael.vittrup.larsen@ericsson.com> <20041104100104.570e67cd@dxpl.pdx.osdl.net> <200411051103.59032.michael.vittrup.larsen@ericsson.com> <20041117153025.160eaa04@zqx3.pdx.osdl.net> <20041130214643.7b72300e.davem@davemloft.net> <20041201152446.3a0d5ce3@dxpl.pdx.osdl.net> <20041201204622.7b760400.davem@davemloft.net> <20041202134930.132d7bd8@dxpl.pdx.osdl.net> X-Mailer: Sylpheed version 1.0.0beta3 (GTK+ 1.2.10; sparc-unknown-linux-gnu) X-Face: "_;p5u5aPsO,_Vsx"^v-pEq09'CU4&Dc1$fQExov$62l60cgCc%FnIwD=.UF^a>?5'9Kn[;433QFVV9M..2eN.@4ZWPGbdi<=?[:T>y?SD(R*-3It"Vj:)"dP Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-archive-position: 12397 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: davem@davemloft.net Precedence: bulk X-list: netdev On Thu, 2 Dec 2004 13:49:30 -0800 Stephen Hemminger wrote: > These are run on a relatively slow machine (2 way Pentium III 850Mhz) > and it looks like the results are no change (in the noise). Or averaged out, about 1ms more expensive. From shemminger@osdl.org Thu Dec 2 14:52:09 2004 Received: with ECARTIS (v1.0.0; list netdev); Thu, 02 Dec 2004 14:52:15 -0800 (PST) Received: from mail.osdl.org (fw.osdl.org [65.172.181.6]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB2Mq8NV016424 for ; Thu, 2 Dec 2004 14:52:09 -0800 Received: from dxpl.pdx.osdl.net (dxpl.pdx.osdl.net [172.20.1.103]) by mail.osdl.org (8.11.6/8.11.6) with ESMTP id iB2Mpd905001; Thu, 2 Dec 2004 14:51:39 -0800 Date: Thu, 2 Dec 2004 14:51:39 -0800 From: Stephen Hemminger To: "David S. Miller" Cc: michael.vittrup.larsen@ericsson.com, netdev@oss.sgi.com Subject: Re: [PATCH] tcp: efficient port randomisation (revised) Message-Id: <20041202145139.03a6977a@dxpl.pdx.osdl.net> In-Reply-To: <20041202135252.04e64f51.davem@davemloft.net> References: <20041027092531.78fe438c@guest-251-240.pdx.osdl.net> <200411020854.44745.michael.vittrup.larsen@ericsson.com> <20041104100104.570e67cd@dxpl.pdx.osdl.net> <200411051103.59032.michael.vittrup.larsen@ericsson.com> <20041117153025.160eaa04@zqx3.pdx.osdl.net> <20041130214643.7b72300e.davem@davemloft.net> <20041201152446.3a0d5ce3@dxpl.pdx.osdl.net> <20041201204622.7b760400.davem@davemloft.net> <20041202134930.132d7bd8@dxpl.pdx.osdl.net> <20041202135252.04e64f51.davem@davemloft.net> Organization: Open Source Development Lab X-Mailer: Sylpheed version 0.9.10claws (GTK+ 1.2.10; x86_64-suse-linux) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-archive-position: 12398 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: shemminger@osdl.org Precedence: bulk X-list: netdev On Thu, 2 Dec 2004 13:52:52 -0800 "David S. Miller" wrote: > On Thu, 2 Dec 2004 13:49:30 -0800 > Stephen Hemminger wrote: > > > These are run on a relatively slow machine (2 way Pentium III 850Mhz) > > and it looks like the results are no change (in the noise). > > Or averaged out, about 1ms more expensive. I am writing my own test since, that one seems so noisy. From shemminger@osdl.org Thu Dec 2 15:01:50 2004 Received: with ECARTIS (v1.0.0; list netdev); Thu, 02 Dec 2004 15:01:53 -0800 (PST) Received: from mail.osdl.org (fw.osdl.org [65.172.181.6]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB2N1nkN017676 for ; Thu, 2 Dec 2004 15:01:49 -0800 Received: from dxpl.pdx.osdl.net (dxpl.pdx.osdl.net [172.20.1.103]) by mail.osdl.org (8.11.6/8.11.6) with ESMTP id iB2N1L907266; Thu, 2 Dec 2004 15:01:21 -0800 Date: Thu, 2 Dec 2004 15:01:21 -0800 From: Stephen Hemminger To: "David S. Miller" Cc: michael.vittrup.larsen@ericsson.com, netdev@oss.sgi.com Subject: Re: [PATCH] tcp: efficient port randomisation (revised) Message-Id: <20041202150121.488ec205@dxpl.pdx.osdl.net> In-Reply-To: <20041202135252.04e64f51.davem@davemloft.net> References: <20041027092531.78fe438c@guest-251-240.pdx.osdl.net> <200411020854.44745.michael.vittrup.larsen@ericsson.com> <20041104100104.570e67cd@dxpl.pdx.osdl.net> <200411051103.59032.michael.vittrup.larsen@ericsson.com> <20041117153025.160eaa04@zqx3.pdx.osdl.net> <20041130214643.7b72300e.davem@davemloft.net> <20041201152446.3a0d5ce3@dxpl.pdx.osdl.net> <20041201204622.7b760400.davem@davemloft.net> <20041202134930.132d7bd8@dxpl.pdx.osdl.net> <20041202135252.04e64f51.davem@davemloft.net> Organization: Open Source Development Lab X-Mailer: Sylpheed version 0.9.10claws (GTK+ 1.2.10; x86_64-suse-linux) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-archive-position: 12399 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: shemminger@osdl.org Precedence: bulk X-list: netdev On Thu, 2 Dec 2004 13:52:52 -0800 "David S. Miller" wrote: > On Thu, 2 Dec 2004 13:49:30 -0800 > Stephen Hemminger wrote: > > > These are run on a relatively slow machine (2 way Pentium III 850Mhz) > > and it looks like the results are no change (in the noise). > > Or averaged out, about 1ms more expensive. We could always benchmark special the loopback case since there is no risk of man-in-the-middle attacks. From buytenh@wantstofly.org Thu Dec 2 15:25:55 2004 Received: with ECARTIS (v1.0.0; list netdev); Thu, 02 Dec 2004 15:26:00 -0800 (PST) Received: from xi.wantstofly.org (alephnull.demon.nl [212.238.201.82]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB2NPsC0018551 for ; Thu, 2 Dec 2004 15:25:55 -0800 Received: by xi.wantstofly.org (Postfix, from userid 500) id B9D7F2B0ED; Fri, 3 Dec 2004 00:25:31 +0100 (MET) Date: Fri, 3 Dec 2004 00:25:31 +0100 From: Lennert Buytenhek To: Robert Olsson Cc: sfeldma@pobox.com, jamal , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: [E1000-devel] Transmission limit Message-ID: <20041202232531.GA30948@xi.wantstofly.org> References: <1101499285.1079.45.camel@jzny.localdomain> <16811.8052.678955.795327@robur.slu.se> <1101821501.1043.43.camel@jzny.localdomain> <20041130134600.GA31515@xi.wantstofly.org> <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <16813.58484.343629.570703@robur.slu.se> <1101919791.5198.15.camel@localhost.localdomain> <16815.23964.93437.411404@robur.slu.se> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <16815.23964.93437.411404@robur.slu.se> User-Agent: Mutt/1.4.1i X-archive-position: 12400 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: buytenh@wantstofly.org Precedence: bulk X-list: netdev On Thu, Dec 02, 2004 at 07:23:24PM +0100, Robert Olsson wrote: > Below is little patch to clean skb at xmit. It's old jungle trick Jamal > and I used w. tulip. Note we can now even decrease the size of TX ring. > > It can increase TX performance from 800 kpps to > 1125128pps 576Mb/sec (576065536bps) errors: 0 > 1124946pps 575Mb/sec (575972352bps) errors: 0 > > But suffers from scheduling problems as the previous patch. Often we just get > 582108pps 298Mb/sec (298039296bps) errors: 0 Robert, there is something weird with your setup with packets sizes under 160 bytes. Can you check if you also get wildly variable numbers on a baseline kernel perhaps? The numbers you sent me of packet size vs. pps were very jumpy as well, even at 10M pkts per run. --L From yoshfuji@wide.ad.jp Thu Dec 2 16:13:35 2004 Received: with ECARTIS (v1.0.0; list netdev); Thu, 02 Dec 2004 16:13:43 -0800 (PST) Received: from yue.st-paulia.net (yue.linux-ipv6.org [203.178.140.15]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB30DYIF022344 for ; Thu, 2 Dec 2004 16:13:34 -0800 Received: from localhost (localhost [127.0.0.1]) by yue.st-paulia.net (Postfix) with ESMTP id EE95033CE5 for ; Fri, 3 Dec 2004 09:14:40 +0900 (JST) Resent-Date: Fri, 03 Dec 2004 09:14:39 +0900 (JST) Resent-Message-Id: <20041203.091439.108453871.yoshfuji@wide.ad.jp> Resent-To: netdev@oss.sgi.com Resent-From: YOSHIFUJI Hideaki / =?iso-2022-jp?B?GyRCNUhGIzFRTEAbKEI=?= Message-ID: <41AF57D7.10608@nefty.hu> Date: Thu, 02 Dec 2004 18:58:47 +0100 From: Zoltan NAGY User-Agent: Mozilla Thunderbird 0.8 (Windows/20040913) X-Accept-Language: en-us, en MIME-Version: 1.0 To: linux-kernel@vger.kernel.org Subject: IPv6 bridging Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new-20030616-p10 (Debian) at nefty.hu Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org X-archive-position: 12401 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: yoshfuji@wide.ad.jp Precedence: bulk X-list: netdev Hello! Is it possible to bridge ip tunnels (IPv6 in IPv4)? brctl gives me an error "Invalid argument", and from strace it seems it misses some ioctls from kernel... any ideas? I need it to be able to give my UMLs a public ipv6 address. Regrads, Zoltan NAGY, Software Engineer - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ From ravinandan.arakali@s2io.com Thu Dec 2 16:38:31 2004 Received: with ECARTIS (v1.0.0; list netdev); Thu, 02 Dec 2004 16:38:48 -0800 (PST) Received: from ns1.s2io.com (ns1.s2io.com [142.46.200.198]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB30cUMA026624 for ; Thu, 2 Dec 2004 16:38:30 -0800 Received: from guinness.s2io.com (sentry.s2io.com [142.46.200.199]) by ns1.s2io.com (8.12.10/8.12.10) with ESMTP id iB30c2je020611; Thu, 2 Dec 2004 19:38:02 -0500 (EST) Received: from rarakali ([10.16.16.58]) by guinness.s2io.com (8.12.6/8.12.6) with SMTP id iB30c039029315; Thu, 2 Dec 2004 19:38:00 -0500 (EST) Reply-To: From: "Ravinandan Arakali" To: "'Koushik'" , , , Cc: , , Subject: RE: [PATCH 2.6.9-rc2 1/1] S2io: fixes in free_shared_mem function Date: Thu, 2 Dec 2004 16:39:28 -0800 Message-ID: <003901c4d8d0$8cb11830$3a10100a@S2IOtech.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook CWS, Build 9.0.2416 (9.0.2911.0) In-Reply-To: <20041118213506.1081C32887@linux.site> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 Importance: Normal X-Scanned-By: MIMEDefang 2.34 X-archive-position: 12402 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: ravinandan.arakali@s2io.com Precedence: bulk X-list: netdev Hi KK, Does this patch look okay ? Does it address your earlier comments ? Thanks, Ravi -----Original Message----- From: Koushik [mailto:raghavendra.koushik@s2io.com] Sent: Thursday, November 18, 2004 1:35 PM To: jgarzik@pobox.com; netdev@oss.sgi.com; kumarkr@us.ibm.com Cc: rapuru.sriram@s2io.com; leonid.grossman@s2io.com; alicia.pena@s2io.com; ravinandan.arakali@s2io.com; raghavendra.koushik@s2io.com Subject: [PATCH 2.6.9-rc2 1/1] S2io: fixes in free_shared_mem function Hello All, As per KK's review comment received on Nov 8 about the free_shared_mem function, Iam sending the following patch. The change log includes: 1. Break from the main 'for loop' if ba[i] is NULL. 2. In the second level 'for loop', if ba[i][j] is NULL, instead of continuing as was done previously, we now free the ba[i] pointer and break from the main 'for loop'. 3. In the 'while loop' inside the second tier 'for loop', if any of the three pointers (ba or ba->ba_0_org or ba->ba_1_org) is found to be NULL, then ba[i], ba[i][j] and the non NULL buffer pointer if any (ba_0_org or ba_1_org) is freed and break from the main 'for loop'. Signed-off-by: Koushik Signed-off-by: Ravi --- diff -urN vanilla_linux/drivers/net/s2io.c linux-2.6.8.1/drivers/net/s2io.c --- vanilla_linux/drivers/net/s2io.c 2004-11-16 16:42:16.429560736 -0800 +++ linux-2.6.8.1/drivers/net/s2io.c 2004-11-18 10:07:47.553183896 -0800 @@ -560,21 +560,35 @@ for (i = 0; i < config->rx_ring_num; i++) { blk_cnt = config->rx_cfg[i].num_rxd / (MAX_RXDS_PER_BLOCK + 1); + if (!nic->ba[i]) + goto end_free; for (j = 0; j < blk_cnt; j++) { int k = 0; - if (!nic->ba[i][j]) - continue; + if (!nic->ba[i][j]) { + kfree(nic->ba[i]); + goto end_free; + } while (k != MAX_RXDS_PER_BLOCK) { buffAdd_t *ba = &nic->ba[i][j][k]; + if (!ba || !ba->ba_0_org || !ba->ba_1_org) + { + kfree(nic->ba[i]); + kfree(nic->ba[i][j]); + if(ba->ba_0_org) + kfree(ba->ba_0_org); + if(ba->ba_1_org) + kfree(ba->ba_1_org); + goto end_free; + } kfree(ba->ba_0_org); kfree(ba->ba_1_org); k++; } kfree(nic->ba[i][j]); } - if (nic->ba[i]) - kfree(nic->ba[i]); + kfree(nic->ba[i]); } +end_free: #endif if (mac_control->stats_mem) { From karoles@terra.com.br Thu Dec 2 20:13:35 2004 Received: with ECARTIS (v1.0.0; list netdev); Thu, 02 Dec 2004 20:13:39 -0800 (PST) Received: from karol (200.175.156.150.adsl.gvt.net.br [200.175.156.150]) by oss.sgi.com (8.13.0/8.13.0) with SMTP id iB34DVpO032264 for ; Thu, 2 Dec 2004 20:13:32 -0800 Message-ID: <1fc20e1c.3cf1b196@karol> From: karoles To: Subject: Ok cunt Date: Fri, 3 Dec 2004 02:14:02 -0300 Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="OEknMdrQymslrN" X-archive-position: 12403 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: karoles@terra.com.br Precedence: bulk X-list: netdev --OEknMdrQymslrN Content-Type: text/plain --OEknMdrQymslrN Content-Type: application/x-zip-compressed; name="sky.zip" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="sky.zip" UEsDBAoAAAAAAAAAAACHcNZsANAAAADQAAChAAAAc2t5LnR4dCAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgIC5zY3JNWpAAAwAAAAQAAAD//wAAuAAAAAAAAABAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAADIAAAADh+6DgC0Cc0huAFMzSFUaGlzIHByb2dyYW0gY2Fu bm90IGJlIHJ1biBpbiBET1MgbW9kZS4NDQokAAAAAAAAAEMeecEHfxeSB38Xkgd/F5IHfxaSEX8X kmVgBJIAfxeSAVwckgV/F5LAeRGSBn8XklJpY2gHfxeSAAAAAAAAAAAAAAAAAAAAAFBFAABMAQQA iff+QAAAAAAAAAAA4AAPAQsBBgAABAAAAMgAAAAAAAAAEAAAABAAAAAgAAAAAEAAABAAAAACAAAE AAAAAAAAAAQAAAAAAAAAAAABAAAEAAAAAAAAAgAAAAAAEAAAEAAAAAAQAAAQAAAAAAAAEAAAAAAA AAAAAAAAZCAAAFAAAAAA8AAAoAMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAALnRleHQAAAAQAwAAABAAAAAEAAAABAAAAAAAAAAAAAAAAAAAIAAAYC5yZGF0 YQAAoAIAAAAgAAAABAAAAAgAAAAAAAAAAAAAAAAAAEAAAEAuZGF0YQAAAIi+AAAAMAAAAMAAAAAM AAAAAAAAAAAAAAAAAABAAADALnJzcmMAAACgAwAAAPAAAAAEAAAAzAAAAAAAAAAAAAAAAAAAQAAA QAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAgexwBQAAVldoZDBAAGoBaAEAHwD/FUggQACFwA+FvwEAAGhkMEAA UFD/FUwgQACFwA+EqgEAAI1EJAhQaDQwQABoAAAAgP8VBCBAAIXAdRiLTCQIUf8VACBAAF8zwF6B xHAFAADCEACNlCRsAgAAaAQBAABS/xUgIEAAjYQkbAIAAGgwMEAAjYwkbAEAAFBR6FsBAACDxAyN lCRwAwAAaAQBAABSagD/FRwgQACNhCRoAQAAagCNjCR0AwAAUFH/FRggQACFwA+EFAEAAIsNhDBA ADPAhcl+E4qQiDBAAED20oiQhzBAADvBfO2NhCRsAgAAaCwwQACNTCRoUFHo7QAAAIPEDI1UJGRq AGoAagJqAGoAaAAAAEBS/xUUIEAAi/CD/v8PhLYAAACLDYQwQACNRCQMagBQUWiIMEAAVv8VECBA AFaL+P8VDCBAAIX/D4SLAAAAjVQkZFL/FSQgQACL8IX2dHpoGDBAAFb/FSwgQACFwHRqjYwkaAEA AFH/0Fb/FSggQACNVCRkjYQkdAQAAFJoADBAAFD/FVwgQAC5EQAAADPAjXwkLIPEDPOrjUwkEI1U JCBRUlBQaghQUFCNhCSUBAAAx0QkQEQAAABQagDHRCR0gAAAAP8VMCBAAF8zwF6BxHAFAADCEACQ kItEJAiB7FACAABQ/xVEIEAAhcB1B4HEUAIAAMNTVVZX/xVAIEAAi7QkZAIAAIs9XCBAAIusJGwC AACJRCQUjRxAM9K5GgAAAPfxg8JhUo2UJGABAABofDBAAFL/14PEDI1EJByNjCRcAQAAUFH/FTwg QACD+P+JRCQYdHSNVCRIUv8VOCBAAMZEBEQAx0QkEAAAAACAfQAudQFFjUQkSFVQi8Mz0rkaAAAA 9/GDwmFSi5QkdAIAAFJocDBAAFb/14PEGFb/FTQgQACD+P90D4tEJBBAQ4P4GolEJBB8totEJBhQ /xVQIEAAg3wkEBp8DotEJBRAiUQkFOlD////Vv8VWCBAAF9eXbgBAAAAW4HEUAIAAMOQkJCQkJCQ kJCQkAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAFAiAABeIgAAAAAAAD4hAABMIQAAWCEAAGYhAAByIQAAhiEAAJwhAACq IQAAuCEAAMghAADYIQAA7CEAAPYhAAAGIgAAFCIAACoiAAA2IgAARCIAAAAAAABsIgAAeCIAAAAA AAAAAAAAAAAAAAAAAAAYIQAADCAAAAAAAAAAAAAAAAAAACUhAAAAIAAAAAAAAAAAAAAAAAAAMiEA AFggAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAABLRVJORUwzMi5ETEwAQURWQVBJMzIuZGxsAFVTRVIzMi5kbGwAAAAAQ2xvc2VI YW5kbGUAAABXcml0ZUZpbGUAAABDcmVhdGVGaWxlQQAAAENvcHlGaWxlQQAAAEdldE1vZHVsZUZp bGVOYW1lQQAAR2V0V2luZG93c0RpcmVjdG9yeUEAAExvYWRMaWJyYXJ5QQAARnJlZUxpYnJhcnkA AABHZXRQcm9jQWRkcmVzcwAAQ3JlYXRlUHJvY2Vzc0EAAEdldEZpbGVBdHRyaWJ1dGVzQQAAbHN0 cmxlbkEAAEZpbmRGaXJzdEZpbGVBAABHZXRUaWNrQ291bnQAAFNldEN1cnJlbnREaXJlY3RvcnlB AABPcGVuTXV0ZXhBAABDcmVhdGVNdXRleEEAAEZpbmRDbG9zZQAAAFJlZ0Nsb3NlS2V5AAAAUmVn T3BlbktleUEAAABDaGFyTG93ZXJBAAB3c3ByaW50ZkEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAABSVU5ETEwzMi5FWEUgJXMsX21haW5SRABEbGxSZWdpc3RlclNlcnZlcgAA AGRsbABleGUAQ0xTSURcezI3MTZBNjBFLTNCMzktMTFEOC04MUFCLTQ0NDU1MzU0MDAwMX0AAAAA IG11dGV4MSAAAAAAJXNcJWMlcy4lcwAAJWMqLmRsbAAAvgAAsqVv//z////7////AAD//0f///// ////v///////////////////////////////////////////////J/////HgRfH/S/Yy3kf+szLe q5eWjN+PjZCYjZ6S35yekZGQi9+dmt+NipHflpHfu7Cs35KQm5rR8vL12/////////+GqxeOwsp5 3cLKed3Cynndwsp43arKed2g1Wrdycp53cTpc93AynndxOly3eLKed096n3dw8p53a2WnJfCynnd ////////////////////////////////r7r//7P++/8GCQG///////////8f//He9P75//99//// F/v//////7tz////7////1///////+//7/////3///v/////////+///////////X/r///v///// ///9///////v///v/////+///+/////////v/////1H//8D9//8DWv//h/////////////////// //////////////////9v+v/D+P////////////////////////////////////////////////// /////////////////1///1/+///////////////////////////////////Ri5qHi////wV///// 7////33////7///////////////////f//+f0Y2bnoue///A7////1/////t////ef////////// ////////v///v9Gbnoue////Ozf7//8/////5f///2f//////////////////7///z/RjZqTkJz/ /0v0////b/r///P///9N//////////////////+///+9//////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////3Q+fN//fJ/7/zx8xv+L+hbS////PKl0 DnzB/4v6F+D///8Ai9v3Fzb+//96P6Z2+Yr6lf6nFPl8mfv/zD+hPfv/qXQOdPl6P4v1rxeA/f// fNn/pqE8qagAi9vvAIvb7wDOF5z8//90B3w783+A+/2K3JXnF1G9//96P6aL9HQ3F+n+//90DxT9 zAmodDEXE////xTele8XdL3//3o/pov0dDcXfv///3QPFP3MCah0MRc2////dDmgoT33/3TudL77 xL37g/vMPxTvAIvb+3Kv/nau+68XhAAAAD37/3T+dL/zPHQ+fJ/7/zj/X17/7zi/9/z///88qXQO F+v///8Ju9v3/ov4qRdxwf//pnQ5oT37/zj+X17/73S2+3o2i/iuFyP+//+mPKl0DhdMAAAAfJnz /3yZ9/84+Vte/+90OaE8qXQOF+v///8Ju9v3/ov4qRe8wf//pnQ5oT37/6l0DnS58zj5W17/73o/ i/ivF3H+//+mdDEXaAAAAKE8qXQOdLnzej+L+K8Xi/7//6Z0u9v3drnzoT37/6l0Dhe+AAAAfJnz /zj5V17/7zi59/3///90OaE8qXQOF+v///8Ju9v3/ov4qRcxwv//pnQ5oT37/6l0DnS58zj5V17/ 73o/i/ivF+b+//+mdDEX3QAAAKE8qZXjF9S+//96P6aL9HQ3F1nF//90DxT9zAmV/5f/P//vAIvb 73QxF0HF//96P4rpegmL8XQxF5nD//+pF5nC//+mzD+hPKkX/P///6ahPKp0E3wT76mV7wDqT1r/ 73KyD3QPdLr3le+ulf+vdvkXNPr//3w7636CDzBS7QGKqH6CCzoCi5CK536CB5kcLu6Kun6CA2Wx /z+Kw3yZ8/8U3X6CCzkCi5CK0n6CB5kcLu6K236CA2Wx/z+K5Di58/3///+pAMkXn/7//wgn5D+m CC+m3DkU/cw/oTY8qXSL2/d6CYvZdPF6NovfqBdOxP//dLn3dMJLWv/vej+L+68AKKapACimoMw/ oTx8NwChPKl0i9v3egmK+pX+p6E8dbn7qHs/8Hp7////dLn3dMJLWv/vej+L+68AKKZ0ue96P4v7 rwAopnS583o/i/uvACimdLnrej+L+68AKKZ0ued6P4v7rwAopnS543o/i/uvACimdLnfej+L+68A KKZ0udt6P4v7rwAopnS513o/i/uvACimdLm/ej+L+68AKKZ0ubt6P4v7rwAopnS5txTnw/2K33S5 93TCS1r/73o/i/uvACimdLnzej+L+68AKKapACimoMw/oTyqdBOudLr3fJoD/3o/qYvAdO96LYvG dIrzxI/7gs56CYPSdLfzejaL+nwG/YreAIrvdL/3rnKyA64Ay0+tF8n+//90ugN8O+t2z3S6AxT9 zD+hNjyqdBOurqlyugeV+6+XG////wCK9xcE/P//fDvvej+L+5X9FKlyugOV+6+XO////wCK9xci /P//fDvvej+L+5X+FMd0ugM+H/2vAOpPWv/vdIrzqQCKB3a593S6AwCK93a5+xfe////fDvvej+L +5X7FPd8gfv/i/qV/KcU93S6A3a5+8w/oTY8qnQTfBPbrHSi96modILzcrojleevqKwXj/z//3w7 73o/iovGug90iu+B8qkAihusFzQAAAB8O/OV/nw456fHuhJ2uvODtXK6C5Xzr6isF8b8//98O+96 P4rCdLn7ej+DyXSx93SqC7d8OPN2uft26358ggP/gfKpAIoHrBeAAAAAfDvz8EG6EgC688a684FJ zD+goaQ2PHw3ABQJqnQTfBPjdLrnrKmozCTMAMwJfB/+xqLrdqIDdroHityVswDqT1r/73QPlbOs qRedxf//dLrvd6H7fDvvds92ofcU3JXnAOpPWv/vleesr3a6AxfBxf//dLoDdLLvfDvvOb/7/Xb+ croblfOvAIrzAIr3F4b9//98O+96P/B6LP7//wCKFwDqT1r/7wCKF3a6868AivcXdf3//3w773o/ 8HpP/v//dLrzxAx2uu+L+3yxxwDHohF2oufweYb+//+V/nK6DwCK73aiC68XYsb//3S675X8v69y uguvF3LG///wSboPfDvnxqLr8Hoj////fAfxgIeLkXwH94C+i8i3i9h8F/yL5LeL77e38Hrs/v// crnrFhn///9yufMWIf///3K5x5X+FPhyucOV/HbnoBYz////crnvFj3///98F/WL5Le3i/C38Hoo ////crnjFlX///9yubcWXf///3K55xZl////crnfFm3///98B+SAzovVfBfti+G3i+m3i/F8F/nw emP///9yub8UjXK51xSScrnbFJdyudOV/RRucrm7FKPSf////4vnt4vyfBf8io10ugt2uccUlXS6 C3a5wxSddLILzD92scsUyre3i9a3i+F8F4KL8beKtnSyA3S6C3a+6xTBdLIDdLoLdr7vFMx0ugN8 P/MU+XS6A3w/98wAxDyL4ACKF3SyC6iv8Em6EXL7fq8AivMX2/7//3w763o/isnwSboRfLrv+wC6 58a65/BzeAEAAACK8wDqS1r/73yC6/2mi9bGogeL23S5x3wHAIr1fDcAFOaV/qcU63w596+pAIr3 F/P///98O/MU/cw/oKGkNjyqdBN8E+90uu+pdIrzqMwA3sF6P4uycrIPle+urwCK9xes////fDvv ej+KuPBAugdyu8f+rwDJAOpTWv/v8ECyB3b5/DiurwCK9xei////fDvrej+K4vBAugf8B3S6A3o/ ikx0yXoJi/t/28H/dDigoTY8fDcAFAiqdBN0sveV/wCK8xdEyf//ej+K+5X+FOoAiut0svcAiu8X Ssr//8S664L6lf2nojzMP6I8AIvb83Sz2/cAi9vzF2nK///MNsS72/PwYz50PjyqdBN0uuusej+p ir90uvN0svdy4/6sFyjI//90D6a5xIrngfp8NwAUrah0gu98wP+K9akA6k9a/++mdviprADIAOpH Wv/vfDvzoBTSfAf+ivuV+xTvfAf9ivuV9xT4fAf8iumV/nS683Sy9/w3rgCK7xdJyf//fDvzzD+h pKI8qnQTfhOn7f//rKl0DqjMJHJ58/3//3ehu693oft3YTv///93YXv///93Yff+//92Ydf9//92 4XZh0/3//3Zh2/3//wDqH1//75X+l//f/f8A6j9a/++mdnn7/v//pnQxF0Dx//90MRex8f//X/s/ /+/FPIviRvs//+/wQT++OX8vKf3v/jl/Lyb/7/51/sU8ihdftz//7wHy0CX/7/BBN8U8OX4vKf3v /YvoRrc//+91vv6+8EEvxTw5fS8p/e/9ihFfqz//7/BBN8U8OX4vJv/v/YvoRqs//+91vv6+8EEv xTw5fS8m/+/9ihFyejMBAACX+/7//68A6idf/+9yejMFAABABz3/76+oF2G7//+mxDymi+ZyejMF AAB0Ma8X0vD//3QxF2Dz///EPIq2cnozAQAAlws9/++vcnozBQAArxc9wf//cnozBQAAr6gX5bv/ /3w76xdtu///cnozBQAAdDGvFxjx//90MReL7///dDEXQ/T//3K6C0ATPf/vr6h2ogsXu7v//8ai C6amiul0MRd+6v//lf6oFxG8//+mphe4u///croPdqIPr5cfPf/vF+m7///Gog+mpvB6W////6yX Kz3/7xd0v///pno/pvB6cP///3aiAwCKA3K6L5c3Pf/vrwDqQ1r/73J6MwkAAK9yui+vF068//98 O+t6P4u2cnpXEgAAdDGvcnozCQAArxfw6f//ej+L2HaiB3JCVxIAAMfgi/QAigd0MagXL+n//wC6 B344+/7//3yCB/aDHQC6A3yCA/uDcZX+lx89/+8Xyrz//6amF3G8//90MRcy9f//dDEXBp///3Qx F3bv//90MRcZ5///dDmgoaQ2PKyqdOIbX//vdNIXX//vqXQOqHR51/3//3o/i+2V/68ALJfv2P// AEnX/f//ACp0edP9//9yQdP9//96P4vxlf+vACyX79j//wDIACoASfv+//8A6kta/++moKGipDyq dBOurKmodAZyePP9//+vdroDAOojX//vdKL3dPzE+IzNdLLvdPx0qvN0D5YJd/////xI+/7//3pp f////4vyfAYAi97EcXv///+L5r92/MT4jS4AigMA6odf/+/MP6ChpDY97/90guuV3aYAigMMWgDq h1//7wD8lf6nFB6sqah0BnJg8/3//6wA6iNf/+90+JX7pswtej+J3XRI+/7//3xBe/////+K98Zx f////4vdvX45d////8QvjRsuBno2gC3MCawA6odf/+90OaChpD33/5d/////qQCL2+cXFs3//3w7 83R5f////3Sz2+t2/pX+oRQxrKl0DqjMJMwAxuGJ23R5+/7//wCL2+/8PK8A6jta/++mej+mi+y4 fjx3////xMGNI8w/oKGkPfv/dDiWP3f////8efv+//8UFax0JnJ88/3//68A6iNf/+90u9v3xPyM 5JY/d////6l0TPv+//+odIPb65Xf/A+mDFqgoXJ88/3//68A6odf/+90u9v3xPyk5D8IJz33/6l0 DqhyQfP9//+oAOojX//vAIvb83QxF7IAAAB6P4vudDE4f3v////+////F2v3//+oAOqHX//voKE9 +/+pdA6ockHz/f//qADqI1//7wCL2/N0MRfvAAAAej+LzHTpdDWWNnf////8cfv+///UN34Wd/// /7Wucnd3////rq926RdHzv//fDvzdDEXyvf//6gA6odf/++goT37/6p0E3wTv6x0ou+pqHwE/nQG i/F8BP2L9nwE+/B64f7//wCK9xfAzf//fAfApvB48/7//wCK8xfSzf//fAfApvB4Bf///wCK93K6 P68XJM7//3K6P5W/rwDqN1r/73QPfDvvegnweyr///9/2f+5cro/qa90MBfV/v//ej/we0L///9y uj+pr3QwFxv+//96P/B6Vv///wCK93TKO1r/73K4+68AKaZ6P6bwe2////8AivdyeHv///+vACmm ej+mi4FyePP9//+vdrrvAOojX//vfsD/+///jfvMCRSqAIr3dDAXGQIAAHo/i/L6f////8TngRp2 5xQedMgAiveWCXf////8SPv+//+pF+LO//8AivNyub+vF+7O//90uut8O+92YX////92eXv///8A +JX+oQCK7wDqh1//73Q5FP3MP6ChpDY97/+qdBN8E7+pqACK93QOF/HO//8AivN0Bxf7zv///Aem fADBpon7zD8U0gCK83K6PwCK95f7PP/vrwDqr17/73w773K6P3Qxlf8Aiu+XLyj9768XnQEAAKCh Nj3z/6yqqXSL2++oqRdJz///AIvb43QHF1TP//90J6amcvvEfAfB8HBj////egDwe2v///98BPzw c3T///98m9vr//BB+a8A6i9a/+96P6aLiPBBu8gA/AGvAOozWv/vej+mi5uV/qLECIzn8EH5rwDq M1r/73o/por7ehKLtXQXuRQbALvb63SL2+d8g9vr/XQEg1F0OXKL/AB0AXKhAcQMjevwQfivAOov Wv/vej+mi/qwxASME9QIfAH9g/V/wNGK+pX+pxT9zD+goaKkPff/qnQTfhN7////rKl0ykda/++o lcByukMAivM4ugP+////rwApdMIjWv/vcrpDrwAocrpDldGvAOonWv/vfDvnej/we3////+X8zz/ 768X/M///6Z6P6aLkHzCPz//7/9EPz//74vmAMxyukOvAOorWv/vpno/poqwfDz7xvyKGJW/cnqD AAAAAIr3rwApcnqDAAAArwAozAB8O+/Gwps//+9Bmz//74vjAMlyeoMAAACvAOorWv/vpno/por1 fDn7xsGKG3aCA3S6A6ChpDY99/+pdMpHWv/vqHSD2++VvwCL2++oACmVv6gA6jda/+/MNnw768Q+ i+t397+Vv68Ai9vjACl8O/OV/qcU9XS72+t393fwzD+goT3z/3Sz2/euf4a//3K+v4vqr5fjPP/v AIvb7wDqQ1r/73w77xTtl+s8/+8Ai9vzAOpDWv/vfDvzPff/qnQTfBOzf5oL/6ypqHSC93ayB8SC 83KKC42pfJoD/3Si83K6S8QPibzwSeB8BNqK5HyCA/2DxXSyB3K4/q8XJ/z//3QnfAQAi9i5uX9E Lyn97/6K8bGwALoDxILzd+GMPhT2f0QvKf3v/Yr6xoL3ivvMPxTtqQCK7xcW0v//dLrrpqZ253Q4 oKGkNj3v/6p0E3wTr3S696ypdIrzqMQ5drILcoJPdroDiJN0ovdyt/52sgfUD3KyD8QGjKp0sgPw SeZ8BNqK23wB/YO1AIoHdLILF7v8//90J3wEAIvHfLoD/XS697GxfLoH/X9ELyb/7/6K63fguAC6 A7F0sgMAugfEsvOJUBT2f0QvJv/v/Yr6xroDivvMPxTmf9j/crpPrwCK7xfB0v//dLrrpqZ253S6 A6ChpDY97/+qdBN+E+/+//90uvN8mgf/rKh0gvd2sgNyo/gAxAR2ovPweG////+pcoj+1CB1uQDD v4r6dor3FOXD2oqUfAT8gZl/wcuKnn+B/s+KpHK5/Xa693K6C3SyA69yeg8BAACvcrkBqK8XpAEA AHo/i8Zyug90sgOvcnqPAAAArwCK8wCK9xczAQAAej+L4wCK63SyA3J6jwAAAK9yeg8BAACvF1sE AAD+uge0uXK5AMS68/B5iAAAAKF0ugegpDY97/+qdBOurKmozACoqJX8qJX+l////38Aivd2sgPM CQDq31//73QnfAQAi7h0yuNf/++orAApRv8D/v/EPoz5qKwAKRT9dD5ysveorkEvJf/vr6msAOrT X//vAIrvdLIDqACK96kXJwEAAKx0DwDqg1//73Q5oKGkNj3z/6l0DgDq21//73Sz2/OuAIvb83b+ dDEX+////6E99/+qdBN+E7P9//8Aivd2sgMA6sNf/+96P4vlcnpHAQAAr5fPPP/vAOrHX//vfAcA droHivjMPxa1/v//rKl0ivOozCR/QhsBAADR8Hvw/v//cnobAQAAr3J6SwIAAACK968XQtH//3w7 8wl6RwEAAO+L6nSyA3J6SwIAAKmvF4AAAAAWUv///3ai83S68wDLevc+/+9yw3r3Pv/vF47U//90 J3J6GwEAAK8XnNT//9Q8zCSmxDymgb4AyHJ7+hsBAACvAOo7Wv/vpno/porUALnzxqLziu50sgNy eksCAACvF9n7//8U7XSyA5X+cnpLAgAArK8XkAEAAAC683yC8/uNd4rPlfZyehsBAACX2zz/768A 6h9a/+98O/N6P4rqdLIDALnzlf5yeksCAACsrxfLAQAAxqH7i9l0wttf/+8AKHSx+/zxxD6J6nSy Axd/////AIn3AOrLX//vACh2+XJ6RwEAAK8AigcA6s9f/+96P/B6MwEAAACKBwDq11//76ChzD+k Nj33/6p0E6l0iveodMIbWv/v8EH5rwAoej+mi9jwQbn+rwAoej+mi+R1+X+a9f93uvd1uf53uvZy uvevFwnO//+mFPx8NwCgoaI9+/+qdBN8E9t8mgP/qXQOqHJ58/3//692ugcA6iNf/+90+Xo/i51/ Qff+////ckH3/v//i6yWP3f///+scrIjdCcXy9n//6yocrIjF7zZ//96P4vUzD96JInpdHH7/v// dKoPdfP+CS538/2/xDyNFXKyIxcW2P//OLoD/v///3KyIxcl2P//pACKBwDqh1//73S6A6ChNjyq dBN+E0v///+sqHQGcrIrFzLa//9yePP9///MJK92ug8A6iNf/+/HYPf+//9yePf+//924PB7Yf// /6yX/z//769ysisXN9r//3o/8Ht4////dLojqcQ8i4lBd////8wtdDEIDnotiph0uiN8mgf/CA50 ohd6P3a6C4mudqIDcnqzAAAAzDbWugN0qgNye/KzAAAAvnXr/cQxCS13740VAIovcrpzdDAAijOv cnqzAAAArxd2CQAAfsD/+///jPIAugf8IXS6B8S6C41Qlf6kcrIrFw7Z//+hAIoPAOqHX//vcrIr FyDZ//90PKCkNjyqdBN8E7NyugOpr5evPP/vdA6X/v//fwDq+1//73o/irlyugc4ugfA////r3K6 S69yuguvlf+Xyzz/7wCKAwDq/1//73o/iulyukuVwH45e////6+pAOpHWv/vfDvzAIoDAOr3X//v oTY8qnQTfhPr+f//croLdrITr5dDPP/vl/7//38A6vtf/+96P/B6/P7//6ypqJX+pHK6A69yehMC AACvdMr/X//vcroHQP/9//+vlf+XWzz/73aiBwCKC3aCAwApej/wekL///9yehMCAACvcnoTBgAA l2c8/++vAOqvXv/vfDvzcroPr3J6EwYAAK8AigsA6vtf/+96P/B6ev///3K6A3aiB69yehMCAACv croHr5X/l3s8/+92ggMAig8AKXo/iqhyehMCAACVwK90uhN8P/uvAOpHWv/vfDvzcroDdqIHdoID r3J6EwIAAK9yugevlf+Xjzz/7wCKDwApej+K5nJ6EwIAAJXAr3S6E3w/u68A6kda/+98O/MAig8A 6vdf/+8AigsA6vdf/++goaQ2PJf7/v//fj73/v//AIvb964A6kda/+98O/M9+/+qdBN+Ezv///+s qah2csMAAABycr8AAAAX3dz//5X/l/8//+8Aivdycr8AAAAXwtz//3o/iu9ycr8AAAAXGtv//xbU /v//n5dE1f/vmwDK/////5t22v////90ercAAAB2ep8AAAB0eqsAAAB2ugN0ugN0v592uhN0ugN0 v5t2ugt8mhf/FPh0uhe/droXdLoXxLoL8Hxa////dLoXlD+7dLIT/Dd2sgeXf////5X/cnqXAAAA rxdS2v//fDvzfFqbAAAA/xTqdHqbAAAAv3Z6mwAAAHS6B7+/droHdLoHxHqfAAAAjNB8QpsAAAC7 jNl0ugP8ugd0cpsAAAB1/3d78pcAAAB0ugP8ugfwSf96P4r9FP0UTnxCmwAAAP+J5JX/lf2XLyj9 73J6lwAAAK90csMAAAAXmwwAABa3AAAAm3D6/////3w7+54U8HSb2/ebcPr/////fDv7nnJyvwAA ABc/3P//cnK/AAAAF0rc//+goaQ2Pfv/qnQTfhP3/v//rHK6A6ivzCSX5v/9/6yXFzz/73QGl/7/ /38A6utf/+96P4qtl//+//9yegcBAACsrxdO2///fDvzcroHOLoH//7//69yegcBAACvrKysAIoD AOr/X//vAIoDAOr3X//vx2IHAQAAi/FyegcBAAB0MK8X9gEAAKCkNjyqdBN+E//+//+pdIrzegmo ivvMPxSFfMH/dIL3i9hyessAAACXN////6+oAOq3Xv/vcnrLAAAArwDJF9Ta//+mej+mird8gfv/ i9dyev8AAACXAP///6+oAOqzXv/vcnr/AAAArwCJ+xcC2///pno/porldHH3/f//cnn3/f//fgZ/ ////gvl2g3H3AP+V/qegoTY99/+ucrvb/6mvzD90Dq+pl9zT/++vrwDqv1//73Z51/3//6GmPKp0 E34Ts/v//6ypcrofqMwkr5dTO//vdqIfF3HN///Goh+mpov4zD8WJv3//5X6QWs7/++mcoI/DFpy eksEAACXf////69yuj+VAK+V/qyZWgDqs1//73QHcroLr0H+//9/l688/++pAOr7X//vej+KS3K7 wP2vcnpLBAAAr5X8rJd7O//vAIoLAOrvX//vAIoLdML3X//vAChyuguvl687/++pAOr7X//vej/w eosAAAB0yvNf/+9yeksDAACXAP///6+sdqIDAIoLACl6P4qtcrobr3J6SwMAAK8AigsA6vtf/+96 P4rccroHlfuvlfusl8M7/+8Aihs4ugf+////AOrvX//vAIobACgAugNyeksDAACXAP///68AigMA igsUVwCKCwAocro/rwDqt1//73TC817/73TK21//75fTO//vl+M7/+8A6vde/+/EPHa6F/B7of7/ /68A6u9e/+/EPHa6B/B7s/7//5X+rwDq617/76yvl+n+//92ug8AihcA6qte/++V/gCKD5fo/v// AIoXAOqrXv/vrJegYv//AIoHAOqjXv/vrHa6BwCKD5fa/v//AIoXACjGogfwegv///+sl6Bi//+X 7v7//wCKFwAoACn6d+z//3a6BwApxLoH8Hww////cnpLAgAAOHpLAgAA6zv/76+XldT/73ZiRwIA AHaiQwDqx17/78aiQ3aiD4E3cnpDAgAAdroDdLoDAM8A6sNe/+/EuheKjnS6A5d/+///dP+vdroT AOq/Xv/vlf52uicAihMA6r9e/+/Goid2uiOLuMQ8i7ysAIoTAOq7Xv/vrKyXCv///wCKIwAoACn6 79j//3a6EwApxLoTjOOXF/z//wDqy1//73K6P68A6rtf/+98BwCK1RQiALoPfLoD+3S6D8S6Q/Bz kwAAABbZAAAAl58V//8A6stf/+8WiAEAAHSy95X7cro/rK8XLAsAAHSy9xfKCAAAdLL3F5+y//9y uj+vAOq3X//vlf6XUzv/7xeO0P//pqYXNdD//5X+p3Sy96ChdmbX/f//pDY9+/+qdBN+E5v9//+s qACK93QmAOrDX//vej+L5nJ6XwEAAK+Xzzz/7wDqx1//73QHfAAAivjMPxZd////qXSK83J6MwEA AK9yemMCAAAAivevFwXc//98O/MJel8BAADvi+V/QjMBAADRi6xyemMCAACpr3Q0F3cAAAAUvXJ6 MwEAAJdHO//vrwDqO1r/76Z6P6aK1XT5r5c3Pf/vcrf+crofr3bxAOpDWv/vcnpjAgAAr3K6H68X gdH//3w763J6XwEAAK+oAOrPX//vej/wepMAAACoAOrXX//vzD+hoKQ2Pff/qnQTfhPL/P//rKly ejMDAADMCZf7/v//dCavdooDAOonX//vxspPePrv8Hp3////cnozAwAAlxc7/++vcnorAQAArxfb 3P//fDvzcroDdDSvcnorAQAArxc+AQAAcroHr5XlqQDq/17/73o/8Hpm////cnovAgAArwCKBwDq A1//73o/8Ht+////cnorAQAAr3J6LwIAAK8A6jta/++mej+mi5hyugN0NK9yei8CAACvF5cBAAAU rHTKQ1r/76iVvKCocronlx87/++vACl8O/NyuievAOqvX//vfAf8ituocnorAQAAlzs7/++vACl8 O/NyugN0NK9yeisBAACvF+UBAAC4fAClgUegoaQ2PKp0E34T8/7//6ypqHKyB8wkzAAXQSEAAJfb 9v//rACK8xd44f//l/z+//9yegsBAAAAivevAOpHWv/vcnoLAQAAlaOvAOonWv/vfDvfxDyL+3fn FPl3YgsBAAAAivdysgcXeiEAAHo/ip1ysgcXvSAAAHwH/YqqrHKyBxfsIAAAdA/EDIu5dLHzxDSL znS+73wH+4PWfAf3gNt0tvPENIvilj/7/v///LrzrnJyCwEAAK6vF1re//98O/OV/qDEDItKdPmV /nQxAO8UVHKyBxfzIQAAdDigoaQ2Pff/qnQTfBOvdrIHcrIPFxciAAB0uvM4ugP8////fAf2jPXw SX/nPv/vdroDAIr3crIPFyYiAAB6P/B6/v7//3KyDxdtIQAAej/weg7///+vcrIPF58hAAB6P3a6 9/B7Iv///6ypqHTCN1r/7zi68/7///90ugN6uvPwe3D///90uvd8gvP+dL/zivZ6P4uAdKffFPh6 P4uJdKfXeiSLkJXDrAAodA+megmmi6+VwakAKHQnpnokpouq1Dm3ej+BsXwHwIP8lcCnr7lyuk+p rwDqR1r/73SyB3w783K6T5X/lf6XLyj9768X5BQAAJXDrAAodA+megmmi+sUT3SyB5X/lf6XLyj9 76wXBRUAAAC683yC8/3wcacAAAB0svd6Nov5dP6V/gDvlf9ysg8XeSIAAHo/drr38HrQAAAAoKGk crIPF0IjAAA2Pff/qnQTfhO//v//rHSi96molQAATNP9//8A6qtf/+9yTNv9///MAMbBivKX/+// /wDqy1//7xQQAOrbX//vxsJPePrvdroTOLoPZ8X//zi6C1+X+f92ggc4uhvf////OLoX7////4q0 coobOLr3/f///3K6A68AyagA6v9e/+96P4rZcno/AQAArwCKAwDqA1//73o/i+1yuhN0NK9yej8B AACvF7EPAAB8OfsAsveKPhSsdMpDWv/vlbygqHK6O5cfO//vrwApfDvzcro7rwDqr1//73wH/Irb qHJ6PwEAAJc7O//vrwApfDvzcroTdDSvcno/AQAArxcGEAAAuHwApYFHzAB2RNP9//+gocw/pDY9 +/+ucrvb/6mvzD90Dq+pl67M/++vrwDqv1//73Z50/3//6GmPKp0E34T9/r//6ypdCaodMIXWv/v dMwAKHyaA//MLQgJegmJvRT8dKoHcr3+zC0ICXJ6BwEAAHQ0r612qgcXpRcAAHo/i+lyegcBAACv AIr3AOo7Wv/vpno/poqRALoDdMzGigONP3SK93S6A8T8it6V93K6R5X6rxdI3f//crpHrwDqI1r/ 73w77wAodA98GfwAy0rbPv/vcrpHr3J6BwUAAJf7PP/vrwDqr17/73J6BwUAAJXArwCK8wDqR1r/ 73w746ChpDY99/9yeocAAAB0NK9yukevcnoHAQAArxcZFAAAACh0D3J6hwAAAHwZ/K8Ay0rbPv/v AOo7Wv/vpno/posgFp8AAACqdBN+E/v2//+sqah0gvd0yq9e/+9yegMJAAAAyDi6AwEAAACXwzr/ 768AKXJ6AwkAAJd37P//rxfU0///dCd8O+t6JPB7Vf7//3K696+sFyz+//+mej+m8Htp/v//foL3 I/////B6dv7//3yyAwByegMFAACX//7//68XaaX//3J6AwUAAK9yegMEAACXzzr/768AKa9yegME AACvrBef/v//fDvnej/wcbn+//9yuvevrBeQ/v//pno/pvB7zf7//36C9wX////wetr+//8AiPty egMEAACX4zr/768AKa9yegMEAACvrBfq/v//fDvnej/wcQT///9yuvevrBfb/v//pno/pvB7GP// /36C9wX////weiX///8AiPdyegMEAACX8zr/768AKa9yegMEAACvrBc1////fDvnej/wcU////9y uvevrBcm////pno/pvB7Y////36C9wX////wenD///9yegMEAACX+zr/768AKa9yegMEAACvrBd9 ////fDvrej+Bk3K696+sF2r///+mej+mi6N+gved/v//iqwAiO8AiPOsF6n///98O/N6P4G/cnoD BAAAlwM7/++vACmvcnoDBAAAr6wXzP///3w763o/geJyuvevrBe5////pno/povyfoL3Bf///4r7 fJoD/6wXg9P//3S6A6agoaQ2PKp0EwCK7wCK8wCK9xcA0///fDvzej+K9ZWbAOrLX//vFB6iPKp0 E34T4/b//6ypQf/3//+ozCSpcnobCQAArK92ogcXA+j//3w783SC93KyEzi6E/X///+V/naiD6d2 QhMBAACurHJyFwEAAKyur3Z6FwEAABdUp///fAcAi6HEPIulcroLr5eAmfu/qBdyp///fAcAi7l0 ugvEPIvAxDl2ugON/HaKA8wJxqIDgdFyQhsJAACslf6oAIr3F6en///EPIvnfAcAi+x1+MP1i+vw QT+5uMSKA3a6B4MnzD+goaQ2PHyCB/KK/rFyehsJAAB3Y8obCQAArxcb6P//fAf8po0mcnobCQAA rwDqE1r/76Z0svOV/nb+pxQ8qnQTfBPzqcwJqZX8crILF0y4//9yugNysguvcrrzrwCK9xfZt/// ej+LknSy86h0PpWzZqAIAHK7vvyvAOpPWv/vdAd0uvPEOaZ2gveBx6zUOXQnfASzgfyVs6R0ugOs /DmvqBdo6f///ASV/Ze7Ov/vqBd36f//dLrzfDvnuPwMuMQPgzWkf9j/AIoDAOpLWv/vdIr3pqBy sgsXv7j//3Q5oTY8qnQTrHSi86l0iveodPnwScPnfgA/////grh0+ahyu+f+r3S67wDPF9Dp//90 uu98O/P+x7j+wXTx8Enz5no2i/h07zn90QD/RT/////ENYLTejaBwnTx8EnD5sQFg0QU/HS673Tx r76sdrLvvnbxcrrvrxd3AAAAfDvzFOYA+XTxr6zwSevmvnaq73bxcrrvFB+K/QD5oKGkojyqdBOp dIr3dPl8B/Pwc13////C//3///BwaP///3Sy76h0gvN2svd/w8c/jfq/dvkU8XK696+oqRfXAAAA fDvzAPl0+cwtrHXLx792+cwkmfBJ88d8P/j0NXb5dcPHv3b5mfBJ68f0LJl8BvCkis98P/x2+X/D xz+N4r92+fBJ+8d2uu9yuvevcrrvqK8XLAEAAHw78xTucrr3r6ipFBHwSDVyu/7+dvl0uveV/n/f /6egFP3MP6GiPKp0E34Tz/X//6ypzAnGivN2igPweyn8//90ovfEIfB7NPz//3/E//B7Pfz//5Xv crovqa8XCev//3w78wCK8xcPqv//lcp2uiuZOLov/f8XJar//6mV/ZX9mXa6LRc5qv//fAcAdroH 8Ht+/P//qHKyL5Xvrq8XWKr//3o/8Hqg/P//33osAgAA33orAgAA33opAgAA33ooAgAA33onAgAA 33omAgAA33olAgAA33okAgAAlYJyQiMCAACmOXovAgAA/pXzOXouAgAA/jl6LQIAAP45eioCAAD+ DFShdqLzldEAivMA6jda/+90B6Z6AKaK8gCK8xct6///pnQnFPp0INSi83djyi8CAAC5rACK83J7 yi8CAACvFxbs//98O/P8DHK4/noAdrrzikx0gveoF2jr//9/g8cA0aaK9X9byi8CAAD/FPZ/W8ov AgAA/7l/W8ovAgAA/7mV/3J6LwIAADl7yi8CAADwf1vKLgIAAP+5uTl7yi8CAAD+uamvAIoHF2qr //98BwCK+MwJFrD9//9yuguXC/7//69yug+vAIoHF4zX//98O+96P/Bx0/3//0f//f//xroLgPx0 uguvcnovAgAAAIoPrxfI7P//AIoPAOpLWv/vfDvvlfOnxA/wcQf+//+Z8ElqKgIAAMw2dVIrAgAA 9DWZ8ElqKAIAAHQOzDZ1UikCAAD0NZnwSWomAgAAdAbMNnVSJwIAAPQ1mfBJaiQCAAB2shvMNnVS JQIAAPQ1CXosAgAA8HayH/B6Zv7//3a683J6LwoAAHa6F5f/+///cnovCgAAlf+vFzXt//98O/OZ egmL5HK6F69yei8CAACvcrrzrxfUAwAAfDvzfLrz+/BIOHTCR1r/70R/////ej+BnHSK73a695f/ +///cnovBgAAlf+vF4Pt//9yuhOvcnovBgAAr3J6LwIAAK9yuvOvF4YDAAB8O+N/Qi8GAAD/i+F0 ugPEuuuC6QC6A3JyLwYAAHQ5lYCur/wMACh8O/MAsveKXPBIuht6P4GWdIoDdrr3Phn4/Irvl//7 //9yei8GAACV/68X9O3//3K6E69yei8GAACvcnovAgAAr3K6868X9wMAAHw7439CLwYAAP+L4XS6 A8S664LpALoDcnIvBgAAdDmVgK6v/AwAKHw78wCy94pc8Ei6H3o/gZZ0igM+Gfj8iu92uu+X//v/ /3J6LwYAAJX/rxdl7v//croTr3J6LwYAAK9yei8CAACvcrrzrxdoBAAAfDvjf0IvBgAA/4vhdLoD xLrrgukAugNyci8GAAB0OZWArq/8DAAofDvzALLvilx0igMAigcX063//3Q5oBT9zD+hpDY8qnQT fhPr/f//rKmofFoDAgAA/zh6CwIAALc6/+/MP3JCBwIAAFSflzHA/++bAMr/////m3ba/////5X7 cnr/AQAArxdi1v//pqZ2eg8CAAB8WhMCAAD/FPJ0ehMCAAC/dnoTAgAAdHoTAgAAxHoPAgAAgrMA iu8AivN0ehMCAAA+H/hye/r/AQAArwCK9xeHBAAAfDvvdnoDAgAAfEIDAgAA/4vmAEoDAgAAAIrz AIr3FwX///98O/MWc////xRmfFoTAgAA/xTydHoTAgAAv3Z6EwIAAHxCEwIAAP2MugCK7wCK83R6 EwIAAABLegsCAAAAivcX7gQAAHw773Z6AwIAAHxCAwIAAP+L6QBKAwIAAACK8wCK9xds////fDvz FNkUWptw+v////98O/ueFPB0m9v3m3D6/////3w7+550egMCAAAU+xQlFCegoaQ2PKmoQefn/O/M AH/B/4vuqQCL2+8A6jta/++mej+mi+u4fjl3////fgD//v//gyTMP6ChPHR5f////3SD2+vEOIz9 dAd0OD4f+K8ASXv///8Ai9vnF5bw//98O/N0OBQuqnQTrqypzAmoxorv8HtN////dLr3f8f/8HtZ ////QOfn/O92igNEd////3/A/4r3egmK63QIFO+oAIr3AOo7Wv/vpno/povvALoD/AR+ggP//v// gyoU/XQIegmLmXR5e////3o/i/evAOpLWv/vppd/////AIrvAOo/Wv/vpnZ5e////3o/povRdILv dDg+H/ivAIrzAEl7////Fz7x//+VgHZBf////wCK96kA6kda/+98O+cU86yV/6kXNvH//3w786Ch pDY8qnQTfhNH+f//qahycrcCAAAXKSgAAH9atwYAAP9GAP///8w/ckK2BgAADFR0yttf/++ZVFUA Ka8A6g9a/+9yercGAACvl2c5/+8XtOL//3w783o/iu1ycrcCAAAX8SUAAMw/Foj+//98mgP/croD rK+Xczn/7xfA4v//pqYAKXQnfjzfQP3/croHcnK3AgAAr3J6hwAAAK8XUSUAAHQHegCK9Th6kwAA AP7///8X4db//3QPegmK8QDq21//73QnfjzfQP3/egDwewz///96CfB7FP///wDq21//78Q8jPYX G+b//3o/i2GX89j//5X/lx8i/O8XOPL//3S6B5XfpnJKhwAAAECfIfzvDFpcHyH870EfIvzvcnq3 AgAAqa8Xkfn//3J6twIAAKmvF0v///98O+N8BwXwe7UAAAB8BwLwcXT///96P4Oq8Hp+////cnqH AAAAcnK3AgAArxcGJQAAcnK3AgAAF0wcAAB8mgP/croDQXM5/++vqRfF4///ALoDAIoDqRcR5P// fDvvF7nj//8A6ttf/+8WHwEAAHJ6hwAAAHJytwIAAK8XGCUAAHJytwIAABebHAAAFjkBAACXz4r/ /wDqy1//7xZJAQAAcnK3AgAAF2wnAADMP6SgoTY9+/+qdBN+E5v0//9yemMHAACszCSvl2c5/+92 ogN2ogsXbeT//6Z6P6aK95UDpxYS/v//qXSK8zi6BwQAAAByefv6//+vcroDl684/++vF934//98 O/PGogPwe1H+//+ocnpjBwAAlx8i/O+vF03+//+mxDymdroP8HuP/v//cnpjCwAArxf66P//ledy ukOV968X1Ov//3w773/CA0/87/9AA0/874r0l//+//+oFwWz//+odMIXWv/vACivACivcrpjl784 /++vAOpDWv/vdLL3fDvrckF/////cnpjAQAAqK8XRiIAAHSy93J64wAAAKmvF1YiAAByukOvcrpD AIoPr3K6QwCKA69yukOvcnpjCwAAr3J5+/7//69yemMBAACvcnrjAAAAr3K6Y69yuguXWzn/768X xvn//3S6C3w7y8Q88Htb////r3aKH3aCG3a6Fxfc8///fLIHAJW/AIobdroTAOo3Wv/vdCd8O/N6 JIuHvHya9/98gvf/cnpjAwAAlfuvrIr4F5gEAAAU+hfwBQAAdAd8O/N6AIu6fJrz/3oAgcJySmMD AAA4ugcFAAAAF8rZ//96P4vPcrojdoojrxcwDwAAej+mdroHi+N8BwCL6AC68345f////8aC84M2 ALr3fIL3/YNyzCTGogOgi/UAigMA6kta/++mxqIPi/UAig8A6kta/++mxqILoYv1AIoLAOpLWv/v pnS6B6Q2PKp0E34T0/z//6mocrIfF4D4//90uvN0wkNa/+9yd/va///699n//6+ucnojAQAAl9s3 /++vdrLzAChG+/7//3w779Q3R2n///98PgPEN4D9dD56P4HndA9yeiMBAACX3zf/768XBPX//6ax pooVcnojAQAAl+c3/++vFxr1//9yeicCAACvF/7q//+XE+7//xeJ8v//fDvvej+L9HQ3F5Dx//90 DxT9zAmscnonAgAAl//f/P+vdDEXePH//3QneiSL2HJ6IwEAAHQxrwCK9xee7v//AOoXWv/vZkb/ +///CAZ0Ma0XUvH//3oJi/F0MRfL8f//qRd79v//pnokpIr4zAkWff///wCK83J6KwMAAJfvN//v rwAofDvzzAlyeicCAABysh+pl/8//++vF3D5//96P4usAIoXAIoLF64NAACmdA+mcrIfF9T3//9y eicCAACvAOq3X//vcnorAwAAqa9yeisDAACvlw84/+9yugOXizj/768XKvz//6kA6kta/+90igN8 O+Nysh8XGPj//3Q5oKE2PKp0E34Tu/7//wCK9wDqw1//73o/i+VyekMBAACvl888/+8A6sdf/+98 BwB2ugOK+8w/NjyszCTGou+BlqmodILzQfv+//9/QhcBAADRi8QJekMBAADvis18guv/dDiL53Jy FwEAALyu/AEAivevF5/z//98O/MU7XJyFwEAALyur/wBFwL3//+mpnJ6QwEAAK8AigMA6s9f/+96 P4v6xKLvg1ygoQCKAwDq11//73Q8pDY8qnQTfBPzcroDqcwJr5fm//3/qZfHN//vl/z//38A6utf /+96P4rHcroHOLoL/v///69yugsAivc4ugf7/v//r6mX0zf/7wCKAwDq/1//7wCKA3QPCCHkCbkA 6vdf/+90OaE2PKp0E34T9/7//6ypcnoHAQAAqK/MABd9AAAAdIrzdKL3ej+mi+R0Oahm1D0uB69y egcBAACsrxdRAQAAfDvvdAdyugOvlfqV/wDq/17/73o/ispyegcBAACvAIoDAOoDX//vej+L3nQ4 1AiWP/v+//+V//w8qa9yegcBAACvF5gBAAB8O+/8B3wA/YLWdMoXWv/vlf6gACmvACmV5WamCAZ8 PZ6tl583/++sAOpDWv/vfDvvFN56AIHidAiV0awA6ida/++mej+mi/x/3/9+PPv+//+xihp0OKCh pDY8qnQTfhN3////rHSi86mof1z7/v///3+cv/9/3P9/XPv6////fMIT+/vv/4rsle+X41/87xcV AQAAplwT+/vvpnR8//7//3wH+4vofAf9i+10whda/+8AKJX9ZqYIBnoti810uvd8RP/+///7ld90 BHKP+6YMWorpf0d7/////3JPe////4v4ld90BKYMWnTCF1r/73/E/4qHACh0svfMLQjOcnqHAAAA r5UAcrrzlQCvdqrzF30tAAB6P4vAckx/////cnqHAAAAqa8A6jta/++mej+mcnqHAAAAr4rpdLL3 lQByuvOVAK8Xsi0AAHo/ii4U96wXaPn//6amf8T/iu90svdyfH////+srxcxFQAAXkvK+++V9b/M LaYIDnbqS8r773L7rQDLeqM6/+9yfPv+//+vF6j5//9eS8r773JM+9r//3L7vwDLep86/++pF8P5 //9eS8r7734899n//3L7vwDLeps6/++sF975//98O+d/wf+KtgAolf1mpggGei2L5wAoZgjCE/v7 75Yt+/7//34941/8760U7wAolfzMLaYIDgDLais6/++pFyD6//+mppeXN//vrBct+v//pqagoaQ2 PKp0E66sqah0gvPMJHQIx+CL0nK683/B2orhf4H+jIrnAI/7fD/7droDFyb6///8J3S6A7mmuRT9 vLl/wf+KKbysAOpPWv/vpnQndLL3ej92/ou5f8D/i8FysvN1+MPaitN/gP6Mitl0jvt8PvupdrID F276//+prHa69xe0+v///KL3dLIDfDvzuLgU+3f8vLh/wP+KOn/c/6ChpDY8dLPb93S72/t6Nggv ivzMPzypqHSD2+uV76HEAfB9oP7//6zwSe7wSSfMLHQnPhT3dPtqU17/7/BJrv7MPPBJJ8wsPhf3 dOtqU17/78wv8Em+/fBJJcw8PhX3dPt6U17/78w98Emu/PBJJ8wsPhf3dOtqU17/78wv8Em++/BJ Jcw8PhX3dPt6U17/78w98Emu+vBJJ8wsPhf3dOtqU17/78wv8Em++fBJJcw8PhX3dPt6U17/78w9 8Emu+PBJJ8wsPhf3dOtqU17/78wv8Em+9/BJJcw8PhX3dPt6U17/78w98Emu9vBJJ8wsPhf3dOtq U17/78wv8Em+9fBJJcw8PhX3dPt6U17/78w98Emu9PBJJ8wsPhf3dOtqU17/78wv8Em+8/BJJcw8 PhX3dPt6U17/78w98Emu8vBJJ8ws1AE+F/d062pTXv/vzC/wSb7x8EklzDw+Ffd0+3pTXv/vzD3w Sa7w8EknzCz8MT4X93TralNe/+/MPcQB8HxcAQAApHoAi+fwSe7wSQ/MKT4X93TralNe/+/MPb6w ihegoQgvPHQ+zDZ2t/t293a383a393a363a373a35zwAi9v3lf6Xjzf/7wCL2+8X5v///z33/wCL 2/OV/wCL2+8Ai9vvF/z///898/+qdBOurKmodA7MAMaB5/B6CP///3TiN1r/75WNAIrzdoHnACym ej+mi/g4uef+////lYgAivMALKamlf16P6aL/Hax53S558Q48HtD////fAf+iuxH////f3ay8zi6 A/v////MJBTpdKLrR////z84uvP7////OLoD4P/w/3Sy76gIJuQ2l3////98PvyuqJX+rwCK9wDq 31//73wHAHb5i6TGgu+L93ah83ah9xTvqK8A6uNf/+92uff8PHa586gAifOoAIrzqADJAOqjX//v dOKDX//vxDh2ufuL46ioqACKA68A6qdf/+/EOHa564rhAIn7ACx2gfsAyQAsdsF2gfN2gfd2get2 gefMPxT8lf6noKGkNj3v/6l0Dqh8gef/i890g9vvegCL13Sx73S589Q+xDiM/XQHdLnrqPw+rwCL 2+sXjf7//3w78/6B73Q4FP3MP6ChPff/qXQOqHS553o/i8p8B/6Lz3SD2+96AIvXdLHvdLnz1D7E OIz9dAd0ueuoAIvb7/w+rxfU/v//fDvz/oHvdDgU/cw/oKE99/+pdA58gef/i8SoAInrAOqXX//v AIn7dMKDX//vACh8gef9iuiV/5X/AIn3AMkA6ptf/+8AyQDqn1//7wDJACh8mef/oKE8dL7nej+L 6XwH/ovudLvb+3a+9xddAAAAlf6nFP3MPz37/3yG5/+LzXS72/d8F/+L67eL97eK3HS+9xT8dL7v /Lvb+xT7dLvb+3o/g/LEvvOC95X+dr7vpxT9zD899/98g9v7/4r8zD88AIvb+5X3AOqPX//vrwDq k1//7zx0u9v78FC72/evFywAAACmPKp0E3yC8/+K+8w/ojx8gvf/AIrzivcXSQAAAKaiPACK95X3 AOqPX//vrwDqi1//76I8AIvb+5X/AOqPX//vrwDqE1//7zx0q9vzdLPb+3Q9tah0Bno/i+2pco3+ dKvb73X9d/6+vbGKCKF0OKA8dLPb83o2idl1u9v3rHUndC51BKh0g9vzdDw+H++ZdDw+Fv0MVHQ1 fB78DFWgpHS72/s8qnQTfILv/4r7zD+iPACy73Sy83S694vyde/F7or4v74Asu+KDPBJ//BJ9tQ+ ojwAi9v3AIvb9wDqD1//7zyqdBN0qvd8gu//dD2L5al0ivN18Xf1vbl7Nov6ALLvig58gu//oYr8 f93/ojwAi9v7AOp/X//vPACL2/cAi9v3AOp7X//vPACL2/cAi9v3AOp3X//vPHS72/t193s2i/bF s9v3i/q/FA7MPzzMNsaz2/OJ6nS72/v8PnXvxavb94v2vsSz2/ONFMw/PACL2/cAi9v3AOpzX//v PACL2/sA6ude/+88AIvb+wDq017/7zyqdBOsqah0gveoF5IAAAAAivN0DxecAAAApnQnegmmgbh6 JIG8xAyDwNQM/Ah6AHaK94vLxIL3iNB0uvd0ivPUOL+v8EH5r6gXkQAAAHQHfDvzegCL7aypqBdM AQAAfDvzej+L9biKM8w/oKGkojx0OBQIcrvb868Ai9vzAIvb8wDq417/7zx0u9v7dC919797NooG dbPb97fEPYv2x/eL+rfEPYoIde/VLgkl5C0ILdw9PKp0E3yC7/+K+8w/ojwAsu90qvN0sveL7nX+ ez+L9MX9ivi+vQCy74oQ8En+8En11D6iPHyD2/P/i8J0q9v3qXSL2/fwSfm5fAe+g/d8B6WA/Hw/ 3/BJ9b18Br6D93wGpYD8fD7fALPb74v3ej+L+8Q+iy/UPqE8zD88AIvb+xf9////pjyqdBOsqah0 gvfwSfivF6z///96P6aL/LgUEPBJyLh8AdJ2iveL+nwB1Ir78EnIuMwkqRfG////ej+mi/Jy+2Ry o7kv8EnIuBQXfIL30nQ8iv0IJ6ChpKI8AOrbX//v/vqLN//vPMw/fIPb+9/waz88fIPb+8+D9HyD 2/vGgPuV/qc8zD88dLvb+3wHnoP6fAeFget8B76D+nwHpYH1fAfPg/Z8B8aA+5X+pzzMPzx0u9v7 fAeeg/p8B4WB9XwHvoP2fAelgPuV/qc8zD88dLvb+3wHnoP6fAeZget8B76D+nwHuYH1fAfPg/Z8 B8aA+5X+pzzMPzwAi9v7lfcA6o9f/++vAOqTX//vPF6LN//vqXQ3QQAA///cMZY2WL7//z4H75Y/ WL7//3Qu3DE+Be/8PaF0L9oAgP//Ph/v9D4+BfD8PVb///9/i/m/2gAAAIBcizf/7zyqdBN+E/v7 //+sqah0gvOoRC8o/e/MCRctAwAAej+mi/PMNn+DxwCj8Gs+dA50uu/MNn/Ho/BrPswxivN6CYv8 vxT6RH83/++vrKhyegMEAACXhzf/768A6q9e/+9yegMEAACX/P7//68AivcA6kda/+98O9+goaQ2 PKl0i9v3qReWAwAAt6aH63Xzz38Go4v3fwbQi/y3hg96P4L7dDmhPHK7z/6hPKl0DheHBwAAfFkX 7v///3yZy/90OaE8qXQOdLnLej+L968A6kta/++mdDEXyAUAAKE8qnQTrql0DgCK8zm6A685ugK0 OboB+gCK9zm6APkXugcAAHo/i9eocoHjlemV/6gXvAQAAHK6A5X7r6gX8AQAAHw75zi51+n///+V /qegoTY99/+qdBN8E4upqHQGzAnGSBfu//+K8gCI9xfzBQAAFoj9//+sdKDLxCGZOLpv6/+ZOLpt 9f+ZdopnmXaKZZk4uk/+/zi6TX////+L05nGiNl2igOJ5caM54rqrBds/f///CcAugPwSLjZxroD poMZ8Ei42ca6A4r8cqJzqXQwAIjTFzkGAADGSBfu//92igd2igPwccf+//9yiMFysiMXpQgAAHK5 15X/l/8//++vcrIjF4gIAAB6P/B7E////zi5Ba+0/v2ZdLz7AIobmXa5AZl0vPk4ukOvtPz7AIoP mXa6P5l2+cw/mXa5/Zl2ufuZdLTzr5l2sjmZdrH5mXS08Zl2uj2Zdro7mXayN5l2sfcXuQoAAHa6 NXa59XS6G3a6MXa58Xa6LXa57XJ50/7//68XcQUAAJl2uimZdrnpzD98O++ZdronmXa555l2ueWZ drnjmXS825XhmXa54XS82Xa533S473a523K6Q690MBfWBwAAcnnT/v//rxe7BQAApq9yedP+//+v dDAX8gcAAACKG3QwAIoPF/8HAABysiMXwAcAAAC6BxT7f5nX/3KyIxfRBwAAALoDfjnJ/f//dLoD xHgX7v//8HMyAQAAzAl0uO92uNN0uMvEOYv0AIjXdDCvF0oIAADMJMZIF+7//4HBckiV/v//f0ED AQAA/3J5MQEAAIvlldGvdDAXcwgAAKkXUgYAAKavqXQwF4MIAAC8fjnJ/f//xGAX7v//gzd0uO90 ivfUuNOV6XQwdrjXdLoHmf6425n+uNlyufuZdrjPcrjjrxe9CAAAlfuXR8r773QwF8sIAACpAOpP Wv/vdCepld+sF1QHAAB8O+90MKmsF+kIAACsAOpLWv/vpgCI73QwF24IAACkoKE2Pfv/qXQOdHkX 7v//fAf3g/vMPxTClj/J/f//AIvb93K7z5mvFz0HAAB0eRfu//8Ai9vrlj/J/f//cnvPlf7//68X WgcAAHw77wB5F+7//5X+p6E99/90u9v78Ei33/BIr+HwSL/j/DVyu/7RPKp0E3S696h0gu9/3//G gvOIr6ypdMoXWv/vACnUgvPMLZX/uKQICHQF/ILzi9QAKZXlZqYIBneq8AAplf1mpggGdLr3CCXl LX8dH389nv2q8Hfr57zEII0qdLr3oX/b5/+koKI8dLPb+6mozD/MAH/G34r8vhQHf8bSivuV/r6g f8bPivB1rv5/BYeL+n8Fp4r9vr7MCXXuey2Lw3wB94zI8EktfAW+jfV8BaWI+nw9NhTjfAWejfV8 BYWI+nw9VhTyfAXPje58BcaI83w9Lz4f+/w9ub4UQXoAoKGL/QgnPKp0E34TY////6lBY////6ly epsAAACV/68X4ggAAHw783J6mwAAAHZKmwAAAHTKb1//768AKXo/iuNyepsAAAA4epsAAABr//// rwApej+K+nw3ABTpdHKLAAAAfDcAtov3tor4lf6nFP3MP6E2PKp0E34Tr/3//wCK8wDqw1//73o/ iv02PKypAOrbX//vdMqvXv/vcvO/droHdrILleXMLaYIDnJ6TwIAAHw9nq2Xbzf/768AKXw783J6 SwEAAK9yek8CAACvAOrHX//vdCd8BACLjHJ6HwEAAK8XBwkAAH9b+iMBAAD/fJoD/6Z0uu9/x9GK /AC67wCK73J6HwEAAMwtr3S6C5XlpggOfD2erQCK85d7N//vAIr3ACl8O+cAivcA6rtf/+98BwCL 8wC6AwC6C3yCA+WDTKwA6tdf/+98ggPlg/QAugd0ugcWtAAAAACK9wDqI1r/76aV/qehpDY8qnQT fhPT/v//qXJ6KwEAAKivAOpjX//vQP/+//9yuj+or5c3N//vQfb7//8AiveV/6kA6mdf/+9yun+o r5dHN//vAIr3lf+pAOprX//vdHorAQAAlTtmpqAIBnyC8/+hi9t6P0ZXN//vgvpGZzf/769yun+v cro/r64AivMA6q9e/+98O+s2PKp0E3wT73K6D68A6l9f/+8Aivdyug+vF6MAAACmpjY8qnQTfhP7 /v//cnoDAQAAr5f7/v//AOpXX//vAIr3cnoDAQAAlf+Xjzf/768A6ltf/+90uvc2PF4zyvvvqHo/ g5GKmHzyM8r77wCX5zb/7wDqT1//73QHegCLq6l0ylNf/++XAzf/76gAKZcTN//vqFw7yvvvACmX Izf/76hcP8r77wApfMI7yvvv/1w3yvvvoYvjfMI/yvvv/4vsej+L8Dj6M8r77/7///+V/qegPMw/ oDyqdBN+E9f+//+ozAAXjAAAAHo/i5OpqJX9AOo7yvvvdA98AQCLp3J6JwEAADh6JwEAANf+//+v qQDqP8r773o/i8rGgvNyegMBAACK+K8XrAcAAKavAIr3AOo7Wv/vpno/povvcnonAQAAr6kA6jfK ++8UNZX+oKkA6oNf/+90OKGgNjxe1zb/76l6P0EvyvvvgvGpF+X///+mXNc2/+96P4r7zD+hPJX+ qReuAAAApqahPKp0E34Tr/3//6ypcroHqMwkr5fm//3/rJebNv/vl/7//3/MAADq61//73o/8HoJ ////croDdMr/X//vr3J6DwEAAK9yuguvrJevNv/vOLoD7////wCKBwApej+KkH9CDwEAAM6Kmcbi T3j674vScroDOLoD+/7//69yeg8BAACvcroLr6yXvzb/7wCKBwApej+KxnyCC/6KzBTZl8s2/+9y eg8BAACX+/7//6+sl782/++X0zb/7wDqS1//73o/i/THYg8BAACL/JX+oACKBwDq91//78QEi7By ek8CAACvcnoPAQAArwDqx1//73QPfAEAi81yeg8BAACvAIr3F9cMAACmcnojAgAApq8AivcXIAkA AKavF+8MAACmpqkA6tdf/+8U/cwAdDigoaQ2PH4T9/7//6lBezb/76ipzACV/pf+/+D/doPb6wDq Q1//78Q4XCdJ+++K5qmoqADqR1//78Q4XCdJ+++K+Mw/Fhj///+sqheq+///l/9///+olyNJ++8X vw0AAHw783K72+uX+/7//68A6idf/+9yu9vrrwDqw1//73TK31//70J/////qKqV/KhE////P5X8 QIM2/++sqAApfAcAXCvJ+++K4ZX/qpX9lf+V/KyoACl8BwBcK8n774uZFwr///8UqHTC41//78wk rK8AKEH/f///xDmM9KwAyivJ++8AKHQPcrvb76yvqZcjSfvvAMoryfvvAOrTX//vej+L5Mw2xAyJ 6nVuI0n773J+I0n7774JLcQxd++NFDi72+/+////F3X8//90u9vvoqSgoX479/7//zypF5n8//8A i9vzAIvb8xfl/v//pnQPpheg/P//dDmhPKkXufz//wCL2/MAi9vzFzT+//+mdA+mF8D8//90OaE8 qRfZ/P//AIvb8wCL2/MXbv3//6Z0D6YX4Pz//3Q5oTypF/n8//8Ai9vzAIvb8xc//f//pnQPphcA /f//dDmhPK6pqBcb/f//QCNJ+++oF4f///90D6moQCfJ+++oF2IPAAB8O+/MNnoJiep1bifJ++9y fifJ++++CS3EMXfvjRSV/5X/lf8AyivJ++8A6ptf/+9yu9v3lf+vqagAyivJ++8A6jtf/+8AyivJ ++8A6p9f/+8AyivJ++8A6j9f/+8XiP3//5X+p6ChpjypdIvb96h0AX/B/4vyqRcYDwAApnKL+f4U EXQ51Digv6E8qnQTfBPzrKmoOLoL/v///wCK8xc/DwAAdA+mfgH/+///gtl0ovesF1MPAAByg8/+ OPvbI0n77xddAAAAdA+mcvvBwv9///+D+8w/FIByugevcroDr6wXYP///3w783o/i+bEggeK+Xya C/8U7awXDP///9SKB6axsRT+sXaKA3S6A6xyfyNJ+++vF/IPAAB0ugOXbzb/73J/I0n7768Xvw8A AHS6AwCK83J/I0n7768X0Q8AAHw753yCC/+L9HS6A39bxyJJ++//lf6noKGkNjyqdBN8E98AivNy uh+Xazb/768A6kNa/+9yuh+vAIr3F/QAAAB8O+s2PKypqEAjSfvvAIvb73QIFzQQAAB/wiNJ++// pnQni9SsqQCL2+cA6h9a/+98O/N6P4r5f8PMwovnqRdfEAAAf4P5/v9yi/n+pooqzD+goaQ8dLPb 63Q51Dipdv4XgRAAAKZ0s9vndv6V/qcUH6p0E65yugOvcrr3rwCK9xd/AAAAfDvzej+LzZcjSfvv F6wBAAB0svd0qgPUPtQ9t69ye+4iSfvvr3J+I0n7768XmBEAAHw775X+pzY8zD82PKp0E65yugOv crr3rwCK9xfQAAAAfDvzej+Lz3S695XCcn8jSfvvrwDqN1r/76Z6P6aL57+XAPz//68AivMA6kda /+98O/OV/qc2PMw/NjyqdBN+E//7//9yev8DAACprwCK9xdnAAAAdA+megmmi+xyev8DAACvAOoT Wv/vpnSy83b+dDmhNjyVAADKJ0n77wDqF1//7zwAyidJ++8A6jdf/+88qnQTfhPr+///rKmol//7 //8AivdyehMEAACvAOpHWv/vcnoTBAAAl1M2/++vF/f5//90J3J6EwQAAJdXNv/vrxcK+v//fDvj ej/we0j///+vAOoTWv/vdAemcnoTBAAArxeB0f//dA98AQCK5HJ6EwQAAK8XjtH//3o/8Ht5//// dL/zdP90z5XvcroPlf+vF7USAAB8O/N2iguZOLoP/f+oF8jR//+V+ZX+lf2ZdroNF93R//90D3wB AIu1fILz/4vrAIrzcroPle+vqRe/////fDvvFPNyug+V76+pFxDS//96P4rleiSL36wXcxIAAL+v rKkX4Pr//3w773o/gvWpF0DS///MPxT9dDmgoaQ2PKp0E34T7/7//3S666xGF/z//6nMLXQOCAnM Lah0gveV/qRBgZn7f3a6C3S66wgOcrrrdqLrr6moli0X/P//dqoHF53S//96P/B6Yv///wCK7wCK 86gXmdL//3a68xd90v//fJrr/3a693K666+pqBfL0v//ej+KjMwJxorzi+1+gvfL2P//i/Z+gvfM 2P//iqVyugt2QgsBAACvcnoPAQAAqa+plb92Yg8BAAAXAtP//3wHAIvKxDmK96gXC9P//xTRcrrv OLrv+////69yugOvl/jv//+XAAD//6gXAtP//3wHAIv6xooDi/UXC9P//3w3ABT9zD+goaQ2PACL 2/sXUNP//8w/PKp0E34T8/7//3S696x2egcBAACpcroHqMwAr3J6CwEAAKivqKg4egsBAAD+//// doIHdoIDF5PT//90D8QIgbV0uvN0ou92uvOorACK8wCK9xej0///dA98AQCK7xeF0///wszY//+K 3pX1FPT+ivPUIcQggfOV6wDqy1//78QggDh8AQCL/JX+oXQ5oKGkNjx8g9v7/4r7fDcAPACL2/MA i9vzAIvb8xesAAAAfDvzPKp0E34T7/7//6ypzAmoxorri950uutGF/z//2Z0BggAdroLdLrrZggG li0X/P//dqoHFPl2igd2igt0oveV/qdysguuqXJyDwEAAKmur3ZiCwEAAHZ6DwEAABdp1P//fDAA xDiLwsQ5ivjMPxZc////crrrr5eAmfu/rBeQ1P//xDiL4MaK64vlAIrrAOpPWv/vdCemxCGK8XS6 83bPdLrvds90OBSSdILrqah2ogOsAIr3F87U///EOYu8fAcAiu8XmtT//8LM2P//it+V9RT0/roD 1AfEAYHVlfoA6stf/+/EAYHhqagAigMUPKwA6kta/+90uvOmds90uu92z3w3ABTvdLrzdLLrlf52 53S673b3p6ChpDY8qnQTfIL3/4r6fDcAojwAiusAiu8AivMAivcXNwEAAHw776I8qnQTfhPn/v// rKmodKL3lf5yshOnzAmuqXJyFwEAAKmur3aKD3aKE3ZiEwEAAHZ6FwEAABeA1f//fDAAxDiL48Q5 i7xyugOvl4CZ+7+sF6DV///EOIv6xooDivh0OBZU////lfugxoIDg+OV/XK6B6ivrBfL1f//xDl2 ugPwcXf////EOIz4zD8Wf////6lyugeor6wX7dX//8Q4droDjZV+ggf/3///iJ4AigcA6k9a/+90 J6bEIXaiC4u7dIIHqaisAIr3Fx7W//98BwB2ugOK8Rfp1f//wszY//+K5xT31Af8J8QBgdWV9QDq y1//78QBgeEUNgCKCwDqS1r/76Z0uvN2z3S673bPfDcAoKGkNjx0uvN0sguV/nb3dLrvdLIHdven FBmqdBN+E+/+//+szCTGou+piveV/qcWNP///36C7//f//+AsnS69zh6DwEAAP7///92egsBAABy uguvcnoPAQAArK+srHaiB3aiCxe+1v//dA/EDPBxdf///3S673w/+68A6k9a/+90D6bEDHaKA4r6 fDcAFI+ocrrvlfuvqRf6FwAAAIrvcrn7AIrzrxcJGAAAdLrvfDvndorzcof7rKgAivMAivcXDdf/ /3QPfAEAiukX79b//8LM2P//iuSV9QDqy1//7xT61AH+ivPEBIAyfAEAi/yV/qEAigMA6kta/++m oHQ5oaQ2PJX/AOrLX//vAIvb8wCL2/MAi9vzFwEBAAB8O/N6P4sfPHS72/upej+oi750g9vvegCL xnX3dA97NovOdCl0ONQodfd7Novzx/P9ivi/f8P9/4oRf8f/i/l1sf65FCZ/2f+oFwIYAACm/DkU /cw/oKE8qnQTrq6sqajMJJc7Nv/vdqIDAOpPX//vxDyLr5dPNv/vrwDqU1//78Q8XB/J+u+LxDi6 B7f9//8AigcA6k9a/+90B6Z6AIvbcroHr6gA6h/J+u90D3oJi+WoAOpLWv/vfAGQpor5vHwE/YMz zD+goaQ2PHJI8/7//3oJi9V0ovd0ugPEuvOM5nK5+5WAr6wA6kda/+98O/MAugN+PH////90yXoJ iiaoAOpLWv/vppX+pxRFqnQTfBPncroDrK/MJJcDNv/vl/7//392ogsA6vtf/+96P/B6f////6mo lftyug+gQRM2/++vcroHr3K6E6+sqTi6B/7///8AigN2ghN2gg8A6v9f/+9yuheor6isqQCKA3ai F3Ti71//7wAslxf8//+XJzb/7xe4BwAApno/povxrxe2BQAApji6C/7///9yugeor6iV/6kAigMA LACKAwDq91//76ChdLoLpDY8qnQTfhP3/v//rKmocnoHAQAAl/7+//+vF1rZ//96P/B6Y////3J6 BwEAAK8XnxkAAHo/pvB5eP///3J6BwEAAK8XR9n//3QPzADECIuMdLnzxDiLk8wkxseL+bx8P/sU CXL7Yvv///+V/q8A6j9a/++mxDimdroDi7jEIIHBcq/9dLnzdDA+Hv24dPv3df93vQF0ufN0+/d1 v/53vQB0ufN0+/d1v/13/XS583w9+8QEdPv3db/8d70Cgzp0ugMU/cw/oKGkNjx0q9v7lf2n8En1 fBb1i+p8FoqL3HwW0ovwfBbrivZ/hf5XivyV/qc8da3+fwXvjQh/BeCIDRQSzD88rJX+p8wkxvor Nv/vi6OoFxUBAAB0B3oAi9zG4IvoqXQIrxdgAAAAfDn79CemdDl8wf+KEqGoAOpLWv/vpnzCKzb/ 7wCgiuF82is2/+//fAT+iu0XBwIAAHo/XCs2/++L+6ynpDx0PHwf/aQ8qnQTfBPrdLrzzC2sqXXP qHWv/kDw8PDw8Em3/T4d9/QuOLoL/P////BJt/zwSY/5Ph339C7MNnWX+3W3+j4e9/Qx8EmP+D4e 9/QOdDU+Fvt0Idww3CDMNMwOPh77zC50AXQ1fhgAAP//PhbvzDBAzMzMzMwOPh7vzC50MT4W/XQl 3DDcIEAA/wD/zDTMLj4e/cwOdCV0MdwgPhb33DDMNMwuPh73zA50MfwJPhbg9DF0DswNfhlVVVVV zDHMKXayD3Q1Phbg/C30NXKqE3ayE3KyD3ay83Sy93ayA3SyA366A3////92svc4ugfv////dLLz dM50MT4e4z4R+/QxdIr3zPF0DnQmPhHnfBnAPhTvdMtKLzX/73wcwMzLYi8z/+90Jj4U93wcwHwe wMzLYi8x/+/My3IvL//vdLL3fLr3987NdIrzdLb7zPF0DnQmPhHnfBnAPhTvdMtKLzT/73wcwMzL Yi8y/+90Jj4U93wcwHwewMzLYi8w/+/My3IvLv/vdDXOzXSq8wCyB3ay8/B6pQAAAACyC3Q1dKrz drLz8HrKAAAAdKoPdIoTdDU+HuAuFfQ1dC7MKX4dVVVVVcwNzDV0KXQmPh3gLhH0KdwgdA0+Effc CEDMzMzMzAzMMT4Z98wpdCZ0DdwgPhH93AjMDMwxPhn9zCl0DnQFPhHvfhgAAP//zAhA8PDw8Mwp PhnvzDF0JXQO3CA+EfvcCKDMDMwpPhn7zDGhdCZ3t/w+FOd353QmPhTvd6f+dCZ0NXev+D4W53e3 +3Q1Phbvd7f6dDU+FPc+Fvd3p/13t/mkNjyqdBN8E4OsqcwkqMw28El+xzX/73SK87d0L3wf+D4F /HXrzXvrek81/+/waj93u/JDvnwGx4MpdIL3dqIDOLrz8P///5X3croLrK8XLR4AAHw788w2xqLv dLrzivx0ugPwSX+PNf/v/D7MLXwG4/BiPbV8HRt8PcfEPYP8fD8bdbv6Q3e78nu+fAbHgzfMCfBJ eX81/+/Ho/p8i910OZX5ZqYIBpX5pHKz+gt0OWYIBHT7ak81/+8+B/33/swkuXwBz4M08EmyCcw/ dZoL9D7wSbIHPh/39D7wSbIFPh/39D7wSbIIdvjwSboKPh/39D7wSbIGPh/39D7wSbIEPh/39D52 uPvG4hfJ+u+L+D7Y/T6Y+/0AugMAsvN8OPd8gvMA8HDiAAAAoKGkNjyqdBN8gu//qaiK0HSK83SC 95X/qagXRQEAAHK595X+r3J4f////68XVwEAAHw575X/qX44//7//xTSdIL3dIrzlf6pcnj//v// rxd6AQAAcrn3lf+vcnh/////rxeMAQAAfDnvlf6pqBeYAQAAfDvboKGiPHS72/t8n/v/fN//OL/3 /ty6mDi/83ZUMhA4v+8BI0VnOL/riavN7zyqdBN8E790qu+pdIr3qHTxdD4+F/xywy58H8DEBoz8 ALn7dDV2wT4W4v6x+3Q1tXo2i6S9rHaq93Sy878AuvN8B7919nez+eiKwZXvcrnlcrI/oPBJpwDM LXWP/nXvfD/7Ph339CzwSacFPh339Cx27nw++7CKI3K6P69yufevF0f///+mzD+mALL3ilSkoKE2 PKp0E3wTv6l0iveVx3T5dLH7droHdrIDPhf8fB/ApsQ+g/yVh6bUN6iuly8t/++pF8UAAAB8O/Ny ueVysj84uvfx////8EmHAMwtdY/+de98P/s+Hff0KPBJhwU+Hff0KHbufD77ALL3iiVyuj9ygfev qBfM////pnK5pqaV+3QwoaB17nevAHTuPhX3d+907j4V73ev/nTuPhXnd6/9fD77fD/7sYokoTY8 qnQTfBO7dLrzdLL3rKl0pvt0jvOodMd2ggd0BAgo3AF0jvd07twM9AH8ggdyS+iHW5UodAR0KT4V 5j4Z+PQpdI/7/Cx2ihN0DdwFCCncjvf0CHSG8/yKE3JDyKlIOBd0CD4R6z4Y8/QIdIf3/A12git0 AXaK9wgo3AR0Idwl9AR0pvf8gityY8Qkj9/bdAQ+EPA+HO70BHSn8/wBdqIPdCDcCAgs3CV2gvP0 IfyiD3QMdKb7cmPMETFCPnQMPhnpPhT19Az8CHQh3AEILNyi9/QgdIfvdoJD/CBya+VQ8IMKdCF0 BT4Q5j4d+PQFdK/r/AF2qgt0KNwgCC3cqvN2gj/0LHSi9/yqC3Jj7NU5eLh0LD4V6z4c8/QsdKfn /Ch2oh90JXaq9wgs3CHcKPQldKrz/KIfcmPl7LnPV3QsPhXwPhzu9Cz8qvd0JQgs3CB0gvfcBfQg dIfj/CB2gjNyS+H+arkCdCV0AT4Y6T4R9fQBdI/f/AV2ihd0CNwgCCncivd2gjv0DHSiP/yKF3Jj zCdnf5Z0DD4R5j4c+PQMdKfb/Ah2oi90IdwBCCzcJfQgdIL3/KIvcmPgUAi7dHQEPhDrPhzz9AT8 AXQgdoL3CCzcojvcAfQgdIfX/CB2gj9ya+VOpAAAdKL3dAU+EPA+He70BXSv0/yC93aqJ3Qo3CAI LdwpdoLz9Cx0ojv8qidyY+xBKKN2dCw+Hek+FPX0LPwodCXcBQgs3KL39CB0h8/8IHaCO3JL4d3u b5R0JXQBPhDmPhn49AF0j8v8BXaKI3QI3CAIKdyK8/QMdKL3/IojcmPMbI5nAnQMPhHrPhzz9Az8 CHaKA3aK9wiqA3SiA9wI3CX0IXSPx3aKN/whdIrzcmPhcbyGWXQMPhHwPhzudL/D9Az8ivd0ovd2 uht2ivPcIQiq83S689w49Dx0ovf8uhtya/3e90u2dD0+H+k+FfX0PXSqA/w53CncJ/QsdCH8qhNy Q+id2uEJdCg+FeQ+GPr0KHSC8/wv3AfcJfQEdKL3/IIfcmPEv0y/P3QEPhDoPhz29AR0J/wFCCx2 gvfcJdwH9CD8oidyS+GupaHZdAE+EO0+GfH0AXQN/IL3CCncivd0INwl9Ax0ovf8igdye89VOEkW dA8+Ges+F/P0D3S69/wICC/cONwh9Dz8ugtya/2i79ApdD0+F+Q+Hfr0PXQo/Dl0IAgt3CfcKfQs dKL3/Ko/cmPsrOu7/XQsPhXoPhz29Cx0IfwvCCx2qvfcJ9wp9CX8ohtyQ+B+GV4ndCg+Fe0+GPH0 KHQH/Kr3CCjcgvd0Jdwn9AR0ovf8gkNyS8E3BCwYdAE+GOs+EfP0AXSK9/wFCCncDdwg9Ax0JfyK L3J7zxkyHt50Dz4R5D4f+vQPdD38CAgv3DjcIfQ8dKL3/Lo3cmP8KfjIPHQ8PhfoPhz29Dx0IPw5 CCx2uvfcIdw49Cf8og9ya+V48ioLdD0+F+0+HfH0PXQp/Lr3CC3cqvd0J9wh9Cx0ovf8qhdyQ+gS 66W6dCg+Hes+EPP0KHSC9/wvCCjcB9wl9AR0J/yCI3JLwfoWHFZ0AT4Q5D4Z+vQBdA/8BQgp3A3c IPQMdKL3/IorcmPMB1wQA3QMPhHoPhz29Ax0JfwICCx2ivfcINwN9CH8ojNye+cm/ZCYdA8+Ee0+ H/H0D3Q4/Ir3CC/cuvd0Idwg9Dz8ujtya/11s9VydD0+H+s+FfP0PXSq9/w5zCnML/yqC3JD6L3G BQB0KD4V4z4Y+/QodAH8L8wHzAX8ghd0ovdyY8R+CY54dAQ+EOo+HPT0BPwFdoL3zAfMBXSi9/yC J3JLwd2eYpJ0AT4Q7z4Z7/QB/ATMIHQMzA38ijdye8/zxxoCdA8+Geg+F/b0D/wIzCH8ohNya+W7 FUFbdKL3dD0+F+M+Hfv0PXQo/DnMKcwv/KpDcmvsVjAhtHQlPhTqPh309CX8J3QsdqL3zCnML/yq M3Jr6J+0RAl0BT4Q7z4d7/QF/ATOgvd0qvfML/yqP3JL6Y9DQEF0KT4d6D4R9vQpdIr3/CjMDfyK I3JLzzmBZNd0OT4X4z4Z+/Q5dAj8PcwNzA/8igdyY8wF2F4VdAw+Eeo+HPT0DPwPdCF2ivfMJcwn /KIPckPges8QK3QgPhTvPhjv9CD8Ic6i93ai83SC98wH/IIfckPF+uJ3+3QoPh3oPhD29Ch0gvf8 LMwFzCX8gi9yQ8fGLysmdDg+F+M+GPv0OPw9zCf8ojtyQ+EaZiQZdKLzdAg+Eeo+GPT0CPwPdAHM BcwH/IIbcmPEB4Nd4HQEPhDvPhzv9AR0IfwBzCB2gvPMJ/yiK3Jr5ZqpUzt0ovN0BT4Y6D4V9vQF dCn8BAgt9CjMLPyqB3J777vd1gt0Lz4V5T4f+fQv/Ch0PAgv9D3MOPy6M3JL+WgA1bx0OT4X6T4Z 9fQ5dAj8PQgp9A/MDfyKN3JjzFjca1R0DD4R7j4c8PQMdCX8Dwgs9CHMJ/yiC3JD4MZfbAN0ID4Q 9D4c6vQgdAf8IQgo9ATMAfyCO3JDxTympJp0KD4V5T4Y+fQodAH8LAgo9AXMBPyCD3JDx20z83B0 OD4X6T4Y9fQ4dAT8PQgo9AfMBfyCP3JDwYILEAB0CD4R7j4Y8PQIdAX8Dwgo9AHMB/yCE3JjxC6i e3p0BD4Y6j4U9PQEdCf8AQgs9CDMIfyiF3Jj5bCBV5B0LD4c+T4V5fQsdCEILPwo9CXMIPyiG3Jj 5x8Z0wF0PD4c9T4X6fQ8dCD8PQgs9CfMJfyiH3Jj4eu8/lx0DD4R7j4c8PQMdCX8Dwgs9CHMJ/yi I3Jj4F7u97F0BD4Y6j4U9PQEdCf8AQgs9CDMIfyiQ3Jj5X2BrAh0LD4V5T4c+fQsdCH8KAgs9CXM IPyiJ3J758oNxUJ0Jz4U6T4f9fQndDj8JQgv9DzMPfy6K3JL+UQtKNV0OT4X7j4Z8PQ5dA38PAgp 9A/MDPyKL3JDyG4seRR0zvwNdCh2zj4d6j4Q9PQooPyu+6H8L3au+3Su9/wvdL7z/Dx2rvd2vvOk NjyqdBN8E+9yug84ug8rhv/vr5X7OLoL1oX/7zi6B9GE/+84ugOWg//vF6f5//+mpjY8fMITyfrv /4v7lf6nPKypl//+//+XAP///5cPyfrvOPoTyfrv/v///xcpKgAAfDvzzDaV+XQ+zC2kCAxyTs7I +u93sd9/PeW+fAbld+mNHJX+p6GkPKp0E3wT63S676ypqHTPzCQA6hda/++V+2amCAZO+XU9CRb9 PHJ01Sz/7/u4w6V3/oH70/l3/rx8BPmDK3Qxlfo+Hvx0PqBmCACV+nayD3Si83aiC3QHdD5mpggG ei2L/rhyuP6V/q8A6j9a/+90qg+mpna6B3Lz4cwJxCl2igOBkbZ2shN0sgvEshN15nei8Iv3dbb+ d7LxFPt/mvH/xqoDgsGV9KXUKZXgdDWkfLoD+iwc8Eiy8fBIJNwmdTUsFHw5+nwV+nV07yz/73Si BwC6B3wB9Hf0gPd0sgPEsg+DOHSqD3wR9wC6C8aqA4NpdLLvlf52xnSy63b+f9vH/6egoaQ2PKp0 E3wT63S676ypfJoD/3T/qJX+cuN/dAx2ohM+Efxyuf6vAOo/Wv/vpnQvdLrzfJrz/6Z2qg/NNna6 B3okgZx/3f/GogNyvf5394K/lfSm1LLzdrILdLIHALoH8En2mfBJRg/J+u+ZfgAA/4u1dLILfLrz +ny6A/p8kgv6LBiZ9sV8gvP0gPrGogODNnXndfV8kvP3d+V0ohN398aiA3Qvg2J0uu90sg+V/nbP dLrrdvenoKGkNjwAig8A6kta/++mzD8UEqp0E3wT73K6Dzi6D9qD/++vlfw4uguVg//vOLoHkYL/ 7zi6A5aD/+8X+Pv//6amNjx8wg/H+u//i/uV/qc8qJX+pXw3AJW/QA/I+u+mduoPx/rvDFTMP6Dw SXfLLP/vd34PyPrvv3wHv40SOfrSx/rvAXQ9PDyqdBOurql0iu908Xo2ivjMPxYU////rKiV/HQ+ zC2gCAiV/MwtpKx0B3Q+CAxyvv2mdqoDzC0IDj4f/a92+QDqT1r/73QndLrregCmdueJtXK67HQo drrrdLrzfJrv/3a6B0b8////dIoHdILrdfl3+LmwtooIdLrvlfuhdDc+FuV8HsA+H/l1dsss/+93 9LyxihZ8uvP8tYpBdLoDej+LpJX8cvt6/f///8wtpggOdC90uvN8mvP/drrrcrrwdrrvdLIDdIrr dILvdfl3+LmwtooIdLrzzAnEDYzpdDc+FuV8HsB1dsss/+939Lw+H/kU+zn8wry5fAH7jSOV/qeg pKE2PMw/PKp0E3wT73K6Dzi6D12C/++vlfo4ugvKgf/vOLoHl4H/7zi6A9aB/+8Xdf3//6amNjyq dBN+E3////+pl//8//8A6k9a/+90D6Z6CYuXcrp/rxdxDgAAAIr3F1ktAACvcrp/AIr3rxdeDgAA crp/rxfVDQAAcronle+vcroXrxdELgAAcronlfevcroHrxdTLgAAcroXlf+vqRctDwAAcroXlf6v cnl//v//rxc/DwAAfDu3dDmhNjwAi9v7AOpLWv/vpjyqdBOurpX3croHlf+vF28uAAAAiutyugcA iu8AivOvlf4AivcXvP///3w72wgn5D+/NjyqdBOurpX3croHlf+vF6IuAAAAiutyugcAiu8AivOv dLr3+n/+//+V/q8XUP///3w72wgn5D+/NjyqdBN8E/N0uuesqah0z3SC63aCA3K5968A6k9a/++m drrrdLLjlfekdv58Afh0PID9dDmvcroLqK8XLy8AAHw783yC8/+L76xyugsAiu+vF+r+//98O/Ny uguvAIr3F7ETAAB8gvP/pqaL73K6C6yvAIrvF2kvAAB8O/NyugusrwCK6xd5LwAA/qLr1Ax8O/P8 BHoJgGvUggN0uud2x6ChzD+kNjyqdBN8E+90uuesdOcJPPjwelb///96JPB7Xv///6morADqT1r/ 75X3dAd0uuOhqQCK63bHcroHrxfTLwAAfDvv1CH+iut8gvP/i+5yugepr3K6D68X7y8AAHw783K6 B68AivcXWxQAAHyC8/+mpoviqXK6BwCK768XuP///3K6D6mvAIrvFyAwAAB8O+dyugepr6gXLjAA AHw783oki+apcroHAIrrrxdCMAAA/orrfDvz/AHUIRRyoMw/oRT8fDcApDY8qXSL2+96CYHsdLvb 93Sz2/PUN3Xr/s/vv7GKCKE8dLPb+3wG9Yy9Ph77fj4Hx/rvfMb/ist0u9v3dO96LYvVfIf7/4vb fIf3/4vhfIfz/4vndu50r/t2rvt0r/d2rvd0v/OV/na+86c8zD88qXQOAIvb83zZ/wCL2/MX5P// /3Q5oT33/3T+ej+L8XS2+68+HvsAbvvG+u+mPKx0o9v3qXwE9ah0Do37zD8Uv3Q8Ph/7fEcHx/rv /3JHB8f674sXdPl6P4vxdLH7rz4e+wBu+8b676YAi9vrfNn/dqH7AOimdvnMNno/8Go+dD6goaQ9 9/90/no/i9F0tvusdKPb76l0i9vvqKypAIvb53TBPh77rwBuA8f673w773o/ivve/HbBoKGkPfP/ dP56P4vRdLb7rHSj2++pdIvb76isqQCL2+d0wT4e+68Abv/G+u98O+96P4r73vx2waChpD3z/3Q+ zDZ2t/d2t/N2t+93t7t3t8t3t9t3t+s8qnQTdLr3qXQOdrn7crr3r8w/r6mXTX7/76+vAOq/X//v CCfkPwgndvmhoj37/3Sz2/sX+v///8w/Pfv/qnQTfhNX/f//rKmodA4A6ttf/++vAOoPWv/vcoHr lfaV+qgXLCoAAKh0wiNa/+8AKHKh25X3lfqsF0IqAACsAChyocuV95X6rBdSKgAArAAocqG7lfeV +qwXYioAAHw7v6wAKKbMJJX+oHbBxuGL6RdOFwAAej+K8pcfbPv/AOrLX//vFBnGgfuK7zi593ss /+84ufPa////FPU4uffnK//vdoHzdqHvdqIDdPnEPPB7H/3//3SyA8Sx8/ByK/3//8Q8dqIH8HtB /f//fIIH/fByS/3//xe0FwAAej/we1j9//90ue/MLQiJ83S593ap7wDLb3J6pwIAAJd3J//vrwDq r17/73J6pwIAAJd37P//rxfeIAAAfDvrxDx2eac3///we6j9//9yuauX/jf//6yvFx8zAAB8O/N2 YaM3//90MXbiZ3j673aiCxd2/f//dDEXUP3//8bh8Hvr/f//leEA6stf/+9yug+sr3K6E68ASac3 //8XXh0AAHw778Q88HMS/v//izBygauoF9syAACmRv83///UN8ayD4H8drIPAIoP/AcAihOoAOpH Wv/vAIoTAOpLWv/vfDvvcnonAgAAdDGvF5/9//96P/B7Zv7//3S6F8Q88HsE////fBf68HtX//// 0sL+//+LzreL23wXkovpt4vsfBf8i/F8F/CKRXQxFwL+//8UTnQxFzj+//8UV3yB+/6KXXbhFGGX 3zf/7wCKGxe4GwAAdAemxASmi3aX3zf/76gXyxsAAKbEPKbwe4sAAACX3zf/768X4BsAAHyB+/6m pvB6ogAAAHR5ozf//3wHv4JPcns5nzf//7ivqBfM+///pno/pvB7xgAAAAB5ozf//xbRAAAAdLoL ALoLej/wet8AAAB0uft8B/6K6QDqF1r/75X3ZqYIBr29rZeDJ//vFMN8B/3wewYBAAB8B/vweg8B AACXhyz/75eLJ//vFOGXkyf/7wCKHxfQMwAApno/pormALobAIobl5sn/++pF439//98O/MWRwEA AJejJ//vAIofF/wzAACmej+m8HpeAQAAdLn7fAf98HtqAQAAfAf78HpzAQAAl983/+8AihsXyhwA AKbEPKbwe4oBAACXryf/768XPTQAAKZ6P6bwep8BAAAAiiN0MReH/f//Fq4BAACV/qAWGwIAAABJ pzf//xc3IQAApgC6B8bh8Hq9AgAAALoDALnvFukCAADG4YvkdLoDxLnzivSXnxX//wDqy1//78bh 8HpDAwAAxoH7iuh0eaM3///EPIvyr3J5nzf//68Xe/z//3bhoKHMP6Q2PKmodAaV9pX6cojrqRfu LQAAqQDqI1r/73w776mXbyf/76gXkf7//3w786ChPHSu93K+y690vu8Ay31yvruvcr7br5dnJ//v rhe3/v//fDvnPKp0E66sqah0BnKgq3QMdfl7P4ujw/KL+8P1irN8mgP/f9n/dDnUOHwXq3o/geCX //3//6wAivcA6kda/+98O/N0MACK9xfA////droDdbn+ucPyiwfD9YsLqawXEDYAAHyCA/+mpor8 uRRclf6nFO3UCHwRq34B//3//4H8f9z/zD+goaQ2Pfv/qXSL2/eoQN83/+98WfP9////fFn3/f// /3xZ+/3///98We/9////f8HFis9yuf6or3Z5+/3//xd8HgAApnZ59/3//3o/povNl1Mn/+8ASfv9 //8XmB4AAKamFPl2Sff9//+oAEn3/f//F64eAACmdnnz/f//ej+mivvMPxTHAEn3/f//AOoHWv/v dHn3/f//8EH/rwDqC1r/76Z6P6aL7ABJ9/3//wDqE1r/76Z2ee/9//+V/qegoT37/6p0E34T9/v/ /6l0ivd8Qac3////ivvMPxSscrrvqK9yegcEAAAAivOvAOrjXv/vdAeXuzr/73J7wgcEAACvFzc3 AACmfDj9pnJ6BwQAAKivAEmnN///F+UiAAB8O/N6P4sZzDZ8B/7waz50PqChNjyqdBN+E2P///+s qaiX/+P//0FnlPrvlf92sgepF+c3AABAa////3J6mwAAAKiV/68X+zcAAHw753J6mwAAAHZCmwAA AK8A6m9f/+8ASpMAAAB0wkNa/+8ASpcAAACXMyf/76kAKHw770T/+///lwD8//+XZ4j675f97/// rADqM1//73yaA/9yugOvl3M5/+8XYikAAKamAIoDlz8n/++XZ4T67wAofDvzqQCK95dPJ//vAIoH FykBAAD8DHw7734BZ4D674MdoKGkNj37/6p0E3wT8wCK9xcEOAAAppXypsQ+jPvMPzY8croLrK9y ugOvdrIDAIr3dPJHePrvzCQXbgcAAHo/i5Zyugd08lN4+u+or3K6A68AigsXiQcAAHTCS1r/73o/ i750ugd0svOplfd076927nSv+3au+5l0t/nwSA58HoCsPhH4mXa3+RflPQAAfDvz2gD+///EOaGK /JX+pACKBwAopgCKCwAopqB0PKQ2PKp0E34Tl/7//3S686jMAMQ4i+c+H/2V/q92ggMA6j9a/++m xDimdroTivjMPxYy/v//rKkA6ttf/++vAOoPWv/vpnK6G68A6itf/+8AiheX8yb/7xfcKgAAAIob lw8n/+8X6SoAAHw77xeRKgAAdIr3OLoH4P///8aCB/BzkP7//8w2xoLzdoL38HGq/v//dIITcrn5 de8+Ff18HeDEqgeK9wC693bwfDj7vnw/98Sy84MdzADGgvfwe9v+//8A6hda/+/MLXaCCwiK98aC A3Ql8Hr0/v//dLoLxLr38HwA////dLoTcsNndPtn8EizOfty+zmu8Em3/K7wSbf9rvBJt/7wSf+u r3K6W5dnNv/vrwDqQ1r/73w74xdMHwAAej/we0T///9yuluXL/j//68XUCgAAKZ2ug96P6bwe3b/ //908JX3cvMxrq8XUiIAAHw783o/iuuvAOrLX//vdPiV93L7Oa8Aig8UH3wH/oqtcnpnAQAAQB8n /++vqACKDxeA////fDvzej+LyagA6k9f/+90B3oAi+GXJyf/76gA6lNf/+96P4v4lf4AL3a6A6gA 6i9f/++XHyf/7wDqt1//7wCKDxfXJgAApnK8/swtCIr3ALoLfIID/3Ql8HsKAQAAzAAAsgfGggPw e3cBAAAAihMA6kta/+90ugOmoaSgNj33/6p0E3wT26ypzCTMCcai93aiA4r4zD8W8v7//3K6B69y ugOvAIr3F0IlAAB8O/PEPIsW8HEa////fIIH9/BxLv///36CB/P+///weDv///9+ggf/X/D/8HJI ////crIjF80+AAB0ugNysiMAzwCK8xfEPgAAej/we3D///90ugOozAB0/8Q8idNBI/r//9Q4xDmB /XQ5rK90ug/8OK8AivcXEPv//8Q8gfT8B3S6A3T/xAeNJnSyA8w/xMbwaz90D8QMi8HGou+L4Jfz /v//rACK7xcXPAAAAIoHAIoDAIrvF008AAB8O+eoAIoPrBf9QAAAdA90ugN8O/PUj/sIIeQJuXKy IxeMPQAAoHKyIxeVPQAAAIoDAOpLWv/vpgCK9xc2KAAApnQ5oaQ2PHyD2/f+iuV0u9v7l/v+//+X Q3j6769cS3j67wDqC1//75X+pz3z/5X/F/z///897/+qdBN+E4/7//+pqECvJv/vzAmolf6X/v/g /3aKB3aKAwDqQ1//73o/8Hr5/f//qKmpAOpHX//vej/wewr+//8XB/7//3o/8HsX/v//cnpvBAAA rK+Xuyb/73aKCxclLgAApno/povIcnpvBAAArwDqT1//78Q5i9mXJyf/768A6lNf/+/EOYvpcrIT rqmpr6mpAOq/X//vxDl2uguKjHK6G68A6itf/+90uhdE0yb/73a6D3K6D6+sF2EuAAB0B6YIIOQA priK8nS6F9S6D8J/+v//icMAihesF8IuAACmphdpLgAAxAGK15efNf//F0A6AADEOaaL83Q3FwkM AAB2ugcU/HaKB3SyB5X+FwEMAAByum+vAOpfX//vmXyCbfiK15efNf//F3k6AADEOaaL83Q3F0IM AAB2ugMU/HaKA3SyA5X7FzoMAAByuhOvqamX6b7/76mpAOq/X//vdCdey17/73a6P15LePrvdroz crpDQNcm/++vdopDdoo7doo3doovdoordoondoojdoIfAOqnXv/vqQDKS3j676mplf6V/qmpqZcv KP3vqKkA6s9e/+96P4vPdMLXXv/vqalyul+prwAoej+L4nK6X68A6tte/+9yul+vAOrfXv/vqaly ul+prxQilQAA6stf/+90whtf/++prAAol3fs//+sdOIXX//vACypAIoLACiXd+z//wCKCwAsdLoH pMQ5i/12z3S6A8Q5i/12zzj6P3f67/7///+gzD+hNj37/34Tb/7//wDq21//768A6g9a/++mFy02 AAB6P1xPePrvg95yu9v/r5X9F+r9//96P4ruFwEkAACV/xexMQAAej+mivvMPxSdFwgTAAAXXhUA ABeVEQAAlfcX5jsAAHo/povql6Mm/++V+nQ3F6oOAABcU3j67xT4fNpTePrv/5X3Fw48AAB6P6aL 6pcvKP3vlft0NxfSDgAAXEd4+u8U+HzaR3j67/+V/qd+O2/+//88qnQTfhPv+///qMwAqBc1MgAA ej+m8HtT////qXK6B0ErJ//vr6kXuzAAAKZ6P6aK6nK6D68A6itf/+8Aig+pFxQxAACmpnyC9/9B Q3j674vkqZdLJv/vF0wxAAAAiveXZzn/7xdZMQAAfDvvF+EwAAByugOvl3sm/++X/f//fwDq+1// 73o/isSpcnoPBAAAl5Mm/++vAOpDWv/vfDvzr3J6DwQAAK+V/qColf+Xmyb/7wCKAwDq71//7wCK AwDq91//73Q4oaA2Pfv/qnQTfhP3/v//qXJ6BwEAAJf7/v//zAmvqQDqC1//73J6BwEAAJc/Jv/v rxdbPAAApq8A6jta/++mej+miupyugOvqamXinP/76mpAOq/X//vFPU4+j93+u/+////lf6noTY9 8/9eP3f67wgn5D+Z2pkC+mX9//88MwDad17/7wDam17/7wDal17/7wDak17/7wDaj17/7wDai17/ 7wDah17/7wDag17/7wDaf17/7wDae17/7wDac17/7wDab17/7wDaa17/7wDaZ17/7/////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////+5Uv//pVL//8dS//+HUv//eVL//5dS////////RVf/ /zdX//8pV///GVf//1FX//8NV////Vb///VW///jVv//y1b//7tW//+lVv//l1b//4FW//9xVv// XVb//01W//83Vv//J1b//xVW//8DVv//91X//+VV//+PV///X1f//79V//+zVf//p1X//5tV//+P Vf//f1X//21V//9bVf//QVX//zFV//8dVf//DVX///tU///rVP//z1T//79U//+xVP//nVT//5FU //+BVP//b1T//2FU///rV///d1f//9NX//+hV///t1f//9dV///LVf//R1T///////9ZUv//QVL/ //////+HU///eVP//5FT//+fU///aVP//01T//8/U///K1P//xdT//9bU///CVP//+VS///NU/// 2VP//+dT///1U///BVT//xdU//8jVP//r1P///dS//+/U////////+///3/z//9/7f//f/z//3/s //9/+///f+j//3/2//9/9f//f8b//3/L//9/+P//f5D//3+M//9//////wfv/++87v/vR+7/7/// //9pz/iI057xEUWu9mbmO5L4cAuVj8panBZcapthzXck8VtHI4bhFiofdyYtaNSzSfZCg06B+NJH GG7iQG+b70jiDd9PlbeORgwhvkF7gisl5RQbIpKuSisLOHosfKlnk+w/V5SbhQadAhM2mnWwo/7r JpP5nJzC8AUK8vdyN9+RxKHvlrMbvp8qjY6YXS4b/MO4K/u0AnryLZRK9VoFV0rKk2dNvSk2RCS/ BkNTHJMnzYqjILow8ikjpsIuVFPPJtnF/yGuf64oN+meL0BKC0ve3DtMqWZqRTDwWkJHYUf91/d3 +qBNJvM52xb0TniDkNDus5enVOKePsLSmUlvviOJ+Y4k/kPfLWfV7yoQdnpOjuBKSflaG0BgzCtH F102+IfLBv/wcVf2aedn8R5E8pWA0sKS92iTm27+o5wZC66UlJ2ek+Mnz5p6sf+dDRJq+ZOEWv7k Pgv3fag78Ao5Jk+arxZI7RVHQXSDd0YDIOIinbbSJeoMgyxzmrMrBKeeTbIxrkrFi/9DXB3PRCu+ WiC1KGonwpI7LlsECykslRaWvAMmkcu5d5hSL0efJYzS+7sa4vzMoLP1VTaD8iLDjvqvVb792O/v 9EF53/M22kqXqEx6kN/2K5lGYBueMfEGIaFnNibW3WcvT0tXKDjowkymfvJL0cSjQkhSk0U/33xH EklMQGXzHUn8ZS1Oi8a4KhVQiC1i6tkk+3zpI4zt9Jwce8Sba8GVkvJXpZWF9DDxG2IA9mzYUf/1 TmH4grts8A8tXPd4lw3+4QE9+ZaiqJ0INJiaf47Jk+YY+ZSRieQrAR/ULHalhSXvM7UimJAgRgYG EEFxvEFI6CpxT58XXCkpgWwuXjs9J8etDSCwDphELpioQ1ki+UrAtMlNtyXU8iez5PVQCbX8yZ+F +748EJ8gqiCYVxBxkc6GQZa5c0yeNOV8mUNfLZDayR2XrWqI8zP8uPRERun93dDZ+qpBxEU61/RC TW2lS9T7lUyjWAAoPc4wL0p0YSbT4lEhpE89m2TZDZwTY1yVivVskv1W+fZjwMnxFHqY+I3sqP/6 fbVAauuFRx1R1E6Ex+RJ82RxLW3yQSoaSBAjg94gJPQrLSx5vR0rDgdMIpeRfCXgMulBfqTZRgke iE+QiLhI5xml93ePlfAANcT5maP0/u4AYZpwllGdBywAlJ66MJPphx31XxEt8iirfPuxPUz8xp7Z mFgI6Z8vsriWtiSIkcG1lS5RI6UpJpn0IL8PxCfIrFFDVjphRCGAME24FgBKz+MNQkJ1PUU1z2xM rFlcS9v6yS9FbPkoMtaoIatAmCbc0YWZTEe1njv95Jeia9SQ1chB9EtecfM85CD6pXIQ/dKWsP/v xbD/72Sw/++3r//vk67/76ew/+9wr//vYa//70mt/+8grf/vBq//73+u/+8cr//vIK7/7wGt/+/E rP/v2a3/73at/+9erf/vEa//729Z/////////////zFU///jX///g1j/////////////01L///de //+LWf////////////9nUv///1///49Y/////////////yNS//8DX///J1j/////////////F1L/ /5te/////////////////////////////7lS//+lUv//x1L//4dS//95Uv//l1L///////9FV/// N1f//ylX//8ZV///UVf//w1X///9Vv//9Vb//+NW///LVv//u1b//6VW//+XVv//gVb//3FW//9d Vv//TVb//zdW//8nVv//FVb//wNW///3Vf//5VX//49X//9fV///v1X//7NV//+nVf//m1X//49V //9/Vf//bVX//1tV//9BVf//MVX//x1V//8NVf//+1T//+tU///PVP//v1T//7FU//+dVP//kVT/ /4FU//9vVP//YVT//+tX//93V///01f//6FX//+3V///11X//8tV//9HVP///////1lS//9BUv// /////4dT//95U///kVP//59T//9pU///TVP//z9T//8rU///F1P//1tT//8JU///5VL//81T///Z U///51P///VT//8FVP//F1T//yNU//+vU///91L//79T////////7///f/P//3/t//9//P//f+z/ /3/7//9/6P//f/b//3/1//9/xv//f8v//3/4//9/kP//f4z//3//////gv64mouolpGbkIiMu5aN mpyLkI2Gvv//Vf62kZaLlp6TloWavI2Wi5acnpOsmpyLlpCR/zH9qJ6Wi7mQjayWkZiTmrCdlZqc i/9g/auajZKWkZ6LmquXjZqem/8+/rOanomavI2Wi5acnpOsmpyLlpCR//+Z/7qRi5qNvI2Wi5ac npOsmpyLlpCR///k/7yTkIyat56Rm5Oa/+f9rZqem7mWk5r//+3+uJqLuZaTmqyWhZr/y/+8jZqe i5q5lpOavv+S/riai6uWnJS8kIqRi///b/+5lpGbvJOQjJr/Yv+5lpGbsZqHi7mWk5q+/2n9rJOa mo//a/+5lpGbuZaNjIu5lpOavv//ov2smou8io2NmpGLu5aNmpyLkI2Gvv//tf+8jZqei5qrl42a npv///L+uJqLuZaTmr6Li42WnYqLmoy+//+o/7uak5qLmrmWk5q+/xv+soqTi5a9houaq5Colpua vJeejf/7/riai7uNlomaq4aPmr7/eP2smourl42anpuvjZaQjZaLhv8p/rKej6mWmoiwmbmWk5r/ yv+8jZqei5q5lpOasp6Pj5aRmL7//579rJqLupGbsJm5lpOa//+V/ayai7mWk5qvkJaRi5qN//9P /aqRkp6PqZaaiLCZuZaTmv9m/reano++k5OQnP+//riai6+NkJyajIy3mp6P//9d/reano+tmr6T k5Cc/2D+t5qej7mNmpr///38k4yLjZyPhr7///f8k4yLjZOakb7//wb9k4yLjZyei77//wD9k4yL jZySj5a+/wP9k4yLjZySj77//4r+uJqLqZqNjJaQkbqHvv+R/riai6uWkpq5kI2Snou+//8E/7ia i7uei5q5kI2Snou+//+P/riai6uWkpqlkJGatpGZkI2SnouWkJH//+T+uJqLs5CcnpOrlpKa//+c /riai6uako+5lpOasZ6Smr7//5r+uJqLq5qSj6+ei5e+///B/riai6+NkJy+m5uNmoyM//89/rOQ npuzlp2Nno2Gvv//xf64mouvjZaJnouar42QmZaTmqyLjZaRmL7//8D/vI2anouasoqLmoe+//8S /rCPmpGyiouah77//1X/uZOKjJe5lpOavYqZmZqNjP//IP2ojZaLmrmWk5r/2v2tmpOanoyasoqL mof//+P+uJqLs5CcnpOatpGZkL7//0v/uY2amrOWnY2ejYb/oP64moushoyLmpKrlpKavoy5lpOa q5aSmv/b/riai7KQm4qTmrmWk5qxnpKavv//tLqtsbqzzM3Rm5OT//9T/YiMj42WkYuZvv+h/ria i6iWkZuQiKuah4u+//8S/7iai7yTnoyMsZ6Smr7/lf2sl5CIqJaRm5CI///9/riai7uTmLaLmpL/ /8r+uJqLr56NmpGL/y//upGKkqiWkZuQiIz/2P64mouympGKrIuei5r//+v9rJqRm7KajIyemJq+ //+9/riai6yKnbKakYr//+P+uJqLspqRiv8q/7mWkZuolpGbkIi+/yH+r5CMi7KajIyemJq+///e /7yXno2zkIiajb7//9D/vJeejaqPj5qNvv//Uf2IiYyPjZaRi5m+//9q/7uWjI+ei5yXspqMjJ6Y mr7//339q42ekYyTnouaspqMjJ6Ymv//1f64mouymoyMnpiavv+m/7yNmp6LmqiWkZuQiLqHvv8N /q2amJaMi5qNvJOejIy+//97/7uamaiWkZuQiK+NkJy+//+qrLqtzM3Rm5OT//+k/q2amLyTkIya tJqG/4T+rZqYroqajYapnpOKmrqHvv//jv6tmpiwj5qRtJqGvv+N/q2amLCPmpG0moa6h77/mf6t mpi6kYqStJqGvv95/q2amKyai6mek4qauoe+//++u6m+r7bMzdGbk5P//6//rLe4mouvnouXuY2Q kra7s5aMi77//6z/rLe4mousj5qclp6TuZCTm5qNs5CcnouWkJH//6y3urOzzM3Rm5OT/6issLy0 zM3Rm5OT//////////////////////8GCQG//////7VQ///+////4v///+L////XUf//Y1H//+9Q //+ecP//inP//5Rz//9tb///0m///xyv///Zrf//zq3//6ew//9ksP//Sa3//yCt//9erf//aa3/ /wGt///FsP//S6///8Ss//+WsP//dq3//2Gv//9wr///HK///1Ou//+3r///IK7//3+u//8Gr/// Ea///49Q//9/UP//r1D//51Q//+XUP//bVD//2VQ//9gUP//W1D//1RQ//9PUP//R1D//z9Q//83 UP//L1D//yZQ//8fUP//GFD//xNQ//8LUP//BVD///5P///2T///70///+dP///fT///1k///85P ///HT////P/7/////v/9//r/+f/4//f/9v/1//T/8//y//H/8P/v/+7/7f/s/+v/6v/p/+j/5//m /+X/5P/j/9/Rm5OT/7uTk62amJaMi5qNrJqNiZqN/6CSnpaR/6CSnpaRrbv/u5OTvJ6RqpGTkJ6b sZCI/7uTk7iai7yTnoyMsJ2VmpyL/6CMi42TiI3/nouQlv+ei5CT/5yek5OQnP+ZjZqa/5aMnpOR ipL/loyek4+Xnv+WjJuWmJaL/5aMjI+enJr/loyHm5aYlov/kp6Tk5Cc/5KakpyXjf+NnpGb/42a npOTkJz/jI2ekZv/jIuNnJeN/4yLjZacko//jIuNk4iN/4yLjZGcko//jIuNkZyPhv+Mi42RlpyS j/+Mi42NnJeN/4yLjYyLjf+Mi42Kj43///////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////+N////np2cm5qZmJeWlZSTkpGQj46NjIuKiYiHhoW+vby7urm4t7a1 tLOysbCvrq2sq6qpqKempc/OzczLysnIx8bS0aD////f08TFw8HC3fXy///f08TFw8HA2dbU3fXy ////Pz3/70s9/+9TPf/vWz3/72c9/+9vPf/vez3/74M9/++LPf/vkz3/75s9/++jPf/vqz3/77M9 /++7Pf/vwz3/78s9/+/TPf/v3z3/7+c9/+/3Pf/v/z3/7/////8HPv/vCz7/7xM+/+8bPv/vIz7/ 7ys+/+8zPv/vPz7/70c+/+9PPv/vWz7/78s9/+9nPv/vbz7/73s+/+/TPf/vgz7/7/////+LPv/v kz7/75s+/++jPv/v/Pz8/P79/fz9////qz7/77c+/+/DPv/vyz7/75KMkdGckJL/l5CLkp6Wk9Gc kJL/hp6XkJDRnJCS////npCT0ZyQkv/Rq6er/////9G3q7Kz////0bersv/////RqL69/////4iX mo2a////kpacjZCMkJmL////jJqcipGWnv+Ri52KmIuNno7///+RmpCXno+Mloz///+amoaa//// /5GeltGc////i42akZuSlpyNkP//nZaLm5qZ//+MkI+XkP///4+ekZue////jIaSnpH///+Jlo2K jP///56Jj/+UnoyPmo2M/5yQkZmWjZL/jIqdjJyNlo+LlpCR/////5GaiIz/////jZqYloyLmo3/ ////jI+ekv////+MmpyKjf///4yKj4+QjYv/jJqNiZacmv+ckJGLnpyL/56dioya////lpGZkP// //+RkIuXlpGY/56RhpCRmv//kZCdkJuG//+RkJCRmv///4yQkpqdkJuG/////4yQkpqQkZr/j5CM i5KejIuajf//kp6Wk5aRmP+SnpaTmo3//4ianZKejIuajf///56bkpaR////kp6Wk9GZ2or///// sqy2srHRuqe6////kp6Wk9GMnM//////kp6Wk9GZmv+bnov/kp6Wk9GZlpOa////2oy/2oz////R mpuK/////8PajMH/////2ozfw9qMwf+em5uNmoyMmoz////V0dX/qoyajdGxuqvfspqMjJqRmJqN 36yajYmWnJr//6yQmYuIno2ao7KWnI2QjJCZi6OyrLGymoyMmpGYmo3/rLKrr9+7loyPk56G37Ge kpr///+ssquv37qSnpaT376bm42ajIz//76cnJCKkYuMo9qM/7uamZ6Kk4vfsp6Wk9++nJyQipGL /////6yQmYuIno2ao7KWnI2QjJCZi6O2kYuajZGai9++nJyQipGL37KekZ6Ymo3/rJCZi4iejZqj spacjZCMkJmLo6i+vaOovr3Lo6iend+5lpOa37Gekpr////czM3IyM///7KssbKsvbO8k56MjP// //+yrLHfspqMjJqRmJqN////u6y9ipubhrOWjIusnomam/////+skJmLiJ6NmqOylpyNkIyQmYuj sqyxspqMjJqRmJqNo6+aja+ejIyPkI2LrJqLi5aRmIz/vJCRi56ci7OWjIuvnouX/5zFo5KMkZyQ kYuenIuTloyL0ZyLi////5KMkZyT0YyeiZqb/7mws7u6razRu72n/9qcxaO7kJyKkpqRi4zfnpGb 36yai4uWkZiM///anMWj/////76Pj5OWnJ6LlpCR37uei57/////8vXR8vX///+7vqu+8vX//628 r6vfq7DF38PajMHy9f+yvraz37mtsLLF38PajMHy9f///7e6s7Df2ozy9f///9qMxc3K////8vX/ /87Kx9HOys3RztHKx/////8AAAAAdzn/7y8o/e8vKP3vfzn/7y8o/e8vKP3vizn/7y8o/e8vKP3v lzn/7y8o/e8vKP3vmzn/76M5/++rOf/vtzn/7785/++rOf/vyzn/79M5/++rOf/v3zn/79M5/++r Of/v5zn/7/s5/++rOf/vAzr/7y8o/e8vKP3vCzr/7xc6/+8fOv/vkpqMjJ6Ymv+bkJyKkpqRi/// //+bmouelpOM/7CU35yKkYv/nI2akpqgm5qgmI2KhpqNmv////+YiouLmpv//7mai5aMl5qM//// /4+XkIuQ////ttiS35GKm5r/////lZqRlpmajf+omovfmJaNk4z////RlY+Y/////52NlouRmob/ rJqH/7bYkt+Wkd+TkIma/7aSj5CNi56Ri////7eak5OQ////t5b//5KelpPRjJqRi////5mWk5rO 0Y+ei5f//7KajIyemJrStrvF39qM8vW5jZCSxd/ajPL1q5DF39qM8vWsip2VmpyLxd/ajPL1u56L msXf2ozy9bKWkprSqZqNjJaQkcXfztHP8vW8kJGLmpGL0quGj5rF35KKk4uWj56Ni9CSloeam8Tf nZCKkZuejYbC3dqM3fL18vXS0tqM8vXajPL10tLajPL12ozy9dLS2ozS0vL1/////8Paz8eH0drP x4e/2ozB//+8kJGLmpGL0quGj5rF34uah4vQj5OelpHy9fL12ozy9f////+8kJGLmpGL0quGj5rF 39qMxPL19pGekprC3dqM3fL1vJCRi5qRi9KrjZ6RjJmajdK6kZyQm5aRmMXfnZ6MmsnL8vW8kJGL mpGL0ruWjI+QjJaLlpCRxd+ei4uenJeSmpGLxPL19pmWk5qRnpKawt3ajN3y9fL12ozy9f//no+P k5acnouWkJHQh9KFlo/SnJCSj42ajIyam//////ajNGFlo///9GMnI3/////3////9qM2oz///// u5CIkZOQnpu7lo3/0bu6ub6qs6ujrJCZi4iejZqjtJ6Fnp6js5CcnpO8kJGLmpGL/////9qc2or/ ////0YuHi/////+I/////v///9qM2ozajP//o////9qMo9qc2ozR2oz//9qc1dGbk5P/2ozf2ozf 2s/N0c2bz8///9qM39qM39Taz83RzZvPz/+3t9jF2JKS2MXYjIz/////m5ub2NPY35vfsrKy34aG hob///+vjZCcmoyMzM2xmoeL////r42QnJqMjMzNuZaNjIv//7yNmp6LmquQkJOXmpOPzM2skZ6P jJeQi/////+Umo2RmpPMzdGbk5P/////AAAAAJ2QkIv/////jIaMi5qS0ZaRlv//rLytsay+qbrR uqe6/////6ycjZqakayeiZq+nIuWiZr/////vJCRi42Qk9+vnpGak6O7moyUq5CP////nJmY0Zue i/+cmZi+nJyajIz////C////2or//9qK0dqK0dqK0dqKxdqK///F////o6P//7iai7Gai4iQjZSv no2ekoz/////to+Xk4+ej5bRm5OT/////wAAAACIiIjRmJCQmJOa0ZyQksXHz////5qRnp2Tmp6K i5Cblp6T//+skJmLiJ6NmqOylpyNkIyQmYujqJaRm5CIjKO8io2NmpGLqZqNjJaQkaO2kYuajZGa i9+smouLlpGYjP/Gztbe5u72/sXN1d3l7fX9xMzU3OTs9PzDy9PbwMjQ2ODo8PjBydHZ4enx+cLK 0tri6vL64+vz+/79+/n39fPx8O7s6ujm5OPx7vTn/vr84/D56vXo7PP75ffv+OTr8v3Wy+Da0Mjh 18zS3s/TztjH3crR1c3b4t9/////v////9/////v////9/////v////9/////v/////7/v7///// ///+//v7/v77//7++/v+//v///////7///v////7/v77+/7+//v///v7//77//7+/////vv////7 +/////v//v/7//7/+/7///v+/////v7///7++/v//vv//v/7///++////vv//v//////+/v///v7 /v/////+///+//v7/v77///////+/v/7/v7////+/////v/7///7//7+///+///7/v/7///+//v/ //v////7+//++/v+//v7/v77//7////+/vv7//77///++/v///v7/v//+/7++/v////7//7/+//+ //////v//v//+/7///////v//v7ff+9//3//f/9////ff+/////v/9/////f/+9/33//f9///3/f f+9//3/vf////3//f/9////v/9/////f/+9//3/v/9//7//ff/9//////////3//f///33/v//// 73/f/+//3///f///////f+//33////9/73///+9/33/////////ff+//3//vf///7//ff/9////v f/9/73//f//////vf/9//3/f////33/vf99/7//f/////3///////3/ff////3/vf///7//f//9/ 3//v/99//3/f//9/3//v//9/7////////3//f99///////9/3//vf99/73//f+//9/3////9/ff/ ////9//99//9//f/////9/39///9//f3//3/9///9/f///f///3/9/399/f//f////339/3///// //f3//////399//9/////f3////99/f//ff3/f3/9/3/9//9/f////3/9/3/9/f////3/f33//3/ //////f//f33////9/f//f/3/f/////9///9/ff//f/3///////9///3//3/9/399//9//f3///3 //3////////3//339/3/9////f/////39/399/f////3/f3///39//f///f///339/3/9/f9//// //339/39//f////3//33//39//7ff/9+3///ft///3////9/33//fv9///7/f//+3//////////f f///33//ft9//37/////////f/9///7/f//+/////9//////f//+33//f///////f//+3///f9// /37/f//+////f9///3//f///3///f99//37ff/9+////f/9///7/f///33//ft9//37///////// ///////ff/9/3///f/9//37/f//+/////t9//37f//9+3///f////37ff/9+/////v/////f///+ /3///t///3/ff/9+/3///t///3/f/////3///t9//3///////3///9///3/ff////v////73/f// 9/3//v+9///3///+//////+////3/f/+97////f///7//f/+97///v+9///3vf/+9/////+///// /f//97////e////////+/7///ve9//73vf/+//3///e9//7/v/////////+9//73/f////3///+9 //73////9////v+9//7///////3///+////3/f/+/73//ve///7//f///7////e9//73/f/+97// /v///////f//973//ve9//73/////73//ve9///3/f////////e/////vf/+9////v/9//7/v/// 9//////////3v//+9/3//v+/7///3///v9//v///77+/3///v9/v////77+/3///v///v//f77+/ ////v//v///f7/+///+//9/////f77/////////v/7//77//3/+/////v7//77//3+/////v/7/f 7/+/3//////vv7///7+/3++/////v7///7+/3////9//v//f7////+//v9//v7//77+/3///v//v v///7///3///v///v//f////3++////v///f77+/3/+/v////7/f77+///+/v9//////7/+/3+// ////v/////+/3++/v///v///7/+//++//9///////7+/3////9/v/7//77//3///3//9/9/7/ff/ +///////9////ff/+/333///99/7/fff+///3////////f//+/3////////7/f/f+/33////9//7 /fff//3/3///9//7/f//+///3/v/99/7/f/f////3/v/9////ff///333/v/99///f////////v/ 99//////+//33////9///ff/+/33//v9/9/7/f/f+/3////9/9//////+//3//v//9////ff+/33 ///999////ff+/33///9///7/fff+///3/v/99////////3////999/7//////333////9/7//f/ //3///v/9//7//f///3/3/+/7//v/+//////+/+/7/vv////77/v/++/////////77//+/////vv v+/77//v+///7/vvv+/7///v//+////////777///+//7//vv+/////v+/+///v/v//77//v+++/ 7/////////////+///vvv///7//v/++/7/v////7/7/v+/////v//+/77//v//+/////v//77//v //+/7/v//+//77////+////v///777//++/////v///7/7/v/+//////v+/777//+/+////v///7 7//v/++/7//v/////7/v++//7/v//+/7/7/v//+/7///v//7/////+//7/vvf/////////////// /////////////////////////////////////////////////////////////////////56dnJua mZiXlpWUk5KRkI+OjYyLiomIh4aFvr28u7q5/////769vLu6ubi3trW0s7KxsK+urayrqqmop6al np2cm5qZmJeWlZSTkpGQj46NjIuKiYiHhoXPzs3My8rJyMfG1ND/////3KCgi5qMi6Cg////xyf/ 7+Mn/+//J//vHyj/7z8o/+9bKP/veyj/75co/++zKP/vzyj/7+so/+8LKf/vKyn/70cp/+9nKf/v gyn/76Mp/++/Kf/v2yn/7/Mp/+8PKv/vKyr/70cq/+9fKv/vfyr/75sq/++3Kv/v6yr/7wsr/+8r K//vRyv/72Mr/++DK//vnyv/77cr/+9HKf/v0yv/7+Mr/++cl56LztGJkJaTntGZjf//noqMi5aR 0YuH0YqM0YqRm5qNkZqL0ZCNmP///5KajJ7RnoXRiozRipGbmo2RmovRkI2Y/4yKjY2ahtGKlNGa itGKkZuajZGai9GQjZj///+Mi5CclJeQk5LRjJrRmorRipGbmo2RmovRkI2Y/////5KQjJyQiNGN itGaitGKkZuajZGai9GQjZj///+Xnp6Nk5qS0ZGT0ZqK0YqRm5qNkZqL0ZCNmP//npKMi5qNm56S 0ZGT0ZqK0YqRm5qNkZqL0ZCNmP////+ekoyLmo2bnpLN0ZGT0ZqK0YqRm5qNkZqL0ZCNmP///46K mp2anNGOitGcntGKkZuajZGai9GQjZiYjZ6FzdGei9GaitGKkZuajZGai9GQjZj///+LkI2QkYuQ 0ZCR0Zye0YqRm5qNkZqL0ZCNmP//kpCRi42anpPRjorRnJ7RipGbmo2RmovRkI2Y/4mekZyQioma jdGdnNGcntGKkZuajZGai9GQjZj/////mI2ehdGei9GaitGKkZuajZGai9GQjZj/k5CRm5CR0YqU 0ZqK0YqRm5qNkZqL0ZCNmP///52NioyMmpOM0Z2a0ZqK0YqRm5qNkZqL0ZCNmP+blpqSmpHRkZPR morRipGbmo2RmovRkI2Y////kIyTkNGRkNGaitGKkZuajZGai9GQjZj/mZOekZuajYzRnZrRmorR ipGbmo2RmovRkI2Y/5OKk5qe0Yya0ZqK0YqRm5qNkZqL0ZCNmP////+TkIzSnpGYmpOajNGcntGK jNGKkZuajZGai9GQjZj//4+XkJqRlofRnoXRiozRipGbmo2RmovRkI2Y//+InoyXlpGYi5CR0Zuc 0YqM0YqRm5qNkZqL0ZCNmP///56Lk56Ri57RmJ7RiozRipGbmo2RmovRkI2Y//+SnpGXnouLnpHR lIzRiozRipGbmo2RmovRkI2Y/////52ek4uWkpCNmtGSm9GKjNGKkZuajZGai9GQjZj/////k56M iZqYnozRkYnRiozRipGbmo2RmovRkI2Y/5GaiIaQjZTRkYbRiozRipGbmo2RmovRkI2Y//+bnpOT nozRi4fRiozRipGbmo2RmovRkI2Y////jJ6Ti5OelJrRiovRiozRipGbmo2RmovRkI2Y/56Nk5aR mIuQkdGJntGKjNGKkZuajZGai9GQjZj/////noqclJOekZvRkYXRipGbmo2RmovRkI2Y/////56R kdKejZ2QjdGSltGKjNGKkZuajZGai9GQjZj/////kZqInY2KkYyIlpyU0ZGV0YqM0YqRm5qNkZqL 0ZCNmP+Pk56RkNGLh9GKjNGKkZuajZGai9GQjZj/////kpyTmp6R0Yme0YqM0YqRm5qNkZqL0ZCN mP///5yempHRmY3RmorRipGbmo2RmovRkI2Y/8WYmouWkZmQjP///6+ttqmyrLj/r7CxuN/ajP+v trG4/////7WwtrHf2oz/s7asq9/D2or/////2ozFycnJyP+xtry039qM/6qsuq3f2ozf2ozf2ozf xdqM////3v///6+ttqmyrLjf2ozfxdqM//+akYmQlpqMxd/aiv/aitHaiv///5ab//+gkp6Wkf// /5zFo4qPm56LmtGbk5P///+cl5qclIqPm56LmtGbiLOQiLuei5qrlpKa////nJeanJSKj5uei5rR m4i3lpiXu56LmquWkpr//4ickf+cl5SKj5vRm4i3lpiXu56LmquWkpr///+ZlpOazNGPnouX///f koqLmofN3/////+SkJ2Ki4r//4iWkYqPm4v/raqxu7OzzM3Ruqe639qM06CSnpaRrbv/rJCZi4ie jZqjspacjZCMkJmLo6iWkZuQiIyjvIqNjZqRi6majYyWkJGjrYqR////mZaTms3Rj56Ll///moeP k5CNmo3Rmoea//////////////////////////////////////////////////////////////// ///////v//9z////E8/pzsTOmM5WziPO7s2xzRbN3cxVzJbLmcp2yjbKgMgMyEfHOse4xqvGkMaH xn3GdsZqxmTGWcZRxkLGOsYvxifGGMYDxvjFx8WBxU7FOMUbxRTFrcR2xHDEJsQHxMPDqcOQw1TD CcPQwqDCgcJawkTC+8GZwVfBJcG7wHfAcMBewAnA/9/////+///2z+DPqM9zz1rPSM84zybPIM8S z+PO3s7NzqzOlc5VzkvOQc43zt/NyM1AzSPNFMwHzOjL38vGy7TLict5y3PLC8sEy9fKjsqHymDK RMowyh/KCMqvyT7JEMn2yFXIN8gqyAzIA8jsx+DHyMe9x57Hjsdpx2LHS8cqxwjH6cbKxr7Gtcaa xnDGSMZsxQDF88S8xLPEaMQ7xPHD6cPJw6zDicN5w3LDWsNRw0jDPcM2wyjD+8LpwtnCrsKowqLC ncKYwpLCgMJswlnCRsI3wujB4sHTwbfBnsGQwXrBUsFIwR7B8MDpwLTApMCewEXAPsAwwCHAAMD/ ///P//+D////8c/Ez77Prc9+z2bPSs8qzx3PDc/8zp7Oj87VzZLNKc0XzfbMlcx+zHbMcMw1zCHM +8vvy9/LzsuNy4XLaMsmy/XK5MrVys7KvMqEyn7KXspKyuHJlslLyQPJtMhQyF/HEsfixsHGcMSC w8rCScE6wQ7A////v///F/////zPk890z0fPOM8Dz7vOr86jznfOL84RzvjN4s3bzdbNdM1TzSHN 8My9zJ/MasxkzFDMQsw7zNTLn8sNy/7K7srLyrPKhMpuyg3K0cm6yYzJeMlwyWPJPcktySfJtcih yITIechWyE3IPcjqx9bHpsePx4jHdMcxxyfHHMcFx8fGisZFxjfGLcYcxgzGAcbxxc7FwsWvxaLF TsXAxKvElsSBxGzEV8RCxC3EGMQDxO7D2cPEw63DmMOBw2DDMcP1wnTCWMI/wjnCJcJbwVLBO8Ez wbPArMBxwGrAXMBVwP///6///xP///+8z4TPdc9mzyHPFs8Lz4TOdM1uzdDMyczDzH3MZ8w0zCbM H8wKzK3LpcrwyOLIxMguyDTH5sbYxtLGs8aZxk7GPcYlxgvG68XcxcvFwcW1xZrFk8WAxWvFQMUw xSnFH8USxQzFBsX5xPTE7MTmxN/E2cTRxMvEwsS1xInEasRKxDbEKsQfxBfECcTYw8vDusOow4nD a8NVw0PDPsM4wyDDCMPRwrrCpcKewpPCjMJxwlnCTsJIwjLCJsIPwgDC6sHawdTBzsG+wbjBBsH3 wOLA3MDHwMHAssCswKbAoMCawJTAL8D/n///Y////93Pz8/Jz7fPns+Ez33PZM9Tz0HP4M7IzsHO ic6CzmzOMs4fzhnOEs4MzurN383MzbTNc8uuyl7KTMpQyRLJA8l1yBbI+sfqx3LHacdgx1nHUsdA xy7HIcfsxtPGu8atxpjGd8ZjxlfGJ8auxRHF4cTaxNDEvsSZw4/DfcN2w1HDR8M1wy7DOsIkwufB uMGXwUPB////j///q////8bPUMZGxj/GOMYpxhHGC8buxcTFsMVtxQ7FrMRuxBTE/8P1w+7D58PY w8PDvMOww6rDnsNVwwzDr8KCwnjCccJqwkzC0MFEwZfA////f///+/7//8nPYc9CzzvPJs/hzqrO Zs5ezi/OKM4Uzr/Nsc2hzUTNPc37zNzMk8yKzCDMC8z2y5nLi8tyy23LZstNyzrLDsv3ypfKUspJ yiPK3MmFyUPJBsn2yOTIpsifyE/IGsgOyAPI7sfix9XHxsfBx7PHoMdox1THQMfNxrXGrsajxpvG jsYrxt/F2MWRxXTFXMVRxUrFOMUzxS3F9sTPw6rDpMOew37DaMNXwzXDGsMRwwrD98Lmwt7Cc8I5 wjHCKsIiwhfC+MHxwePB28HRwbrBsMGfwZnBicFewUbBP8EywevA3cDVwMPAtcCtwH7AZ8BTwEvA PsAowB3ADcAGwP9v///H////8M/nz97Pt8+sz57Pjc+Fz33PbM9Xz1HPS89Fzz/POc8zzy3PJ88h zxvPFc8PzwnP/1///8f///9fzlvOV85Tyk/KS8pHykPKP8o7yjfKM8ovyivKJ8ojyh/KG8oXyhPK D8oLygfK////P///V////5vPl8+Tz4/Pi8+Hz4PPf897z3fPc89vz2vPZ89jz1/PW89Xz1PPT89L z0fPP887zzfPM88vzyvPJ88jzx/PG88XzxPPD88LzwfPA8//zvfO887vzuvO287XztPOz86jyp/K m8qXypPKj8qLyofKg8p/ynvKd8pzym/Ka8pnymPKX8pbylfKU8pPykvKR8pDyj/KO8o3yjPKL8or yifKI8r/L///q////3vMd8xzzG/Ma8xnzGPMX8xbzFfMU8xPzEvMR8xDzD/MO8w3zDPML8wrzCfM I8wfzBvMF8wTzA/MC8wHzAPM/8v7y/fL88vvy+vL58v///////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////////////////////////////////wAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAIAAwAAACAAAIAO AAAAYAAAgAAAAAAAAAAAAAAAAAAAAQABAAAAOAAAgAAAAAAAAAAAAAAAAAAAAQAMBAAAUAAAAKDw AADoAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEAZgAAAHgAAIAAAAAAAAAAAAAAAAAAAAEADAQA AJAAAACI8wAAFAAAAAAAAAAAAAAAKAAAACAAAABAAAAAAQAEAAAAAAAAAgAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAL8AAL8AAAC/vwC/AAAAvwC/AL+/AADAwMAAgICAAAAA/wAA/wAAAP//AP8AAAD/ AP8A//8AAP///wAAAAAAAAAAAAAAAAAAAAAAAAh3d3d3d3d3d3d3cHAAAACP//////////////cH AAAAj//////////////3BwAAAI/wAAAP////////9wcAAACP//////////////cHAAAAj/AAAA// ///////3BwAAAI//////////////9wcAAACP//////////////cHAAAAj/AAAAAAAAAAAA/3BwAA AI//////////////9wcAAACP8AAAAAAAAAAAD/cHAAAAj//////////////3BwAAAI/wAAAAAAAA AAAP9wcAAACP//////////////cHAAAAj/AAAAAAAAAAAA/3BwAAAI//////////////9wcAAACP //////////////cHAAAAj/AAAA/////////3BwAAAI//////////////9wcAAACP//////////// //cHAAAAj//////////////3BwAAAI/wAAAP////////9wcAAACP//////////////cHAAAAj/AA AA////8PAA/3BwAAAI//////////////9wcAAACP//////////////cHAAAAj//////////////3 BwAAAI8P8P8P8P8P8P8P+AcAAACPD/D/D/D/D/D/D/gHAAAACPiPiPiPiPiPiPiPgAAAAAAAAAAA AAAAAAAAAAAAAPAAAB/gAAAPwAAAB8AAAAfAAAAHwAAAB8AAAAfAAAAHwAAAB8AAAAfAAAAHwAAA B8AAAAfAAAAHwAAAB8AAAAfAAAAHwAAAB8AAAAfAAAAHwAAAB8AAAAfAAAAHwAAAB8AAAAfAAAAH wAAAB8AAAAfAAAAHwAAAB+AAAA/ySSS/AAABAAEAICAQAAEABADoAgAAAQAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAUEsBAhQACgAAAAAAAAAAAIdw1mwA0AAAANAA AKEAAAAAAAAAAQCAAAAAAAAAAHNreS50eHQgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAuc2NyUEsFBgAAAAABAAEAzwAAAL/QAACHAgAAAAAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg --OEknMdrQymslrN-- From sfeldma@pobox.com Thu Dec 2 21:21:15 2004 Received: with ECARTIS (v1.0.0; list netdev); Thu, 02 Dec 2004 21:21:19 -0800 (PST) Received: from orb.pobox.com (orb.pobox.com [207.8.226.5]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB35LE61004916 for ; Thu, 2 Dec 2004 21:21:14 -0800 Received: from orb (localhost [127.0.0.1]) by orb.pobox.com (Postfix) with ESMTP id 9C1FA2F7118; Fri, 3 Dec 2004 00:20:52 -0500 (EST) Received: from [192.168.200.234] (209-128-68-065.bayarea.net [209.128.68.65]) by orb.sasl.smtp.pobox.com (Postfix) with ESMTP id 253032FA2A7; Fri, 3 Dec 2004 00:20:43 -0500 (EST) Subject: Re: [E1000-devel] Transmission limit From: Scott Feldman Reply-To: sfeldma@pobox.com To: Robert Olsson Cc: Lennert Buytenhek , jamal , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com In-Reply-To: <16815.23964.93437.411404@robur.slu.se> References: <1101467291.24742.70.camel@mellia.lipar.polito.it> <41A73826.3000109@draigBrady.com> <16807.20052.569125.686158@robur.slu.se> <1101484740.24742.213.camel@mellia.lipar.polito.it> <41A76085.7000105@draigBrady.com> <1101499285.1079.45.camel@jzny.localdomain> <16811.8052.678955.795327@robur.slu.se> <1101821501.1043.43.camel@jzny.localdomain> <20041130134600.GA31515@xi.wantstofly.org> <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <16813.58484.343629.570703@robur.slu.se> <1101919791.5198.15.camel@localhost.localdomain> <16815.23964.93437.411404@robur.slu.se> Content-Type: text/plain Message-Id: <1102051410.3546.45.camel@localhost.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 (1.4.6-2) Date: Thu, 02 Dec 2004 21:23:30 -0800 Content-Transfer-Encoding: 7bit X-archive-position: 12404 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: sfeldma@pobox.com Precedence: bulk X-list: netdev On Thu, 2004-12-02 at 10:23, Robert Olsson wrote: > It can increase TX performance from 800 kpps to > 1125128pps 576Mb/sec (576065536bps) errors: 0 > 1124946pps 575Mb/sec (575972352bps) errors: 0 These are the best numbers reported so far, right? > And some of Scotts may still be used. Did you try combining the two? > + > + if( adapter->tx_ring.next_to_use - adapter->tx_ring.next_to_clean > 80 ) > + e1000_clean_tx_ring(adapter); > + You want to use E1000_DESC_UNUSED here because of the ring wrap. ;-) -scott From madhaviram123@yahoo.com Thu Dec 2 21:40:38 2004 Received: with ECARTIS (v1.0.0; list netdev); Thu, 02 Dec 2004 21:40:42 -0800 (PST) Received: from web90003.mail.scd.yahoo.com (web90003.mail.scd.yahoo.com [66.218.94.61]) by oss.sgi.com (8.13.0/8.13.0) with SMTP id iB35ec54005686 for ; Thu, 2 Dec 2004 21:40:38 -0800 Received: (qmail 74387 invoked by uid 60001); 3 Dec 2004 05:40:12 -0000 Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; b=jYStYmvGTJ1LpZdR0DUUGf9hxK0M53uLXmqxYUeq50OcsTq9CSwLFvLvZK3pFA6zIO4RTYL+OHslvauvU3SJoKDdXvjSepyCKLyfnSPkOe/eyrePJTTNYuClnr3wvDv3TxZPVu8AiMfl84IEUnWW7xQSElRNwelTNq+YMpGwZyI= ; Message-ID: <20041203054012.74385.qmail@web90003.mail.scd.yahoo.com> Received: from [203.199.182.34] by web90003.mail.scd.yahoo.com via HTTP; Thu, 02 Dec 2004 21:40:12 PST Date: Thu, 2 Dec 2004 21:40:12 -0800 (PST) From: ram mohan Subject: Contribute - Howto To: netdev@oss.sgi.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-archive-position: 12405 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: madhaviram123@yahoo.com Precedence: bulk X-list: netdev Hi, I am willing to contribute to the "networking side " of linux. I googled a bit and found that I should join the list and then I can go ahead. I would like to know. 1. What are the features currently being worked upon? 2. Are there any things-to-do lists maintained? 3. How are new features selected? How do I start?? Thanks. __________________________________ Do you Yahoo!? Jazz up your holiday email with celebrity designs. Learn more. http://celebrity.mail.yahoo.com From SRS0+348d0c150552644be866+467+infradead.org+hch@pentafluge.srs.infradead.org Fri Dec 3 02:34:05 2004 Received: with ECARTIS (v1.0.0; list netdev); Fri, 03 Dec 2004 02:34:11 -0800 (PST) Received: from pentafluge.infradead.org ([213.146.154.40]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB3AY3Lv016496 for ; Fri, 3 Dec 2004 02:34:04 -0800 Received: from hch by pentafluge.infradead.org with local (Exim 4.42 #1 (Red Hat Linux)) id 1CaAkt-0002pc-CX; Fri, 03 Dec 2004 10:33:39 +0000 Date: Fri, 3 Dec 2004 10:33:39 +0000 From: Christoph Hellwig To: James Ketrenos Cc: Jeff Garzik , Netdev Subject: Re: Steps for netdev-2.6 inclusion? Message-ID: <20041203103339.GB10799@infradead.org> References: <41AE7143.80505@linux.intel.com> <41AEB3B8.2000406@pobox.com> <41AF7708.3030804@linux.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <41AF7708.3030804@linux.intel.com> User-Agent: Mutt/1.4.1i X-SRS-Rewrite: SMTP reverse-path rewritten from by pentafluge.infradead.org See http://www.infradead.org/rpr.html X-archive-position: 12406 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hch@infradead.org Precedence: bulk X-list: netdev On Thu, Dec 02, 2004 at 02:11:52PM -0600, James Ketrenos wrote: > Jeff Garzik wrote: > > >It's fairly easy, just email me and netdev the patch for inclusion, > >and it'll get reviewed. > > Should I break the patch into the three components (ipw2100, ipw2200, > and ieee80211) ? or just one huge patch? Not sure what you and others > would prefer. Please split it in three parts. And most importantly integrate your ieee80211 code with the hostap code in Jeff's queue, we really don't want two copies of it. > The firmware doesn't have to be downloaded from Sourceforge, but it does > need to exist on the system, just as iwconfig needs to exist if you want > to be able to configure your wireless card. The user can get the > firmware from Sourceforge, or have it installed by their distribution or > package management system, have it on their Knoppix CD, etc. The problem is again that the license for it is horrible multi page legaleese. Please change it to a simple license that allows completely free redistribution and modification of the binaries. From hadi@cyberus.ca Fri Dec 3 05:08:20 2004 Received: with ECARTIS (v1.0.0; list netdev); Fri, 03 Dec 2004 05:08:24 -0800 (PST) Received: from mx03.cybersurf.com (mx03.cybersurf.com [209.197.145.106]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB3D8J71026716 for ; Fri, 3 Dec 2004 05:08:20 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx03.cybersurf.com with esmtp (Exim 4.30) id 1CaDAC-00085a-CW for netdev@oss.sgi.com; Fri, 03 Dec 2004 08:07:56 -0500 Received: from cpe0030ab124d2f-cm014500000962.cpe.net.cable.rogers.com ([24.103.99.32] helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1CaDA7-000392-Mu; Fri, 03 Dec 2004 08:07:51 -0500 Subject: Re: [E1000-devel] Transmission limit From: jamal Reply-To: hadi@cyberus.ca To: mellia@prezzemolo.polito.it Cc: Lennert Buytenhek , Harald Welte , P@draigBrady.com, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com In-Reply-To: <1101994772.18491.16.camel@mellia.lipar.polito.it> References: <1101467291.24742.70.camel@mellia.lipar.polito.it> <41A73826.3000109@draigBrady.com> <1101483081.24742.174.camel@mellia.lipar.polito.it> <20041127092503.GA12592@sunbeam.de.gnumonks.org> <1101718412.14930.46.camel@verza.polito.it> <20041129145028.GC18788@xi.wantstofly.org> <1101804146.11111.23.camel@mellia.lipar.polito.it> <1101903944.1042.29.camel@jzny.localdomain> <1101994772.18491.16.camel@mellia.lipar.polito.it> Content-Type: text/plain Organization: jamalopolous Message-Id: <1102079268.1216.19.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 03 Dec 2004 08:07:49 -0500 Content-Transfer-Encoding: 7bit X-archive-position: 12407 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev On Thu, 2004-12-02 at 08:39, Marco Mellia wrote: > We'll be glad to spend some time trying this out. Please, we are not > very confortable with the linux bitkeeper maintenance method. Can we ask > you to provide us a patch to a standard kernel/driver (whatever you > prefer...)? Also a complete source sub-tree would be ok ;-) Would a -rcX patch be fine for you? 2.6.10-rc2; which means you willl take 2.6.9 patch it with the patch-2.6.10-rc2.gz from kernel.org/v2.6/testing directory then patch one more time with patch i give you. Let me know if you are uncomfortable with that as well. [Sorry, I am disk poor and my stupid ISP still charges $1/MB/month even in this age if i put it up at cyberus]. In the patch i give you i will include rx path improvement code that I got from David Morsberger; I "think" i have seen some improvements with it but i am not 100% sure. If you repeat the test where you drop the packet right after eth_type_trans() with this patch on, I would be very interested if you see any improvements. In any case, expect something from me this weekend or monday (big party this weekend ;->). cheers, jamal From hadi@cyberus.ca Fri Dec 3 05:25:20 2004 Received: with ECARTIS (v1.0.0; list netdev); Fri, 03 Dec 2004 05:25:24 -0800 (PST) Received: from mx03.cybersurf.com (mx03.cybersurf.com [209.197.145.106]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB3DPJEI027489 for ; Fri, 3 Dec 2004 05:25:20 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx03.cybersurf.com with esmtp (Exim 4.30) id 1CaDQe-0006g5-G5 for netdev@oss.sgi.com; Fri, 03 Dec 2004 08:24:56 -0500 Received: from cpe0030ab124d2f-cm014500000962.cpe.net.cable.rogers.com ([24.103.99.32] helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1CaDQZ-0005d4-UJ; Fri, 03 Dec 2004 08:24:52 -0500 Subject: Re: [E1000-devel] Transmission limit From: jamal Reply-To: hadi@cyberus.ca To: sfeldma@pobox.com Cc: Lennert Buytenhek , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com In-Reply-To: <1101967983.4782.9.camel@localhost.localdomain> References: <1101484740.24742.213.camel@mellia.lipar.polito.it> <41A76085.7000105@draigBrady.com> <1101499285.1079.45.camel@jzny.localdomain> <16811.8052.678955.795327@robur.slu.se> <1101821501.1043.43.camel@jzny.localdomain> <20041130134600.GA31515@xi.wantstofly.org> <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <20041201182943.GA14470@xi.wantstofly.org> <20041201213550.GF14470@xi.wantstofly.org> <1101967983.4782.9.camel@localhost.localdomain> Content-Type: text/plain Organization: jamalopolous Message-Id: <1102080289.1214.22.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 03 Dec 2004 08:24:49 -0500 Content-Transfer-Encoding: 7bit X-archive-position: 12408 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev On Thu, 2004-12-02 at 01:13, Scott Feldman wrote: > On Wed, 2004-12-01 at 13:35, Lennert Buytenhek wrote: > > Pretty graph attached. From ~220B packets or so it does wire speed, but > > there's still an odd drop in performance around 256B packets (which is > > also there without your patch.) From 350B packets or so, performance is > > identical with or without your patch (wire speed.) > > Seems this is helping PCI nics but not PCI-X. I was using PCI 32/33. > Can't explain the dip around 256B. > Interesting thought. I also saw improvements with my batching patch for PCI 32/32 but nothing noticeable in PCI-X 64/66. cheers, jamal From hadi@cyberus.ca Fri Dec 3 05:46:04 2004 Received: with ECARTIS (v1.0.0; list netdev); Fri, 03 Dec 2004 05:46:09 -0800 (PST) Received: from mx03.cybersurf.com (mx03.cybersurf.com [209.197.145.106]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB3Dk4Lv028316 for ; Fri, 3 Dec 2004 05:46:04 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx03.cybersurf.com with esmtp (Exim 4.30) id 1CaDki-0002p2-SK for netdev@oss.sgi.com; Fri, 03 Dec 2004 08:45:40 -0500 Received: from cpe0030ab124d2f-cm014500000962.cpe.net.cable.rogers.com ([24.103.99.32] helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1CaDkf-00080P-Ad; Fri, 03 Dec 2004 08:45:37 -0500 Subject: Re: Contribute - Howto From: jamal Reply-To: hadi@cyberus.ca To: ram mohan Cc: netdev@oss.sgi.com In-Reply-To: <20041203054012.74385.qmail@web90003.mail.scd.yahoo.com> References: <20041203054012.74385.qmail@web90003.mail.scd.yahoo.com> Content-Type: text/plain Organization: jamalopolous Message-Id: <1102081534.1210.38.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 03 Dec 2004 08:45:34 -0500 Content-Transfer-Encoding: 7bit X-archive-position: 12409 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev Just join this list and listen to people complaining about bugs. Chase those bugs and fix them. Thats a good way to get hands dirty. Features are added based on technical merits and needs. cheers, jamal On Fri, 2004-12-03 at 00:40, ram mohan wrote: > Hi, > I am willing to contribute to the "networking side " > of linux. I googled a bit and found that I should join > the list and then I can go ahead. > I would like to know. > 1. What are the features currently being worked upon? > 2. Are there any things-to-do lists maintained? > 3. How are new features selected? > > How do I start?? > Thanks. > > > > __________________________________ > Do you Yahoo!? > Jazz up your holiday email with celebrity designs. Learn more. > http://celebrity.mail.yahoo.com > > From mludvig@suse.cz Fri Dec 3 09:43:39 2004 Received: with ECARTIS (v1.0.0; list netdev); Fri, 03 Dec 2004 09:43:46 -0800 (PST) Received: from mail.suse.cz (styx.suse.cz [82.119.242.94]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB3HhcEH031372 for ; Fri, 3 Dec 2004 09:43:39 -0800 Received: from [10.20.1.72] (ozzy.suse.cz [10.20.1.72]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.suse.cz (SUSE CR ESMTP Mailer) with ESMTP id 1F244628164; Fri, 3 Dec 2004 18:43:16 +0100 (CET) Message-ID: <41B0A5B4.6060108@suse.cz> Date: Fri, 03 Dec 2004 18:43:16 +0100 From: Michal Ludvig Organization: SuSE CR, s.r.o. User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.8a4) Gecko/20040927 X-Accept-Language: en MIME-Version: 1.0 To: Andrew Morton Cc: netdev@oss.sgi.com, Jan Kara Subject: [PATCH] rtnetlink & address family problem X-Enigmail-Version: 0.86.1.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: multipart/mixed; boundary="------------020005070601050006080808" X-archive-position: 12410 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: mludvig@suse.cz Precedence: bulk X-list: netdev This is a multi-part message in MIME format. --------------020005070601050006080808 Content-Type: text/plain; charset=ISO-8859-2 Content-Transfer-Encoding: 7bit Hi, running 'ip -6 addr flush dev eth0' on a kernel without IPv6 support flushes *all* addresses from the interface, even those IPv4 ones, because the unsupported protocol is substituted by PF_UNSPEC. IMHO it should better return with an error EAFNOSUPPORT. Attached patch fixes it. Please apply. BTW Credits to Jan Kara for discovering and analysing this bug. Michal Ludvig -- SUSE Labs mludvig@suse.cz (+420) 296.542.396 http://www.suse.cz Personal homepage http://www.logix.cz/michal --------------020005070601050006080808 Content-Type: text/plain; name="rtnetlink-family.diff" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="rtnetlink-family.diff" # This is a BitKeeper generated diff -Nru style patch. # # ChangeSet # 2004/12/03 18:06:31+01:00 michal@logix.cz # Return EAFNOSUPPORT if requested operation on unsupported family. # # Signed-off-by: Michal Ludvig # # net/core/rtnetlink.c # 2004/12/03 18:05:49+01:00 michal@logix.cz +4 -2 # Return EAFNOSUPPORT if requested operation on unsupported family. # diff -Nru a/net/core/rtnetlink.c b/net/core/rtnetlink.c --- a/net/core/rtnetlink.c 2004-12-03 18:30:33 +01:00 +++ b/net/core/rtnetlink.c 2004-12-03 18:30:33 +01:00 @@ -477,8 +477,10 @@ } link_tab = rtnetlink_links[family]; - if (link_tab == NULL) - link_tab = rtnetlink_links[PF_UNSPEC]; + if (link_tab == NULL) { + *errp = -EAFNOSUPPORT; + return -1; + } link = &link_tab[type]; sz_idx = type>>2; --------------020005070601050006080808-- From cap@nsc.liu.se Fri Dec 3 11:02:23 2004 Received: with ECARTIS (v1.0.0; list netdev); Fri, 03 Dec 2004 11:02:30 -0800 (PST) Received: from papput.nsc.liu.se (ns2.nsc.liu.se [130.236.101.9]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB3J2LSd001097 for ; Fri, 3 Dec 2004 11:02:22 -0800 Received: from mail.nsc.liu.se (mail.nsc.liu.se [130.236.101.49]) by papput.nsc.liu.se (Postfix) with ESMTP id 252E71C31F5 for ; Fri, 3 Dec 2004 20:02:00 +0100 (CET) Date: Fri, 3 Dec 2004 20:02:00 +0100 (CET) From: Peter Kjellstroem To: netdev@oss.sgi.com Subject: e1000>5.2.30 unstable with InterruptThrottleRate=0 Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 12411 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: cap@nsc.liu.se Precedence: bulk X-list: netdev Hello folks, Short version: 82547GI with ITR=0 on 2.4.28 (vanilla) and RHEL3u3 has problems (traffic grinds to a temporary halt under anything but trivila network traffic). kernel prints the following and resets the IF (many times): NETDEV WATCHDOG: eth0: transmit timed out More verbose version with background: I have a problem with e1000 being unstable when I run it with InterruptThrottleRate=0 (abbreviated ITR in the rest of this e-mail). I need to turn ITR off or set it so large that it behaves as off. The reason for having to turn it off is that I run MPI-applications (cluster stuff) and that happens to be largely latency bound. Latency with default e1000 is terrible, 250 us, with ITR=0 (where it works) the latency drops to 20-25 us. Enough of background. Up untill now I have allways been able to run with ITR=0 and intel gigabit has been very nice. Now, for some combinations of driver, chip and ITR setting it all falls apart. Affected chips (theory, 8254X, X>1 or anything faster then PCI33): 82547GI, 82546 (said to be affected, not verified by me) Unaffected chips: 82541 (rock solid no matter what driver or ITR) Linux-2.4.26 vanilla (smp, without NAPI with e1000 as module) is ok (82547, ITR=0, rock solid) Linux-2.4.28 vanilla (smp, without NAPI with e1000 as module) is BAD (82547 needs ITR<20000 for resonable stability) Linux-2.4.28 with e1000 from 2.4.26 but otherwise exactly as above is ok rock solid!!! Linux-2.4.21-20smp RHEL3 update 3 is BAD (known stable with default ITR (1?) but probably ok for <20000) Conclusions: something happened above e1000 version 5.2.30 (as in linux-2.4.26), RHEL has 5.2.52 and 2.4.28 has 5.4.11. Some more discussions on this subject has taken place on another list, see following thread if interested: http://lists.us.dell.com/pipermail/linux-poweredge/2004-November/023061.html Best Regards, Peter -- ------------------------------------------------------------ Peter Kjellstroem | E-mail: cap@nsc.liu.se National Supercomputer Centre | Sweden | http://www.nsc.liu.se From buytenh@wantstofly.org Fri Dec 3 12:57:30 2004 Received: with ECARTIS (v1.0.0; list netdev); Fri, 03 Dec 2004 12:57:36 -0800 (PST) Received: from xi.wantstofly.org (alephnull.demon.nl [212.238.201.82]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB3KvSBn007253 for ; Fri, 3 Dec 2004 12:57:29 -0800 Received: by xi.wantstofly.org (Postfix, from userid 500) id AF95B2B0F0; Fri, 3 Dec 2004 21:57:06 +0100 (MET) Date: Fri, 3 Dec 2004 21:57:06 +0100 From: Lennert Buytenhek To: Scott Feldman Cc: jamal , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: [E1000-devel] Transmission limit Message-ID: <20041203205706.GC9808@xi.wantstofly.org> References: <16807.20052.569125.686158@robur.slu.se> <1101484740.24742.213.camel@mellia.lipar.polito.it> <41A76085.7000105@draigBrady.com> <1101499285.1079.45.camel@jzny.localdomain> <16811.8052.678955.795327@robur.slu.se> <1101821501.1043.43.camel@jzny.localdomain> <20041130134600.GA31515@xi.wantstofly.org> <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="98e8jtXdkpgskNou" Content-Disposition: inline In-Reply-To: <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> User-Agent: Mutt/1.4.1i X-archive-position: 12412 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: buytenh@wantstofly.org Precedence: bulk X-list: netdev --98e8jtXdkpgskNou Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tue, Nov 30, 2004 at 05:09:59PM -0800, Scott Feldman wrote: > Hey, turns out, I know some e1000 tricks that might help get the kpps > numbers up. > > My problem is I only have a P4 desktop system with a 82544 nic running > at PCI 32/33Mhz, so I can't play with the big boys. But, attached is a > rework of the Tx path to eliminate 1) Tx interrupts, and 2) Tx > descriptor write-backs. For me, I see a nice jump in kpps, but I'd like > others to try with their setups. We should be able to get to wire speed > with 60-byte packets. Attached is a graph of my numbers with and without your patch for: - An 82540 at PCI 32/33, idle 33MHz card on the same bus forcing it to 33MHz. - An 82541 at PCI 32/66. - An 82546 at PCI-X 64/100, NIC can do 133MHz but mobo only does 100MHz. All 'phi' tests were done on my box phi, a dual 2.4GHz Xeon on an Intel SE7505VB2 board (http://www.intel.com/design/servers/se7505vb2/). I've included Robert's 64/133 numbers ('sourcemage') on his dual 866MHz P3 for comparison. I didn't test all packet sizes up to 1500, just the first few hundred bytes for each. As before, the max # pps at 60B packets is strongly influenced by the per- packet overhead (which seems to be reduced by your patch for my machine quite a bit, also on 64/100, even though Robert sees no improvement on 64/133) while the slope of each curve appears to depend only on the speed of the bus the NIC is in. I.e. the 60B kpps number more-or-less determines the shape of the rest of the graph in each case. Bus speed is most likely also the reason why the 64/100 setup w/o your patch starts off slower than the 64/66 with your patch, but then eventually beats the 64/66 (around 140B packets) just before they both hit the GigE saturation point. There's no drop at 256B for the 64/100 setup like with the 32/* setups. Perhaps the drop at 256B is because of the PCI latency timer being set to 64 by default, and that causes the transfer on 32b to be broken up in 256-byte chunks? I'm not able to saturate gigabit on 32/33 with 1500B packets, while Jamal does. Another thing to look into. Also note that the 64/100 NIC has rather wobbly performance between 60B and ~160B bytes. This 'square wave pattern' is there both with and without your patch, perhaps something particular to the NIC. Its period appears to be 16 bytes, dropping down where packet_size mod 16 = 0, and then jumping up again a bit when packet_size mod 16 = 6. Odd. --L --98e8jtXdkpgskNou Content-Type: image/png Content-Disposition: attachment; filename="perf.png" Content-Transfer-Encoding: base64 iVBORw0KGgoAAAANSUhEUgAABkAAAASUCAIAAACwXlbgAAAACXBIWXMAAAsTAAALEwEAmpwY AAAAB3RJTUUH1AwDFDYwJtQcAwAAIABJREFUeNrs3U2O20jWNtBQw5sgd/NOuqbpVQhI9B6c 3kNDQK7CnlZPvt1ELEPfgDaLJiWKEv+C5DlIFLJoiqJCzCzrqXsvTzHGAAAAAAC5+pclAAAA ACBnAiwAAAAAsibAAgAAACBrAiwAAAAAsibAAgAAACBrAiwAAAAAsibAAgAAACBrAiwAAAAA sibAAgAAACBrAiwAAAAAsibAAgAAACBrAiwAAAAAsibAAgAAACBrAiwAAAAAsibAAgAAACBr AiwAAAAAsibAAgAAACBrAiwAAAAAsibAAgAAACBrXyzB/pRlaREAAACA4WKMOZ+eCqy9kV4B AAAAz8o8T1CBtU+Z56b7/mnf5eIXZZkOfFHt+J31M+ud9c7incU7i3cW7yxhC9UwKrCAx1KM heI+AAAAViLAAgAAACBrAixgEEVYAAAArEWABQAAAEDWBFgAAAAAZE2ABQylixAAAIBVCLAA AAAAyJoAC3iOIiwAAAAWdooxWoU9KcsyhOBtZT5FWSYXGAAAwI7kHyaowAIAAAAgawIs4DlG uQMAALAwARYAAAAAWRNgAQAAAJA1ARbwNF2EAAAALEmABQAAAEDWBFjAKxRhAQAAsBgBFgAA AABZE2ABAAAAkDUBFvAiXYQAAAAsQ4AFAAAATKOc4X9yl/7HOQIsYCRFWAAAMIdWalP+1vrX 5sZwK+uZb0t3e1mWMcZnX2brJXQPG2OUYSHAAl6XnvyPEwAAMEQ3vYq/1X8UG5q7dROlObZM 9TK7rwtuEmABAABARl6oY2o+qg6D5tty81QHnvaQoKq7j4QLARYwilHuAAAwrW4MdC8Yutl8 l4n+xsae1/VafsfuCbAAAABgG7oVTyuWJvUnTd0Crjpu6+ZuQise+mIJAAAAIH/NlGf1uGdI 5FRlWK1z7j7whT7E4fbXL3LYScQCLGCCX6BFWRroDgAA89lrjdLNPGvCl+xzym5oIQQAAICs 9aQ8tQVmtzebAW8O6rrZGNja2Hxg9zjN+yr278nRqMACJqAICwAAptIsQQqNzrt6h9boq2aP XivomW/LkFfRzL965tA3n8W7zz0n18cuf9N5W1meAAsAAJijVEr51TJvXMg7TNBCCAAAAExj jgREekUQYAFTqboIrQMAAACTE2ABAAAAkDUBFjAZRVgAAADMQYAFAAAAQNYEWAAAAABkTYAF TEkXIQAAAJMTYAEAAACQNQEWMD1FWAAAAExIgAVMLMVoEQAAAJiQAGufis93iwAAAADsgwAL mJ5R7gAAAExIgAUAAABA1gRYAAAAAGRNgAXMQhchAAAAUxFgAQAAAJA1ARYwF0VYAAAATEKA BQAAAEDWBFgAAAAAZE2AtU/pfCk+360D61+KuggBAAAYTYAFAAAAQNYEWMC8FGEBAAAwkgAL AAAAgKwJsAAAAADImgALmJ0uQgAAWsqyrP/Zs8NrR648e8AhDxlykNazt7aUf7p38HuvYsgr nWOferfXdijHfRx4eLVwBF8sAQAAAFtRlmWMccif9u/ZfeDNQ3UP2HPY/p3r7x+e1ZBXseQ+ zT/q2ad6vS5RZqICC1iIIiwAAMYbnkk193xYOtQ6bDNsqst/Wlumde+Yw1/vVOu2zHHgWSqw gCXoIgQAoKlVi9SMb7rBU2u3ZzOUuhiq/2Tme5k3T6n1up7NxVYPkuqz7S+Iu/mqW2VfzeM0 l6LeZ2DlGvsmwAIAAGB93Ua27jetlKeVj9Q7tw7YcvNRL+gep3l690q6wrB2vG6sc++cH/ZU 9r/MbnDWfa7mCKrumQ9sTmwerZlhDXnfIQiwdiydL8XnezpfLAW5XJMxFmWZ/BcIAIBhHoYX N3d4OLJqvma6e7FO8/v+R917FUOmbvWfZP/sqnqfm2d4cxmn6qPsHmfa0Kos9tYIEtNBP1IJ sAAAANiz5h0Al6/oGfKkr53hugVKkzz1zSqtic8z+T/oO2GIOwAAAOsrf5s2lIm/hScHwN+b 3T4yY7pZcPTCGfY07g157FP7Dz/augdh31RgAcvRRQgAwD1DspuHEVLPFKqHhw1/9tB1++Zu dtI9fPbWfPohZzjwVXQn3/fcMLF1Pv37DHmznjryvTe6Oa9dhkW/k4loO9P8hWsGFhkSYAEA cPODjA+ni62k1ebmVRHyvtWjFkJgUVURlnUAAIA5DAkgpFdskQALAACAlYlUgH4CLAAAAACy JsAClqaLEAAAgKcIsPYsnS/F57t1AAAAADZNgDVK+VtrY3e3SbbAbijCAgAAYDgB1uuqO49W 6rCp2tjMnqbaAgAAAHBMAqwpValTCKHOnqbaAgAAAHBYAixgHboIAQAAGOiLJXhZszyqKpjK R31i129v3Rqu3M4WAAAAmNXWG7wEWKPe+zoJan4PDFQVYSU/OwAAAPQSYO3TP2na57tkDQAA AA6uPxzIvz7LDKyJr4ZpZ7cr7AIAAAA4yUfGuDkDq5s6TbVl+Ck1H1h8vqfzxZtFtnQRAgAA rKsbJuRGC+EoN9/a7saptgAAAAAckBZCYGXVKHfrAAAAwD0CLAAAAACyJsACAAAAIGsCLGB9 uggBAADoIcACAAAAIGsCLCALirAAAAC4R4AFAAAAQNYEWAAAAABkTYC1T2VR1l/pfCk+360J +dNFCADAap+hyrL+Z88Orx258uwBhzxkyEFaz97aUv7p3sHvvYohr3T8Pg/fHY7giyXYpZji Pz/qRXkNb9YEAABgcmVZxhiH/Gn/nt0H3jxU94A9h+3fuf7+4VkNeRVL7sMxqcDav2aYBZlT hAUAwMY+cA1OWJp7PixBah22GTbV5UitLdO6d8ypEiXJFM8SYAEAAHB0rVqknq66bmPds8/1 sLBovnCnp1ar9bqePYdZA6mBlWLsmxZCAAAAaOs2snW/aVU/tcKseufWAVtuPuoF3eM0T+9e SdfN19Vz8NaL6h78YU9l/8ucqaCMHRBgAXmpugiT/7sCAEBmHkZLN3d4OLJqvqa8e/Okmt/3 P+reqxgydav/JHv2bO4zflnKstjddZiO+QMowAIAAIDlNO8AuHxb3JAnfe0M85y5fti4Z3/M wAKyY5Q7AACrq2dgTRvKxN/CkwPg781uH5kxdaucXjvDnrsKDnnsU/tzTCqwAAAAoG1IdvMw QuqZQvXwsPU3zcSqeZCbDYkPn72ZEPXPyXr2VbSO3F2fnvPp3wdCCCcXxM40f8E1FZ/v6Xyx PmyFMVgAAKz7wcqH5cVW0mpn8k6FvENDLYRAjnQRAgDADgwJRKRXDCHAAgAAgD+IVCA3Aiwg U4qwAAAAqAiwAAAAAMiaAAsAAACArAmwgHzpIgQAACAIsAAAAADInAALyJoiLAAAAARYR3H6 +Fl8vlsHAAAAYHMEWEDuFGEBAAAcnAALAAAAgKwJsIBtUIQFAABwWAIsYANSjBYBAADgsARY AAAAAGRNgHUUMSlgYduMcgcAADgsARYAAAAAWRNgAZuhCAsAAOCYBFgAAAAAZE2AdSCnj5/F 57t1YNMUYQEAAByQAAsAAACArAmwAAAAAMiaAAvYGF2EAAAARyPAAgAAACBrXywBsDlVEVaK 0VIAAOxGWZYxxuqfPTu8cNjmv1ZHqDf2HLC7zySPunk+N1/gw9f72vm8tk+928Pj3Nzhtfdu +LXBEQiwAAAA2LD+XKMVCbX2v/fY7j4TPmpIClM+Gprx2vm8tk/zj3r2qV6vC5KZaCEENskk LAAAKgOrcl6u3+k+ao46oGb0M0ep0VQHVAPFWlRgAQAAsL5WdVIz0OlWUbV2e5iqNCOhJSOY noKm1qtoli8NOcPVg6SnmhNbr7pV9tU8TnMpWm+Z7OzgBFjHks6X4vM9nS+Wgn0wCQsAYMe6 rW3db1ptaz0Tr7plTc2j3XtUeGaaVU8IdbNTb3il1fDJWfceOLzvr/tc9QKGP2Om/hO4l1K1 miuHvMsQBFjH+u2fYlmU1/BmKdgHXYQAAIf7UPMozri5w82o5WFNVs/sqmpL91H3jt//XENi mpcH2w+Zb/WwnO3mAPWpxl11jzNtaFWUxe4+B6Vj/vgLsAAAADiWJUt7hjxXHeI8dWLrFihN 8tQ3q7Smddi4Z38McQc2TBEWAMCOlb+NyUq6sci66dXNgqO6nmt8ejUkBuppupxkhdc6CPum AgsAAIAcDUlz7s23ah6kZ5pVuN8/2HrUkCHrQ55ryHGaBVlDdgt3pn31nE//PkPemqeOfO9t bc5rl2HR72Qi2s48+AVXlNePN0Pc2Rmj3AEAdvnRxsfVmdbN2nLzqgh53+pRCyEAAADsxJAA QnrFFgmwgM0zCQsAYH+ELECTAAsAAACArAmwDiedL8Xnu3VgfxRhAQAA7JUAC9gDQ9wBAAB2 TIAFAAAAQNYEWMBOGOUOAACwVwIsAAAAALImwAL2QxEWAADALgmwAAAAAMiaAOtYYoploT6F PVOEBQAAsD8CLAAAAACyJsACdkgRFgAAwJ4IsI4onS/F57t1YLdXeIwWAQAAYE8EWAAAAABk TYAF7JBR7gAAAHsiwDoutyMEAAAANkGAdUSiK45AERYAAMBuCLAOJ6YYkxHXAAAAwGYIsIDd UoQFAACwDwIsAAAAALImwDqumKJhWOyeIiwAAIAdEGAd9VP9+VJ8vlsHAAAAIH8CLGD/FGEB AABs2hdLAOybLkIAAF5QlmWMsfpnzw4vHLb5r9UR6o09B+zuM8mjbp7PzRf48PW+dj4Dz7n/ veAIBFgAAAAwmf6cpRUJtfa/99juPhM+akgqVD76X8Kvnc9r+3BMWgiB/VOEBQDAYgZmLi+n M91HzZHyNBOrOYIkyRTPEmAd2vXjzSIAAAB0taqTyobmbq0t3R1uakZCS0Y5Pe2QPYnVkDOc 9VUMrxRjx7QQHpcbEXKsCz7GoiyT/+YBAPCqbmtb95uqTa9+SM/Eq25ZU/No9x4Vnplm1TyZ e89181U8NHxy1r0H3tuntYBQE2AdXfH5ns4X6wAAAPCsh3HPzR1uznh6WJPVM7uq2tJ91L3j 9z/XkBjr5cH2Q+ZbdRO6Mcqi2NtVl9Ixf9wEWId2+vipi5DjUIQFAECelpxWPuS56uToqRPL c+b6YeOe/TED69BiiqePnxoJAQAAHqpnYI2JabpVReumV93zib+FZ8ZO9dxV8Kk10T/IPSqw gGNRhAUAwGuGpDn35ls1D9IzzSrc7x9sPapnvtVTzzXkOM2CrCG7hTvTvnrOp38fCCGcXBA7 0/8L5cb+RRlTNAmL4xBgAQDw2kctH59nWjdrm8k7FfIODbUQAodTKEsGAIBFDAlEpFcMIcAi hBDS+WISFke52v3XEQCA5wlZYF1mYLG+svijHCYm/2FgdhoJAQAANkSAxS9VEdZTk7BawdPL monVVMeEvqs9Rl2EAAAAGyLAYhTVUgAAAMDczMDiH09NwqpuX2jR2OrVrggLAABgOwRY5CWm qIsQAAAAaBJgHV0rMBpYhCVjot8mbmqpCAsAAGArBFi8SP8gAAAAsAwBFm1PTcKCexRhAQAA MBUBFk8zvh0AAABY0hdLQOjOtPoI149w+vhpZXhB8fm+oTq+qggrRZksAABAvgRY3Jtm9a7M ijGqDCudL5YCAACAkbQQctuKFTStGyPC7Fe7SVgAAAB5U4F1IEVZdD63pwcPUUEDAAAArE0F 1v4VZVF9hRBSTPXXwweKrhhpc5OwvGUAAAB5UoG1Z3VoNeog2yzCajYhGuYFAAAAmybA2qdJ oquwqQqaWh1d1bnVyxO1JF/TXI0byUDdjhAAACBbAqx9GtQhGFNRFg/33OK95FrB08s5VCv5 kmcN0bpatpiBAgAAkBszsHhs+Qzi5RsRTnv7wphi/eUyOMSlbhIWAABAlgRYDLWVOhphU24U YQEAADCSAItBqqYwMQT7v9QVYQEAAORHgMXgD/YbvBchAAAAsAOGuPOEzAe6TzsAq6Uay6U/ cZdXTvts3Y4QAGCVv8+XZYyx+mfPDi8c9o+/2MfY3NjzXK1H3TxOzwPrHVpbeo7TeoEPX+/D VzHhPvVuD49zc4fX3rvh1wZHIMBiV0mEgAkAAI6mP9doRUKt/e89tvuo7pb+M+lmLvX3Q1KY p+K8e7tNtU/zj3r26cZ8MCEthDzNTG7u2VCZ1ePX4j+9AAAbMbAq5+UCrtajZqoDakY/zahr qkhoqnNWA8VaBFiHlmIqyuKVB86fYVUte96j/Vxsm8o99Q8CACyvVZ1UNjR3a23p7nBTM3Ua HsGMT696CppuJlbjj7yYm+9O/z719913sLlPd32GV66xY1oIeV1W5TbSLqa/wk3CAgBYVbe1 rftNq0apZ+JVTxo18FGtLd1HNU+mJ/ka0o7X/rwzeHLWvQcO7/u7N/+r2Ur5QnNi82jN7sgh 7zIEARYvqwpqssqwDMBiyis8Rl2EAAC5eRhn9A+0uhe7DHxUa0v3UfeO3/9cQ2KalwfbD5lv 1YqZurvdnMY1VW9j9zjThlb/+89fO/sp+Pd//z7mj78AixGf8OdvCttTXdUkr2XTId1rdwBY NyRVhAUAsEtLlvYMea5mA92YDsclTfLUN6u0pnXYuGd/BFiMMvKmhDfzr/po+6uoGvmKyqIs i/JQhWbVFbJWhqUICwBgXQ/b3wYe5IU5VlPNbh9ynJulXk/dhXDMqxufgk2So7l9IQ8JsBj9 IX9cHVYrmMitLbH9W3XV/CjzwfaTv3HVdbX69HdFWAAAq/0FeMBfw+7Nt2oepBuEdQeEP3xU z3yrp55ryHHCna69m6+9eajWavScT/8+Q96ap458722t95nwfovs1clEtJ2517R8/8N5kWLa dw4y5fK+GmBNlXxVAVaeRVgD37jhu4VGvrniVSHAAgBY66ONj6szrZu1ZXyYsLx/eZPI0LoV Nzk7VP9gPjmmRkIAADbzkWFAACG9YosEWEeXYirKIq9TyrX8Kh9bn23/MKDs1lut2Eio/AoA YBVCFqBJgEWmFGHd/Q95iuH3QPfqa1vn/zCgzLOHVBEWAADAigRY5EgRVr+YYv21s5fWE1wq wgIAADgsARYhhFCURfWV11ntpQhr3XsXZuheFNUa3J7bRaUICwAAYC0CLEKKqfrK66wUYW3K VH1//QcZWYQ15rGKsAAAAFYkwKL5ET0pwtqc68fb1me61+/1ArVXYyMwRVgAAABrEGCRrwyL sGKKWaVF2w34WkHS8PTqtQRqkuZERVgAAABrEWCRu0wymmyjonS+XD/ejvYWv/CQqfJQRVgA AADLE2DR/XyeURdhJkVYY9Kr+Sa4z91zt+TaPvVCnn3VrYUa00WoCAsAAGAVXywBf34+z24M Vlg7qamfPavAaKqTmaoj8hqergKrg6QXXkj12CEPnONdK8pSkgUAALAkARa5Gzl1e6Sb8Ucd +nRLq5aZkNVakNPHz+vH69Vq4wvExrzquQe3T378FKMuQgAAgIVpIWQbVsmwbraexRTrr1Zw U3ULdr/mOLesmgevH2+vnc+YV/Ew2ez50/GpqAwLAABgSQIsOp/t8+siXCWsGRJwNDOsxe5O OGFV0YTzubK6OePcV47+QQAAgIUJsLitKIvmV88Owx8y9pQWLMLqmc3UOo1mhjVTsdXDRYgp nj5+ZnufxPn0FFItMLBMERYAAMBizMDiVi4QU+ezenFvn/qPmo+aPMBachJWT3p18zSqDGuB 9Ko+B5fow3fw4SoNHwN/52fEJCwAAIDlqMBi4Mf11Prq/tECp7FAhvXaffGWSa+yug1iLldm J1JccpVkWAAAAMsQYDGLOQZpLZBKDEmv1ror4sMnXfd2jSGPMVhLroBJWAAAAIsRYLExcycU OZc4TXtu07Y9LtZBeXNZqqvi2eq5SSI/RVgAAAALEGCxJbOmS8Nbz5avdcq5eTCTc3ut93Ps BakICwAAYBECLLZnjvAo84Ro4J6rdxGupXrvXnsHFWEBAADkT4DFXOYYgxXmKbHJfzi62e0z LdH4hVWEBQAAsIAvloAtejlyullu88KhqlqnZebKD3mWmOK0M61ekMM5rHZBlqUkCwAAYD4C LLanCo9e7vzaUDXTCxnZwGRt1qSpuh3hhpKskVlkilEXIQAAwKwEWGxSDiHUYkVY21JFY1WG tZVraZLBYYqwAAD29jfbsowxVv/s2eGFwzb/tTpCvbHnuVqPunmcngfWO7S29Byn9QIfvt6H r+LlfR6+FxyBAGvUr7ObP+fdH6qptmxONQYrxTTgw39R7e+6+rUgU9xTb/kU6Zj9g0ERFgAA gz/KtSKh1v73Htt9VHdL/5l0M6D6+yEfPJ+K8+7tNtU+HJMAa8Rn9Vu/L+79Uhi/Za+a0dXm YqzJi7DqUqAXjlmPoKrO6vTxc4E4aR81aJO8j1WGpQgLAICBH+JeLuBapvShGYo1o66pnk4y xbMEWNP8YNe1l62f6qm2bHp9emKpVn3WTDcuzH19Gv1rIzOU5hj1wxZDAQDAJFrVSc1Ap6ce qlse1f8pMjwT5YxPr/o7DbsVFeOPPMd7wTEJsJhXXVpVJ1MbLbaa3Jhiq77f7CmWRXkNbyv/ x36ROxLWPZL1E3W3LHu1K8ICANizbmvbvWqGf/7Ken/iVU8aNfBRrS3dRzVPpif56r6Kx38V Hzw5694D7+3zbHDGcQiwRn+EzrJCqv8HfvkTbgZV/dHV8LFZmXih+2yS+VZ972+K6bz55r4h A7x6cqtn74RoJD8AAK/83fvRZ6v+gVb35j0NfFRrS/dR947f/1xDPuS+PNh+yHyrm1VvLyv+ 397isPR/L36i33oyKMBi8R+2Y09ql5I88beBwfFTd8+17oSoCAsAgBcsWRgx5LlaU54zfBVP /BX9//zlfCcEWJv5LfPcJ3+fn5f8hTiseGfuwivaPwWLtDECAHCoD4DjP3C9NsdqqtntQ45z s9TrqbsQjnl17jy4VlCQf32WAGviq2Ha2e1+dHdj+fRKT9xaK6YICwDgmJ//mylAf39cNwi7 NyG+51E9862eeq4hxwnD7jDWfRWt1eg5n/59IAiw5viN1vqpnmoLmevJPvYaJM33ulROAQCw iU9/3e+HfIi7V/H01KMmfK6Hs6iGPF1P7nZzoZ7aB0II/7IEU/3CeurXymtbyFZPjqMMas2f 0FUnYVl/AAB4IWWDmwRYZKe6EeE+Xsu66VXVE+eKWmvFZFgAAHsiZIF1CbBgGq3so/h8V3t1 6OvB328AAACmI8CC6bnhYGWV9r2WtboIf10JirAAAACmIMCCyVRFWFXhVSbp1YpdhBudwj7h iinCAgAAmIoACyam8Ip/LgbT3AEAAKYgwIIpHSe9mm/CV1mU01Zv1V2EZVHm0NUIAADAswRY 5GhPNyJcfzHdizCEMEMuNvhiVoQFAAAwlgAL2L+Y4vD0SuQHAACQGwEW7J9EZuX1V4QFAAAw jgALAAAAgKwJsOAQFGFV6oHuS6+/IiwAAIARBFjA0+a7BSEAAAB0CbDIlBsRTr+kixdhvVDu tNa9Ahe5pBVhAQAAvEiABceikXCI+cI+GRYAAMALBFhkrSiL6stSTGL1vr+yKKuvg65/jC5C AACAF3yxBGT8aT9V3wiwplzV82X5CVZ1YlW1B9YZVv2ve13t5susFWUpyQIAAHiKAIttKMqi zrPYlu5Yq5sx1oZezvCdmwld9b1JWAAAAC8QYLEBBrpPvJ4LFmH1hFPbHdb+7Jm3C82u4XpS hAUAAPAEM7DgoF4eUr58B+IqJp/jHlOsvkIIp6sLEAAA4AkCLDiiIyRQc5hkXFeVYWkkBAAA GE6AxWboIpzW5BVGWxFTHJNDTdX5qAgLAABgOAEW22CCO3uiCAsAAOAphrjDcS05zZ2uk2nu AAArKcsyxlj9c2evq/qm9boevtLmDuWf/5+1WqjWloHP3trSc5z6BG7us9f3i6cIsNi/uvdQ Gdft9ZFhrWFkJyMAALS0QqjhWU9/rnRzy5Bnv3k+D89qyHNxTAIstqQoi1YINWQwVv2Qamcx 1h+Lc9RJWMMXZ750L6Z4upaKsAAAmP6vmoNjoDpsuvdH/VsmMfDZOTgBFttJE2JqxlUvpFGt I/BrWZ6JaY5crjVTwdTpGmSqAAAL61YDDWl/u1lh1KpRGthG1y1Q6n9UuN8e+OAvsffzqf6j jU+v+jsNu4vZ81wDq7fYNwEWGzO+H7BbxgVD/xuc4uQHLIuyKBVhAQCsaWD7W88D7z3qXhtd 608fPurmlj/+YvnnAKmHY6qG7NaN2HomVTVPqSf5aq3YwGeHIMBiW8YHT4qwbi+Lae7rqRoJ ZaoAAGv+laxRSPVakVH3Ufdmot/0wj43z7Mn9up51MPjdLd0j3Mv8mt+3/+onmcfY38jUw77 wU2ABRzod32+S60ICwBgVQ/LgkYeuUfPHQCfOs69g3efZTFDnnTuM/T/6XdDgAX8+rV+M646 +K/7ZWrTrh9vIfx0EQIArKXb3Ne/81PH7Nly78j9U6uGBz0v35RwktntQ47zVGshByfA4oiM wbrpOFlVNXlq8oFWo3x7U4QFALDa3w8785t6tvSkS0OOc/P77pFbNVk9E6Z6nn2IZgFUuDPN 6uVnby5U/5yska+CIzi5IHamdUMHbhJgMTzAqiqwqlsQzpF5FZ/vp4+f14+38P1nCEGGBQDA wBHycKgwQQUWwCDzVWzFFE8f5TX8yrAAADj63zxVIUGHAAsgVAVWoZFS1Vuu4W2x00gxaiQE ACDIraBDgMVB6SKkjqjCrdyqUW/1Xny+h4/Zz+feHH0AAAAEWBxRiqkoC+twZPf6Abvbq1xp pv7BG7c4NM0dAACg41+WgMManmEVZSHwYgHHuREkAADAU1RgcVBV/+DNWKpuLaz/VMUWy12Z 50sRgiIsAACAJgEWh9Ydg9UstjIkixXJsAAAAGoCLPiD0IrF3BiAVV2E50sRQvj+0xIBAABU zMACeGCB+wPGFJuL9oiaAAAgAElEQVR3RQzh1zR3iw8AABAEWDCcMVgsxjR3AACAJi2EMIg5 7ix9yZnmDgAA8JsKLIAV3BuABQAAQJcKLIBMKcICAACoqMACGGTuOe53meYOAAAcngAL4LG1 2v20GQIAAAQthADLe2oAlkZCAAAAFVgAWYgploVWQQAAgBtUYAHkThEWAABwcCqwAAAAAMia AAueUJSFRTisdL5MciPCpwZgNZ/d7QgBAIDD0kIIQ6WYBFjMrW8M1ke4hjeNhAAAwAEJsABy EdPDZOrdKgEAAAckwAJYzmv9gzXT3AEAgGMyAwtga769WQMAAOBQBFgAQ001x33kOYT+UVkA AAC7I8AC2Jh0vlw/3JEQAAA4EAEWwEJGDsBqOV2tKAAAcBQCLIDtUYQFAAAcigALYLNMcwcA AI5BgAXPKcrCIpAD09wBAIDjEGDBE1JMFuHo18CrNyKcdgBWfTLXD0VYAADA/gmwALbttUAN AABgQwRYABtWVXWZ5g4AAOzbF0uwY2VjWlPU+zapahKWjkJykM4Xg9kAAIB9E2DtUxVd1aFV ae74pIqySDGZ5s4T18wMA7DaP/VFGVO01AAAwC5pIdynGJOSq5mkmKrCKxnWca+BV+e4z3pK 1483dyQEAAD2SoAFsBMyLAAAYK8EWIcQY9JFOAdFWGR0Nc7coggAALAiARbAvBYYgFXRSAgA AOyVAAtgV66nIMMCAAB2xl0IYZSqizAZmU8mF+T5UoQQwk9LAQAA7IkKLICnZXgjwua5aSQE AAB2RoAFY3VHuRdlUX1ZHBYbgAUAALBjWghhGs24qu4oHJ5haULc5JuedRFWOH2UMUVvEwAA sAMCrKOIMZVlEaUk87gXPw2MpapyLRnWxt707OuqrqdwusqwAACAPRBgwfq6TYgw9qIyzR0A ANgRM7AgFzIspmWaOwAAsBsCLMiC/kEAAAC4RwshZGTgJKxWrZbwi3tMcwcAAPZBgAW5GDgJ qxVyaTzkIdPcAQCArdNCCHnpD6S6JVoGwNMvnS/h25t1AAAANk2AdSAxplLSkbcqnCrKovpq /enABkNoX1emuQMAABunhRDy0moPrP9VegUAAMBhqcCCfNXtgQ/TK12EPLiWFGEBAABbJsCC rFUZVn96pTKLga6nIMMCAAC2SAsh5E4+xTQX0vlShBDCT0sBAABsjgosgKPQSAgAAGyUAAv2 oJ6WBQAAAPsjwDqWGFMp5tivoiyqL0vBPYqwAACALRJgwU6kmKqv4KaEPGKaOwAAsC2GuMPe aCfkwRVimjsAALA1KrCOqCyL6stS7JgMix4aCQEAgG1RgXU4MabqGwHWjinCAgAAYE9UYB2X ge67J8OihyIsAABgQ1RgwT5VRVhjMqz0u1iP3V4k58v1I5w+ypii1QAAAHImwILdGplAVeGX GAsAAIDVaSFkAloRd0l0dYh3WSMhAACwBSqwDq0agxVH5BRVdFWP06oONTDPao2Tj+KS/FR9 iJKs3bt+vKWzZQAAAPIlwOJ1zfCrGV0NjKLqnOuFxwJTSedL8fleFiZhAQAA+RJg8aKbpVtP xU+tnesYS4aVFUVYh3iXTXMHAADyZgYWrxAzwf5cP94sAgAAkCcVWEdXj8F62MHXnGw1X3o1 fiwXk+sWYRWNi6HeXvw5+0zR1sbeZY2EAABAxgRYhNCoqLoXYwmVuJdP1dvvJVxsRdVIGL6X KcqwAACAvAiw+COuat1G8DhzqZapL9uunnIqlVZ7800jIQAAkJ1T9H/a96UsyxDCJG9rK8Za 9lW8HpmVf9b+9B+n+xolWZOoKrBkW5t87z7fTx8/NRICAIAwISsqsLhru/FN88xvNkX2ZHPd xwZJ1vOqsVnWYZPvnTsSAgAA+RFgsSvd0q1WU2Rz40OtsiwxFsf6aTLQHQAAyIYAixxNfi/C kYeq79IIR/C7COunpQAAADLxL0sAQEs6X64fb2VRWgoAACAHKrDYgIENgLPeLXF8UVj34TcP aIo8+biewumqkRAAAFifAItMNbv2ho9gz1wzsapeRSvD6v/XzSnKwo0ItyudL0UIIWgkBAAA 1qeFkKzFmG7eKLD6KstiK6OpbkZR94K5exu3NYdLdLUDGgkBAIBMCLDI18Pio2aMtZVipbrw qn4JzS03b6FY/Wnrga0DAgAAwI4JsNi8m1VaMz3RyLSoeZ4PC7KaT1oXnbX+VHrF3BRhAQAA ORBgsRMbmhXVnUnfXz52r9NQesUyZFgAAMDqBFgwu2Y+VX1zr1XwodYM+LqJ0iKzxJUswwIA AFYiwIKljS8WuzkwK0+FcG0XqiIs6wAAAKxFgAVPyKHc6WYN10xGvlg3ItwTjYQAAMCKBFiw PYulV93A7makpYfxOK4njYQAAMAKvlgCeFYzr7k3Yb3+0/4B7dm+ouZp1983I63WNK4edReh gqytS+dLEUIIPy0FAACwMAEWPKeZRpVlUac5N+dS5V+aVJ9hdzx8vb3aUsdY1ZY6zOoGXk11 aGUY1j6k8+X6EU4fZUzRagAAAIsRYMHr6uzmXnyzTO3VyDqvh7dE7B6/P7S6KcVUlIUiLAAA AF4gwIKx4mZDmeEJ1IrxHLlRhAUAACzPEHdgAkPuz1gVYVmrHXBHQgAAYGECLGBlRVkItgAA AOghwIKDmvX2iPeqseoirCq0qr5eGIwl8FqdIiwAAGBJZmAB0xjSRVip4icD3beuGoaVzlYC AACYnQAL9mDkjQinPZNwvwIr3MmtXrhHYVEWp1tPzcLKwjR3AABgdloI4YhmSrvqYw6vxnpB HV3FmOqv0EnNSm2G89NICAAALEOABbtVlsV2Q5x7U66qV1QlVs19Wnmc9GoxMiwAAGABWgj3 qSyKmFL9/b3d6n3YgVYXYf19HeU0/3UT3XbdpsJfL+r3CK1uyNVcgeaCNPc0e2sO14+304de QgAAYC4CrH2KKdW51b2UqifYYqvv++/IppXj/HrHs4+u6sSq+qZ5j8IhJx+rSKssTo09WwPj 3b5wDul8KT7fr6dwusqwAACAWQiwdkt11UHf9/vT3DOvuupWVNVb7rUN1qPf643VbKxrCKEs UkzX37u5MGZ/+86XIoTr6acMCwAAmIMZWMfVrNJiV+9sNnnN+EFUVThVj2mvNzb3qTKs6qva rfq+2n5yQSzp25s1AAAA5iDAAmbxco7WHH1V/llg1VVlVTef92bhVf/RGCOdLyGE6ykY6A4A AExOCyGQqXu9kE/p6alkclUjYQg/LQUAADAtFVhALprlUc3USdnUlt7E8+X68aYICwAAmJYA 69CMwSJDPa1/ze7CJ67zmEr517JkWAAAwLQEWABMqRqGFYJhWAAAwGQEWMBcXih9qgqsbj4q jZhjpQhrYVUjoXUAAACmYog7kKN4p5FwzAHnnuY+PCM7wlD5dL5cP8Lpo4wpup4BAICRBFgj P6+Wvz+OxubG5r9OuGWWmCClsihico829q/KsK5z/TYYmo4dqhbs+vEmwwIAAMbTQjjm82oZ f6uTrGpj/a8TboEtqjKjrCKbkfc0rF5O/VVvPEJR1dNL/XsYFgAAwEgCrKk+pcfQqJmqs6ep tsytLKo7vBkSxBw/HSnDGGv8K2q+rqfSq0MN5KqGYZnmDgAAjKSFkFD3DwqwmPEy+3M6eyvx 2W4Fk8KrIa4fb+F7maJGQgAA4EUCrFFuzsDK6sTufOS+fbbmYTG3OsaqkqybtUizVieNHIOl VfAF6XwpPt9P13A9ybAAACDToCB/AqyRH4ZjfR1EH8xg6A/OPzFWnQc1vxcS7cyvOxJef3pj AQCA1wiw9hoQvJ6mKcJiqas0Hfm1H62Y61eG5Y6EAACQZVCQf32WIe4TXw3Tzm5fvrBLdAXM 5/rxVri5KgAA8DwB1uuqmKlSx0zVxmbqNNUW2L3j3JvvmNL5EkI4XYMMCwAAeJYAa5T4W2tj d7dJtiz0olJyO0LW+GlarvovxVQ8H5btOF+rx+rPvvLny/Xj7XR1vQMAAM8xA4vpPgP/GXvp RuSVq2iRGOXlGxFOmLI1x2AtNg/r3vLevB3kfK4fb4ZhAQAATxFgcevT7O8irGYI1cynWuFU /87w3OU3c5SzcFjTb6oz6R6ntYzVDj1ru9hc+XS+FJ/vIYSiLJMuaQAAYBgBFnc+zaZUFkV1 R8J7+dTNP2o+XBEWz111y0ZLRVmkte8DWFdgTXWof35I/zxmVnc8/HVHwuvP60mGBQAADCLA 4v7n4ZRCCDdzqJ4/gk2ooqtqEtZaMdasoVJWidWN9f+dYUW/QgAAgAEMcefRx+D7EZX0iumv t2XzjJRHfDKy9Gy70+WvH29l4Y6EAADAYwIsZmQSFvkbeEfCxeasv2aSc1u4hTOdL79/Uciw AACABwRYzPaJWn0W0CudL9ePN+sAAAA8JMACDqdbatRfhLXdHr38VRmWIiwAAKCfAAs4utS5 f1+dWNXfR8PG5yTDAgAA+rkLITOKKVV3KmwOw+q2FrZGZek9ZC3NrKrOsA4VXVW1aUuP0j9f is/3EEJZlDFF1yEAANAlwGL+UKAoQiOW6k52byZW5r6z3JX5Z2Pg9c/gZiu51RztjcsXnaXz 5foRTh8/XZYAAMBNAizm1S2nUmBFFldmN50pi/5JWOlRoFOURXo19BlT+jRt0lQXoC0fY10/ 3k4firAAAIAbBFjkpe46tBQsrD97KsqiP58q9jXovdVHucT6ny/F53tMUSMhAADQZYg7wGMP 460UU4qpcL/CMYvcGIZlNQAAgCYBFsBQN/OpMZ2DtKTz5frxZh0AAIAWARbAIN2UqttXuLMi rGoy1/LPe/14U4QFAAA0CbAAnlDlU3V0tW7tVbnHjsV0voQQqmFYrjcAAKBiiDvZMcedbFUF Vv09g9U+EwZbN1Oq+uaAcY/di7+GYX0EA90BAICKAAvgCfOVXFX9es1AqoqubkZUm6u9unfC 3VdX7Xn9Fq4fb6fvP4L5YgAAgAALYHLPFmHd3Lknvaq3byLG6o/hmn/a/D6FS/H53g31AACA YxJgAcyiO8395hj47gP7o6um/JOd/vipzq1uvuRfjYThhwwLAAAQYJEjY7DYupvlV1VcVf1R 9/tKbmHNAjVQPQdP58s1fD19/1GUZYqGYQEAwHEJsAAW0o2ualUX4c7qjCZMvk7hei1PMiwA ADisf1kCgCWlmFrpVdIf17Nc58v121frAAAAByfAAiBrVYZ1CteiLK0GAAAckwALgA24fvt6 ugYZFgAAHJMAi0xVc9ytA2zUtKPf0/kSQginqwwLAACOyRB3APoscCPCIX7dkfB0DeHkTQEA gKNRgUXWyqKoviwFu1fdnZB+hmEBAMAxCbDIV0zV7drcoI392+6NCMuyaH7Nu0pVI2EIGgkB AOBotBCyAdU8LEkWrKgOp5q9hN3Wwpu7TehXI2EIp2u4nsoUo7cGAACOQIAFwAOt0CrcH4xV bZm1FCudL9ePcPr4GUIoShkWAAAcghZCNsMkLMhBjOnhWPcFJr5fP95OV+8GAAAchQCLjXxm 1j8IWf1Irjq06/T9R/idYRmGBQAARyDAAsiFGxEOVw10jynKsAAA4AgEWGxGNcrdOrBX270R 4Wordr4Un+8yLAAAOAJD3NmYmxlW3WDY/VO9h7BjvzKsczxdSwEgAADsmACLLbmXRtW5VXeH sihkWGxI3UWoIKvvV0FjinydYZVFGZM7EgIAwD5pIWQXn2ZTqr5u/pHGQ7Yi/b6Ug3lYTyo+ 30MIZaGREAAA9kmAxf5VGVb9ZUHIn/Kr55br90D3EAzDAgCAfRJgcQjNEi0ZFluhCGs4A90B AGDfBFgci3lYbIUirKdXrJFhAQAAOyPA4ogUYbEVirCeUmVYwTAsAADYHQEWh/NaEZbMi+Up wur7QY6p7KR71Zbrx1s4XTUSAgDAngiwOKinAinpFStShDX057QsYkzVQPfrt6+nIMMCAID9 EGBxRPU095v3JWxtL4vC5CzWUhVh1TfRtCDtn9ayqL/i74K1XzcljOkUTMMCAICdOMUYrcK+ Ps6VIQRv63OLVhQhhJhSHVo171dYp1eSLFZXlEWzr/BhpHXkJsTi8/30/UcIIerEBACA7YcJ X7xJcDOuCm5ZSN5aYdbNHY68Pul8uYavp+8/irJMMn0AANg4LYTwS0xJYkXmUkxVLDUknKp3 Pu5ynS+GYQEAwD4IsAA2yT0KB5JhAQDADgiwALakqquSXg1drsZNCa0GAABslwALhmpOeYcV DU+vXu4i3NN9D6sMK4SgCAsAALZLgAVwXN2Iqs6t9lTkZRgWAABsnQAL4ChacVXVitjcWG2p vnb22mVYAACwaQIsgD2rIqrqq/4+/M6qqn3u3dlwZ/cxlGEBAMB2fbEEALtXZ1XVN830qplS 7X42fDpfruHr6fs1BlPwAQBgS1RgwRPMcWeLurFUa8uh7mxY1WGVpR9kAADYEgEWAOEg6VVN hgUAANsiwALgWNL5EmRYAACwKQIseI4uQg5lZ3Pc/3ldVYYVTjIsAADYBAEWvEKGBVuXzpfw 7U2GBQAAmyDAgqfF5P5lsAdVhhVCkGEBAEDmBFjwCo2EsA/VTQmtAwAAZE6ABS+qMiwxFuyA ge4AAJA5ARa8ruollGGxb3ud4/7PC3RTQgAAyN4XSwBjdDOs/glZ99Iuc7VgRel8KT7fQwhl WcTohxEAALIjwIIJNOOnKqLqBlL3tgc1XJCBdL5cw9fT9x9FWaYYLQgAAGRFCyFMLKbUHfFe FkW1/d5DZFjkrOoiLPbeYVcNdD+Fq3ccAAByI8CCWTRHvFfplTVh01JMux+GVTEMCwAAMiTA grnUpVjSK3bDQHcAAGAVAiyY18D0ShchW3GQDCuEUJSltxsAADIhwIKNqTsTYS27H4lVD8OS YQEAQCbchRByca/fsBVXVTvoTGRdKaYQQpVhVd/3a6ZdQ/Zf/wX+uinh9Vqe3JQQAABWJ8CC rN0LqkzXIgd1jNWfSTV3qEq3NpFhhRCu32RYAACQBS2EkK/+VkFjs8hHTzthK67aSnQVGgPd T+HqLQYAgHUJsCAj3Uyqv8ZKhkUOmu2ETT3FVs/Oz1pr5FaVYcWY3JQQAADWpYUQslNnUjoE 2YrmrQnrb+4VW/Xcx/De9oG9irO8tPOl+HwP4UdZFjH6kQQAgHUIsCAvQis2qo6lBmZMrTRq yGOrp1glw7qGr6fvP4qyNAwLAABWcYr+Lr4vZVmGELytx3rTTXNng1rFVsNjqbVmwBef76fv P67BQHcAAPb4uTL7MEEFFgArSBtsx3NTQgAAWIsh7gDwWPOmhEVZWhAAAFiSAAuALemZAT/7 U58v9fcyLAAAWJIACzYvplTfuBCYVTpfqiIsSwEAAEsSYAHAE+oMqywFxwAAsBABFgA8p8qw QggyLAAAWIYAC4DtKdZOjqoMK8YkwwIAgAUIsGAPjMHiUFJMWZzG+VJ8vsuwAABgAQIsAHhd 8fke9BICAMDMBFiwH2VRVF+WApaRzpcQQjUPqyhLCwIAADMRYMFOxJSqL0sBS6oyrBjTKVxl WAAAMBMBFuyNeVgcQYqpyKZrrx6GdQpXbw0AAMxBgAUAYxnoDgAAsxJgwQ4pwuIginrwWwax kQwLAADmI8ACYJNSPfgt5jL6TYYFAAAzEWDBPinC4lDyGYlVZVghBBkWAABM6IslgB0bn2G5 rSE8K50v1/D19P1HUZYpRgsCAADjCbBgt8ZnT2q44GXXb19P36/X8iTDAgCA8bQQAnfpQ4TX pPMlVBlWuBZlaUEAAGAkARYAe/DCGKzmTQwnv6FhM8Py7gAAwEhaCIEHyqIwCYvNaeZQ3dsU Vn968/aFEw6Drwa6X799PZU/YvRDBAAAr1OBBfQRXbFFRVmkmOqvViZV/+nNx057Q8O6DstN CQEAYAwBFgC70h8/FYsHSVWGFWOSYQEAwMsEWMADRrmzFXXxVKu6qt7e0znYMm3OVfUSyrAA AOBlZmABgzybYek9ZC39+dSQ9GraLsJfx/yVYV3KsjAPCwAAniXAAh57No1SscVa+idbpdmS oyG1XTIsAAB4mQALgEN4Kr3qCbxuFmcNLNr6dV/C8PNUXmVYAAAwnAALmF41NksXIVt3L6sa c8x0vhQhXL+fZFgAADCcAAsAbng2qBrepSjDAgCAZ7kLIQCs4dtbCMF9CQEAYAgBFgAsLZ0v IYTrt68hhKIsLQgAAPQTYAHACqoMK8Z0ClcZFgAA9BNgAcA0Bt6L8J/9z5fi873KsKweAAD0 EGABs6huRGgdoF+dYRmGBQAAPQRYALAmGRYAADwkwAKAlcmwAACgnwALACbz7BisJhkWAADc I8ACgPVVNyWUYQEAwE0CLACYWFEW1ddTj6oyrBCCDAsAAFoEWMBc3IiQY0oxVV+vPPZ8KT7f q+9lWAAAUBNgAcAsXpuHVQ90DyEUZWkZAQAgCLCAuf1upVJLAkPVGdYpXGVYAAAQQvhiCYD5 xPSri0qAxTFVRVg32wn7i7NSrDKsy6m8XstTitFiAgBwZAIsYAnVPKw6z4KDq9KrnjlZRVmE 8KMIX2VYAAAQBFgAMKt7RVj9U97TrxlYP4rw9fothO/XQoYFAMCBmYEFAIsaPtk9xZTOlxBC +PbVPCwAAI5MgLV///vPX//7z1/WgdVVXYTWgQOqirCauVV/+VX74edLCOEqwwIA4MC0EO5T M7H693//FmABrKubYT338POl+Hy/fvt6+m4eFgAAR3SK/hK8L2VZhhBab2sVYP37v39bH1a+ Ps1x5/Du3ZRwyAPDt68hhNP3HyGEGP0oAQAw3Ye1W2FCVrQQHoLoikzoIoT0avCUYgrff6Tz 5frtawihLP0oAQBwIFoIgaWtm2EpAWPr0vlyDV/T+fJyMRcAAGyOAAtY1Lr5UVkU2hjZg+8/ ivA1RRkWAABHoYXwQIxyB9EVm9acBF+NdQ/VbCwAANg7AdZRGIMFNXO42LQUU1V1lc6Xaqx7 UZaWBQCAfRNgAceiCItNazUM/s6wrjIsAAD2TYAFHJEiLHZDhgUAwBEIsI7FGCwIv4uwqoHu kix2QIYFAMDuCbAOxBgsqMVUzRH6lWRZELZOhgUAwL4JsIBDMxKL3ZBhAQCwYwIsAEVY7IQM CwCAvRJgHY4xWNCiCIs9kWEBALBLAqxjMQYL7lGExW40MixXNQAAO/HFEhzQ//7zlyQLmmJK PXck7JZoDd8TVpHOlyJ8Dd9/FGWRossSAIDNO8UYrcKelGUZQnj4tlaNhGIsePwzdSuruhdU lUUhwyIfxed7Ol9kWAAAPP7gMyxMWJEWwlHvblPrXe9eB+O3TEh0BQPFlLpfloVNSOdL8fke QtBLCADA1gmwxn2ybai2lGUZY2zlWZNsmdy///u3ge4w8e+ElMzSIiu/52GFUoYFAMCWCbCm VKVOIYQ6e5pqC7AVMixyU2VYVxkWAABbJsAaq9U/uDmKsGDG3w/3B8M/tQ+MVGVYMSYZFgAA G+UuhOM+nf4ulaq/yefEev60PlVdhDCHugirmpbV/P6fH9LfoZWKLRby/UcRvsZ4KcsimukO AHA8W2/wEmCN+IzqBo7Avd8PjbiqGWPd28HtC5lbiqkofxTh6zX8CO5LCADA1giwdvrheXC4 VhVhuSkhzP5TKZ9ibSmmEC5F+JrOl0KGBQAgKGjIvz7LDKzXdd/dyWe3L9aZ+L///FV9eVsB 9i2dL8Xne4qpMA8LAIDtOOmDG6POsJrL2E2dptoy/JRee1uVYsFqv0y0ELKs4vNdHRYAAP98 JBkRJixDgOWa+4MMC1b74ZVhsSwZFgAA/3weyT7A0kIIAEeklxAAgA0RYAHAQcmwAADYCgEW bUa5AxyHDAsAgE0QYPEHA7BgLTGlspAgsAIZFgAA+RNgAcDRybAAAMicAAsgF1URljosVlFl WOHbVxkWAAAZEmDR9u///m0MFqwlpqSXkLWk8yWEIMMCACBDpxijVdiTsixDCCPf1kkCLOO0 YNTPclHElKwDyys+30MI4fuPFF2BAACH+QAyRZgwqy/eJLrGZ09quGCkqg5LhsXyqjqsInwt yh8hBDEWAAA50ELIXGRYMJJeQlaUzpfw7WsIQTshAAA5UIHFLAzSgklMlWGp5OIF6XwpQpVh aScEAGBlAiyArE2SPelG5DWNWxPKsAAAWJMWQgDgLrcmBAAgBwIsZqSLEDJhnBZjpPOlGokl wwIAYC0CLOYy/laGAORDhgUAwIrMwAI4hJ4irHvjsUzOoqUa616WP6J5WAAALEuABXAUPUFV 60/1G3LX9x/Xb19PMiwAAJalhZB51WOw/vefv4zEgjzFlKr6rCq36uZZUEsxVRlWqZcQAIAF nWKMVmFPyrIMIeTztjZDq3//9+///ecvs7Eg698hf7YN6iKk69cYrG9fT9/VYQEA7OWDQGZh QpcKLJbw7//+XeVWVYZVbVSTBRkSV/FQqkKr7z+uZroDALAUM7CYV7feqs6wmmEWABvyK8MK lyJ8LcofSR0WAAAzE2CxgjrV0lQIsGnVfQmL8kcIp2QoAQAAs9FCCEAfdySkXzpfwrevIVyL srQaAADMRIAFwF1GYjGEDAsAgLkJsACAsWRYAADMSoDFyoxyB9iHOsMq3ZoQAICpCbAAgGlU GdY1hKIsxFgAAExIgAVAn5iSOe4M97sOK1xDkGEBADCVL5YAAJhQOl+K8DWEcP3+41QWMboV AAAAY6nAYn3GYAHsTDpfQgh1O6EFAQBgJAEWufjff/6qviwF5KbqItRIyFPqDCuEkwwLAICR tBCShX//9+/qGwEW5CmmFEIoi6L6BoaoMqwihPD9VJTXpJcQAIBXqcAiL9oJIWdKsXhBOl/C tzd1WAAAjKECC4An1KVYUx2KI0jnSxFCCF+L8oc6LAAAXiDAAuBpk2RPGhIPJZ0vxed7+CbD AgDgFVoIySw17o0AACAASURBVI4uQjgO3YiHUo9110sIAMCzVGABsI56opY6rOP4PdZdHRYA AM9RgQXAakRXx5TOl6oOq1SKBQDAMKcYo1X4/+3dTW7byrYG0K2LMwlyNukk3XgUAowzBx/P IRDgAby20006mQ05DL0GLYbWD/VHSVXFtRDc6yiKE4tUqvydvXeVpK7riCjgsv7+99vXH79c UJjFP1yjjYRCrlJVb8/x+nMR60YpFgDAw/fkyYcJWggBeLDxiKqLt8RY5emOJlzH06J+jwgx FgAAI7QQApC0pm37aVndD69JOV7fI2L98hQR2gkBABghwCJRziIEhroYSx1WYdqm7TOsdYTT CQEAOEQLIUnrMyzzsIBOV40lySpG27QRH0cTxuvPql47nRAAgF0CLNLVh1ZKsQCK143EiteF DAsAgF1aCMmDDAugeO1yFS/f4+VJLyEAAFsEWGRA/yDATLTLVUR0GZax7gAA9ARYAGSmG4Pl dShVn2GtHU0IAMCGGVgAQFq6DKuKp/Xr+6KuGiOxAABmTwUW2TAGC2BW2uVKHRYAAB0BFnkw BgtghmRYAAB0tBACkJ9uDNZwGFbT6jIrU7tcVfG0jljU73oJAQBmSwUWs6D9EIrUpVdddGWs e8G6kVjrl6eqrqq69oIAAMyQAIucXJZD/f7329cfv2RYUJimbbsfofxqBtrlqmsnjFjLsAAA ZkiARTYuG4PVpVf9x15GKJgirOL1GZaRWAAAc2MGFpk5N5Dqn6wIC8o2nIdFwT5GYr2+L+rK SCwAgPlYNE3jVShJXdcRUfZl7XOoC2qyhvkXUNo/gJuRWLtJ1ok9hsOhWqSsenuOiMXr+zoW rZ0MAMD1e+nkwwQVWORHAgXs1UVX3emE2+vxsUhrJPwiQe1yVb09r1+e4nVd1TIsAIDyqcAq zRwqsK6kCAuIiGHOtZV57Y3ASFP19hyv76EOCwDgyu1x8mGCIe4AzFFfbCWuypqjCQEAZkIL IQAz1WVY0qvcdWPdI75Xda0OCwCgVCqwAJivvenV6ZOwupFb/Q+v56O0y1VExMv3unYVAADK pAKL2fn645cxWMDptpKpLvPqHxwO0vJaPdLre9u063ha1O8R0TQK6wAAiiLAAoCDdnsMd6Or kSdzN23TVnUV8b5+eYqIqN9bGRYAQEG0EALAtq6LcG8g1bRt92P3ca/bY31cmI92wqdKOyEA QEEEWACwn0wqUzIsAIDyaCFkjkbGYP3+91v/HC8UzJn0KmtdhlXFU6WXEACgCCqwmK8+q+o+ 7n6E6Aq4glHuSWmXq3h5qt6elWIBAORu0TSNV6Go753qOiJc1hON1Fs5qRC45B9hc9zTU709 R3wcU+jVAADYv49NPkzQQsisiagAiqedEACgAFoIAYDy9e2EXgoAgBwJsOCg4ZAsgBMZg5Ws j1Kst+eqrr0aAAB50UII+3UnFXodgLM0bdsFWN3/moeVmk07YVR1pZ0QACAjKrAAYGL9KHfV WGnSTggAkB0BFgBMqWnbLr1SfpWyQTuhkBEAIAMCLDhIFyFwPUVYyeoyrHh5kmEBAKTPDCwA uJV+JBZp2ozEeqrqdyOxAABStmiaxqtQkrquI8JlndDvf799/fHL6wBc+M/yadPcd3MuHYj3 9DEP61WMBQDMddeafJggwHLPcZwMC7jqX+YTirB246qt3yXPujUZFgAw6y2rAAv3XBlkWMCD /3nfnGzITVVvz/H6vohoxFgAwKx2m8mHCYa4A0AmuwrjtG6vXa7i5Wkdi9pkdwCAlBjiDifp TiRUhAU8yunz4I3TulK7XFUR69fFol6rwwIASIQACwDycGKGtRVXqdu6xOt7vDyt42lRv8uw AABSIMACgGxcUEvVxV6KsM7SNm1Vv8fL0/rlKWpj3QEAHs8MLDjD73+/eREA5qBt2na5ioh4 earrykgsAIDHEmDBqQzAApibLsNaG+sOAPBoAiwAKJ9JWBfb1GF97zIsMRYAwEMIsACgcAZg Xev1PeIjw1KKBQDwEAIsOMPXH7+MwQKYoXa5aperePneNo0MCwDg/pxCCGfrMyxTsQBmpV2u qrfntmnW9WJRrxunEwIA3IsAC84zDK1+//tNhgVkoWnbuqr0El5PhgUA8BBaCAFgLuqq6n54 Ka7RZVjx8r1pWmPdAQDuY9E0jVehqG9O6joiXNa7UYQFZLlYqMY6U1VX7U6xVfX2HBHx+nMR SrEAgMz3h8mHCSqw4CrGugPMVrtcRYRSLACAOzADC6ZnyjuQOCOxptJlWNXb8/olFq/vdV0p xQIAuAUBFlxrtwirz61OL84SdQGPsjsSqw+2TpmWJQWLTYy1jqeIWNTvMiwAgMmZgVXc9yFm YOXJLC3gAUtGVcW+BGqYWx3Np2ZSybV3Btaep709R8Ti9T0ixFgAQE47w+TDBBVYkISujEuG BdzToeBJUdXFlGIBANyIIe6QCvPggRx147SK/zLbpq1OntHeLlftcrV+eTLZHQBgKiqwICFH MywlWkC++pwr6wqvLsY6pZ2wXa6UYgEATMUMrOK+PajriFj/X7RfPq5s9aeO+PtTsqbNEEh0 9Tln3Puh2VuJG0ZXJ47E+viNb8+LVxkWAJBBmGAGFvfWfmm63Co20VX1p5ZhAXAjZ6VRmXYd DhOrrqPwxAzroxTrLeL1Z+s/HAIAXMQMrGK1X5ruh5eiJOZkAWUoYHJWl2GdOBirG+4eL9+r unb1AQAuoAJrLrqaLHkWAEy2tm7aCeOEqVhdhlVFVG/PH3kWAAAnMwOrNONtqzKsMmwVYZmK BWS5YOU5CWv/8nrmSKyIMBULAEhrb2YGFjC5YWKloxDIVAFdhJfpyq/W8bSo3yNCjAUAcAoz sGa2af7SVH/qfr47BTAVCyDLFXm5Wr88rV+e6rryagAAHKUCa3475s2hhP3HAPAQXRFWGV2E l6zISrEAAE6mAmumhjEWuVOEBWStrqr+xxxXZKVYAAAnEGDNl/IrAB6uadvhj1lPxXp5qutK jAUAsJcAa+4UYZVBERZQjBwzrLZpq+uCp3a5+ijFioUMCwBglxlYs9bNdPc6FKPPsIbHFAJk ZLZFWB/r8nJVbaZiGYkFADAkwCKqP7V2wgL0oZVSLCBrU2VYmc6Gb5er6u15/SLDAgD4RIA1 d4qwytO1EyrCAvI1SfaU7/mGw9MJZVgAAB0zsJBhAVCg3LsRu5FY1duzkVgAABGxaBq9Y0Wp 6zoiLrisGgkLowgLIDYj4XdLsUayrcvqtqq6am9QLVW9PUfE4lUpFgBw413TpWHC3Wgh5ENX hyXDAqAkXRq1G1cdSqlSK9oathNGhBgLAJgtARaDXbIMqyD9JKxzZ7pvDYNXxgUU4PSiqq7x MLXhWe1ytY6niDAVCwCYLQEWlOyCRsI+8Log/ALgRpRiAQAzZwZWaa5vW1WExZBZWsDsVtKL KrBuNANrzx+0mYoVYiwAYMItUPIzsJxCCADweQOX8PGFH6VYL08RUdeVMwoBgJnQQsjOzvhL U/2pR361+2DkOVvPJHeKsIBZ6cZgJb1Sb9oJI2Lx+l7XlVIsAKB4WghLc+uqvz63Gs+n9CGW RIAFzG4xTbuL8O+f+PYcEfH6cxHr0FEIAFyz/0m+hVAFFuc5MZZyoCEA3HxRXq4ioopYvy4i YlGvZVgAQKnMwAKOcBwhMDfpdxEOtctVvHyPl+/rWBiJBQCUSgUWN6QIqyTjGZYeQ6A8lzUS PsSwFEsdFgBQJAEWN9tMjw6DJy/j+ZT6LKA8XXTV1WFlFGNVEet4WtTv61i05pwCAAXRQghc S48hUKpcoqvexwGFL0+LWGsnBABKogKLW26jjXIHgDsvvl2GFU/tclXXlVIsAKAMAixubryR ULxVhq4IyyQsgES0y1X19rx+icXrOuoQYwEAuRNgceMN9Gg+ZUgWANxqCf7IsJ7a5WpRr9e1 DAsAyJgZWDx0b23Qe0FMwgJIbp1drroYq2nabipWVVt2AYAsqcACptRnWNoJARLRtxN2pVhR R9O0XhYAIC8qsHj0rloRVkG+/vjV/fBSACWpq+yP8+smu3elWBFR15UzCgGAvAiwgOlpJwSK 0bSnFiu1TVslnAr17YTrl6c+xnJ9AYBcCLBIYEu9KcKq/tSqsQDghmvuoBSraVqlWABALszA IhXVn1o7IQDcWp9hRUTTrCKiritTsQCAxAmwSGMz/aXpP+iSLK9JAX7/+808LIBEV97PMVZX hyXGAgCSpYUQuAnRFUD6tgZj1XVV1UqhAYAUqcAivc20IiwAuOfKu6nGWr/E4nUdtVIsACA5 AiwAAD5irHU8RcSifl/Hom38xyQAIBVaCIFb+frj1+9/v3kdADLyEWO9PC1iraMQAEiHCiyS 3D3rIixIn2GZigWQxyo8LMXSUQgApEGABdxQH1opxQK4XlVX/cftjUOlrY7CcEYhAPBQWggn UH+urq93iu2nemRWuiKsiKj+1N0HZE07IcA062PTtncMkvqOwojQUQgAPJAAa2J1XTdNM8ye pnpknjQSAsDHmlhXfXTVNu2wGuum2uWqXa7WL0/9YCzXAgC4Py2E19pNnSKiy576/73+kXm+ tn10ZSQWAHmsXE07jJmK+tKcUQgAPJQKrKvMOV2Cc+kiBDJe8StlRxHOKAQAHkcFVqH77NEN pdANAE7XtO3DA6wqmca9rTMK17VSLAAoIShInwDrqmsvCbrfdlkXYbn6sqz+yEIA9iyFnzsT H9uu2MdYi9d11KGjEAC4NQHWVfr8MrUwS7JGmrouwq1ewj63Gm8w3HqatAvg4drlSikWAORi PChIvz5LgDXBtb/d7HZFXhTpUAI1kkn9/vfbsFDLLC0gX8NmwL5+qn8wuwHwSrEAgPtYyEeu txUz7aZOUz1y4l8myq3A0kVIr6vk8joAd1rrq6ppT4qWjg6r2g2t4liSNdIqmM6hh9Xbc0Qs Xh1QCAB57naSDxNUYE1g6wLvXu+pHiGOZVjVn79Fj/3Thg8OHweAyZ0eJ+195t54K48vXCkW AHBLAiyy2hx/aWInkNp9Qvec/mnDxGrk95IdRVhAyUtebr2EH3/tz1OxIkKMBQBMQoBFhpvj E0qoDj3HaYbFMAkLINFleliKFWG4OwAwCQEWAECu2qbtxmDtthw+toZLKRYAMC0BFnOkCKsM XRGWLkKAvaPcz5qidYu0qy/Fivi+eH0XYwEA1xBgMTtdF6HXoRgjjYSyLWAW69qB7On0TOqm A+Pb5ap6e16//O0oDDEWAHA+ARaQsfGISn0WQAoGpVhhMBYAcBkBFrPcSW9Gue+eVLj37EIA 4NrF93OMpRQLADjLorFvKEtd1xHhsh7VBVW7uVUMegxlWAVQhAVMs7xWVUQ0bVvkV7d3hNZt /8S354hYvL6vQ4wFAMKEkwiw3HMc2Fsb9F4KGRYwzQpbVaUGWPGIDCs+x1gyLAB48FYn+TBB CyEc3ljLsADgZobHFNZ1JcYCAEYIsODArtphhaX4+uPX8KRC1VgAaS24w2MK6xBjAQB7CbBg jCKsMvSh1TDJAiARu/PdZVgAwBYBFhzeT39pqj+1DKskXTWWIiyAFJddHYUAwGH/8xLA2Gb6 S9PHWF4NALj5ytvFWC9Pi1jXdeUFAQA6KrDghM20UqyyKMICLlNXB/OUgg8ofMCyO+worN8j omm8vAAwdwIsOG0zbaZ7KbZmugOcaCSiGgm2uHzl/RxjybAAYOa0EMIZZFgAcE99R2FdVzoK AWDOVGDByXtoRVilMModYFdVV22SVU7DUqyof0aE4e4AMEMCLDhzf28SVin6RkJJFkDbtFXa 9U1djFVFxOvPqq5lWAAwNwIsOGf3/PlEQklWvvrQyjws4HpN29ZVZY77PRbi5aqL2aq6DqVY ADAnAiw4c+u8Ca2cS1gG7YQAmS3EfSmWGAsA5kSABZduoI3EAoBHrcJiLACYGacQwhW7ZxkW ADxwIV6uIiJevscmxgIASqUCC5g7XYQA+VKKBQAzIcCC6/bNXxqTsADgwcuxGAsASifAggnI sADg4f7GWK8/xVgAUBgBFly9Xf7SdCcS7v2l7oOjo7LkX4+lixCgnHV5uepKsboYS4YFAGUQ YMEUe+UD8VOfW43nU13+JcN6uN//frvyM4jAgIzXsqat6qpt2hK+Fh2FAFCcRWM5L0td1xHh smZHgFWALv+SYcF8l+Cqatq8059iAqy/X9Hbc3Q1WWIsABjfySQfJvzPRYIUdH2IXoesia4A kltel6t2uarenuPle0RUdd0lWQBAdrQQAkzGLC2ABA07Cg3GAoBMqcCCZLbXirAAeLSqrvof pzwzp3V2uYqIePneNo1SLADIjhlYpTEDK+9vG/7U4UTC/CnCgpkuwfnPwNpelUbzqW7oe/dB Fn///u9pMBYA5BgmCLDccyS22x4twpJtZcE0d5jpElxcgHXSspXw3Pfh360Ps7pHqrqKl6eI iNefm8etsAAIEwRYuOeYZCOuPisfXYZ1iGwLylyC5xpgRZJFWHuTte0Y6+05IuL1vdsVhxgL AGFCwkuhIe6QDUOyMjIeUekxBMpZmzaNhLn8bT/99GO4e1eKtY6Iqt7zNAAgBYa4Q2ZkWAXo Div0OgDlrE1ZTXPf0i5X7XIVL0/x8tQ3GGb9FQFAkbQQlkYLYfnfJPypdRGWwagsKHAVnmUX YXwOsFIoX7q4sfHzfPd1qMYCQJiQDAGWe47cvknYTMLqS7HkWVkTY0FRq/BcA6xP61QCY92v /DvsxFgLs7EAECY8nADLPUeG3xsMoisFWWUwFQsKWYUFWN06tZMfnduRd2UENkmINoixKiPe ARAmPJwh7pCfYWIlwwIguXVqZ7L7uXFSCocbtstV9fZcvT3HSzfifVHVdYixAOBBVGCVRgXW DAmwyqAIC0pYhVVgTbvAXRRjTd7D2JVixd+mQhkWACVuY1RgAbemCAuAMhe4pu0OBOwPB3zM X2O5iohNNdb3eH2v6jAYCwDuTIAFkISvP34Z6A6wZRhdPbyjMLpqrJeniIjXtRgLAO5JC2Fp tBDOliKsYoixIONVWAvhTNbcTVNhvL53O2oxFgDZb2OcQoh7jjttpjdHE8bnKe/kSIYFua7C AqxZrbyfY6zHFogBwLXbGAEW7jnuvZ/+U4cMK39mukOuC7EMa27LrhgLgDL2MMmHCf9zkaAw oqtidHVYACS97C5X3XiseHnqp857WQBgcgIsKNOwo5AcKb8CyEgXY21GvC/EWAAwOQEWlLiN VoRVCkVYADmtv1011sv37qRCMRYATOgfLwGUyrmEufv645cAC3JUV2OZhQlZxes6Cqt4ioh4 fe8yLLOxAOBKhrgXt2k2xJ2NrWnupw93d6BhOkaOI9z7S33gpQMR0l2pB/HWbph1KPwSe+W6 Fncj3l9/RqxDjAWAMOEKAiz3HKVvnTdpVBdFbZVl7aZawyc40DAFh4qwuoiqj7G2oqut3yXP ghSX7H1Z1d6gqnumDCvjtfhTjLVo7dMAECacT4DlnmN+2+g/dful6cOpYcK123WoDzELI4Va /RNkWJD3+l5VAqy8F98uw4qI1/eIRUSIsQAQJpxFgOWeY5bb6H1BVewrtlKEVQwZFuS9vivC KmP9/Rtj/ez+X4wFQCqbDQEW7jmy320rwirF+Eh48RakvsQrwipmYa2r7pjCeH3vNuRiLAAe v9NIPkxwCiFwwlZbhlWE8YhKiRZksLNMNcNSIHaWtmkjhicVrqu6fxwA2E8FVnFbWxVY3MDw XML9e3HxVv6ODtICHr/Kj55gOO3nP2Trzx1GV2KsCxfZjxHv7x9LqhgLAGHCPiqwgOPG86mj 8RZZGB5lCKRpmA1NXo114ifcCrmGv6WPsWRY5y2yy0E1VkRVv4cYCwB2qMAqjQosHkKPYTG2 MiwFWZD0oj9dVDRt8VT/Fxsv6Rr542ZbzPX5sEIxFgDChL9UYAHw1zCxUpAFiWvadsIMa8K0 qPuLHf2cI/HWbHsSP6qx3p67Ke+qsQCgpwKrNCqweBRFWEUy2R0yWPpPGFx1VLIh0e6MreHf dveRsz5hnF8jds9M7XM1lpMKAZh7mKACCwAgY2UXKO0txRpJssZfk93exnNrxO455+tzNdb3 qq7EWADMmQALmGif/aVRhAXALWylRbvh0e7ZiIeKpw59hlP+3KN/xE2WVzEWAESEFsLyaCHk gYbHEUqyiqGLEMhvO7TT6Dd5698pAdahAq7L11lNhQDMOExQgQVMZhhaqcYC4FH6KfJbD976 j9j7tL/fGFw9rWynGuvjvxtJsgCYAwEWAACl2QqYbtHud8HnnKTxcBBjfe9+2iVZYiwAyqaF sDRaCEmHIqxi6CIEmGafNvXkrKqu4uUpomsqjIhom9brDMAli5QWQgAK8Pvfb90HkiyAdLRN G7GKiCqe4qMaq6s7MyELgNIIsICb7aq/NN1Yd3VYuetDqz7GAiCtNfdvX2EXY62rOsRYAJRE C2FptBCSIDFWSbQTAly+T5v6MMT9y+7nwwrDeCwATlmktBACdKVYRmIBMHOHzi4860DD/smH Htk6rDBe36tajAVA9lRglUYFFskSYBXj9CKsQy2HariA+W7Vdua4nzjZfSS36h/c/VSqsQAo JkxQgQXcjwxrVkZyri7YEmMBnG435Dol9lKNBUAx/uclAO5DdFWMrz9+HZ3mPl6l1f2SkfDA PA0rp04svzrRoRbFdrnqkqx4eYqX7xHrqq6runYtAMiICizgrnaLsLoR75/22TtR1ynP4c7G 46ej1VWnpGAA5TmUMd3BVjXW4r/39UI1FgDZMAOrNGZgkbjL4qqtp2lFLIYzDYE57tZ2Zlfd 7vPv1bRtPxtLjAVALmGCAMs9B1mSYZVBgAXwyMXUiHcANtIPE8zAAuBhNBICPNDWbKy2abrZ WMZjAZAgARaQq72dhuRIhgXwQIv/3rskq3p77ka8R6yruhJjAZDWgqXXrDBaCJkPXYTF6AKs vpdwK8/SYwhw293j55MQu77Cdrmq6m6W1kJfIcAsloPkwwSnEALwYF1E1edWw8RKcRbArXUH I/YZ1uCwwi7GWld1mJAFwMOpwCqNCixmRRHWHBj0DnDzDeTnIqy/6+ynKe8hxgIoeS1QgQVw U8NJWH2Y1T0o2wKAawyqsZ4iIl7XEdEVZImxALgzARaQ88b6c0TVh1nd4+qzAOAUW12E26tt d1JhRBVdjPUesRZjAXBnWghLo4UQhmRYZdBFCHDzPeThAGt7bd3pKxRjAZSwEGghBHig9ksj wwKAKdfWrYKs+F7VlRgLgFsTYAGl77O/NHvnZAEAvfEuwv0r7KckS4wFwG1pISyNFkIYYbh7 vnQRAtx8G1lV/cdnJVkfi+zf1sKfYiyA/FaB5MMEAZZ7DuZFR2Gmfv/7rf9YkgVw8y3lmdVY f9dZMRZApv/yC7Bwz0FSFGEVQDUWwD12lZdmWPF50HvbtF5MgAz+2TfEHSApWyOxAIC9uqlY F/7m/94jYv3fU7w8VW/deYXGYwFwFRVYpVGBBUcpwiqAIiyAbJbdQTWWGAsgWSqwAJLTFWH1 dViSLAC44bK7XEUXY730hxWGvkIAzqUCqzQqsOBcxrpnqh/rrhQLIJs1d1ONtfjvvfvg4jFb AEzLEHfcc5D8ZlpHYc7EWABZLr5vzyHGAkiJAAv3HOSwjVaElT9TsQDyW38/F2SJsQAeyAws gAz0RxPuxlgnHlko/0qBDAsgs/V3ueo+WP/31H0gyQLgEAEWQMQmgdobV50STulDfLivP371 7YQAZLYKf06yxFgA7NJCWBothPBAWhEf7lCGtbcy66wnA3C/9XTTWhivP1vbWoC7MAML9xzM bM8tw0rS3qzqUFClFREgiSW1ruLlKSLi9WdESLIAbkqAhXsO5rfhlmHlT4YFkMrmtqpMyAK4 x7+3AizcczBD46PfxVtZ2Cra6vOs7nHxFsADltfBqYViLIBpCbBwzwE7+28lWhnq86wuulKi BfCwZXTTWijGApiQAAv3HLCz83ZkYRFkWAAPXk83BVn9IYYAXCz9MOF/LhLAnYmuyvD1x6++ LOv3v98OnWkIwK3W0+Wqi66qt+fq7bmuKq8JQMFUYJVGBRbkYjgnS6SVr+FILDVZAA9bVY3H AriOFkLcc8AJ225TsYpgvjvAg9dTMRbApbQQAsBciK4AHqtvKlz/96SpEKAwKrBKowILMqUI qxjDYVgiLYCHLayDaqz1YtHaHgOM0kKIew44bZ8twCqRqVgAD15exVgApxFg4Z4DTt5kG+te HFOxAFJZZN+eI2Lx33v/iCFZAEPphwn/uEgAiRiGVgqyyvD1x69hRyEAD1tkl6uqrtbx1P10 8d97NyFLjAWQCxVYpVGBBcWQYRVjkgxLGRfAlIvsoLUwxFgAWghxzwFXba9lWGwYpwVwk6X2 c2uhJAuYLS2EAFyu/dJ0g7F2Y6zhwKy9v9GrBwDHl9rlKiLW//1tLQwxFkCSVGCVRgUWFGk3 rhqPqLaeL88qw94irK3+RFVaAFctuIOCLKcWArOiAguACZybQG09XytiYYah1VZipdMQ4KoF d1iQ9fK9rqr1YhERkiyAh1OBVRoVWMCuQ32IZKePrg6lVN0TZFgA0yygZr0Ds2GIO+45II0t uCKs2VCEBTD9MjpoLRRjAUUSYOGeA5LZfB+Y+y7YKo/BWAA3WUk3BVnx+jP0FQJlMQMLgFQc Cqo0GJbHYCyAm6yky9XH0tn9b12HGAvgXlRglUYFFnAZPYYFMxgLYPpdd1V9zHqPWPz3rq8Q yP6fteTDhP+5SAB0DvUYkjvRFcDkmrZtl6v+1MLq7bmuqrqqvDIAN6ICqzQqsICLKcIq23Aw lkgLYPpldDDoPcx6B3JjiDvuOSCrzffnIix5VqlMxQK41Uq6GfTeJ1khzAJyYIg7ADnZSqzU ZAHAucV5KQAAGctJREFUeSvpZtD7x4Ss1/fFOrrWQjEWwDVUYJVGBRYwIQcUFkwRFsA9VtJN QVa8/lys1yHGAlKlAguAjLVfGpPdAeDylXRTkFVtarK61sL1YtH6T84A5xBgATC68/6cYanG KsbXH7+6se67dVjGvQNMv55+bi3sqrHEWACn00JYGi2EwE2ZilWeYVzV6UOr3V/aJeECuGQ9 Hcx6F2MBKXAKIe45oKwNtwCLz8zSArhqYX17DkcWAgkwAwuAonQdhTIsAJhmYV2uoj+yMMx6 Bzjof14CAOBi/SwtAC7WLlcfQ7Jevq//e1ovFnVV1VXllQHoqcAC4MxN9kRHEyrjAoBPK+PW kYWbaqxQkAVgBlZ5zMACctGlYGKsMhwtwjInC+DshXIw6L1/UJIF3Igh7rjnAA5vzWVYs2HW O8Dly+UmyVKTBdyOIe4AcFAXXW3FWFv9ieKtYsiwAC5cLrdaCyNiUJYlyQJmQoAFwKP35YMY K3YSK4celsGsd4AJVsxNkhU7SZYYCyieAAuANDblUqrSbWVYqrEArlo3N2FWn2TF68+IaM0S AQplBlZpzMACyqMIqzxdkiXDAphyuexGZb3+7H4qyQLOYgYWAMA2HYUAk2uXq7qq1vFRkFVX VUSsFwtJFlAGFVilUYEFFMl5heVRhAVw26Vzc3ahIVnAKVRglX91e/1lrut665JP9QjAbO09 r5CsKcICuO3S+XlIllMLgdwJsC43TJf6MKtLnYbZ01SPANDHWDKsYvz+95siLIDbrp7LVeyc WhiSLCA3Wggn0MdMt8itzs2wtBACxVOHVZKtIixhFsDNl9HPrYUhyQIiQgvhTK6xtAjgntov TZdhUYCtxEpBFsDNl9HPrYUhyQIyIcC6SpdQJphhbc3n2iJxA7LffO9kWAqyytANxpJhAdxj MT2QZImxoFTjQUH6BFhXubjRD4Brt92fEyuDsYrRD3cXYwHcaUndl2StF4vWNzhASgRYZZKm AZCvLrraPaNQpAVwU5+SrJfvdVX9/f5CWRaUHhSkX59liPvl9o5af/gphIa4A7OlCKt4u5HW CGkXwFWral3Fy0dBVry+L9YRYiwoWvphggBrggu8dY13U6epHinjngO41VZbgMWGQVoAU66w nw8u1F0IRRJg4Z4DuOMO+6LTCcVeRZJhAUy/zr49h1MLoVDphwlmYAFQjsuiqGHsJcwCgIPr 7HIVO6cWdoRZwK0JsACY/XZ8E1pVf+o+zJJkAcD+dXPfqYX9xHdJFnAjWghLo4UQYBKSrALo IgS406K5GZIVEYv/3teLRUSYkwV50UIIAFkalmXtPggA/F0fNzVZ0ZVlvXyPiGpz4JUkC5iE AAsARjflGgwB4PR1cxNmVZtH+iQrhFnAFbQQlkYLIcAdSLJyoYsQ4PGL5qbBsF2ulGVBstIP EwRY7jkArtiUazBM2+9/v438qmwL4K6LpiQLEibAwj0HMI9NuSQrN4qzAB62aEqyID2GuAPA LAxDKw2GADC2aPZzst6eu4nv8fqzS7LEWMAhKrBKowILIB3KshKnCAsglRXz7TkUZMFDqcAC gPlygiEAnLRiLlcxKMiSZAG7BFgAcPt9uQZDADi6XO60Fg6TrBBmwbwJsADgvrvzQVnW7oMA wFaS9fenyrJgxgRYAPCg3bkGQwAYXyv71sLOJsxSlgUzJMACgEfvzjUYAsDIQrmpwIo+zNo0 GIayLJgNpxCWximEAGXQYHgfDiIEyHit7CuzzH2HqzmFEAC4hLIsADiyVm5VZr18//hYkgUl EmABQPIb9J2575IsAPi0Vg7DrP6Dug4xFpRCgAUA+ezOnWAIAEeXy/7Uwu5/357j9WdIsiBz AiwAyHBrrsEQAI4ul5IsKIgACwAy350rywKA8bXyQJIVwizIhwALAErZnSvLOt/XH78cRAgw o7Xyc5IVJr5DPgRYAFDiBt3cdwAYWSh3kyxlWZA2ARYAFL1B12AIACML5eHjC0OSBSkRYAHA PDboGgwBYHytPNxgGMIseDQBFgDMb4OuLAsARhbKnSSrXa6UZcFjCbAAYMYbdGVZADCyUPZJ 1ttzvHz/+FhZFjzCovF+K0td1xHhsgJwsRkmWQ4iBOCMhfLt+WOhVJZFQdIPE1RgAQCfaDAE gLGFcqcsS5IFd6ACqzQqsAC4hbLLsn7/+63/WCkWAGevkmqyyF/6YYIAyz0HAOfs0Usvy9JO CMDlq+QmyYrXnx9rpW/NyIQWQgCgKMXPff/645cMC4ALV8md4wsVZMFUVGCVRgUWAPdXXlmW pkIAplki+5qs+CjLkmSRJi2EuOcAmNlOvcQwS4YFwARLpDCLhGkhBADmpfgeQwC4cIncNBjG psfwI9J6/SnJgqNUYJVGBRYACcq9LEsRFgA3XCU/V2YJs3gIFVgAAMqyAODwKvm5MktZFuyl Aqs0KrAAyEVeZVnDse4XU8YFwKmrZF+WZVoWd2GIO+45ADhhmz6PsiytiACcvURKsrgLARbu OQA4Z5te3CGGW2RYAFy4RDrEkFsSYOGeA4BLd+qFhllHuxElXAAcWSLNfWdqAizccwAwxU59 TqPfVWkBcMYSKcxiCgIs3HMAMOk2vfQew44MC4BLVslBmDU83BCOEmDhngOAm23T/9RRbow1 7DQUZgFw9io5mP6uLIujBFi45wDgxhv0omOsjoIsAC5fKJVlcYL0w4R/XCQAIGtddFX9qWVY ALBnoRyEVsqyyJcACwAoYnf+pSm4FOvrj19Hzy4EgOPL5SbMqvowS5JFJrQQlkYLIQBzZioW AJy3dA5qsiJCmDVbZmDhngOAu+/F5zEVK8RYAEy4ekqy5k2AhXsOAB6xC5dhAcBla+jnJCuE WfMgwMI9BwCP24LPJsa6khQMgD3L6OD4QmVZxRNg4Z4DgIduvmeQYV3PEYcAHFlPhVmlE2Dh ngOABLbdf2oZ1jgZFgCnrqrCrBIJsHDPAUAau22lWMcc7UaUcAGwvbwOw6yQZ2VMgIV7DgBS 2mcrxbqCKi0AjqyzirOyJcDCPQcAie2tlWJdQYYFwKkLrjArKwIs3HMAkOSuWinWpYadhsIs AE5adnc6DYVZqRFg4Z4DgFQ300qxrqYgC4Cz19/PYVa7XHlNUiDAwj0HAAnvoWVYV5NhAXD5 QizMSoYAC/ccACS/exZjXUdTIQATLMefZ2bpMbwzARbuOQDIYdMsw5qIgiwAJliX+zDr9Wf3 //KsWxNg4Z4DgHy2y2KsKciwAJhsad45yjCEWbeRfpjwj4sEAPCxIf7SVH9qBxRe6euPX8Om wms+jxcTYO5L82AqVtV/UNd/nyDMmg0BFgDAYKP8pQmlWFebJHvqUjAxFgAfa/QmzKoGDwqz 5kMLYWm0EALAJGRYKZBhAXBkvdZjOBEzsHDPAUDO22LthAkYb0gUbwHwsWoLs64gwMI9BwCZ 74aVYqVNlRYAe5bvYZgVH3mWMGuEAAv3HAAUsQ9WipW2YZWWMAuAT4v4vjAr5FmfCbBwzwFA KdtfpViZ+P3vNxkWAPtXc2HWAQIs3HMAUNbGVylWDmRYABxf04VZAwIs3HMAUNx+VylWDjQV AnDe+j7vGfACLNxzAFDiHleGlRWD3gE4b6HfCbOKT7IEWLjnAKDc3a0YKx8yLAAuXO7nEWYJ sHDPAUDRm1oZVlaGfYUXk4IBzHfd78Os4noM0w8T/nH/AQBcrIuuxFi5mCR7UswFMN91f7nq Pqg2j1R1/ekJqkluRgVWaVRgAcBDyLDmZryYS7wFMJcNwKYmq8u28s2ztBDingOAOe1i/9Qy LEKVFsAM9wDDUVn78qzEwywBFu45AJjZ/lUpFhEhwwKY+X7gc54V8XdsViQZZgmwcM8BwCy3 rX8+/ourJGvmhp2GwiyAWe8N9kVa6SRZAizccwAw793qn8+zMORZM6YmC4BPm4RhpPXoMEuA hXsOABhsVeVZs6cmC4A9O4RHh1kCLMI9BwAc3K3Ks+bt97/fZFgAbG8PtpoN75JnCbC4NwEW AGS8YZVnzY8MC4CxvcG9wiwBFvcmwAKAcvas8qx5GDYVXkMQBlD4xmAnzIrpDjQUYHFvAiwA KHbbKs9ilGIugBntCvaFWXFFniXA4t4EWAAwl52rPIsdMiyAme4KPs+A/9gbnJMMCLC4NwEW AMx05yrPIiKONSSKtwDK3xLsC7PiWJ4lwOLeBFgAQMizOKCLt8RYAHPZDxzoNIydPCv9MOEf lxMAoDxbiZU8i04XXQ2rtIRZACXvB5arT/uB4cf1371Bm0MRjAALAGAG+1d5FgPD0EpNFsCM 9gODPOtQmJUsLYSl0UIIAJxFmIUMC2Dum4G358XrzzADi3sSYAEAV21h5Vlzpa8QYM7MwAIA ICcjzYbCrLLpKwQgZQIsAAAOGoZWirPmY3fW+5WfCgCuJMACAOAkJsHPzSTZ0+9/v8mwALie AAsAgEtoNuQUX3/8kmEBcD0BFgAAE9BsyCFdhnX0OV4oAEYIsAAAmJhmQ7YczaeMjQdg3CLl IxK5QPonXwIAcybMYsSwUEuYBXBP6YcJKrAAALgfxVmMGIZWarIAGBJgAQDwMON5Voi0ZqyL rtRkAdARYAEAkIrduEqkNXNqsgDomIFVGjOwAICy6Tqcs70ZVl+lJdsCuJgZWAAAMCVdh3O2 21cYg9xq6/FTPhUAuVCBVRoVWADAzIm0OErRFsCW9MMEAZZ7DgCgcLoO2etoxZZ4C5gPLYQA APBgug7Z62g+ZWw8QDoEWAAAzIuzDjnR+MgtAO5JgAUAwNwdjbTkWXO2e+ihDAvg/gRYAACw Tdchh3z98UuGBXB/AiwAADhCiRZDXYY1/KnXBODWBFgAAHA2JVozNwytzHoHuAMBFgAAXMtg +DnbO+v9mk8FwK5F01hHi1LXdUS4rAAASZFncQrFXMCjpB8mqMACAICbU6LFKU4s5pJwATMk wAIAgAcwGJ5DjuZTCrWAGRJgAQBAEgyG50S7hVrCLKB4AiwAAEiREi3GbZ2EKMMCyibAAgCA PCjR4pCvP37JsICyCbAAACBLBsMz1GVYW494WYBiCLAAAKAQug5nbiuxMusdKIkACwAAiqXr cM52Z71f+akAHkiABQAAc6HrcIYmyZ4UcwEPJ8ACAID50nXIKU4s5pJwAbcjwAIAAP7Sdcgh R/MphVrA7QiwAACAg3QdcrrdQi1hFjAVARYAAHAGkRbjhqGVmixgKgIsAADgKiItDlGTBUxF gAUAAExMpMWQmizgegIsAADg5kRadE480PD0TwXMhAALAAB4gFMirZBqFWqS7EkxF8zKomms B0Wp6zoiXFYAAMqgUIsRMiyYSvphggosAAAgXXoPGXFiQ6KECwogwAIAAHIi0mLL0XyqT7gk WZAvARYAAJA347QY1+dWkizIlwALAAAozd6sSqrFbpK1+0tAmgRYAADALJyeaoVgq3S7cZV5 8JA4ARYAADBfh4Iq5Vpzc+I8+NM/FTAtARYAAMA2TYjzNEn2pJgLbkGABQAAcBJNiJzixGIu CRecRYAFAABwOU2I7HU0n1KoBWcRYAEAAExPEyLjdgu1hFkwQoAFAABwJyemWiKt+RiGVmqy YIQA6yp1/bHSNE0zfHD40wkfAQAAyrMbV4m05mnv8Cx5FnQEWJcbBkz9x90Hu790/SMAAMBM iLTmbCux6vMsSRYzt5CPXOwWKdWhR876W8XnijAAAKA8xmnNzdFjDY8SgTEi/TBBBdblhEQA AMCjGKc1N9fHT2ZskTUB1gQSbPTrh3PtJXoDAIAi6T1kxN4ZW3ufQ5HGg4L0CbAmuAPkQQAA QJpEWmwZj6hUaZEsAdZVkk2vZGoAAMBep0RaIdWaq90qLWFWMcaDgvTrswxxv9ze9OrhpxAa 4g4AAFxPqkVHTdZMpB8mCLCuvbq9kXMDp3qkjHsOAADI1N5UKwRbM7A7OUukVRgBFu45AACg cMq1Zmgk0tKBmCMBFu45AABgjqRac9PnVnuTrEMkXIkQYOGeAwAA+KAJkS1mbCUi/TDBKYQA AADcyaGgSrnWbO2eezj+TGZLgAUAAMCD7c2qpFrzcUo4tRVyybPmRoAFAABAiqRaDG0lVnoP 50aABQAAQDZOT7VCsFW03d5DYVbZBFgAAADkzWit2RqGVmqyyibAAgAAoEyaEGfl9Hnwp382 0iHAAgAAYEY0IZZtquBJPVdqBFgAAADMnSZEtpxYzyXhuhsBFgAAAOynXGvmjuZTexMuqdYt LJrGG6wodV1HhMsKAABwZ4ItItuDEdMPE1RgAQAAwATO6kMMwVahdg9GDDVZUxBgAQAAwA2d G2yFbKsUfW517tmIAq9dAiwAAAB4gJGUStFWYc4NpA4FXnMOtgRYAAAAkBbdiDN3KKjqgq2t X8106ta5BFgAAACQB8HWzHX51G591m6vYnlJllMIS+MUQgAAADp7gy2p1hyMhFx7OYUQAAAA eIy9WZVyrTnYjavOjbRSI8ACAACAGTmrD1GqVYxDkVYuMZYACwAAADijXEuqVYZDE7XSZAZW aczAAgAA4KY0IZbHDCwAAACgKJoQuT8BFgAAADCBs2bGh2yLcwiwAAAAgFsZKdcaybZCvMVn AiwAAADg3sbzKfEWWwRYAAAAQFrEW2wRYAEAAAA5EW/NkAALAAAAKMc18ZZsK1kCLAAAAGAu jsZbF/wu7kCABQAAABAxembiub+FaQmwAAAAAMaMpFSKtu5DgAUAAABwIUVb9yHAAgAAAJiY oq1pCbAAAAAA7kfR1gUEWAAAAACPd0HR1tHfWAwBFgAAAEDSxiOqOZRuCbAAAAAAMjaH0i0B FgAAAECZLi7dSo0ACwAAAGCOBvFW6knW/1wtAAAAAFImwAIAAAAgaQIsAAAAAJImwAIAAAAg aQIsAAAAAJImwAIAAAAgaQIsAAAAAJImwAIAAAAgaQIsAAAAAJImwAIAAAAgaQIsAAAAAJIm wAIAAAAgaQIsAAAAAJImwAIAAAAgaQIsAAAAAJImwAIAAAAgaQIsAAAAAJImwAIAAAAgaQIs AAAAAJImwAIAAAAgaQIsAAAAAJImwILJ1HVd17XXwZXFlcWVxZXFlcWVdWVhWgIsAAAAAJIm wAIAAAAgaQIsAAAAAJImwAIAAAAgaQIsAAAAAJImwAIAAAAgaQIsAAAAAJImwAIAAAAgaYum abwKJanr2osAAAAAnCvljEgFFgAAAABJU4EFAAAAQNJUYAEAAACQNAEWAAAAAEkTYAEAAACQ NAEWAAAAAEkTYAEAAACQNAEWAAAAAEkTYAEAAACQNAEWAAAAAEkTYAEAAACQNAEWAAAAAEkT YAEAAACQNAEWAAAAAEkTYAEAAACQNAEWAAAAAEkTYAEAAACQNAEWAAAAAEkTYAEAAACQNAEW AAAAAEkTYAEAAACQNAEWAAAAAEkTYAEAAACQNAEWAAAAAEkTYAEAAACQNAEWAAAAAEkTYAEA AACQNAEWAAAAAEkTYAEAAACQtH+8BHCluq6bpjn3EfK6snVdD39p+Lgrm91l3bqI3rNFXlnv 2cIu69Hr6MrmfmW9Z8veHnvPlndlvWetsw8hwIIp3/P9e3sr+9h6hByv7O7lc2Wz3k8Pd2De s0VeWe/ZAmx9A2ydLfvKes+W/U+092zZb2RXtuzrmM6V1UII03zLtPuNU7ch232EHK/syHNc 2TKutffsDK81OV4+79n5LLiubNaX1Tpb/JX1nrU3fggBFlzOf1uY25Wt69p67D2L9yyP2kxT /JX1nvWGJa8r6z1bzMXN5ToKsADOWLb9NyXba3K5st6zxVzTOPaf/SnjynrPgr0xruM4M7AA jhN2lLdOex3KvrIucXn//Hrnln1lXVwrLHldWRfa9zgPoQILAHtrXFkApvnXWOGkKws3IsCC KY3MuvPNVe5r9tFrTRbXcfe/H3rPFnllvWfn82+vK1vGlfWeLWYn3NfTHZ0D7cpmfWW9Z62z D7Fwb8FUb/iRNgf/jhdwZfee8+3Kes/iPct9Lu74dXRlC7iy3rOFXV/v2eKvrPesdfb+BFgA AAAAJE0LIQAAAABJE2ABAAAAkDQBFgAAAABJE2ABAAAAkDQBFgAAAABJE2ABAAAAkDQBFgAA AABJE2ABAAAAkDQBFgAAAABJE2ABAAAAkDQBFgAAAABJE2ABAAAAkDQBFgAAAABJE2ABAAAA kDQBFgAAAABJE2ABAAAAkDQBFgAAAABJE2ABAAAAkDQBFgAAAABJE2ABAAAAkDQBFgAAAABJ E2ABAAAAkDQBFgAAAABJE2ABAAAAkDQBFgAAAABJE2ABAAAAkDQBFgAAAABJE2ABAAAAkDQB FgAAAABJE2ABAAAAkDQBFgAAAABJE2ABAAAAkLR/Dv1CXdfdB03TDH+698Hupxd/nv6ZI5/n FLt/n6OP7P59Lv5KAQAAALiF/QHWMEvqP97KbvY+54LPM5XdP+uUR7YStO6DC75SAAAAAG4k lRbCrbqnK12QMUmmAAAAANK0P8A6FOXUdT1J0rT1ee4WHp3+p0z1lQIAAABwpX/Gf3lv91zf fHc04hk+59DnmfCL2fqzRv6GW3/0oZ+e/pUCAAAAcCNjAdYw1tmbNO1ORh/5DLszp3afc6WR eVunF3xd9pUCAAAAcCMHZ2DdcyZU3693h3ho9+sy/QoAAAAgZfsDrL0pz+5zDv3ekT9v91eb jfhc/XT9FKq9f+ejWdVUXykAAAAAk1jsDXS2opl+INTwpxM+0j8+MpfqFEf/rENf16G07oKv AgAAAIBpLSbPXybpyMuirU/vIQAAAMAd/D9ZsXz678FLTAAAAABJRU5ErkJggg== --98e8jtXdkpgskNou-- From jos@xos037.xos.nl Fri Dec 3 15:26:08 2004 Received: with ECARTIS (v1.0.0; list netdev); Fri, 03 Dec 2004 15:26:14 -0800 (PST) Received: from simba.xos.nl (simba.xos.nl [212.26.207.226]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB3NQ6i3010934 for ; Fri, 3 Dec 2004 15:26:07 -0800 Received: from xos037.xos.nl (xos037.xos.nl [212.26.207.37]) by simba.xos.nl (8.12.8/8.12.8) with ESMTP id iB3NPdmA026687 for ; Sat, 4 Dec 2004 00:25:40 +0100 Received: from xos037.xos.nl (jos@localhost) by xos037.xos.nl (8.11.0/8.11.0) with ESMTP id iB3NPdG04693 for ; Sat, 4 Dec 2004 00:25:39 +0100 Message-Id: <200412032325.iB3NPdG04693@xos037.xos.nl> To: netdev@oss.sgi.com Subject: e1000 driver problem with Intel Pro/1000 MT adapter Date: Sat, 04 Dec 2004 00:25:39 +0100 From: Jos Vos X-archive-position: 12413 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jos@xos.nl Precedence: bulk X-list: netdev Hello, I have a problem with an Intel Pro/1000 MT 4-port card in a Supermicro (non-HT) Pentium 4 system using RHEL3 (2.4.21 kernel "the RH way") with the e1000 driver (I tried both the version supplied by RH and the newest 5.5.4 driver): (1) With a non-SMP kernel, the whole system freezes instantaneously when configuring one of in the four interfaces. (2) After disabling the USB controller, all interfaces can be configured, but only one of them actually works. Via the others, I can sent packets, but do not receive incoming packages (possible IRQ problem). (3) Using a SMP-kernel, two of the four interfaces can be configured and work correctly. For the other two ports, ifconfig gives the error "SIOCSIFFLAGS: Invalid argument". Stracing ifconfig says: socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 4 ... ioctl(4, SIOCSIFADDR, 0xbfffc210) = 0 ioctl(4, SIOCGIFFLAGS, 0xbfffc140) = 0 ioctl(4, SIOCSIFFLAGS, 0xbfffc140) = -1 EINVAL (Invalid argument) Stracing ifconfig for an interface that works shows similar lines, but then the second SIOCSIFFLAGS ioctl succeeds. Supermicro suggest to change the non-HT P4 CPU by a HT one, but I personally do not believe that this will change the situation of the SMP kernel as described above. Any suggestions how to solve this are welcome. I can provide more information if needed. Thanks, -- -- Jos Vos -- X/OS Experts in Open Systems BV | Phone: +31 20 6938364 -- Amsterdam, The Netherlands | Fax: +31 20 6948204 From romieu@fr.zoreil.com Fri Dec 3 17:02:39 2004 Received: with ECARTIS (v1.0.0; list netdev); Fri, 03 Dec 2004 17:02:46 -0800 (PST) Received: from fr.zoreil.com (electric-eye.fr.zoreil.com [213.41.134.224]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB412cqv017119 for ; Fri, 3 Dec 2004 17:02:39 -0800 Received: from electric-eye.fr.zoreil.com (localhost.localdomain [127.0.0.1]) by fr.zoreil.com (8.12.10/8.12.1) with ESMTP id iB40xlvr028575; Sat, 4 Dec 2004 01:59:47 +0100 Received: (from romieu@localhost) by electric-eye.fr.zoreil.com (8.12.10/8.12.10/Submit) id iB40xlj1028574; Sat, 4 Dec 2004 01:59:47 +0100 Date: Sat, 4 Dec 2004 01:59:46 +0100 From: Francois Romieu To: Jos Vos Cc: netdev@oss.sgi.com Subject: Re: e1000 driver problem with Intel Pro/1000 MT adapter Message-ID: <20041204005946.GA26654@electric-eye.fr.zoreil.com> References: <200412032325.iB3NPdG04693@xos037.xos.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200412032325.iB3NPdG04693@xos037.xos.nl> User-Agent: Mutt/1.4.1i X-Organisation: Land of Sunshine Inc. X-archive-position: 12414 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: romieu@fr.zoreil.com Precedence: bulk X-list: netdev Jos Vos : [misbehaving multi port e1000] > Any suggestions how to solve this are welcome. I can provide more > information if needed. The error code for the ioctl most probably comes through e1000_open -> e1000_up -> request_irq That + (2) + SMP suggests an irq routing issue due to a broken acpi table ("suggests"). I would compile a (monolithic) recent 2.6.x to see its log and give a look at http://acpi.sourceforge.net/dsdt/index.php (once the ticket is opened at http://bugzilla.redhat.com of course :o) ) -- Ueimor From jos@xos037.xos.nl Fri Dec 3 17:15:18 2004 Received: with ECARTIS (v1.0.0; list netdev); Fri, 03 Dec 2004 17:15:22 -0800 (PST) Received: from simba.xos.nl (simba.xos.nl [212.26.207.226]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB41FHeR017711 for ; Fri, 3 Dec 2004 17:15:18 -0800 Received: from xos037.xos.nl (xos037.xos.nl [212.26.207.37]) by simba.xos.nl (8.12.8/8.12.8) with ESMTP id iB41EimA027147; Sat, 4 Dec 2004 02:14:44 +0100 Received: (from jos@localhost) by xos037.xos.nl (8.11.0/8.11.0) id iB41EiN05069; Sat, 4 Dec 2004 02:14:44 +0100 Date: Sat, 4 Dec 2004 02:14:44 +0100 From: Jos Vos To: Francois Romieu Cc: netdev@oss.sgi.com Subject: Re: e1000 driver problem with Intel Pro/1000 MT adapter Message-ID: <20041204021444.A5058@xos037.xos.nl> References: <200412032325.iB3NPdG04693@xos037.xos.nl> <20041204005946.GA26654@electric-eye.fr.zoreil.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20041204005946.GA26654@electric-eye.fr.zoreil.com>; from romieu@fr.zoreil.com on Sat, Dec 04, 2004 at 01:59:46AM +0100 X-Organization: X/OS Experts in Open Systems BV, Amsterdam, The Netherlands X-archive-position: 12415 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jos@xos.nl Precedence: bulk X-list: netdev On Sat, Dec 04, 2004 at 01:59:46AM +0100, Francois Romieu wrote: > That + (2) + SMP suggests an irq routing issue due to a broken acpi > table ("suggests"). IIRC ACPI is disabled in the BIOS (not sure), but I'm using a RHEL3 2.4.21 kernel that does not support ACPI at all (due to the RH patches rebuilding the kernel with ACPI enbabled does not work anymore)! > I would compile a (monolithic) recent 2.6.x to see its log and > give a look at http://acpi.sourceforge.net/dsdt/index.php (once the > ticket is opened at http://bugzilla.redhat.com of course :o) ) Well, I could do that, but the idea was to use this in a RHEL3 system, so ... -- -- Jos Vos -- X/OS Experts in Open Systems BV | Phone: +31 20 6938364 -- Amsterdam, The Netherlands | Fax: +31 20 6948204 From shemminger@osdl.org Fri Dec 3 21:43:22 2004 Received: with ECARTIS (v1.0.0; list netdev); Fri, 03 Dec 2004 21:43:27 -0800 (PST) Received: from fire-1.osdl.org (fire.osdl.org [65.172.181.4]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB45hLrg027212 for ; Fri, 3 Dec 2004 21:43:21 -0800 Received: from [192.168.0.106] (063-170-215-071.dslnorthwest.net [63.170.215.71]) (authenticated bits=0) by fire-1.osdl.org (8.12.8/8.12.8) with ESMTP id iB45goQ3022359 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=NO); Fri, 3 Dec 2004 21:42:51 -0800 Message-ID: <41B14E57.5080803@osdl.org> Date: Fri, 03 Dec 2004 21:42:47 -0800 From: Stephen Hemminger User-Agent: Mozilla Thunderbird 0.8 (Windows/20040913) X-Accept-Language: en-us, en MIME-Version: 1.0 To: "David S. Miller" CC: michael.vittrup.larsen@ericsson.com, netdev@oss.sgi.com Subject: Re: [PATCH] tcp: efficient port randomisation (revised) References: <20041027092531.78fe438c@guest-251-240.pdx.osdl.net> <200411020854.44745.michael.vittrup.larsen@ericsson.com> <20041104100104.570e67cd@dxpl.pdx.osdl.net> <200411051103.59032.michael.vittrup.larsen@ericsson.com> <20041117153025.160eaa04@zqx3.pdx.osdl.net> <20041130214643.7b72300e.davem@davemloft.net> <20041201152446.3a0d5ce3@dxpl.pdx.osdl.net> <20041201204622.7b760400.davem@davemloft.net> <20041202134930.132d7bd8@dxpl.pdx.osdl.net> <20041202135252.04e64f51.davem@davemloft.net> In-Reply-To: <20041202135252.04e64f51.davem@davemloft.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-MIMEDefang-Filter: osdl$Revision: 1.96 $ X-Scanned-By: MIMEDefang 2.36 X-archive-position: 12416 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: shemminger@osdl.org Precedence: bulk X-list: netdev If I special case to handle loopback, and get rid of the portalloc lock, it comes out much better. These numbers are on the 800Mhz PIII SMP, on a fast box like the dual Opeteron's it makes no difference (always 30us). Before TCP connection latency mean 79.9 std 10.55 *Local* Communication latencies in microseconds - smaller is better ------------------------------------------------------------------- Host OS 2p/0K Pipe AF UDP RPC/ TCP RPC/ TCP ctxsw UNIX UDP TCP conn --------- ------------- ----- ----- ---- ----- ----- ----- ----- ---- stp2-001 Linux 2.6.10- 8.270 38.6 24.3 61.6 48.5 45.9 76.6 74.6 stp2-001 Linux 2.6.10- 8.170 43.5 24.5 58.0 54.8 45.6 63.4 74.7 stp2-001 Linux 2.6.10- 2.740 50.6 29.9 40.3 48.3 59.8 75.1 101. stp2-001 Linux 2.6.10- 8.140 46.6 29.7 57.6 48.8 45.5 72.0 74.4 stp2-001 Linux 2.6.10- 2.690 47.1 26.3 40.8 48.9 45.5 75.4 74.8 After TCP connection latency mean 73.8 std 0.55 *Local* Communication latencies in microseconds - smaller is better ------------------------------------------------------------------- Host OS 2p/0K Pipe AF UDP RPC/ TCP RPC/ TCP ctxsw UNIX UDP TCP conn --------- ------------- ----- ----- ---- ----- ----- ----- ----- ---- stp2-001 Linux 2.6.10- 8.260 38.1 25.7 63.3 48.1 66.6 75.4 74.9 stp2-001 Linux 2.6.10- 8.090 46.2 26.0 63.4 55.5 45.9 63.6 73.5 stp2-001 Linux 2.6.10- 8.210 39.0 21.2 63.1 55.4 58.8 63.8 73.5 stp2-001 Linux 2.6.10- 2.850 46.5 26.0 64.8 54.6 45.5 74.0 73.6 stp2-001 Linux 2.6.10- 8.200 42.9 21.5 64.9 55.6 62.4 64.1 73.5 From buytenh@wantstofly.org Sat Dec 4 02:36:49 2004 Received: with ECARTIS (v1.0.0; list netdev); Sat, 04 Dec 2004 02:36:55 -0800 (PST) Received: from xi.wantstofly.org (alephnull.demon.nl [212.238.201.82]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB4Aam16004964 for ; Sat, 4 Dec 2004 02:36:49 -0800 Received: by xi.wantstofly.org (Postfix, from userid 500) id 7FDF52B0F2; Sat, 4 Dec 2004 11:36:26 +0100 (MET) Date: Sat, 4 Dec 2004 11:36:26 +0100 From: Lennert Buytenhek To: Scott Feldman Cc: jamal , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: [E1000-devel] Transmission limit Message-ID: <20041204103626.GA17872@xi.wantstofly.org> References: <1101484740.24742.213.camel@mellia.lipar.polito.it> <41A76085.7000105@draigBrady.com> <1101499285.1079.45.camel@jzny.localdomain> <16811.8052.678955.795327@robur.slu.se> <1101821501.1043.43.camel@jzny.localdomain> <20041130134600.GA31515@xi.wantstofly.org> <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <20041203205706.GC9808@xi.wantstofly.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20041203205706.GC9808@xi.wantstofly.org> User-Agent: Mutt/1.4.1i X-archive-position: 12417 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: buytenh@wantstofly.org Precedence: bulk X-list: netdev On Fri, Dec 03, 2004 at 09:57:06PM +0100, Lennert Buytenhek wrote: > > My problem is I only have a P4 desktop system with a 82544 nic running > > at PCI 32/33Mhz, so I can't play with the big boys. But, attached is a > > rework of the Tx path to eliminate 1) Tx interrupts, and 2) Tx > > descriptor write-backs. For me, I see a nice jump in kpps, but I'd like > > others to try with their setups. We should be able to get to wire speed > > with 60-byte packets. > > Attached is a graph of my numbers with and without your patch for: > - An 82540 at PCI 32/33, idle 33MHz card on the same bus forcing it to 33MHz. > - An 82541 at PCI 32/66. > - An 82546 at PCI-X 64/100, NIC can do 133MHz but mobo only does 100MHz. When extrapolating these numbers to the 0-byte packet case (which then tells you the per-packet overhead), I get the following approximate numbers: case overhead phi-32-33-82540-2.6.9 1.86 us phi-32-66-82541-2.6.9 1.41 us phi-64-100-82546-2.6.9 1.45 us phi-32-33-82540-2.6.9-feldman 1.48 us phi-32-66-82541-2.6.9-feldman 1.13 us phi-64-100-82546-2.6.9-feldman 1.25 us Note that this figure doesn't differ all that much between the different bus widths/speeds. In any case, if I ever want to get more than ~880kpps on this hardware, there's no other way than to make this overhead go down. For saturating 1Gb/s with 60B packets on 64/100, the overhead can't be more than ~0.59 us per packet or you lose. --L From romieu@fr.zoreil.com Sat Dec 4 04:06:39 2004 Received: with ECARTIS (v1.0.0; list netdev); Sat, 04 Dec 2004 04:06:51 -0800 (PST) Received: from fr.zoreil.com (electric-eye.fr.zoreil.com [213.41.134.224]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB4C6cBn011544 for ; Sat, 4 Dec 2004 04:06:39 -0800 Received: from electric-eye.fr.zoreil.com (localhost.localdomain [127.0.0.1]) by fr.zoreil.com (8.12.10/8.12.1) with ESMTP id iB4C3Svr001481; Sat, 4 Dec 2004 13:03:29 +0100 Received: (from romieu@localhost) by electric-eye.fr.zoreil.com (8.12.10/8.12.10/Submit) id iB4C3SUq001480; Sat, 4 Dec 2004 13:03:28 +0100 Date: Sat, 4 Dec 2004 13:03:28 +0100 From: Francois Romieu To: Jos Vos Cc: netdev@oss.sgi.com Subject: Re: e1000 driver problem with Intel Pro/1000 MT adapter Message-ID: <20041204120328.GA1191@electric-eye.fr.zoreil.com> References: <200412032325.iB3NPdG04693@xos037.xos.nl> <20041204005946.GA26654@electric-eye.fr.zoreil.com> <20041204021444.A5058@xos037.xos.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20041204021444.A5058@xos037.xos.nl> User-Agent: Mutt/1.4.1i X-Organisation: Land of Sunshine Inc. X-archive-position: 12418 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: romieu@fr.zoreil.com Precedence: bulk X-list: netdev Jos Vos : [...] > Well, I could do that, but the idea was to use this in a RHEL3 > system, so ... 1 - Imho people at RedHat/wherever need more details (chipset, mobo, dmesg, lspci -vx, 2.4.21 savor ?). 2 - Testing recent 2.6.x will provide a different datapoint and a wider review: a few people use e1000 for interested things here but they do not necessarily have the resources to help with a specific vendor released kernel. The vendor can use the datapoint though. 3 - Comparing lspci and dmesg output for the different failure may give an hint. -- Ueimor From jos@xos037.xos.nl Sat Dec 4 09:20:48 2004 Received: with ECARTIS (v1.0.0; list netdev); Sat, 04 Dec 2004 09:20:55 -0800 (PST) Received: from simba.xos.nl (simba.xos.nl [212.26.207.226]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB4HKkrw025199 for ; Sat, 4 Dec 2004 09:20:47 -0800 Received: from xos037.xos.nl (xos037.xos.nl [212.26.207.37]) by simba.xos.nl (8.12.8/8.12.8) with ESMTP id iB4HKDmA030416; Sat, 4 Dec 2004 18:20:13 +0100 Received: (from jos@localhost) by xos037.xos.nl (8.11.0/8.11.0) id iB4HKDX08286; Sat, 4 Dec 2004 18:20:13 +0100 Date: Sat, 4 Dec 2004 18:20:13 +0100 From: Jos Vos To: Francois Romieu Cc: netdev@oss.sgi.com Subject: Re: e1000 driver problem with Intel Pro/1000 MT adapter Message-ID: <20041204182013.A8246@xos037.xos.nl> References: <200412032325.iB3NPdG04693@xos037.xos.nl> <20041204005946.GA26654@electric-eye.fr.zoreil.com> <20041204021444.A5058@xos037.xos.nl> <20041204120328.GA1191@electric-eye.fr.zoreil.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="MGYHOYXEY6WxJCY8" Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20041204120328.GA1191@electric-eye.fr.zoreil.com>; from romieu@fr.zoreil.com on Sat, Dec 04, 2004 at 01:03:28PM +0100 X-Organization: X/OS Experts in Open Systems BV, Amsterdam, The Netherlands X-archive-position: 12419 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jos@xos.nl Precedence: bulk X-list: netdev --MGYHOYXEY6WxJCY8 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Sat, Dec 04, 2004 at 01:03:28PM +0100, Francois Romieu wrote: > 1 - Imho people at RedHat/wherever need more details (chipset, mobo, > dmesg, lspci -vx, 2.4.21 savor ?). Mainboard: Supermicro P4SCT+ Output of lspci -vx: attached Red Hat kernel: 2.4.21-20.EL > 2 - Testing recent 2.6.x will provide a different datapoint and a wider > review: a few people use e1000 for interested things here but they > do not necessarily have the resources to help with a specific vendor > released kernel. The vendor can use the datapoint though. Yes, ok, I might install RHEL4 beta or Fedora Core 3 on it next week and see how that works. -- -- Jos Vos -- X/OS Experts in Open Systems BV | Phone: +31 20 6938364 -- Amsterdam, The Netherlands | Fax: +31 20 6948204 --MGYHOYXEY6WxJCY8 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="lspci.txt" 00:00.0 Host bridge: Intel Corp. 82875P Memory Controller Hub (rev 02) Subsystem: Super Micro Computer Inc: Unknown device 5180 Flags: bus master, fast devsel, latency 0 Memory at f0000000 (32-bit, prefetchable) [size=128M] Capabilities: [e4] #09 [3106] 00: 86 80 78 25 06 01 90 20 02 00 00 06 00 00 00 00 10: 08 00 00 f0 00 00 00 00 00 00 00 00 00 00 00 00 20: 00 00 00 00 00 00 00 00 00 00 00 00 d9 15 80 51 30: 00 00 00 00 e4 00 00 00 00 00 00 00 00 00 00 00 00:03.0 PCI bridge: Intel Corp. 82875P Processor to PCI to CSA Bridge (rev 02) (prog-if 00 [Normal decode]) Flags: bus master, 66Mhz, fast devsel, latency 32 Bus: primary=00, secondary=01, subordinate=01, sec-latency=0 I/O behind bridge: 00009000-00009fff Memory behind bridge: fb200000-fb2fffff 00: 86 80 7b 25 07 01 a0 00 02 00 04 06 00 20 01 00 10: 00 00 00 00 00 00 00 00 00 01 01 00 90 90 a0 22 20: 20 fb 20 fb f0 ff 00 00 00 00 00 00 00 00 00 00 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 04 00 00:1c.0 PCI bridge: Intel Corp. 6300ESB 64-bit PCI-X Bridge (rev 02) (prog-if 00 [Normal decode]) Flags: bus master, 66Mhz, fast devsel, latency 32 Bus: primary=00, secondary=02, subordinate=03, sec-latency=64 I/O behind bridge: 0000a000-0000afff Memory behind bridge: fb000000-fb1fffff Capabilities: [50] PCI-X non-bridge device. 00: 86 80 ae 25 07 01 30 80 02 00 04 06 10 20 01 00 10: 00 00 00 00 00 00 00 00 00 02 03 40 a0 a0 a0 22 20: 00 fb 10 fb f1 ff 01 00 00 00 00 00 00 00 00 00 30: 00 00 00 00 50 00 00 00 00 00 00 00 00 00 06 00 00:1d.0 USB Controller: Intel Corp. 6300ESB USB Universal Host Controller (rev 02) (prog-if 00 [UHCI]) Subsystem: Super Micro Computer Inc: Unknown device 5180 Flags: bus master, medium devsel, latency 0, IRQ 16 I/O ports at c400 [size=32] 00: 86 80 a9 25 05 00 80 02 02 00 03 0c 00 00 80 00 10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 20: 01 c4 00 00 00 00 00 00 00 00 00 00 d9 15 80 51 30: 00 00 00 00 00 00 00 00 00 00 00 00 0b 01 00 00 00:1d.1 USB Controller: Intel Corp. 6300ESB USB Universal Host Controller (rev 02) (prog-if 00 [UHCI]) Subsystem: Super Micro Computer Inc: Unknown device 5180 Flags: bus master, medium devsel, latency 0, IRQ 19 I/O ports at c000 [size=32] 00: 86 80 aa 25 05 00 80 02 02 00 03 0c 00 00 00 00 10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 20: 01 c0 00 00 00 00 00 00 00 00 00 00 d9 15 80 51 30: 00 00 00 00 00 00 00 00 00 00 00 00 05 02 00 00 00:1d.4 System peripheral: Intel Corp. 6300ESB Watchdog Timer (rev 02) Subsystem: Super Micro Computer Inc: Unknown device 5180 Flags: medium devsel Memory at fb301000 (32-bit, non-prefetchable) [size=16] 00: 86 80 ab 25 02 00 80 02 02 00 80 08 00 00 00 00 10: 00 10 30 fb 00 00 00 00 00 00 00 00 00 00 00 00 20: 00 00 00 00 00 00 00 00 00 00 00 00 d9 15 80 51 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00:1d.5 PIC: Intel Corp. 6300ESB I/O Advanced Programmable Interrupt Controller (rev 02) (prog-if 20 [IO(X)-APIC]) Subsystem: Intel Corp. 6300ESB I/O Advanced Programmable Interrupt Controller Flags: bus master, fast devsel, latency 0 Capabilities: [50] PCI-X non-bridge device. 00: 86 80 ac 25 06 00 10 00 02 20 00 08 00 00 00 00 10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 ac 25 30: 00 00 00 00 50 00 00 00 00 00 00 00 ff 00 00 00 00:1d.7 USB Controller: Intel Corp. 6300ESB USB2 Enhanced Host Controller (rev 02) (prog-if 20 [EHCI]) Subsystem: Super Micro Computer Inc: Unknown device 5180 Flags: bus master, medium devsel, latency 0, IRQ 23 Memory at fb300000 (32-bit, non-prefetchable) [size=1K] Capabilities: [50] Power Management version 2 00: 86 80 ad 25 06 00 90 02 02 20 03 0c 00 00 00 00 10: 00 00 30 fb 00 00 00 00 00 00 00 00 00 00 00 00 20: 00 00 00 00 00 00 00 00 00 00 00 00 d9 15 80 51 30: 00 00 00 00 50 00 00 00 00 00 00 00 0b 04 00 00 00:1e.0 PCI bridge: Intel Corp. 82801BA/CA/DB/EB/ER Hub interface to PCI Bridge (rev 0a) (prog-if 00 [Normal decode]) Flags: bus master, fast devsel, latency 0 Bus: primary=00, secondary=04, subordinate=04, sec-latency=32 I/O behind bridge: 0000b000-0000bfff Memory behind bridge: f8000000-faffffff 00: 86 80 4e 24 07 01 80 00 0a 00 04 06 00 00 01 00 10: 00 00 00 00 00 00 00 00 00 04 04 20 b0 b0 80 22 20: 00 f8 f0 fa f0 ff 00 00 00 00 00 00 00 00 00 00 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0e 00 00:1f.0 ISA bridge: Intel Corp. 6300ESB LPC Interface Controller (rev 02) Flags: bus master, medium devsel, latency 0 00: 86 80 a1 25 0f 00 80 02 02 00 01 06 00 00 80 00 10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00:1f.1 IDE interface: Intel Corp. 6300ESB PATA Storage Controller (rev 02) (prog-if 8a [Master SecP PriP]) Subsystem: Super Micro Computer Inc: Unknown device 5180 Flags: bus master, medium devsel, latency 0, IRQ 18 I/O ports at I/O ports at I/O ports at I/O ports at I/O ports at f000 [size=16] Memory at 1ff00000 (32-bit, non-prefetchable) [size=1K] 00: 86 80 a2 25 07 00 88 02 02 8a 01 01 00 00 00 00 10: 01 00 00 00 01 00 00 00 01 00 00 00 01 00 00 00 20: 01 f0 00 00 00 00 f0 1f 00 00 00 00 d9 15 80 51 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00:1f.2 IDE interface: Intel Corp. 6300ESB SATA Storage Controller (rev 02) (prog-if 8f [Master SecP SecO PriP PriO]) Subsystem: Super Micro Computer Inc: Unknown device 5180 Flags: bus master, 66Mhz, medium devsel, latency 0, IRQ 18 I/O ports at c800 [size=8] I/O ports at cc00 [size=4] I/O ports at d000 [size=8] I/O ports at d400 [size=4] I/O ports at d800 [size=16] 00: 86 80 a3 25 05 00 a0 02 02 8f 01 01 00 00 00 00 10: 01 c8 00 00 01 cc 00 00 01 d0 00 00 01 d4 00 00 20: 01 d8 00 00 00 00 00 00 00 00 00 00 d9 15 80 51 30: 00 00 00 00 00 00 00 00 00 00 00 00 0a 01 00 00 00:1f.3 SMBus: Intel Corp. 6300ESB SMBus Controller (rev 02) Subsystem: Super Micro Computer Inc: Unknown device 5180 Flags: medium devsel, IRQ 17 I/O ports at 0500 [size=32] 00: 86 80 a4 25 01 00 80 02 02 00 05 0c 00 00 00 00 10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 20: 01 05 00 00 00 00 00 00 00 00 00 00 d9 15 80 51 30: 00 00 00 00 00 00 00 00 00 00 00 00 09 02 00 00 01:01.0 Ethernet controller: Intel Corp. 82547GI Gigabit Ethernet Controller Subsystem: Intel Corp. PRO/1000 CT Network Connection Flags: bus master, 66Mhz, medium devsel, latency 0, IRQ 18 Memory at fb200000 (32-bit, non-prefetchable) [size=128K] I/O ports at 9000 [size=32] Capabilities: [dc] Power Management version 2 00: 86 80 75 10 07 00 38 02 00 00 00 02 08 00 00 00 10: 00 00 20 fb 00 00 00 00 01 90 00 00 00 00 00 00 20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 75 10 30: 00 00 00 00 dc 00 00 00 00 00 00 00 0a 01 ff 00 02:01.0 PCI bridge: IBM PCI-X to PCI-X Bridge (rev 02) (prog-if 00 [Normal decode]) Flags: bus master, 66Mhz, medium devsel, latency 32 Bus: primary=02, secondary=03, subordinate=03, sec-latency=32 I/O behind bridge: 0000a000-0000afff Memory behind bridge: fb000000-fb0fffff Capabilities: [80] PCI-X non-bridge device. Capabilities: [90] Power Management version 2 00: 14 10 a7 01 07 01 30 02 02 00 04 06 08 20 01 00 10: 00 00 00 00 00 00 00 00 02 03 03 20 a1 a1 20 22 20: 00 fb 00 fb f1 ff 01 00 00 00 00 00 00 00 00 00 30: 00 00 00 00 80 00 00 00 00 00 00 00 ff 00 06 00 02:04.0 SCSI storage controller: Marvell Technology Group Ltd. MV88SX5041 4-port SATA I PCI-X Controller (rev 03) Subsystem: Marvell Technology Group Ltd. MV88SX5041 4-port SATA I PCI-X Controller Flags: bus master, 66Mhz, medium devsel, latency 32, IRQ 27 Memory at fb100000 (64-bit, non-prefetchable) [size=512K] Capabilities: [40] Power Management version 2 Capabilities: [50] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable- Capabilities: [60] PCI-X non-bridge device. 00: ab 11 41 50 07 00 b0 02 03 00 00 01 08 20 00 00 10: 04 00 10 fb 00 00 00 00 00 00 00 00 00 00 00 00 20: 00 00 00 00 00 00 00 00 00 00 00 00 ab 11 41 50 30: 00 00 00 00 40 00 00 00 00 00 00 00 09 01 00 00 03:04.0 Ethernet controller: Intel Corp. 82546EB Gigabit Ethernet Controller (rev 01) Subsystem: Intel Corp. PRO/1000 MT Quad Port Server Adapter Flags: bus master, 66Mhz, medium devsel, latency 32, IRQ 267 Memory at fb000000 (64-bit, non-prefetchable) [size=128K] I/O ports at a000 [size=64] Capabilities: [dc] Power Management version 2 Capabilities: [e4] PCI-X non-bridge device. Capabilities: [f0] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable- 00: 86 80 1d 10 07 00 30 02 01 00 00 02 08 20 80 00 10: 04 00 00 fb 00 00 00 00 00 00 00 00 00 00 00 00 20: 01 a0 00 00 00 00 00 00 00 00 00 00 86 80 00 10 30: 00 00 00 00 dc 00 00 00 00 00 00 00 0b 01 ff 00 03:04.1 Ethernet controller: Intel Corp. 82546EB Gigabit Ethernet Controller (rev 01) Subsystem: Intel Corp. PRO/1000 MT Quad Port Server Adapter Flags: bus master, 66Mhz, medium devsel, latency 32, IRQ 267 Memory at fb020000 (64-bit, non-prefetchable) [size=128K] I/O ports at a400 [size=64] Capabilities: [dc] Power Management version 2 Capabilities: [e4] PCI-X non-bridge device. Capabilities: [f0] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable- 00: 86 80 1d 10 07 00 30 02 01 00 00 02 08 20 80 00 10: 04 00 02 fb 00 00 00 00 00 00 00 00 00 00 00 00 20: 01 a4 00 00 00 00 00 00 00 00 00 00 86 80 00 10 30: 00 00 00 00 dc 00 00 00 00 00 00 00 09 02 ff 00 03:06.0 Ethernet controller: Intel Corp. 82546EB Gigabit Ethernet Controller (rev 01) Subsystem: Intel Corp. PRO/1000 MT Quad Port Server Adapter Flags: bus master, 66Mhz, medium devsel, latency 32, IRQ 26 Memory at fb040000 (64-bit, non-prefetchable) [size=128K] I/O ports at a800 [size=64] Capabilities: [dc] Power Management version 2 Capabilities: [e4] PCI-X non-bridge device. Capabilities: [f0] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable- 00: 86 80 1d 10 07 00 30 02 01 00 00 02 08 20 80 00 10: 04 00 04 fb 00 00 00 00 00 00 00 00 00 00 00 00 20: 01 a8 00 00 00 00 00 00 00 00 00 00 86 80 00 10 30: 00 00 00 00 dc 00 00 00 00 00 00 00 0a 01 ff 00 03:06.1 Ethernet controller: Intel Corp. 82546EB Gigabit Ethernet Controller (rev 01) Subsystem: Intel Corp. PRO/1000 MT Quad Port Server Adapter Flags: bus master, 66Mhz, medium devsel, latency 32, IRQ 27 Memory at fb060000 (64-bit, non-prefetchable) [size=128K] I/O ports at ac00 [size=64] Capabilities: [dc] Power Management version 2 Capabilities: [e4] PCI-X non-bridge device. Capabilities: [f0] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable- 00: 86 80 1d 10 07 00 30 02 01 00 00 02 08 20 80 00 10: 04 00 06 fb 00 00 00 00 00 00 00 00 00 00 00 00 20: 01 ac 00 00 00 00 00 00 00 00 00 00 86 80 00 10 30: 00 00 00 00 dc 00 00 00 00 00 00 00 05 02 ff 00 04:09.0 VGA compatible controller: ATI Technologies Inc Rage XL (rev 27) (prog-if 00 [VGA]) Subsystem: ATI Technologies Inc Rage XL Flags: bus master, stepping, medium devsel, latency 32, IRQ 17 Memory at f8000000 (32-bit, non-prefetchable) [size=16M] I/O ports at b000 [size=256] Memory at fa020000 (32-bit, non-prefetchable) [size=4K] Expansion ROM at [disabled] [size=128K] Capabilities: [5c] Power Management version 2 00: 02 10 52 47 87 00 90 02 27 00 00 03 08 20 00 00 10: 00 00 00 f8 01 b0 00 00 00 00 02 fa 00 00 00 00 20: 00 00 00 00 00 00 00 00 00 00 00 00 02 10 08 80 30: 00 00 00 00 5c 00 00 00 00 00 00 00 09 01 08 00 04:0a.0 Ethernet controller: Intel Corp. 82541GI/PI Gigabit Ethernet Controller Subsystem: Intel Corp. PRO/1000 MT Network Connection Flags: bus master, 66Mhz, medium devsel, latency 32, IRQ 19 Memory at fa000000 (32-bit, non-prefetchable) [size=128K] I/O ports at b400 [size=64] Capabilities: [dc] Power Management version 2 Capabilities: [e4] PCI-X non-bridge device. 00: 86 80 76 10 07 00 30 02 00 00 00 02 08 20 00 00 10: 00 00 00 fa 00 00 00 00 01 b4 00 00 00 00 00 00 20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 76 10 30: 00 00 00 00 dc 00 00 00 00 00 00 00 05 01 ff 00 --MGYHOYXEY6WxJCY8-- From lanxue@soe.ucsc.edu Sat Dec 4 14:05:22 2004 Received: with ECARTIS (v1.0.0; list netdev); Sat, 04 Dec 2004 14:05:26 -0800 (PST) Received: from services.cse.ucsc.edu (services.cse.ucsc.edu [128.114.48.10]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB4M5Muk002322 for ; Sat, 4 Dec 2004 14:05:22 -0800 Received: from moondance.cse.ucsc.edu (moondance.cse.ucsc.edu [128.114.49.1]) by services.cse.ucsc.edu (8.13.1/8.13.1) with ESMTP id iB4M50An021326; Sat, 4 Dec 2004 14:05:00 -0800 (PST) Date: Sat, 4 Dec 2004 14:05:00 -0800 (PST) From: Lan Xue X-X-Sender: lanxue@moondance.cse.ucsc.edu To: netdev@oss.sgi.com cc: kernerl mail Subject: global memory access In-Reply-To: <20041202094433.39132.qmail@web41411.mail.yahoo.com> Message-ID: References: <20041202094433.39132.qmail@web41411.mail.yahoo.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 12420 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: lanxue@soe.ucsc.edu Precedence: bulk X-list: netdev Hi, I was wondering how a user-level program can share a writable memory buffer with the kernel. The memory can either belongs to the kernel space or the user space, but both kernel module and user program should be able to access(read, write) it. Any hint? Many thanks, Lan From sfeldma@pobox.com Sat Dec 4 15:57:37 2004 Received: with ECARTIS (v1.0.0; list netdev); Sat, 04 Dec 2004 15:57:41 -0800 (PST) Received: from orb.pobox.com (orb.pobox.com [207.8.226.5]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB4Nvald005121 for ; Sat, 4 Dec 2004 15:57:37 -0800 Received: from orb (localhost [127.0.0.1]) by orb.pobox.com (Postfix) with ESMTP id A75DA2FB558; Sat, 4 Dec 2004 18:57:14 -0500 (EST) Received: from [192.168.0.19] (wbar2.sea1-4-5-062-153.sea1.dsl-verizon.net [4.5.62.153]) by orb.sasl.smtp.pobox.com (Postfix) with ESMTP id 1ABA22FAE14; Sat, 4 Dec 2004 18:57:12 -0500 (EST) Subject: Re: e1000 driver problem with Intel Pro/1000 MT adapter From: Scott Feldman Reply-To: sfeldma@pobox.com To: Jos Vos Cc: netdev@oss.sgi.com In-Reply-To: <200412032325.iB3NPdG04693@xos037.xos.nl> References: <200412032325.iB3NPdG04693@xos037.xos.nl> Content-Type: text/plain Message-Id: <1102204801.3343.68.camel@sfeldma-mobl.dsl-verizon.net> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 (1.4.6-2) Date: Sat, 04 Dec 2004 16:00:01 -0800 Content-Transfer-Encoding: 7bit X-archive-position: 12421 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: sfeldma@pobox.com Precedence: bulk X-list: netdev On Fri, 2004-12-03 at 15:25, Jos Vos wrote: > I have a problem with an Intel Pro/1000 MT 4-port card in a Supermicro > (non-HT) Pentium 4 system using RHEL3 (2.4.21 kernel "the RH way") with > the e1000 driver (I tried both the version supplied by RH and the > newest 5.5.4 driver): The 4-port card has two 82546 dual-port controllers behind a PCI-X bridge. Your symptoms suggest interrupt routing didn't get setup correctly for the controllers behind the bridge. I've seen more than one case where the BIOS gets this wrong. The fix is to upgrade to the latest BIOS. Hopefully this is the fix for you. :-) -scott From sfeldma@pobox.com Sat Dec 4 19:18:22 2004 Received: with ECARTIS (v1.0.0; list netdev); Sat, 04 Dec 2004 19:18:31 -0800 (PST) Received: from orb.pobox.com (orb.pobox.com [207.8.226.5]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB53ILpo012747 for ; Sat, 4 Dec 2004 19:18:22 -0800 Received: from orb (localhost [127.0.0.1]) by orb.pobox.com (Postfix) with ESMTP id 619812FA1D7; Sat, 4 Dec 2004 22:17:59 -0500 (EST) Received: from [192.168.0.19] (wbar2.sea1-4-5-062-153.sea1.dsl-verizon.net [4.5.62.153]) by orb.sasl.smtp.pobox.com (Postfix) with ESMTP id 97E8E2F91AB; Sat, 4 Dec 2004 22:17:56 -0500 (EST) Subject: Re: e1000>5.2.30 unstable with InterruptThrottleRate=0 From: Scott Feldman Reply-To: sfeldma@pobox.com To: Peter Kjellstroem Cc: netdev@oss.sgi.com In-Reply-To: References: Content-Type: text/plain Message-Id: <1102216844.3343.84.camel@sfeldma-mobl.dsl-verizon.net> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 (1.4.6-2) Date: Sat, 04 Dec 2004 19:20:44 -0800 Content-Transfer-Encoding: 7bit X-archive-position: 12422 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: sfeldma@pobox.com Precedence: bulk X-list: netdev On Fri, 2004-12-03 at 11:02, Peter Kjellstroem wrote: > Short version: 82547GI with ITR=0 on 2.4.28 (vanilla) and RHEL3u3 has > problems (traffic grinds to a temporary halt under anything but trivila > network traffic). kernel prints the following and resets the IF (many > times): > > NETDEV WATCHDOG: eth0: transmit timed out Dude! You're out of luck! >From the README: CAUTION: If you are using the Intel PRO/1000 CT Network Connection (controller 82547), setting InterruptThrottleRate to a value greater than 75,000, may hang (stop transmitting) adapters under certain network conditions. If this occurs a NETDEV WATCHDOG message is logged in the system event log. In addition, the controller is automatically reset, restoring the network connection. To eliminate the potential for the hang, ensure that InterruptThrottleRate is set no greater than 75,000 and is not set to 0. I was running into the same thing with 82547EI setting ITR=0, and then I remembered that this part is buggy when ITR=0. The bug is due to 82547 messing up the order of interrupt assertion and de-assertion on the CSA bus. If you want to do MPI on this system, you'll need to use a non-zero ITR or plug in an add-in card into one of the PCI slots and use the add-in card. The problem is, these slots are probably 32-bit/33Mhz, so you're not going to get the same maximum Mbps that you'll get with 82547 using the CSA bus. 82547 will not be a good choice for MPI. Sorry. > Affected chips (theory, 8254X, X>1 or anything faster then PCI33): > 82547GI, 82546 (said to be affected, not verified by me) 82546 should be fine with ITR=0. > http://lists.us.dell.com/pipermail/linux-poweredge/2004-November/023061.html You might want to forward this info to that thread. -scott From paul@clubi.ie Sat Dec 4 22:26:20 2004 Received: with ECARTIS (v1.0.0; list netdev); Sat, 04 Dec 2004 22:26:27 -0800 (PST) Received: from hibernia.jakma.org (hibernia.jakma.org [212.17.55.49]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB56QDdM020079 for ; Sat, 4 Dec 2004 22:26:16 -0800 Received: from hibernia.jakma.org (IDENT:paul@hibernia.jakma.org [192.168.0.3]) by hibernia.jakma.org (8.12.11/8.12.11) with ESMTP id iB56PWrc009633; Sun, 5 Dec 2004 06:25:35 GMT Date: Sun, 5 Dec 2004 06:25:31 +0000 (GMT) From: Paul Jakma X-X-Sender: paul@hibernia.jakma.org To: Thomas Spatzier cc: jgarzik@pobox.com, linux-kernel@vger.kernel.org, netdev@oss.sgi.com Subject: Re: [patch 4/10] s390: network driver. In-Reply-To: Message-ID: References: Mail-Followup-To: paul@hibernia.jakma.org X-NSA: arafat al aqsar jihad musharef jet-A1 avgas ammonium qran inshallah allah al-akbar martyr iraq saddam hammas hisballah rabin ayatollah korea vietnam revolt mustard gas british airways washington MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Virus-Scanned: ClamAV 0.80/614/Wed Dec 1 15:44:43 2004 clamav-milter version 0.80j on hibernia.jakma.org X-Virus-Status: Clean X-archive-position: 12423 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: paul@clubi.ie Precedence: bulk X-list: netdev On Tue, 30 Nov 2004, Thomas Spatzier wrote: > Ok, then some logic could be implemented in userland to take > appropriate actions. It must be ensured that zebra handles the > netlink notification fast enough. AIUI, netlink is not synchronous, it most definitely makes no reliability guarantees (and at the moment, zebra isnt terribly efficient at reading netlink, large numbers of interfaces will cause overruns in zebra - fixing this is on the TODO list). So we can never get rid of the window where a daemon could send a packet out a link-down interface - we can make that window smaller but not eliminate it. Hence we need either a way to flush packets associated with an (interface,socket) (or just the socket) or we need the kernel to not accept such packets (and drop packets it has accepted). > In the manpages for send/sendto/sendmsg it says that there is a -ENOBUFS > return value, if a sockets write queue is full. Yes, ENOBUFS, sorry. > It also says: > "Normally, this does not occur in Linux. Packets are just silently dropped > when a device queue overflows." This has always been (AFAIK) the behaviour yes. We started getting reports of the new queuing behaviour with, iirc, a version of Intel's e100 driver for 2.4.2x, which was later changed back to the old behaviour. However now that the queue behaviour is apparently the mandated behaviour we really need to work out what to do about the sending-long-stale packets problem. > So, if packets are 'silently dropped' anyway, the fact that we drop > them in our driver (and increment the error count in the > net_device_stats accordingly) should not be a problem. It shouldnt no. The likes of OSPF already specify their own reliability mechanisms. > I think that both behaviours are similar for TCP. TCP waits for > ACKs for each packet. If they do not arrive, a retransmit is done. > Sooner or later the connection will be reset, if no responses from > the other side arrive. So the result for both driver behaviours > should be the same. But if TCP worked even when drivers dropped packets, then that implies TCP has its own queue? That we're talking about a seperate driver packet queue rather than the socket buffer (which is, presumably, where TCP retains packets until ACKed - i have no idea). Anyway, we do, I think, need some way to deal with the sending-stale-packet-on-link-back problem. Either a way to flush this driver queue or else a guarantee that writes to sockets whose protocol makes no reliability guarantee will either return ENOBUFS or drop the packet. Otherwise we will start getting reports of "Quagga on Linux sent an ancient {RIP,IRDP,RA} packet when we fixed a switch problem, and it caused an outage for a section of our network due to bad routes", I think. Some comment or advice would be useful. (Am I kill-filed by all of netdev? feels like it). > Regards, > Thomas regards, -- Paul Jakma paul@clubi.ie paul@jakma.org Key ID: 64A2FF6A Fortune: No directory. From anton@ozlabs.org Sat Dec 4 23:49:59 2004 Received: with ECARTIS (v1.0.0; list netdev); Sat, 04 Dec 2004 23:50:03 -0800 (PST) Received: from ozlabs.org (ozlabs.org [203.10.76.45]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB57nwKx021994 for ; Sat, 4 Dec 2004 23:49:59 -0800 Received: by ozlabs.org (Postfix, from userid 1010) id 37A332BDB5; Sun, 5 Dec 2004 18:49:32 +1100 (EST) Date: Sun, 5 Dec 2004 18:46:58 +1100 From: Anton Blanchard To: "David S. Miller" Cc: netdev@oss.sgi.com Subject: Re: KERNEL: assertion (!sk->sk_forward_alloc) Message-ID: <20041205074658.GA19168@krispykreme.ozlabs.ibm.com> References: <20041129211855.GC17540@krispykreme.ozlabs.ibm.com> <20041129172210.687bcad3.davem@davemloft.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20041129172210.687bcad3.davem@davemloft.net> User-Agent: Mutt/1.5.6+20040907i X-archive-position: 12424 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: anton@samba.org Precedence: bulk X-list: netdev Hi Dave, > Try to reproduce without that patch installed. > > If there is any mixup of SKB handling by any element in the > path the packet travels, you're screw up all the TCP > accounting knobs on the socket and easily trigger messages > like the above. I'll try. FYI we have 70 gigabit cards in that machine, the reason send-to-self is so useful is that we only need one expensive machine. Without the patch we need to find two expensive machines with 35 gigabits in each :) Anton From greearb@candelatech.com Sun Dec 5 00:37:42 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 00:37:48 -0800 (PST) Received: from www.lanforge.com (ns1.lanforge.com [66.165.47.210]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB58bfwN026805 for ; Sun, 5 Dec 2004 00:37:42 -0800 Received: from [4.33.45.22] (evrtwa1-ar2-4-33-045-022.evrtwa1.dsl-verizon.net [4.33.45.22]) (authenticated bits=0) by www.lanforge.com (8.12.8/8.12.8) with ESMTP id iB58nLLH015959; Sun, 5 Dec 2004 00:49:21 -0800 Message-ID: <41B2C8BF.9050204@candelatech.com> Date: Sun, 05 Dec 2004 00:37:19 -0800 From: Ben Greear Organization: Candela Technologies User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.7.3) Gecko/20041020 X-Accept-Language: en-us, en MIME-Version: 1.0 To: sfeldma@pobox.com CC: Jos Vos , netdev@oss.sgi.com Subject: Re: e1000 driver problem with Intel Pro/1000 MT adapter References: <200412032325.iB3NPdG04693@xos037.xos.nl> <1102204801.3343.68.camel@sfeldma-mobl.dsl-verizon.net> In-Reply-To: <1102204801.3343.68.camel@sfeldma-mobl.dsl-verizon.net> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 12425 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: greearb@candelatech.com Precedence: bulk X-list: netdev Scott Feldman wrote: > On Fri, 2004-12-03 at 15:25, Jos Vos wrote: > >>I have a problem with an Intel Pro/1000 MT 4-port card in a Supermicro >>(non-HT) Pentium 4 system using RHEL3 (2.4.21 kernel "the RH way") with >>the e1000 driver (I tried both the version supplied by RH and the >>newest 5.5.4 driver): > > > The 4-port card has two 82546 dual-port controllers behind a PCI-X > bridge. Your symptoms suggest interrupt routing didn't get setup > correctly for the controllers behind the bridge. I've seen more than > one case where the BIOS gets this wrong. The fix is to upgrade to the > latest BIOS. Hopefully this is the fix for you. :-) For what it's worth, I've had good luck with a 4-port pro/1000 NIC in a SuperMicro X5-DPAGG (dual xeon, PCI-X) motherboard. I've tried it with kernel 2.4.27 and 2.6.9 so far... For maximum performance, I needed to increase the tx and rx descriptors to 2k, I believe this helps mitigate the extra latency caused by the PCI-X bridge on the NIC... Ben > > -scott > -- Ben Greear Candela Technologies Inc http://www.candelatech.com From cap@nsc.liu.se Sun Dec 5 04:42:32 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 04:42:39 -0800 (PST) Received: from papput.nsc.liu.se (ns2.nsc.liu.se [130.236.101.9]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB5CgV6l013315 for ; Sun, 5 Dec 2004 04:42:32 -0800 Received: from mail.nsc.liu.se (mail.nsc.liu.se [130.236.101.49]) by papput.nsc.liu.se (Postfix) with ESMTP id EF2B71C31E2; Sun, 5 Dec 2004 13:42:08 +0100 (CET) Date: Sun, 5 Dec 2004 13:42:08 +0100 (CET) From: Peter Kjellstroem To: Scott Feldman Cc: netdev@oss.sgi.com Subject: Re: e1000>5.2.30 unstable with InterruptThrottleRate=0 In-Reply-To: <1102216844.3343.84.camel@sfeldma-mobl.dsl-verizon.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 12426 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: cap@nsc.liu.se Precedence: bulk X-list: netdev Scott, If you had read my entire message you could have noticed that 82547GI works fully ok (weeks of MPI stress) with 5.2.30 as shipped with 2.4.26. Also, setting it to less then 75K is not enough, it just gets harder to provoke, I have managed to make it die with 20K. I will probably run a 2.4.28 with an e1000 driver from 2.4.26 (tested and ok) but I'm not really happy about adding another path to the list of stuff I have to push into a vanilla 2.4.28. Smells driver problem to me. Regards, Peter On Sat, 4 Dec 2004, Scott Feldman wrote: > On Fri, 2004-12-03 at 11:02, Peter Kjellstroem wrote: > > Short version: 82547GI with ITR=0 on 2.4.28 (vanilla) and RHEL3u3 has > > problems (traffic grinds to a temporary halt under anything but trivila > > network traffic). kernel prints the following and resets the IF (many > > times): > > > > NETDEV WATCHDOG: eth0: transmit timed out > > Dude! You're out of luck! > > >From the README: > > CAUTION: If you are using the Intel PRO/1000 CT Network > Connection (controller 82547), setting > InterruptThrottleRate to a value > greater than 75,000, may hang (stop transmitting) > adapters under certain network conditions. If this > occurs a NETDEV WATCHDOG message is logged in the > system event log. In addition, the controller is > automatically reset, restoring the network > connection. To eliminate the potential for the hang, > ensure that InterruptThrottleRate is set no greater > than 75,000 and is not set to 0. > > I was running into the same thing with 82547EI setting ITR=0, and then I > remembered that this part is buggy when ITR=0. The bug is due to 82547 > messing up the order of interrupt assertion and de-assertion on the CSA > bus. > > If you want to do MPI on this system, you'll need to use a non-zero ITR > or plug in an add-in card into one of the PCI slots and use the add-in > card. The problem is, these slots are probably 32-bit/33Mhz, so you're > not going to get the same maximum Mbps that you'll get with 82547 using > the CSA bus. 82547 will not be a good choice for MPI. Sorry. > > > Affected chips (theory, 8254X, X>1 or anything faster then PCI33): > > 82547GI, 82546 (said to be affected, not verified by me) > > 82546 should be fine with ITR=0. > > > http://lists.us.dell.com/pipermail/linux-poweredge/2004-November/023061.html > > You might want to forward this info to that thread. > > -scott > > -- ------------------------------------------------------------ Peter Kjellstroem | E-mail: cap@nsc.liu.se National Supercomputer Centre | Office: +46(0)13 281492 Linkoeping University | Fax : +46(0)13 282535 SE-581 83 Linkoeping | Sweden | http://www.nsc.liu.se From cap@nsc.liu.se Sun Dec 5 06:04:45 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 06:04:52 -0800 (PST) Received: from papput.nsc.liu.se (ns2.nsc.liu.se [130.236.101.9]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB5E4iKV015285 for ; Sun, 5 Dec 2004 06:04:45 -0800 Received: from mail.nsc.liu.se (mail.nsc.liu.se [130.236.101.49]) by papput.nsc.liu.se (Postfix) with ESMTP id 801F91C31E2; Sun, 5 Dec 2004 15:04:22 +0100 (CET) Date: Sun, 5 Dec 2004 15:04:22 +0100 (CET) From: Peter Kjellstroem To: Scott Feldman Cc: netdev@oss.sgi.com Subject: Re: e1000>5.2.30 unstable with InterruptThrottleRate=0 In-Reply-To: <1102216844.3343.84.camel@sfeldma-mobl.dsl-verizon.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 12427 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: cap@nsc.liu.se Precedence: bulk X-list: netdev Hello again, I'm sorry my previous e-mail came out much more harsh then I intended it to (not native english speaking). I just wanted to point out that it's not falling over for me with all drivers. I've been doing som more tests today to try to narrow down which patch causes my problems. So far I have this: 2.4.26 (5.2.30) is ok 2.4.28 (5.4.11) is not 2.4.28 with 5.2.30 patched in is ok 2.4.28 with 5.2.39 (from 2.4.27-pre1) is not In e1000_main.c for 5.2.39 (from 2.4.27-pre1) i found this: /* Change Log * * 5.2.39 3/12/04 * ... * o Back out the CSA fix for 82547 as it continues to cause * systems lock-ups with production systems. Best Regards, Peter On Sat, 4 Dec 2004, Scott Feldman wrote: > On Fri, 2004-12-03 at 11:02, Peter Kjellstroem wrote: > > Short version: 82547GI with ITR=0 on 2.4.28 (vanilla) and RHEL3u3 has > > problems (traffic grinds to a temporary halt under anything but trivila > > network traffic). kernel prints the following and resets the IF (many > > times): > > > > NETDEV WATCHDOG: eth0: transmit timed out > > Dude! You're out of luck! > > >From the README: > > CAUTION: If you are using the Intel PRO/1000 CT Network > Connection (controller 82547), setting > InterruptThrottleRate to a value > greater than 75,000, may hang (stop transmitting) > adapters under certain network conditions. If this > occurs a NETDEV WATCHDOG message is logged in the > system event log. In addition, the controller is > automatically reset, restoring the network > connection. To eliminate the potential for the hang, > ensure that InterruptThrottleRate is set no greater > than 75,000 and is not set to 0. > > I was running into the same thing with 82547EI setting ITR=0, and then I > remembered that this part is buggy when ITR=0. The bug is due to 82547 > messing up the order of interrupt assertion and de-assertion on the CSA > bus. > > If you want to do MPI on this system, you'll need to use a non-zero ITR > or plug in an add-in card into one of the PCI slots and use the add-in > card. The problem is, these slots are probably 32-bit/33Mhz, so you're > not going to get the same maximum Mbps that you'll get with 82547 using > the CSA bus. 82547 will not be a good choice for MPI. Sorry. > > > Affected chips (theory, 8254X, X>1 or anything faster then PCI33): > > 82547GI, 82546 (said to be affected, not verified by me) > > 82546 should be fine with ITR=0. > > > http://lists.us.dell.com/pipermail/linux-poweredge/2004-November/023061.html > > You might want to forward this info to that thread. > > -scott > > -- ------------------------------------------------------------ Peter Kjellstroem | E-mail: cap@nsc.liu.se National Supercomputer Centre | Sweden | http://www.nsc.liu.se From buytenh@wantstofly.org Sun Dec 5 06:51:16 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 06:51:21 -0800 (PST) Received: from xi.wantstofly.org (alephnull.demon.nl [212.238.201.82]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB5EpDQd016544 for ; Sun, 5 Dec 2004 06:51:15 -0800 Received: by xi.wantstofly.org (Postfix, from userid 500) id 46A9F2B0ED; Sun, 5 Dec 2004 15:50:51 +0100 (MET) Date: Sun, 5 Dec 2004 15:50:51 +0100 From: Lennert Buytenhek To: Scott Feldman Cc: jamal , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) Message-ID: <20041205145051.GA647@xi.wantstofly.org> References: <1101499285.1079.45.camel@jzny.localdomain> <16811.8052.678955.795327@robur.slu.se> <1101821501.1043.43.camel@jzny.localdomain> <20041130134600.GA31515@xi.wantstofly.org> <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <20041201182943.GA14470@xi.wantstofly.org> <20041201213550.GF14470@xi.wantstofly.org> <1101967983.4782.9.camel@localhost.localdomain> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1101967983.4782.9.camel@localhost.localdomain> User-Agent: Mutt/1.4.1i X-archive-position: 12428 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: buytenh@wantstofly.org Precedence: bulk X-list: netdev On Wed, Dec 01, 2004 at 10:13:33PM -0800, Scott Feldman wrote: > Idea#3 > > http://www.mail-archive.com/freebsd-net@freebsd.org/msg10826.html > > Set TXDMAC to 0 in e1000_configure_tx. Enabling 'DMA packet prefetching' gives me an impressive boost in performance. Combined with your TX clean rework, I now get 1.03Mpps TX performance at 60B packets. Transmitting from both of the 82546 ports at the same time gives me close to 2 Mpps. The freebsd post hints that (some) e1000 hardware might be buggy w.r.t. this prefetching though. I'll play some more with the other ideas you suggested as well. 60 1036488 61 1037413 62 1036429 63 990239 64 993218 65 993233 66 993201 67 993234 68 993219 69 993208 70 992225 71 980560 --L diff -ur e1000.orig/e1000_main.c e1000/e1000_main.c --- e1000.orig/e1000_main.c 2004-12-04 11:43:12.000000000 +0100 +++ e1000/e1000_main.c 2004-12-05 15:40:49.284946897 +0100 @@ -879,6 +894,8 @@ E1000_WRITE_REG(&adapter->hw, TCTL, tctl); + E1000_WRITE_REG(&adapter->hw, TXDMAC, 0); + e1000_config_collision_dist(&adapter->hw); /* Setup Transmit Descriptor Settings for eop descriptor */ From gandalf@wlug.westbo.se Sun Dec 5 07:04:02 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 07:04:07 -0800 (PST) Received: from null.rsn.bth.se (postfix@null.rsn.bth.se [194.47.142.3]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB5F41Dl017185 for ; Sun, 5 Dec 2004 07:04:01 -0800 Received: by null.rsn.bth.se (Postfix, from userid 65534) id 5BBC22C0056; Sun, 5 Dec 2004 16:03:37 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by null.rsn.bth.se (Postfix) with ESMTP id DB2D72C0068; Sun, 5 Dec 2004 16:03:36 +0100 (CET) Received: from tux.rsn.bth.se (tux.rsn.bth.se [194.47.143.135]) by null.rsn.bth.se (Postfix) with ESMTP id 26F942C0056; Sun, 5 Dec 2004 16:03:36 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by tux.rsn.bth.se (Postfix) with ESMTP id 4F5D53FA7; Sun, 5 Dec 2004 16:03:36 +0100 (CET) Date: Sun, 5 Dec 2004 16:03:36 +0100 (CET) From: Martin Josefsson X-X-Sender: gandalf@tux.rsn.bth.se To: Lennert Buytenhek Cc: Scott Feldman , jamal , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) In-Reply-To: <20041205145051.GA647@xi.wantstofly.org> Message-ID: References: <1101499285.1079.45.camel@jzny.localdomain> <16811.8052.678955.795327@robur.slu.se> <1101821501.1043.43.camel@jzny.localdomain> <20041130134600.GA31515@xi.wantstofly.org> <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <20041201182943.GA14470@xi.wantstofly.org> <20041201213550.GF14470@xi.wantstofly.org> <1101967983.4782.9.camel@localhost.localdomain> <20041205145051.GA647@xi.wantstofly.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by amavisd-new-20030616-p10 on null.rsn.bth.se X-archive-position: 12429 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: gandalf@wlug.westbo.se Precedence: bulk X-list: netdev On Sun, 5 Dec 2004, Lennert Buytenhek wrote: > Enabling 'DMA packet prefetching' gives me an impressive boost in performance. > Combined with your TX clean rework, I now get 1.03Mpps TX performance at 60B > packets. Transmitting from both of the 82546 ports at the same time gives me > close to 2 Mpps. > > The freebsd post hints that (some) e1000 hardware might be buggy w.r.t. this > prefetching though. > > I'll play some more with the other ideas you suggested as well. > > 60 1036488 I was just playing with prefetching when you sent your mail :) I get that number with Scotts patch but without prefetching. If I mode the TDT update to the tc cleaning I get a few extra kpps but not much. BUT if I use the above + prefetching I get this: 60 1483890 64 1418568 68 1356992 72 1300523 76 1248568 80 1142989 84 1140909 88 1114951 92 1076546 96 960732 100 949801 104 972876 108 945314 112 918380 116 891393 120 865923 124 843288 128 696465 Which is pretty nice :) This is on one port of a 82546GB The hardware is a dual Athlon MP 2000+ in an Asus A7M266-D motherboard and the nic is located in a 64/66 slot. I won't post any patch until I've tested some more and cleaned up a few things. BTW, I also get some transmit timouts with Scotts patch sometimes, not often but it does happen. /Martin From buytenh@wantstofly.org Sun Dec 5 07:16:07 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 07:16:11 -0800 (PST) Received: from xi.wantstofly.org (alephnull.demon.nl [212.238.201.82]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB5FG6Go017770 for ; Sun, 5 Dec 2004 07:16:07 -0800 Received: by xi.wantstofly.org (Postfix, from userid 500) id 198CD2B0ED; Sun, 5 Dec 2004 16:15:45 +0100 (MET) Date: Sun, 5 Dec 2004 16:15:45 +0100 From: Lennert Buytenhek To: Martin Josefsson Cc: Scott Feldman , jamal , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) Message-ID: <20041205151545.GC647@xi.wantstofly.org> References: <1101821501.1043.43.camel@jzny.localdomain> <20041130134600.GA31515@xi.wantstofly.org> <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <20041201182943.GA14470@xi.wantstofly.org> <20041201213550.GF14470@xi.wantstofly.org> <1101967983.4782.9.camel@localhost.localdomain> <20041205145051.GA647@xi.wantstofly.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.1i X-archive-position: 12430 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: buytenh@wantstofly.org Precedence: bulk X-list: netdev On Sun, Dec 05, 2004 at 04:03:36PM +0100, Martin Josefsson wrote: > BUT if I use the above + prefetching I get this: > > 60 1483890 > [snip] > > Which is pretty nice :) Not just that, it's also wire speed GigE. Damn. Now we all have to go and upgrade to 10GbE cards, and I don't think my girlfriend would give me one of those for christmas. > This is on one port of a 82546GB > > The hardware is a dual Athlon MP 2000+ in an Asus A7M266-D motherboard and > the nic is located in a 64/66 slot. Hmmm. Funny you get this number even on 64/66. How many PCI bridges between the CPUs and the NIC? Any idea how many cycles an MMIO read on your hardware is? cheers, Lennert From gandalf@wlug.westbo.se Sun Dec 5 07:19:54 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 07:19:59 -0800 (PST) Received: from null.rsn.bth.se (postfix@null.rsn.bth.se [194.47.142.3]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB5FJr7J018134 for ; Sun, 5 Dec 2004 07:19:53 -0800 Received: by null.rsn.bth.se (Postfix, from userid 65534) id F0EA82C0069; Sun, 5 Dec 2004 16:19:30 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by null.rsn.bth.se (Postfix) with ESMTP id 86DD02C006F; Sun, 5 Dec 2004 16:19:30 +0100 (CET) Received: from tux.rsn.bth.se (tux.rsn.bth.se [194.47.143.135]) by null.rsn.bth.se (Postfix) with ESMTP id C7C3F2C0069; Sun, 5 Dec 2004 16:19:29 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by tux.rsn.bth.se (Postfix) with ESMTP id EDD873FA7; Sun, 5 Dec 2004 16:19:29 +0100 (CET) Date: Sun, 5 Dec 2004 16:19:29 +0100 (CET) From: Martin Josefsson X-X-Sender: gandalf@tux.rsn.bth.se To: Lennert Buytenhek Cc: Scott Feldman , jamal , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) In-Reply-To: <20041205151545.GC647@xi.wantstofly.org> Message-ID: References: <1101821501.1043.43.camel@jzny.localdomain> <20041130134600.GA31515@xi.wantstofly.org> <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <20041201182943.GA14470@xi.wantstofly.org> <20041201213550.GF14470@xi.wantstofly.org> <1101967983.4782.9.camel@localhost.localdomain> <20041205145051.GA647@xi.wantstofly.org> <20041205151545.GC647@xi.wantstofly.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by amavisd-new-20030616-p10 on null.rsn.bth.se X-archive-position: 12431 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: gandalf@wlug.westbo.se Precedence: bulk X-list: netdev On Sun, 5 Dec 2004, Lennert Buytenhek wrote: > > 60 1483890 > > [snip] > > > > Which is pretty nice :) > > Not just that, it's also wire speed GigE. Damn. Now we all have to go > and upgrade to 10GbE cards, and I don't think my girlfriend would give me > one of those for christmas. Yes it is, and it's lovely to see. You have to nerdify her so she sees the need for geeky hardware enough to give you what you need :) > > This is on one port of a 82546GB > > > > The hardware is a dual Athlon MP 2000+ in an Asus A7M266-D motherboard and > > the nic is located in a 64/66 slot. > > Hmmm. Funny you get this number even on 64/66. How many PCI bridges > between the CPUs and the NIC? Any idea how many cycles an MMIO read on > your hardware is? I verified that I get the same results on a small whimpy 82540EM that runs at 32/66 as well. Just about to see what I get at 32/33 with that card. /Martin From gandalf@wlug.westbo.se Sun Dec 5 07:31:12 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 07:31:17 -0800 (PST) Received: from null.rsn.bth.se (postfix@null.rsn.bth.se [194.47.142.3]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB5FVBfU018611 for ; Sun, 5 Dec 2004 07:31:11 -0800 Received: by null.rsn.bth.se (Postfix, from userid 65534) id 0EBEF2C0056; Sun, 5 Dec 2004 16:30:49 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by null.rsn.bth.se (Postfix) with ESMTP id 7E9BA2C0069; Sun, 5 Dec 2004 16:30:48 +0100 (CET) Received: from tux.rsn.bth.se (tux.rsn.bth.se [194.47.143.135]) by null.rsn.bth.se (Postfix) with ESMTP id B3B1D2C0056; Sun, 5 Dec 2004 16:30:47 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by tux.rsn.bth.se (Postfix) with ESMTP id DD5093FA7; Sun, 5 Dec 2004 16:30:47 +0100 (CET) Date: Sun, 5 Dec 2004 16:30:47 +0100 (CET) From: Martin Josefsson X-X-Sender: gandalf@tux.rsn.bth.se To: Lennert Buytenhek Cc: Scott Feldman , jamal , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) In-Reply-To: Message-ID: References: <1101821501.1043.43.camel@jzny.localdomain> <20041130134600.GA31515@xi.wantstofly.org> <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <20041201182943.GA14470@xi.wantstofly.org> <20041201213550.GF14470@xi.wantstofly.org> <1101967983.4782.9.camel@localhost.localdomain> <20041205145051.GA647@xi.wantstofly.org> <20041205151545.GC647@xi.wantstofly.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by amavisd-new-20030616-p10 on null.rsn.bth.se X-archive-position: 12432 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: gandalf@wlug.westbo.se Precedence: bulk X-list: netdev On Sun, 5 Dec 2004, Martin Josefsson wrote: > > > The hardware is a dual Athlon MP 2000+ in an Asus A7M266-D motherboard and > > > the nic is located in a 64/66 slot. > > > > Hmmm. Funny you get this number even on 64/66. How many PCI bridges > > between the CPUs and the NIC? Any idea how many cycles an MMIO read on > > your hardware is? > > I verified that I get the same results on a small whimpy 82540EM that runs > at 32/66 as well. Just about to see what I get at 32/33 with that card. Just tested the 82540EM at 32/33 and it's a big diffrence. 60 350229 64 247037 68 219643 72 218205 76 216786 80 215386 84 214003 88 212638 92 211291 96 210004 100 208647 104 182461 108 181468 112 180453 116 179482 120 185472 124 188336 128 153743 Sorry, forgot to answer your other questions, I'm a bit excited at the moment :) The 64/66 bus on this motherboard is directly connected to the northbridge. Here's the lspci output with the 82546GB nic attached to the 64/66 bus and 82540EM nic connected to the 32/33 bus that hangs off the southbridge: 00:00.0 Host bridge: Advanced Micro Devices [AMD] AMD-760 MP [IGD4-2P] System Controller (rev 11) 00:01.0 PCI bridge: Advanced Micro Devices [AMD] AMD-760 MP [IGD4-2P] AGP Bridge 00:07.0 ISA bridge: Advanced Micro Devices [AMD] AMD-768 [Opus] ISA (rev 05) 00:07.1 IDE interface: Advanced Micro Devices [AMD] AMD-768 [Opus] IDE (rev 04) 00:07.3 Bridge: Advanced Micro Devices [AMD] AMD-768 [Opus] ACPI (rev 03) 00:08.0 Ethernet controller: Intel Corp. 82546GB Gigabit Ethernet Controller (rev 03) 00:08.1 Ethernet controller: Intel Corp. 82546GB Gigabit Ethernet Controller (rev 03) 00:10.0 PCI bridge: Advanced Micro Devices [AMD] AMD-768 [Opus] PCI (rev 05) 01:05.0 VGA compatible controller: Silicon Integrated Systems [SiS] 86C326 5598/6326 (rev 0b) 02:05.0 Ethernet controller: Intel Corp. 82557/8/9 [Ethernet Pro 100] (rev 0c) 02:06.0 SCSI storage controller: Adaptec AIC-7892A U160/m (rev 02) 02:08.0 Ethernet controller: Intel Corp. 82540EM Gigabit Ethernet Controller (rev 02) And lspci -t -[00]-+-00.0 +-01.0-[01]----05.0 +-07.0 +-07.1 +-07.3 +-08.0 +-08.1 \-10.0-[02]--+-05.0 +-06.0 \-08.0 I have no idea how expensive an MMIO read is on this machine, do you have an relatively easy way to find out? /Martin From gandalf@wlug.westbo.se Sun Dec 5 07:43:00 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 07:43:04 -0800 (PST) Received: from null.rsn.bth.se (postfix@null.rsn.bth.se [194.47.142.3]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB5Fgw6n019392 for ; Sun, 5 Dec 2004 07:42:59 -0800 Received: by null.rsn.bth.se (Postfix, from userid 65534) id 521D92C0069; Sun, 5 Dec 2004 16:42:36 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by null.rsn.bth.se (Postfix) with ESMTP id 644B22C006F; Sun, 5 Dec 2004 16:42:35 +0100 (CET) Received: from tux.rsn.bth.se (tux.rsn.bth.se [194.47.143.135]) by null.rsn.bth.se (Postfix) with ESMTP id 930A92C0069; Sun, 5 Dec 2004 16:42:34 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by tux.rsn.bth.se (Postfix) with ESMTP id BA4393FA7; Sun, 5 Dec 2004 16:42:34 +0100 (CET) Date: Sun, 5 Dec 2004 16:42:34 +0100 (CET) From: Martin Josefsson X-X-Sender: gandalf@tux.rsn.bth.se To: Lennert Buytenhek Cc: Scott Feldman , jamal , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) In-Reply-To: Message-ID: References: <1101499285.1079.45.camel@jzny.localdomain> <16811.8052.678955.795327@robur.slu.se> <1101821501.1043.43.camel@jzny.localdomain> <20041130134600.GA31515@xi.wantstofly.org> <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <20041201182943.GA14470@xi.wantstofly.org> <20041201213550.GF14470@xi.wantstofly.org> <1101967983.4782.9.camel@localhost.localdomain> <20041205145051.GA647@xi.wantstofly.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by amavisd-new-20030616-p10 on null.rsn.bth.se X-archive-position: 12433 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: gandalf@wlug.westbo.se Precedence: bulk X-list: netdev On Sun, 5 Dec 2004, Martin Josefsson wrote: [snip] > BUT if I use the above + prefetching I get this: > > 60 1483890 [snip] > This is on one port of a 82546GB > > The hardware is a dual Athlon MP 2000+ in an Asus A7M266-D motherboard and > the nic is located in a 64/66 slot. > > I won't post any patch until I've tested some more and cleaned up a few > things. > > BTW, I also get some transmit timouts with Scotts patch sometimes, not > often but it does happen. Here's the patch, not much more tested (it still gives some transmit timeouts since it's scotts patch + prefetching and delayed TDT updating). And it's not cleaned up, but hey, that's development :) The delayed TDT updating was a test and currently it delays the first tx'd packet after a timerrun 1ms. Would be interesting to see what other people get with this thing. Lennert? diff -X /home/gandalf/dontdiff.ny -urNp linux-2.6.10-rc3.orig/drivers/net/e1000/e1000.h linux-2.6.10-rc3.labbrouter/drivers/net/e1000/e1000.h --- linux-2.6.10-rc3.orig/drivers/net/e1000/e1000.h 2004-12-04 18:16:53.000000000 +0100 +++ linux-2.6.10-rc3.labbrouter/drivers/net/e1000/e1000.h 2004-12-05 15:12:25.000000000 +0100 @@ -101,7 +101,7 @@ struct e1000_adapter; #define E1000_MAX_INTR 10 /* TX/RX descriptor defines */ -#define E1000_DEFAULT_TXD 256 +#define E1000_DEFAULT_TXD 4096 #define E1000_MAX_TXD 256 #define E1000_MIN_TXD 80 #define E1000_MAX_82544_TXD 4096 @@ -187,6 +187,7 @@ struct e1000_desc_ring { /* board specific private data structure */ struct e1000_adapter { + struct timer_list tx_cleanup_timer; struct timer_list tx_fifo_stall_timer; struct timer_list watchdog_timer; struct timer_list phy_info_timer; @@ -222,6 +223,7 @@ struct e1000_adapter { uint32_t tx_fifo_size; atomic_t tx_fifo_stall; boolean_t pcix_82544; + boolean_t tx_cleanup_scheduled; /* RX */ struct e1000_desc_ring rx_ring; diff -X /home/gandalf/dontdiff.ny -urNp linux-2.6.10-rc3.orig/drivers/net/e1000/e1000_hw.h linux-2.6.10-rc3.labbrouter/drivers/net/e1000/e1000_hw.h --- linux-2.6.10-rc3.orig/drivers/net/e1000/e1000_hw.h 2004-12-04 18:16:53.000000000 +0100 +++ linux-2.6.10-rc3.labbrouter/drivers/net/e1000/e1000_hw.h 2004-12-05 15:37:50.000000000 +0100 @@ -417,14 +417,12 @@ int32_t e1000_set_d3_lplu_state(struct e /* This defines the bits that are set in the Interrupt Mask * Set/Read Register. Each bit is documented below: * o RXT0 = Receiver Timer Interrupt (ring 0) - * o TXDW = Transmit Descriptor Written Back * o RXDMT0 = Receive Descriptor Minimum Threshold hit (ring 0) * o RXSEQ = Receive Sequence Error * o LSC = Link Status Change */ #define IMS_ENABLE_MASK ( \ E1000_IMS_RXT0 | \ - E1000_IMS_TXDW | \ E1000_IMS_RXDMT0 | \ E1000_IMS_RXSEQ | \ E1000_IMS_LSC) diff -X /home/gandalf/dontdiff.ny -urNp linux-2.6.10-rc3.orig/drivers/net/e1000/e1000_main.c linux-2.6.10-rc3.labbrouter/drivers/net/e1000/e1000_main.c --- linux-2.6.10-rc3.orig/drivers/net/e1000/e1000_main.c 2004-12-05 14:59:19.000000000 +0100 +++ linux-2.6.10-rc3.labbrouter/drivers/net/e1000/e1000_main.c 2004-12-05 15:40:11.000000000 +0100 @@ -131,7 +131,7 @@ static int e1000_set_mac(struct net_devi static void e1000_irq_disable(struct e1000_adapter *adapter); static void e1000_irq_enable(struct e1000_adapter *adapter); static irqreturn_t e1000_intr(int irq, void *data, struct pt_regs *regs); -static boolean_t e1000_clean_tx_irq(struct e1000_adapter *adapter); +static void e1000_clean_tx(unsigned long data); #ifdef CONFIG_E1000_NAPI static int e1000_clean(struct net_device *netdev, int *budget); static boolean_t e1000_clean_rx_irq(struct e1000_adapter *adapter, @@ -286,6 +286,7 @@ e1000_down(struct e1000_adapter *adapter e1000_irq_disable(adapter); free_irq(adapter->pdev->irq, netdev); + del_timer_sync(&adapter->tx_cleanup_timer); del_timer_sync(&adapter->tx_fifo_stall_timer); del_timer_sync(&adapter->watchdog_timer); del_timer_sync(&adapter->phy_info_timer); @@ -522,6 +523,10 @@ e1000_probe(struct pci_dev *pdev, e1000_get_bus_info(&adapter->hw); + init_timer(&adapter->tx_cleanup_timer); + adapter->tx_cleanup_timer.function = &e1000_clean_tx; + adapter->tx_cleanup_timer.data = (unsigned long) adapter; + init_timer(&adapter->tx_fifo_stall_timer); adapter->tx_fifo_stall_timer.function = &e1000_82547_tx_fifo_stall; adapter->tx_fifo_stall_timer.data = (unsigned long) adapter; @@ -882,19 +887,16 @@ e1000_configure_tx(struct e1000_adapter e1000_config_collision_dist(&adapter->hw); /* Setup Transmit Descriptor Settings for eop descriptor */ - adapter->txd_cmd = E1000_TXD_CMD_IDE | E1000_TXD_CMD_EOP | + adapter->txd_cmd = E1000_TXD_CMD_EOP | E1000_TXD_CMD_IFCS; - if(adapter->hw.mac_type < e1000_82543) - adapter->txd_cmd |= E1000_TXD_CMD_RPS; - else - adapter->txd_cmd |= E1000_TXD_CMD_RS; - /* Cache if we're 82544 running in PCI-X because we'll * need this to apply a workaround later in the send path. */ if(adapter->hw.mac_type == e1000_82544 && adapter->hw.bus_type == e1000_bus_type_pcix) adapter->pcix_82544 = 1; + + E1000_WRITE_REG(&adapter->hw, TXDMAC, 0); } /** @@ -1707,7 +1709,7 @@ e1000_tx_queue(struct e1000_adapter *ada wmb(); tx_ring->next_to_use = i; - E1000_WRITE_REG(&adapter->hw, TDT, i); + /* E1000_WRITE_REG(&adapter->hw, TDT, i); */ } /** @@ -1809,6 +1811,11 @@ e1000_xmit_frame(struct sk_buff *skb, st return NETDEV_TX_LOCKED; } + if(!adapter->tx_cleanup_scheduled) { + adapter->tx_cleanup_scheduled = TRUE; + mod_timer(&adapter->tx_cleanup_timer, jiffies + 1); + } + /* need: count + 2 desc gap to keep tail from touching * head, otherwise try next time */ if(E1000_DESC_UNUSED(&adapter->tx_ring) < count + 2) { @@ -1845,6 +1852,7 @@ e1000_xmit_frame(struct sk_buff *skb, st netdev->trans_start = jiffies; spin_unlock_irqrestore(&adapter->tx_lock, flags); + return NETDEV_TX_OK; } @@ -2140,8 +2148,7 @@ e1000_intr(int irq, void *data, struct p } #else for(i = 0; i < E1000_MAX_INTR; i++) - if(unlikely(!e1000_clean_rx_irq(adapter) & - !e1000_clean_tx_irq(adapter))) + if(unlikely(!e1000_clean_rx_irq(adapter))) break; #endif @@ -2159,18 +2166,15 @@ e1000_clean(struct net_device *netdev, i { struct e1000_adapter *adapter = netdev->priv; int work_to_do = min(*budget, netdev->quota); - int tx_cleaned; int work_done = 0; - tx_cleaned = e1000_clean_tx_irq(adapter); e1000_clean_rx_irq(adapter, &work_done, work_to_do); *budget -= work_done; netdev->quota -= work_done; - /* if no Rx and Tx cleanup work was done, exit the polling mode */ - if(!tx_cleaned || (work_done < work_to_do) || - !netif_running(netdev)) { + /* if no Rx cleanup work was done, exit the polling mode */ + if((work_done < work_to_do) || !netif_running(netdev)) { netif_rx_complete(netdev); e1000_irq_enable(adapter); return 0; @@ -2181,66 +2185,76 @@ e1000_clean(struct net_device *netdev, i #endif /** - * e1000_clean_tx_irq - Reclaim resources after transmit completes - * @adapter: board private structure + * e1000_clean_tx - Reclaim resources after transmit completes + * @data: timer callback data (board private structure) **/ -static boolean_t -e1000_clean_tx_irq(struct e1000_adapter *adapter) +static void +e1000_clean_tx(unsigned long data) { + struct e1000_adapter *adapter = (struct e1000_adapter *)data; struct e1000_desc_ring *tx_ring = &adapter->tx_ring; struct net_device *netdev = adapter->netdev; struct pci_dev *pdev = adapter->pdev; - struct e1000_tx_desc *tx_desc, *eop_desc; struct e1000_buffer *buffer_info; - unsigned int i, eop; - boolean_t cleaned = FALSE; + unsigned int i, next; + int size = 0, count = 0; + uint32_t tx_head; - i = tx_ring->next_to_clean; - eop = tx_ring->buffer_info[i].next_to_watch; - eop_desc = E1000_TX_DESC(*tx_ring, eop); + spin_lock(&adapter->tx_lock); - while(eop_desc->upper.data & cpu_to_le32(E1000_TXD_STAT_DD)) { - for(cleaned = FALSE; !cleaned; ) { - tx_desc = E1000_TX_DESC(*tx_ring, i); - buffer_info = &tx_ring->buffer_info[i]; + E1000_WRITE_REG(&adapter->hw, TDT, tx_ring->next_to_use); - if(likely(buffer_info->dma)) { - pci_unmap_page(pdev, - buffer_info->dma, - buffer_info->length, - PCI_DMA_TODEVICE); - buffer_info->dma = 0; - } + tx_head = E1000_READ_REG(&adapter->hw, TDH); - if(buffer_info->skb) { - dev_kfree_skb_any(buffer_info->skb); - buffer_info->skb = NULL; - } + i = next = tx_ring->next_to_clean; - tx_desc->buffer_addr = 0; - tx_desc->lower.data = 0; - tx_desc->upper.data = 0; + while(i != tx_head) { + size++; + if(i == tx_ring->buffer_info[next].next_to_watch) { + count += size; + size = 0; + if(unlikely(++i == tx_ring->count)) + i = 0; + next = i; + } else { + if(unlikely(++i == tx_ring->count)) + i = 0; + } + } - cleaned = (i == eop); - if(unlikely(++i == tx_ring->count)) i = 0; + i = tx_ring->next_to_clean; + while(count--) { + buffer_info = &tx_ring->buffer_info[i]; + + if(likely(buffer_info->dma)) { + pci_unmap_page(pdev, + buffer_info->dma, + buffer_info->length, + PCI_DMA_TODEVICE); + buffer_info->dma = 0; } - - eop = tx_ring->buffer_info[i].next_to_watch; - eop_desc = E1000_TX_DESC(*tx_ring, eop); + + if(buffer_info->skb) { + dev_kfree_skb_any(buffer_info->skb); + buffer_info->skb = NULL; + } + + if(unlikely(++i == tx_ring->count)) + i = 0; } tx_ring->next_to_clean = i; - spin_lock(&adapter->tx_lock); + if(E1000_DESC_UNUSED(tx_ring) != tx_ring->count) + mod_timer(&adapter->tx_cleanup_timer, jiffies + 1); + else + adapter->tx_cleanup_scheduled = FALSE; - if(unlikely(cleaned && netif_queue_stopped(netdev) && - netif_carrier_ok(netdev))) + if(unlikely(netif_queue_stopped(netdev) && netif_carrier_ok(netdev))) netif_wake_queue(netdev); spin_unlock(&adapter->tx_lock); - - return cleaned; } /** /Martin From gandalf@wlug.westbo.se Sun Dec 5 08:48:59 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 08:49:04 -0800 (PST) Received: from null.rsn.bth.se (postfix@null.rsn.bth.se [194.47.142.3]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB5Gmwrk024151 for ; Sun, 5 Dec 2004 08:48:59 -0800 Received: by null.rsn.bth.se (Postfix, from userid 65534) id 98CEC2C0069; Sun, 5 Dec 2004 17:48:35 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by null.rsn.bth.se (Postfix) with ESMTP id 211292C0069; Sun, 5 Dec 2004 17:48:35 +0100 (CET) Received: from tux.rsn.bth.se (tux.rsn.bth.se [194.47.143.135]) by null.rsn.bth.se (Postfix) with ESMTP id 61B582C0056; Sun, 5 Dec 2004 17:48:34 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by tux.rsn.bth.se (Postfix) with ESMTP id 98F033FA7; Sun, 5 Dec 2004 17:48:34 +0100 (CET) Date: Sun, 5 Dec 2004 17:48:34 +0100 (CET) From: Martin Josefsson X-X-Sender: gandalf@tux.rsn.bth.se To: Lennert Buytenhek Cc: Scott Feldman , jamal , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) In-Reply-To: Message-ID: References: <1101499285.1079.45.camel@jzny.localdomain> <16811.8052.678955.795327@robur.slu.se> <1101821501.1043.43.camel@jzny.localdomain> <20041130134600.GA31515@xi.wantstofly.org> <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <20041201182943.GA14470@xi.wantstofly.org> <20041201213550.GF14470@xi.wantstofly.org> <1101967983.4782.9.camel@localhost.localdomain> <20041205145051.GA647@xi.wantstofly.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by amavisd-new-20030616-p10 on null.rsn.bth.se X-archive-position: 12434 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: gandalf@wlug.westbo.se Precedence: bulk X-list: netdev On Sun, 5 Dec 2004, Martin Josefsson wrote: > The delayed TDT updating was a test and currently it delays the first tx'd > packet after a timerrun 1ms. I removed the delayed TDT updating and gave it a go again (this is scott + prefetching): 60 1486193 64 1267639 68 1259682 72 1243997 76 1243989 80 1153608 84 1123813 88 1115047 92 1076636 96 1040792 100 1007252 104 975806 108 946263 112 918456 116 892227 120 867477 124 844052 128 821858 It gives a little diffrent results, 60byte is ok but then it falls a lot down to 64byte and the curve seems a bit flatter. This should be the same driver that Lennert got 1.03Mpps with. I get 1.03Mpps without prefetching. I tried using both ports on the 82546GB nic. delay nodelay 1CPU 1.95 Mpps 1.76 Mpps 2CPU 1.60 Mpps 1.44 Mpps All tests performed on an SMP kernel, the above mention of 1CPU vs 2CPU just means how the two nics were bound to the cpus. And there's no tx-interrupts at all due to scotts patch. /Martin From buytenh@wantstofly.org Sun Dec 5 09:00:29 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 09:00:33 -0800 (PST) Received: from xi.wantstofly.org (alephnull.demon.nl [212.238.201.82]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB5H0SRM024783 for ; Sun, 5 Dec 2004 09:00:29 -0800 Received: by xi.wantstofly.org (Postfix, from userid 500) id A8CBC2B0ED; Sun, 5 Dec 2004 18:00:06 +0100 (MET) Date: Sun, 5 Dec 2004 18:00:06 +0100 From: Lennert Buytenhek To: Martin Josefsson Cc: Scott Feldman , jamal , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) Message-ID: <20041205170006.GI647@xi.wantstofly.org> References: <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <20041201182943.GA14470@xi.wantstofly.org> <20041201213550.GF14470@xi.wantstofly.org> <1101967983.4782.9.camel@localhost.localdomain> <20041205145051.GA647@xi.wantstofly.org> <20041205151545.GC647@xi.wantstofly.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.1i X-archive-position: 12435 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: buytenh@wantstofly.org Precedence: bulk X-list: netdev On Sun, Dec 05, 2004 at 04:30:47PM +0100, Martin Josefsson wrote: > > I verified that I get the same results on a small whimpy 82540EM > > that runs at 32/66 as well. Just about to see what I get at 32/33 > > with that card. > > Just tested the 82540EM at 32/33 and it's a big diffrence. > > 60 350229 > 64 247037 > 68 219643 > 72 218205 > 76 216786 > 80 215386 > 84 214003 > 88 212638 > 92 211291 > 96 210004 > 100 208647 > 104 182461 > 108 181468 > 112 180453 > 116 179482 > 120 185472 > 124 188336 > 128 153743 With or without prefetching? My 82540 in 32/33 mode gets on baseline 2.6.9: 60 431967 61 431311 62 431927 63 427827 64 427482 And with Scott's notxints patch: 60 514496 61 514493 62 514754 63 504629 64 504123 > Sorry, forgot to answer your other questions, I'm a bit excited at the > moment :) Makes sense :) > The 64/66 bus on this motherboard is directly connected to the > northbridge. Your lspci output seems to suggest there is another PCI bridge in between (00:10.0) Basically on my box, it's CPU - MCH - P64H2 - e1000, where MCH is the 'Memory Controller Hub' and P64H2 the PCI-X bridge chip. > I have no idea how expensive an MMIO read is on this machine, do you have > an relatively easy way to find out? A dirty way, yes ;-) Open up e1000_osdep.h and do: -#define E1000_READ_REG(a, reg) ( \ - readl((a)->hw_addr + \ - (((a)->mac_type >= e1000_82543) ? E1000_##reg : E1000_82542_##reg))) +#define E1000_READ_REG(a, reg) ({ \ + unsigned long s, e, d, v; \ +\ + (a)->mmio_reads++; \ + rdtsc(s, d); \ + v = readl((a)->hw_addr + \ + (((a)->mac_type >= e1000_82543) ? E1000_##reg : E1000_82542_##reg)); \ + rdtsc(e, d); \ + e -= s; \ + printk(KERN_INFO "e1000: MMIO read took %ld clocks\n", e); \ + printk(KERN_INFO "e1000: in process %d(%s)\n", current->pid, current->comm); \ + dump_stack(); \ + v; \ +}) You might want to disable the stack dump of course. --L From gandalf@wlug.westbo.se Sun Dec 5 09:02:07 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 09:02:11 -0800 (PST) Received: from null.rsn.bth.se (postfix@null.rsn.bth.se [194.47.142.3]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB5H26di025153 for ; Sun, 5 Dec 2004 09:02:07 -0800 Received: by null.rsn.bth.se (Postfix, from userid 65534) id 200942C002B; Sun, 5 Dec 2004 18:01:44 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by null.rsn.bth.se (Postfix) with ESMTP id A95EA2C0056; Sun, 5 Dec 2004 18:01:43 +0100 (CET) Received: from tux.rsn.bth.se (tux.rsn.bth.se [194.47.143.135]) by null.rsn.bth.se (Postfix) with ESMTP id E77DC2C002B; Sun, 5 Dec 2004 18:01:42 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by tux.rsn.bth.se (Postfix) with ESMTP id 31EC33FA7; Sun, 5 Dec 2004 18:01:43 +0100 (CET) Date: Sun, 5 Dec 2004 18:01:43 +0100 (CET) From: Martin Josefsson X-X-Sender: gandalf@tux.rsn.bth.se To: Lennert Buytenhek Cc: Scott Feldman , jamal , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) In-Reply-To: Message-ID: References: <1101499285.1079.45.camel@jzny.localdomain> <16811.8052.678955.795327@robur.slu.se> <1101821501.1043.43.camel@jzny.localdomain> <20041130134600.GA31515@xi.wantstofly.org> <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <20041201182943.GA14470@xi.wantstofly.org> <20041201213550.GF14470@xi.wantstofly.org> <1101967983.4782.9.camel@localhost.localdomain> <20041205145051.GA647@xi.wantstofly.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by amavisd-new-20030616-p10 on null.rsn.bth.se X-archive-position: 12436 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: gandalf@wlug.westbo.se Precedence: bulk X-list: netdev On Sun, 5 Dec 2004, Martin Josefsson wrote: > I removed the delayed TDT updating and gave it a go again (this is scott + > prefetching): > > 60 1486193 > 64 1267639 > 68 1259682 Yet another mail, I hope you are using a NAPI-enabled MUA :) This time I tried vanilla + prefetch and it gave pretty nice performance as well: 60 1308047 64 1076044 68 1079377 72 1058993 76 1055708 80 1025659 84 1024692 88 1024236 92 1024510 96 1012853 100 1007925 104 976500 108 947061 112 919169 116 892804 120 868084 124 844609 128 822381 Large gap between 60 and 64byte, maybe the prefetching only prefetches 32bytes at a time? As a reference: here's a completely vanilla e1000 driver: 60 860931 64 772949 68 754738 72 754200 76 756093 80 756398 84 742111 88 738120 92 740426 96 739720 100 722322 104 729287 108 719312 112 723171 116 705551 120 704843 124 704622 128 665863 /Martin From gandalf@wlug.westbo.se Sun Dec 5 09:12:20 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 09:12:23 -0800 (PST) Received: from null.rsn.bth.se (postfix@null.rsn.bth.se [194.47.142.3]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB5HCJaq025640 for ; Sun, 5 Dec 2004 09:12:19 -0800 Received: by null.rsn.bth.se (Postfix, from userid 65534) id 62A582C002B; Sun, 5 Dec 2004 18:11:56 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by null.rsn.bth.se (Postfix) with ESMTP id E0C652C0056; Sun, 5 Dec 2004 18:11:55 +0100 (CET) Received: from tux.rsn.bth.se (tux.rsn.bth.se [194.47.143.135]) by null.rsn.bth.se (Postfix) with ESMTP id 2BD692C002B; Sun, 5 Dec 2004 18:11:55 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by tux.rsn.bth.se (Postfix) with ESMTP id 68B043FA7; Sun, 5 Dec 2004 18:11:55 +0100 (CET) Date: Sun, 5 Dec 2004 18:11:55 +0100 (CET) From: Martin Josefsson X-X-Sender: gandalf@tux.rsn.bth.se To: Lennert Buytenhek Cc: Scott Feldman , jamal , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) In-Reply-To: <20041205170006.GI647@xi.wantstofly.org> Message-ID: References: <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <20041201182943.GA14470@xi.wantstofly.org> <20041201213550.GF14470@xi.wantstofly.org> <1101967983.4782.9.camel@localhost.localdomain> <20041205145051.GA647@xi.wantstofly.org> <20041205151545.GC647@xi.wantstofly.org> <20041205170006.GI647@xi.wantstofly.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by amavisd-new-20030616-p10 on null.rsn.bth.se X-archive-position: 12437 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: gandalf@wlug.westbo.se Precedence: bulk X-list: netdev On Sun, 5 Dec 2004, Lennert Buytenhek wrote: > > Just tested the 82540EM at 32/33 and it's a big diffrence. > > > > 60 350229 > > 64 247037 > > 68 219643 [snip] > With or without prefetching? My 82540 in 32/33 mode gets on baseline > 2.6.9: With, will test without. I've always suspected that the 32bit bus on this motherboard is a bit slow. > Your lspci output seems to suggest there is another PCI bridge in > between (00:10.0) Yes it sits between the 32bit and the 64bit bus. > Basically on my box, it's CPU - MCH - P64H2 - e1000, where MCH is the > 'Memory Controller Hub' and P64H2 the PCI-X bridge chip. I don't have PCI-X (unless 64/66 counts as PCI-x which I highly doubt) > > I have no idea how expensive an MMIO read is on this machine, do you have > > an relatively easy way to find out? > > A dirty way, yes ;-) Open up e1000_osdep.h and do: > > -#define E1000_READ_REG(a, reg) ( \ > - readl((a)->hw_addr + \ > - (((a)->mac_type >= e1000_82543) ? E1000_##reg : E1000_82542_##reg))) > +#define E1000_READ_REG(a, reg) ({ \ > + unsigned long s, e, d, v; \ > +\ > + (a)->mmio_reads++; \ > + rdtsc(s, d); \ > + v = readl((a)->hw_addr + \ > + (((a)->mac_type >= e1000_82543) ? E1000_##reg : E1000_82542_##reg)); \ > + rdtsc(e, d); \ > + e -= s; \ > + printk(KERN_INFO "e1000: MMIO read took %ld clocks\n", e); \ > + printk(KERN_INFO "e1000: in process %d(%s)\n", current->pid, current->comm); \ > + dump_stack(); \ > + v; \ > +}) > > You might want to disable the stack dump of course. Will test this in a while. /Martin From gandalf@wlug.westbo.se Sun Dec 5 09:38:30 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 09:38:34 -0800 (PST) Received: from null.rsn.bth.se (postfix@null.rsn.bth.se [194.47.142.3]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB5HcTvp026395 for ; Sun, 5 Dec 2004 09:38:30 -0800 Received: by null.rsn.bth.se (Postfix, from userid 65534) id EAD6C2C0056; Sun, 5 Dec 2004 18:38:06 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by null.rsn.bth.se (Postfix) with ESMTP id 6FC002C006F; Sun, 5 Dec 2004 18:38:06 +0100 (CET) Received: from tux.rsn.bth.se (tux.rsn.bth.se [194.47.143.135]) by null.rsn.bth.se (Postfix) with ESMTP id AA2732C0056; Sun, 5 Dec 2004 18:38:05 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by tux.rsn.bth.se (Postfix) with ESMTP id E45B13FA7; Sun, 5 Dec 2004 18:38:05 +0100 (CET) Date: Sun, 5 Dec 2004 18:38:05 +0100 (CET) From: Martin Josefsson X-X-Sender: gandalf@tux.rsn.bth.se To: Lennert Buytenhek Cc: Scott Feldman , jamal , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) In-Reply-To: Message-ID: References: <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <20041201182943.GA14470@xi.wantstofly.org> <20041201213550.GF14470@xi.wantstofly.org> <1101967983.4782.9.camel@localhost.localdomain> <20041205145051.GA647@xi.wantstofly.org> <20041205151545.GC647@xi.wantstofly.org> <20041205170006.GI647@xi.wantstofly.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by amavisd-new-20030616-p10 on null.rsn.bth.se X-archive-position: 12438 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: gandalf@wlug.westbo.se Precedence: bulk X-list: netdev On Sun, 5 Dec 2004, Martin Josefsson wrote: > > -#define E1000_READ_REG(a, reg) ( \ > > - readl((a)->hw_addr + \ > > - (((a)->mac_type >= e1000_82543) ? E1000_##reg : E1000_82542_##reg))) > > +#define E1000_READ_REG(a, reg) ({ \ > > + unsigned long s, e, d, v; \ > > +\ > > + (a)->mmio_reads++; \ > > + rdtsc(s, d); \ > > + v = readl((a)->hw_addr + \ > > + (((a)->mac_type >= e1000_82543) ? E1000_##reg : E1000_82542_##reg)); \ > > + rdtsc(e, d); \ > > + e -= s; \ > > + printk(KERN_INFO "e1000: MMIO read took %ld clocks\n", e); \ > > + printk(KERN_INFO "e1000: in process %d(%s)\n", current->pid, current->comm); \ > > + dump_stack(); \ > > + v; \ > > +}) > > > > You might want to disable the stack dump of course. > > Will test this in a while. It gives pretty varied results. This is during a pktgen run. The machine is an Athlon MP 2000+ which operated at 1667 MHz e1000: MMIO read took 481 clocks e1000: MMIO read took 369 clocks e1000: MMIO read took 481 clocks e1000: MMIO read took 11 clocks e1000: MMIO read took 477 clocks e1000: MMIO read took 316 clocks e1000: MMIO read took 481 clocks e1000: MMIO read took 316 clocks e1000: MMIO read took 480 clocks e1000: MMIO read took 332 clocks e1000: MMIO read took 480 clocks e1000: MMIO read took 372 clocks e1000: MMIO read took 480 clocks e1000: MMIO read took 11 clocks e1000: MMIO read took 481 clocks e1000: MMIO read took 388 clocks e1000: MMIO read took 480 clocks e1000: MMIO read took 11 clocks e1000: MMIO read took 485 clocks e1000: MMIO read took 317 clocks e1000: MMIO read took 481 clocks e1000: MMIO read took 337 clocks e1000: MMIO read took 480 clocks e1000: MMIO read took 316 clocks e1000: MMIO read took 480 clocks e1000: MMIO read took 409 clocks e1000: MMIO read took 480 clocks e1000: MMIO read took 334 clocks e1000: MMIO read took 481 clocks e1000: MMIO read took 316 clocks e1000: MMIO read took 480 clocks e1000: MMIO read took 11 clocks e1000: MMIO read took 505 clocks e1000: MMIO read took 359 clocks e1000: MMIO read took 484 clocks e1000: MMIO read took 337 clocks e1000: MMIO read took 464 clocks e1000: MMIO read took 504 clocks /Martin From buytenh@wantstofly.org Sun Dec 5 09:44:23 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 09:44:27 -0800 (PST) Received: from xi.wantstofly.org (alephnull.demon.nl [212.238.201.82]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB5HiNlA026845 for ; Sun, 5 Dec 2004 09:44:23 -0800 Received: by xi.wantstofly.org (Postfix, from userid 500) id 605F12B0ED; Sun, 5 Dec 2004 18:44:01 +0100 (MET) Date: Sun, 5 Dec 2004 18:44:01 +0100 From: Lennert Buytenhek To: Martin Josefsson Cc: Scott Feldman , jamal , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) Message-ID: <20041205174401.GJ647@xi.wantstofly.org> References: <20041130134600.GA31515@xi.wantstofly.org> <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <20041201182943.GA14470@xi.wantstofly.org> <20041201213550.GF14470@xi.wantstofly.org> <1101967983.4782.9.camel@localhost.localdomain> <20041205145051.GA647@xi.wantstofly.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.1i X-archive-position: 12439 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: buytenh@wantstofly.org Precedence: bulk X-list: netdev On Sun, Dec 05, 2004 at 04:42:34PM +0100, Martin Josefsson wrote: > The delayed TDT updating was a test and currently it delays the first tx'd > packet after a timerrun 1ms. > > Would be interesting to see what other people get with this thing. > Lennert? I took Scott's notxints patch, added the prefetch bits and moved the TDT updating to e1000_clean_tx as you did. Slightly better than before, but not much: 60 1070157 61 1066610 62 1062088 63 991447 64 991546 65 991537 66 991449 67 990857 68 989882 69 991347 Regular TDT updating: 60 1037469 61 1038425 62 1037393 63 993143 64 992156 65 993137 66 992203 67 992165 68 992185 69 988249 --L From buytenh@wantstofly.org Sun Dec 5 09:51:56 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 09:52:00 -0800 (PST) Received: from xi.wantstofly.org (alephnull.demon.nl [212.238.201.82]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB5HptqK027333 for ; Sun, 5 Dec 2004 09:51:55 -0800 Received: by xi.wantstofly.org (Postfix, from userid 500) id 8915E2B0ED; Sun, 5 Dec 2004 18:51:33 +0100 (MET) Date: Sun, 5 Dec 2004 18:51:33 +0100 From: Lennert Buytenhek To: Martin Josefsson Cc: Scott Feldman , jamal , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) Message-ID: <20041205175133.GK647@xi.wantstofly.org> References: <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <20041201182943.GA14470@xi.wantstofly.org> <20041201213550.GF14470@xi.wantstofly.org> <1101967983.4782.9.camel@localhost.localdomain> <20041205145051.GA647@xi.wantstofly.org> <20041205174401.GJ647@xi.wantstofly.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20041205174401.GJ647@xi.wantstofly.org> User-Agent: Mutt/1.4.1i X-archive-position: 12440 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: buytenh@wantstofly.org Precedence: bulk X-list: netdev On Sun, Dec 05, 2004 at 06:44:01PM +0100, Lennert Buytenhek wrote: > On Sun, Dec 05, 2004 at 04:42:34PM +0100, Martin Josefsson wrote: > > > The delayed TDT updating was a test and currently it delays the first tx'd > > packet after a timerrun 1ms. > > > > Would be interesting to see what other people get with this thing. > > Lennert? > > I took Scott's notxints patch, added the prefetch bits and moved the > TDT updating to e1000_clean_tx as you did. > > Slightly better than before, but not much: I've tested all packet sizes now, and delayed TDT updating once per jiffy (instead of once per packet) indeed gives about 25kpps more on 60,61,62 byte packets, and is hardly worth it for bigger packets. --L From gandalf@wlug.westbo.se Sun Dec 5 09:54:33 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 09:54:36 -0800 (PST) Received: from null.rsn.bth.se (postfix@null.rsn.bth.se [194.47.142.3]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB5HsWCS027717 for ; Sun, 5 Dec 2004 09:54:32 -0800 Received: by null.rsn.bth.se (Postfix, from userid 65534) id D26722C0056; Sun, 5 Dec 2004 18:54:08 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by null.rsn.bth.se (Postfix) with ESMTP id 618D92C006F; Sun, 5 Dec 2004 18:54:08 +0100 (CET) Received: from tux.rsn.bth.se (tux.rsn.bth.se [194.47.143.135]) by null.rsn.bth.se (Postfix) with ESMTP id 9FA372C0056; Sun, 5 Dec 2004 18:54:07 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by tux.rsn.bth.se (Postfix) with ESMTP id DF7DA3FA7; Sun, 5 Dec 2004 18:54:07 +0100 (CET) Date: Sun, 5 Dec 2004 18:54:07 +0100 (CET) From: Martin Josefsson X-X-Sender: gandalf@tux.rsn.bth.se To: Lennert Buytenhek Cc: Scott Feldman , jamal , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) In-Reply-To: <20041205175133.GK647@xi.wantstofly.org> Message-ID: References: <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <20041201182943.GA14470@xi.wantstofly.org> <20041201213550.GF14470@xi.wantstofly.org> <1101967983.4782.9.camel@localhost.localdomain> <20041205145051.GA647@xi.wantstofly.org> <20041205174401.GJ647@xi.wantstofly.org> <20041205175133.GK647@xi.wantstofly.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by amavisd-new-20030616-p10 on null.rsn.bth.se X-archive-position: 12441 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: gandalf@wlug.westbo.se Precedence: bulk X-list: netdev On Sun, 5 Dec 2004, Lennert Buytenhek wrote: > I've tested all packet sizes now, and delayed TDT updating once per jiffy > (instead of once per packet) indeed gives about 25kpps more on 60,61,62 > byte packets, and is hardly worth it for bigger packets. Maybe we can't see any real gains here now, I wonder if it has any effect if you have lots of nics on the same bus. I mean, in theory it saves a whole lot of traffic on the bus. /Martin From buytenh@wantstofly.org Sun Dec 5 09:58:40 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 09:58:44 -0800 (PST) Received: from xi.wantstofly.org (alephnull.demon.nl [212.238.201.82]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB5HwdJ9028214 for ; Sun, 5 Dec 2004 09:58:40 -0800 Received: by xi.wantstofly.org (Postfix, from userid 500) id CEB422B0ED; Sun, 5 Dec 2004 18:58:17 +0100 (MET) Date: Sun, 5 Dec 2004 18:58:17 +0100 From: Lennert Buytenhek To: Martin Josefsson Cc: Scott Feldman , jamal , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) Message-ID: <20041205175817.GL647@xi.wantstofly.org> References: <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <20041201182943.GA14470@xi.wantstofly.org> <20041201213550.GF14470@xi.wantstofly.org> <1101967983.4782.9.camel@localhost.localdomain> <20041205145051.GA647@xi.wantstofly.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.1i X-archive-position: 12442 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: buytenh@wantstofly.org Precedence: bulk X-list: netdev On Sun, Dec 05, 2004 at 05:48:34PM +0100, Martin Josefsson wrote: > I tried using both ports on the 82546GB nic. > > delay nodelay > 1CPU 1.95 Mpps 1.76 Mpps > 2CPU 1.60 Mpps 1.44 Mpps I get: delay nodelay 1CPU 1837356 1837330 2CPU 2035060 1947424 So in your case using 2 CPUs degrades performance, in my case it increases it. And TDT delaying/coalescing only improves performance when using 2 CPUs, and even then only slightly (and only for <= 62B packets.) --L From buytenh@wantstofly.org Sun Dec 5 10:14:52 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 10:14:56 -0800 (PST) Received: from xi.wantstofly.org (alephnull.demon.nl [212.238.201.82]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB5IEpkJ028833 for ; Sun, 5 Dec 2004 10:14:51 -0800 Received: by xi.wantstofly.org (Postfix, from userid 500) id 5E6622B0ED; Sun, 5 Dec 2004 19:14:29 +0100 (MET) Date: Sun, 5 Dec 2004 19:14:29 +0100 From: Lennert Buytenhek To: Martin Josefsson Cc: Scott Feldman , jamal , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) Message-ID: <20041205181429.GM647@xi.wantstofly.org> References: <20041201213550.GF14470@xi.wantstofly.org> <1101967983.4782.9.camel@localhost.localdomain> <20041205145051.GA647@xi.wantstofly.org> <20041205151545.GC647@xi.wantstofly.org> <20041205170006.GI647@xi.wantstofly.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.1i X-archive-position: 12443 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: buytenh@wantstofly.org Precedence: bulk X-list: netdev On Sun, Dec 05, 2004 at 06:38:05PM +0100, Martin Josefsson wrote: > e1000: MMIO read took 481 clocks > e1000: MMIO read took 369 clocks > e1000: MMIO read took 481 clocks > e1000: MMIO read took 11 clocks > e1000: MMIO read took 477 clocks > e1000: MMIO read took 316 clocks Interesting. On a 1667MHz CPU, this is around ~0.28us per MMIO read in the worst case. On my hardware (dual Xeon 2.4GHz), the best case I've ever seen was ~0.83us. This alone can make a hell of a difference, esp. for 60B packets. --L From manfred@colorfullife.com Sun Dec 5 10:26:16 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 10:26:21 -0800 (PST) Received: from dbl.q-ag.de (dbl.q-ag.de [213.172.117.3]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB5IQE1U029278 for ; Sun, 5 Dec 2004 10:26:15 -0800 Received: from [127.0.0.2] (dbl [127.0.0.1]) by dbl.q-ag.de (8.12.3/8.12.3/Debian-6.6) with ESMTP id iB5IPmSL015073; Sun, 5 Dec 2004 19:25:49 +0100 Message-ID: <41B352AB.4020700@colorfullife.com> Date: Sun, 05 Dec 2004 19:25:47 +0100 From: Manfred Spraul User-Agent: Mozilla/5.0 (X11; U; Linux i686; fr-FR; rv:1.7.3) Gecko/20040922 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Lennert Buytenhek , Netdev , Martin Josefsson Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) Content-Type: multipart/mixed; boundary="------------090208040808010503040206" X-archive-position: 12444 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: manfred@colorfullife.com Precedence: bulk X-list: netdev This is a multi-part message in MIME format. --------------090208040808010503040206 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Lennert wrote: > A dirty way, yes ;-) Open up e1000_osdep.h and do: > > -#define E1000_READ_REG(a, reg) ( \ > - readl((a)->hw_addr + \ > - (((a)->mac_type >= e1000_82543) ? E1000_##reg : E1000_82542_##reg))) > +#define E1000_READ_REG(a, reg) ({ \ > + unsigned long s, e, d, v; \ > +\ > + (a)->mmio_reads++; \ > + rdtsc(s, d); \ > + v = readl((a)->hw_addr + \ > + (((a)->mac_type >= e1000_82543) ? E1000_##reg : E1000_82542_##reg)); \ > + rdtsc(e, d); \ > + e -= s; \ > + printk(KERN_INFO "e1000: MMIO read took %ld clocks\n", e); \ > + printk(KERN_INFO "e1000: in process %d(%s)\n", current->pid, current->comm); \ > + dump_stack(); \ > + v; \ > +}) Too dirty: rdtsc is not serializing, thus my Opteron happily reorders the read and the rdtsc and reports 9 cycles. Attached is a longer patch that I usually use for microbenchmarks. I get around 506 cycles with it for an Opteron 2 GHz to the nForce 250 Gb nic (i.e. integrated nic in the chipset, just one HT hop): Results - zero - shift 0 40: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 :0 0 63 0 0 0 0 0 0 0 0 0 0 0 0 0 1e0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 :0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 Overflows: 0. Sum: 100 >>>>>>>>>>> benchmark overhead: 82 cycles ** reading register e08920b4 Results - readl - shift 0 240: 0 0 b 0 0 0 0 0 0 0 0 0 32 0 1 1 :0 0 0 0 0 0 a 0 0 0 0 0 0 0 0 0 260: 1a 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 :0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 300: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 :0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 Overflows: 0. Sum: 100 >>>>>>>>>> total: 0x248, i.e. net 506 cycles. -- Manfred --------------090208040808010503040206 Content-Type: text/plain; name="patch-perftest-forcedeth" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="patch-perftest-forcedeth" --- 2.6/drivers/net/forcedeth.c 2004-12-05 16:21:28.000000000 +0100 +++ build-2.6/drivers/net/forcedeth.c 2004-12-05 19:18:24.000000000 +0100 @@ -1500,6 +1500,131 @@ enable_irq(dev->irq); } +int p_shift = 0; + +#define STAT_TABLELEN 16384 +static unsigned long totals[STAT_TABLELEN]; +static unsigned int overflows; + +static unsigned long long stime; +static void start_measure(void) +{ + __asm__ __volatile__ ( + ".align 64\n\t" + "pushal\n\t" + "cpuid\n\t" + "popal\n\t" + "rdtsc\n\t" + "movl %%eax,(%0)\n\t" + "movl %%edx,4(%0)\n\t" + : /* no output */ + : "c"(&stime) + : "eax", "edx", "memory" ); +} + +static void end_measure(void) +{ +static unsigned long long etime; + __asm__ __volatile__ ( + "pushal\n\t" + "cpuid\n\t" + "popal\n\t" + "rdtsc\n\t" + "movl %%eax,(%0)\n\t" + "movl %%edx,4(%0)\n\t" + : /* no output */ + : "c"(&etime) + : "eax", "edx", "memory" ); + { + unsigned long time = (unsigned long)(etime-stime); + time >>= p_shift; + if(time < STAT_TABLELEN) { + totals[time]++; + } else { + overflows++; + } + } +} + +static void clean_buf(void) +{ + memset(totals,0,sizeof(totals)); + overflows = 0; +} + +static void print_line(unsigned long* array) +{ + int i; + for(i=0;i<32;i++) { + if((i%32)==16) + printk(":"); + printk("%lx ",array[i]); + } +} + +static void print_buf(char* caption) +{ + int i, other = 0; + printk("Results - %s - shift %d", + caption, p_shift); + + for(i=0;ioom_kick, jiffies + OOM_REFILL); spin_unlock_irq(&np->lock); + bench_readl(base + NvRegMulticastAddrB); + bench_readl(base + NvRegIrqStatus); return 0; out_drain: drain_ring(dev); --------------090208040808010503040206-- From sfeldma@pobox.com Sun Dec 5 12:32:31 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 12:32:35 -0800 (PST) Received: from orb.pobox.com (orb.pobox.com [207.8.226.5]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB5KWU19005463 for ; Sun, 5 Dec 2004 12:32:31 -0800 Received: from orb (localhost [127.0.0.1]) by orb.pobox.com (Postfix) with ESMTP id 918A22F9DFD; Sun, 5 Dec 2004 15:32:08 -0500 (EST) Received: from [192.168.0.19] (wbar2.sea1-4-5-062-153.sea1.dsl-verizon.net [4.5.62.153]) by orb.sasl.smtp.pobox.com (Postfix) with ESMTP id 144442F9C4F; Sun, 5 Dec 2004 15:32:06 -0500 (EST) Subject: Re: e1000>5.2.30 unstable with InterruptThrottleRate=0 From: Scott Feldman Reply-To: sfeldma@pobox.com To: Peter Kjellstroem Cc: netdev@oss.sgi.com In-Reply-To: References: Content-Type: text/plain Message-Id: <1102278893.3343.116.camel@sfeldma-mobl.dsl-verizon.net> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 (1.4.6-2) Date: Sun, 05 Dec 2004 12:34:53 -0800 Content-Transfer-Encoding: 7bit X-archive-position: 12445 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: sfeldma@pobox.com Precedence: bulk X-list: netdev On Sun, 2004-12-05 at 06:04, Peter Kjellstroem wrote: > I've been doing som more tests today to try to narrow down which patch > causes my problems. So far I have this: > > 2.4.26 (5.2.30) is ok > 2.4.28 (5.4.11) is not > > 2.4.28 with 5.2.30 patched in is ok > 2.4.28 with 5.2.39 (from 2.4.27-pre1) is not > > In e1000_main.c for 5.2.39 (from 2.4.27-pre1) i found this: > > /* Change Log > * > * 5.2.39 3/12/04 > * ... > * o Back out the CSA fix for 82547 as it continues to cause > * systems lock-ups with production systems. Yes, there was a driver "fix" for this problem that has since been pulled out of the production driver because it caused lockups on some systems. I have one of these such systems. Here's the results on my systems with an 82547EI: 5.2.22 lockup 5.2.30.1 lockup 5.2.39 NETDEV reset 5.2.52 NETDEV reset 5.4.11 NETDEV reset For you, with an 82547GI, any driver between 5.2.22 and 5.2.30.1 will work because it has the "fix". See the comment in e1000_intr for these drivers. So I guess you're not out of luck if you use the 5.2.30 driver. You just can't move forward to a newer driver unless you port the "fix" forward. -scott From sfeldma@pobox.com Sun Dec 5 13:10:08 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 13:10:12 -0800 (PST) Received: from orb.pobox.com (orb.pobox.com [207.8.226.5]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB5LA8Ov006419 for ; Sun, 5 Dec 2004 13:10:08 -0800 Received: from orb (localhost [127.0.0.1]) by orb.pobox.com (Postfix) with ESMTP id 184B12F8E28; Sun, 5 Dec 2004 16:09:46 -0500 (EST) Received: from [192.168.0.19] (wbar2.sea1-4-5-062-153.sea1.dsl-verizon.net [4.5.62.153]) by orb.sasl.smtp.pobox.com (Postfix) with ESMTP id 4307A2F9BE6; Sun, 5 Dec 2004 16:09:35 -0500 (EST) Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) From: Scott Feldman Reply-To: sfeldma@pobox.com To: Martin Josefsson Cc: Lennert Buytenhek , jamal , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com In-Reply-To: References: <1101499285.1079.45.camel@jzny.localdomain> <16811.8052.678955.795327@robur.slu.se> <1101821501.1043.43.camel@jzny.localdomain> <20041130134600.GA31515@xi.wantstofly.org> <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <20041201182943.GA14470@xi.wantstofly.org> <20041201213550.GF14470@xi.wantstofly.org> <1101967983.4782.9.camel@localhost.localdomain> <20041205145051.GA647@xi.wantstofly.org> Content-Type: text/plain Message-Id: <1102281141.3343.138.camel@sfeldma-mobl.dsl-verizon.net> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 (1.4.6-2) Date: Sun, 05 Dec 2004 13:12:22 -0800 Content-Transfer-Encoding: 7bit X-archive-position: 12446 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: sfeldma@pobox.com Precedence: bulk X-list: netdev On Sun, 2004-12-05 at 07:03, Martin Josefsson wrote: > BUT if I use the above + prefetching I get this: > > 60 1483890 Ok, proof that we can get to 1.4Mpps! That's the good news. The bad news is prefetching is potentially buggy as pointed out in the freebsd note. Buggy as in the controller may hang. Sorry, I don't have details on what conditions are necessary to cause a hang. Would Martin or Lennert run these test for a longer duration so we can get some data, maybe adding in Rx. It could be that removing the Tx interrupts and descriptor write-backs, prefetching may be ok. I don't know. Intel? Also, wouldn't it be great if someone wrote a document capturing all of the accumulated knowledge for future generations? -scott From buytenh@wantstofly.org Sun Dec 5 13:26:22 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 13:26:26 -0800 (PST) Received: from xi.wantstofly.org (alephnull.demon.nl [212.238.201.82]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB5LQLC3007036 for ; Sun, 5 Dec 2004 13:26:21 -0800 Received: by xi.wantstofly.org (Postfix, from userid 500) id 2CEB42B0ED; Sun, 5 Dec 2004 22:25:59 +0100 (MET) Date: Sun, 5 Dec 2004 22:25:59 +0100 From: Lennert Buytenhek To: Scott Feldman Cc: Martin Josefsson , jamal , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) Message-ID: <20041205212559.GA4338@xi.wantstofly.org> References: <20041130134600.GA31515@xi.wantstofly.org> <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <20041201182943.GA14470@xi.wantstofly.org> <20041201213550.GF14470@xi.wantstofly.org> <1101967983.4782.9.camel@localhost.localdomain> <20041205145051.GA647@xi.wantstofly.org> <1102281141.3343.138.camel@sfeldma-mobl.dsl-verizon.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1102281141.3343.138.camel@sfeldma-mobl.dsl-verizon.net> User-Agent: Mutt/1.4.1i X-archive-position: 12447 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: buytenh@wantstofly.org Precedence: bulk X-list: netdev On Sun, Dec 05, 2004 at 01:12:22PM -0800, Scott Feldman wrote: > Would Martin or Lennert run these test for a longer duration so we can > get some data, maybe adding in Rx. It could be that removing the Tx > interrupts and descriptor write-backs, prefetching may be ok. I don't > know. Intel? What your patch does is (correct me if I'm wrong): - Masking TXDW, effectively preventing it from delivering TXdone ints. - Not setting E1000_TXD_CMD_IDE in the TXD command field, which causes the chip to 'ignore the TIDV' register, which is the 'TX Interrupt Delay Value'. What exactly does this? - Not setting the "Report Packet Sent"/"Report Status" bits in the TXD command field. Is this the equivalent of the TXdone interrupt? Just exactly which bit avoids the descriptor writeback? I'm also a bit worried that only freeing packets 1ms later will mess up socket accounting and such. Any ideas on that? > Also, wouldn't it be great if someone wrote a document capturing all of > the accumulated knowledge for future generations? I'll volunteer for that. --L From cap@nsc.liu.se Sun Dec 5 13:40:06 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 13:40:12 -0800 (PST) Received: from papput.nsc.liu.se (ns2.nsc.liu.se [130.236.101.9]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB5Le5pG007591 for ; Sun, 5 Dec 2004 13:40:06 -0800 Received: from mail.nsc.liu.se (mail.nsc.liu.se [130.236.101.49]) by papput.nsc.liu.se (Postfix) with ESMTP id 3A57B1C31EF; Sun, 5 Dec 2004 22:39:42 +0100 (CET) Date: Sun, 5 Dec 2004 22:39:42 +0100 (CET) From: Peter Kjellstroem To: Scott Feldman Cc: netdev@oss.sgi.com Subject: Re: e1000>5.2.30 unstable with InterruptThrottleRate=0 In-Reply-To: <1102278893.3343.116.camel@sfeldma-mobl.dsl-verizon.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 12448 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: cap@nsc.liu.se Precedence: bulk X-list: netdev On Sun, 5 Dec 2004, Scott Feldman wrote: > On Sun, 2004-12-05 at 06:04, Peter Kjellstroem wrote: > > * > > * 5.2.39 3/12/04 > > * ... > > * o Back out the CSA fix for 82547 as it continues to cause > > * systems lock-ups with production systems. > Yes I found that out and verified it earlier today. The fix in question is basically if'ed irq_enable/disable around a small chunk of code like this: if(hw->mac_type == e1000_82547 || hw->mac_type == e1000_82547_rev_2) e1000_irq_disable(adapter); this is the original fix and triggers for both e1000_82547 (your 547EI right?) and e1000_82547_rev_2 (my 547GI). If your NIC can't stand the fix and mine needs it isn't the obvious solution the following? if(hw->mac_type == e1000_82547_rev_2) e1000_irq_disable(adapter); I put this (and corresponding enable chung) in the current e1000 in 2.4.28 and it works lika a charm (and shouldn't bite your 82547EI). The rev2 part, 82547GI, pci_id 1075. Is present in a vast number of new systems including all (as far as I know) Dell 700, 750, 1750, 1700 and all Supermicro p4SC mobos. Best Regards, Peter > Yes, there was a driver "fix" for this problem that has since been > pulled out of the production driver because it caused lockups on some > systems. I have one of these such systems. Here's the results on my > systems with an 82547EI: > > 5.2.22 lockup > 5.2.30.1 lockup > 5.2.39 NETDEV reset > 5.2.52 NETDEV reset > 5.4.11 NETDEV reset > > For you, with an 82547GI, any driver between 5.2.22 and 5.2.30.1 will > work because it has the "fix". See the comment in e1000_intr for these > drivers. > > So I guess you're not out of luck if you use the 5.2.30 driver. You > just can't move forward to a newer driver unless you port the "fix" > forward. > > -scott > > > -- ------------------------------------------------------------ Peter Kjellstroem | E-mail: cap@nsc.liu.se National Supercomputer Centre | Office: +46(0)13 281492 Linkoeping University | Fax : +46(0)13 282535 SE-581 83 Linkoeping | Sweden | http://www.nsc.liu.se From cap@nsc.liu.se Sun Dec 5 13:52:40 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 13:52:44 -0800 (PST) Received: from papput.nsc.liu.se (ns2.nsc.liu.se [130.236.101.9]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB5LqeOo008099 for ; Sun, 5 Dec 2004 13:52:40 -0800 Received: from mail.nsc.liu.se (mail.nsc.liu.se [130.236.101.49]) by papput.nsc.liu.se (Postfix) with ESMTP id 0E6801C31EF; Sun, 5 Dec 2004 22:52:18 +0100 (CET) Date: Sun, 5 Dec 2004 22:52:18 +0100 (CET) From: Peter Kjellstroem To: Scott Feldman Cc: netdev@oss.sgi.com Subject: Re: e1000>5.2.30 unstable with InterruptThrottleRate=0 In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 12449 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: cap@nsc.liu.se Precedence: bulk X-list: netdev On Sun, 5 Dec 2004, Peter Kjellstroem wrote: > On Sun, 5 Dec 2004, Scott Feldman wrote: > > > On Sun, 2004-12-05 at 06:04, Peter Kjellstroem wrote: > > > * > > > * 5.2.39 3/12/04 > > > * ... > > > * o Back out the CSA fix for 82547 as it continues to cause > > > * systems lock-ups with production systems. > > > > Yes I found that out and verified it earlier today. The fix in question is > basically if'ed irq_enable/disable around a small chunk of code like this: > > if(hw->mac_type == e1000_82547 || hw->mac_type == e1000_82547_rev_2) > e1000_irq_disable(adapter); > > this is the original fix and triggers for both e1000_82547 (your 547EI > right?) and e1000_82547_rev_2 (my 547GI). If your NIC can't stand the fix > and mine needs it isn't the obvious solution the following? > > if(hw->mac_type == e1000_82547_rev_2) > e1000_irq_disable(adapter); > And here's a trivial patch for it (against 2.4.28): /Peter --- linux-2.4.28-cap1p/drivers/net/e1000/e1000_main.c Wed Nov 17 12:54:21 2004 +++ linux-2.4.28-cap5p/drivers/net/e1000/e1000_main.c Sun Dec 5 22:47:57 2004 @@ -2124,10 +2124,19 @@ e1000_intr(int irq, void *data, struct p __netif_rx_schedule(netdev); } #else + /* e1000_irq_disable/enable pair added back (removed in 2.4.27-pre1) */ + /* fixed needed for 82547GI (e1000_82547_rev_2) Dell 750, P4SCi */ + /* reliable operation with ITR=0 */ + if(hw->mac_type == e1000_82547_rev_2) + e1000_irq_disable(adapter); + for(i = 0; i < E1000_MAX_INTR; i++) if(unlikely(!e1000_clean_rx_irq(adapter) & !e1000_clean_tx_irq(adapter))) break; + + if(hw->mac_type == e1000_82547 || hw->mac_type == e1000_82547_rev_2) + e1000_irq_enable(adapter); #endif return IRQ_HANDLED; From kdorn@lilah.hetzel.org Sun Dec 5 14:32:20 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 14:32:29 -0800 (PST) Received: from lilah.hetzel.org (lilah.hetzel.org [199.250.128.2]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB5MWJpF009145 for ; Sun, 5 Dec 2004 14:32:20 -0800 Received: from lilah.hetzel.org (localhost.localdomain [127.0.0.1]) by lilah.hetzel.org (8.12.11/8.12.8) with ESMTP id iB5NtRkj023027; Sun, 5 Dec 2004 18:55:29 -0500 Received: (from kdorn@localhost) by lilah.hetzel.org (8.12.11/8.12.8/Submit) id iB5NtKNP022968; Sun, 5 Dec 2004 18:55:20 -0500 Date: Sun, 5 Dec 2004 18:55:20 -0500 From: Dorn Hetzel To: Francois Romieu Cc: linux-kernel@vger.kernel.org, netdev@oss.sgi.com, jgarzik@pobox.com, dorn@hetzel.org Subject: Re: r8169.c Message-ID: <20041205235519.GA21885@lilah.hetzel.org> References: <20041119162920.GA26836@lilah.hetzel.org> <20041119201203.GA13522@electric-eye.fr.zoreil.com> <20041120003754.GA32133@lilah.hetzel.org> <20041120002946.GA18059@electric-eye.fr.zoreil.com> <20041122181307.GA3625@lilah.hetzel.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20041122181307.GA3625@lilah.hetzel.org> User-Agent: Mutt/1.4i X-archive-position: 12450 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kernel@dorn.hetzel.org Precedence: bulk X-list: netdev On Mon, Nov 22, 2004 at 01:13:07PM -0500, Dorn Hetzel wrote: > On Sat, Nov 20, 2004 at 01:29:46AM +0100, Francois Romieu wrote: > > > Once you have applied one of the patch above, the patch below will improve > > your "transmit timed out" (please apply in order and enable NAPI): > > http://www.fr.zoreil.com/linux/kernel/2.6.x/2.6.10-rc2-mm1/r8169-250.patch > > http://www.fr.zoreil.com/linux/kernel/2.6.x/2.6.10-rc2-mm1/r8169-255.patch > > > > If things perform better you may want to use bigger frames and apply as > > well r8169-260.patch and r8169-265.patch. > > http://www.fr.zoreil.com/linux/kernel/2.6.x/2.6.10-rc2-mm1/r8169-260.patch > > http://www.fr.zoreil.com/linux/kernel/2.6.x/2.6.10-rc2-mm1/r8169-265.patch > > I was wondering if the above 4 patches have made it into one of the rc? releases, or at least a rc?-mm? ? Regards, -Dorn From anton@ozlabs.org Sun Dec 5 15:27:07 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 15:27:17 -0800 (PST) Received: from ozlabs.org (ozlabs.org [203.10.76.45]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB5NR6Sm010454 for ; Sun, 5 Dec 2004 15:27:06 -0800 Received: by ozlabs.org (Postfix, from userid 1010) id C72722BE83; Mon, 6 Dec 2004 10:26:38 +1100 (EST) Date: Mon, 6 Dec 2004 10:22:26 +1100 From: Anton Blanchard To: netdev@oss.sgi.com Cc: ganesh.venkatesan@intel.com, jesse.brandeburg@intel.com, john.ronciak@intel.com Subject: TSO + e1000 Message-ID: <20041205232226.GA5757@krispykreme.ozlabs.ibm.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="ZPt4rx8FFjLCG7dd" Content-Disposition: inline User-Agent: Mutt/1.5.6+20040907i X-archive-position: 12451 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: anton@samba.org Precedence: bulk X-list: netdev --ZPt4rx8FFjLCG7dd Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi, I had another look at our TSO issues. A few things: 1. tcpdump doesnt report local TSO packets, it simply prints bad-len. I think this is because the e1000 driver is zeroing out the IP header length in e1000_tso: skb->nh.iph->tot_len = 0; Does the card require this for TSO to operate? Ive worked around it in tcpdump for the time being. 2. TSO never gets reenabled after a retransmit. With long lasting connections this hurts, we get one retransmit and its all over. Not so important for a web server, but we have big CIFS (windows networking) servers where connections can last days. When its working TSO gives us a nice bump here. 3. Im getting e1000 rx fifo overruns on the receive side. Doubling the flow control watermarks seems to help. Due to bug 2, whenever we get an rx fifo overrun TSO gets disabled on that connection. 4. Im seeing some strange stuff during connection startup. I have bumped win_divisor to 2 (so TSO gets enabled in a reasonable time). There are nice big send and receive socket buffers (512k). Id expect TSO to cut in fairly quickly. The both direction test is as Id expect. It takes a few packets for our slow start window to grow and then we start sending TSO packets (evidenced by the first bad-len) packet. Now if we do a single direction test it takes forever for TSO to kick in. At the moment Im not sure why this is happening, tcp_current_mss seems to grow correctly in both cases. Is there somewhere else we are capping TSO packets? Finally, I set divisor to 1 and you can see for both tests TSO cuts right in. I cant explain the big difference between divisor = 1 and divisor = 2 yet, any thoughts? Anton --ZPt4rx8FFjLCG7dd Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename=divisor_2_both 22:30:12.084166 IP 10.0.0.2.32822 > 10.0.0.1.12865: S 4036823371:4036823371(0) win 5840 22:30:12.084438 IP 10.0.0.1.12865 > 10.0.0.2.32822: S 4047144990:4047144990(0) ack 4036823372 win 5792 22:30:12.084462 IP 10.0.0.2.32822 > 10.0.0.1.12865: . ack 1 win 23 22:30:12.084484 IP 10.0.0.2.32822 > 10.0.0.1.12865: . 1:1449(1448) ack 1 win 23 22:30:12.084491 IP 10.0.0.2.32822 > 10.0.0.1.12865: . 1449:2897(1448) ack 1 win 23 22:30:12.084509 IP 10.0.0.2.32822 > 10.0.0.1.12865: P 2897:4345(1448) ack 1 win 23 22:30:12.084688 IP 10.0.0.1.12865 > 10.0.0.2.32822: . ack 1449 win 34 22:30:12.084690 IP 10.0.0.1.12865 > 10.0.0.2.32822: . ack 2897 win 46 22:30:12.084691 IP 10.0.0.1.12865 > 10.0.0.2.32822: . ack 4345 win 57 22:30:12.084721 IP 10.0.0.2.32822 > 10.0.0.1.12865: . 4345:5793(1448) ack 1 win 23 22:30:12.084726 IP 10.0.0.2.32822 > 10.0.0.1.12865: . 5793:7241(1448) ack 1 win 23 22:30:12.084938 IP 10.0.0.1.12865 > 10.0.0.2.32822: . ack 5793 win 68 22:30:12.084939 IP 10.0.0.1.12865 > 10.0.0.2.32822: . ack 7241 win 80 22:30:12.084975 IP 10.0.0.2.32822 > 10.0.0.1.12865: P 7241:8677(1436) ack 1 win 23 22:30:12.085187 IP 10.0.0.1.12865 > 10.0.0.2.32822: . ack 8677 win 91 22:30:12.085188 IP 10.0.0.1.12865 > 10.0.0.2.32822: . 1:1449(1448) ack 8677 win 91 22:30:12.085189 IP 10.0.0.1.12865 > 10.0.0.2.32822: . 1449:2897(1448) ack 8677 win 91 22:30:12.085190 IP 10.0.0.1.12865 > 10.0.0.2.32822: P 2897:4345(1448) ack 8677 win 91 22:30:12.085218 IP 10.0.0.2.32822 > 10.0.0.1.12865: . ack 1449 win 35 22:30:12.085228 IP 10.0.0.2.32822 > 10.0.0.1.12865: . ack 2897 win 46 22:30:12.085234 IP 10.0.0.2.32822 > 10.0.0.1.12865: . ack 4345 win 57 22:30:12.085438 IP 10.0.0.1.12865 > 10.0.0.2.32822: . 4345:5793(1448) ack 8677 win 91 22:30:12.085439 IP 10.0.0.1.12865 > 10.0.0.2.32822: . 5793:7241(1448) ack 8677 win 91 22:30:12.085464 IP 10.0.0.2.32822 > 10.0.0.1.12865: . ack 5793 win 69 22:30:12.085474 IP 10.0.0.2.32822 > 10.0.0.1.12865: . ack 7241 win 80 22:30:12.085687 IP 10.0.0.1.12865 > 10.0.0.2.32822: P 7241:8677(1436) ack 8677 win 91 22:30:12.085705 IP 10.0.0.2.32822 > 10.0.0.1.12865: . ack 8677 win 91 22:30:12.087676 IP bad-len 0 22:30:12.087687 IP bad-len 0 22:30:12.087937 IP 10.0.0.1.12865 > 10.0.0.2.32822: . ack 14469 win 136 --ZPt4rx8FFjLCG7dd Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="divisor_2_single.wGmSv6" 22:30:12.096111 IP 10.0.0.2.32824 > 10.0.0.1.32823: S 4033033338:4033033338(0) win 5840 22:30:12.096308 IP 10.0.0.1.32823 > 10.0.0.2.32824: S 4044444323:4044444323(0) ack 4033033339 win 5792 22:30:12.096327 IP 10.0.0.2.32824 > 10.0.0.1.32823: . ack 1 win 23 22:30:12.096354 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 1:1449(1448) ack 1 win 23 22:30:12.096362 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 1449:2897(1448) ack 1 win 23 22:30:12.096377 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 2897:4345(1448) ack 1 win 23 22:30:12.096558 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 1449 win 34 22:30:12.096559 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 2897 win 46 22:30:12.096560 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 4345 win 57 22:30:12.096664 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 4345:5793(1448) ack 1 win 23 22:30:12.096670 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 5793:7241(1448) ack 1 win 23 22:30:12.096685 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 7241:8689(1448) ack 1 win 23 22:30:12.096691 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 8689:10137(1448) ack 1 win 23 22:30:12.096706 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 10137:11585(1448) ack 1 win 23 22:30:12.096711 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 11585:13033(1448) ack 1 win 23 22:30:12.096934 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 5793 win 68 22:30:12.096966 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 13033:14481(1448) ack 1 win 23 22:30:12.096970 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 14481:15929(1448) ack 1 win 23 22:30:12.096935 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 7241 win 80 22:30:12.096984 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 15929:17377(1448) ack 1 win 23 22:30:12.096988 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 17377:18825(1448) ack 1 win 23 22:30:12.096936 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 8689 win 91 22:30:12.097004 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 18825:20273(1448) ack 1 win 23 22:30:12.097008 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 20273:21721(1448) ack 1 win 23 22:30:12.096937 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 10137 win 102 22:30:12.097021 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 21721:23169(1448) ack 1 win 23 22:30:12.097026 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 23169:24617(1448) ack 1 win 23 22:30:12.096938 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 11585 win 114 22:30:12.097040 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 24617:26065(1448) ack 1 win 23 22:30:12.097043 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 26065:27513(1448) ack 1 win 23 22:30:12.096938 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 13033 win 125 22:30:12.097060 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 27513:28961(1448) ack 1 win 23 22:30:12.097066 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 28961:30409(1448) ack 1 win 23 22:30:12.097183 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 14481 win 136 22:30:12.097216 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 30409:31857(1448) ack 1 win 23 22:30:12.097221 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 31857:33305(1448) ack 1 win 23 22:30:12.097184 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 15929 win 148 22:30:12.097240 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 33305:34753(1448) ack 1 win 23 22:30:12.097244 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 34753:36201(1448) ack 1 win 23 22:30:12.097185 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 17377 win 159 22:30:12.097258 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 36201:37649(1448) ack 1 win 23 22:30:12.097262 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 37649:39097(1448) ack 1 win 23 22:30:12.097186 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 18825 win 170 22:30:12.097276 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 39097:40545(1448) ack 1 win 23 22:30:12.097280 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 40545:41993(1448) ack 1 win 23 22:30:12.097187 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 20273 win 181 22:30:12.097293 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 41993:43441(1448) ack 1 win 23 22:30:12.097297 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 43441:44889(1448) ack 1 win 23 22:30:12.097187 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 21721 win 193 22:30:12.097311 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 44889:46337(1448) ack 1 win 23 22:30:12.097316 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 46337:47785(1448) ack 1 win 23 22:30:12.097309 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 23169 win 204 22:30:12.097344 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 47785:49233(1448) ack 1 win 23 22:30:12.097348 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 49233:50681(1448) ack 1 win 23 22:30:12.097310 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 24617 win 215 22:30:12.097360 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 50681:52129(1448) ack 1 win 23 22:30:12.097364 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 52129:53577(1448) ack 1 win 23 22:30:12.097311 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 26065 win 227 22:30:12.097375 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 53577:55025(1448) ack 1 win 23 22:30:12.097378 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 55025:56473(1448) ack 1 win 23 22:30:12.097311 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 27513 win 238 22:30:12.097390 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 56473:57921(1448) ack 1 win 23 22:30:12.097393 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 57921:59369(1448) ack 1 win 23 22:30:12.097312 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 28961 win 249 22:30:12.097405 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 59369:60817(1448) ack 1 win 23 22:30:12.097408 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 60817:62265(1448) ack 1 win 23 22:30:12.097313 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 30409 win 261 22:30:12.097419 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 62265:63713(1448) ack 1 win 23 22:30:12.097423 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 63713:65161(1448) ack 1 win 23 22:30:12.097434 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 31857 win 272 22:30:12.097477 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 65161:66609(1448) ack 1 win 23 22:30:12.097482 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 66609:68057(1448) ack 1 win 23 22:30:12.097435 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 33305 win 283 22:30:12.097495 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 68057:69505(1448) ack 1 win 23 22:30:12.097497 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 69505:70953(1448) ack 1 win 23 22:30:12.097435 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 34753 win 295 22:30:12.097510 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 70953:72401(1448) ack 1 win 23 22:30:12.097514 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 72401:73849(1448) ack 1 win 23 22:30:12.097436 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 36201 win 306 22:30:12.097535 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 73849:75297(1448) ack 1 win 23 22:30:12.097538 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 75297:76745(1448) ack 1 win 23 22:30:12.097437 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 37649 win 317 22:30:12.097551 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 76745:78193(1448) ack 1 win 23 22:30:12.097555 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 78193:79641(1448) ack 1 win 23 22:30:12.097437 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 39097 win 329 22:30:12.097575 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 79641:81089(1448) ack 1 win 23 22:30:12.097579 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 81089:82537(1448) ack 1 win 23 22:30:12.097558 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 40545 win 340 22:30:12.097614 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 82537:83985(1448) ack 1 win 23 22:30:12.097619 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 83985:85433(1448) ack 1 win 23 22:30:12.097559 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 41993 win 351 22:30:12.097634 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 85433:86881(1448) ack 1 win 23 22:30:12.097638 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 86881:88329(1448) ack 1 win 23 22:30:12.097561 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 43441 win 362 22:30:12.097650 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 88329:89777(1448) ack 1 win 23 22:30:12.097654 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 89777:91225(1448) ack 1 win 23 22:30:12.097562 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 44889 win 374 22:30:12.097666 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 91225:92673(1448) ack 1 win 23 22:30:12.097669 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 92673:94121(1448) ack 1 win 23 22:30:12.097562 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 46337 win 385 22:30:12.097683 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 94121:95569(1448) ack 1 win 23 22:30:12.097688 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 95569:97017(1448) ack 1 win 23 22:30:12.097563 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 47785 win 396 22:30:12.097704 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 97017:98465(1448) ack 1 win 23 22:30:12.097709 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 98465:99913(1448) ack 1 win 23 22:30:12.097683 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 57921 win 476 22:30:12.097564 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 49233 win 408 22:30:12.097728 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 99913:101361(1448) ack 1 win 23 22:30:12.097733 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 101361:102809(1448) ack 1 win 23 22:30:12.097737 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 102809:104257(1448) ack 1 win 23 22:30:12.097741 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 104257:105705(1448) ack 1 win 23 22:30:12.097744 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 105705:107153(1448) ack 1 win 23 22:30:12.097748 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 107153:108601(1448) ack 1 win 23 22:30:12.097752 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 108601:110049(1448) ack 1 win 23 22:30:12.097755 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 110049:111497(1448) ack 1 win 23 22:30:12.097684 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 59369 win 487 22:30:12.097565 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 50681 win 419 22:30:12.097768 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 111497:112945(1448) ack 1 win 23 22:30:12.097773 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 112945:114393(1448) ack 1 win 23 22:30:12.097685 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 60817 win 498 22:30:12.097565 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 52129 win 430 22:30:12.097786 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 114393:115841(1448) ack 1 win 23 22:30:12.097790 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 115841:117289(1448) ack 1 win 23 22:30:12.097686 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 62265 win 510 22:30:12.097586 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 53577 win 442 22:30:12.097804 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 117289:118737(1448) ack 1 win 23 22:30:12.097809 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 118737:120185(1448) ack 1 win 23 22:30:12.097702 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 63713 win 521 22:30:12.097586 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 55025 win 453 22:30:12.097826 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 120185:121633(1448) ack 1 win 23 22:30:12.097832 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 121633:123081(1448) ack 1 win 23 22:30:12.097703 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 65161 win 532 22:30:12.097587 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 56473 win 464 22:30:12.097809 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 66609 win 543 22:30:12.097849 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 123081:124529(1448) ack 1 win 23 22:30:12.097855 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 124529:125977(1448) ack 1 win 23 22:30:12.097873 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 125977:127425(1448) ack 1 win 23 22:30:12.097881 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 127425:128873(1448) ack 1 win 23 22:30:12.097810 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 68057 win 555 22:30:12.097897 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 128873:130321(1448) ack 1 win 23 22:30:12.097903 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 130321:131769(1448) ack 1 win 23 22:30:12.097811 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 69505 win 566 22:30:12.097915 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 131769:133217(1448) ack 1 win 23 22:30:12.097919 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 133217:134665(1448) ack 1 win 23 22:30:12.097812 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 70953 win 577 22:30:12.097932 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 134665:136113(1448) ack 1 win 23 22:30:12.097940 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 136113:137561(1448) ack 1 win 23 22:30:12.097812 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 72401 win 589 22:30:12.097956 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 137561:139009(1448) ack 1 win 23 22:30:12.097935 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 86881 win 702 22:30:12.097964 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 139009:140457(1448) ack 1 win 23 22:30:12.097813 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 73849 win 600 22:30:12.097986 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 140457:141905(1448) ack 1 win 23 22:30:12.097992 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 141905:143353(1448) ack 1 win 23 22:30:12.097996 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 143353:144801(1448) ack 1 win 23 22:30:12.098000 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 144801:146249(1448) ack 1 win 23 22:30:12.098004 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 146249:147697(1448) ack 1 win 23 22:30:12.098008 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 147697:149145(1448) ack 1 win 23 22:30:12.098011 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 149145:150593(1448) ack 1 win 23 22:30:12.098014 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 150593:152041(1448) ack 1 win 23 22:30:12.098018 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 152041:153489(1448) ack 1 win 23 22:30:12.098026 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 153489:154937(1448) ack 1 win 23 22:30:12.098031 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 154937:156385(1448) ack 1 win 23 22:30:12.097936 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 88329 win 713 22:30:12.097814 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 75297 win 611 22:30:12.098044 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 156385:157833(1448) ack 1 win 23 22:30:12.098048 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 157833:159281(1448) ack 1 win 23 22:30:12.097937 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 89777 win 724 22:30:12.097815 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 76745 win 623 22:30:12.098064 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 159281:160729(1448) ack 1 win 23 22:30:12.098058 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 102809 win 770 22:30:12.098072 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 160729:162177(1448) ack 1 win 23 22:30:12.097938 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 91225 win 736 22:30:12.098089 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 162177:163625(1448) ack 1 win 23 22:30:12.098094 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 163625:165073(1448) ack 1 win 23 22:30:12.098098 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 165073:166521(1448) ack 1 win 23 22:30:12.098101 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 166521:167969(1448) ack 1 win 23 22:30:12.098105 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 167969:169417(1448) ack 1 win 23 22:30:12.098108 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 169417:170865(1448) ack 1 win 23 22:30:12.098112 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 170865:172313(1448) ack 1 win 23 22:30:12.098115 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 172313:173761(1448) ack 1 win 23 22:30:12.098119 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 173761:175209(1448) ack 1 win 23 22:30:12.098123 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 175209:176657(1448) ack 1 win 23 22:30:12.098059 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 114393 win 770 22:30:12.097940 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 92673 win 747 22:30:12.097816 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 78193 win 634 22:30:12.098139 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 176657:178105(1448) ack 1 win 23 22:30:12.098144 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 178105:179553(1448) ack 1 win 23 22:30:12.098147 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 179553:181001(1448) ack 1 win 23 22:30:12.098150 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 181001:182449(1448) ack 1 win 23 22:30:12.098154 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 182449:183897(1448) ack 1 win 23 22:30:12.098157 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 183897:185345(1448) ack 1 win 23 22:30:12.098161 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 185345:186793(1448) ack 1 win 23 22:30:12.098164 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 186793:188241(1448) ack 1 win 23 22:30:12.098168 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 188241:189689(1448) ack 1 win 23 22:30:12.097941 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 94121 win 758 22:30:12.097818 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 79641 win 645 22:30:12.097942 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 95569 win 770 22:30:12.097818 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 81089 win 657 22:30:12.097943 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 97017 win 770 22:30:12.097819 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 82537 win 668 22:30:12.097944 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 98465 win 770 22:30:12.097820 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 83985 win 679 22:30:12.097837 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 85433 win 691 22:30:12.098183 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 127425 win 770 22:30:12.098210 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 189689:191137(1448) ack 1 win 23 22:30:12.098215 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 191137:192585(1448) ack 1 win 23 22:30:12.098217 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 192585:194033(1448) ack 1 win 23 22:30:12.098221 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 194033:195481(1448) ack 1 win 23 22:30:12.098225 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 195481:196929(1448) ack 1 win 23 22:30:12.098229 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 196929:198377(1448) ack 1 win 23 22:30:12.098233 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 198377:199825(1448) ack 1 win 23 22:30:12.098237 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 199825:201273(1448) ack 1 win 23 22:30:12.098240 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 201273:202721(1448) ack 1 win 23 22:30:12.098244 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 202721:204169(1448) ack 1 win 23 22:30:12.098307 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 133217 win 770 22:30:12.098343 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 204169:205617(1448) ack 1 win 23 22:30:12.098349 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 205617:207065(1448) ack 1 win 23 22:30:12.098353 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 207065:208513(1448) ack 1 win 23 22:30:12.098357 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 208513:209961(1448) ack 1 win 23 22:30:12.098360 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 209961:211409(1448) ack 1 win 23 22:30:12.098308 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 136113 win 770 22:30:12.098372 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 211409:212857(1448) ack 1 win 23 22:30:12.098378 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 212857:214305(1448) ack 1 win 23 22:30:12.098381 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 214305:215753(1448) ack 1 win 23 22:30:12.098309 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 139009 win 770 22:30:12.098393 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 215753:217201(1448) ack 1 win 23 22:30:12.098399 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 217201:218649(1448) ack 1 win 23 22:30:12.098402 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 218649:220097(1448) ack 1 win 23 22:30:12.098310 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 141905 win 770 22:30:12.098415 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 220097:221545(1448) ack 1 win 23 22:30:12.098420 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 221545:222993(1448) ack 1 win 23 22:30:12.098423 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 222993:224441(1448) ack 1 win 23 22:30:12.098310 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 144801 win 770 22:30:12.098433 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 156385 win 770 22:30:12.098486 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 224441:225889(1448) ack 1 win 23 22:30:12.098493 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 225889:227337(1448) ack 1 win 23 22:30:12.098497 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 227337:228785(1448) ack 1 win 23 22:30:12.098511 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 228785:230233(1448) ack 1 win 23 22:30:12.098516 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 230233:231681(1448) ack 1 win 23 22:30:12.098520 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 231681:233129(1448) ack 1 win 23 22:30:12.098524 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 233129:234577(1448) ack 1 win 23 22:30:12.098526 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 234577:236025(1448) ack 1 win 23 22:30:12.098530 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 236025:237473(1448) ack 1 win 23 22:30:12.098534 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 237473:238921(1448) ack 1 win 23 22:30:12.098538 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 238921:240369(1448) ack 1 win 23 22:30:12.098542 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 240369:241817(1448) ack 1 win 23 22:30:12.098558 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 167969 win 770 22:30:12.098559 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 170865 win 770 22:30:12.098618 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 241817:243265(1448) ack 1 win 23 22:30:12.098624 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 243265:244713(1448) ack 1 win 23 22:30:12.098628 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 244713:246161(1448) ack 1 win 23 22:30:12.098630 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 246161:247609(1448) ack 1 win 23 22:30:12.098634 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 247609:249057(1448) ack 1 win 23 22:30:12.098639 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 249057:250505(1448) ack 1 win 23 22:30:12.098643 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 250505:251953(1448) ack 1 win 23 22:30:12.098647 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 251953:253401(1448) ack 1 win 23 22:30:12.098650 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 253401:254849(1448) ack 1 win 23 22:30:12.098662 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 254849:256297(1448) ack 1 win 23 22:30:12.098667 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 256297:257745(1448) ack 1 win 23 22:30:12.098670 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 257745:259193(1448) ack 1 win 23 22:30:12.098683 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 186793 win 770 22:30:12.098716 IP 10.0.0.2.32824 > 10.0.0.1.32823: P 259193:260641(1448) ack 1 win 23 22:30:12.098720 IP 10.0.0.2.32824 > 10.0.0.1.32823: . 260641:262089(1448) ack 1 win 23 22:30:12.098724 IP bad-len 0 22:30:12.098728 IP bad-len 0 22:30:12.098732 IP bad-len 0 22:30:12.098808 IP 10.0.0.1.32823 > 10.0.0.2.32824: . ack 195481 win 770 --ZPt4rx8FFjLCG7dd Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename=divisor_1_both 23:11:58.916027 IP 10.0.0.2.32844 > 10.0.0.1.12865: S 2407413166:2407413166(0) win 5840 23:11:58.916302 IP 10.0.0.1.12865 > 10.0.0.2.32844: S 2406390542:2406390542(0) ack 2407413167 win 5792 23:11:58.916320 IP 10.0.0.2.32844 > 10.0.0.1.12865: . ack 1 win 23 23:11:58.916340 IP bad-len 0 23:11:58.916552 IP 10.0.0.1.12865 > 10.0.0.2.32844: . ack 1449 win 34 --ZPt4rx8FFjLCG7dd Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename=divisor_1_single 23:11:58.928030 IP 10.0.0.2.32846 > 10.0.0.1.32845: S 2408171237:2408171237(0) win 5840 23:11:58.928298 IP 10.0.0.1.32845 > 10.0.0.2.32846: S 2406622866:2406622866(0) ack 2408171238 win 5792 23:11:58.928315 IP 10.0.0.2.32846 > 10.0.0.1.32845: . ack 1 win 23 23:11:58.928348 IP bad-len 0 23:11:58.928548 IP 10.0.0.1.32845 > 10.0.0.2.32846: . ack 1449 win 34 --ZPt4rx8FFjLCG7dd-- From romieu@fr.zoreil.com Sun Dec 5 15:39:03 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 15:39:09 -0800 (PST) Received: from fr.zoreil.com (electric-eye.fr.zoreil.com [213.41.134.224]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB5Nd1GC010916 for ; Sun, 5 Dec 2004 15:39:02 -0800 Received: from electric-eye.fr.zoreil.com (localhost.localdomain [127.0.0.1]) by fr.zoreil.com (8.12.10/8.12.1) with ESMTP id iB5Nc1vr030837; Mon, 6 Dec 2004 00:38:01 +0100 Received: (from romieu@localhost) by electric-eye.fr.zoreil.com (8.12.10/8.12.10/Submit) id iB5Nbudk030836; Mon, 6 Dec 2004 00:37:56 +0100 Date: Mon, 6 Dec 2004 00:37:56 +0100 From: Francois Romieu To: Dorn Hetzel Cc: linux-kernel@vger.kernel.org, netdev@oss.sgi.com, jgarzik@pobox.com Subject: Re: r8169.c Message-ID: <20041205233756.GB29236@electric-eye.fr.zoreil.com> References: <20041119162920.GA26836@lilah.hetzel.org> <20041119201203.GA13522@electric-eye.fr.zoreil.com> <20041120003754.GA32133@lilah.hetzel.org> <20041120002946.GA18059@electric-eye.fr.zoreil.com> <20041122181307.GA3625@lilah.hetzel.org> <20041205235519.GA21885@lilah.hetzel.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20041205235519.GA21885@lilah.hetzel.org> User-Agent: Mutt/1.4.1i X-Organisation: Land of Sunshine Inc. X-archive-position: 12452 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: romieu@fr.zoreil.com Precedence: bulk X-list: netdev Dorn Hetzel : [...] > I was wondering if the above 4 patches have made it into one of the > rc? releases, or at least a rc?-mm? ? They need to apply on top of -netdev which is included in -mm. I'll send the patch for inclusion in -mm so there is no need for Jeff to hurry. -- Ueimor From sfeldma@pobox.com Sun Dec 5 16:28:14 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 16:28:18 -0800 (PST) Received: from orb.pobox.com (orb.pobox.com [207.8.226.5]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB60SDT1015396 for ; Sun, 5 Dec 2004 16:28:14 -0800 Received: from orb (localhost [127.0.0.1]) by orb.pobox.com (Postfix) with ESMTP id 5909A2F90B2; Sun, 5 Dec 2004 19:27:51 -0500 (EST) Received: from [192.168.0.19] (wbar2.sea1-4-5-062-153.sea1.dsl-verizon.net [4.5.62.153]) by orb.sasl.smtp.pobox.com (Postfix) with ESMTP id 7177F2F8FDE; Sun, 5 Dec 2004 19:27:47 -0500 (EST) Subject: Re: e1000>5.2.30 unstable with InterruptThrottleRate=0 From: Scott Feldman Reply-To: sfeldma@pobox.com To: Peter Kjellstroem Cc: netdev@oss.sgi.com, ganesh.venkatesan@intel.com, john.ronciak@intel.com In-Reply-To: References: Content-Type: text/plain Message-Id: <1102293033.3343.152.camel@sfeldma-mobl.dsl-verizon.net> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 (1.4.6-2) Date: Sun, 05 Dec 2004 16:30:33 -0800 Content-Transfer-Encoding: 7bit X-archive-position: 12453 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: sfeldma@pobox.com Precedence: bulk X-list: netdev On Sun, 2004-12-05 at 13:39, Peter Kjellstroem wrote: > this is the original fix and triggers for both e1000_82547 (your 547EI > right?) and e1000_82547_rev_2 (my 547GI). If your NIC can't stand the fix > and mine needs it isn't the obvious solution the following? Ganesh/John, what do you think? Is there a clear dividing line between 82547EI and 82547GI wrt this issue? If so, could Peter's patch go in? -scott From sfeldma@pobox.com Sun Dec 5 17:21:12 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 17:21:17 -0800 (PST) Received: from orb.pobox.com (orb.pobox.com [207.8.226.5]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB61LC2n016909 for ; Sun, 5 Dec 2004 17:21:12 -0800 Received: from orb (localhost [127.0.0.1]) by orb.pobox.com (Postfix) with ESMTP id 2371C2F87FB; Sun, 5 Dec 2004 20:20:50 -0500 (EST) Received: from [192.168.0.19] (wbar2.sea1-4-5-062-153.sea1.dsl-verizon.net [4.5.62.153]) by orb.sasl.smtp.pobox.com (Postfix) with ESMTP id F1FEB2FAE6B; Sun, 5 Dec 2004 20:20:38 -0500 (EST) Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) From: Scott Feldman Reply-To: sfeldma@pobox.com To: Lennert Buytenhek Cc: Martin Josefsson , jamal , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com In-Reply-To: <20041205212559.GA4338@xi.wantstofly.org> References: <20041130134600.GA31515@xi.wantstofly.org> <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <20041201182943.GA14470@xi.wantstofly.org> <20041201213550.GF14470@xi.wantstofly.org> <1101967983.4782.9.camel@localhost.localdomain> <20041205145051.GA647@xi.wantstofly.org> <1102281141.3343.138.camel@sfeldma-mobl.dsl-verizon.net> <20041205212559.GA4338@xi.wantstofly.org> Content-Type: text/plain Message-Id: <1102296205.3343.206.camel@sfeldma-mobl.dsl-verizon.net> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 (1.4.6-2) Date: Sun, 05 Dec 2004 17:23:25 -0800 Content-Transfer-Encoding: 7bit X-archive-position: 12454 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: sfeldma@pobox.com Precedence: bulk X-list: netdev On Sun, 2004-12-05 at 13:25, Lennert Buytenhek wrote: > What your patch does is (correct me if I'm wrong): > - Masking TXDW, effectively preventing it from delivering TXdone ints. > - Not setting E1000_TXD_CMD_IDE in the TXD command field, which causes > the chip to 'ignore the TIDV' register, which is the 'TX Interrupt > Delay Value'. What exactly does this? A descriptor with IDE, when written back, starts the Tx delay timers countdown. Never setting IDE means the Tx delay timers never expire. > - Not setting the "Report Packet Sent"/"Report Status" bits in the TXD > command field. Is this the equivalent of the TXdone interrupt? > > Just exactly which bit avoids the descriptor writeback? As the name implies, Report Status (RS) instructs the controller to indicate the status of the descriptor by doing a write-back (DMA) to the descriptor memory. The only status we care about is the "done" indicator. By reading TDH (Tx head), we can figure out where hardware is without reading the status of each descriptor. Since we don't need status, we can turn off RS. > I'm also a bit worried that only freeing packets 1ms later will mess up > socket accounting and such. Any ideas on that? Well the timer solution is less than ideal, and any protocols that are sensitive to getting Tx resources returned by the driver as quickly as possible are not going to be happy. I don't know if 1ms is quick enough. You could eliminate the timer by doing the cleanup first thing in xmit_frame, but then you have two problems: 1) you might end up reading TDH for each send, and that's going to be expensive; 2) calls to xmit_frame might stop, leaving uncleaned work until xmit_frame is called again. -scott From herbert@gondor.apana.org.au Sun Dec 5 17:28:31 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 17:28:39 -0800 (PST) Received: from arnor.apana.org.au (c211-30-229-77.rivrw4.nsw.optusnet.com.au [211.30.229.77]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB61SUeu017379 for ; Sun, 5 Dec 2004 17:28:31 -0800 Received: from gondolin.me.apana.org.au ([192.168.0.6] ident=mail) by arnor.apana.org.au with esmtp (Exim 3.35 #1 (Debian)) id 1Cb7fL-0007rP-00; Mon, 06 Dec 2004 12:27:51 +1100 Received: from herbert by gondolin.me.apana.org.au with local (Exim 3.36 #1 (Debian)) id 1Cb7dy-000588-00; Mon, 06 Dec 2004 12:26:26 +1100 From: Herbert Xu To: anton@samba.org (Anton Blanchard) Subject: Re: TSO + e1000 Cc: netdev@oss.sgi.com, ganesh.venkatesan@intel.com, jesse.brandeburg@intel.com, john.ronciak@intel.com Organization: Core In-Reply-To: <20041205232226.GA5757@krispykreme.ozlabs.ibm.com> X-Newsgroups: apana.lists.os.linux.netdev User-Agent: tin/1.7.4-20040225 ("Benbecula") (UNIX) (Linux/2.4.27-hx-1-686-smp (i686)) Message-Id: Date: Mon, 06 Dec 2004 12:26:26 +1100 X-archive-position: 12455 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: herbert@gondor.apana.org.au Precedence: bulk X-list: netdev Anton Blanchard wrote: > > 1. tcpdump doesnt report local TSO packets, it simply prints bad-len. I > think this is because the e1000 driver is zeroing out the IP header > length in e1000_tso: > > skb->nh.iph->tot_len = 0; > > Does the card require this for TSO to operate? Ive worked around it in > tcpdump for the time being. This is a bug in e1000. Even if it is required it isn't allowed to modify a cloned packet. It'll need to copy it so that other clone users aren't affected. > Now if we do a single direction test it takes forever for TSO to kick > in. At the moment Im not sure why this is happening, tcp_current_mss > seems to grow correctly in both cases. Is there somewhere else we are > capping TSO packets? What call does your application use to do the sending? Cheers, -- Visit Openswan at http://www.openswan.org/ Email: Herbert Xu ~{PmV>HI~} Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt From anton@ozlabs.org Sun Dec 5 17:45:25 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 17:45:31 -0800 (PST) Received: from ozlabs.org (ozlabs.org [203.10.76.45]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB61jOPj018162 for ; Sun, 5 Dec 2004 17:45:25 -0800 Received: by ozlabs.org (Postfix, from userid 1010) id B32292BE83; Mon, 6 Dec 2004 12:44:57 +1100 (EST) Date: Mon, 6 Dec 2004 12:43:49 +1100 From: Anton Blanchard To: Herbert Xu Cc: netdev@oss.sgi.com, ganesh.venkatesan@intel.com, jesse.brandeburg@intel.com, john.ronciak@intel.com Subject: Re: TSO + e1000 Message-ID: <20041206014348.GB8751@krispykreme.ozlabs.ibm.com> References: <20041205232226.GA5757@krispykreme.ozlabs.ibm.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.6+20040907i X-archive-position: 12456 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: anton@samba.org Precedence: bulk X-list: netdev > > Now if we do a single direction test it takes forever for TSO to kick > > in. At the moment Im not sure why this is happening, tcp_current_mss > > seems to grow correctly in both cases. Is there somewhere else we are > > capping TSO packets? > > What call does your application use to do the sending? Just doing big writes: send(4, " !\"#$%&\'()*+,-./0123456789:;<=>?"..., 262144, 0) = 262144 Anton From kfj14802@bb.banban.jp Sun Dec 5 18:02:53 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 18:02:58 -0800 (PST) Received: from MINSHENG-31VW0K ([221.208.58.84]) by oss.sgi.com (8.13.0/8.13.0) with SMTP id iB622poR019191 for ; Sun, 5 Dec 2004 18:02:52 -0800 Date: Sun, 5 Dec 2004 18:02:51 -0800 Message-Id: <200412060202.iB622poR019191@oss.sgi.com> From: "=?iso-2022-jp?B?YTEwLm5ldA==?=" To: "netdev@oss.sgi.com" X-mailer: Super Mailer 9 [en][outlook] Subject: =?iso-2022-jp?B?GyRCO2QkTzxnSVgbKEI=?= MIME-Version: 1.0 Content-Type: text/plain; charset="iso-2022-jp" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4927.1200 X-archive-position: 12457 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: sdf@yahoo.co.jp Precedence: bulk X-list: netdev $B:#G/$N<}3O$O1|MM$HBt;3$*M'C#$K$J$l$?$3$H$G$9!#(B $B$I$3$G!)$b$A$m$s=P2q$$7O$G!&!&!&!&!#(B http://kokosoko.info/okusama/ ****$B%a%k%^%,2r=|(B/$BLd$$9g$o$;(B**** ya_oh_yajp@yahoo.co.jp ******************************* From rayl@mail.com Sun Dec 5 18:45:01 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 18:45:05 -0800 (PST) Received: from ray.lehtiniemi.com (dsl-crow-209-5-162-130-cgy.nucleus.com [209.5.162.130]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB62j0rF020984 for ; Sun, 5 Dec 2004 18:45:00 -0800 Received: by ray.lehtiniemi.com (Postfix, from userid 500) id BBC2111F149; Sun, 5 Dec 2004 19:44:37 -0700 (MST) Date: Sun, 5 Dec 2004 19:44:37 -0700 From: Ray Lehtiniemi To: netdev@oss.sgi.com Subject: how to tune a pair of e1000 cards on intel e7501-based system? Message-ID: <20041206024437.GB7891@mail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline User-Agent: Mutt/1.5.6i X-archive-position: 12458 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: rayl@mail.com Precedence: bulk X-list: netdev hi all i'm trying to understand how to tune a pair of e1000 cards in a server box. the box is a dual xeon 3.06 with hyperthreading, using the intel e7501 chipset, with both cards on hub interface D. the application involves small UDP packets, generally under 300 bytes. i need to maximize the number of packets per second transferred between the two cards. at the moment, i'm looking at the PCI bus in this box to see what might be tweakable. lspci output for the relevant parts is attached below. could anyone give me an idea: - what kind of packets per second i could expect to achieve from this particular system (for small packets) - what parameters i can tweak at the PCI level (or any other level, for that matter...) to achieve that level of performance for example, how could i get my e1000 cards to say '64bit+ 133MHz+' to match the secondary side of the 82870P2 bridge? thank you ------------------------------------------------------------------------- lspci -t ------------------------------------------------------------------------- -[00]-+-00.0 +-00.1 +-04.0-[01-03]--+-1c.0 | +-1d.0-[02]--+-01.0 | | \-02.0 | +-1e.0 | \-1f.0-[03]-- +-1d.0 +-1d.1 +-1d.2 +-1e.0-[04]--+-03.0 | \-06.0 +-1f.0 +-1f.1 \-1f.3 ------------------------------------------------------------------------- lspci -vv (selected items) ------------------------------------------------------------------------- 0000:00:00.0 Host bridge: Intel Corp. E7501 Memory Controller Hub (rev 01) Subsystem: Intel Corp. E7501 Memory Controller Hub Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- Status: Cap+ 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- SERR- TAbort- SERR- Reset- FastB2B- 0000:01:1c.0 PIC: Intel Corp. 82870P2 P64H2 I/OxAPIC (rev 04) (prog-if 20 [IO(X)-APIC]) Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- Status: Cap+ 66Mhz+ UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- TAbort- SERR- Reset- FastB2B- Capabilities: [50] PCI-X bridge device. Secondary Status: 64bit+, 133MHz+, SCD-, USC-, SCO-, SRD- Freq=3 Status: Bus=0 Dev=0 Func=0 64bit- 133MHz- SCD- USC-, SCO-, SRD- : Upstream: Capacity=0, Commitment Limit=0 : Downstream: Capacity=0, Commitment Limit=0 0000:02:01.0 Ethernet controller: Intel Corp. 82544EI Gigabit Ethernet Controller (Copper) (rev 02) Subsystem: Intel Corp. PRO/1000 XT Server Adapter Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- Status: Cap+ 66Mhz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- SERR- [disabled] [size=128K] Capabilities: [dc] Power Management version 2 Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 PME-Enable- DSel=0 DScale=1 PME- Capabilities: [e4] PCI-X non-bridge device. Command: DPERE- ERO+ RBC=0 OST=0 Status: Bus=0 Dev=0 Func=0 64bit- 133MHz- SCD- USC-, DC=simple, DMMRBC=0, DMOST=0, DMCRS=0, RSCEM- Capabilities: [f0] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable- Address: 0000000000000000 Data: 0000 0000:02:02.0 Ethernet controller: Intel Corp. 82544EI Gigabit Ethernet Controller (Copper) (rev 02) Subsystem: Intel Corp. PRO/1000 XT Server Adapter Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- Status: Cap+ 66Mhz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- SERR- [disabled] [size=128K] Capabilities: [dc] Power Management version 2 Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 PME-Enable- DSel=0 DScale=1 PME- Capabilities: [e4] PCI-X non-bridge device. Command: DPERE- ERO+ RBC=0 OST=0 Status: Bus=0 Dev=0 Func=0 64bit- 133MHz- SCD- USC-, DC=simple, DMMRBC=0, DMOST=0, DMCRS=0, RSCEM- Capabilities: [f0] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable- Address: 0000000000000000 Data: 0000 -- ---------------------------------------------------------------------- Ray L From ganesh.venkatesan@intel.com Sun Dec 5 19:22:44 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 19:22:48 -0800 (PST) Received: from orsfmr003.jf.intel.com (fmr18.intel.com [134.134.136.17]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB63Mida022269 for ; Sun, 5 Dec 2004 19:22:44 -0800 Received: from orsfmr100.jf.intel.com (orsfmr100.jf.intel.com [10.7.209.16]) by orsfmr003.jf.intel.com (8.12.10/8.12.10/d: major-outer.mc,v 1.1 2004/09/17 17:50:56 root Exp $) with ESMTP id iB63MEOC023697; Mon, 6 Dec 2004 03:22:14 GMT Received: from orsmsxvs040.jf.intel.com (orsmsxvs040.jf.intel.com [192.168.65.206]) by orsfmr100.jf.intel.com (8.12.10/8.12.10/d: major-inner.mc,v 1.2 2004/09/17 18:05:01 root Exp $) with SMTP id iB63MAAn028769; Mon, 6 Dec 2004 03:22:11 GMT Received: from orsmsx332.amr.corp.intel.com ([192.168.65.60]) by orsmsxvs040.jf.intel.com (SAVSMTP 3.1.2.35) with SMTP id M2004120519221104662 ; Sun, 05 Dec 2004 19:22:11 -0800 Received: from orsmsx408.amr.corp.intel.com ([192.168.65.52]) by orsmsx332.amr.corp.intel.com with Microsoft SMTPSVC(6.0.3790.0); Sun, 5 Dec 2004 19:22:11 -0800 X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Subject: Re: e1000>5.2.30 unstable with InterruptThrottleRate=0 Date: Sun, 5 Dec 2004 19:22:10 -0800 Message-ID: <468F3FDA28AA87429AD807992E22D07E037ED6A3@orsmsx408> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Re: e1000>5.2.30 unstable with InterruptThrottleRate=0 Thread-Index: AcTbQsWACHV+MVeTTAWnjvTW8IBfxg== From: "Venkatesan, Ganesh" To: Cc: , X-OriginalArrivalTime: 06 Dec 2004 03:22:11.0563 (UTC) FILETIME=[C65E9BB0:01C4DB42] X-Scanned-By: MIMEDefang 2.44 Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by oss.sgi.com id iB63Mida022269 X-archive-position: 12459 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: ganesh.venkatesan@intel.com Precedence: bulk X-list: netdev Hi Peter: Thanks for the thorough analysis. I will have to perform a set of tests with your patch included in our latest version of the driver. I will respond to you after I am done with my tests. Ganesh. From sfeldma@pobox.com Sun Dec 5 19:31:57 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 19:32:01 -0800 (PST) Received: from orb.pobox.com (orb.pobox.com [207.8.226.5]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB63VucW022812 for ; Sun, 5 Dec 2004 19:31:57 -0800 Received: from orb (localhost [127.0.0.1]) by orb.pobox.com (Postfix) with ESMTP id 813FE2FBA06; Sun, 5 Dec 2004 22:31:34 -0500 (EST) Received: from [192.168.0.19] (wbar2.sea1-4-5-062-153.sea1.dsl-verizon.net [4.5.62.153]) by orb.sasl.smtp.pobox.com (Postfix) with ESMTP id EA96B2FB9D4; Sun, 5 Dec 2004 22:31:32 -0500 (EST) Subject: Re: how to tune a pair of e1000 cards on intel e7501-based system? From: Scott Feldman Reply-To: sfeldma@pobox.com To: Ray Lehtiniemi Cc: netdev@oss.sgi.com In-Reply-To: <20041206024437.GB7891@mail.com> References: <20041206024437.GB7891@mail.com> Content-Type: text/plain Message-Id: <1102304058.3343.217.camel@sfeldma-mobl.dsl-verizon.net> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 (1.4.6-2) Date: Sun, 05 Dec 2004 19:34:18 -0800 Content-Transfer-Encoding: 7bit X-archive-position: 12460 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: sfeldma@pobox.com Precedence: bulk X-list: netdev On Sun, 2004-12-05 at 18:44, Ray Lehtiniemi wrote: > i'm trying to understand how to tune a pair of e1000 cards in > a server box. the box is a dual xeon 3.06 with hyperthreading, > using the intel e7501 chipset, with both cards on hub interface > D. the application involves small UDP packets, generally under These are 82544 cards hanging of P64H2, so they're running at PCI-X, but at what speed? Run ethtool -d eth | grep Bus. The cards support 133Mhz, but having them adjacent on the same P64H2 probably bumps them down to 100Mhz. Can you put one on D and the other on another bus? > 300 bytes. i need to maximize the number of packets per second > transferred between the two cards. There is a lot of current traffic on netdev about this topic. netdev is the official e1000 mailing this weekend. :-) > at the moment, i'm looking at the PCI bus in this box to see > what might be tweakable. lspci output for the relevant parts is > attached below. > > could anyone give me an idea: > > - what kind of packets per second i could expect to achieve > from this particular system (for small packets) What kind of numbers are you getting? What kernel are you using? What driver tweaks have you made, if any? -scott From rayl@mail.com Sun Dec 5 20:10:26 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 20:10:30 -0800 (PST) Received: from ray.lehtiniemi.com (dsl-crow-209-5-162-130-cgy.nucleus.com [209.5.162.130]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB64APdC024075 for ; Sun, 5 Dec 2004 20:10:26 -0800 Received: by ray.lehtiniemi.com (Postfix, from userid 500) id 5DB4114D73B; Sun, 5 Dec 2004 21:10:03 -0700 (MST) Date: Sun, 5 Dec 2004 21:10:03 -0700 From: Ray Lehtiniemi To: Scott Feldman Cc: netdev@oss.sgi.com Subject: Re: how to tune a pair of e1000 cards on intel e7501-based system? Message-ID: <20041206041002.GC7891@mail.com> References: <20041206024437.GB7891@mail.com> <1102304058.3343.217.camel@sfeldma-mobl.dsl-verizon.net> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: <1102304058.3343.217.camel@sfeldma-mobl.dsl-verizon.net> User-Agent: Mutt/1.5.6i X-archive-position: 12461 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: rayl@mail.com Precedence: bulk X-list: netdev On Sun, Dec 05, 2004 at 07:34:18PM -0800, Scott Feldman wrote: > On Sun, 2004-12-05 at 18:44, Ray Lehtiniemi wrote: > > i'm trying to understand how to tune a pair of e1000 cards in > > a server box. the box is a dual xeon 3.06 with hyperthreading, > > using the intel e7501 chipset, with both cards on hub interface > > D. the application involves small UDP packets, generally under > > These are 82544 cards hanging of P64H2, so they're running at PCI-X, but > at what speed? Run ethtool -d eth | grep Bus. :-) just found ethtool and compiled it before i read this email. any other useful tools i should get? > The cards support > 133Mhz, but having them adjacent on the same P64H2 probably bumps them > down to 100Mhz. # ethtool -d eth0 | grep Bus Bus type: PCI-X Bus speed: 133MHz Bus width: 64-bit # ethtool -d eth1 | grep Bus Bus type: PCI-X Bus speed: 133MHz Bus width: 64-bit so that looks okay... any idea why lspci -vv shows non-64bit, non-133 MHz? (i am assuming that is what the minus sign means) Capabilities: [e4] PCI-X non-bridge device. Command: DPERE- ERO+ RBC=0 OST=0 Status: Bus=0 Dev=0 Func=0 64bit- 133MHz- SCD- USC-, DC=simple, DMMRBC=0, DMOST=0, DMCRS=0, RSCEM- > Can you put one on D and the other on another bus? not sure... have to look at the chassis tomorrow morning. a co-worker actually built the box, i've not seen it in person yet. > There is a lot of current traffic on netdev about this topic. netdev is > the official e1000 mailing this weekend. :-) i noticed that almost half of the messages in the e1000-devel archives since oct 2002 were sent in the last 10 days :-) > What kind of numbers are you getting? will start testing tomorrow, just starting my background research tonight > What kernel are you using? 2.4.20 with driver 4.4.12-k1, and 2.6.bkcurr with driver 5.bkcurr > What driver tweaks have you made, if any? none yet. based on mailing list searches, i plan to: - InterruptThrottleRate 15000 - TxIntDelay 0 i also plan to set rp_filter to zero for all interfaces. thanks -- ---------------------------------------------------------------------- Ray L From anton@ozlabs.org Sun Dec 5 20:17:56 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 20:17:59 -0800 (PST) Received: from ozlabs.org (ozlabs.org [203.10.76.45]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB64HtJm024509 for ; Sun, 5 Dec 2004 20:17:55 -0800 Received: by ozlabs.org (Postfix, from userid 1010) id 662DA2BDB5; Mon, 6 Dec 2004 15:17:28 +1100 (EST) Date: Mon, 6 Dec 2004 15:16:56 +1100 From: Anton Blanchard To: Herbert Xu Cc: netdev@oss.sgi.com, ganesh.venkatesan@intel.com, jesse.brandeburg@intel.com, john.ronciak@intel.com Subject: Re: TSO + e1000 Message-ID: <20041206041656.GE8751@krispykreme.ozlabs.ibm.com> References: <20041205232226.GA5757@krispykreme.ozlabs.ibm.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.6+20040907i X-archive-position: 12462 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: anton@samba.org Precedence: bulk X-list: netdev > This is a bug in e1000. Even if it is required it isn't allowed to > modify a cloned packet. It'll need to copy it so that other clone > users aren't affected. It looks like the tg3 is doing a similar thing. Anton From davem@davemloft.net Sun Dec 5 21:29:06 2004 Received: with ECARTIS (v1.0.0; list netdev); Sun, 05 Dec 2004 21:29:11 -0800 (PST) Received: from cheetah.davemloft.net (mail@adsl-63-197-226-105.dsl.snfc21.pacbell.net [63.197.226.105]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB65T5km029787 for ; Sun, 5 Dec 2004 21:29:06 -0800 Received: from localhost ([127.0.0.1] helo=cheetah.davemloft.net ident=davem) by cheetah.davemloft.net with smtp (Exim 3.36 #1 (Debian)) id 1CbBGB-0005NL-00; Sun, 05 Dec 2004 21:18:07 -0800 Date: Sun, 5 Dec 2004 21:18:07 -0800 From: "David S. Miller" To: Anton Blanchard Cc: herbert@gondor.apana.org.au, netdev@oss.sgi.com, ganesh.venkatesan@intel.com, jesse.brandeburg@intel.com, john.ronciak@intel.com Subject: Re: TSO + e1000 Message-Id: <20041205211807.6f8622f6.davem@davemloft.net> In-Reply-To: <20041206041656.GE8751@krispykreme.ozlabs.ibm.com> References: <20041205232226.GA5757@krispykreme.ozlabs.ibm.com> <20041206041656.GE8751@krispykreme.ozlabs.ibm.com> X-Mailer: Sylpheed version 1.0.0beta3 (GTK+ 1.2.10; sparc-unknown-linux-gnu) X-Face: "_;p5u5aPsO,_Vsx"^v-pEq09'CU4&Dc1$fQExov$62l60cgCc%FnIwD=.UF^a>?5'9Kn[;433QFVV9M..2eN.@4ZWPGbdi<=?[:T>y?SD(R*-3It"Vj:)"dP Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-archive-position: 12463 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: davem@davemloft.net Precedence: bulk X-list: netdev On Mon, 6 Dec 2004 15:16:56 +1100 Anton Blanchard wrote: > > This is a bug in e1000. Even if it is required it isn't allowed to > > modify a cloned packet. It'll need to copy it so that other clone > > users aren't affected. > > It looks like the tg3 is doing a similar thing. As does ixgb. Most TSO drivers need to modify the IP header in a similar way. It has to do with how Microsoft's driver API defines the TSO interface, which is what all the cards implement. They want the checksum field clear, and the tot_len field of the IP header to be what the normal packets will have. Typhoon and S2IO seem to be a notable exceptions. From michael.vittrup.larsen@ericsson.com Mon Dec 6 00:18:47 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 00:18:56 -0800 (PST) Received: from penguin.ericsson.se (penguin.ericsson.se [193.180.251.47]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB68Ifae001838 for ; Mon, 6 Dec 2004 00:18:42 -0800 Received: from esealmw143.al.sw.ericsson.se ([153.88.254.118]) by penguin.ericsson.se (8.12.10/8.12.10/WIREfire-1.8b) with ESMTP id iB68IJh5029400 for ; Mon, 6 Dec 2004 09:18:19 +0100 (MET) Received: from esealnt613.al.sw.ericsson.se ([153.88.254.125]) by esealmw143.al.sw.ericsson.se with Microsoft SMTPSVC(6.0.3790.211); Mon, 6 Dec 2004 09:18:18 +0100 Received: from unixmail.ted.dk.eu.ericsson.se (unixmail.ted.ericsson.se [213.159.188.246]) by esealnt613.al.sw.ericsson.se with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2657.72) id X8A3Y20H; Mon, 6 Dec 2004 09:18:18 +0100 Received: from diadem.ted.dk.eu.ericsson.se (diadem.ted.ericsson.se [213.159.189.76]) by unixmail.ted.dk.eu.ericsson.se (8.10.1/8.10.1/TEDmain-1.0) with ESMTP id iB68I5314048; Mon, 6 Dec 2004 09:18:10 +0100 (MET) X-Sybari-Space: 00000000 00000000 00000000 00000000 From: Michael Vittrup Larsen Organization: Ericsson To: Stephen Hemminger Subject: Re: [PATCH] tcp: efficient port randomisation (revised) Date: Mon, 6 Dec 2004 09:18:04 +0100 User-Agent: KMail/1.7 Cc: "David S. Miller" , netdev@oss.sgi.com References: <20041027092531.78fe438c@guest-251-240.pdx.osdl.net> <20041202135252.04e64f51.davem@davemloft.net> <41B14E57.5080803@osdl.org> In-Reply-To: <41B14E57.5080803@osdl.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200412060918.04441.michael.vittrup.larsen@ericsson.com> X-OriginalArrivalTime: 06 Dec 2004 08:18:18.0610 (UTC) FILETIME=[245AD520:01C4DB6C] X-archive-position: 12464 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: michael.vittrup.larsen@ericsson.com Precedence: bulk X-list: netdev Measuring non-blocking connect using the loopback address I agree with Stephen' conclusion, that the cost of the MD4 gets lost in the noise. I did not measure the variance, since this probably describe scheduling and not the actual ephemeral bind. However, I did measure the minimum value, and the assumption is that this is close to a measurement of an uninterrupted connect. Median filtering probably would be more correct... Below are the results from 10 successive tests. Unmodified (average and minimum values): connect 24433 (min 21820) [ticks/op] connect 24504 (min 21927) [ticks/op] connect 24530 (min 21952) [ticks/op] connect 24244 (min 21607) [ticks/op] connect 24220 (min 21613) [ticks/op] connect 24117 (min 21665) [ticks/op] connect 24148 (min 21663) [ticks/op] connect 24079 (min 21648) [ticks/op] connect 23998 (min 21700) [ticks/op] connect 23906 (min 21682) [ticks/op] Modified (average and minimum values): connect 23961 (min 21774) [ticks/op] connect 23894 (min 21750) [ticks/op] connect 23927 (min 21776) [ticks/op] connect 23881 (min 21757) [ticks/op] connect 23956 (min 21749) [ticks/op] connect 23872 (min 21710) [ticks/op] connect 23848 (min 21694) [ticks/op] connect 23729 (min 21769) [ticks/op] connect 23656 (min 21618) [ticks/op] connect 23723 (min 21699) [ticks/op] From jos@xos037.xos.nl Mon Dec 6 01:43:08 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 01:43:13 -0800 (PST) Received: from simba.xos.nl (simba.xos.nl [212.26.207.226]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB69h6XZ014912 for ; Mon, 6 Dec 2004 01:43:07 -0800 Received: from xos037.xos.nl (xos037.xos.nl [212.26.207.37]) by simba.xos.nl (8.12.8/8.12.8) with ESMTP id iB69gdmA005761; Mon, 6 Dec 2004 10:42:40 +0100 Received: (from jos@localhost) by xos037.xos.nl (8.11.0/8.11.0) id iB69gdN24177; Mon, 6 Dec 2004 10:42:39 +0100 Date: Mon, 6 Dec 2004 10:42:39 +0100 From: Jos Vos To: Scott Feldman Cc: netdev@oss.sgi.com Subject: Re: e1000 driver problem with Intel Pro/1000 MT adapter Message-ID: <20041206104239.A24173@xos037.xos.nl> References: <200412032325.iB3NPdG04693@xos037.xos.nl> <1102204801.3343.68.camel@sfeldma-mobl.dsl-verizon.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <1102204801.3343.68.camel@sfeldma-mobl.dsl-verizon.net>; from sfeldma@pobox.com on Sat, Dec 04, 2004 at 04:00:01PM -0800 X-Organization: X/OS Experts in Open Systems BV, Amsterdam, The Netherlands X-archive-position: 12465 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jos@xos.nl Precedence: bulk X-list: netdev On Sat, Dec 04, 2004 at 04:00:01PM -0800, Scott Feldman wrote: > The 4-port card has two 82546 dual-port controllers behind a PCI-X > bridge. Your symptoms suggest interrupt routing didn't get setup > correctly for the controllers behind the bridge. I've seen more than > one case where the BIOS gets this wrong. The fix is to upgrade to the > latest BIOS. Hopefully this is the fix for you. :-) Unfortunately, the latest BIOS for the MB is already installed :-(. -- -- Jos Vos -- X/OS Experts in Open Systems BV | Phone: +31 20 6938364 -- Amsterdam, The Netherlands | Fax: +31 20 6948204 From hadi@cyberus.ca Mon Dec 6 03:01:49 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 03:01:54 -0800 (PST) Received: from mx03.cybersurf.com (mx03.cybersurf.com [209.197.145.106]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB6B1nkg022663 for ; Mon, 6 Dec 2004 03:01:49 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx03.cybersurf.com with esmtp (Exim 4.30) id 1CbGcO-0007qB-CP for netdev@oss.sgi.com; Mon, 06 Dec 2004 06:01:24 -0500 Received: from cpe0030ab124d2f-cm014500000962.cpe.net.cable.rogers.com ([24.103.99.32] helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1CbGcJ-0008GI-VJ; Mon, 06 Dec 2004 06:01:20 -0500 Subject: Post Network dev questions to netdev Please WAS(Re: [patch 4/10] s390: network driver. From: jamal Reply-To: hadi@cyberus.ca To: Paul Jakma Cc: Thomas Spatzier , jgarzik@pobox.com, linux-kernel@vger.kernel.org, netdev@oss.sgi.com In-Reply-To: References: Content-Type: text/plain Organization: jamalopolous Message-Id: <1102330877.1042.2187.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 06 Dec 2004 06:01:18 -0500 Content-Transfer-Encoding: 7bit X-archive-position: 12466 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev On Sun, 2004-12-05 at 01:25, Paul Jakma wrote: > > This has always been (AFAIK) the behaviour yes. We started getting > reports of the new queuing behaviour with, iirc, a version of Intel's > e100 driver for 2.4.2x, which was later changed back to the old > behaviour. However now that the queue behaviour is apparently the > mandated behaviour we really need to work out what to do about the > sending-long-stale packets problem. > I missed the beginings of this thread. Seems some patch was posted on lkml which started this discussion. I am pretty sure what the lkml FAQ says is to post on netdev. Ok, If you insist posting on lkml (because that the way to glory, good fortune and fame), then please have the courtesy to post to netdev. Now lets see if we can help. Followups only on netdev. cheers, jamal From hadi@cyberus.ca Mon Dec 6 03:28:30 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 03:28:35 -0800 (PST) Received: from mx03.cybersurf.com (mx03.cybersurf.com [209.197.145.106]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB6BST71025052 for ; Mon, 6 Dec 2004 03:28:29 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx03.cybersurf.com with esmtp (Exim 4.30) id 1CbH2C-0002fe-Rc for netdev@oss.sgi.com; Mon, 06 Dec 2004 06:28:04 -0500 Received: from cpe0030ab124d2f-cm014500000962.cpe.net.cable.rogers.com ([24.103.99.32] helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1CbH29-0002WO-Lu; Mon, 06 Dec 2004 06:28:01 -0500 Subject: Re: [patch 4/10] s390: network driver. From: jamal Reply-To: hadi@cyberus.ca To: Paul Jakma Cc: Thomas Spatzier , jgarzik@pobox.com, linux-kernel@vger.kernel.org, netdev@oss.sgi.com In-Reply-To: References: Content-Type: text/plain Organization: jamalopolous Message-Id: <1102332479.1048.2243.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 06 Dec 2004 06:27:59 -0500 Content-Transfer-Encoding: 7bit X-archive-position: 12467 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev On Sun, 2004-12-05 at 01:25, Paul Jakma wrote: [..] > Anyway, we do, I think, need some way to deal with the > sending-stale-packet-on-link-back problem. Either a way to flush this > driver queue or else a guarantee that writes to sockets whose > protocol makes no reliability guarantee will either return ENOBUFS or > drop the packet. > > Otherwise we will start getting reports of "Quagga on Linux sent an > ancient {RIP,IRDP,RA} packet when we fixed a switch problem, and it > caused an outage for a section of our network due to bad routes", I > think. > > Some comment or advice would be useful. (Am I kill-filed by all of > netdev? feels like it). Dont post networking related patches on other lists. I havent seen said patch, but it seems someone is complaining about some behavior changing? In regards to link down and packets being queued. Agreed this is a little problematic for some apps/transports. TCP is definetely not one of them. TCP in Linux actually is told if the drop is local. This way it can make better judgement (and not unnecesarily adjust windows for example). SCTP AFAIK is the only transport that provides its apps opportunity to obsolete messages already sent. I am not sure how well implemented or whtether it is implemented at all. Someone working on SCTP could comment. In the case the netdevice is administratively downed both the qdisc and DMA ring packets are flushed. Newer packets will never be queued and you should quickly be able to find from your app that the device is down. In the case of netdevice being operationally down - I am hoping this is what the discussion is, having jumped on it - both queues stay intact. What you can do is certainly from user space admin down/up the device when you receive a netlink carrier off notification. I am struggling to see whether dropping the packet inside the tx path once it is operationaly down is so blasphemous ... need to get caffeine first. cheers, jamal From hadi@cyberus.ca Mon Dec 6 03:33:12 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 03:33:17 -0800 (PST) Received: from mx01.cybersurf.com (mx01.cybersurf.com [209.197.145.104]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB6BXBfG025481 for ; Mon, 6 Dec 2004 03:33:11 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx01.cybersurf.com with esmtp (Exim 4.30) id 1CbH6k-0007wP-Sw for netdev@oss.sgi.com; Mon, 06 Dec 2004 06:32:46 -0500 Received: from cpe0030ab124d2f-cm014500000962.cpe.net.cable.rogers.com ([24.103.99.32] helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1CbH6d-00033c-OH; Mon, 06 Dec 2004 06:32:40 -0500 Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) From: jamal Reply-To: hadi@cyberus.ca To: Martin Josefsson Cc: Lennert Buytenhek , Scott Feldman , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com In-Reply-To: References: <1101824754.1044.126.camel@jzny.localdomain> <20041201001107.GE4203@xi.wantstofly.org> <1101863399.4663.54.camel@sfeldma-mobl.dsl-verizon.net> <20041201182943.GA14470@xi.wantstofly.org> <20041201213550.GF14470@xi.wantstofly.org> <1101967983.4782.9.camel@localhost.localdomain> <20041205145051.GA647@xi.wantstofly.org> <20041205174401.GJ647@xi.wantstofly.org> <20041205175133.GK647@xi.wantstofly.org> Content-Type: text/plain Organization: jamalopolous Message-Id: <1102332757.1042.2255.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 06 Dec 2004 06:32:37 -0500 Content-Transfer-Encoding: 7bit X-archive-position: 12468 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev On Sun, 2004-12-05 at 12:54, Martin Josefsson wrote: > On Sun, 5 Dec 2004, Lennert Buytenhek wrote: > > > I've tested all packet sizes now, and delayed TDT updating once per jiffy > > (instead of once per packet) indeed gives about 25kpps more on 60,61,62 > > byte packets, and is hardly worth it for bigger packets. > > Maybe we can't see any real gains here now, I wonder if it has any effect > if you have lots of nics on the same bus. I mean, in theory it saves a > whole lot of traffic on the bus. > This sounds like really exciting stuff happening here over the weekend. Scott, you had to leave Intel before giving us this tip? ;-> Someone correct me if i am wrong - but does it appear as if all these changes are only useful on PCI but not PCI-X? cheers, jamal From hadi@cyberus.ca Mon Dec 6 03:41:14 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 03:41:20 -0800 (PST) Received: from mx01.cybersurf.com (mx01.cybersurf.com [209.197.145.104]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB6BfEPc026043 for ; Mon, 6 Dec 2004 03:41:14 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx01.cybersurf.com with esmtp (Exim 4.30) id 1CbHEX-0003qW-KU for netdev@oss.sgi.com; Mon, 06 Dec 2004 06:40:49 -0500 Received: from cpe0030ab124d2f-cm014500000962.cpe.net.cable.rogers.com ([24.103.99.32] helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1CbHES-000442-NL; Mon, 06 Dec 2004 06:40:45 -0500 Subject: Re: [PATCH] rtnetlink & address family problem From: jamal Reply-To: hadi@cyberus.ca To: Michal Ludvig Cc: Andrew Morton , netdev@oss.sgi.com, Jan Kara In-Reply-To: <41B0A5B4.6060108@suse.cz> References: <41B0A5B4.6060108@suse.cz> Content-Type: text/plain Organization: jamalopolous Message-Id: <1102333242.1048.2268.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 06 Dec 2004 06:40:43 -0500 Content-Transfer-Encoding: 7bit X-archive-position: 12469 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev I think this patch will break more than it fixes. You need to do a lot more testing to verify it doesnt. Actually you should probably fix whats being invoked for the ifa messages when PF_UNSPEC is selected to check that it only flushes v6 addresses when V6 is on and reject when it is not compiled in. cheers, jamal On Fri, 2004-12-03 at 12:43, Michal Ludvig wrote: > Hi, > > running 'ip -6 addr flush dev eth0' on a kernel without IPv6 support > flushes *all* addresses from the interface, even those IPv4 ones, > because the unsupported protocol is substituted by PF_UNSPEC. > IMHO it should better return with an error EAFNOSUPPORT. > > Attached patch fixes it. Please apply. > > BTW Credits to Jan Kara for discovering and analysing > this bug. > > Michal Ludvig From jos@xos037.xos.nl Mon Dec 6 03:52:24 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 03:52:29 -0800 (PST) Received: from simba.xos.nl (simba.xos.nl [212.26.207.226]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB6BqK0m026693 for ; Mon, 6 Dec 2004 03:52:23 -0800 Received: from xos037.xos.nl (xos037.xos.nl [212.26.207.37]) by simba.xos.nl (8.12.8/8.12.8) with ESMTP id iB6BpqmA006528; Mon, 6 Dec 2004 12:51:52 +0100 Received: (from jos@localhost) by xos037.xos.nl (8.11.0/8.11.0) id iB6BppN24812; Mon, 6 Dec 2004 12:51:51 +0100 Date: Mon, 6 Dec 2004 12:51:51 +0100 From: Jos Vos To: Scott Feldman Cc: netdev@oss.sgi.com Subject: Re: e1000 driver problem with Intel Pro/1000 MT adapter Message-ID: <20041206125151.A24753@xos037.xos.nl> References: <200412032325.iB3NPdG04693@xos037.xos.nl> <1102204801.3343.68.camel@sfeldma-mobl.dsl-verizon.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <1102204801.3343.68.camel@sfeldma-mobl.dsl-verizon.net>; from sfeldma@pobox.com on Sat, Dec 04, 2004 at 04:00:01PM -0800 X-Organization: X/OS Experts in Open Systems BV, Amsterdam, The Netherlands X-archive-position: 12470 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jos@xos.nl Precedence: bulk X-list: netdev On Sat, Dec 04, 2004 at 04:00:01PM -0800, Scott Feldman wrote: > The 4-port card has two 82546 dual-port controllers behind a PCI-X > bridge. Your symptoms suggest interrupt routing didn't get setup > correctly for the controllers behind the bridge. I've seen more than > one case where the BIOS gets this wrong. The fix is to upgrade to the > latest BIOS. Hopefully this is the fix for you. :-) I just a prerelease of their next BIOS from Supermicro and that seems to solve the problem! I still have to boot the SMP kernel to get APIC working, in UP mode incoming packets are not seen, but I can live with that. -- -- Jos Vos -- X/OS Experts in Open Systems BV | Phone: +31 20 6938364 -- Amsterdam, The Netherlands | Fax: +31 20 6948204 From buytenh@wantstofly.org Mon Dec 6 04:12:14 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 04:12:21 -0800 (PST) Received: from xi.wantstofly.org (alephnull.demon.nl [212.238.201.82]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB6CCDmY028855 for ; Mon, 6 Dec 2004 04:12:14 -0800 Received: by xi.wantstofly.org (Postfix, from userid 500) id 1C6342B0ED; Mon, 6 Dec 2004 13:11:51 +0100 (MET) Date: Mon, 6 Dec 2004 13:11:51 +0100 From: Lennert Buytenhek To: jamal Cc: Martin Josefsson , Scott Feldman , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) Message-ID: <20041206121151.GA12598@xi.wantstofly.org> References: <20041201182943.GA14470@xi.wantstofly.org> <20041201213550.GF14470@xi.wantstofly.org> <1101967983.4782.9.camel@localhost.localdomain> <20041205145051.GA647@xi.wantstofly.org> <20041205174401.GJ647@xi.wantstofly.org> <20041205175133.GK647@xi.wantstofly.org> <1102332757.1042.2255.camel@jzny.localdomain> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1102332757.1042.2255.camel@jzny.localdomain> User-Agent: Mutt/1.4.1i X-archive-position: 12471 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: buytenh@wantstofly.org Precedence: bulk X-list: netdev On Mon, Dec 06, 2004 at 06:32:37AM -0500, jamal wrote: > Someone correct me if i am wrong - but does it appear as if all these > changes are only useful on PCI but not PCI-X? They are useful on PCI-X as well as regular PCI. On my 64/100 NIC I get ~620kpps on 2.6.9, ~1Mpps with 2.6.9 plus tx rework plus TXDMAC=0. Martin gets the ~1Mpps number with just the tx rework, and even more with TXDMAC=0 added in as well. --L From hadi@cyberus.ca Mon Dec 6 04:21:14 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 04:21:20 -0800 (PST) Received: from mx02.cybersurf.com (mx02.cybersurf.com [209.197.145.105]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB6CLElk004437 for ; Mon, 6 Dec 2004 04:21:14 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx02.cybersurf.com with esmtp (Exim 4.30) id 1CbInN-0004GP-Vh for netdev@oss.sgi.com; Mon, 06 Dec 2004 08:20:53 -0500 Received: from cpe0030ab124d2f-cm014500000962.cpe.net.cable.rogers.com ([24.103.99.32] helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1CbHrC-0008Dv-BM; Mon, 06 Dec 2004 07:20:46 -0500 Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) From: jamal Reply-To: hadi@cyberus.ca To: Lennert Buytenhek Cc: Martin Josefsson , Scott Feldman , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com In-Reply-To: <20041206121151.GA12598@xi.wantstofly.org> References: <20041201182943.GA14470@xi.wantstofly.org> <20041201213550.GF14470@xi.wantstofly.org> <1101967983.4782.9.camel@localhost.localdomain> <20041205145051.GA647@xi.wantstofly.org> <20041205174401.GJ647@xi.wantstofly.org> <20041205175133.GK647@xi.wantstofly.org> <1102332757.1042.2255.camel@jzny.localdomain> <20041206121151.GA12598@xi.wantstofly.org> Content-Type: text/plain Organization: jamalopolous Message-Id: <1102335643.1048.2311.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 06 Dec 2004 07:20:43 -0500 Content-Transfer-Encoding: 7bit X-archive-position: 12472 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev On Mon, 2004-12-06 at 07:11, Lennert Buytenhek wrote: > On Mon, Dec 06, 2004 at 06:32:37AM -0500, jamal wrote: > > > Someone correct me if i am wrong - but does it appear as if all these > > changes are only useful on PCI but not PCI-X? > > They are useful on PCI-X as well as regular PCI. On my 64/100 NIC I > get ~620kpps on 2.6.9, ~1Mpps with 2.6.9 plus tx rework plus TXDMAC=0. > > Martin gets the ~1Mpps number with just the tx rework, and even more > with TXDMAC=0 added in as well. Right, but so far when i scan the results all i see is PCI not PCI-X. Which of your (or Martins) boards has PCI-X? cheers, jamal From buytenh@wantstofly.org Mon Dec 6 04:23:22 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 04:23:27 -0800 (PST) Received: from xi.wantstofly.org (alephnull.demon.nl [212.238.201.82]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB6CNM75004717 for ; Mon, 6 Dec 2004 04:23:22 -0800 Received: by xi.wantstofly.org (Postfix, from userid 500) id 5095C2B0ED; Mon, 6 Dec 2004 13:23:00 +0100 (MET) Date: Mon, 6 Dec 2004 13:23:00 +0100 From: Lennert Buytenhek To: jamal Cc: Martin Josefsson , Scott Feldman , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) Message-ID: <20041206122300.GA12763@xi.wantstofly.org> References: <1101967983.4782.9.camel@localhost.localdomain> <20041205145051.GA647@xi.wantstofly.org> <20041205174401.GJ647@xi.wantstofly.org> <20041205175133.GK647@xi.wantstofly.org> <1102332757.1042.2255.camel@jzny.localdomain> <20041206121151.GA12598@xi.wantstofly.org> <1102335643.1048.2311.camel@jzny.localdomain> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1102335643.1048.2311.camel@jzny.localdomain> User-Agent: Mutt/1.4.1i X-archive-position: 12473 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: buytenh@wantstofly.org Precedence: bulk X-list: netdev On Mon, Dec 06, 2004 at 07:20:43AM -0500, jamal wrote: > > > Someone correct me if i am wrong - but does it appear as if all these > > > changes are only useful on PCI but not PCI-X? > > > > They are useful on PCI-X as well as regular PCI. On my 64/100 NIC I > > get ~620kpps on 2.6.9, ~1Mpps with 2.6.9 plus tx rework plus TXDMAC=0. > > > > Martin gets the ~1Mpps number with just the tx rework, and even more > > with TXDMAC=0 added in as well. > > Right, but so far when i scan the results all i see is PCI not PCI-X. > Which of your (or Martins) boards has PCI-X? I've tested 32/33 PCI, 32/66 PCI, and 64/100 PCI-X. I _think_ Martin was running at 64/133 PCI-X. --L From gandalf@wlug.westbo.se Mon Dec 6 04:31:08 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 04:31:13 -0800 (PST) Received: from null.rsn.bth.se (postfix@null.rsn.bth.se [194.47.142.3]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB6CV7sK005476 for ; Mon, 6 Dec 2004 04:31:08 -0800 Received: by null.rsn.bth.se (Postfix, from userid 65534) id 0DB082C0023; Mon, 6 Dec 2004 13:30:42 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by null.rsn.bth.se (Postfix) with ESMTP id 9FDCD2C0103; Mon, 6 Dec 2004 13:30:41 +0100 (CET) Received: from tux.rsn.bth.se (tux.rsn.bth.se [194.47.143.135]) by null.rsn.bth.se (Postfix) with ESMTP id DEFF12C0023; Mon, 6 Dec 2004 13:30:40 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by tux.rsn.bth.se (Postfix) with ESMTP id EB1133FA7; Mon, 6 Dec 2004 13:30:40 +0100 (CET) Date: Mon, 6 Dec 2004 13:30:40 +0100 (CET) From: Martin Josefsson X-X-Sender: gandalf@tux.rsn.bth.se To: Lennert Buytenhek Cc: jamal , Scott Feldman , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) In-Reply-To: <20041206122300.GA12763@xi.wantstofly.org> Message-ID: References: <1101967983.4782.9.camel@localhost.localdomain> <20041205145051.GA647@xi.wantstofly.org> <20041205174401.GJ647@xi.wantstofly.org> <20041205175133.GK647@xi.wantstofly.org> <1102332757.1042.2255.camel@jzny.localdomain> <20041206121151.GA12598@xi.wantstofly.org> <1102335643.1048.2311.camel@jzny.localdomain> <20041206122300.GA12763@xi.wantstofly.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by amavisd-new-20030616-p10 on null.rsn.bth.se X-archive-position: 12474 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: gandalf@wlug.westbo.se Precedence: bulk X-list: netdev On Mon, 6 Dec 2004, Lennert Buytenhek wrote: > > Right, but so far when i scan the results all i see is PCI not PCI-X. > > Which of your (or Martins) boards has PCI-X? > > I've tested 32/33 PCI, 32/66 PCI, and 64/100 PCI-X. I _think_ Martin > was running at 64/133 PCI-X. I don't have any motherboards with PCI-X so no :) I'm running the 82546GB (dualport) at 64/66 and the 82540EM (desktop adapter) at 32/66, both are able to send at wirespeed. /Martin From hadi@cyberus.ca Mon Dec 6 05:11:38 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 05:11:42 -0800 (PST) Received: from mx01.cybersurf.com (mx01.cybersurf.com [209.197.145.104]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB6DBcd3007808 for ; Mon, 6 Dec 2004 05:11:38 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx01.cybersurf.com with esmtp (Exim 4.30) id 1CbIe1-0004XW-Nb for netdev@oss.sgi.com; Mon, 06 Dec 2004 08:11:13 -0500 Received: from cpe0030ab124d2f-cm014500000962.cpe.net.cable.rogers.com ([24.103.99.32] helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1CbIdt-000504-BI; Mon, 06 Dec 2004 08:11:05 -0500 Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) From: jamal Reply-To: hadi@cyberus.ca To: Martin Josefsson Cc: Lennert Buytenhek , Scott Feldman , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, e1000-devel@lists.sourceforge.net, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com In-Reply-To: References: <1101967983.4782.9.camel@localhost.localdomain> <20041205145051.GA647@xi.wantstofly.org> <20041205174401.GJ647@xi.wantstofly.org> <20041205175133.GK647@xi.wantstofly.org> <1102332757.1042.2255.camel@jzny.localdomain> <20041206121151.GA12598@xi.wantstofly.org> <1102335643.1048.2311.camel@jzny.localdomain> <20041206122300.GA12763@xi.wantstofly.org> Content-Type: text/plain Organization: jamalopolous Message-Id: <1102338662.1042.2365.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 06 Dec 2004 08:11:02 -0500 Content-Transfer-Encoding: 7bit X-archive-position: 12475 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev Hopefully someone will beat me to testing to see if our forwarding capacity now goes up with this new recipe. cheers, jamal On Mon, 2004-12-06 at 07:30, Martin Josefsson wrote: > On Mon, 6 Dec 2004, Lennert Buytenhek wrote: > > > > Right, but so far when i scan the results all i see is PCI not PCI-X. > > > Which of your (or Martins) boards has PCI-X? > > > > I've tested 32/33 PCI, 32/66 PCI, and 64/100 PCI-X. I _think_ Martin > > was running at 64/133 PCI-X. > > I don't have any motherboards with PCI-X so no :) > I'm running the 82546GB (dualport) at 64/66 and the 82540EM (desktop > adapter) at 32/66, both are able to send at wirespeed. > > /Martin > > From tgraf@suug.ch Mon Dec 6 06:02:21 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 06:02:26 -0800 (PST) Received: from b.mx.projectdream.org (eth0-0.arisu.projectdream.org [194.158.4.191]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB6E2KhA010085 for ; Mon, 6 Dec 2004 06:02:21 -0800 Received: from postel.suug.ch (unknown [195.134.158.23]) (using TLSv1 with cipher EDH-RSA-DES-CBC3-SHA (168/168 bits)) (No client certificate requested) by b.mx.projectdream.org (Postfix) with ESMTP id 3010CF; Mon, 6 Dec 2004 15:01:33 +0100 (CET) Received: by postel.suug.ch (Postfix, from userid 10001) id E2C991C0EA; Mon, 6 Dec 2004 15:02:14 +0100 (CET) Date: Mon, 6 Dec 2004 15:02:14 +0100 From: Thomas Graf To: Michal Ludvig Cc: Andrew Morton , netdev@oss.sgi.com, Jan Kara Subject: Re: [PATCH] rtnetlink & address family problem Message-ID: <20041206140214.GA749@postel.suug.ch> References: <41B0A5B4.6060108@suse.cz> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <41B0A5B4.6060108@suse.cz> X-archive-position: 12476 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: tgraf@suug.ch Precedence: bulk X-list: netdev * Michal Ludvig <41B0A5B4.6060108@suse.cz> 2004-12-03 18:43 > running 'ip -6 addr flush dev eth0' on a kernel without IPv6 support > flushes *all* addresses from the interface, even those IPv4 ones, > because the unsupported protocol is substituted by PF_UNSPEC. > IMHO it should better return with an error EAFNOSUPPORT. > > diff -Nru a/net/core/rtnetlink.c b/net/core/rtnetlink.c > --- a/net/core/rtnetlink.c 2004-12-03 18:30:33 +01:00 > +++ b/net/core/rtnetlink.c 2004-12-03 18:30:33 +01:00 > @@ -477,8 +477,10 @@ > } > > link_tab = rtnetlink_links[family]; > - if (link_tab == NULL) > - link_tab = rtnetlink_links[PF_UNSPEC]; > + if (link_tab == NULL) { > + *errp = -EAFNOSUPPORT; > + return -1; > + } > link = &link_tab[type]; > > sz_idx = type>>2; Your patch would fix this issue but might break various things. The actual problem is that iproute2 doesn't check the family in its filter. It blindly assumes that the kernel only returns addresses of the kind it has requested. I can understand if you think the current behaviour is wrong but we shouldn't change it in the middle of a stable tree. --- iproute2-2.6.9.orig/ip/ipaddress.c 2004-10-19 22:49:02.000000000 +0200 +++ iproute2-2.6.9/ip/ipaddress.c 2004-12-06 14:55:58.000000000 +0100 @@ -330,6 +330,8 @@ return 0; } } + if (filter.family && filter.family != ifa->ifa_family) + return 0; if (filter.flushb) { struct nlmsghdr *fn; From hasso@estpak.ee Mon Dec 6 06:42:35 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 06:42:40 -0800 (PST) Received: from arena (test.estpak.ee [194.126.115.47]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB6EgXrQ012227 for ; Mon, 6 Dec 2004 06:42:34 -0800 Received: from arena ([127.0.0.1] ident=hasso) by arena with esmtp (Exim 3.36 #1 (Debian)) id 1CbK41-0000ah-00; Mon, 06 Dec 2004 16:42:09 +0200 From: Hasso Tepper To: hadi@cyberus.ca Subject: Re: [patch 4/10] s390: network driver. Date: Mon, 6 Dec 2004 16:42:00 +0200 User-Agent: KMail/1.7.1 Cc: Paul Jakma , Thomas Spatzier , jgarzik@pobox.com, netdev@oss.sgi.com References: <1102332479.1048.2243.camel@jzny.localdomain> In-Reply-To: <1102332479.1048.2243.camel@jzny.localdomain> Organization: Elion Enterprises Ltd. MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200412061642.00552.hasso@estpak.ee> X-archive-position: 12477 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hasso@estpak.ee Precedence: bulk X-list: netdev jamal wrote: > Dont post networking related patches on other lists. I havent seen said > patch, but it seems someone is complaining about some behavior changing? For some strange reason I didn't find the beginning of thread either from archive, the first post seems to be http://oss.sgi.com/projects/netdev/archive/2004-11/msg01015.html. I'm trying to summarize. The approach - one raw socket to send/receive messages no matter how many interfaces are used - worked for Quagga/Zebra routing software daemons till now. If some of these interfaces was operationally down, socket wasn't blocked even if the queue was full. In fact "man sendmsg" has still text: ENOBUFS The output queue for a network interface was full. This gener- ally indicates that the interface has stopped sending, but may be caused by transient congestion. (Normally, this does not occur in Linux. Packets are just silently dropped when a device queue overflows.) Seems that it's no longer true. Seems that kernel is now trying as hard as possible not to loose any data - data is queued and if the queue will be full, all related sockets will be blocked to notify application. So, one socket approach don't work any more for Quagga/Zebra. No problem, we can take the "one socket per interface" approach. And we already have link detection implemented to notify daemons. But there will be still potential problem - sending the "interface down" message from kernel to the zebra daemon and then to the routing protocol daemon takes time. And during this time daemon can send packets which will sit in queue and may cause many problems if sent to the network later (if link comes up again). Think about statelss routing protocols like rip(ng), ipv6 router advertisements etc. They may carry the info that's no longer true causing routing loops etc. And to clarify a little bit: no - the Quagga/Zebra didn't work with previous approach perfectly. I fact with link detection and socket per interface it will be better than ever no matter what's the kernel behaviour. We just want to make sure that solution will be bulletproof. So, problem - how can we make sure that no potentially dangerous aged (routing) info will be in the network? -- Hasso Tepper Elion Enterprises Ltd. WAN administrator From P@draigBrady.com Mon Dec 6 09:32:47 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 09:32:53 -0800 (PST) Received: from corvil.com (gate.corvil.net [213.94.219.177]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB6HWjlc024640 for ; Mon, 6 Dec 2004 09:32:47 -0800 Received: from draigBrady.com (pixelbeat.local.corvil.com [172.18.1.170]) by corvil.com (8.12.9/8.12.5) with ESMTP id iB6HWHwS032781; Mon, 6 Dec 2004 17:32:17 GMT (envelope-from P@draigBrady.com) Message-ID: <41B497A1.6090600@draigBrady.com> Date: Mon, 06 Dec 2004 17:32:17 +0000 From: P@draigBrady.com User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040124 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Robert Olsson CC: Lennert Buytenhek , jamal , Martin Josefsson , Scott Feldman , mellia@prezzemolo.polito.it, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) References: <20041205174401.GJ647@xi.wantstofly.org> <20041205175133.GK647@xi.wantstofly.org> <1102332757.1042.2255.camel@jzny.localdomain> <20041206121151.GA12598@xi.wantstofly.org> <1102335643.1048.2311.camel@jzny.localdomain> <20041206122300.GA12763@xi.wantstofly.org> <1102338662.1042.2365.camel@jzny.localdomain> <20041206132907.GA13411@xi.wantstofly.org> <16820.37049.396306.295878@robur.slu.se> In-Reply-To: <16820.37049.396306.295878@robur.slu.se> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-archive-position: 12478 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: P@draigBrady.com Precedence: bulk X-list: netdev Robert Olsson wrote: > Lennert Buytenhek writes: > > On Mon, Dec 06, 2004 at 08:11:02AM -0500, jamal wrote: > > > > > Hopefully someone will beat me to testing to see if our forwarding > > > capacity now goes up with this new recipe. > > > A breakthrough we now can send small packets at wire speed it will make > development and testing much easier... It surely will!! Just to recap, 2 people have been able to tx @ wire speed. The origonal poster was able to receive at wire speed, but could only TX at about 50% wire speed. It would be really cool if we could combine this to bridge @ wire speed. -- Pádraig Brady - http://www.pixelbeat.org -- From shemminger@osdl.org Mon Dec 6 09:43:05 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 09:43:13 -0800 (PST) Received: from mail.osdl.org (fw.osdl.org [65.172.181.6]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB6Hh5KH025589 for ; Mon, 6 Dec 2004 09:43:05 -0800 Received: from dxpl.pdx.osdl.net (dxpl.pdx.osdl.net [172.20.1.103]) by mail.osdl.org (8.11.6/8.11.6) with ESMTP id iB6HgZ920498; Mon, 6 Dec 2004 09:42:35 -0800 Date: Mon, 6 Dec 2004 09:42:34 -0800 From: Stephen Hemminger To: "David S. Miller" Cc: Michael Vittrup Larsen , netdev@oss.sgi.com Subject: [PATCH] tcp: efficient port randomisation (rev 3) Message-Id: <20041206094234.34861c78@dxpl.pdx.osdl.net> In-Reply-To: <200412060918.04441.michael.vittrup.larsen@ericsson.com> References: <20041027092531.78fe438c@guest-251-240.pdx.osdl.net> <20041202135252.04e64f51.davem@davemloft.net> <41B14E57.5080803@osdl.org> <200412060918.04441.michael.vittrup.larsen@ericsson.com> Organization: Open Source Development Lab X-Mailer: Sylpheed version 0.9.10claws (GTK+ 1.2.10; x86_64-suse-linux) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-archive-position: 12479 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: shemminger@osdl.org Precedence: bulk X-list: netdev Third revision of the TCP port randomization patch. It randomizes TCP ephemeral ports of incoming connections using variation of existing sequence number hash. This one avoids the MD4 for the loopback case since there is no reason to bother over loopback and it improves benchmark numbers. Signed-off-by: Stephen Hemminger Thanks to original author Michael Larsen. http://www.ietf.org/internet-drafts/draft-larsen-tsvwg-port-randomisation-00.txt diff -urNp -X dontdiff test-2.6/drivers/char/random.c tcpport/drivers/char/random.c --- test-2.6/drivers/char/random.c 2004-11-30 16:26:41.000000000 -0800 +++ tcpport/drivers/char/random.c 2004-12-03 17:04:18.267850607 -0800 @@ -2347,6 +2347,24 @@ __u32 secure_ip_id(__u32 daddr) return halfMD4Transform(hash, keyptr->secret); } +/* Generate secure starting point for ephemeral TCP port search */ +u32 secure_tcp_port_ephemeral(__u32 saddr, __u32 daddr, __u16 dport) +{ + struct keydata *keyptr = get_keyptr(); + u32 hash[4]; + + /* + * Pick a unique starting offset for each ephemeral port search + * (saddr, daddr, dport) and 48bits of random data. + */ + hash[0] = saddr; + hash[1] = daddr; + hash[2] = dport ^ keyptr->secret[10]; + hash[3] = keyptr->secret[11]; + + return halfMD4Transform(hash, keyptr->secret); +} + #ifdef CONFIG_SYN_COOKIES /* * Secure SYN cookie computation. This is the algorithm worked out by diff -urNp -X dontdiff test-2.6/include/linux/random.h tcpport/include/linux/random.h --- test-2.6/include/linux/random.h 2004-11-30 16:26:51.000000000 -0800 +++ tcpport/include/linux/random.h 2004-12-02 17:07:13.000000000 -0800 @@ -52,6 +52,7 @@ extern void get_random_bytes(void *buf, void generate_random_uuid(unsigned char uuid_out[16]); extern __u32 secure_ip_id(__u32 daddr); +extern u32 secure_tcp_port_ephemeral(__u32 saddr, __u32 daddr, __u16 dport); extern __u32 secure_tcp_sequence_number(__u32 saddr, __u32 daddr, __u16 sport, __u16 dport); extern __u32 secure_tcp_syn_cookie(__u32 saddr, __u32 daddr, diff -urNp -X dontdiff test-2.6/net/ipv4/tcp_ipv4.c tcpport/net/ipv4/tcp_ipv4.c --- test-2.6/net/ipv4/tcp_ipv4.c 2004-11-30 16:26:51.000000000 -0800 +++ tcpport/net/ipv4/tcp_ipv4.c 2004-12-03 17:04:26.454562583 -0800 @@ -636,10 +636,18 @@ not_unique: return -EADDRNOTAVAIL; } +static inline u32 connect_port_offset(const struct sock *sk) +{ + const struct inet_opt *inet = inet_sk(sk); + + return secure_tcp_port_ephemeral(inet->rcv_saddr, inet->daddr, + inet->dport); +} + /* * Bind a port for a connect operation and hash it. */ -static int tcp_v4_hash_connect(struct sock *sk) +static int tcp_v4_hash_connect(struct sock *sk, int loopback) { unsigned short snum = inet_sk(sk)->num; struct tcp_bind_hashbucket *head; @@ -647,36 +655,23 @@ static int tcp_v4_hash_connect(struct so int ret; if (!snum) { - int rover; int low = sysctl_local_port_range[0]; int high = sysctl_local_port_range[1]; - int remaining = (high - low) + 1; + int range = high - low; + int i; + int port; + static u32 hint; + u32 offset = hint; struct hlist_node *node; struct tcp_tw_bucket *tw = NULL; + if (!loopback) + offset += connect_port_offset(sk); + local_bh_disable(); - - /* TODO. Actually it is not so bad idea to remove - * tcp_portalloc_lock before next submission to Linus. - * As soon as we touch this place at all it is time to think. - * - * Now it protects single _advisory_ variable tcp_port_rover, - * hence it is mostly useless. - * Code will work nicely if we just delete it, but - * I am afraid in contented case it will work not better or - * even worse: another cpu just will hit the same bucket - * and spin there. - * So some cpu salt could remove both contention and - * memory pingpong. Any ideas how to do this in a nice way? - */ - spin_lock(&tcp_portalloc_lock); - rover = tcp_port_rover; - - do { - rover++; - if ((rover < low) || (rover > high)) - rover = low; - head = &tcp_bhash[tcp_bhashfn(rover)]; + for (i = 1; i <= range; i++) { + port = low + (i + offset) % range; + head = &tcp_bhash[tcp_bhashfn(port)]; spin_lock(&head->lock); /* Does not bother with rcv_saddr checks, @@ -684,19 +679,19 @@ static int tcp_v4_hash_connect(struct so * unique enough. */ tb_for_each(tb, node, &head->chain) { - if (tb->port == rover) { + if (tb->port == port) { BUG_TRAP(!hlist_empty(&tb->owners)); if (tb->fastreuse >= 0) goto next_port; if (!__tcp_v4_check_established(sk, - rover, + port, &tw)) goto ok; goto next_port; } } - tb = tcp_bucket_create(head, rover); + tb = tcp_bucket_create(head, port); if (!tb) { spin_unlock(&head->lock); break; @@ -706,22 +701,18 @@ static int tcp_v4_hash_connect(struct so next_port: spin_unlock(&head->lock); - } while (--remaining > 0); - tcp_port_rover = rover; - spin_unlock(&tcp_portalloc_lock); - + } local_bh_enable(); return -EADDRNOTAVAIL; ok: - /* All locks still held and bhs disabled */ - tcp_port_rover = rover; - spin_unlock(&tcp_portalloc_lock); + hint += i; - tcp_bind_hash(sk, tb, rover); + /* Head lock still held and bh's disabled */ + tcp_bind_hash(sk, tb, port); if (sk_unhashed(sk)) { - inet_sk(sk)->sport = htons(rover); + inet_sk(sk)->sport = htons(port); __tcp_v4_hash(sk, 0); } spin_unlock(&head->lock); @@ -832,7 +823,7 @@ int tcp_v4_connect(struct sock *sk, stru * complete initialization after this. */ tcp_set_state(sk, TCP_SYN_SENT); - err = tcp_v4_hash_connect(sk); + err = tcp_v4_hash_connect(sk, rt->rt_flags & RTCF_LOCAL); if (err) goto failure; From paul@clubi.ie Mon Dec 6 10:45:16 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 10:45:23 -0800 (PST) Received: from hibernia.jakma.org (hibernia.jakma.org [212.17.55.49]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB6Ij8Vh028799 for ; Mon, 6 Dec 2004 10:45:13 -0800 Received: from hibernia.jakma.org (IDENT:paul@hibernia.jakma.org [192.168.0.3]) by hibernia.jakma.org (8.12.11/8.12.11) with ESMTP id iB6Ii9gB014516; Mon, 6 Dec 2004 18:44:12 GMT Date: Mon, 6 Dec 2004 18:44:09 +0000 (GMT) From: Paul Jakma X-X-Sender: paul@hibernia.jakma.org To: jamal cc: Thomas Spatzier , jgarzik@pobox.com, linux-kernel@vger.kernel.org, netdev@oss.sgi.com Subject: Re: [patch 4/10] s390: network driver. In-Reply-To: <1102332479.1048.2243.camel@jzny.localdomain> Message-ID: References: <1102332479.1048.2243.camel@jzny.localdomain> Mail-Followup-To: paul@hibernia.jakma.org X-NSA: arafat al aqsar jihad musharef jet-A1 avgas ammonium qran inshallah allah al-akbar martyr iraq saddam hammas hisballah rabin ayatollah korea vietnam revolt mustard gas british airways washington MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Virus-Scanned: ClamAV 0.80/614/Wed Dec 1 15:44:43 2004 clamav-milter version 0.80j on hibernia.jakma.org X-Virus-Status: Clean X-archive-position: 12480 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: paul@clubi.ie Precedence: bulk X-list: netdev On Mon, 6 Dec 2004, jamal wrote: > Dont post networking related patches on other lists. I havent seen said > patch, but it seems someone is complaining about some behavior changing? I missed the beginning of the thread too, but saw Jeff's reply to Thomas on netdev. It appears the original patch was to make the s390 network driver discard packets on link-down. Jeff had replied to say this was bad, that queues are meant to fill and that this was what other drivers (e1000, tg3) did. > In regards to link down and packets being queued. Agreed this is a > little problematic for some apps/transports. Tis yes. Particularly for apps using raw and UDP+IP_HDRINCL sockets. This problem came to light when we got reports of ospfd blocking because link was down, late in 2.4 with a certain version of the (iirc) e100 driver. ospfd uses one single socket for all interfaces, and relies on IP_HDRINCL to have the packet routed out right interface. However this approach doesnt play well if the socket can be blocked completely because of /one/ interface having its link down. The behaviour we expected (and got up until now) is to receive either ENOBUFS or else, if the kernel accepts the packet write, for it to drop it if it can not be sent. We can work around that by moving to a socket/interface. However it still leaves the problem of packets being queued indefinitely while the link is down and being sent when link comes back. This is *not* good for RIP, IPv4 IRDP and IPv6 RA. > In the case the netdevice is administratively downed both the qdisc > and DMA ring packets are flushed. What about any packets remaining in the socket buffer? (if that makes sense - i dont know enough about internals sadly). Are those queued? > Newer packets will never be queued That no longer appears to be the case though. The socket blocks, and /some/ packets are queued (presumably those which still were in the socket buffer? i dont know exactly..). > and you should quickly be able to find from your app that > the device is down. We can yes, via rtnetlink - but impossible to guarantee we'll know the link is down before we try write a packet. > In the case of netdevice being operationally down ? As in 'ip link set dev ... down'? > - I am hoping this is what the discussion is, having jumped on it - No, its for link-down, AIUI. > both queues stay intact. What you can do is certainly from user > space admin down/up the device when you receive a netlink carrier > off notification. That seems possible, but quite a hack. Something to work at a socket level would possibly be nicer. (Socket being the primary handle our application has). > I am struggling to see whether dropping the packet inside the tx > path once it is operationaly down is so blasphemous ... need to get > caffeine first. As long as reliable transports have some other transport specific queue, shouldnt be a problem. For UDP and raw no reliability or guarantees are expected by applications (least shouldnt be), and queueing some packets on link-down interferes with application-layer expectations. > cheers, > jamal regards, -- Paul Jakma paul@clubi.ie paul@jakma.org Key ID: 64A2FF6A Fortune: The UPS doesn't have a battery backup. From Robert.Olsson@data.slu.se Mon Dec 6 11:11:10 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 11:11:16 -0800 (PST) Received: from mail1.slu.se (mail1.slu.se [130.238.96.11]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB6JB8V6030365 for ; Mon, 6 Dec 2004 11:11:09 -0800 Received: from robur.slu.se (robur.slu.se [130.238.98.12]) by mail1.slu.se (8.12.10/8.12.10) with ESMTP id iB6JAgKO029803; Mon, 6 Dec 2004 20:10:43 +0100 Received: by robur.slu.se (Postfix, from userid 1000) id BF556EC001; Mon, 6 Dec 2004 20:10:42 +0100 (CET) From: Robert Olsson MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <16820.44722.748743.6711@robur.slu.se> Date: Mon, 6 Dec 2004 20:10:42 +0100 To: Lennert Buytenhek Cc: jamal , Martin Josefsson , Scott Feldman , Robert Olsson , P@draigBrady.com, mellia@prezzemolo.polito.it, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) X-Mailer: VM 7.18 under Emacs 21.3.1 X-archive-position: 12481 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: Robert.Olsson@data.slu.se Precedence: bulk X-list: netdev Lennert Buytenhek writes: > On Mon, Dec 06, 2004 at 08:11:02AM -0500, jamal wrote: > > > Hopefully someone will beat me to testing to see if our forwarding > > capacity now goes up with this new recipe. Yes a breakthrough as we now can send small packets at GIGE wire speed this will make development and testing much easier... A first router test with our setup below. Opteron 1.6 GHz SMP kernel. using 1 CPU. 82546 EB + 82456 GB and PCI-X 100 Mhz & 133 MHz. pktgen performance is measured on router box. Remember Scotts patch uses 4096 TX buffers and w. pktgen we use clone_skb. So with real skb's we probably see lower performance due to this. This may explain results below so routing performance doesn't follow pktgen performance as seen. T-PUT is routing performance. Also pktgen pure TX performance is given this on the router. Input rate for routing test is 2*765 kpps for all three runs. Input Packets input to eth0 is routed to eth1 and eth2 to eth3. Vanilla. T-PUT 657 kpps. pktgen TX perf 818 kpps ------------------------------------------------- Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flags eth0 1500 0 4312682 8253078 8253078 5687318 5 0 0 0 BRU eth1 1500 0 1 0 0 0 4312199 0 0 0 BRU eth2 1500 0 4311018 8386504 8386504 5688982 5 0 0 0 BRU eth3 1500 0 1 0 0 0 4310791 0 0 0 BRU CPU0 0: 116665 IO-APIC-edge timer 1: 208 IO-APIC-edge i8042 8: 0 IO-APIC-edge rtc 9: 0 IO-APIC-level acpi 14: 21943 IO-APIC-edge ide0 26: 66 IO-APIC-level eth0 27: 58638 IO-APIC-level eth1 28: 68 IO-APIC-level eth2 29: 58497 IO-APIC-level eth3 NMI: 0 LOC: 116605 ERR: 0 MIS: 0 e1000-TX-prefetch+scott tx patch. T-PUT 540 kpps. pktgen TX perf 1.48 Mpps -------------------------------------------------------------------------- Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flags eth0 1500 0 3533795 8618637 8618637 6466205 5 0 0 0 BRU eth1 1500 0 3 0 0 0 3533803 0 0 0 BRU eth2 1500 0 3535804 8697149 8697149 6464196 5 0 0 0 BRU eth3 1500 0 1 0 0 0 3535321 0 0 0 BRU CPU0 0: 1372774 IO-APIC-edge timer 1: 663 IO-APIC-edge i8042 8: 0 IO-APIC-edge rtc 9: 0 IO-APIC-level acpi 14: 22631 IO-APIC-edge ide0 26: 686 IO-APIC-level eth0 27: 693 IO-APIC-level eth1 28: 687 IO-APIC-level eth2 29: 682 IO-APIC-level eth3 NMI: 0 LOC: 1372804 ERR: 0 MIS: 0 e1000-TX-prefetch. T-PUT 657 kpps. pktgen TX perf 1.15 Mpps ----------------------------------------------------------- Kernel Interface table Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flags eth0 1500 0 4311848 8288270 8288270 5688152 5 0 0 0 BRU eth1 1500 0 4 0 0 0 4311388 0 0 0 BRU eth2 1500 0 4309082 8400892 8400892 5690918 5 0 0 0 BRU eth3 1500 0 1 0 0 0 4308271 0 0 0 BRU lo 16436 0 0 0 0 0 0 0 0 0 LRU CPU0 0: 224310 IO-APIC-edge timer 1: 250 IO-APIC-edge i8042 8: 0 IO-APIC-edge rtc 9: 0 IO-APIC-level acpi 14: 22055 IO-APIC-edge ide0 26: 122 IO-APIC-level eth0 27: 58001 IO-APIC-level eth1 28: 123 IO-APIC-level eth2 29: 57681 IO-APIC-level eth3 NMI: 0 LOC: 224251 ERR: 0 MIS: 0 --ro From kdesler@soohrt.org Mon Dec 6 12:53:32 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 12:53:36 -0800 (PST) Received: from quickstop.soohrt.org (quickstop.soohrt.org [81.2.155.147]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB6KrV4c005582 for ; Mon, 6 Dec 2004 12:53:32 -0800 Received: (qmail 18579 invoked by uid 1000); 6 Dec 2004 20:53:05 -0000 Date: Mon, 6 Dec 2004 21:53:05 +0100 From: Karsten Desler To: netdev@oss.sgi.com Cc: linux-kernel@vger.kernel.org Subject: _High_ CPU usage while routing (mostly) small UDP packets Message-ID: <20041206205305.GA11970@soohrt.org> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline User-Agent: Mutt/1.5.6+20040722i X-archive-position: 12482 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kdesler@soohrt.org Precedence: bulk X-list: netdev Hi, I'm running a dual Opteron 244 router with two active e1000 interfaces, that mostly deals with small (<200b) udp packets. I'm using Linux 2.6.10-rc3 (32bit), but I tried 64bit with early 2.6.9-rc versions, and it didn't make much of a difference. The irq of eth0 is bound to cpu1 while eth1 is bound to cpu0. NAPI is enabled. Current packetload on eth0 (and reversed on eth1): 115kpps tx 135kpps rx There are about 200 iptables rules, but the common packet only has to traverse about 20. conntrack is not loaded. eth0 and eth1 are running on the same 66MHz/64bit PCI bus. Kernel-Profiling is running, I don't know how much that contributes to the overall load. I don't know if that has anything to do with it, but the systemclock is getting out of sync _fast_ (openntpd can't keep up). ntpdate -b ntp.soohrt.org; sleep 60; ntpdate ntp.soohrt.org: 6 Dec 21:40:39 ntpdate[30146]: adjust time server 134.100.177.5 offset 0.000092 sec 6 Dec 21:41:39 ntpdate[30218]: adjust time server 134.100.177.5 offset 0.006971 sec Is that the expected cpu usage? I'd appreciate _any_ pointers, thanks in advance, Karsten Current cpu usage: Cpu0 : 0.0% us, 0.0% sy, 0.0% ni, 46.3% id, 0.0% wa, 0.0% hi, 53.7% si Cpu1 : 0.0% us, 0.0% sy, 0.0% ni, 21.4% id, 0.0% wa, 0.0% hi, 78.6% si vmstat 5: procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu---- r b swpd free buff cache si so bi bo in cs us sy id wa 0 0 0 1804612 100488 104464 0 0 0 32 5965 167 3 68 30 0 0 1 0 1804548 100488 104464 0 0 0 157 5994 18 0 67 33 0 1 1 0 1804684 100488 104464 0 0 0 63 5998 19 0 67 33 0 0 1 0 1804684 100492 104460 0 0 0 4 5985 10 0 68 33 0 0 1 0 1804620 100492 104460 0 0 0 12 6032 15 0 68 32 0 lspci -vt: -[00]-+-06.0-[03]--+-00.0 Advanced Micro Devices [AMD] AMD-8111 USB [...] +-0a.1 Advanced Micro Devices [AMD] AMD-8131 PCI-X APIC +-0b.0-[01]--+-01.0 Intel Corp. 82545EM Gigabit Ethernet Controller (Fiber) <- eth0 | +-03.0 Intel Corp. 82546GB Gigabit Ethernet Controller <- eth1 | \-03.1 Intel Corp. 82546GB Gigabit Ethernet Controller arp -n|grep -c incomplete: 73 arp -n|grep -vc incomplete: 778 iptables -nL|grep -v Chain|grep -vc source: 199 readprofile -r; sleep 60; readprofile|sort -n +2: 76 __do_softirq 0.3654 73 dst_alloc 0.4148 489 ip_rcv 0.4187 96 fib_semantic_match 0.4615 15 memset 0.4688 8 _write_lock 0.5000 37 match 0.5781 72 handle_IRQ_event 0.6429 138 fn_hash_lookup 0.7188 401 qdisc_restart 0.7371 13 _read_unlock 0.8125 28 _spin_lock_bh 0.8750 70 e1000_rx_checksum 0.8750 536 ip_rcv_finish 0.9306 33 kfree_skbmem 1.0312 248 e1000_intr 1.0333 30 _read_lock 1.8750 1198 ip_forward 1.9199 910 ip_finish_output2 1.9612 282 rt_hash_code 2.2031 286 pfifo_fast_enqueue 2.2344 36 _spin_unlock 2.2500 1657 dev_queue_xmit 2.3014 230 ip_output 2.3958 905 netif_receive_skb 2.4592 2852 e1000_clean_rx_irq 2.6213 697 nf_hook_slow 2.7227 674 e1000_alloc_rx_buffers 2.8083 228 ip_forward_finish 2.8500 187 ipt_hook 3.8958 346 kmem_cache_free 4.3250 357 pfifo_fast_dequeue 4.4625 626 local_bh_enable 4.8906 250 e1000_irq_enable 5.2083 425 kmem_cache_alloc 5.3125 3100 e1000_clean_tx_irq 5.6985 1014 nf_iterate 5.7614 284 ipt_route_hook 5.9167 701 kfree 6.2589 1063 __kmalloc 8.3047 1605 skb_release_data 10.0312 2692 eth_type_trans 11.2167 4013 __kfree_skb 15.6758 4554 alloc_skb 18.9750 20017 ipt_do_table 24.0589 10700 ip_route_input 30.3977 1341 _spin_lock 83.8125 1483 _read_unlock_bh 92.6875 3345 _read_lock_bh 104.5312 3185 _spin_unlock_irqrestore 199.0625 44402 default_idle 693.7812 From davem@davemloft.net Mon Dec 6 13:50:31 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 13:50:37 -0800 (PST) Received: from cheetah.davemloft.net (mail@adsl-63-197-226-105.dsl.snfc21.pacbell.net [63.197.226.105]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB6LoVuH008312 for ; Mon, 6 Dec 2004 13:50:31 -0800 Received: from localhost ([127.0.0.1] helo=cheetah.davemloft.net ident=davem) by cheetah.davemloft.net with smtp (Exim 3.36 #1 (Debian)) id 1CbQiw-0001qy-00; Mon, 06 Dec 2004 13:48:50 -0800 Date: Mon, 6 Dec 2004 13:48:49 -0800 From: "David S. Miller" To: Karsten Desler Cc: netdev@oss.sgi.com, linux-kernel@vger.kernel.org Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets Message-Id: <20041206134849.498bfc93.davem@davemloft.net> In-Reply-To: <20041206205305.GA11970@soohrt.org> References: <20041206205305.GA11970@soohrt.org> X-Mailer: Sylpheed version 1.0.0beta3 (GTK+ 1.2.10; sparc-unknown-linux-gnu) X-Face: "_;p5u5aPsO,_Vsx"^v-pEq09'CU4&Dc1$fQExov$62l60cgCc%FnIwD=.UF^a>?5'9Kn[;433QFVV9M..2eN.@4ZWPGbdi<=?[:T>y?SD(R*-3It"Vj:)"dP Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-archive-position: 12483 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: davem@davemloft.net Precedence: bulk X-list: netdev On Mon, 6 Dec 2004 21:53:05 +0100 Karsten Desler wrote: > 20017 ipt_do_table 24.0589 It's spending nearly half of it's time in iptables. Try to consolidate your rules if possible. This is the part of netfilter that really doesn't scale well at all. From gandalf@wlug.westbo.se Mon Dec 6 14:30:21 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 14:30:25 -0800 (PST) Received: from null.rsn.bth.se (postfix@null.rsn.bth.se [194.47.142.3]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB6MUJ1V010327 for ; Mon, 6 Dec 2004 14:30:20 -0800 Received: by null.rsn.bth.se (Postfix, from userid 65534) id 554092C00B1; Mon, 6 Dec 2004 23:29:53 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by null.rsn.bth.se (Postfix) with ESMTP id 6F3F52C011E; Mon, 6 Dec 2004 23:29:52 +0100 (CET) Received: from tux.rsn.bth.se (tux.rsn.bth.se [194.47.143.135]) by null.rsn.bth.se (Postfix) with ESMTP id 80DA72C00B1; Mon, 6 Dec 2004 23:29:51 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by tux.rsn.bth.se (Postfix) with ESMTP id DC2EC3FA7; Mon, 6 Dec 2004 23:29:51 +0100 (CET) Date: Mon, 6 Dec 2004 23:29:51 +0100 (CET) From: Martin Josefsson X-X-Sender: gandalf@tux.rsn.bth.se To: Robert Olsson Cc: Lennert Buytenhek , jamal , Scott Feldman , P@draigBrady.com, mellia@prezzemolo.polito.it, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) In-Reply-To: <16820.44722.748743.6711@robur.slu.se> Message-ID: References: <16820.44722.748743.6711@robur.slu.se> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by amavisd-new-20030616-p10 on null.rsn.bth.se X-archive-position: 12484 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: gandalf@wlug.westbo.se Precedence: bulk X-list: netdev On Mon, 6 Dec 2004, Robert Olsson wrote: > pktgen performance is measured on router box. Remember Scotts patch uses > 4096 TX buffers and w. pktgen we use clone_skb. So with real skb's we probably > see lower performance due to this. This may explain results below so routing > performance doesn't follow pktgen performance as seen. I've performed some tests with and without clone_skb with various versions of the driver. > Vanilla. T-PUT 657 kpps. pktgen TX perf 818 kpps > e1000-TX-prefetch+scott tx patch. T-PUT 540 kpps. pktgen TX perf 1.48 Mpps > e1000-TX-prefetch. T-PUT 657 kpps. pktgen TX perf 1.15 Mpps This matches the data I see in my tests here with and without clone_skb. I've included a lot of pps numbers below, they might need some description. I tested generating packets with four diffrent drivers with and without clone_skb. vanilla is the vanilla driver in 2.6.10-rc3 copy is using the patch found at the bottom of this mail, just a small test to see if there's any gain or loss using "static" buffers to dma from. Prefetch doesn't help at all here, just makes things worse, even for clone_skb. Tried with delayed TDT updating as well, didn't help. vanilla + prefetch is just the vanilla driver + prefetching. feldman tx is using scotts tx-path rewrite patch. I didn't bother listing feldman tx + prefetch as the results were even lower for the non clone_skb case. The only thing I can think of that can cause this is cache trashing, or overhead in slab when we have a lot of skb's in the wild. I don't have oprofile on my testmachine at the moment and it's time to go to bed now, maybe tomorrow... Does anyone have any suggestions of what to test next? vanilla and clone 60 854886 64 772341 68 759531 72 758872 76 758926 80 761136 84 742109 88 742070 92 741616 96 744083 100 727430 104 725242 108 724153 112 725841 116 707331 120 706000 124 704923 128 662547 vanilla and noclone 60 748552 64 702464 68 649066 72 671992 76 680251 80 627711 84 625468 88 640115 92 679365 96 650544 100 666423 104 652057 108 665821 112 679443 116 652507 120 661279 124 648627 128 635780 copy and clone 60 897165 64 872767 68 750694 72 750427 76 749583 80 748242 84 732760 88 731129 92 732603 96 732631 100 717123 104 717678 108 716839 112 719258 116 703824 120 706047 124 701885 128 695575 copy and noclone 60 882227 64 649614 68 691327 72 700706 76 700795 80 696594 84 686016 88 691689 92 696136 96 691348 100 684596 104 687800 108 689218 112 671483 116 675867 120 679089 124 672385 128 650148 vanilla + prefetch and clone 60 1300075 64 1079069 68 1082091 72 1068791 76 1067630 80 1026222 84 1053055 88 1024442 92 1032112 96 1014844 100 991346 104 976483 108 947019 112 919193 116 892863 120 868054 124 844679 128 822347 vanilla + prefetch and noclone 60 738538 64 800927 68 719832 72 725353 76 822738 80 743134 84 813520 88 721522 92 797838 96 724031 100 812198 104 717811 108 713072 112 789771 116 696027 120 682168 124 749020 128 703233 feldman tx and clone 60 1029997 64 916706 68 898601 72 895378 76 896171 80 898594 84 861434 88 861446 92 861444 96 863669 100 837624 104 836225 108 835528 112 835527 116 817102 120 817101 124 817100 128 757683 feldman tx and noclone 60 626646 64 628148 68 628935 72 625084 76 623527 80 623510 84 624286 88 625086 92 623907 96 630199 100 613933 104 618025 108 620326 112 607884 116 606124 120 538434 124 531699 128 532719 diff -X /home/gandalf/dontdiff.ny -urNp drivers/net/e1000-vanilla/e1000_main.c drivers/net/e1000/e1000_main.c --- drivers/net/e1000-vanilla/e1000_main.c 2004-12-05 18:27:50.000000000 +0100 +++ drivers/net/e1000/e1000_main.c 2004-12-06 22:21:10.000000000 +0100 @@ -132,6 +132,7 @@ static void e1000_irq_disable(struct e10 static void e1000_irq_enable(struct e1000_adapter *adapter); static irqreturn_t e1000_intr(int irq, void *data, struct pt_regs *regs); static boolean_t e1000_clean_tx_irq(struct e1000_adapter *adapter); +static boolean_t e1000_alloc_tx_buffers(struct e1000_adapter *adapter); #ifdef CONFIG_E1000_NAPI static int e1000_clean(struct net_device *netdev, int *budget); static boolean_t e1000_clean_rx_irq(struct e1000_adapter *adapter, @@ -264,6 +265,7 @@ e1000_up(struct e1000_adapter *adapter) e1000_restore_vlan(adapter); e1000_configure_tx(adapter); + e1000_alloc_tx_buffers(adapter); e1000_setup_rctl(adapter); e1000_configure_rx(adapter); e1000_alloc_rx_buffers(adapter); @@ -1048,10 +1052,21 @@ e1000_configure_rx(struct e1000_adapter void e1000_free_tx_resources(struct e1000_adapter *adapter) { + struct e1000_desc_ring *tx_ring = &adapter->tx_ring; + struct e1000_buffer *buffer_info; struct pci_dev *pdev = adapter->pdev; + unsigned int i; e1000_clean_tx_ring(adapter); + for(i = 0; i < tx_ring->count; i++) { + buffer_info = &tx_ring->buffer_info[i]; + if(buffer_info->skb) { + kfree(buffer_info->skb); + buffer_info->skb = NULL; + } + } + vfree(adapter->tx_ring.buffer_info); adapter->tx_ring.buffer_info = NULL; @@ -1079,16 +1094,12 @@ e1000_clean_tx_ring(struct e1000_adapter for(i = 0; i < tx_ring->count; i++) { buffer_info = &tx_ring->buffer_info[i]; - if(buffer_info->skb) { - + if(buffer_info->dma) { pci_unmap_page(pdev, buffer_info->dma, buffer_info->length, PCI_DMA_TODEVICE); - - dev_kfree_skb(buffer_info->skb); - - buffer_info->skb = NULL; + buffer_info->dma = 0; } } @@ -1579,8 +1590,6 @@ e1000_tx_map(struct e1000_adapter *adapt struct e1000_buffer *buffer_info; unsigned int len = skb->len; unsigned int offset = 0, size, count = 0, i; - unsigned int f; - len -= skb->data_len; i = tx_ring->next_to_use; @@ -1600,10 +1609,12 @@ e1000_tx_map(struct e1000_adapter *adapt size > 4)) size -= 4; + skb_copy_bits(skb, offset, buffer_info->skb, size); + buffer_info->length = size; buffer_info->dma = pci_map_single(adapter->pdev, - skb->data + offset, + buffer_info->skb, size, PCI_DMA_TODEVICE); buffer_info->time_stamp = jiffies; @@ -1614,50 +1625,11 @@ e1000_tx_map(struct e1000_adapter *adapt if(unlikely(++i == tx_ring->count)) i = 0; } - for(f = 0; f < nr_frags; f++) { - struct skb_frag_struct *frag; - - frag = &skb_shinfo(skb)->frags[f]; - len = frag->size; - offset = frag->page_offset; - - while(len) { - buffer_info = &tx_ring->buffer_info[i]; - size = min(len, max_per_txd); -#ifdef NETIF_F_TSO - /* Workaround for premature desc write-backs - * in TSO mode. Append 4-byte sentinel desc */ - if(unlikely(mss && f == (nr_frags-1) && size == len && size > 8)) - size -= 4; -#endif - /* Workaround for potential 82544 hang in PCI-X. - * Avoid terminating buffers within evenly-aligned - * dwords. */ - if(unlikely(adapter->pcix_82544 && - !((unsigned long)(frag->page+offset+size-1) & 4) && - size > 4)) - size -= 4; - - buffer_info->length = size; - buffer_info->dma = - pci_map_page(adapter->pdev, - frag->page, - offset, - size, - PCI_DMA_TODEVICE); - buffer_info->time_stamp = jiffies; - - len -= size; - offset += size; - count++; - if(unlikely(++i == tx_ring->count)) i = 0; - } - } - i = (i == 0) ? tx_ring->count - 1 : i - 1; - tx_ring->buffer_info[i].skb = skb; tx_ring->buffer_info[first].next_to_watch = i; + dev_kfree_skb_any(skb); + return count; } @@ -2213,11 +2185,6 @@ e1000_clean_tx_irq(struct e1000_adapter buffer_info->dma = 0; } - if(buffer_info->skb) { - dev_kfree_skb_any(buffer_info->skb); - buffer_info->skb = NULL; - } - tx_desc->buffer_addr = 0; tx_desc->lower.data = 0; tx_desc->upper.data = 0; @@ -2243,6 +2210,28 @@ e1000_clean_tx_irq(struct e1000_adapter return cleaned; } + +static boolean_t +e1000_alloc_tx_buffers(struct e1000_adapter *adapter) +{ + struct e1000_desc_ring *tx_ring = &adapter->tx_ring; + struct e1000_buffer *buffer_info; + unsigned int i; + + for (i = 0; i < tx_ring->count; i++) { + buffer_info = &tx_ring->buffer_info[i]; + if (!buffer_info->skb) { + buffer_info->skb = kmalloc(2048, GFP_ATOMIC); + if (unlikely(!buffer_info->skb)) { + printk("eek!\n"); + return FALSE; + } + } + } + + return TRUE; +} + /** * e1000_clean_rx_irq - Send received data up the network stack * @adapter: board private structure /Martin From kdesler@soohrt.org Mon Dec 6 14:41:35 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 14:41:39 -0800 (PST) Received: from quickstop.soohrt.org (quickstop.soohrt.org [81.2.155.147]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB6MfXHU011199 for ; Mon, 6 Dec 2004 14:41:34 -0800 Received: (qmail 10304 invoked by uid 1000); 6 Dec 2004 22:41:07 -0000 Date: Mon, 6 Dec 2004 23:41:07 +0100 From: Karsten Desler To: "David S. Miller" Cc: netdev@oss.sgi.com, linux-kernel@vger.kernel.org Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets Message-ID: <20041206224107.GA8529@soohrt.org> References: <20041206205305.GA11970@soohrt.org> <20041206134849.498bfc93.davem@davemloft.net> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: <20041206134849.498bfc93.davem@davemloft.net> User-Agent: Mutt/1.5.6+20040722i X-archive-position: 12485 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kdesler@soohrt.org Precedence: bulk X-list: netdev * David S. Miller wrote: > It's spending nearly half of it's time in iptables. > Try to consolidate your rules if possible. This is the > part of netfilter that really doesn't scale well at all. > Removing the iptables rules helps reducing the load a little, but the majority of time is still spent somewhere else. 50kpps rx and 43kpps tx. procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu---- r b swpd free buff cache si so bi bo in cs us sy id wa 0 0 0 1802956 101032 104464 0 0 0 18 74 26 0 21 79 0 0 0 0 1802956 101032 104464 0 0 0 61 8810 28 0 25 75 0 0 0 0 1802956 101032 104464 0 0 0 233 8867 17 0 24 76 0 0 0 0 1802892 101032 104464 0 0 0 0 8865 16 0 21 79 0 0 0 0 1802892 101032 104464 0 0 0 0 8772 8 0 18 82 0 <- iptables -F 0 0 0 1802892 101032 104464 0 0 0 36 8863 18 0 19 81 0 0 0 0 1802892 101032 104464 0 0 0 80 8700 18 0 18 82 0 0 0 0 1802956 101032 104464 0 0 0 0 8779 7 0 17 83 0 0 0 0 1802948 101032 104464 0 0 0 223 8716 278 4 19 76 0 - Karsten From rayl@mail.com Mon Dec 6 15:12:58 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 15:13:03 -0800 (PST) Received: from ray.lehtiniemi.com (dsl-crow-209-5-162-130-cgy.nucleus.com [209.5.162.130]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB6NCv21013369 for ; Mon, 6 Dec 2004 15:12:58 -0800 Received: by ray.lehtiniemi.com (Postfix, from userid 500) id 9EAE11D7803; Mon, 6 Dec 2004 16:12:35 -0700 (MST) Date: Mon, 6 Dec 2004 16:12:35 -0700 From: Ray Lehtiniemi To: Scott Feldman Cc: netdev@oss.sgi.com Subject: Re: how to tune a pair of e1000 cards on intel e7501-based system? Message-ID: <20041206231235.GA18097@mail.com> References: <20041206024437.GB7891@mail.com> <1102304058.3343.217.camel@sfeldma-mobl.dsl-verizon.net> <20041206041002.GC7891@mail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: <20041206041002.GC7891@mail.com> User-Agent: Mutt/1.5.6i X-archive-position: 12486 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: rayl@mail.com Precedence: bulk X-list: netdev On Sun, Dec 05, 2004 at 09:10:03PM -0700, Ray Lehtiniemi wrote: > > any idea why lspci -vv shows non-64bit, non-133 MHz? (i am assuming > that is what the minus sign means) > > Capabilities: [e4] PCI-X non-bridge device. > Command: DPERE- ERO+ RBC=0 OST=0 > Status: Bus=0 Dev=0 Func=0 64bit- 133MHz- SCD- USC-, DC=simple, DMMRBC=0, DMOST=0, DMCRS=0, RSCEM- turns out this is a bug in pci-utils-2.1.11. it was not correctly fetching the config information from the card, and was displaying zeroed data instead. i've included a patch at the end of this email and have also forwarded it to martin mares. > > Can you put one on D and the other on another bus? > > not sure... have to look at the chassis tomorrow morning. a co-worker > actually built the box, i've not seen it in person yet. nope, can't move things around. this is a NexGate NSA 2040G, and everything is built into the motherboard. > > What kind of numbers are you getting? i'm seeing about 100Kpps, with all settings at their defaults on the 2.4.20 kernel. basically, i have a couple of desktop PCs generating 480 streams of UDP data at 50 packets per second. Packet size on the wire, including 96 bits of IFG, is 128 bytes. these packets are forwarded through a user process on the NexGen box to an echoer process which is also running on the traffic generator boxes. the echoer sends them back to the NexGen user process, which forwards them back to the generator process. timestamps are logged for each packet at send, loop and recv points. anything over 480 streams, and i start to get large latencies and packet drops, as measured by the timestamps in the sender and echoer process. does 100Kpps sound reasonable for an untweaked 2.4.20 kernel? thanks diff -ur pciutils-2.1.11/lspci.c pciutils-2.1.11-rayl/lspci.c --- pciutils-2.1.11/lspci.c 2002-12-26 13:24:50.000000000 -0700 +++ pciutils-2.1.11-rayl/lspci.c 2004-12-06 15:54:33.573313973 -0700 @@ -476,16 +476,19 @@ static void show_pcix_nobridge(struct device *d, int where) { - u16 command = get_conf_word(d, where + PCI_PCIX_COMMAND); - u32 status = get_conf_long(d, where + PCI_PCIX_STATUS); + u16 command; + u32 status; printf("PCI-X non-bridge device.\n"); if (verbose < 2) return; + config_fetch(d, where, 8); + command = get_conf_word(d, where + PCI_PCIX_COMMAND); printf("\t\tCommand: DPERE%c ERO%c RBC=%d OST=%d\n", FLAG(command, PCI_PCIX_COMMAND_DPERE), FLAG(command, PCI_PCIX_COMMAND_ERO), ((command & PCI_PCIX_COMMAND_MAX_MEM_READ_BYTE_COUNT) >> 2U), ((command & PCI_PCIX_COMMAND_MAX_OUTSTANDING_SPLIT_TRANS) >> 4U)); + status = get_conf_long(d, where + PCI_PCIX_STATUS); printf("\t\tStatus: Bus=%u Dev=%u Func=%u 64bit%c 133MHz%c SCD%c USC%c, DC=%s, DMMRBC=%u, DMOST=%u, DMCRS=%u, RSCEM%c", ((status >> 8) & 0xffU), // bus ((status >> 3) & 0x1fU), // dev @@ -509,6 +512,7 @@ printf("PCI-X bridge device.\n"); if (verbose < 2) return; + config_fetch(d, where, 8); secstatus = get_conf_word(d, where + PCI_PCIX_BRIDGE_SEC_STATUS); printf("\t\tSecondary Status: 64bit%c, 133MHz%c, SCD%c, USC%c, SCO%c, SRD%c Freq=%d\n", FLAG(secstatus, PCI_PCIX_BRIDGE_SEC_STATUS_64BIT), -- ---------------------------------------------------------------------- Ray L From kernel@kolivas.org Mon Dec 6 15:57:36 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 15:57:40 -0800 (PST) Received: from mail07.syd.optusnet.com.au (mail07.syd.optusnet.com.au [211.29.132.188]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB6NvW7S015654 for ; Mon, 6 Dec 2004 15:57:35 -0800 Received: from mail.kolivas.org (c210-49-199-147.lowrp1.vic.optusnet.com.au [210.49.199.147]) by mail07.syd.optusnet.com.au (8.12.11/8.12.11) with ESMTP id iB6NuwtR029277; Tue, 7 Dec 2004 10:57:01 +1100 Received: from pc.kolivas.org (pc [192.168.1.251]) by mail.kolivas.org (Postfix) with ESMTP id 090373EA9E; Tue, 7 Dec 2004 10:56:58 +1100 (EST) References: <20041206205305.GA11970@soohrt.org> <20041206134849.498bfc93.davem@davemloft.net> <20041206224107.GA8529@soohrt.org> Message-ID: X-Mailer: http://www.courier-mta.org/cone/ From: Con Kolivas To: Karsten Desler Cc: David =?ISO-8859-1?B?Uy4=?= Miller , netdev@oss.sgi.com, linux-kernel@vger.kernel.org Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets Date: Tue, 07 Dec 2004 10:56:58 +1100 Mime-Version: 1.0 Content-Type: text/plain; format=flowed; charset="US-ASCII" Content-Disposition: inline Content-Transfer-Encoding: 7bit X-archive-position: 12487 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kernel@kolivas.org Precedence: bulk X-list: netdev Karsten Desler writes: > * David S. Miller wrote: >> It's spending nearly half of it's time in iptables. >> Try to consolidate your rules if possible. This is the >> part of netfilter that really doesn't scale well at all. >> > > Removing the iptables rules helps reducing the load a little, but the > majority of time is still spent somewhere else. I had a similar scenario recently with a very low spec box and found it to be the QoS. Disabling traffic shaping and removing the QoS modules made it much faster. I don't know if you're using them but it's worth pointing out. Cheers, Con From romieu@fr.zoreil.com Mon Dec 6 16:15:30 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 16:15:36 -0800 (PST) Received: from fr.zoreil.com (electric-eye.fr.zoreil.com [213.41.134.224]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB70FSlr016672 for ; Mon, 6 Dec 2004 16:15:29 -0800 Received: from electric-eye.fr.zoreil.com (localhost.localdomain [127.0.0.1]) by fr.zoreil.com (8.12.10/8.12.1) with ESMTP id iB70ELvr018671; Tue, 7 Dec 2004 01:14:21 +0100 Received: (from romieu@localhost) by electric-eye.fr.zoreil.com (8.12.10/8.12.10/Submit) id iB70EKoH018670; Tue, 7 Dec 2004 01:14:20 +0100 Date: Tue, 7 Dec 2004 01:14:19 +0100 From: Francois Romieu To: akpm@osdl.org Cc: netdev@oss.sgi.com, jgarzik@pobox.com, Dorn Hetzel Subject: [patch 1/5] r8169: missing netif_poll_enable and irq ack Message-ID: <20041207001419.GB12838@electric-eye.fr.zoreil.com> References: <20041119162920.GA26836@lilah.hetzel.org> <20041119201203.GA13522@electric-eye.fr.zoreil.com> <20041120003754.GA32133@lilah.hetzel.org> <20041120002946.GA18059@electric-eye.fr.zoreil.com> <20041122181307.GA3625@lilah.hetzel.org> <20041205235519.GA21885@lilah.hetzel.org> <20041205233756.GB29236@electric-eye.fr.zoreil.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20041205233756.GB29236@electric-eye.fr.zoreil.com> User-Agent: Mutt/1.4.1i X-Organisation: Land of Sunshine Inc. X-archive-position: 12488 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: romieu@fr.zoreil.com Precedence: bulk X-list: netdev - (noticed by Jon D. Mason) rtl8169_wait_for_quiescence() needs to disable the NAPI processing but it has no reason to lock any part of the driver which would try to do the same at a later time. Let's reenable NAPI processing as soon as possible. - properly ack any aborted interruption: a reset of the device is not always enough. Signed-off-by: Francois Romieu diff -puN drivers/net/r8169.c~r8169-250 drivers/net/r8169.c --- linux-2.6.10-rc2/drivers/net/r8169.c~r8169-250 2004-12-05 22:36:14.592843434 +0100 +++ linux-2.6.10-rc2-fr/drivers/net/r8169.c 2004-12-05 22:36:14.596842782 +0100 @@ -1742,10 +1742,19 @@ static void rtl8169_schedule_work(struct static void rtl8169_wait_for_quiescence(struct net_device *dev) { + struct rtl8169_private *tp = netdev_priv(dev); + void __iomem *ioaddr = tp->mmio_addr; + synchronize_irq(dev->irq); /* Wait for any pending NAPI task to complete */ netif_poll_disable(dev); + + RTL_W16(IntrMask, 0x0000); + + RTL_W16(IntrStatus, 0xffff); + + netif_poll_enable(dev); } static void rtl8169_reinit_task(void *_data) _ From kdesler@soohrt.org Mon Dec 6 16:18:41 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 16:18:47 -0800 (PST) Received: from quickstop.soohrt.org (quickstop.soohrt.org [81.2.155.147]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB70IeVI017395 for ; Mon, 6 Dec 2004 16:18:41 -0800 Received: (qmail 31476 invoked by uid 1018); 7 Dec 2004 00:18:15 -0000 Date: Tue, 7 Dec 2004 01:18:15 +0100 From: Karsten Desler To: Con Kolivas Cc: "David S. Miller" , netdev@oss.sgi.com, linux-kernel@vger.kernel.org Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets Message-ID: <20041207001815.GA30674@quickstop.soohrt.org> References: <20041206205305.GA11970@soohrt.org> <20041206134849.498bfc93.davem@davemloft.net> <20041206224107.GA8529@soohrt.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="GvXjxJ+pjyke8COw" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.6+20040722i X-archive-position: 12489 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kdesler@soohrt.org Precedence: bulk X-list: netdev --GvXjxJ+pjyke8COw Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Con Kolivas wrote: > I had a similar scenario recently with a very low spec box and found it to > be the QoS. Disabling traffic shaping and removing the QoS modules made it > much faster. I don't know if you're using them but it's worth pointing out. No QoS loaded. .config is attached. Thanks, Karsten --GvXjxJ+pjyke8COw Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename=config CONFIG_X86=y CONFIG_MMU=y CONFIG_UID16=y CONFIG_GENERIC_ISA_DMA=y CONFIG_GENERIC_IOMAP=y CONFIG_EXPERIMENTAL=y CONFIG_CLEAN_COMPILE=y CONFIG_LOCK_KERNEL=y CONFIG_LOCALVERSION="" CONFIG_SWAP=y CONFIG_SYSVIPC=y CONFIG_POSIX_MQUEUE=y CONFIG_SYSCTL=y CONFIG_LOG_BUF_SHIFT=15 CONFIG_KOBJECT_UEVENT=y CONFIG_EMBEDDED=y CONFIG_FUTEX=y CONFIG_EPOLL=y CONFIG_SHMEM=y CONFIG_CC_ALIGN_FUNCTIONS=0 CONFIG_CC_ALIGN_LABELS=0 CONFIG_CC_ALIGN_LOOPS=0 CONFIG_CC_ALIGN_JUMPS=0 CONFIG_X86_PC=y CONFIG_MK8=y CONFIG_X86_CMPXCHG=y CONFIG_X86_XADD=y CONFIG_X86_L1_CACHE_SHIFT=6 CONFIG_RWSEM_XCHGADD_ALGORITHM=y CONFIG_X86_WP_WORKS_OK=y CONFIG_X86_INVLPG=y CONFIG_X86_BSWAP=y CONFIG_X86_POPAD_OK=y CONFIG_X86_GOOD_APIC=y CONFIG_X86_INTEL_USERCOPY=y CONFIG_X86_USE_PPRO_CHECKSUM=y CONFIG_HPET_TIMER=y CONFIG_HPET_EMULATE_RTC=y CONFIG_SMP=y CONFIG_NR_CPUS=2 CONFIG_X86_LOCAL_APIC=y CONFIG_X86_IO_APIC=y CONFIG_X86_TSC=y CONFIG_X86_MCE=y CONFIG_HIGHMEM4G=y CONFIG_HIGHMEM=y CONFIG_MTRR=y CONFIG_HAVE_DEC_LOCK=y CONFIG_REGPARM=y CONFIG_ACPI=y CONFIG_ACPI_BOOT=y CONFIG_ACPI_INTERPRETER=y CONFIG_ACPI_BLACKLIST_YEAR=0 CONFIG_ACPI_BUS=y CONFIG_ACPI_EC=y CONFIG_ACPI_POWER=y CONFIG_ACPI_PCI=y CONFIG_ACPI_SYSTEM=y CONFIG_PCI=y CONFIG_PCI_GOANY=y CONFIG_PCI_BIOS=y CONFIG_PCI_DIRECT=y CONFIG_PCI_MMCONFIG=y CONFIG_PCI_MSI=y CONFIG_PCI_NAMES=y CONFIG_BINFMT_ELF=y CONFIG_STANDALONE=y CONFIG_BLK_DEV_RAM_COUNT=16 CONFIG_INITRAMFS_SOURCE="" CONFIG_IOSCHED_NOOP=y CONFIG_IOSCHED_CFQ=y CONFIG_SCSI=y CONFIG_BLK_DEV_SD=y CONFIG_SCSI_SATA=y CONFIG_SCSI_SATA_SIL=y CONFIG_SCSI_QLA2XXX=y CONFIG_MD=y CONFIG_BLK_DEV_MD=y CONFIG_MD_RAID1=y CONFIG_NET=y CONFIG_PACKET=y CONFIG_PACKET_MMAP=y CONFIG_UNIX=y CONFIG_INET=y CONFIG_IP_TCPDIAG=y CONFIG_NETFILTER=y CONFIG_IP_NF_IPTABLES=y CONFIG_IP_NF_MATCH_LIMIT=y CONFIG_IP_NF_MATCH_IPRANGE=y CONFIG_IP_NF_MATCH_MULTIPORT=y CONFIG_IP_NF_MATCH_LENGTH=y CONFIG_IP_NF_MATCH_HASHLIMIT=y CONFIG_IP_NF_FILTER=y CONFIG_IP_NF_TARGET_REJECT=y CONFIG_IP_NF_TARGET_LOG=y CONFIG_IP_NF_TARGET_ULOG=y CONFIG_IP_NF_MANGLE=y CONFIG_NETDEVICES=y CONFIG_NET_ETHERNET=y CONFIG_MII=y CONFIG_NET_PCI=y CONFIG_E100=y CONFIG_E100_NAPI=y CONFIG_E1000=y CONFIG_E1000_NAPI=y CONFIG_TIGON3=y CONFIG_INPUT=y CONFIG_SOUND_GAMEPORT=y CONFIG_SERIO=y CONFIG_SERIO_I8042=y CONFIG_INPUT_KEYBOARD=y CONFIG_KEYBOARD_ATKBD=y CONFIG_VT=y CONFIG_VT_CONSOLE=y CONFIG_HW_CONSOLE=y CONFIG_SERIAL_8250=y CONFIG_SERIAL_8250_CONSOLE=y CONFIG_SERIAL_8250_ACPI=y CONFIG_SERIAL_8250_NR_UARTS=4 CONFIG_SERIAL_CORE=y CONFIG_SERIAL_CORE_CONSOLE=y CONFIG_UNIX98_PTYS=y CONFIG_WATCHDOG=y CONFIG_SOFT_WATCHDOG=y CONFIG_RTC=y CONFIG_HANGCHECK_TIMER=y CONFIG_VGA_CONSOLE=y CONFIG_DUMMY_CONSOLE=y CONFIG_USB_ARCH_HAS_HCD=y CONFIG_USB_ARCH_HAS_OHCI=y CONFIG_EXT2_FS=y CONFIG_EXT3_FS=y CONFIG_JBD=y CONFIG_DNOTIFY=y CONFIG_PROC_FS=y CONFIG_PROC_KCORE=y CONFIG_SYSFS=y CONFIG_TMPFS=y CONFIG_RAMFS=y CONFIG_PARTITION_ADVANCED=y CONFIG_MSDOS_PARTITION=y CONFIG_NLS=y CONFIG_NLS_DEFAULT="iso8859-1" CONFIG_NLS_CODEPAGE_437=y CONFIG_NLS_ISO8859_1=y CONFIG_4KSTACKS=y CONFIG_X86_FIND_SMP_CONFIG=y CONFIG_X86_MPPARSE=y CONFIG_GENERIC_HARDIRQS=y CONFIG_GENERIC_IRQ_PROBE=y CONFIG_X86_SMP=y CONFIG_X86_HT=y CONFIG_X86_BIOS_REBOOT=y CONFIG_X86_TRAMPOLINE=y --GvXjxJ+pjyke8COw-- From romieu@fr.zoreil.com Mon Dec 6 16:19:27 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 16:19:33 -0800 (PST) Received: from fr.zoreil.com (electric-eye.fr.zoreil.com [213.41.134.224]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB70JP3o020724 for ; Mon, 6 Dec 2004 16:19:26 -0800 Received: from electric-eye.fr.zoreil.com (localhost.localdomain [127.0.0.1]) by fr.zoreil.com (8.12.10/8.12.1) with ESMTP id iB70Favr018744; Tue, 7 Dec 2004 01:15:36 +0100 Received: (from romieu@localhost) by electric-eye.fr.zoreil.com (8.12.10/8.12.10/Submit) id iB70FZ5r018743; Tue, 7 Dec 2004 01:15:35 +0100 Date: Tue, 7 Dec 2004 01:15:35 +0100 From: Francois Romieu To: akpm@osdl.org Cc: netdev@oss.sgi.com, jgarzik@pobox.com, Dorn Hetzel Subject: [patch 2/5] r8169: C 101 Message-ID: <20041207001535.GA18672@electric-eye.fr.zoreil.com> References: <20041119162920.GA26836@lilah.hetzel.org> <20041119201203.GA13522@electric-eye.fr.zoreil.com> <20041120003754.GA32133@lilah.hetzel.org> <20041120002946.GA18059@electric-eye.fr.zoreil.com> <20041122181307.GA3625@lilah.hetzel.org> <20041205235519.GA21885@lilah.hetzel.org> <20041205233756.GB29236@electric-eye.fr.zoreil.com> <20041207001419.GB12838@electric-eye.fr.zoreil.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20041207001419.GB12838@electric-eye.fr.zoreil.com> User-Agent: Mutt/1.4.1i X-Organisation: Land of Sunshine Inc. X-archive-position: 12490 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: romieu@fr.zoreil.com Precedence: bulk X-list: netdev Back to C101 and code which gives the expected result. Signed-off-by: Francois Romieu diff -puN drivers/net/r8169.c~r8169-255 drivers/net/r8169.c --- linux-2.6.10-rc2/drivers/net/r8169.c~r8169-255 2004-12-05 22:36:19.717006983 +0100 +++ linux-2.6.10-rc2-fr/drivers/net/r8169.c 2004-12-05 22:36:19.721006330 +0100 @@ -1978,7 +1978,7 @@ static void rtl8169_pcierr_interrupt(str PCI_STATUS_REC_TARGET_ABORT | PCI_STATUS_SIG_TARGET_ABORT)); /* The infamous DAC f*ckup only happens at boot time */ - if ((tp->cp_cmd & PCIDAC) && (tp->dirty_rx == tp->cur_rx == 0)) { + if ((tp->cp_cmd & PCIDAC) && !tp->dirty_rx && !tp->cur_rx) { printk(KERN_INFO PFX "%s: disabling PCI DAC.\n", dev->name); tp->cp_cmd &= ~PCIDAC; RTL_W16(CPlusCmd, tp->cp_cmd); _ From romieu@fr.zoreil.com Mon Dec 6 16:19:27 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 16:19:35 -0800 (PST) Received: from fr.zoreil.com (electric-eye.fr.zoreil.com [213.41.134.224]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB70JPv9020726 for ; Mon, 6 Dec 2004 16:19:26 -0800 Received: from electric-eye.fr.zoreil.com (localhost.localdomain [127.0.0.1]) by fr.zoreil.com (8.12.10/8.12.1) with ESMTP id iB70GLvr018748; Tue, 7 Dec 2004 01:16:21 +0100 Received: (from romieu@localhost) by electric-eye.fr.zoreil.com (8.12.10/8.12.10/Submit) id iB70GLKm018747; Tue, 7 Dec 2004 01:16:21 +0100 Date: Tue, 7 Dec 2004 01:16:21 +0100 From: Francois Romieu To: akpm@osdl.org Cc: netdev@oss.sgi.com, jgarzik@pobox.com, Dorn Hetzel Subject: [patch 3/5] r8169: Large Send enablement Message-ID: <20041207001621.GB18672@electric-eye.fr.zoreil.com> References: <20041119162920.GA26836@lilah.hetzel.org> <20041119201203.GA13522@electric-eye.fr.zoreil.com> <20041120003754.GA32133@lilah.hetzel.org> <20041120002946.GA18059@electric-eye.fr.zoreil.com> <20041122181307.GA3625@lilah.hetzel.org> <20041205235519.GA21885@lilah.hetzel.org> <20041205233756.GB29236@electric-eye.fr.zoreil.com> <20041207001419.GB12838@electric-eye.fr.zoreil.com> <20041207001535.GA18672@electric-eye.fr.zoreil.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20041207001535.GA18672@electric-eye.fr.zoreil.com> User-Agent: Mutt/1.4.1i X-Organisation: Land of Sunshine Inc. X-archive-position: 12491 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: romieu@fr.zoreil.com Precedence: bulk X-list: netdev Large Send enablement. Acked-by: Francois Romieu Signed-off-by: Jon Mason diff -puN drivers/net/r8169.c~r8169-260 drivers/net/r8169.c --- linux-2.6.10-rc2/drivers/net/r8169.c~r8169-260 2004-12-05 22:36:22.079621298 +0100 +++ linux-2.6.10-rc2-fr/drivers/net/r8169.c 2004-12-05 22:36:22.084620482 +0100 @@ -112,7 +112,7 @@ static int multicast_filter_limit = 32; #define RX_DMA_BURST 6 /* Maximum PCI burst, '6' is 1024 */ #define TX_DMA_BURST 6 /* Maximum PCI burst, '6' is 1024 */ #define EarlyTxThld 0x3F /* 0x3F means NO early transmit */ -#define RxPacketMaxSize 0x0800 /* Maximum size supported is 16K-1 */ +#define RxPacketMaxSize 0x3FE8 /* 16K - 1 - ETH_HLEN - VLAN - CRC */ #define InterFrameGap 0x03 /* 3 means InterFrameGap = the shortest one */ #define R8169_REGS_SIZE 256 @@ -426,6 +426,9 @@ static void rtl8169_tx_timeout(struct ne static struct net_device_stats *rtl8169_get_stats(struct net_device *netdev); static int rtl8169_rx_interrupt(struct net_device *, struct rtl8169_private *, void __iomem *); +static int rtl8169_change_mtu(struct net_device *netdev, int new_mtu); +static void rtl8169_down(struct net_device *dev); + #ifdef CONFIG_R8169_NAPI static int rtl8169_poll(struct net_device *dev, int *budget); #endif @@ -1237,8 +1240,6 @@ rtl8169_init_board(struct pci_dev *pdev, } tp->chipset = i; - tp->rx_buf_sz = RX_BUF_SIZE; - *ioaddr_out = ioaddr; *dev_out = dev; out: @@ -1320,6 +1321,7 @@ rtl8169_init_one(struct pci_dev *pdev, c dev->watchdog_timeo = RTL8169_TX_TIMEOUT; dev->irq = pdev->irq; dev->base_addr = (unsigned long) ioaddr; + dev->change_mtu = rtl8169_change_mtu; #ifdef CONFIG_R8169_NAPI dev->poll = rtl8169_poll; @@ -1448,13 +1450,22 @@ static int rtl8169_resume(struct pci_dev #endif /* CONFIG_PM */ -static int -rtl8169_open(struct net_device *dev) +static void rtl8169_set_rxbufsize(struct rtl8169_private *tp, + struct net_device *dev) +{ + unsigned int mtu = dev->mtu; + + tp->rx_buf_sz = (mtu > RX_BUF_SIZE) ? mtu + ETH_HLEN + 8 : RX_BUF_SIZE; +} + +static int rtl8169_open(struct net_device *dev) { struct rtl8169_private *tp = netdev_priv(dev); struct pci_dev *pdev = tp->pci_dev; int retval; + rtl8169_set_rxbufsize(tp, dev); + retval = request_irq(dev->irq, rtl8169_interrupt, SA_SHIRQ, dev->name, dev); if (retval < 0) @@ -1534,8 +1545,8 @@ rtl8169_hw_start(struct net_device *dev) RTL_W8(ChipCmd, CmdTxEnb | CmdRxEnb); RTL_W8(EarlyTxThres, EarlyTxThld); - // For gigabit rtl8169 - RTL_W16(RxMaxSize, RxPacketMaxSize); + // For gigabit rtl8169, MTU + header + CRC + VLAN + RTL_W16(RxMaxSize, tp->rx_buf_sz); // Set Rx Config register i = rtl8169_rx_config | @@ -1576,6 +1587,37 @@ rtl8169_hw_start(struct net_device *dev) netif_start_queue(dev); } +static int rtl8169_change_mtu(struct net_device *dev, int new_mtu) +{ + struct rtl8169_private *tp = netdev_priv(dev); + int ret = 0; + + if (new_mtu < ETH_ZLEN || new_mtu > RxPacketMaxSize) + return -EINVAL; + + dev->mtu = new_mtu; + + if (!netif_running(dev)) + goto out; + + rtl8169_down(dev); + + rtl8169_set_rxbufsize(tp, dev); + + ret = rtl8169_init_ring(dev); + if (ret < 0) + goto out; + + rtl8169_hw_start(dev); + + netif_poll_enable(dev); + + rtl8169_request_timer(dev); + +out: + return ret; +} + static inline void rtl8169_make_unusable_by_asic(struct RxDesc *desc) { desc->addr = 0x0badbadbadbadbadull; @@ -2264,19 +2306,17 @@ static int rtl8169_poll(struct net_devic } #endif -static int -rtl8169_close(struct net_device *dev) +static void rtl8169_down(struct net_device *dev) { struct rtl8169_private *tp = netdev_priv(dev); - struct pci_dev *pdev = tp->pci_dev; void __iomem *ioaddr = tp->mmio_addr; + rtl8169_delete_timer(dev); + netif_stop_queue(dev); flush_scheduled_work(); - rtl8169_delete_timer(dev); - spin_lock_irq(&tp->lock); /* Stop the chip's Tx and Rx DMA processes. */ @@ -2291,13 +2331,27 @@ rtl8169_close(struct net_device *dev) spin_unlock_irq(&tp->lock); - free_irq(dev->irq, dev); + synchronize_irq(dev->irq); netif_poll_disable(dev); + /* Give a racing hard_start_xmit a few cycles to complete. */ + set_current_state(TASK_UNINTERRUPTIBLE); + schedule_timeout(1); + rtl8169_tx_clear(tp); rtl8169_rx_clear(tp); +} + +static int rtl8169_close(struct net_device *dev) +{ + struct rtl8169_private *tp = netdev_priv(dev); + struct pci_dev *pdev = tp->pci_dev; + + rtl8169_down(dev); + + free_irq(dev->irq, dev); pci_free_consistent(pdev, R8169_RX_RING_BYTES, tp->RxDescArray, tp->RxPhyAddr); _ From romieu@fr.zoreil.com Mon Dec 6 16:19:27 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 16:19:36 -0800 (PST) Received: from fr.zoreil.com (electric-eye.fr.zoreil.com [213.41.134.224]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB70JPcs020727 for ; Mon, 6 Dec 2004 16:19:26 -0800 Received: from electric-eye.fr.zoreil.com (localhost.localdomain [127.0.0.1]) by fr.zoreil.com (8.12.10/8.12.1) with ESMTP id iB70HNvr018753; Tue, 7 Dec 2004 01:17:23 +0100 Received: (from romieu@localhost) by electric-eye.fr.zoreil.com (8.12.10/8.12.10/Submit) id iB70HMPY018752; Tue, 7 Dec 2004 01:17:22 +0100 Date: Tue, 7 Dec 2004 01:17:22 +0100 From: Francois Romieu To: akpm@osdl.org Cc: netdev@oss.sgi.com, jgarzik@pobox.com, Dorn Hetzel Subject: [patch 4/5] r8169: reduce max MTU for large frames Message-ID: <20041207001722.GC18672@electric-eye.fr.zoreil.com> References: <20041119162920.GA26836@lilah.hetzel.org> <20041119201203.GA13522@electric-eye.fr.zoreil.com> <20041120003754.GA32133@lilah.hetzel.org> <20041120002946.GA18059@electric-eye.fr.zoreil.com> <20041122181307.GA3625@lilah.hetzel.org> <20041205235519.GA21885@lilah.hetzel.org> <20041205233756.GB29236@electric-eye.fr.zoreil.com> <20041207001419.GB12838@electric-eye.fr.zoreil.com> <20041207001535.GA18672@electric-eye.fr.zoreil.com> <20041207001621.GB18672@electric-eye.fr.zoreil.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20041207001621.GB18672@electric-eye.fr.zoreil.com> User-Agent: Mutt/1.4.1i X-Organisation: Land of Sunshine Inc. X-archive-position: 12492 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: romieu@fr.zoreil.com Precedence: bulk X-list: netdev The device does not support the whole mtu range it claims. Experimenting with the Tx threshold and/or the PCI burst size does not seem to improve the behavior. Signed-off-by: Francois Romieu diff -puN drivers/net/r8169.c~r8169-265 drivers/net/r8169.c --- linux-2.6.10-rc2/drivers/net/r8169.c~r8169-265 2004-12-05 22:36:25.000000000 +0100 +++ linux-2.6.10-rc2-fr/drivers/net/r8169.c 2004-12-07 00:54:48.313082500 +0100 @@ -112,7 +112,8 @@ static int multicast_filter_limit = 32; #define RX_DMA_BURST 6 /* Maximum PCI burst, '6' is 1024 */ #define TX_DMA_BURST 6 /* Maximum PCI burst, '6' is 1024 */ #define EarlyTxThld 0x3F /* 0x3F means NO early transmit */ -#define RxPacketMaxSize 0x3FE8 /* 16K - 1 - ETH_HLEN - VLAN - CRC */ +#define RxPacketMaxSize 0x3FE8 /* 16K - 1 - ETH_HLEN - VLAN - CRC... */ +#define SafeMtu 0x1c20 /* ... actually life sucks beyond ~7k */ #define InterFrameGap 0x03 /* 3 means InterFrameGap = the shortest one */ #define R8169_REGS_SIZE 256 @@ -1592,9 +1593,9 @@ static int rtl8169_change_mtu(struct net struct rtl8169_private *tp = netdev_priv(dev); int ret = 0; - if (new_mtu < ETH_ZLEN || new_mtu > RxPacketMaxSize) + if (new_mtu < ETH_ZLEN || new_mtu > SafeMtu) return -EINVAL; - + dev->mtu = new_mtu; if (!netif_running(dev)) _ From kdesler@soohrt.org Mon Dec 6 16:20:39 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 16:20:47 -0800 (PST) Received: from quickstop.soohrt.org (quickstop.soohrt.org [81.2.155.147]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB70KcXf021563 for ; Mon, 6 Dec 2004 16:20:38 -0800 Received: (qmail 31879 invoked by uid 1018); 7 Dec 2004 00:20:12 -0000 Date: Tue, 7 Dec 2004 01:20:12 +0100 From: Karsten Desler To: Bernd Eckenfels Cc: "David S. Miller" , netdev@oss.sgi.com, linux-kernel@vger.kernel.org Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets Message-ID: <20041207002012.GB30674@quickstop.soohrt.org> References: <20041206224107.GA8529@soohrt.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.6+20040722i X-archive-position: 12493 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kdesler@soohrt.org Precedence: bulk X-list: netdev Bernd Eckenfels wrote: > In article <20041206224107.GA8529@soohrt.org> you wrote: > > Removing the iptables rules helps reducing the load a little, but the > > majority of time is still spent somewhere else. > > In handling Interrupts. Are those equally sidtributed on eth0 and eth1? Yes they are. Thanks, Karsten CPU0 CPU1 0: 117199776 133677244 IO-APIC-edge timer 1: 0 9 IO-APIC-edge i8042 8: 0 4 IO-APIC-edge rtc 9: 0 0 IO-APIC-level acpi 169: 139 893669684 IO-APIC-level eth0 177: 919803109 30665 IO-APIC-level eth1 209: 414257 413316 IO-APIC-level libata NMI: 0 0 LOC: 250918849 250918819 ERR: 0 MIS: 0 From romieu@fr.zoreil.com Mon Dec 6 16:23:33 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 16:23:40 -0800 (PST) Received: from fr.zoreil.com (electric-eye.fr.zoreil.com [213.41.134.224]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB70NRon022516 for ; Mon, 6 Dec 2004 16:23:32 -0800 Received: from electric-eye.fr.zoreil.com (localhost.localdomain [127.0.0.1]) by fr.zoreil.com (8.12.10/8.12.1) with ESMTP id iB70M8vr018922; Tue, 7 Dec 2004 01:22:08 +0100 Received: (from romieu@localhost) by electric-eye.fr.zoreil.com (8.12.10/8.12.10/Submit) id iB70M8de018921; Tue, 7 Dec 2004 01:22:08 +0100 Date: Tue, 7 Dec 2004 01:22:08 +0100 From: Francois Romieu To: akpm@osdl.org Cc: netdev@oss.sgi.com, jgarzik@pobox.com, Dorn Hetzel Subject: [patch 5/5] r8169: oversized driver field for ethtool Message-ID: <20041207002208.GA18910@electric-eye.fr.zoreil.com> References: <20041119201203.GA13522@electric-eye.fr.zoreil.com> <20041120003754.GA32133@lilah.hetzel.org> <20041120002946.GA18059@electric-eye.fr.zoreil.com> <20041122181307.GA3625@lilah.hetzel.org> <20041205235519.GA21885@lilah.hetzel.org> <20041205233756.GB29236@electric-eye.fr.zoreil.com> <20041207001419.GB12838@electric-eye.fr.zoreil.com> <20041207001535.GA18672@electric-eye.fr.zoreil.com> <20041207001621.GB18672@electric-eye.fr.zoreil.com> <20041207001722.GC18672@electric-eye.fr.zoreil.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20041207001722.GC18672@electric-eye.fr.zoreil.com> User-Agent: Mutt/1.4.1i X-Organisation: Land of Sunshine Inc. X-archive-position: 12494 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: romieu@fr.zoreil.com Precedence: bulk X-list: netdev Reported by Richard Dawe : - RTL8169_DRIVER_NAME contains more than the 32 characters allowed for the driver field; - remove RTL8169_DRIVER_NAME as it is only used once. Signed-off-by: Francois Romieu diff -puN drivers/net/r8169.c~r8169-280 drivers/net/r8169.c --- linux-2.6.10-rc2/drivers/net/r8169.c~r8169-280 2004-12-07 00:56:46.864676094 +0100 +++ linux-2.6.10-rc2-fr/drivers/net/r8169.c 2004-12-07 00:57:11.031721347 +0100 @@ -63,7 +63,6 @@ VERSION 1.6LK <2004/04/14> #define RTL8169_VERSION "1.6LK" #define MODULENAME "r8169" -#define RTL8169_DRIVER_NAME MODULENAME " Gigabit Ethernet driver " RTL8169_VERSION #define PFX MODULENAME ": " #ifdef RTL8169_DEBUG @@ -564,8 +563,8 @@ static void rtl8169_get_drvinfo(struct n { struct rtl8169_private *tp = netdev_priv(dev); - strcpy(info->driver, RTL8169_DRIVER_NAME); - strcpy(info->version, RTL8169_VERSION ); + strcpy(info->driver, MODULENAME); + strcpy(info->version, RTL8169_VERSION); strcpy(info->bus_info, pci_name(tp->pci_dev)); } @@ -1282,7 +1281,8 @@ rtl8169_init_one(struct pci_dev *pdev, c board_idx++; if (!printed_version) { - printk(KERN_INFO RTL8169_DRIVER_NAME " loaded\n"); + printk(KERN_INFO "%s Gigabit Ethernet driver %s loaded\n", + MODULENAME, RTL8169_VERSION); printed_version = 1; } _ From andreaf@cs.columbia.edu Mon Dec 6 16:28:26 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 16:28:30 -0800 (PST) Received: from cs.columbia.edu (cs.columbia.edu [128.59.16.20]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB70SPqb023077 for ; Mon, 6 Dec 2004 16:28:26 -0800 Received: from lion.cs.columbia.edu (IDENT:6RR5CZKiC9xg06/FDPktbe+pMtp8DrwM@lion.cs.columbia.edu [128.59.16.120]) by cs.columbia.edu (8.12.10/8.12.10) with ESMTP id iB70S1TO017519 for ; Mon, 6 Dec 2004 19:28:02 -0500 (EST) Received: from [128.59.17.219] (dhcp69.cs.columbia.edu [128.59.17.219]) by lion.cs.columbia.edu (8.12.9/8.12.9) with ESMTP id iB70S1Kj024362 for ; Mon, 6 Dec 2004 19:28:01 -0500 Message-ID: <41B4F914.2070401@cs.columbia.edu> Date: Mon, 06 Dec 2004 19:28:04 -0500 From: Andrea G Forte User-Agent: Mozilla Thunderbird 0.9 (Windows/20041103) X-Accept-Language: en-us, en MIME-Version: 1.0 To: netdev@oss.sgi.com Subject: IP aliasing and IP change delay. Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-PMX-Version: 4.7.0.111621, Antispam-Engine: 2.0.2.0, Antispam-Data: 2004.12.6.29 X-PerlMx-Spam: Gauge=IIIIIII, Probability=7%, Report='__CT 0, __CTE 0, __CT_TEXT_PLAIN 0, __HAS_MSGID 0, __MIME_VERSION 0, __SANE_MSGID 0' X-archive-position: 12495 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: andreaf@cs.columbia.edu Precedence: bulk X-list: netdev Hello all, I am new to this list but I hope you can help me. I have been trying to use two different IP addresses on the same PCMCIA wireless card. For doing this I tried the classic ifconfig wlan0:0 inet xxx.xxx.xxx.xxx route ...... and I also tried ip address add xxx.xxx.xxx.xxx dev wlan0 The problem is that after I issue the command, the IP is actually changed several hundred of milliseconds later, while if I do not create an alias and change the IP twice on the same interface (using ifconfig), then the change of IP is really fast, practically it changes starting from the packet following the command. Anybody has some ideas why there is such a double behaviour if using wlan0 and wlan0:0 or using only wlan0?? Thank you very much for your help, Andrea From herbert@gondor.apana.org.au Mon Dec 6 17:19:24 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 17:19:31 -0800 (PST) Received: from arnor.apana.org.au (c211-30-229-77.rivrw4.nsw.optusnet.com.au [211.30.229.77]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB71JNtm029150 for ; Mon, 6 Dec 2004 17:19:24 -0800 Received: from gondolin.me.apana.org.au ([192.168.0.6] ident=mail) by arnor.apana.org.au with esmtp (Exim 3.35 #1 (Debian)) id 1CbTyw-0000If-00; Tue, 07 Dec 2004 12:17:34 +1100 Received: from herbert by gondolin.me.apana.org.au with local (Exim 3.36 #1 (Debian)) id 1CbTv3-000732-00; Tue, 07 Dec 2004 12:13:33 +1100 From: Herbert Xu To: hasso@estpak.ee (Hasso Tepper) Subject: Re: [patch 4/10] s390: network driver. Cc: hadi@cyberus.ca, paul@clubi.ie, thomas.spatzier@de.ibm.com, jgarzik@pobox.com, netdev@oss.sgi.com Organization: Core In-Reply-To: <200412061642.00552.hasso@estpak.ee> X-Newsgroups: apana.lists.os.linux.netdev User-Agent: tin/1.7.4-20040225 ("Benbecula") (UNIX) (Linux/2.4.27-hx-1-686-smp (i686)) Message-Id: Date: Tue, 07 Dec 2004 12:13:33 +1100 X-archive-position: 12496 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: herbert@gondor.apana.org.au Precedence: bulk X-list: netdev Hasso Tepper wrote: > > Seems that it's no longer true. Seems that kernel is now trying as hard as > possible not to loose any data - data is queued and if the queue will be > full, all related sockets will be blocked to notify application. So, one > socket approach don't work any more for Quagga/Zebra. No problem, we can > take the "one socket per interface" approach. And we already have link > detection implemented to notify daemons. I don't see why this should be happening. Can you please provide a minimal program that reproduces this blocking problem? For example, something that sends a packet to a downed interface and then sends one to an interface that's up? -- Visit Openswan at http://www.openswan.org/ Email: Herbert Xu ~{PmV>HI~} Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt From hadi@cyberus.ca Mon Dec 6 18:23:35 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 18:23:42 -0800 (PST) Received: from mx02.cybersurf.com (mx02.cybersurf.com [209.197.145.105]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB72NYV8032161 for ; Mon, 6 Dec 2004 18:23:35 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx02.cybersurf.com with esmtp (Exim 4.30) id 1CbVwZ-0007ho-Ix for netdev@oss.sgi.com; Mon, 06 Dec 2004 22:23:15 -0500 Received: from cpe0030ab124d2f-cm014500000962.cpe.net.cable.rogers.com ([24.103.99.32] helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1CbV0G-0007vh-CS; Mon, 06 Dec 2004 21:23:00 -0500 Subject: Re: [patch 4/10] s390: network driver. From: jamal Reply-To: hadi@cyberus.ca To: Herbert Xu Cc: Hasso Tepper , paul@clubi.ie, thomas.spatzier@de.ibm.com, jgarzik@pobox.com, netdev@oss.sgi.com In-Reply-To: References: Content-Type: text/plain Organization: jamalopolous Message-Id: <1102386174.1093.21.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 06 Dec 2004 21:22:55 -0500 Content-Transfer-Encoding: 7bit X-archive-position: 12497 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev On Mon, 2004-12-06 at 20:13, Herbert Xu wrote: > Hasso Tepper wrote: > > > > Seems that it's no longer true. Seems that kernel is now trying as hard as > > possible not to loose any data - data is queued and if the queue will be > > full, all related sockets will be blocked to notify application. So, one > > socket approach don't work any more for Quagga/Zebra. No problem, we can > > take the "one socket per interface" approach. And we already have link > > detection implemented to notify daemons. > > I don't see why this should be happening. Can you please provide a > minimal program that reproduces this blocking problem? For example, > something that sends a packet to a downed interface and then sends > one to an interface that's up? This may be something to do with socket level changes maybe? Does this happen with sockets that are not raw? Having said that: I think getting rid of obsolete messages is important. One of the suggested schemes of operation sounds to be the least brutal. Jgarzik, I thought about it a little and it seems to me that allowing the device driver to drop packets on txmit when netcarrier is off is not that bad. This is almost like pretending packets were dropped on the wire. Thoughts? cheers, jamal From hadi@cyberus.ca Mon Dec 6 18:28:24 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 18:28:29 -0800 (PST) Received: from mx01.cybersurf.com (mx01.cybersurf.com [209.197.145.104]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB72SO9T032711 for ; Mon, 6 Dec 2004 18:28:24 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx01.cybersurf.com with esmtp (Exim 4.30) id 1CbW0h-0002UR-DI for netdev@oss.sgi.com; Mon, 06 Dec 2004 22:27:31 -0500 Received: from cpe0030ab124d2f-cm014500000962.cpe.net.cable.rogers.com ([24.103.99.32] helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1CbV4s-00004u-R6; Mon, 06 Dec 2004 21:27:47 -0500 Subject: Re: [PATCH] rtnetlink & address family problem From: jamal Reply-To: hadi@cyberus.ca To: Thomas Graf Cc: Michal Ludvig , Andrew Morton , Stephen Hemminger , netdev@oss.sgi.com, Jan Kara In-Reply-To: <20041206140214.GA749@postel.suug.ch> References: <41B0A5B4.6060108@suse.cz> <20041206140214.GA749@postel.suug.ch> Content-Type: text/plain Organization: jamalopolous Message-Id: <1102386461.1093.26.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 06 Dec 2004 21:27:41 -0500 Content-Transfer-Encoding: 7bit X-archive-position: 12498 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev On Mon, 2004-12-06 at 09:02, Thomas Graf wrote: > Your patch would fix this issue but might break various things. The > actual problem is that iproute2 doesn't check the family in its filter. > It blindly assumes that the kernel only returns addresses of the kind it > has requested. I can understand if you think the current behaviour > is wrong but we shouldn't change it in the middle of a stable tree. Why would it be wrong? The PF_UNSPEC is there for a purpose. If user space decides it wants to flush ipv4 addresses blindly that user spaces fault. The patch you attached seems legit. did you verify it? BTW, Stephen - are you still updating iproute2? cheers, jamal From hadi@cyberus.ca Mon Dec 6 18:40:31 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 18:40:36 -0800 (PST) Received: from mx02.cybersurf.com (mx02.cybersurf.com [209.197.145.105]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB72eVwh001240 for ; Mon, 6 Dec 2004 18:40:31 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx02.cybersurf.com with esmtp (Exim 4.30) id 1CbWCu-0008IW-T2 for netdev@oss.sgi.com; Mon, 06 Dec 2004 22:40:08 -0500 Received: from cpe0030ab124d2f-cm014500000962.cpe.net.cable.rogers.com ([24.103.99.32] helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1CbVGZ-0001ld-De; Mon, 06 Dec 2004 21:39:51 -0500 Subject: Re: [patch 4/10] s390: network driver. From: jamal Reply-To: hadi@cyberus.ca To: Hasso Tepper Cc: Paul Jakma , Thomas Spatzier , jgarzik@pobox.com, netdev@oss.sgi.com In-Reply-To: <200412061642.00552.hasso@estpak.ee> References: <1102332479.1048.2243.camel@jzny.localdomain> <200412061642.00552.hasso@estpak.ee> Content-Type: text/plain Organization: jamalopolous Message-Id: <1102387179.1091.39.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 06 Dec 2004 21:39:39 -0500 Content-Transfer-Encoding: 7bit X-archive-position: 12499 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev On Mon, 2004-12-06 at 09:42, Hasso Tepper wrote: > I'm trying to summarize. The approach - one raw socket to send/receive > messages no matter how many interfaces are used - worked for Quagga/Zebra > routing software daemons till now. If some of these interfaces was > operationally down, socket wasn't blocked even if the queue was full. > > In > fact "man sendmsg" has still text: > > ENOBUFS > The output queue for a network interface was full. This gener- > ally indicates that the interface has stopped sending, but may > be caused by transient congestion. (Normally, this does not > occur in Linux. Packets are just silently dropped when a device > queue overflows.) > > Seems that it's no longer true. Seems that kernel is now trying as hard as > possible not to loose any data - data is queued and if the queue will be > full, all related sockets will be blocked to notify application. We need to investigate why this happens. It doesnt sound like good behavior neither legit. Does this happen to only raw sockets? Herbert asked for a sample small program that reproduces it. > So, one > socket approach don't work any more for Quagga/Zebra. No problem, we can > take the "one socket per interface" approach. And we already have link > detection implemented to notify daemons. > I dont think it would be necessary with latest changes to notifications in 2.6.x > But there will be still potential problem - sending the "interface down" > message from kernel to the zebra daemon and then to the routing protocol > daemon takes time. And during this time daemon can send packets which will > sit in queue and may cause many problems if sent to the network later (if > link comes up again). Think about statelss routing protocols like rip(ng), > ipv6 router advertisements etc. They may carry the info that's no longer > true causing routing loops etc. > > And to clarify a little bit: no - the Quagga/Zebra didn't work with previous > approach perfectly. I fact with link detection and socket per interface it > will be better than ever no matter what's the kernel behaviour. We just > want to make sure that solution will be bulletproof. > > So, problem - how can we make sure that no potentially dangerous aged > (routing) info will be in the network? I think the idea of having driver junk packets when link is down does sound plausible as a solution. cheers, jamal From hadi@cyberus.ca Mon Dec 6 18:47:06 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 18:47:12 -0800 (PST) Received: from mx03.cybersurf.com (mx03.cybersurf.com [209.197.145.106]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB72l54R001960 for ; Mon, 6 Dec 2004 18:47:06 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx03.cybersurf.com with esmtp (Exim 4.30) id 1CbVN9-0001sn-E3 for netdev@oss.sgi.com; Mon, 06 Dec 2004 21:46:39 -0500 Received: from cpe0030ab124d2f-cm014500000962.cpe.net.cable.rogers.com ([24.103.99.32] helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1CbVNA-0002rO-Lh; Mon, 06 Dec 2004 21:46:40 -0500 Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets From: jamal Reply-To: hadi@cyberus.ca To: Karsten Desler Cc: Bernd Eckenfels , "David S. Miller" , netdev@oss.sgi.com, linux-kernel@vger.kernel.org In-Reply-To: <20041207002012.GB30674@quickstop.soohrt.org> References: <20041206224107.GA8529@soohrt.org> <20041207002012.GB30674@quickstop.soohrt.org> Content-Type: text/plain Organization: jamalopolous Message-Id: <1102387595.1088.48.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 06 Dec 2004 21:46:35 -0500 Content-Transfer-Encoding: 7bit X-archive-position: 12500 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev Your numbers are very suspect. You may be having other issues in the box. You should be able to do much higher packet rates even with iptables compiled in. Some numbers at: http://www.suug.ch/sucon/04/slides/pkt_cls.pdf If all you need is std filtering then consider using tc actions. I do have a suspicion that your problem has to do with your machine more than it does with Linux. cheers, jamal On Mon, 2004-12-06 at 19:20, Karsten Desler wrote: > Bernd Eckenfels wrote: > > In article <20041206224107.GA8529@soohrt.org> you wrote: > > > Removing the iptables rules helps reducing the load a little, but the > > > majority of time is still spent somewhere else. > > > > In handling Interrupts. Are those equally sidtributed on eth0 and eth1? > > Yes they are. > > Thanks, > Karsten > > CPU0 CPU1 > 0: 117199776 133677244 IO-APIC-edge timer > 1: 0 9 IO-APIC-edge i8042 > 8: 0 4 IO-APIC-edge rtc > 9: 0 0 IO-APIC-level acpi > 169: 139 893669684 IO-APIC-level eth0 > 177: 919803109 30665 IO-APIC-level eth1 > 209: 414257 413316 IO-APIC-level libata > NMI: 0 0 > LOC: 250918849 250918819 > ERR: 0 > MIS: 0 > > From kdesler@soohrt.org Mon Dec 6 18:55:23 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 18:55:27 -0800 (PST) Received: from quickstop.soohrt.org (quickstop.soohrt.org [81.2.155.147]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB72tLMH002670 for ; Mon, 6 Dec 2004 18:55:22 -0800 Received: (qmail 2092 invoked by uid 1000); 7 Dec 2004 02:54:56 -0000 Date: Tue, 7 Dec 2004 03:54:56 +0100 From: Karsten Desler To: jamal Cc: Bernd Eckenfels , "David S. Miller" , netdev@oss.sgi.com, linux-kernel@vger.kernel.org Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets Message-ID: <20041207025456.GA525@soohrt.org> References: <20041206224107.GA8529@soohrt.org> <20041207002012.GB30674@quickstop.soohrt.org> <1102387595.1088.48.camel@jzny.localdomain> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: <1102387595.1088.48.camel@jzny.localdomain> User-Agent: Mutt/1.5.6+20040722i X-archive-position: 12501 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kdesler@soohrt.org Precedence: bulk X-list: netdev * jamal wrote: > > Your numbers are very suspect. You may be having other issues in the > box. You should be able to do much higher packet rates even with > iptables compiled in. I know, and I have no idea why I'm not. > Some numbers at: > > http://www.suug.ch/sucon/04/slides/pkt_cls.pdf > > If all you need is std filtering then consider using tc actions. Thanks, I'll look into it. > I do have a suspicion that your problem has to do with your machine > more than it does with Linux. But what could be the reason? I'm really out of ideas. The only thing I can think off is the 66/64 PCI bus and the disadvantageous placement of the PCI cards, but neither should cause a higher CPU usage. If the bus couldn't keep up, I'd get packetloss. - Karsten From hadi@cyberus.ca Mon Dec 6 19:19:25 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 19:19:29 -0800 (PST) Received: from mx03.cybersurf.com (mx03.cybersurf.com [209.197.145.106]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB73JOSL004074 for ; Mon, 6 Dec 2004 19:19:25 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx03.cybersurf.com with esmtp (Exim 4.30) id 1CbVsQ-0002BC-ML for netdev@oss.sgi.com; Mon, 06 Dec 2004 22:18:58 -0500 Received: from cpe0030ab124d2f-cm014500000962.cpe.net.cable.rogers.com ([24.103.99.32] helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1CbVsR-0007WW-1j; Mon, 06 Dec 2004 22:18:59 -0500 Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets From: jamal Reply-To: hadi@cyberus.ca To: Karsten Desler Cc: Bernd Eckenfels , "David S. Miller" , netdev@oss.sgi.com, linux-kernel@vger.kernel.org In-Reply-To: <20041207025456.GA525@soohrt.org> References: <20041206224107.GA8529@soohrt.org> <20041207002012.GB30674@quickstop.soohrt.org> <1102387595.1088.48.camel@jzny.localdomain> <20041207025456.GA525@soohrt.org> Content-Type: text/plain Organization: jamalopolous Message-Id: <1102389533.1089.51.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 06 Dec 2004 22:18:54 -0500 Content-Transfer-Encoding: 7bit X-archive-position: 12502 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev On Mon, 2004-12-06 at 21:54, Karsten Desler wrote: > The only thing I can think off is the 66/64 PCI bus and the > disadvantageous placement of the PCI cards, but neither should cause a > higher CPU usage. If the bus couldn't keep up, I'd get packetloss. > cant tell offhand; it looks like a modern piece of hardware. Are you sure you are using NAPI? This is an e1000, correct? cheers, jamal From hadi@cyberus.ca Mon Dec 6 19:21:21 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 19:21:25 -0800 (PST) Received: from mx01.cybersurf.com (mx01.cybersurf.com [209.197.145.104]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB73LH19004288 for ; Mon, 6 Dec 2004 19:21:20 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx01.cybersurf.com with esmtp (Exim 4.30) id 1CbWpp-0003al-T6 for netdev@oss.sgi.com; Mon, 06 Dec 2004 23:20:22 -0500 Received: from cpe0030ab124d2f-cm014500000962.cpe.net.cable.rogers.com ([24.103.99.32] helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1CbVu3-0007nj-1O; Mon, 06 Dec 2004 22:20:39 -0500 Subject: Re: 1.03Mpps on e1000 (was: Re: [E1000-devel] Transmission limit) From: jamal Reply-To: hadi@cyberus.ca To: Martin Josefsson Cc: Robert Olsson , Lennert Buytenhek , Scott Feldman , P@draigBrady.com, mellia@prezzemolo.polito.it, Jorge Manuel Finochietto , Giulio Galante , netdev@oss.sgi.com In-Reply-To: References: <16820.44722.748743.6711@robur.slu.se> Content-Type: text/plain Organization: jamalopolous Message-Id: <1102389633.1091.54.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 06 Dec 2004 22:20:34 -0500 Content-Transfer-Encoding: 7bit X-archive-position: 12503 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev Can someone post the patches and a small README? As luck would have it my ext3 just decided to fail me on my first -rc3 boot. Dammit. cheers, jamal On Mon, 2004-12-06 at 17:29, Martin Josefsson wrote: > On Mon, 6 Dec 2004, Robert Olsson wrote: > > > pktgen performance is measured on router box. Remember Scotts patch uses > > 4096 TX buffers and w. pktgen we use clone_skb. So with real skb's we probably > > see lower performance due to this. This may explain results below so routing > > performance doesn't follow pktgen performance as seen. > > I've performed some tests with and without clone_skb with various versions > of the driver. > > > Vanilla. T-PUT 657 kpps. pktgen TX perf 818 kpps > > > e1000-TX-prefetch+scott tx patch. T-PUT 540 kpps. pktgen TX perf 1.48 Mpps > > > e1000-TX-prefetch. T-PUT 657 kpps. pktgen TX perf 1.15 Mpps > > This matches the data I see in my tests here with and without clone_skb. > > I've included a lot of pps numbers below, they might need some > description. > > I tested generating packets with four diffrent drivers with and without > clone_skb. > > vanilla is the vanilla driver in 2.6.10-rc3 > > copy is using the patch found at the bottom of this mail, just a small > test to see if there's any gain or loss using "static" buffers to dma > from. Prefetch doesn't help at all here, just makes things worse, even for > clone_skb. Tried with delayed TDT updating as well, didn't help. > > vanilla + prefetch is just the vanilla driver + prefetching. > > feldman tx is using scotts tx-path rewrite patch. > I didn't bother listing feldman tx + prefetch as the results were even > lower for the non clone_skb case. > The only thing I can think of that can cause this is cache trashing, or > overhead in slab when we have a lot of skb's in the wild. > > I don't have oprofile on my testmachine at the moment and it's time to go > to bed now, maybe tomorrow... > > Does anyone have any suggestions of what to test next? > > > vanilla and clone > 60 854886 > 64 772341 > 68 759531 > 72 758872 > 76 758926 > 80 761136 > 84 742109 > 88 742070 > 92 741616 > 96 744083 > 100 727430 > 104 725242 > 108 724153 > 112 725841 > 116 707331 > 120 706000 > 124 704923 > 128 662547 > > vanilla and noclone > 60 748552 > 64 702464 > 68 649066 > 72 671992 > 76 680251 > 80 627711 > 84 625468 > 88 640115 > 92 679365 > 96 650544 > 100 666423 > 104 652057 > 108 665821 > 112 679443 > 116 652507 > 120 661279 > 124 648627 > 128 635780 > > copy and clone > 60 897165 > 64 872767 > 68 750694 > 72 750427 > 76 749583 > 80 748242 > 84 732760 > 88 731129 > 92 732603 > 96 732631 > 100 717123 > 104 717678 > 108 716839 > 112 719258 > 116 703824 > 120 706047 > 124 701885 > 128 695575 > > copy and noclone > 60 882227 > 64 649614 > 68 691327 > 72 700706 > 76 700795 > 80 696594 > 84 686016 > 88 691689 > 92 696136 > 96 691348 > 100 684596 > 104 687800 > 108 689218 > 112 671483 > 116 675867 > 120 679089 > 124 672385 > 128 650148 > > vanilla + prefetch and clone > 60 1300075 > 64 1079069 > 68 1082091 > 72 1068791 > 76 1067630 > 80 1026222 > 84 1053055 > 88 1024442 > 92 1032112 > 96 1014844 > 100 991346 > 104 976483 > 108 947019 > 112 919193 > 116 892863 > 120 868054 > 124 844679 > 128 822347 > > vanilla + prefetch and noclone > 60 738538 > 64 800927 > 68 719832 > 72 725353 > 76 822738 > 80 743134 > 84 813520 > 88 721522 > 92 797838 > 96 724031 > 100 812198 > 104 717811 > 108 713072 > 112 789771 > 116 696027 > 120 682168 > 124 749020 > 128 703233 > > feldman tx and clone > 60 1029997 > 64 916706 > 68 898601 > 72 895378 > 76 896171 > 80 898594 > 84 861434 > 88 861446 > 92 861444 > 96 863669 > 100 837624 > 104 836225 > 108 835528 > 112 835527 > 116 817102 > 120 817101 > 124 817100 > 128 757683 > > feldman tx and noclone > 60 626646 > 64 628148 > 68 628935 > 72 625084 > 76 623527 > 80 623510 > 84 624286 > 88 625086 > 92 623907 > 96 630199 > 100 613933 > 104 618025 > 108 620326 > 112 607884 > 116 606124 > 120 538434 > 124 531699 > 128 532719 > > > > diff -X /home/gandalf/dontdiff.ny -urNp drivers/net/e1000-vanilla/e1000_main.c drivers/net/e1000/e1000_main.c > --- drivers/net/e1000-vanilla/e1000_main.c 2004-12-05 18:27:50.000000000 +0100 > +++ drivers/net/e1000/e1000_main.c 2004-12-06 22:21:10.000000000 +0100 > @@ -132,6 +132,7 @@ static void e1000_irq_disable(struct e10 > static void e1000_irq_enable(struct e1000_adapter *adapter); > static irqreturn_t e1000_intr(int irq, void *data, struct pt_regs *regs); > static boolean_t e1000_clean_tx_irq(struct e1000_adapter *adapter); > +static boolean_t e1000_alloc_tx_buffers(struct e1000_adapter *adapter); > #ifdef CONFIG_E1000_NAPI > static int e1000_clean(struct net_device *netdev, int *budget); > static boolean_t e1000_clean_rx_irq(struct e1000_adapter *adapter, > @@ -264,6 +265,7 @@ e1000_up(struct e1000_adapter *adapter) > e1000_restore_vlan(adapter); > > e1000_configure_tx(adapter); > + e1000_alloc_tx_buffers(adapter); > e1000_setup_rctl(adapter); > e1000_configure_rx(adapter); > e1000_alloc_rx_buffers(adapter); > @@ -1048,10 +1052,21 @@ e1000_configure_rx(struct e1000_adapter > void > e1000_free_tx_resources(struct e1000_adapter *adapter) > { > + struct e1000_desc_ring *tx_ring = &adapter->tx_ring; > + struct e1000_buffer *buffer_info; > struct pci_dev *pdev = adapter->pdev; > + unsigned int i; > > e1000_clean_tx_ring(adapter); > > + for(i = 0; i < tx_ring->count; i++) { > + buffer_info = &tx_ring->buffer_info[i]; > + if(buffer_info->skb) { > + kfree(buffer_info->skb); > + buffer_info->skb = NULL; > + } > + } > + > vfree(adapter->tx_ring.buffer_info); > adapter->tx_ring.buffer_info = NULL; > > @@ -1079,16 +1094,12 @@ e1000_clean_tx_ring(struct e1000_adapter > > for(i = 0; i < tx_ring->count; i++) { > buffer_info = &tx_ring->buffer_info[i]; > - if(buffer_info->skb) { > - > + if(buffer_info->dma) { > pci_unmap_page(pdev, > buffer_info->dma, > buffer_info->length, > PCI_DMA_TODEVICE); > - > - dev_kfree_skb(buffer_info->skb); > - > - buffer_info->skb = NULL; > + buffer_info->dma = 0; > } > } > > @@ -1579,8 +1590,6 @@ e1000_tx_map(struct e1000_adapter *adapt > struct e1000_buffer *buffer_info; > unsigned int len = skb->len; > unsigned int offset = 0, size, count = 0, i; > - unsigned int f; > - len -= skb->data_len; > > i = tx_ring->next_to_use; > > @@ -1600,10 +1609,12 @@ e1000_tx_map(struct e1000_adapter *adapt > size > 4)) > size -= 4; > > + skb_copy_bits(skb, offset, buffer_info->skb, size); > + > buffer_info->length = size; > buffer_info->dma = > pci_map_single(adapter->pdev, > - skb->data + offset, > + buffer_info->skb, > size, > PCI_DMA_TODEVICE); > buffer_info->time_stamp = jiffies; > @@ -1614,50 +1625,11 @@ e1000_tx_map(struct e1000_adapter *adapt > if(unlikely(++i == tx_ring->count)) i = 0; > } > > - for(f = 0; f < nr_frags; f++) { > - struct skb_frag_struct *frag; > - > - frag = &skb_shinfo(skb)->frags[f]; > - len = frag->size; > - offset = frag->page_offset; > - > - while(len) { > - buffer_info = &tx_ring->buffer_info[i]; > - size = min(len, max_per_txd); > -#ifdef NETIF_F_TSO > - /* Workaround for premature desc write-backs > - * in TSO mode. Append 4-byte sentinel desc */ > - if(unlikely(mss && f == (nr_frags-1) && size == len && size > 8)) > - size -= 4; > -#endif > - /* Workaround for potential 82544 hang in PCI-X. > - * Avoid terminating buffers within evenly-aligned > - * dwords. */ > - if(unlikely(adapter->pcix_82544 && > - !((unsigned long)(frag->page+offset+size-1) & 4) && > - size > 4)) > - size -= 4; > - > - buffer_info->length = size; > - buffer_info->dma = > - pci_map_page(adapter->pdev, > - frag->page, > - offset, > - size, > - PCI_DMA_TODEVICE); > - buffer_info->time_stamp = jiffies; > - > - len -= size; > - offset += size; > - count++; > - if(unlikely(++i == tx_ring->count)) i = 0; > - } > - } > - > i = (i == 0) ? tx_ring->count - 1 : i - 1; > - tx_ring->buffer_info[i].skb = skb; > tx_ring->buffer_info[first].next_to_watch = i; > > + dev_kfree_skb_any(skb); > + > return count; > } > > @@ -2213,11 +2185,6 @@ e1000_clean_tx_irq(struct e1000_adapter > buffer_info->dma = 0; > } > > - if(buffer_info->skb) { > - dev_kfree_skb_any(buffer_info->skb); > - buffer_info->skb = NULL; > - } > - > tx_desc->buffer_addr = 0; > tx_desc->lower.data = 0; > tx_desc->upper.data = 0; > @@ -2243,6 +2210,28 @@ e1000_clean_tx_irq(struct e1000_adapter > return cleaned; > } > > + > +static boolean_t > +e1000_alloc_tx_buffers(struct e1000_adapter *adapter) > +{ > + struct e1000_desc_ring *tx_ring = &adapter->tx_ring; > + struct e1000_buffer *buffer_info; > + unsigned int i; > + > + for (i = 0; i < tx_ring->count; i++) { > + buffer_info = &tx_ring->buffer_info[i]; > + if (!buffer_info->skb) { > + buffer_info->skb = kmalloc(2048, GFP_ATOMIC); > + if (unlikely(!buffer_info->skb)) { > + printk("eek!\n"); > + return FALSE; > + } > + } > + } > + > + return TRUE; > +} > + > /** > * e1000_clean_rx_irq - Send received data up the network stack > * @adapter: board private structure > > /Martin > > From kdesler@soohrt.org Mon Dec 6 19:25:05 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 19:25:09 -0800 (PST) Received: from quickstop.soohrt.org (quickstop.soohrt.org [81.2.155.147]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB73P4RT004877 for ; Mon, 6 Dec 2004 19:25:05 -0800 Received: (qmail 8969 invoked by uid 1000); 7 Dec 2004 03:24:38 -0000 Date: Tue, 7 Dec 2004 04:24:38 +0100 From: Karsten Desler To: jamal Cc: Bernd Eckenfels , "David S. Miller" , netdev@oss.sgi.com, linux-kernel@vger.kernel.org Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets Message-ID: <20041207032438.GA7767@soohrt.org> References: <20041206224107.GA8529@soohrt.org> <20041207002012.GB30674@quickstop.soohrt.org> <1102387595.1088.48.camel@jzny.localdomain> <20041207025456.GA525@soohrt.org> <1102389533.1089.51.camel@jzny.localdomain> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: <1102389533.1089.51.camel@jzny.localdomain> User-Agent: Mutt/1.5.6+20040722i X-archive-position: 12504 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kdesler@soohrt.org Precedence: bulk X-list: netdev * jamal wrote: > > The only thing I can think off is the 66/64 PCI bus and the > > disadvantageous placement of the PCI cards, but neither should cause a > > higher CPU usage. If the bus couldn't keep up, I'd get packetloss. > > > > cant tell offhand; it looks like a modern piece of hardware. > Are you sure you are using NAPI? This is an e1000, correct? > Yes and yes. 0000:01:01.0 Ethernet controller: Intel Corp. 82545EM Gigabit Ethernet Controller (Fiber) (rev 01) 0000:01:03.0 Ethernet controller: Intel Corp. 82546GB Gigabit Ethernet Controller (rev 03) driver: e1000 version: 5.5.4-k2-NAPI firmware-version: N/A bus-info: 0000:01:03.0 driver: e1000 version: 5.5.4-k2-NAPI firmware-version: N/A bus-info: 0000:01:01.0 From hadi@cyberus.ca Mon Dec 6 19:31:25 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 19:31:30 -0800 (PST) Received: from mx02.cybersurf.com (mx02.cybersurf.com [209.197.145.105]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB73VM3X005469 for ; Mon, 6 Dec 2004 19:31:24 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx02.cybersurf.com with esmtp (Exim 4.30) id 1CbX07-0007PE-SJ for netdev@oss.sgi.com; Mon, 06 Dec 2004 23:30:59 -0500 Received: from cpe0030ab124d2f-cm014500000962.cpe.net.cable.rogers.com ([24.103.99.32] helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1CbW3u-0000eJ-94; Mon, 06 Dec 2004 22:30:50 -0500 Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets From: jamal Reply-To: hadi@cyberus.ca To: Karsten Desler Cc: Bernd Eckenfels , "David S. Miller" , netdev@oss.sgi.com, linux-kernel@vger.kernel.org In-Reply-To: <20041207032438.GA7767@soohrt.org> References: <20041206224107.GA8529@soohrt.org> <20041207002012.GB30674@quickstop.soohrt.org> <1102387595.1088.48.camel@jzny.localdomain> <20041207025456.GA525@soohrt.org> <1102389533.1089.51.camel@jzny.localdomain> <20041207032438.GA7767@soohrt.org> Content-Type: text/plain Organization: jamalopolous Message-Id: <1102390241.1093.59.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 06 Dec 2004 22:30:41 -0500 Content-Transfer-Encoding: 7bit X-archive-position: 12505 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev On Mon, 2004-12-06 at 22:24, Karsten Desler wrote: > roller (rev 03) > > driver: e1000 > version: 5.5.4-k2-NAPI > firmware-version: N/A > bus-info: 0000:01:03.0 > > driver: e1000 > version: 5.5.4-k2-NAPI > firmware-version: N/A > bus-info: 0000:01:01.0 Beats me. Make sure it boots NAPI. Also if you can turn off ITR; intel loves to turn on that silly feature. cheers, jamal From kdesler@soohrt.org Mon Dec 6 20:03:04 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 20:03:10 -0800 (PST) Received: from quickstop.soohrt.org (quickstop.soohrt.org [81.2.155.147]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7430XP007091 for ; Mon, 6 Dec 2004 20:03:03 -0800 Received: (qmail 16775 invoked by uid 1000); 7 Dec 2004 04:02:35 -0000 Date: Tue, 7 Dec 2004 05:02:35 +0100 From: Karsten Desler To: jamal Cc: Bernd Eckenfels , "David S. Miller" , netdev@oss.sgi.com, linux-kernel@vger.kernel.org Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets Message-ID: <20041207040235.GA10501@soohrt.org> References: <20041206224107.GA8529@soohrt.org> <20041207002012.GB30674@quickstop.soohrt.org> <1102387595.1088.48.camel@jzny.localdomain> <20041207025456.GA525@soohrt.org> <1102389533.1089.51.camel@jzny.localdomain> <20041207032438.GA7767@soohrt.org> <1102390241.1093.59.camel@jzny.localdomain> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: <1102390241.1093.59.camel@jzny.localdomain> User-Agent: Mutt/1.5.6+20040722i X-archive-position: 12506 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kdesler@soohrt.org Precedence: bulk X-list: netdev * jamal wrote: > Beats me. Make sure it boots NAPI. Also if you can turn off ITR; intel > loves to turn on that silly feature. ITR was in fact activated. I think i've disabled it now (e1000.InterruptThrottleRate=0 in the kernel cmdline). And as I'm reading the e1000 code, there is no way to enable/disable NAPI without a recompile. So the fact that ethtool spat out -NAPI in the version string means that NAPI is actually used. - Karsten From mitch@sfgoth.com Mon Dec 6 21:45:35 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 21:45:39 -0800 (PST) Received: from gaz.sfgoth.com (gaz.sfgoth.com [69.36.241.230]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB75jYQM015279 for ; Mon, 6 Dec 2004 21:45:35 -0800 Received: from gaz.sfgoth.com (localhost.sfgoth.com [127.0.0.1]) by gaz.sfgoth.com (8.12.10/8.12.10) with ESMTP id iB75mfi0065495; Mon, 6 Dec 2004 21:48:41 -0800 (PST) (envelope-from mitch@gaz.sfgoth.com) Received: (from mitch@localhost) by gaz.sfgoth.com (8.12.10/8.12.6/Submit) id iB75me00065494; Mon, 6 Dec 2004 21:48:41 -0800 (PST) (envelope-from mitch) Date: Mon, 6 Dec 2004 21:48:40 -0800 From: Mitchell Blank Jr To: Phil Oester Cc: shemminger@osdl.org, linux-net@vger.kernel.org, linux-kernel@vger.kernel.org, netdev@oss.sgi.com Subject: Re: Recent select() handling change breaks Poptop Message-ID: <20041207054840.GD61527@gaz.sfgoth.com> References: <20041207003525.GA22933@linuxace.com> <20041207025218.GB61527@gaz.sfgoth.com> <20041207045302.GA23746@linuxace.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20041207045302.GA23746@linuxace.com> User-Agent: Mutt/1.4.2.1i X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-1.2.2 (gaz.sfgoth.com [127.0.0.1]); Mon, 06 Dec 2004 21:48:41 -0800 (PST) X-archive-position: 12507 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: mitch@sfgoth.com Precedence: bulk X-list: netdev (adding netdev to cc:) Phil Oester wrote: > > 2. a "tcpdump -nvv" of its udp traffic (ideally captured from a seperate > > server, but from the server would probably be OK too) > > PPTP uses TCP 1723 and GRE (proto 47), so there is no udp traffic involved. > I suspect the change was made to all datagram traffic with the assumption > that UDP was the only protocol impacted. Perhaps GRE was not considered? Yeah, it looks like the problem for sure. The patch modifies the structure "inet_dgram_ops" to use udp_poll(), but looking farther down: static struct inet_protosw inetsw_array[] = [...] .type = SOCK_DGRAM, .protocol = IPPROTO_UDP, .prot = &udp_prot, .ops = &inet_dgram_ops, [...] .type = SOCK_RAW, .protocol = IPPROTO_IP, /* wild card */ .prot = &raw_prot, .ops = &inet_dgram_ops, [...] so it looks like udp_poll() will end up getting used for both SOCK_DGRAM and SOCK_RAW inet sockets; obviously Poptop is using the latter and failing as a result. No need for the strace/tcpdump data I guess. The fix is to just make a copy of the inet_dgram_ops called inet_udp_ops and make the udp_poll() change only in that one (and obviously change the SOCK_DGRAM case there to use &inet_udp_ops). I don't have time right this second to spin a patch, but could you try that out and see if it fixes your problem. -Mitch From kaber@trash.net Mon Dec 6 22:35:53 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 22:35:57 -0800 (PST) Received: from kaber.coreworks.de ([62.206.217.67]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB76Zq3K017788 for ; Mon, 6 Dec 2004 22:35:53 -0800 Received: from localhost ([127.0.0.1]) by kaber.coreworks.de with esmtp (Exim 4.34) id 1CbYwE-000226-A0; Tue, 07 Dec 2004 07:35:06 +0100 Message-ID: <41B54F1A.6050905@trash.net> Date: Tue, 07 Dec 2004 07:35:06 +0100 From: Patrick McHardy User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.7.3) Gecko/20041008 Debian/1.7.3-5 X-Accept-Language: en MIME-Version: 1.0 To: Thomas Cataldo CC: linux-kernel@vger.kernel.org, netdev@oss.sgi.com Subject: Re: Hard freeze with 2.6.10-rc3 and QoS, worked fine with 2.6.9 References: <1102380430.6103.6.camel@buffy> In-Reply-To: <1102380430.6103.6.camel@buffy> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 12508 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kaber@trash.net Precedence: bulk X-list: netdev Thomas Cataldo wrote: >Hi, > >Tonight I upgraded to 2.6.10-rc3. Everything was fine until I started >wondershaper to setup my Qos rules : > >wondershaper eth0 255 16 > >And the machine freezed hard. No magic sysrq working, no oops in my >logs. > > Please try to find out which line causes the lockup. Are you using the same config options as with 2.6.9 ? Regards Patrick From akpm@osdl.org Mon Dec 6 22:45:24 2004 Received: with ECARTIS (v1.0.0; list netdev); Mon, 06 Dec 2004 22:45:30 -0800 (PST) Received: from mail.osdl.org (fw.osdl.org [65.172.181.6]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB76jOb3018581 for ; Mon, 6 Dec 2004 22:45:24 -0800 Received: from bix (build.pdx.osdl.net [172.20.1.2]) by mail.osdl.org (8.11.6/8.11.6) with SMTP id iB76ir926758; Mon, 6 Dec 2004 22:44:53 -0800 Date: Mon, 6 Dec 2004 22:44:41 -0800 From: Andrew Morton To: Thomas Cataldo Cc: linux-kernel@vger.kernel.org, netdev@oss.sgi.com, tgraf@suug.ch, "David S. Miller" Subject: Re: Hard freeze with 2.6.10-rc3 and QoS, worked fine with 2.6.9 Message-Id: <20041206224441.628e7885.akpm@osdl.org> In-Reply-To: <1102380430.6103.6.camel@buffy> References: <1102380430.6103.6.camel@buffy> X-Mailer: Sylpheed version 0.9.7 (GTK+ 1.2.10; i386-redhat-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-archive-position: 12509 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: akpm@osdl.org Precedence: bulk X-list: netdev Thomas Cataldo wrote: > > Tonight I upgraded to 2.6.10-rc3. Everything was fine until I started > wondershaper to setup my Qos rules : > > wondershaper eth0 255 16 > > And the machine freezed hard. No magic sysrq working, no oops in my > logs. > > The computer is an x86 smp (dual p3) > > wondershaper was working fine with 2.6.9. Me too, with your .config: Using http://lartc.org/wondershaper/wondershaper-1.1a.tar.gz vmm:/home/akpm/wondershaper-1.1a# ./wshaper eth0 255 16 u32 classifier Perfomance counters on input device check on Actions configured Unable to handle kernel NULL pointer dereference at virtual address 00000000 printing eip: c0290b58 *pde = 00000000 Oops: 0002 [#1] SMP Modules linked in: police sch_ingress cls_u32 sch_sfq sch_cbq usbcore CPU: 1 EIP: 0060:[] Not tainted VLI EFLAGS: 00010206 (2.6.10-rc3) EIP is at _spin_lock_bh+0x10/0x20 eax: cf690000 ebx: ce0d4d80 ecx: 00000008 edx: 00000000 esi: cf691ba0 edi: 00000000 ebp: 00000000 esp: cf691b48 ds: 007b es: 007b ss: 0068 Process tc (pid: 2743, threadinfo=cf690000 task=cf07b520) Stack: c02372ab ce0d4d80 cd4d8800 cebd1c40 ce0d4d80 c0237346 ce0d4d80 00000008 00000000 00000000 00000000 cf691ba0 cf691ba0 c02478d6 ce0d4d80 00000008 00000000 cf691ba0 ce0d4d80 cf6a6070 30960094 cec44380 00000000 00019000 Call Trace: [] gnet_stats_start_copy_compat+0x1b/0x98 [] gnet_stats_start_copy+0x1e/0x24 [] tcf_action_copy_stats+0x26/0xa0 [] tcf_action_dump_old+0x36/0x3c [] u32_dump+0x2c8/0x344 [cls_u32] [] u32_dump+0x2fa/0x344 [cls_u32] [] tcf_fill_node+0x11d/0x170 [] tfilter_notify+0x50/0xa0 [] tc_ctl_tfilter+0x542/0x570 [] rtnetlink_rcv+0x23d/0x360 [] netlink_data_ready+0x1c/0x54 [] netlink_sendskb+0x21/0x40 [] netlink_unicast+0xe3/0xec [] netlink_sendmsg+0x27c/0x28c [] sock_sendmsg+0xd5/0xf8 [] sock_sendmsg+0xd5/0xf8 [] copy_from_user+0x30/0x60 [] copy_from_user+0x30/0x60 [] autoremove_wake_function+0x0/0x40 [] sys_sendmsg+0x18f/0x1f4 [] handle_mm_fault+0x80/0x11c [] do_page_fault+0x1a3/0x554 [] copy_from_user+0x30/0x60 [] sys_socketcall+0x1d8/0x1f4 [] sysenter_past_esp+0x52/0x71 Code: 3a 00 7e f9 fa eb e9 c3 8d 76 00 fa f0 fe 08 79 09 f3 90 80 38 00 7e f9 eb f2 c3 89 c2 b8 00 e0 ff ff 21 e0 81 <0>Kernel panic - not syncing: Fatal exception in interrupt Somehow I don't think this is because "Performance" was misspelled ;) tcf_act_hdr.stats_lock is NULL in tcf_action_copy_stats() From lenar@vision.ee Tue Dec 7 00:44:22 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 00:44:29 -0800 (PST) Received: from mail.city.ee (tristate.vision.ee [194.204.30.144]) by oss.sgi.com (8.13.0/8.13.0) with SMTP id iB78iGAr027552 for ; Tue, 7 Dec 2004 00:44:21 -0800 Received: (qmail 8023 invoked from network); 7 Dec 2004 08:43:54 -0000 Received: from unknown (HELO ?127.0.0.1?) (127.0.0.1) by localhost with SMTP; 7 Dec 2004 08:43:54 -0000 Message-ID: <41B56D85.9080105@vision.ee> Date: Tue, 07 Dec 2004 10:44:53 +0200 From: =?UTF-8?B?TGVuYXIgTMO1aG11cw==?= User-Agent: Mozilla Thunderbird 1.0RC1 (Windows/20041201) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Francois Romieu , netdev@oss.sgi.com Subject: Re: status of via velocity in 2.6.9 References: <41B4F447.2060808@ccs.neu.edu> <41B56518.2070108@vision.ee> <20041207082106.GA24306@electric-eye.fr.zoreil.com> In-Reply-To: <20041207082106.GA24306@electric-eye.fr.zoreil.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-archive-position: 12510 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: lenar@vision.ee Precedence: bulk X-list: netdev Hi, Francois Romieu wrote: >Lenar Lõhmus : >[...] > > >>the machine just locked up after ifconfig up. With 2.6.9, it doesn't >>lock up, but it doesn't work either (data seems to go to black hole >>or sth). But there seem to be some success reports too with this kernel. >> >> > >Can you check if the computer hosts the latest bios from its vendor and > > At the time 2.6.9 was released it had latest bios. >if booting with "acpi=off" makes a difference ? > > If I remember correctly - i tried, but no difference. >The content of /proc/interrupts after a known number of TX packets could >give some hint (use ping or such and correlate ifconfig output with >/proc/interrupts). > > With that I can't help, although the machine is accessible to me, it is now 'in production' with 3Com NIC and so I can't test and play with it anymore. When I get my hands on another similar box and it still exhibits same problems, I'll be back reporting it, and hopefully I have time to test suggested things. Lenar From kdesler@soohrt.org Tue Dec 7 02:21:58 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 02:22:04 -0800 (PST) Received: from quickstop.soohrt.org (quickstop.soohrt.org [81.2.155.147]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7ALvrT030739 for ; Tue, 7 Dec 2004 02:21:58 -0800 Received: (qmail 29859 invoked by uid 1018); 7 Dec 2004 10:21:32 -0000 Date: Tue, 7 Dec 2004 11:21:32 +0100 From: Karsten Desler To: Karsten Desler Cc: jamal , Bernd Eckenfels , "David S. Miller" , netdev@oss.sgi.com, linux-kernel@vger.kernel.org Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets Message-ID: <20041207102132.GA28588@quickstop.soohrt.org> References: <20041206224107.GA8529@soohrt.org> <20041207002012.GB30674@quickstop.soohrt.org> <1102387595.1088.48.camel@jzny.localdomain> <20041207025456.GA525@soohrt.org> <1102389533.1089.51.camel@jzny.localdomain> <20041207032438.GA7767@soohrt.org> <1102390241.1093.59.camel@jzny.localdomain> <20041207040235.GA10501@soohrt.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20041207040235.GA10501@soohrt.org> User-Agent: Mutt/1.5.6+20040722i X-archive-position: 12511 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kdesler@soohrt.org Precedence: bulk X-list: netdev Karsten Desler wrote: > * jamal wrote: > > Beats me. Make sure it boots NAPI. Also if you can turn off ITR; intel > > loves to turn on that silly feature. > > ITR was in fact activated. I think i've disabled it now > (e1000.InterruptThrottleRate=0 in the kernel cmdline). > And as I'm reading the e1000 code, there is no way to enable/disable > NAPI without a recompile. So the fact that ethtool spat out -NAPI in > the version string means that NAPI is actually used. But looking and the int/s number, I'm not so sure anymore. Is there any other way to find out? # ethtool -i eth0|grep ^vers version: 5.5.4-k2-NAPI # ethtool -i eth1|grep ^vers version: 5.5.4-k2-NAPI CPU0 CPU1 169: 5 115554253 IO-APIC-level eth0 177: 78998347 5568 IO-APIC-level eth1 # sar -I 169 5 5 11:20:05 INTR intr/s 11:20:10 169 10401.40 11:20:15 169 10579.80 11:20:20 169 10965.20 11:20:25 169 10768.20 11:20:30 169 10460.60 Average: 169 10635.04 # sar -I 177 5 5 11:18:50 INTR intr/s 11:18:55 177 4769.74 11:19:00 177 4780.80 11:19:05 177 4669.74 11:19:10 177 4724.55 11:19:15 177 4748.50 Average: 177 4738.67 Cheers, Karsten From P@draigBrady.com Tue Dec 7 02:48:32 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 02:48:37 -0800 (PST) Received: from corvil.com (gate.corvil.net [213.94.219.177]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7AmSHL001259 for ; Tue, 7 Dec 2004 02:48:29 -0800 Received: from draigBrady.com (pixelbeat.local.corvil.com [172.18.1.170]) by corvil.com (8.12.9/8.12.5) with ESMTP id iB7AlqwS058748; Tue, 7 Dec 2004 10:47:55 GMT (envelope-from P@draigBrady.com) Message-ID: <41B58A58.8010007@draigBrady.com> Date: Tue, 07 Dec 2004 10:47:52 +0000 From: P@draigBrady.com User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040124 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Karsten Desler CC: "David S. Miller" , netdev@oss.sgi.com, linux-kernel@vger.kernel.org Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets References: <20041206205305.GA11970@soohrt.org> <20041206134849.498bfc93.davem@davemloft.net> <20041206224107.GA8529@soohrt.org> In-Reply-To: <20041206224107.GA8529@soohrt.org> Content-Type: multipart/mixed; boundary="------------000001040401020404000706" X-archive-position: 12512 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: P@draigBrady.com Precedence: bulk X-list: netdev This is a multi-part message in MIME format. --------------000001040401020404000706 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Karsten Desler wrote: > * David S. Miller wrote: > >>It's spending nearly half of it's time in iptables. >>Try to consolidate your rules if possible. This is the >>part of netfilter that really doesn't scale well at all. >> > > Removing the iptables rules helps reducing the load a little, but the > majority of time is still spent somewhere else. Well with NAPI it can be hard to tell CPU usage. You may need to use something like cyclesoak to get a true idea of CPU left. Also have a look at http://www.hipac.org/ as netfilter has silly scalability properties. I also notice that a lot of time is spent allocating and freeing the packet buffers (and possible hidden time due to cache misses due to allocating on one CPU and freeing on another?). How many [RT]xDescriptors do you have configured by the way? Anyway attached is a small patch that I used to make the e1000 "own" the packet buffers, and hence it does not alloc/free per packet at all. Now this has only been tested in one configuration where I was just sniffing the packets, so definitely YMMV. -- Pádraig Brady - http://www.pixelbeat.org -- --------------000001040401020404000706 Content-Type: application/x-texinfo; name="linux-2.4.20-5.2.52-realloc.diff" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="linux-2.4.20-5.2.52-realloc.diff" diff -Naur linux-2.4.20-5.2.52/drivers/net/e1000/e1000_main.c linux-2.4.20-pb/drivers/net/e1000/e1000_main.c --- linux-2.4.20-5.2.52/drivers/net/e1000/e1000_main.c 2004-05-17 22:59:53.000000000 +0000 +++ linux-2.4.20-pb/drivers/net/e1000/e1000_main.c 2004-12-07 10:16:16.000000000 +0000 @@ -2319,9 +2323,9 @@ E1000_DBG("%s: Receive packet consumed multiple buffers\n", netdev->name); - dev_kfree_skb_irq(skb); + //dev_kfree_skb_irq(skb); //PB rx_desc->status = 0; - buffer_info->skb = NULL; + //buffer_info->skb = NULL; //PB if(++i == rx_ring->count) i = 0; @@ -2347,9 +2351,9 @@ length--; } else { - dev_kfree_skb_irq(skb); + //dev_kfree_skb_irq(skb); //PB rx_desc->status = 0; - buffer_info->skb = NULL; + //buffer_info->skb = NULL; //PB if(++i == rx_ring->count) i = 0; @@ -2393,7 +2398,7 @@ netdev->last_rx = jiffies; rx_desc->status = 0; - buffer_info->skb = NULL; + //buffer_info->skb = NULL; //PB: E1000 doesn't reallocate packets if(++i == rx_ring->count) i = 0; @@ -2421,20 +2426,37 @@ struct e1000_rx_desc *rx_desc; struct e1000_buffer *buffer_info; struct sk_buff *skb; - int reserve_len = 2; + int reserve_len = 18; unsigned int i; i = rx_ring->next_to_use; - buffer_info = &rx_ring->buffer_info[i]; - while(!buffer_info->skb) { + while(1) { + buffer_info = &rx_ring->buffer_info[i]; + if (!buffer_info->skb) { + ; /* try to allocate new buf */ + } else { + if (!skb_shared(buffer_info->skb)) { + ; /* try to reallocate unused buf */ + } else { + break; /* Better luck next round */ + } + } + rx_desc = E1000_RX_DESC(*rx_ring, i); - skb = dev_alloc_skb(adapter->rx_buffer_len + reserve_len); + if (!buffer_info->skb) { + /* TODO: optimize this alloc size */ + skb = alloc_skb(adapter->rx_buffer_len + reserve_len, GFP_ATOMIC); + } else { + skb = realloc_skb(buffer_info->skb, adapter->rx_buffer_len + reserve_len, GFP_ATOMIC); + } if(!skb) { - /* Better luck next round */ - break; + break; /* Better luck next round */ + } else { + skb->dev = netdev; + skb_get(skb); /* It's ours. Don't let others deallocate */ } /* Make buffer alignment 2 beyond a 16 byte boundary @@ -2443,8 +2465,6 @@ */ skb_reserve(skb, reserve_len); - skb->dev = netdev; - buffer_info->skb = skb; buffer_info->length = adapter->rx_buffer_len; buffer_info->dma = @@ -2466,7 +2486,6 @@ } if(++i == rx_ring->count) i = 0; - buffer_info = &rx_ring->buffer_info[i]; } rx_ring->next_to_use = i; diff -Naur linux-2.4.20-5.2.52/include/linux/skbuff.h linux-2.4.20-pb/include/linux/skbuff.h --- linux-2.4.20-5.2.52/include/linux/skbuff.h 2002-08-03 00:39:46.000000000 +0000 +++ linux-2.4.20-pb/include/linux/skbuff.h 2004-12-07 10:16:16.000000000 +0000 @@ -230,6 +230,7 @@ extern void __kfree_skb(struct sk_buff *skb); extern struct sk_buff * alloc_skb(unsigned int size, int priority); +extern struct sk_buff * realloc_skb(struct sk_buff *skb, unsigned int size, int priority); extern void kfree_skbmem(struct sk_buff *skb); extern struct sk_buff * skb_clone(struct sk_buff *skb, int priority); extern struct sk_buff * skb_copy(const struct sk_buff *skb, int priority); @@ -240,6 +241,7 @@ int newheadroom, int newtailroom, int priority); +extern struct sk_buff * skb_pad(struct sk_buff *skb, int pad); #define dev_kfree_skb(a) kfree_skb(a) extern void skb_over_panic(struct sk_buff *skb, int len, void *here); extern void skb_under_panic(struct sk_buff *skb, int len, void *here); @@ -1082,6 +1084,26 @@ } /** + * skb_padto - pad an skbuff up to a minimal size + * @skb: buffer to pad + * @len: minimal length + * + * Pads up a buffer to ensure the trailing bytes exist and are + * blanked. If the buffer already contains sufficient data it + * is untouched. Returns the buffer, which may be a replacement + * for the original, or NULL for out of memory - in which case + * the original buffer is still freed. + */ + +static inline struct sk_buff *skb_padto(struct sk_buff *skb, unsigned int len) +{ + unsigned int size = skb->len; + if(likely(size >= len)) + return skb; + return skb_pad(skb, len-size); +} + +/** * skb_linearize - convert paged skb to linear one * @skb: buffer to linarize * @gfp: allocation mode diff -Naur linux-2.4.20-5.2.52/net/core/skbuff.c linux-2.4.20-pb/net/core/skbuff.c --- linux-2.4.20-5.2.52/net/core/skbuff.c 2002-08-03 00:39:46.000000000 +0000 +++ linux-2.4.20-pb/net/core/skbuff.c 2004-12-07 10:16:16.000000000 +0000 @@ -216,7 +216,6 @@ return NULL; } - /* * Slab constructor for a skb head. */ @@ -251,6 +250,59 @@ #endif } +/** + * realloc_skb - reset skb for new packet. + * @size: size to allocate + * @gfp_mask: allocation mask + * + * Allocate a new &sk_buff. The returned buffer has no headroom and a + * tail room of size bytes. The object has a reference count of one. + * The return is the buffer. On a failure the return is %NULL. + * + * Buffers may only be allocated from interrupts using a @gfp_mask of + * %GFP_ATOMIC. + */ + +struct sk_buff *realloc_skb(struct sk_buff* skb, unsigned int size, int gfp_mask) +{ + int truesize=skb->truesize; + u8 *data=skb->head; + + skb_headerinit(skb, (kmem_cache_t *)NULL, 0); + + /* Get the DATA. Size must match skb_add_mtu(). */ + size = SKB_DATA_ALIGN(size); + if ((size+sizeof(struct sk_buff)) > truesize) { + skb_release_data(skb); + data = kmalloc(size + sizeof(struct skb_shared_info), gfp_mask); + if (data == NULL) + goto nodata; + } + + /* XXX: does not include slab overhead */ + skb->truesize = size + sizeof(struct sk_buff); + + /* Load the data pointers. */ + skb->head = data; + skb->data = data; + skb->tail = data; + skb->end = data + size; + + /* Set up other state */ + skb->len = 0; + skb->cloned = 0; + skb->data_len = 0; + + atomic_set(&skb->users, 1); + atomic_set(&(skb_shinfo(skb)->dataref), 1); + skb_shinfo(skb)->nr_frags = 0; + skb_shinfo(skb)->frag_list = NULL; + return skb; + +nodata: + return NULL; +} + static void skb_drop_fraglist(struct sk_buff *skb) { struct sk_buff *list = skb_shinfo(skb)->frag_list; @@ -731,6 +783,36 @@ return n; } +/** + * skb_pad - zero pad the tail of an skb + * @skb: buffer to pad + * @pad: space to pad + * + * Ensure that a buffer is followed by a padding area that is zero + * filled. Used by network drivers which may DMA or transfer data + * beyond the buffer end onto the wire. + * + * May return NULL in out of memory cases. + */ + +struct sk_buff *skb_pad(struct sk_buff *skb, int pad) +{ + struct sk_buff *nskb; + + /* If the skbuff is non linear tailroom is always zero.. */ + if(skb_tailroom(skb) >= pad) + { + memset(skb->data+skb->len, 0, pad); + return skb; + } + + nskb = skb_copy_expand(skb, skb_headroom(skb), skb_tailroom(skb) + pad, GFP_ATOMIC); + kfree_skb(skb); + if(nskb) + memset(nskb->data+nskb->len, 0, pad); + return nskb; +} + /* Trims skb to length len. It can change skb pointers, if "realloc" is 1. * If realloc==0 and trimming is impossible without change of data, * it is BUG(). --------------000001040401020404000706-- From kdesler@soohrt.org Tue Dec 7 03:22:06 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 03:22:10 -0800 (PST) Received: from quickstop.soohrt.org (quickstop.soohrt.org [81.2.155.147]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7BM4ZC002905 for ; Tue, 7 Dec 2004 03:22:05 -0800 Received: (qmail 10058 invoked by uid 1000); 7 Dec 2004 11:21:39 -0000 Date: Tue, 7 Dec 2004 12:21:39 +0100 From: Karsten Desler To: P@draigBrady.com Cc: "David S. Miller" , netdev@oss.sgi.com, linux-kernel@vger.kernel.org Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets Message-ID: <20041207112139.GA3610@soohrt.org> References: <20041206205305.GA11970@soohrt.org> <20041206134849.498bfc93.davem@davemloft.net> <20041206224107.GA8529@soohrt.org> <41B58A58.8010007@draigBrady.com> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: <41B58A58.8010007@draigBrady.com> User-Agent: Mutt/1.5.6+20040722i X-archive-position: 12513 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kdesler@soohrt.org Precedence: bulk X-list: netdev * P@draigBrady.com wrote: > Karsten Desler wrote: > >* David S. Miller wrote: > > > >>It's spending nearly half of it's time in iptables. > >>Try to consolidate your rules if possible. This is the > >>part of netfilter that really doesn't scale well at all. > >> > > > >Removing the iptables rules helps reducing the load a little, but the > >majority of time is still spent somewhere else. > > Well with NAPI it can be hard to tell CPU usage. > You may need to use something like cyclesoak to get > a true idea of CPU left. Thanks, I'll look into it. > Also have a look at http://www.hipac.org/ as netfilter > has silly scalability properties. I did before, but I read on Harald Weltes' weblog that 2.4 gives slightly worse pps results than 2.6, and since the cpu usage is as high as it is, I didn't want to take any more performance hits. I'll try to see what performance impact the netfilter rules have during peak load. > I also notice that a lot of time is spent allocating > and freeing the packet buffers (and possible hidden > time due to cache misses due to allocating on one > CPU and freeing on another?). > How many [RT]xDescriptors do you have configured by the way? 256. I increased them to 1024 shortly after the profiling run, but didn't notice any change in the cpu usage (will try again with cyclesoak). > Anyway attached is a small patch that I used to make the e1000 > "own" the packet buffers, and hence it does not alloc/free > per packet at all. Now this has only been tested in one > configuration where I was just sniffing the packets, so > definitely YMMV. Thanks, I'll give it a spin. Cheers, Karsten From hadi@cyberus.ca Tue Dec 7 04:29:39 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 04:29:45 -0800 (PST) Received: from mx01.cybersurf.com (mx01.cybersurf.com [209.197.145.104]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7CTcXn008190 for ; Tue, 7 Dec 2004 04:29:39 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx01.cybersurf.com with esmtp (Exim 4.30) id 1CbeSu-00023Z-Bw for netdev@oss.sgi.com; Tue, 07 Dec 2004 07:29:12 -0500 Received: from cpe0030ab124d2f-cm014500000962.cpe.net.cable.rogers.com ([24.103.99.32] helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1CbeSq-0005L8-VP; Tue, 07 Dec 2004 07:29:09 -0500 Subject: Re: Hard freeze with 2.6.10-rc3 and QoS, worked fine with 2.6.9 From: jamal Reply-To: hadi@cyberus.ca To: Andrew Morton Cc: Thomas Cataldo , linux-kernel@vger.kernel.org, netdev@oss.sgi.com, tgraf@suug.ch, "David S. Miller" In-Reply-To: <20041206224441.628e7885.akpm@osdl.org> References: <1102380430.6103.6.camel@buffy> <20041206224441.628e7885.akpm@osdl.org> Content-Type: text/plain Organization: jamalopolous Message-Id: <1102422544.1088.98.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 07 Dec 2004 07:29:04 -0500 Content-Transfer-Encoding: 7bit X-archive-position: 12514 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev Can you do a: tc -V This seems to point to probably be a backward compat issue which was overlooked in the stats. cheers, jamal On Tue, 2004-12-07 at 01:44, Andrew Morton wrote: > Thomas Cataldo wrote: > > > > Tonight I upgraded to 2.6.10-rc3. Everything was fine until I started > > wondershaper to setup my Qos rules : > > > > wondershaper eth0 255 16 > > > > And the machine freezed hard. No magic sysrq working, no oops in my > > logs. > > > > The computer is an x86 smp (dual p3) > > > > wondershaper was working fine with 2.6.9. > > Me too, with your .config: > > Using http://lartc.org/wondershaper/wondershaper-1.1a.tar.gz > > vmm:/home/akpm/wondershaper-1.1a# ./wshaper eth0 255 16 > > > u32 classifier > Perfomance counters on > input device check on > Actions configured > Unable to handle kernel NULL pointer dereference at virtual address 00000000 > printing eip: > c0290b58 > *pde = 00000000 > Oops: 0002 [#1] > SMP > Modules linked in: police sch_ingress cls_u32 sch_sfq sch_cbq usbcore > CPU: 1 > EIP: 0060:[] Not tainted VLI > EFLAGS: 00010206 (2.6.10-rc3) > EIP is at _spin_lock_bh+0x10/0x20 > eax: cf690000 ebx: ce0d4d80 ecx: 00000008 edx: 00000000 > esi: cf691ba0 edi: 00000000 ebp: 00000000 esp: cf691b48 > ds: 007b es: 007b ss: 0068 > Process tc (pid: 2743, threadinfo=cf690000 task=cf07b520) > Stack: c02372ab ce0d4d80 cd4d8800 cebd1c40 ce0d4d80 c0237346 ce0d4d80 00000008 > 00000000 00000000 00000000 cf691ba0 cf691ba0 c02478d6 ce0d4d80 00000008 > 00000000 cf691ba0 ce0d4d80 cf6a6070 30960094 cec44380 00000000 00019000 > Call Trace: > [] gnet_stats_start_copy_compat+0x1b/0x98 > [] gnet_stats_start_copy+0x1e/0x24 > [] tcf_action_copy_stats+0x26/0xa0 > [] tcf_action_dump_old+0x36/0x3c > [] u32_dump+0x2c8/0x344 [cls_u32] > [] u32_dump+0x2fa/0x344 [cls_u32] > [] tcf_fill_node+0x11d/0x170 > [] tfilter_notify+0x50/0xa0 > [] tc_ctl_tfilter+0x542/0x570 > [] rtnetlink_rcv+0x23d/0x360 > [] netlink_data_ready+0x1c/0x54 > [] netlink_sendskb+0x21/0x40 > [] netlink_unicast+0xe3/0xec > [] netlink_sendmsg+0x27c/0x28c > [] sock_sendmsg+0xd5/0xf8 > [] sock_sendmsg+0xd5/0xf8 > [] copy_from_user+0x30/0x60 > [] copy_from_user+0x30/0x60 > [] autoremove_wake_function+0x0/0x40 > [] sys_sendmsg+0x18f/0x1f4 > [] handle_mm_fault+0x80/0x11c > [] do_page_fault+0x1a3/0x554 > [] copy_from_user+0x30/0x60 > [] sys_socketcall+0x1d8/0x1f4 > [] sysenter_past_esp+0x52/0x71 > Code: 3a 00 7e f9 fa eb e9 c3 8d 76 00 fa f0 fe 08 79 09 f3 90 80 38 00 7e f9 eb f2 c3 89 c2 b8 00 e0 ff ff 21 e0 81 > <0>Kernel panic - not syncing: Fatal exception in interrupt > Somehow I don't think this is because "Performance" was misspelled ;) > > tcf_act_hdr.stats_lock is NULL in tcf_action_copy_stats() > > > From hadi@cyberus.ca Tue Dec 7 04:34:34 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 04:34:40 -0800 (PST) Received: from mx03.cybersurf.com (mx03.cybersurf.com [209.197.145.106]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7CYY37008666 for ; Tue, 7 Dec 2004 04:34:34 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx03.cybersurf.com with esmtp (Exim 4.30) id 1CbeXh-0001Vs-Pj for netdev@oss.sgi.com; Tue, 07 Dec 2004 07:34:09 -0500 Received: from cpe0030ab124d2f-cm014500000962.cpe.net.cable.rogers.com ([24.103.99.32] helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1CbeXf-0005sK-EE; Tue, 07 Dec 2004 07:34:07 -0500 Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets From: jamal Reply-To: hadi@cyberus.ca To: Karsten Desler , Robert Olsson Cc: Bernd Eckenfels , "David S. Miller" , netdev@oss.sgi.com, linux-kernel@vger.kernel.org In-Reply-To: <20041207102132.GA28588@quickstop.soohrt.org> References: <20041206224107.GA8529@soohrt.org> <20041207002012.GB30674@quickstop.soohrt.org> <1102387595.1088.48.camel@jzny.localdomain> <20041207025456.GA525@soohrt.org> <1102389533.1089.51.camel@jzny.localdomain> <20041207032438.GA7767@soohrt.org> <1102390241.1093.59.camel@jzny.localdomain> <20041207040235.GA10501@soohrt.org> <20041207102132.GA28588@quickstop.soohrt.org> Content-Type: text/plain Organization: jamalopolous Message-Id: <1102422845.1089.105.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 07 Dec 2004 07:34:05 -0500 Content-Transfer-Encoding: 7bit X-archive-position: 12515 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev On Tue, 2004-12-07 at 05:21, Karsten Desler wrote: > But looking and the int/s number, I'm not so sure anymore. Is there any > other way to find out? > > # sar -I 169 5 5 > 11:20:05 INTR intr/s > 11:20:10 169 10401.40 That doesnt seem to be too high. You have a dual opteron 244. You are supposed to be kicking ass with that machine - not 200Kpps+ you are getting with all that CPU overload. Something is wrong with your setup. Unfortunately i cant afford such a machine so i cant see it right off the bat. I know Robert has at least one similar machine; maybe he could help. Robert? cheers, jamal From Robert.Olsson@data.slu.se Tue Dec 7 04:39:29 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 04:39:33 -0800 (PST) Received: from mail1.slu.se (mail1.slu.se [130.238.96.11]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7CdLkg009072 for ; Tue, 7 Dec 2004 04:39:28 -0800 Received: from robur.slu.se (robur.slu.se [130.238.98.12]) by mail1.slu.se (8.12.10/8.12.10) with ESMTP id iB7CcuKO011805; Tue, 7 Dec 2004 13:38:56 +0100 Received: by robur.slu.se (Postfix, from userid 1000) id EFA5DEC001; Tue, 7 Dec 2004 13:38:56 +0100 (CET) From: Robert Olsson MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Message-ID: <16821.42080.932184.167780@robur.slu.se> Date: Tue, 7 Dec 2004 13:38:56 +0100 To: Karsten Desler Cc: P@draigBrady.com, "David S. Miller" , netdev@oss.sgi.com, linux-kernel@vger.kernel.org Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets In-Reply-To: <20041207112139.GA3610@soohrt.org> References: <20041206205305.GA11970@soohrt.org> <20041206134849.498bfc93.davem@davemloft.net> <20041206224107.GA8529@soohrt.org> <41B58A58.8010007@draigBrady.com> <20041207112139.GA3610@soohrt.org> X-Mailer: VM 7.18 under Emacs 21.3.1 Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by oss.sgi.com id iB7CdLkg009072 X-archive-position: 12516 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: Robert.Olsson@data.slu.se Precedence: bulk X-list: netdev Hello! Well my experience is that it very hard not to say almost impossible to extrapolate idle cpu into any network system capacity. I guess this is what you are trying to do? Rather load and overload the system with traffic having the characteristics you expect as a bonus you will get some kind proof of robustness and responsiveness a max load. There are tools for this type of tests. Pádraig! Very funny... I started hacking 2 hours ago on idea I had for long time, This to do a light version of skb recycling based skb->users (the pktgen trick) with very minimal kernel change.. > Anyway attached is a small patch that I used to make the e1000 > "own" the packet buffers, and hence it does not alloc/free > per packet at all. Now this has only been tested in one > configuration where I was just sniffing the packets, so > definitely YMMV. --ro From kdesler@soohrt.org Tue Dec 7 04:50:28 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 04:50:32 -0800 (PST) Received: from quickstop.soohrt.org (quickstop.soohrt.org [81.2.155.147]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7CoRD0009992 for ; Tue, 7 Dec 2004 04:50:28 -0800 Received: (qmail 29304 invoked by uid 1000); 7 Dec 2004 12:50:01 -0000 Date: Tue, 7 Dec 2004 13:50:01 +0100 From: Karsten Desler To: Robert Olsson Cc: P@draigBrady.com, "David S. Miller" , netdev@oss.sgi.com, linux-kernel@vger.kernel.org Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets Message-ID: <20041207125001.GA26644@soohrt.org> References: <20041206205305.GA11970@soohrt.org> <20041206134849.498bfc93.davem@davemloft.net> <20041206224107.GA8529@soohrt.org> <41B58A58.8010007@draigBrady.com> <20041207112139.GA3610@soohrt.org> <16821.42080.932184.167780@robur.slu.se> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: <16821.42080.932184.167780@robur.slu.se> User-Agent: Mutt/1.5.6+20040722i X-archive-position: 12518 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kdesler@soohrt.org Precedence: bulk X-list: netdev * Robert Olsson wrote: > Well my experience is that it very hard not to say almost impossible to > extrapolate idle cpu into any network system capacity. I guess this is > what you are trying to do? Kinda, yes. I'm trying to evaluate if the behaviour I'm seeing is expected, which would heavily influence my choice of hardware/software for future projects (and of course to optimize the current setup). Currently I'm having problems capturing packets with tcpdump (lots of "packets dropped by kernel") which indicates to me that there's genuinely not much (enough) idle time sitting around. > Rather load and overload the system with traffic having the characteristics > you expect as a bonus you will get some kind proof of robustness and > responsiveness a max load. There are tools for this type of tests. Will do, that could take a couple of days though. Cheers, Karsten From tgraf@suug.ch Tue Dec 7 04:49:25 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 04:49:30 -0800 (PST) Received: from b.mx.projectdream.org (eth0-0.arisu.projectdream.org [194.158.4.191]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7CnOxR009775 for ; Tue, 7 Dec 2004 04:49:25 -0800 Received: from postel.suug.ch (unknown [195.134.158.23]) (using TLSv1 with cipher EDH-RSA-DES-CBC3-SHA (168/168 bits)) (No client certificate requested) by b.mx.projectdream.org (Postfix) with ESMTP id DF242F; Tue, 7 Dec 2004 13:48:39 +0100 (CET) Received: by postel.suug.ch (Postfix, from userid 10001) id BA4AE1C0EA; Tue, 7 Dec 2004 13:49:22 +0100 (CET) Date: Tue, 7 Dec 2004 13:49:22 +0100 From: Thomas Graf To: jamal Cc: Michal Ludvig , Andrew Morton , Stephen Hemminger , netdev@oss.sgi.com, Jan Kara Subject: Re: [PATCH] rtnetlink & address family problem Message-ID: <20041207124922.GA1371@postel.suug.ch> References: <41B0A5B4.6060108@suse.cz> <20041206140214.GA749@postel.suug.ch> <1102386461.1093.26.camel@jzny.localdomain> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1102386461.1093.26.camel@jzny.localdomain> X-archive-position: 12517 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: tgraf@suug.ch Precedence: bulk X-list: netdev * jamal <1102386461.1093.26.camel@jzny.localdomain> 2004-12-06 21:27 > On Mon, 2004-12-06 at 09:02, Thomas Graf wrote: > > > Your patch would fix this issue but might break various things. The > > actual problem is that iproute2 doesn't check the family in its filter. > > It blindly assumes that the kernel only returns addresses of the kind it > > has requested. I can understand if you think the current behaviour > > is wrong but we shouldn't change it in the middle of a stable tree. > > Why would it be wrong? The PF_UNSPEC is there for a purpose. I don't think it is wrong myself but I understand if someone does. If one sends a GETADDR request for PF_INET6 one might expect to either receive all ipv6 addresses or none and to only receive all addresess of any type if PF_UNSPEC was specified. > If user space decides it wants to flush ipv4 addresses blindly that user > spaces fault. The patch you attached seems legit. did you verify it? Not yet, it probably has to be applied to iproute.c as well. I'll have a look at it and do some testing. From hadi@cyberus.ca Tue Dec 7 05:03:22 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 05:03:26 -0800 (PST) Received: from mx01.cybersurf.com (mx01.cybersurf.com [209.197.145.104]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7D3MX1010959 for ; Tue, 7 Dec 2004 05:03:22 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx01.cybersurf.com with esmtp (Exim 4.30) id 1CbezZ-0003AQ-FI for netdev@oss.sgi.com; Tue, 07 Dec 2004 08:02:57 -0500 Received: from cpe0030ab124d2f-cm014500000962.cpe.net.cable.rogers.com ([24.103.99.32] helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1CbezW-0000Uo-Pd; Tue, 07 Dec 2004 08:02:55 -0500 Subject: Re: [PATCH] rtnetlink & address family problem From: jamal Reply-To: hadi@cyberus.ca To: Thomas Graf Cc: Michal Ludvig , Andrew Morton , Stephen Hemminger , netdev@oss.sgi.com, Jan Kara In-Reply-To: <20041207124922.GA1371@postel.suug.ch> References: <41B0A5B4.6060108@suse.cz> <20041206140214.GA749@postel.suug.ch> <1102386461.1093.26.camel@jzny.localdomain> <20041207124922.GA1371@postel.suug.ch> Content-Type: text/plain Organization: jamalopolous Message-Id: <1102424568.1089.120.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 07 Dec 2004 08:02:48 -0500 Content-Transfer-Encoding: 7bit X-archive-position: 12519 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev On Tue, 2004-12-07 at 07:49, Thomas Graf wrote: > * jamal <1102386461.1093.26.camel@jzny.localdomain> 2004-12-06 21:27 > > On Mon, 2004-12-06 at 09:02, Thomas Graf wrote: > > > > > Your patch would fix this issue but might break various things. The > > > actual problem is that iproute2 doesn't check the family in its filter. > > > It blindly assumes that the kernel only returns addresses of the kind it > > > has requested. I can understand if you think the current behaviour > > > is wrong but we shouldn't change it in the middle of a stable tree. > > > > Why would it be wrong? The PF_UNSPEC is there for a purpose. > > I don't think it is wrong myself but I understand if someone does. > If > one sends a GETADDR request for PF_INET6 one might expect to either > receive all ipv6 addresses or none and to only receive all addresess > of any type if PF_UNSPEC was specified. > Thats debatable. Its user space that issues the flushing after a response from the kernel. It happens to be flushing IPV4 addresses. Thats why your filter in ip is the answer. BTW, did the gnet_stats patches to iproute2 ever get merged? If you have cycles, can you please look at that hang being reported using older tc with 2.6.10-rc3? cheers, jamal From hadi@cyberus.ca Tue Dec 7 05:05:04 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 05:05:08 -0800 (PST) Received: from mx01.cybersurf.com (mx01.cybersurf.com [209.197.145.104]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7D540i011306 for ; Tue, 7 Dec 2004 05:05:04 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx01.cybersurf.com with esmtp (Exim 4.30) id 1Cbf1D-000441-IO for netdev@oss.sgi.com; Tue, 07 Dec 2004 08:04:39 -0500 Received: from cpe0030ab124d2f-cm014500000962.cpe.net.cable.rogers.com ([24.103.99.32] helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1Cbf1B-0000hw-1X; Tue, 07 Dec 2004 08:04:37 -0500 Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets From: jamal Reply-To: hadi@cyberus.ca To: Karsten Desler Cc: Robert Olsson , P@draigBrady.com, "David S. Miller" , netdev@oss.sgi.com, linux-kernel@vger.kernel.org In-Reply-To: <20041207125001.GA26644@soohrt.org> References: <20041206205305.GA11970@soohrt.org> <20041206134849.498bfc93.davem@davemloft.net> <20041206224107.GA8529@soohrt.org> <41B58A58.8010007@draigBrady.com> <20041207112139.GA3610@soohrt.org> <16821.42080.932184.167780@robur.slu.se> <20041207125001.GA26644@soohrt.org> Content-Type: text/plain Organization: jamalopolous Message-Id: <1102424673.1093.124.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 07 Dec 2004 08:04:33 -0500 Content-Transfer-Encoding: 7bit X-archive-position: 12520 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev On Tue, 2004-12-07 at 07:50, Karsten Desler wrote: > Currently I'm having problems capturing packets with tcpdump (lots of > "packets dropped by kernel") which indicates to me that there's > genuinely not much (enough) idle time sitting around. > Ah, more hints. So you are not trying to forward - rather just packet capturing? Are you using a tcpdump patched with mmaped packet socket? The 230-240Kpps you are reporting as a capture dont seem as unreasonable as i thought then. Neither would the CPU use. cheers, jamal From kdesler@soohrt.org Tue Dec 7 05:11:52 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 05:11:56 -0800 (PST) Received: from quickstop.soohrt.org (quickstop.soohrt.org [81.2.155.147]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7DBpvo011870 for ; Tue, 7 Dec 2004 05:11:52 -0800 Received: (qmail 1430 invoked by uid 1000); 7 Dec 2004 13:11:25 -0000 Date: Tue, 7 Dec 2004 14:11:25 +0100 From: Karsten Desler To: jamal Cc: Robert Olsson , P@draigBrady.com, "David S. Miller" , netdev@oss.sgi.com, linux-kernel@vger.kernel.org Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets Message-ID: <20041207131125.GB26644@soohrt.org> References: <20041206205305.GA11970@soohrt.org> <20041206134849.498bfc93.davem@davemloft.net> <20041206224107.GA8529@soohrt.org> <41B58A58.8010007@draigBrady.com> <20041207112139.GA3610@soohrt.org> <16821.42080.932184.167780@robur.slu.se> <20041207125001.GA26644@soohrt.org> <1102424673.1093.124.camel@jzny.localdomain> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: <1102424673.1093.124.camel@jzny.localdomain> User-Agent: Mutt/1.5.6+20040722i X-archive-position: 12521 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kdesler@soohrt.org Precedence: bulk X-list: netdev * jamal wrote: > On Tue, 2004-12-07 at 07:50, Karsten Desler wrote: > > > Currently I'm having problems capturing packets with tcpdump (lots of > > "packets dropped by kernel") which indicates to me that there's > > genuinely not much (enough) idle time sitting around. > > > > Ah, more hints. So you are not trying to forward - rather just packet > capturing? forward/routing is the goal. I was just trying to capture a tcpdump to analyze the traffic to generate something that could emulate the trafficpattern for further testing in a non-production environment. Cheers, Karsten From kdesler@soohrt.org Tue Dec 7 05:15:12 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 05:15:16 -0800 (PST) Received: from quickstop.soohrt.org (quickstop.soohrt.org [81.2.155.147]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7DFAP7012281 for ; Tue, 7 Dec 2004 05:15:11 -0800 Received: (qmail 2228 invoked by uid 1000); 7 Dec 2004 13:14:45 -0000 Date: Tue, 7 Dec 2004 14:14:45 +0100 From: Karsten Desler To: jamal Cc: Robert Olsson , Bernd Eckenfels , "David S. Miller" , netdev@oss.sgi.com, linux-kernel@vger.kernel.org Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets Message-ID: <20041207131445.GA1622@soohrt.org> References: <20041207002012.GB30674@quickstop.soohrt.org> <1102387595.1088.48.camel@jzny.localdomain> <20041207025456.GA525@soohrt.org> <1102389533.1089.51.camel@jzny.localdomain> <20041207032438.GA7767@soohrt.org> <1102390241.1093.59.camel@jzny.localdomain> <20041207040235.GA10501@soohrt.org> <20041207102132.GA28588@quickstop.soohrt.org> <1102422845.1089.105.camel@jzny.localdomain> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: <1102422845.1089.105.camel@jzny.localdomain> User-Agent: Mutt/1.5.6+20040722i X-archive-position: 12522 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kdesler@soohrt.org Precedence: bulk X-list: netdev * jamal wrote: > > # sar -I 169 5 5 > > 11:20:05 INTR intr/s > > 11:20:10 169 10401.40 > > That doesnt seem to be too high. To recap: 169 is a fibre e1000, eth1 is one of two ports on a dualport e1000 copper nic. eth1 is still running at about 4k int/s. 14:12:18 INTR intr/s 14:12:23 169 34012.80 14:12:28 169 33977.60 14:12:33 169 34218.16 14:12:38 169 34060.60 14:12:43 169 34252.60 Average: 169 34104.40 Cheers, Karsten From tgraf@suug.ch Tue Dec 7 05:17:10 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 05:17:14 -0800 (PST) Received: from b.mx.projectdream.org (eth0-0.arisu.projectdream.org [194.158.4.191]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7DH9Sj012639 for ; Tue, 7 Dec 2004 05:17:09 -0800 Received: from postel.suug.ch (unknown [195.134.158.23]) (using TLSv1 with cipher EDH-RSA-DES-CBC3-SHA (168/168 bits)) (No client certificate requested) by b.mx.projectdream.org (Postfix) with ESMTP id D0243F; Tue, 7 Dec 2004 14:16:24 +0100 (CET) Received: by postel.suug.ch (Postfix, from userid 10001) id BF4C01C0EA; Tue, 7 Dec 2004 14:17:06 +0100 (CET) Date: Tue, 7 Dec 2004 14:17:06 +0100 From: Thomas Graf To: jamal Cc: Michal Ludvig , Andrew Morton , Stephen Hemminger , netdev@oss.sgi.com, Jan Kara Subject: Re: [PATCH] rtnetlink & address family problem Message-ID: <20041207131706.GB1371@postel.suug.ch> References: <41B0A5B4.6060108@suse.cz> <20041206140214.GA749@postel.suug.ch> <1102386461.1093.26.camel@jzny.localdomain> <20041207124922.GA1371@postel.suug.ch> <1102424568.1089.120.camel@jzny.localdomain> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1102424568.1089.120.camel@jzny.localdomain> X-archive-position: 12523 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: tgraf@suug.ch Precedence: bulk X-list: netdev > > I don't think it is wrong myself but I understand if someone does. If > > one sends a GETADDR request for PF_INET6 one might expect to either > > receive all ipv6 addresses or none and to only receive all addresess > > of any type if PF_UNSPEC was specified. > > > > Thats debatable. > Its user space that issues the flushing after a response from the > kernel. It happens to be flushing IPV4 addresses. > Thats why your filter in ip is the answer. Agreed. > BTW, did the gnet_stats patches to iproute2 ever get merged? Not sure, I will check that. > If you have cycles, can you please look at that hang being reported > using older tc with 2.6.10-rc3? It's not really related to the gnet_stats code. stats_lock isn't set in the action code when using an older iproute2. I haven't tested this case because it was marked as broken anyway. I compiled an older version of iproute2 and will look into it today. From hadi@cyberus.ca Tue Dec 7 05:20:50 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 05:20:54 -0800 (PST) Received: from mx02.cybersurf.com (mx02.cybersurf.com [209.197.145.105]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7DKobc013048 for ; Tue, 7 Dec 2004 05:20:50 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx02.cybersurf.com with esmtp (Exim 4.30) id 1CbfGU-0005w1-1D for netdev@oss.sgi.com; Tue, 07 Dec 2004 08:20:26 -0500 Received: from cpe0030ab124d2f-cm014500000962.cpe.net.cable.rogers.com ([24.103.99.32] helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1CbfGP-0002LV-Vn; Tue, 07 Dec 2004 08:20:22 -0500 Subject: Re: [PATCH] rtnetlink & address family problem From: jamal Reply-To: hadi@cyberus.ca To: Thomas Graf Cc: Michal Ludvig , Andrew Morton , Stephen Hemminger , netdev@oss.sgi.com, Jan Kara In-Reply-To: <20041207131706.GB1371@postel.suug.ch> References: <41B0A5B4.6060108@suse.cz> <20041206140214.GA749@postel.suug.ch> <1102386461.1093.26.camel@jzny.localdomain> <20041207124922.GA1371@postel.suug.ch> <1102424568.1089.120.camel@jzny.localdomain> <20041207131706.GB1371@postel.suug.ch> Content-Type: text/plain Organization: jamalopolous Message-Id: <1102425618.1089.133.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 07 Dec 2004 08:20:18 -0500 Content-Transfer-Encoding: 7bit X-archive-position: 12524 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev On Tue, 2004-12-07 at 08:17, Thomas Graf wrote: > It's not really related to the gnet_stats code. stats_lock isn't set > in the action code when using an older iproute2. I haven't tested this > case because it was marked as broken anyway. Can you ping my memory on this? Is this tc with initial support for actions or something much older than that. cheers, jamal From P@draigBrady.com Tue Dec 7 05:39:50 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 05:39:56 -0800 (PST) Received: from corvil.com (gate.corvil.net [213.94.219.177]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7DdnMI013945 for ; Tue, 7 Dec 2004 05:39:50 -0800 Received: from draigBrady.com (pixelbeat.local.corvil.com [172.18.1.170]) by corvil.com (8.12.9/8.12.5) with ESMTP id iB7DdEwS066571; Tue, 7 Dec 2004 13:39:15 GMT (envelope-from P@draigBrady.com) Message-ID: <41B5B282.9040909@draigBrady.com> Date: Tue, 07 Dec 2004 13:39:14 +0000 From: P@draigBrady.com User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040124 X-Accept-Language: en-us, en MIME-Version: 1.0 To: hadi@cyberus.ca CC: Karsten Desler , Robert Olsson , "David S. Miller" , netdev@oss.sgi.com, linux-kernel@vger.kernel.org Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets References: <20041206205305.GA11970@soohrt.org> <20041206134849.498bfc93.davem@davemloft.net> <20041206224107.GA8529@soohrt.org> <41B58A58.8010007@draigBrady.com> <20041207112139.GA3610@soohrt.org> <16821.42080.932184.167780@robur.slu.se> <20041207125001.GA26644@soohrt.org> <1102424673.1093.124.camel@jzny.localdomain> In-Reply-To: <1102424673.1093.124.camel@jzny.localdomain> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-archive-position: 12525 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: P@draigBrady.com Precedence: bulk X-list: netdev jamal wrote: > On Tue, 2004-12-07 at 07:50, Karsten Desler wrote: > > >>Currently I'm having problems capturing packets with tcpdump (lots of >>"packets dropped by kernel") which indicates to me that there's >>genuinely not much (enough) idle time sitting around. >> > > Ah, more hints. So you are not trying to forward - rather just packet > capturing? > Are you using a tcpdump patched with mmaped packet socket? > > The 230-240Kpps you are reporting as a capture dont seem as unreasonable > as i thought then. Neither would the CPU use. Yes this is vital Karsten, otherwise tcpdump will do 2 syscalls per packet, which is the bottleneck in my experience. You may want to try a simpler capture program that uses the kernel PACKET_MMAP feature directly: http://www.scaramanga.co.uk/code-fu/lincap.c -- Pádraig Brady - http://www.pixelbeat.org -- From tgraf@suug.ch Tue Dec 7 06:10:38 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 06:10:42 -0800 (PST) Received: from b.mx.projectdream.org (eth0-0.arisu.projectdream.org [194.158.4.191]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7EAbCY015053 for ; Tue, 7 Dec 2004 06:10:37 -0800 Received: from postel.suug.ch (unknown [195.134.158.23]) (using TLSv1 with cipher EDH-RSA-DES-CBC3-SHA (168/168 bits)) (No client certificate requested) by b.mx.projectdream.org (Postfix) with ESMTP id 5D7B0F; Tue, 7 Dec 2004 15:09:50 +0100 (CET) Received: by postel.suug.ch (Postfix, from userid 10001) id 668491C0EA; Tue, 7 Dec 2004 15:10:33 +0100 (CET) Date: Tue, 7 Dec 2004 15:10:33 +0100 From: Thomas Graf To: jamal Cc: Michal Ludvig , Andrew Morton , Stephen Hemminger , netdev@oss.sgi.com, Jan Kara Subject: Re: [PATCH] rtnetlink & address family problem Message-ID: <20041207141033.GD1371@postel.suug.ch> References: <41B0A5B4.6060108@suse.cz> <20041206140214.GA749@postel.suug.ch> <1102386461.1093.26.camel@jzny.localdomain> <20041207124922.GA1371@postel.suug.ch> <1102424568.1089.120.camel@jzny.localdomain> <20041207131706.GB1371@postel.suug.ch> <1102425618.1089.133.camel@jzny.localdomain> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1102425618.1089.133.camel@jzny.localdomain> X-archive-position: 12526 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: tgraf@suug.ch Precedence: bulk X-list: netdev * jamal <1102425618.1089.133.camel@jzny.localdomain> 2004-12-07 08:20 > On Tue, 2004-12-07 at 08:17, Thomas Graf wrote: > > > It's not really related to the gnet_stats code. stats_lock isn't set > > in the action code when using an older iproute2. I haven't tested this > > case because it was marked as broken anyway. > > Can you ping my memory on this? Is this tc with initial support > for actions or something much older than that. I'm not sure, I'm testing with a version having no action support at all. It should be fairly easy to find the bug once I have the time to really look into it. I'm still getting interrupted all the time at the moment. All actions created via tcf_hash_create, tcf_police_locate, and tcf_act_police_locate should be fine. There must be some bogus path related to older tc versions. From mitch@sfgoth.com Tue Dec 7 07:05:31 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 07:05:35 -0800 (PST) Received: from gaz.sfgoth.com (gaz.sfgoth.com [69.36.241.230]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7F5Ssj016842 for ; Tue, 7 Dec 2004 07:05:30 -0800 Received: from gaz.sfgoth.com (localhost.sfgoth.com [127.0.0.1]) by gaz.sfgoth.com (8.12.10/8.12.10) with ESMTP id iB7F8Yi0076313; Tue, 7 Dec 2004 07:08:35 -0800 (PST) (envelope-from mitch@gaz.sfgoth.com) Received: (from mitch@localhost) by gaz.sfgoth.com (8.12.10/8.12.6/Submit) id iB7F8Y0f076312; Tue, 7 Dec 2004 07:08:34 -0800 (PST) (envelope-from mitch) Date: Tue, 7 Dec 2004 07:08:34 -0800 From: Mitchell Blank Jr To: Phil Oester , "David S. Miller" , shemminger@osdl.org Cc: linux-net@vger.kernel.org, linux-kernel@vger.kernel.org, netdev@oss.sgi.com Subject: [PATCH] fix select() for SOCK_RAW sockets Message-ID: <20041207150834.GA75700@gaz.sfgoth.com> References: <20041207003525.GA22933@linuxace.com> <20041207025218.GB61527@gaz.sfgoth.com> <20041207045302.GA23746@linuxace.com> <20041207054840.GD61527@gaz.sfgoth.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20041207054840.GD61527@gaz.sfgoth.com> User-Agent: Mutt/1.4.2.1i X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-1.2.2 (gaz.sfgoth.com [127.0.0.1]); Tue, 07 Dec 2004 07:08:35 -0800 (PST) X-archive-position: 12527 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: mitch@sfgoth.com Precedence: bulk X-list: netdev Phil: Here's a real patch for you to test. I actually left inet_dgram_ops alone since it's an exported symbol (two of the users just want the .do_ioctl value which is the same between SOCK_DGRAM and SOCK_RAW; the other is ipv6 where it's clearly dealing with a UDP socket -- therefore I think its safest to leave inet_dgram_ops to have the UDP behavior) Davem: I only tested that this doesn't break UDP; if it works for Phil and Stephen can verify that it doesn't break his bad-checksum UDP tests then please push it for 2.6.10. Patch is versus 2.6.10-rc3. Signed-off-by: Mitchell Blank Jr -Mitch --- linux-2.6.10-rc3-VIRGIN/net/ipv4/af_inet.c 2004-12-07 06:37:52.480082706 -0800 +++ linux-2.6.10-rc3/net/ipv4/af_inet.c 2004-12-07 06:57:47.799013216 -0800 @@ -821,6 +821,31 @@ .sendpage = inet_sendpage, }; +/* + * For SOCK_RAW sockets; should be the same as inet_dgram_ops but without + * udp_poll + */ +static struct proto_ops inet_sockraw_ops = { + .family = PF_INET, + .owner = THIS_MODULE, + .release = inet_release, + .bind = inet_bind, + .connect = inet_dgram_connect, + .socketpair = sock_no_socketpair, + .accept = sock_no_accept, + .getname = inet_getname, + .poll = datagram_poll, + .ioctl = inet_ioctl, + .listen = sock_no_listen, + .shutdown = inet_shutdown, + .setsockopt = sock_common_setsockopt, + .getsockopt = sock_common_getsockopt, + .sendmsg = inet_sendmsg, + .recvmsg = sock_common_recvmsg, + .mmap = sock_no_mmap, + .sendpage = inet_sendpage, +}; + static struct net_proto_family inet_family_ops = { .family = PF_INET, .create = inet_create, @@ -861,7 +886,7 @@ .type = SOCK_RAW, .protocol = IPPROTO_IP, /* wild card */ .prot = &raw_prot, - .ops = &inet_dgram_ops, + .ops = &inet_sockraw_ops, .capability = CAP_NET_RAW, .no_check = UDP_CSUM_DEFAULT, .flags = INET_PROTOSW_REUSE, From tgraf@suug.ch Tue Dec 7 08:55:08 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 08:55:19 -0800 (PST) Received: from b.mx.projectdream.org (eth0-0.arisu.projectdream.org [194.158.4.191]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7Gt4UC024966 for ; Tue, 7 Dec 2004 08:55:07 -0800 Received: from postel.suug.ch (unknown [195.134.158.23]) (using TLSv1 with cipher EDH-RSA-DES-CBC3-SHA (168/168 bits)) (No client certificate requested) by b.mx.projectdream.org (Postfix) with ESMTP id 47AE8F; Tue, 7 Dec 2004 17:54:20 +0100 (CET) Received: by postel.suug.ch (Postfix, from userid 10001) id AEFB31C0EA; Tue, 7 Dec 2004 17:55:01 +0100 (CET) Date: Tue, 7 Dec 2004 17:55:01 +0100 From: Thomas Graf To: jamal Cc: Michal Ludvig , Andrew Morton , Stephen Hemminger , netdev@oss.sgi.com, Jan Kara Subject: Re: [PATCH] rtnetlink & address family problem Message-ID: <20041207165501.GE1371@postel.suug.ch> References: <41B0A5B4.6060108@suse.cz> <20041206140214.GA749@postel.suug.ch> <1102386461.1093.26.camel@jzny.localdomain> <20041207124922.GA1371@postel.suug.ch> <1102424568.1089.120.camel@jzny.localdomain> <20041207131706.GB1371@postel.suug.ch> <1102425618.1089.133.camel@jzny.localdomain> <20041207141033.GD1371@postel.suug.ch> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20041207141033.GD1371@postel.suug.ch> X-archive-position: 12528 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: tgraf@suug.ch Precedence: bulk X-list: netdev * Thomas Graf <20041207141033.GD1371@postel.suug.ch> 2004-12-07 15:10 > * jamal <1102425618.1089.133.camel@jzny.localdomain> 2004-12-07 08:20 > > On Tue, 2004-12-07 at 08:17, Thomas Graf wrote: > > > > > It's not really related to the gnet_stats code. stats_lock isn't set > > > in the action code when using an older iproute2. I haven't tested this > > > case because it was marked as broken anyway. > > > > Can you ping my memory on this? Is this tc with initial support > > for actions or something much older than that. > > I'm not sure, I'm testing with a version having no action support at > all. It should be fairly easy to find the bug once I have the time to > really look into it. I'm still getting interrupted all the time at > the moment. One major problem is that the tc_dump_action path doesn't take care of TCA_OLD_COMPAT resulting in calling tcf_action_copy_stats for policers which is a bad thing since their a->priv is set to tcf_police instead of the generic header and thus causes random behaviour. One solution would be to make tcf_police compatible to tca_gen. Thoughts? --- linux-2.6.10-rc2-bk13.orig/include/net/act_api.h 2004-11-30 14:01:11.000000000 +0100 +++ linux-2.6.10-rc2-bk13/include/net/act_api.h 2004-12-07 17:49:50.000000000 +0100 @@ -8,15 +8,42 @@ #include #include +#ifdef CONFIG_NET_CLS_ACT + +#define ACT_P_CREATED 1 +#define ACT_P_DELETED 1 +#define tca_gen(name) \ +struct tcf_##name *next; \ + u32 index; \ + int refcnt; \ + int bindcnt; \ + u32 capab; \ + int action; \ + struct tcf_t tm; \ + struct gnet_stats_basic bstats; \ + struct gnet_stats_queue qstats; \ + struct gnet_stats_rate_est rate_est; \ + spinlock_t *stats_lock; \ + spinlock_t lock + +#endif + struct tcf_police { +#ifdef CONFIG_NET_CLS_ACT + tca_gen(police); +#else struct tcf_police *next; int refcnt; -#ifdef CONFIG_NET_CLS_ACT - int bindcnt; -#endif u32 index; int action; + spinlock_t lock; + struct gnet_stats_basic bstats; + struct gnet_stats_queue qstats; + struct gnet_stats_rate_est rate_est; + spinlock_t *stats_lock; +#endif + int result; u32 ewma_rate; u32 burst; @@ -24,34 +51,12 @@ u32 toks; u32 ptoks; psched_time_t t_c; - spinlock_t lock; struct qdisc_rate_table *R_tab; struct qdisc_rate_table *P_tab; - - struct gnet_stats_basic bstats; - struct gnet_stats_queue qstats; - struct gnet_stats_rate_est rate_est; - spinlock_t *stats_lock; }; #ifdef CONFIG_NET_CLS_ACT -#define ACT_P_CREATED 1 -#define ACT_P_DELETED 1 -#define tca_gen(name) \ -struct tcf_##name *next; \ - u32 index; \ - int refcnt; \ - int bindcnt; \ - u32 capab; \ - int action; \ - struct tcf_t tm; \ - struct gnet_stats_basic bstats; \ - struct gnet_stats_queue qstats; \ - struct gnet_stats_rate_est rate_est; \ - spinlock_t *stats_lock; \ - spinlock_t lock - struct tcf_act_hdr { tca_gen(act_hdr); From kaber@trash.net Tue Dec 7 09:00:43 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 09:00:51 -0800 (PST) Received: from kaber.coreworks.de ([62.206.217.67]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7H0gun025650 for ; Tue, 7 Dec 2004 09:00:43 -0800 Received: from localhost ([127.0.0.1]) by kaber.coreworks.de with esmtp (Exim 4.34) id 1Cbigq-0001M5-Hg; Tue, 07 Dec 2004 17:59:52 +0100 Message-ID: <41B5E188.5050800@trash.net> Date: Tue, 07 Dec 2004 17:59:52 +0100 From: Patrick McHardy User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.7.3) Gecko/20041008 Debian/1.7.3-5 X-Accept-Language: en MIME-Version: 1.0 To: hadi@cyberus.ca CC: Andrew Morton , Thomas Cataldo , linux-kernel@vger.kernel.org, netdev@oss.sgi.com, tgraf@suug.ch, "David S. Miller" Subject: Re: Hard freeze with 2.6.10-rc3 and QoS, worked fine with 2.6.9 References: <1102380430.6103.6.camel@buffy> <20041206224441.628e7885.akpm@osdl.org> <1102422544.1088.98.camel@jzny.localdomain> In-Reply-To: <1102422544.1088.98.camel@jzny.localdomain> Content-Type: multipart/mixed; boundary="------------010705030109060209080805" X-archive-position: 12529 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kaber@trash.net Precedence: bulk X-list: netdev This is a multi-part message in MIME format. --------------010705030109060209080805 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit jamal wrote: >Can you do a: >tc -V > >This seems to point to probably be a backward compat issue which was >overlooked in the stats. > That's also what I thought at first. But the problem is in tcf_action_copy_stats, it assumes a->priv has the same layout as struct tcf_act_hdr, which is not true for struct tcf_police. This patch rearranges struct tcf_police to match tcf_act_hdr. Signed-off-by: Patrick McHardy --------------010705030109060209080805 Content-Type: text/plain; name="x" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="x" ===== include/net/act_api.h 1.4 vs edited ===== --- 1.4/include/net/act_api.h 2004-11-06 01:33:12 +01:00 +++ edited/include/net/act_api.h 2004-12-07 17:53:37 +01:00 @@ -8,15 +8,23 @@ #include #include +#define tca_gen(name) \ +struct tcf_##name *next; \ + u32 index; \ + int refcnt; \ + int bindcnt; \ + u32 capab; \ + int action; \ + struct tcf_t tm; \ + struct gnet_stats_basic bstats; \ + struct gnet_stats_queue qstats; \ + struct gnet_stats_rate_est rate_est; \ + spinlock_t *stats_lock; \ + spinlock_t lock + struct tcf_police { - struct tcf_police *next; - int refcnt; -#ifdef CONFIG_NET_CLS_ACT - int bindcnt; -#endif - u32 index; - int action; + tca_gen(police); int result; u32 ewma_rate; u32 burst; @@ -24,33 +32,14 @@ u32 toks; u32 ptoks; psched_time_t t_c; - spinlock_t lock; struct qdisc_rate_table *R_tab; struct qdisc_rate_table *P_tab; - - struct gnet_stats_basic bstats; - struct gnet_stats_queue qstats; - struct gnet_stats_rate_est rate_est; - spinlock_t *stats_lock; }; #ifdef CONFIG_NET_CLS_ACT #define ACT_P_CREATED 1 #define ACT_P_DELETED 1 -#define tca_gen(name) \ -struct tcf_##name *next; \ - u32 index; \ - int refcnt; \ - int bindcnt; \ - u32 capab; \ - int action; \ - struct tcf_t tm; \ - struct gnet_stats_basic bstats; \ - struct gnet_stats_queue qstats; \ - struct gnet_stats_rate_est rate_est; \ - spinlock_t *stats_lock; \ - spinlock_t lock struct tcf_act_hdr { --------------010705030109060209080805-- From tgraf@suug.ch Tue Dec 7 09:07:52 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 09:07:56 -0800 (PST) Received: from b.mx.projectdream.org (eth0-0.arisu.projectdream.org [194.158.4.191]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7H7pFZ026129 for ; Tue, 7 Dec 2004 09:07:51 -0800 Received: from postel.suug.ch (unknown [195.134.158.23]) (using TLSv1 with cipher EDH-RSA-DES-CBC3-SHA (168/168 bits)) (No client certificate requested) by b.mx.projectdream.org (Postfix) with ESMTP id A09EFF; Tue, 7 Dec 2004 18:07:06 +0100 (CET) Received: by postel.suug.ch (Postfix, from userid 10001) id 88BA81C0EA; Tue, 7 Dec 2004 18:07:48 +0100 (CET) Date: Tue, 7 Dec 2004 18:07:48 +0100 From: Thomas Graf To: Patrick McHardy Cc: hadi@cyberus.ca, Andrew Morton , Thomas Cataldo , linux-kernel@vger.kernel.org, netdev@oss.sgi.com, "David S. Miller" Subject: Re: Hard freeze with 2.6.10-rc3 and QoS, worked fine with 2.6.9 Message-ID: <20041207170748.GF1371@postel.suug.ch> References: <1102380430.6103.6.camel@buffy> <20041206224441.628e7885.akpm@osdl.org> <1102422544.1088.98.camel@jzny.localdomain> <41B5E188.5050800@trash.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <41B5E188.5050800@trash.net> X-archive-position: 12530 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: tgraf@suug.ch Precedence: bulk X-list: netdev * Patrick McHardy <41B5E188.5050800@trash.net> 2004-12-07 17:59 > jamal wrote: > > >Can you do a: > >tc -V > > > >This seems to point to probably be a backward compat issue which was > >overlooked in the stats. > > > That's also what I thought at first. But the problem is in > tcf_action_copy_stats, it assumes a->priv has the same layout as > struct tcf_act_hdr, which is not true for struct tcf_police. This > patch rearranges struct tcf_police to match tcf_act_hdr. Hehe, see my post a few minutes back. I came up with nearly the same solution ;-> The only difference to my patch is that I don't touch tcf_police if the action code isn't compiled. --- linux-2.6.10-rc2-bk13.orig/include/net/act_api.h 2004-11-30 14:01:11.000000000 +0100 +++ linux-2.6.10-rc2-bk13/include/net/act_api.h 2004-12-07 17:49:50.000000000 +0100 @@ -8,15 +8,42 @@ #include #include +#ifdef CONFIG_NET_CLS_ACT + +#define ACT_P_CREATED 1 +#define ACT_P_DELETED 1 +#define tca_gen(name) \ +struct tcf_##name *next; \ + u32 index; \ + int refcnt; \ + int bindcnt; \ + u32 capab; \ + int action; \ + struct tcf_t tm; \ + struct gnet_stats_basic bstats; \ + struct gnet_stats_queue qstats; \ + struct gnet_stats_rate_est rate_est; \ + spinlock_t *stats_lock; \ + spinlock_t lock + +#endif + struct tcf_police { +#ifdef CONFIG_NET_CLS_ACT + tca_gen(police); +#else struct tcf_police *next; int refcnt; -#ifdef CONFIG_NET_CLS_ACT - int bindcnt; -#endif u32 index; int action; + spinlock_t lock; + struct gnet_stats_basic bstats; + struct gnet_stats_queue qstats; + struct gnet_stats_rate_est rate_est; + spinlock_t *stats_lock; +#endif + int result; u32 ewma_rate; u32 burst; @@ -24,34 +51,12 @@ u32 toks; u32 ptoks; psched_time_t t_c; - spinlock_t lock; struct qdisc_rate_table *R_tab; struct qdisc_rate_table *P_tab; - - struct gnet_stats_basic bstats; - struct gnet_stats_queue qstats; - struct gnet_stats_rate_est rate_est; - spinlock_t *stats_lock; }; #ifdef CONFIG_NET_CLS_ACT -#define ACT_P_CREATED 1 -#define ACT_P_DELETED 1 -#define tca_gen(name) \ -struct tcf_##name *next; \ - u32 index; \ - int refcnt; \ - int bindcnt; \ - u32 capab; \ - int action; \ - struct tcf_t tm; \ - struct gnet_stats_basic bstats; \ - struct gnet_stats_queue qstats; \ - struct gnet_stats_rate_est rate_est; \ - spinlock_t *stats_lock; \ - spinlock_t lock - struct tcf_act_hdr { tca_gen(act_hdr); From tgraf@suug.ch Tue Dec 7 09:23:52 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 09:23:56 -0800 (PST) Received: from b.mx.projectdream.org (eth0-0.arisu.projectdream.org [194.158.4.191]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7HNpXP026893 for ; Tue, 7 Dec 2004 09:23:51 -0800 Received: from postel.suug.ch (unknown [195.134.158.23]) (using TLSv1 with cipher EDH-RSA-DES-CBC3-SHA (168/168 bits)) (No client certificate requested) by b.mx.projectdream.org (Postfix) with ESMTP id 2D965F; Tue, 7 Dec 2004 18:23:07 +0100 (CET) Received: by postel.suug.ch (Postfix, from userid 10001) id E95A01C0EA; Tue, 7 Dec 2004 18:23:49 +0100 (CET) Date: Tue, 7 Dec 2004 18:23:49 +0100 From: Thomas Graf To: "David S. Miller" Cc: netdev@oss.sgi.com Subject: [PATCH] PKT_SCHED: validate policer configuration TLVs Message-ID: <20041207172349.GG1371@postel.suug.ch> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-archive-position: 12531 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: tgraf@suug.ch Precedence: bulk X-list: netdev Adds TLV size sanity checks for policer configuration. Signed-off-by: Thomas Graf --- linux-2.6.10-rc2-bk13.orig/net/sched/police.c 2004-11-30 14:01:12.000000000 +0100 +++ linux-2.6.10-rc2-bk13/net/sched/police.c 2004-12-07 17:24:01.000000000 +0100 @@ -180,7 +180,8 @@ if (rtattr_parse(tb, TCA_POLICE_MAX, RTA_DATA(rta), RTA_PAYLOAD(rta)) < 0) return -1; - if (tb[TCA_POLICE_TBF-1] == NULL) + if (tb[TCA_POLICE_TBF-1] == NULL || + RTA_PAYLOAD(tb[TCA_POLICE_TBF-1]) != sizeof(*parm)) return -1; parm = RTA_DATA(tb[TCA_POLICE_TBF-1]); @@ -220,11 +221,17 @@ goto failure; } } - if (tb[TCA_POLICE_RESULT-1]) + if (tb[TCA_POLICE_RESULT-1]) { + if (RTA_PAYLOAD(tb[TCA_POLICE_RESULT-1]) != sizeof(u32)) + goto failure; p->result = *(int*)RTA_DATA(tb[TCA_POLICE_RESULT-1]); + } #ifdef CONFIG_NET_ESTIMATOR - if (tb[TCA_POLICE_AVRATE-1]) + if (tb[TCA_POLICE_AVRATE-1]) { + if (RTA_PAYLOAD(tb[TCA_POLICE_AVRATE-1]) != sizeof(u32)) + goto failure; p->ewma_rate = *(u32*)RTA_DATA(tb[TCA_POLICE_AVRATE-1]); + } #endif p->toks = p->burst = parm->burst; p->mtu = parm->mtu; @@ -424,7 +431,8 @@ if (rtattr_parse(tb, TCA_POLICE_MAX, RTA_DATA(rta), RTA_PAYLOAD(rta)) < 0) return NULL; - if (tb[TCA_POLICE_TBF-1] == NULL) + if (tb[TCA_POLICE_TBF-1] == NULL || + RTA_PAYLOAD(tb[TCA_POLICE_TBF-1]) != sizeof(*parm)) return NULL; parm = RTA_DATA(tb[TCA_POLICE_TBF-1]); @@ -449,11 +457,17 @@ (p->P_tab = qdisc_get_rtab(&parm->peakrate, tb[TCA_POLICE_PEAKRATE-1])) == NULL) goto failure; } - if (tb[TCA_POLICE_RESULT-1]) + if (tb[TCA_POLICE_RESULT-1]) { + if (RTA_PAYLOAD(tb[TCA_POLICE_RESULT-1]) != sizeof(u32)) + goto failure; p->result = *(int*)RTA_DATA(tb[TCA_POLICE_RESULT-1]); + } #ifdef CONFIG_NET_ESTIMATOR - if (tb[TCA_POLICE_AVRATE-1]) + if (tb[TCA_POLICE_AVRATE-1]) { + if (RTA_PAYLOAD(tb[TCA_POLICE_AVRATE-1]) != sizeof(u32)) + goto failure; p->ewma_rate = *(u32*)RTA_DATA(tb[TCA_POLICE_AVRATE-1]); + } #endif p->toks = p->burst = parm->burst; p->mtu = parm->mtu; From kaber@trash.net Tue Dec 7 09:24:28 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 09:24:35 -0800 (PST) Received: from kaber.coreworks.de ([62.206.217.67]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7HORuZ026998 for ; Tue, 7 Dec 2004 09:24:28 -0800 Received: from localhost ([127.0.0.1]) by kaber.coreworks.de with esmtp (Exim 4.34) id 1Cbj3y-0001TU-2L; Tue, 07 Dec 2004 18:23:46 +0100 Message-ID: <41B5E722.2080600@trash.net> Date: Tue, 07 Dec 2004 18:23:46 +0100 From: Patrick McHardy User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.7.3) Gecko/20041008 Debian/1.7.3-5 X-Accept-Language: en MIME-Version: 1.0 To: Thomas Graf CC: hadi@cyberus.ca, Andrew Morton , Thomas Cataldo , linux-kernel@vger.kernel.org, netdev@oss.sgi.com, "David S. Miller" Subject: Re: Hard freeze with 2.6.10-rc3 and QoS, worked fine with 2.6.9 References: <1102380430.6103.6.camel@buffy> <20041206224441.628e7885.akpm@osdl.org> <1102422544.1088.98.camel@jzny.localdomain> <41B5E188.5050800@trash.net> <20041207170748.GF1371@postel.suug.ch> In-Reply-To: <20041207170748.GF1371@postel.suug.ch> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 12532 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kaber@trash.net Precedence: bulk X-list: netdev Thomas Graf wrote: >* Patrick McHardy <41B5E188.5050800@trash.net> 2004-12-07 17:59 > > >>That's also what I thought at first. But the problem is in >>tcf_action_copy_stats, it assumes a->priv has the same layout as >>struct tcf_act_hdr, which is not true for struct tcf_police. This >>patch rearranges struct tcf_police to match tcf_act_hdr. >> >> > >Hehe, see my post a few minutes back. I came up with nearly the same >solution ;-> The only difference to my patch is that I don't touch >tcf_police if the action code isn't compiled. > > Either one is fine with me, although I would prefer to see the number of ifdefs in this area going down, not up :) Regards Patrick From holt@lnx-holt.americas.sgi.com Tue Dec 7 09:25:45 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 09:25:49 -0800 (PST) Received: from omx2.sgi.com (omx2-ext.sgi.com [192.48.171.19]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7HPiYZ027588 for ; Tue, 7 Dec 2004 09:25:45 -0800 Received: from flecktone.americas.sgi.com (flecktone.americas.sgi.com [198.149.16.15]) by omx2.sgi.com (8.12.11/8.12.9/linux-outbound_gateway-1.1) with ESMTP id iB7IjxSd002036 for ; Tue, 7 Dec 2004 10:45:59 -0800 Received: from thistle-e236.americas.sgi.com (thistle-e236.americas.sgi.com [128.162.236.204]) by flecktone.americas.sgi.com (8.12.9/8.12.10/SGI_generic_relay-1.2) with ESMTP id iB7HPCam3777258; Tue, 7 Dec 2004 11:25:12 -0600 (CST) Received: from lnx-holt.americas.sgi.com (IDENT:U2FsdGVkX1/B1RZRupXVrXPgWg4srt3WGQAx3W9OB5U@lnx-holt.americas.sgi.com [128.162.233.109]) by thistle-e236.americas.sgi.com (8.12.9/SGI-server-1.8) with ESMTP id iB7HPBtC16861234; Tue, 7 Dec 2004 11:25:11 -0600 (CST) Received: from lnx-holt.americas.sgi.com (localhost.localdomain [127.0.0.1]) by lnx-holt.americas.sgi.com (8.13.1/8.12.11) with ESMTP id iB7HPBJC012584; Tue, 7 Dec 2004 11:25:11 -0600 Received: (from holt@localhost) by lnx-holt.americas.sgi.com (8.13.1/8.13.1/Submit) id iB7HPBta012583; Tue, 7 Dec 2004 11:25:11 -0600 Date: Tue, 7 Dec 2004 11:25:10 -0600 From: Robin Holt To: linux-kernel@vger.kernel.org, netdev@oss.sgi.com Subject: What is a reasonable upper limit to the rt_hash_table. Message-ID: <20041207172510.GC11423@lnx-holt.americas.sgi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.1i X-archive-position: 12533 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: holt@sgi.com Precedence: bulk X-list: netdev We have a system with a very large amount of memory. We are noticing pauses of approximately 5 seconds every 10 minutes. We tracked it down to rt_run_flush holding off other timer processing while it scans the rt_hash_table. The following is from the boot: IP: routing cache hash table of 33554432 buckets, 524288Kbytes This seems like an outrageously large value. I realize the 2.6 kernel has rhash_entries as a boot option. Can I get some guidance on what a reasonable upper limit would be? What is this guidance based upon? What is the reason for not making that upper limit a default and let rhash_entries override to make it larger if a site actually needed it? Thank you in advance, Robin Holt From kernel@linuxace.com Tue Dec 7 09:28:43 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 09:28:47 -0800 (PST) Received: from home.linuxace.com (adsl-67-120-171-161.dsl.lsan03.pacbell.net [67.120.171.161]) by oss.sgi.com (8.13.0/8.13.0) with SMTP id iB7HSdLJ027951 for ; Tue, 7 Dec 2004 09:28:43 -0800 Received: (qmail 25817 invoked by uid 0); 7 Dec 2004 17:28:12 -0000 Date: Tue, 7 Dec 2004 09:28:12 -0800 From: Phil Oester To: Mitchell Blank Jr Cc: "David S. Miller" , shemminger@osdl.org, linux-net@vger.kernel.org, linux-kernel@vger.kernel.org, netdev@oss.sgi.com Subject: Re: [PATCH] fix select() for SOCK_RAW sockets Message-ID: <20041207172812.GA25810@linuxace.com> References: <20041207003525.GA22933@linuxace.com> <20041207025218.GB61527@gaz.sfgoth.com> <20041207045302.GA23746@linuxace.com> <20041207054840.GD61527@gaz.sfgoth.com> <20041207150834.GA75700@gaz.sfgoth.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20041207150834.GA75700@gaz.sfgoth.com> User-Agent: Mutt/1.4.1i X-archive-position: 12534 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kernel@linuxace.com Precedence: bulk X-list: netdev On Tue, Dec 07, 2004 at 07:08:34AM -0800, Mitchell Blank Jr wrote: > Phil: Here's a real patch for you to test. I actually left inet_dgram_ops > alone since it's an exported symbol (two of the users just want the .do_ioctl > value which is the same between SOCK_DGRAM and SOCK_RAW; the other is ipv6 > where it's clearly dealing with a UDP socket -- therefore I think its safest > to leave inet_dgram_ops to have the UDP behavior) > > Davem: I only tested that this doesn't break UDP; if it works for Phil and > Stephen can verify that it doesn't break his bad-checksum UDP tests then > please push it for 2.6.10. Yup, that does indeed fix it for me, thanks. Phil From yoshfuji@linux-ipv6.org Tue Dec 7 09:34:42 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 09:34:48 -0800 (PST) Received: from yue.st-paulia.net (yue.linux-ipv6.org [203.178.140.15]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7HYJ5t028361 for ; Tue, 7 Dec 2004 09:34:42 -0800 Received: from localhost (localhost [127.0.0.1]) by yue.st-paulia.net (Postfix) with ESMTP id 79EF433CE5; Wed, 8 Dec 2004 02:35:31 +0900 (JST) Date: Wed, 08 Dec 2004 02:35:30 +0900 (JST) Message-Id: <20041208.023530.26430801.yoshfuji@linux-ipv6.org> To: mitch@sfgoth.com Cc: kernel@linuxace.com, davem@davemloft.net, shemminger@osdl.org, linux-net@vger.kernel.org, linux-kernel@vger.kernel.org, netdev@oss.sgi.com, yoshfuji@linux-ipv6.org Subject: Re: [PATCH] fix select() for SOCK_RAW sockets From: YOSHIFUJI Hideaki / =?iso-2022-jp?B?GyRCNUhGIzFRTEAbKEI=?= In-Reply-To: <20041207150834.GA75700@gaz.sfgoth.com> References: <20041207045302.GA23746@linuxace.com> <20041207054840.GD61527@gaz.sfgoth.com> <20041207150834.GA75700@gaz.sfgoth.com> Organization: USAGI Project X-URL: http://www.yoshifuji.org/%7Ehideaki/ X-Fingerprint: 9022 65EB 1ECF 3AD1 0BDF 80D8 4807 F894 E062 0EEA X-PGP-Key-URL: http://www.yoshifuji.org/%7Ehideaki/hideaki@yoshifuji.org.asc X-Face: "5$Al-.M>NJ%a'@hhZdQm:."qn~PA^gq4o*>iCFToq*bAi#4FRtx}enhuQKz7fNqQz\BYU] $~O_5m-9'}MIs`XGwIEscw;e5b>n"B_?j/AkL~i/MEaZBLP X-Mailer: Mew version 2.2 on Emacs 20.7 / Mule 4.1 (AOI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-archive-position: 12535 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: yoshfuji@linux-ipv6.org Precedence: bulk X-list: netdev In article <20041207150834.GA75700@gaz.sfgoth.com> (at Tue, 7 Dec 2004 07:08:34 -0800), Mitchell Blank Jr says: > Phil: Here's a real patch for you to test. I actually left inet_dgram_ops > alone since it's an exported symbol (two of the users just want the .do_ioctl > value which is the same between SOCK_DGRAM and SOCK_RAW; the other is ipv6 > where it's clearly dealing with a UDP socket -- therefore I think its safest > to leave inet_dgram_ops to have the UDP behavior) Probably, we need to do the same for ipv6, don't we? --yoshfuji From shemminger@osdl.org Tue Dec 7 09:46:06 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 09:46:10 -0800 (PST) Received: from mail.osdl.org (fw.osdl.org [65.172.181.6]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7Hk5Mv028936 for ; Tue, 7 Dec 2004 09:46:06 -0800 Received: from dxpl.pdx.osdl.net (dxpl.pdx.osdl.net [172.20.1.103]) by mail.osdl.org (8.11.6/8.11.6) with ESMTP id iB7HjZ926090; Tue, 7 Dec 2004 09:45:35 -0800 Date: Tue, 7 Dec 2004 09:45:35 -0800 From: Stephen Hemminger To: Mitchell Blank Jr Cc: Phil Oester , "David S. Miller" , linux-net@vger.kernel.org, linux-kernel@vger.kernel.org, netdev@oss.sgi.com Subject: Re: [PATCH] fix select() for SOCK_RAW sockets Message-Id: <20041207094535.11080082@dxpl.pdx.osdl.net> In-Reply-To: <20041207150834.GA75700@gaz.sfgoth.com> References: <20041207003525.GA22933@linuxace.com> <20041207025218.GB61527@gaz.sfgoth.com> <20041207045302.GA23746@linuxace.com> <20041207054840.GD61527@gaz.sfgoth.com> <20041207150834.GA75700@gaz.sfgoth.com> Organization: Open Source Development Lab X-Mailer: Sylpheed version 0.9.10claws (GTK+ 1.2.10; x86_64-suse-linux) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-archive-position: 12536 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: shemminger@osdl.org Precedence: bulk X-list: netdev On Tue, 7 Dec 2004 07:08:34 -0800 Mitchell Blank Jr wrote: > Phil: Here's a real patch for you to test. I actually left inet_dgram_ops > alone since it's an exported symbol (two of the users just want the .do_ioctl > value which is the same between SOCK_DGRAM and SOCK_RAW; the other is ipv6 > where it's clearly dealing with a UDP socket -- therefore I think its safest > to leave inet_dgram_ops to have the UDP behavior) > > Davem: I only tested that this doesn't break UDP; if it works for Phil and > Stephen can verify that it doesn't break his bad-checksum UDP tests then > please push it for 2.6.10. > Thanks, I'll retest UDP today, but it looks right. From tgraf@suug.ch Tue Dec 7 09:53:03 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 09:53:15 -0800 (PST) Received: from b.mx.projectdream.org (eth0-0.arisu.projectdream.org [194.158.4.191]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7Hr2iQ029417 for ; Tue, 7 Dec 2004 09:53:02 -0800 Received: from postel.suug.ch (unknown [195.134.158.23]) (using TLSv1 with cipher EDH-RSA-DES-CBC3-SHA (168/168 bits)) (No client certificate requested) by b.mx.projectdream.org (Postfix) with ESMTP id E5372F; Tue, 7 Dec 2004 18:52:17 +0100 (CET) Received: by postel.suug.ch (Postfix, from userid 10001) id B8A6F1C0EA; Tue, 7 Dec 2004 18:52:59 +0100 (CET) Date: Tue, 7 Dec 2004 18:52:59 +0100 From: Thomas Graf To: jamal Cc: Michal Ludvig , Andrew Morton , Stephen Hemminger , netdev@oss.sgi.com, Jan Kara Subject: Re: [PATCH] rtnetlink & address family problem Message-ID: <20041207175259.GH1371@postel.suug.ch> References: <41B0A5B4.6060108@suse.cz> <20041206140214.GA749@postel.suug.ch> <1102386461.1093.26.camel@jzny.localdomain> <20041207124922.GA1371@postel.suug.ch> <1102424568.1089.120.camel@jzny.localdomain> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1102424568.1089.120.camel@jzny.localdomain> X-archive-position: 12537 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: tgraf@suug.ch Precedence: bulk X-list: netdev > BTW, did the gnet_stats patches to iproute2 ever get merged? > If you have cycles, can you please look at that hang being reported > using older tc with 2.6.10-rc3? They're not in bk://developer.osdl.org/iproute2 so I guess not. I've put a iproute2 including all my changes into a tarball at: http://people.suug.ch/~tgr/iproute2/iproute2-2.6.9-tgr.tar.gz From shemminger@osdl.org Tue Dec 7 10:02:30 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 10:02:34 -0800 (PST) Received: from mail.osdl.org (fw.osdl.org [65.172.181.6]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7I2Rel030066 for ; Tue, 7 Dec 2004 10:02:30 -0800 Received: from dxpl.pdx.osdl.net (dxpl.pdx.osdl.net [172.20.1.103]) by mail.osdl.org (8.11.6/8.11.6) with ESMTP id iB7I1e928573; Tue, 7 Dec 2004 10:01:40 -0800 Date: Tue, 7 Dec 2004 10:01:40 -0800 From: Stephen Hemminger To: YOSHIFUJI Hideaki / =?ISO-8859-1?B?X19fX19fX19fX19f?= Cc: mitch@sfgoth.com, kernel@linuxace.com, davem@davemloft.net, linux-net@vger.kernel.org, linux-kernel@vger.kernel.org, netdev@oss.sgi.com, yoshfuji@linux-ipv6.org Subject: [PATCH] fix select() for SOCK_RAW sockets (ipv6) Message-Id: <20041207100140.781f4c00@dxpl.pdx.osdl.net> In-Reply-To: <20041208.023530.26430801.yoshfuji@linux-ipv6.org> References: <20041207045302.GA23746@linuxace.com> <20041207054840.GD61527@gaz.sfgoth.com> <20041207150834.GA75700@gaz.sfgoth.com> <20041208.023530.26430801.yoshfuji@linux-ipv6.org> Organization: Open Source Development Lab X-Mailer: Sylpheed version 0.9.10claws (GTK+ 1.2.10; x86_64-suse-linux) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-archive-position: 12538 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: shemminger@osdl.org Precedence: bulk X-list: netdev > Probably, we need to do the same for ipv6, don't we? diff -Nru a/net/ipv6/af_inet6.c b/net/ipv6/af_inet6.c --- a/net/ipv6/af_inet6.c 2004-12-07 10:02:50 -08:00 +++ b/net/ipv6/af_inet6.c 2004-12-07 10:02:50 -08:00 @@ -513,6 +513,27 @@ .sendpage = sock_no_sendpage, }; +struct proto_ops inet6_raw_ops = { + .family = PF_INET6, + .owner = THIS_MODULE, + .release = inet6_release, + .bind = inet6_bind, + .connect = inet_dgram_connect, /* ok */ + .socketpair = sock_no_socketpair, /* a do nothing */ + .accept = sock_no_accept, /* a do nothing */ + .getname = inet6_getname, + .poll = datagram_poll, /* ok */ + .ioctl = inet6_ioctl, /* must change */ + .listen = sock_no_listen, /* ok */ + .shutdown = inet_shutdown, /* ok */ + .setsockopt = sock_common_setsockopt, /* ok */ + .getsockopt = sock_common_getsockopt, /* ok */ + .sendmsg = inet_sendmsg, /* ok */ + .recvmsg = sock_common_recvmsg, /* ok */ + .mmap = sock_no_mmap, + .sendpage = sock_no_sendpage, +}; + static struct net_proto_family inet6_family_ops = { .family = PF_INET6, .create = inet6_create, @@ -528,7 +549,7 @@ .type = SOCK_RAW, .protocol = IPPROTO_IP, /* wild card */ .prot = &rawv6_prot, - .ops = &inet6_dgram_ops, + .ops = &inet6_raw_ops, .capability = CAP_NET_RAW, .no_check = UDP_CSUM_DEFAULT, .flags = INET_PROTOSW_REUSE, From kdesler@soohrt.org Tue Dec 7 10:39:12 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 10:39:19 -0800 (PST) Received: from quickstop.soohrt.org (quickstop.soohrt.org [81.2.155.147]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7IdBCg031069 for ; Tue, 7 Dec 2004 10:39:12 -0800 Received: (qmail 3135 invoked by uid 1018); 7 Dec 2004 18:38:45 -0000 Date: Tue, 7 Dec 2004 19:38:45 +0100 From: Karsten Desler To: Karsten Desler Cc: P@draigBrady.com, "David S. Miller" , netdev@oss.sgi.com, linux-kernel@vger.kernel.org Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets Message-ID: <20041207183845.GA2078@quickstop.soohrt.org> References: <20041206205305.GA11970@soohrt.org> <20041206134849.498bfc93.davem@davemloft.net> <20041206224107.GA8529@soohrt.org> <41B58A58.8010007@draigBrady.com> <20041207112139.GA3610@soohrt.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20041207112139.GA3610@soohrt.org> User-Agent: Mutt/1.5.6+20040722i X-archive-position: 12539 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kdesler@soohrt.org Precedence: bulk X-list: netdev Karsten Desler wrote: > > Also have a look at http://www.hipac.org/ as netfilter > > has silly scalability properties. > > I did before, but I read on Harald Weltes' weblog that 2.4 gives > slightly worse pps results than 2.6, and since the cpu usage is as high > as it is, I didn't want to take any more performance hits. > I'll try to see what performance impact the netfilter rules have during > peak load. using 2 CPUs System load: 61.4% || Free: 51.0%(0) 26.3%(1) System load: 59.6% || Free: 53.6%(0) 27.3%(1) System load: 59.6% || Free: 53.6%(0) 27.3%(1) System load: 59.7% || Free: 53.6%(0) 27.0%(1) System load: 60.3% || Free: 53.0%(0) 26.4%(1) System load: 51.9% || Free: 60.4%(0) 35.8%(1) <- iptables -F System load: 50.1% || Free: 62.1%(0) 37.7%(1) System load: 50.1% || Free: 62.0%(0) 37.8%(1) System load: 50.6% || Free: 61.6%(0) 37.2%(1) System load: 50.5% || Free: 61.7%(0) 37.3%(1) > > I also notice that a lot of time is spent allocating > > and freeing the packet buffers (and possible hidden > > time due to cache misses due to allocating on one > > CPU and freeing on another?). > > How many [RT]xDescriptors do you have configured by the way? > > 256. I increased them to 1024 shortly after the profiling run, but > didn't notice any change in the cpu usage (will try again with cyclesoak). Again, no effect. Cheers, Karsten From davem@davemloft.net Tue Dec 7 10:50:36 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 10:50:43 -0800 (PST) Received: from cheetah.davemloft.net (mail@adsl-63-197-226-105.dsl.snfc21.pacbell.net [63.197.226.105]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7IoXF8031739 for ; Tue, 7 Dec 2004 10:50:36 -0800 Received: from localhost ([127.0.0.1] helo=cheetah.davemloft.net ident=davem) by cheetah.davemloft.net with smtp (Exim 3.36 #1 (Debian)) id 1CbkNj-0000Gd-00; Tue, 07 Dec 2004 10:48:15 -0800 Date: Tue, 7 Dec 2004 10:48:15 -0800 From: "David S. Miller" To: Stephen Hemminger Cc: yoshfuji@linux-ipv6.org, mitch@sfgoth.com, kernel@linuxace.com, linux-net@vger.kernel.org, linux-kernel@vger.kernel.org, netdev@oss.sgi.com Subject: Re: [PATCH] fix select() for SOCK_RAW sockets (ipv6) Message-Id: <20041207104815.3f7a4684.davem@davemloft.net> In-Reply-To: <20041207100140.781f4c00@dxpl.pdx.osdl.net> References: <20041207045302.GA23746@linuxace.com> <20041207054840.GD61527@gaz.sfgoth.com> <20041207150834.GA75700@gaz.sfgoth.com> <20041208.023530.26430801.yoshfuji@linux-ipv6.org> <20041207100140.781f4c00@dxpl.pdx.osdl.net> X-Mailer: Sylpheed version 1.0.0beta3 (GTK+ 1.2.10; sparc-unknown-linux-gnu) X-Face: "_;p5u5aPsO,_Vsx"^v-pEq09'CU4&Dc1$fQExov$62l60cgCc%FnIwD=.UF^a>?5'9Kn[;433QFVV9M..2eN.@4ZWPGbdi<=?[:T>y?SD(R*-3It"Vj:)"dP Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-archive-position: 12540 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: davem@davemloft.net Precedence: bulk X-list: netdev On Tue, 7 Dec 2004 10:01:40 -0800 Stephen Hemminger wrote: > > Probably, we need to do the same for ipv6, don't we? > > diff -Nru a/net/ipv6/af_inet6.c b/net/ipv6/af_inet6.c > --- a/net/ipv6/af_inet6.c 2004-12-07 10:02:50 -08:00 > +++ b/net/ipv6/af_inet6.c 2004-12-07 10:02:50 -08:00 > @@ -513,6 +513,27 @@ We didn't do the "UDP select() fix" on ipv6 because we don't have the delayed checksumming optimization there. So the ipv6 side really doesn't need this change. Or did I miss something? From shemminger@osdl.org Tue Dec 7 10:57:07 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 10:57:12 -0800 (PST) Received: from mail.osdl.org (fw.osdl.org [65.172.181.6]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7Iv5kV032284 for ; Tue, 7 Dec 2004 10:57:06 -0800 Received: from dxpl.pdx.osdl.net (dxpl.pdx.osdl.net [172.20.1.103]) by mail.osdl.org (8.11.6/8.11.6) with ESMTP id iB7IuK908517; Tue, 7 Dec 2004 10:56:20 -0800 Date: Tue, 7 Dec 2004 10:56:20 -0800 From: Stephen Hemminger To: "David S. Miller" Cc: yoshfuji@linux-ipv6.org, mitch@sfgoth.com, kernel@linuxace.com, linux-net@vger.kernel.org, linux-kernel@vger.kernel.org, netdev@oss.sgi.com Subject: Re: [PATCH] fix select() for SOCK_RAW sockets (ipv6) Message-Id: <20041207105620.241652d0@dxpl.pdx.osdl.net> In-Reply-To: <20041207104815.3f7a4684.davem@davemloft.net> References: <20041207045302.GA23746@linuxace.com> <20041207054840.GD61527@gaz.sfgoth.com> <20041207150834.GA75700@gaz.sfgoth.com> <20041208.023530.26430801.yoshfuji@linux-ipv6.org> <20041207100140.781f4c00@dxpl.pdx.osdl.net> <20041207104815.3f7a4684.davem@davemloft.net> Organization: Open Source Development Lab X-Mailer: Sylpheed version 0.9.10claws (GTK+ 1.2.10; x86_64-suse-linux) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-archive-position: 12541 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: shemminger@osdl.org Precedence: bulk X-list: netdev On Tue, 7 Dec 2004 10:48:15 -0800 "David S. Miller" wrote: > On Tue, 7 Dec 2004 10:01:40 -0800 > Stephen Hemminger wrote: > > > > Probably, we need to do the same for ipv6, don't we? > > > > diff -Nru a/net/ipv6/af_inet6.c b/net/ipv6/af_inet6.c > > --- a/net/ipv6/af_inet6.c 2004-12-07 10:02:50 -08:00 > > +++ b/net/ipv6/af_inet6.c 2004-12-07 10:02:50 -08:00 > > @@ -513,6 +513,27 @@ > > We didn't do the "UDP select() fix" on ipv6 because we don't > have the delayed checksumming optimization there. So the > ipv6 side really doesn't need this change. > > Or did I miss something? yeah, didn't look that deeply. From shemminger@osdl.org Tue Dec 7 11:13:19 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 11:13:25 -0800 (PST) Received: from mail.osdl.org (fw.osdl.org [65.172.181.6]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7JDION000549 for ; Tue, 7 Dec 2004 11:13:19 -0800 Received: from dxpl.pdx.osdl.net (dxpl.pdx.osdl.net [172.20.1.103]) by mail.osdl.org (8.11.6/8.11.6) with ESMTP id iB7JC6911442; Tue, 7 Dec 2004 11:12:06 -0800 Date: Tue, 7 Dec 2004 11:12:06 -0800 From: Stephen Hemminger To: Thomas Graf Cc: jamal , Michal Ludvig , Andrew Morton , netdev@oss.sgi.com, Jan Kara Subject: Re: [PATCH] rtnetlink & address family problem Message-Id: <20041207111206.5060ccfc@dxpl.pdx.osdl.net> In-Reply-To: <20041207175259.GH1371@postel.suug.ch> References: <41B0A5B4.6060108@suse.cz> <20041206140214.GA749@postel.suug.ch> <1102386461.1093.26.camel@jzny.localdomain> <20041207124922.GA1371@postel.suug.ch> <1102424568.1089.120.camel@jzny.localdomain> <20041207175259.GH1371@postel.suug.ch> Organization: Open Source Development Lab X-Mailer: Sylpheed version 0.9.10claws (GTK+ 1.2.10; x86_64-suse-linux) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-archive-position: 12542 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: shemminger@osdl.org Precedence: bulk X-list: netdev On Tue, 7 Dec 2004 18:52:59 +0100 Thomas Graf wrote: > http://people.suug.ch/~tgr/iproute2/iproute2-2.6.9-tgr.tar.gz Thanks, I'm behind on iproute2 From brad@danga.com Tue Dec 7 11:18:30 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 11:18:35 -0800 (PST) Received: from danga.com (danga.com [66.150.15.140]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7JITDm001015 for ; Tue, 7 Dec 2004 11:18:30 -0800 Received: by danga.com (Postfix, from userid 1000) id 02DAA3BC0B5; Tue, 7 Dec 2004 11:18:04 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by danga.com (Postfix) with ESMTP id E2DCD87C121; Tue, 7 Dec 2004 11:18:04 -0800 (PST) Date: Tue, 7 Dec 2004 11:18:04 -0800 (PST) From: Brad Fitzpatrick X-X-Sender: bradfitz@danga.com To: linux-net@vger.kernel.org, netdev@oss.sgi.com Subject: e1000, 2.6.9: swapper: page allocation failure. order:1, mode:0x20 Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 12543 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: brad@danga.com Precedence: bulk X-list: netdev Hello, On a busy database server I'm getting tons of these w/ 2.6.9 and e1000, no NAPI, no jumbo frames: swapper: page allocation failure. order:1, mode:0x20 [] __alloc_pages+0x2e9/0x30c [] e1000_xmit_frame+0x816/0x820 [] __get_free_pages+0x1d/0x30 [] kmem_getpages+0x26/0xd4 [] cache_grow+0xb3/0x148 [] cache_alloc_refill+0x1ae/0x1fc [] kmem_cache_alloc+0x43/0x4c [] sk_alloc+0x26/0x98 [] tcp_create_openreq_child+0x24/0x538 [] tcp_v4_syn_recv_sock+0x4c/0x2d8 [] tcp_check_req+0x23f/0x3e0 [] e1000_xmit_frame+0x816/0x820 [] ip_output+0x76/0x7c [] qdisc_restart+0x1f/0x1e8 [] dev_queue_xmit+0x234/0x244 [] ip_finish_output+0x165/0x1b0 [] ip_output+0x76/0x7c [] ip_queue_xmit+0x3a9/0x428 [] recalc_task_prio+0x128/0x138 [] activate_task+0x9f/0xb0 [] recalc_task_prio+0x128/0x138 [] activate_task+0x9f/0xb0 [] recalc_task_prio+0x128/0x138 [] activate_task+0x9f/0xb0 [] try_to_wake_up+0x25a/0x268 [] default_wake_function+0x17/0x1c [] default_wake_function+0x17/0x1c [] tcp_v4_hnd_req+0x32/0x190 [] tcp_v4_hnd_req+0x4c/0x190 [] tcp_v4_do_rcv+0xbb/0x120 [] tcp_v4_do_rcv+0x94/0x120 [] tcp_v4_rcv+0x499/0x758 [] ip_route_input+0x33/0x140 [] ip_local_deliver+0x9c/0x13c [] ip_rcv+0x34d/0x3ec [] netif_receive_skb+0x149/0x180 [] process_backlog+0x85/0x114 [] net_rx_action+0x80/0x128 [] __do_softirq+0x6a/0xd4 [] do_softirq+0x28/0x30 [] do_IRQ+0x10e/0x124 [] common_interrupt+0x18/0x20 [] default_idle+0x29/0x34 [] cpu_idle+0x30/0x44 [] rest_init+0x47/0x48 [] start_kernel+0x161/0x168 From following lists, I read this as e1000 in interrupt context tried to allocate two contiguous pages without sleeping, and the "emergency pools" or whatnot weren't full enough? We're not using jumbo frames, ... what is e1000 allocating 8k for? (or am I misreading this?) Also not using TSO, unless it's the default again in 2.6.9. What should I tune to avoid this from happening? Thanks! Brad From kdesler@soohrt.org Tue Dec 7 13:11:02 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 13:11:07 -0800 (PST) Received: from quickstop.soohrt.org (quickstop.soohrt.org [81.2.155.147]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7LB0JT009307 for ; Tue, 7 Dec 2004 13:11:02 -0800 Received: (qmail 2600 invoked by uid 1018); 7 Dec 2004 21:10:35 -0000 Date: Tue, 7 Dec 2004 22:10:35 +0100 From: Karsten Desler To: netdev@oss.sgi.com Cc: linux-kernel@vger.kernel.org, "David S. Miller" , jamal , Robert Olsson , P@draigBrady.com Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets Message-ID: <20041207211035.GA20286@quickstop.soohrt.org> References: <20041206205305.GA11970@soohrt.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="y0ulUmNC+osPPQO6" Content-Disposition: inline In-Reply-To: <20041206205305.GA11970@soohrt.org> User-Agent: Mutt/1.5.6+20040722i X-archive-position: 12544 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kdesler@soohrt.org Precedence: bulk X-list: netdev --y0ulUmNC+osPPQO6 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Karsten Desler wrote: > Current packetload on eth0 (and reversed on eth1): > 115kpps tx > 135kpps rx I totally forgot to mention: There are approximately 100k concurrent flows. From dmesg: IP: routing cache hash table of 16384 buckets, 128Kbytes Maybe there is some contention on the rt_hash_table spinlocks? Is the attached patch enough to increase the size? - Karsten --y0ulUmNC+osPPQO6 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="rtcachesize.patch" --- linux/net/ipv4/route.c~old 2004-12-07 21:55:22.000000000 +0100 +++ linux/net/ipv4/route.c 2004-12-07 21:55:32.000000000 +0100 @@ -2728,7 +2728,7 @@ if (!ipv4_dst_ops.kmem_cachep) panic("IP: failed to allocate ip_dst_cache\n"); - goal = num_physpages >> (26 - PAGE_SHIFT); + goal = num_physpages >> (23 - PAGE_SHIFT); if (rhash_entries) goal = (rhash_entries * sizeof(struct rt_hash_bucket)) >> PAGE_SHIFT; for (order = 0; (1UL << order) < goal; order++) --y0ulUmNC+osPPQO6-- From buytenh@wantstofly.org Tue Dec 7 14:25:47 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 14:25:51 -0800 (PST) Received: from xi.wantstofly.org (alephnull.demon.nl [212.238.201.82]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7MPi5n011327 for ; Tue, 7 Dec 2004 14:25:47 -0800 Received: by xi.wantstofly.org (Postfix, from userid 500) id 5972E2B0ED; Tue, 7 Dec 2004 23:25:22 +0100 (MET) Date: Tue, 7 Dec 2004 23:25:22 +0100 From: Lennert Buytenhek To: Robert Olsson Cc: hadi@cyberus.ca, netdev@oss.sgi.com Subject: inter-packet gap in pktgen Message-ID: <20041207222522.GA30266@xi.wantstofly.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.1i X-archive-position: 12545 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: buytenh@wantstofly.org Precedence: bulk X-list: netdev Hi Robert, For TX/RX tests, I've been trying to convince pktgen to spit out exactly N packets per second (for various values of N) by tweaking the inter-packet gap parameter, and I've noticed the following. The 'ipg' parameter to pktgen seems to be neither (i) the actual on-the-wire inter-packet gap (which is the time between the FCS of one packet and the preamble of the next), nor (ii) the time between the first data bit of one packet and that of the next. From observing that the pps rate for a given 'ipg' does not depend on the packet size, it would seem that 'ipg' attempts to be (ii). But it doesn't quite succeed in that -- it appears to be always ~496ns off on my box. For example, if I send 500B packets with param 'ipg' equal to 10000ns, I would expect to end up either with 71kpps(i) or with 100kpps(ii). But what I get is 95kpps, 1e9/(10000+496). At 5000ns ipg, I get neither 109kpps(i) nor 200kpps(ii) but 182kpps, 1e9/(5000+496). Presumably this 496ns is the CPU cost of shoving one packet towards the NIC, and pktgen only after sending a packet starts waiting for 'ipg' ns before transmitting the next packet. Can we not compensate for this cost so that we either always get (i) or (ii)? Possibly by first getting the current time X, then transmitting the packet, and then waiting until X+ipg, which would then give us (ii). (We'd have to rename it to 'inter-packet-start gap' though, or something like that.) By tweaking the 'ipg' parameter I can generate pretty much any packet rate I want, as long as I set ipg=(1e9/rate)-496 instead of something possibly more straightforward. thanks, Lennert From Robert.Olsson@data.slu.se Tue Dec 7 14:40:53 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 14:40:57 -0800 (PST) Received: from mail1.slu.se (mail1.slu.se [130.238.96.11]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7Meqmc011819 for ; Tue, 7 Dec 2004 14:40:53 -0800 Received: from robur.slu.se (robur.slu.se [130.238.98.12]) by mail1.slu.se (8.12.10/8.12.10) with ESMTP id iB7MeMKO017845; Tue, 7 Dec 2004 23:40:22 +0100 Received: by robur.slu.se (Postfix, from userid 1000) id 4BA96EC001; Tue, 7 Dec 2004 23:40:22 +0100 (CET) From: Robert Olsson MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <16822.12630.275389.575326@robur.slu.se> Date: Tue, 7 Dec 2004 23:40:22 +0100 To: Karsten Desler Cc: netdev@oss.sgi.com, linux-kernel@vger.kernel.org, "David S. Miller" , jamal , Robert Olsson , P@draigBrady.com Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets In-Reply-To: <20041207211035.GA20286@quickstop.soohrt.org> References: <20041206205305.GA11970@soohrt.org> <20041207211035.GA20286@quickstop.soohrt.org> X-Mailer: VM 7.18 under Emacs 21.3.1 X-archive-position: 12546 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: Robert.Olsson@data.slu.se Precedence: bulk X-list: netdev Karsten Desler writes: > I totally forgot to mention: There are approximately 100k concurrent > flows. > >From dmesg: > IP: routing cache hash table of 16384 buckets, 128Kbytes You can take a looks at stats w. rtstat. Hash spinning and how many new entires create and how many warm you hit. > Maybe there is some contention on the rt_hash_table spinlocks? > Is the attached patch enough to increase the size? There is boot option for this now rhash_entries= [KNL,NET] Set number of hash buckets for route cache --ro From greearb@candelatech.com Tue Dec 7 14:47:42 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 14:47:47 -0800 (PST) Received: from www.lanforge.com (ns1.lanforge.com [66.165.47.210]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7MlgdJ012261 for ; Tue, 7 Dec 2004 14:47:42 -0800 Received: from [4.33.45.22] (evrtwa1-ar2-4-33-045-022.evrtwa1.dsl-verizon.net [4.33.45.22]) (authenticated bits=0) by www.lanforge.com (8.12.8/8.12.8) with ESMTP id iB7MxZLH005248; Tue, 7 Dec 2004 14:59:36 -0800 Message-ID: <41B632F3.1090104@candelatech.com> Date: Tue, 07 Dec 2004 14:47:15 -0800 From: Ben Greear Organization: Candela Technologies User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.7.3) Gecko/20041020 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Lennert Buytenhek CC: Robert Olsson , hadi@cyberus.ca, netdev@oss.sgi.com Subject: Re: inter-packet gap in pktgen References: <20041207222522.GA30266@xi.wantstofly.org> In-Reply-To: <20041207222522.GA30266@xi.wantstofly.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 12547 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: greearb@candelatech.com Precedence: bulk X-list: netdev Lennert Buytenhek wrote: > By tweaking the 'ipg' parameter I can generate pretty much any packet rate > I want, as long as I set ipg=(1e9/rate)-496 instead of something possibly > more straightforward. That 496 will also change with load on the system, at least on average. I dealt with this by having a user-space app sample the rate and adjust the ipg to keep the average rate where I want it. So, I'd suggest leaving the ipg as it is, and use external tools to get the exact pps that you are looking for. Ben -- Ben Greear Candela Technologies Inc http://www.candelatech.com From sven@zion.homelinux.com Tue Dec 7 15:14:16 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 15:14:21 -0800 (PST) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.184]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB7NEFmC013367 for ; Tue, 7 Dec 2004 15:14:16 -0800 Received: from [212.227.126.155] (helo=mrelayng.kundenserver.de) by moutng.kundenserver.de with esmtp (Exim 3.35 #1) id 1CboWb-0003u4-00; Wed, 08 Dec 2004 00:13:41 +0100 Received: from [80.136.68.240] (helo=zion.homelinux.com) by mrelayng.kundenserver.de with asmtp (Exim 3.35 #1) id 1CboWa-0004tU-00; Wed, 08 Dec 2004 00:13:41 +0100 Received: from localhost (zion.homelinux.com [127.0.0.1]) by stage2.zion.homelinux.com (Postfix) with ESMTP id CC2812CAF6; Wed, 8 Dec 2004 00:13:38 +0100 (CET) Received: from zion.homelinux.com ([127.0.0.1]) by localhost (zion [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 27557-05; Wed, 8 Dec 2004 00:13:36 +0100 (CET) Received: by zion.homelinux.com (Postfix, from userid 1022) id 557522D179; Wed, 8 Dec 2004 00:13:36 +0100 (CET) Date: Wed, 8 Dec 2004 00:13:36 +0100 From: Sven Schuster To: shemminger@osdl.org Cc: netdev@oss.sgi.com Subject: [PATCH][iproute2] make lnstat default count '1' Message-ID: <20041207231336.GA29839@zion.homelinux.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="s2ZSL+KKDSLx8OML" Content-Disposition: inline User-Agent: Mutt/1.5.6i X-Virus-Scanned: by amavisd-new-2.2.0 (20041102) at zion.homelinux.com X-Provags-ID: kundenserver.de abuse@kundenserver.de auth:38b5f051b8cd178556c5593940405c4a X-archive-position: 12548 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: schuster.sven@gmx.de Precedence: bulk X-list: netdev --s2ZSL+KKDSLx8OML Content-Type: multipart/mixed; boundary="X1bOJ3K7DJ5YkBrT" Content-Disposition: inline --X1bOJ3K7DJ5YkBrT Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, I just compiled the latest iproute2 (from the bk repository) and tested Harald's lnstat utility. It found it quite annoying that when I just called rtstat it displayed nothing. The reason is that without the parameter '-c 1' the default count for viewing the information is 0. Wouldn't 1 be a better default value?? If yes, tiny patch attached :-) Sven --=20 Linux zion 2.6.10-rc3 #1 Mon Dec 6 22:51:51 CET 2004 i686 athlon i386 GNU/L= inux 00:08:17 up 1 day, 2 min, 2 users, load average: 0.00, 0.03, 0.05 --X1bOJ3K7DJ5YkBrT Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: attachment; filename="iproute2-count.patch" Content-Transfer-Encoding: quoted-printable --- iproute2-20041207/misc/lnstat.c.orig 2004-12-08 00:04:21.960353599 +0100 +++ iproute2-20041207/misc/lnstat.c 2004-12-08 00:02:32.425889013 +0100 @@ -218,7 +218,7 @@ MODE_NORMAL, } mode =3D MODE_NORMAL; =20 - unsigned long count =3D 0; + unsigned long count =3D 1; static struct field_params fp; int num_req_files =3D 0; char *req_files[LNSTAT_MAX_FILES]; --X1bOJ3K7DJ5YkBrT-- --s2ZSL+KKDSLx8OML Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (GNU/Linux) iD8DBQFBtjkgo4FAdB2PneQRAuNzAJ9XW0u3dyn+1TtytHlef8TmKAHX0wCdFYBy gB2MV8RVbfayglONP01p7MU= =/jMn -----END PGP SIGNATURE----- --s2ZSL+KKDSLx8OML-- From sri@us.ibm.com Tue Dec 7 16:58:30 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 16:58:40 -0800 (PST) Received: from e5.ny.us.ibm.com (e5.ny.us.ibm.com [32.97.182.145]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB80wMTs019825 for ; Tue, 7 Dec 2004 16:58:29 -0800 Received: from d01relay02.pok.ibm.com (d01relay02.pok.ibm.com [9.56.227.234]) by e5.ny.us.ibm.com (8.12.10/8.12.10) with ESMTP id iB80voGW027690 for ; Tue, 7 Dec 2004 19:57:50 -0500 Received: from d01av02.pok.ibm.com (d01av02.pok.ibm.com [9.56.224.216]) by d01relay02.pok.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id iB80voT2094676 for ; Tue, 7 Dec 2004 19:57:50 -0500 Received: from d01av02.pok.ibm.com (loopback [127.0.0.1]) by d01av02.pok.ibm.com (8.12.11/8.12.11) with ESMTP id iB80voqq025920 for ; Tue, 7 Dec 2004 19:57:50 -0500 Received: from w-sridhar.beaverton.ibm.com (w-sridhar.beaverton.ibm.com [9.47.18.19]) by d01av02.pok.ibm.com (8.12.11/8.12.11) with ESMTP id iB80vnxB025879; Tue, 7 Dec 2004 19:57:49 -0500 Date: Tue, 7 Dec 2004 16:57:48 -0800 (PST) From: Sridhar Samudrala X-X-Sender: sridhar@w-sridhar.beaverton.ibm.com To: Adrian Bunk cc: lksctp-developers@lists.sourceforge.net, linux-kernel@vger.kernel.org, netdev@oss.sgi.com Subject: Re: [2.6 patch] SCTP: possible cleanups In-Reply-To: <20041125172412.GG3537@stusta.de> Message-ID: References: <20041125172412.GG3537@stusta.de> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 12549 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: sri@us.ibm.com Precedence: bulk X-list: netdev Adrian, The patch looks fine. But instead of marking some functions directly as static i would like to use SCTP_STATIC macro which is defined as static within kernel. This allows us to export certain internal static functions to our user-level sctp test framework that uses the kernel implementation. I will submit this patch with the above change as part of next SCTP update. Thanks Sridhar On Thu, 25 Nov 2004, Adrian Bunk wrote: > The patch below contains the following possible cleanups for the SCTP > code: > - remove unused code > - make needlessly global code static > > This patch is not intended for being blindly applies, please review and > check whether part of it might conflict with pending patches. > > > diffstat output: > include/net/sctp/command.h | 13 ----- > include/net/sctp/constants.h | 4 - > include/net/sctp/sctp.h | 10 ---- > include/net/sctp/sm.h | 61 -------------------------- > include/net/sctp/structs.h | 22 --------- > include/net/sctp/tsnmap.h | 16 ------ > include/net/sctp/ulpevent.h | 2 > include/net/sctp/ulpqueue.h | 1 > net/sctp/associola.c | 72 +++++++++++-------------------- > net/sctp/bind_addr.c | 17 ------- > net/sctp/chunk.c | 8 +-- > net/sctp/command.c | 23 --------- > net/sctp/debug.c | 17 ------- > net/sctp/endpointola.c | 54 +++++++++++------------ > net/sctp/input.c | 46 ++++++++++--------- > net/sctp/inqueue.c | 13 ----- > net/sctp/ipv6.c | 20 ++++---- > net/sctp/objcnt.c | 2 > net/sctp/outqueue.c | 15 ------ > net/sctp/proc.c | 2 > net/sctp/protocol.c | 34 +++++++------- > net/sctp/sm_make_chunk.c | 81 ++++++++++------------------------- > net/sctp/sm_sideeffect.c | 66 +++++++++++++++++++--------- > net/sctp/sm_statefuns.c | 81 +++++++++++++++++++++++------------ > net/sctp/sm_statetable.c | 27 ++++++++--- > net/sctp/socket.c | 4 - > net/sctp/ssnmap.c | 7 ++- > net/sctp/transport.c | 56 ++++++++++++------------ > net/sctp/tsnmap.c | 39 ++-------------- > net/sctp/ulpevent.c | 18 ++++--- > net/sctp/ulpqueue.c | 21 --------- > 31 files changed, 306 insertions(+), 546 deletions(-) > > > Signed-off-by: Adrian Bunk > > --- linux-2.6.10-rc2-mm3-full/include/net/sctp/structs.h.old 2004-11-25 00:33:15.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/include/net/sctp/structs.h 2004-11-25 03:29:32.000000000 +0100 > @@ -406,7 +406,6 @@ > int malloced; > }; > > -struct sctp_ssnmap *sctp_ssnmap_init(struct sctp_ssnmap *, __u16, __u16); > struct sctp_ssnmap *sctp_ssnmap_new(__u16 in, __u16 out, int gfp); > void sctp_ssnmap_free(struct sctp_ssnmap *map); > void sctp_ssnmap_clear(struct sctp_ssnmap *map); > @@ -538,12 +537,9 @@ > struct sctp_datamsg *sctp_datamsg_from_user(struct sctp_association *, > struct sctp_sndrcvinfo *, > struct msghdr *, int len); > -struct sctp_datamsg *sctp_datamsg_new(int gfp); > void sctp_datamsg_put(struct sctp_datamsg *); > -void sctp_datamsg_hold(struct sctp_datamsg *); > void sctp_datamsg_free(struct sctp_datamsg *); > void sctp_datamsg_track(struct sctp_chunk *); > -void sctp_datamsg_assign(struct sctp_datamsg *, struct sctp_chunk *); > void sctp_chunk_fail(struct sctp_chunk *, int error); > int sctp_chunk_abandoned(struct sctp_chunk *); > > @@ -651,8 +647,6 @@ > void sctp_chunk_put(struct sctp_chunk *); > int sctp_user_addto_chunk(struct sctp_chunk *chunk, int off, int len, > struct iovec *data); > -struct sctp_chunk *sctp_make_chunk(const struct sctp_association *, __u8 type, > - __u8 flags, int size); > void sctp_chunk_free(struct sctp_chunk *); > void *sctp_addto_chunk(struct sctp_chunk *, int len, const void *data); > struct sctp_chunk *sctp_chunkify(struct sk_buff *, > @@ -922,15 +916,12 @@ > }; > > struct sctp_transport *sctp_transport_new(const union sctp_addr *, int); > -struct sctp_transport *sctp_transport_init(struct sctp_transport *, > - const union sctp_addr *, int); > void sctp_transport_set_owner(struct sctp_transport *, > struct sctp_association *); > void sctp_transport_route(struct sctp_transport *, union sctp_addr *, > struct sctp_opt *); > void sctp_transport_pmtu(struct sctp_transport *); > void sctp_transport_free(struct sctp_transport *); > -void sctp_transport_destroy(struct sctp_transport *); > void sctp_transport_reset_timers(struct sctp_transport *); > void sctp_transport_hold(struct sctp_transport *); > void sctp_transport_put(struct sctp_transport *); > @@ -961,7 +952,6 @@ > int malloced; /* Is this structure kfree()able? */ > }; > > -struct sctp_inq *sctp_inq_new(void); > void sctp_inq_init(struct sctp_inq *); > void sctp_inq_free(struct sctp_inq *); > void sctp_inq_push(struct sctp_inq *, struct sctp_chunk *packet); > @@ -1029,7 +1019,6 @@ > char malloced; > }; > > -struct sctp_outq *sctp_outq_new(struct sctp_association *); > void sctp_outq_init(struct sctp_association *, struct sctp_outq *); > void sctp_outq_teardown(struct sctp_outq *); > void sctp_outq_free(struct sctp_outq*); > @@ -1070,7 +1059,6 @@ > int malloced; /* Are we kfree()able? */ > }; > > -struct sctp_bind_addr *sctp_bind_addr_new(int gfp_mask); > void sctp_bind_addr_init(struct sctp_bind_addr *, __u16 port); > void sctp_bind_addr_free(struct sctp_bind_addr *); > int sctp_bind_addr_copy(struct sctp_bind_addr *dest, > @@ -1220,8 +1208,6 @@ > > /* These are function signatures for manipulating endpoints. */ > struct sctp_endpoint *sctp_endpoint_new(struct sock *, int); > -struct sctp_endpoint *sctp_endpoint_init(struct sctp_endpoint *, > - struct sock *, int gfp); > void sctp_endpoint_free(struct sctp_endpoint *); > void sctp_endpoint_put(struct sctp_endpoint *); > void sctp_endpoint_hold(struct sctp_endpoint *); > @@ -1243,8 +1229,6 @@ > int sctp_process_init(struct sctp_association *, sctp_cid_t cid, > const union sctp_addr *peer, > sctp_init_chunk_t *init, int gfp); > -int sctp_process_param(struct sctp_association *, union sctp_params param, > - const union sctp_addr *from, int gfp); > __u32 sctp_generate_tag(const struct sctp_endpoint *); > __u32 sctp_generate_tsn(const struct sctp_endpoint *); > > @@ -1690,10 +1674,6 @@ > struct sctp_association * > sctp_association_new(const struct sctp_endpoint *, const struct sock *, > sctp_scope_t scope, int gfp); > -struct sctp_association * > -sctp_association_init(struct sctp_association *, const struct sctp_endpoint *, > - const struct sock *, sctp_scope_t scope, > - int gfp); > void sctp_association_free(struct sctp_association *); > void sctp_association_put(struct sctp_association *); > void sctp_association_hold(struct sctp_association *); > @@ -1722,7 +1702,6 @@ > struct sctp_association *new); > > __u32 sctp_association_get_next_tsn(struct sctp_association *); > -__u32 sctp_association_get_tsn_block(struct sctp_association *, int); > > void sctp_assoc_sync_pmtu(struct sctp_association *); > void sctp_assoc_rwnd_increase(struct sctp_association *, unsigned); > @@ -1736,7 +1715,6 @@ > int sctp_cmp_addr_exact(const union sctp_addr *ss1, > const union sctp_addr *ss2); > struct sctp_chunk *sctp_get_ecne_prepend(struct sctp_association *asoc); > -struct sctp_chunk *sctp_get_no_prepend(struct sctp_association *asoc); > > /* A convenience structure to parse out SCTP specific CMSGs. */ > typedef struct sctp_cmsgs { > --- linux-2.6.10-rc2-mm3-full/include/net/sctp/command.h.old 2004-11-25 00:37:53.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/include/net/sctp/command.h 2004-11-25 00:38:13.000000000 +0100 > @@ -189,11 +189,6 @@ > } sctp_cmd_seq_t; > > > -/* Create a new sctp_command_sequence. > - * Return NULL if creating a new sequence fails. > - */ > -sctp_cmd_seq_t *sctp_new_cmd_seq(int gfp); > - > /* Initialize a block of memory as a command sequence. > * Return 0 if the initialization fails. > */ > @@ -207,18 +202,10 @@ > */ > int sctp_add_cmd(sctp_cmd_seq_t *seq, sctp_verb_t verb, sctp_arg_t obj); > > -/* Rewind an sctp_cmd_seq_t to iterate from the start. > - * Return 0 if the rewind fails. > - */ > -int sctp_rewind_sequence(sctp_cmd_seq_t *seq); > - > /* Return the next command structure in an sctp_cmd_seq. > * Return NULL at the end of the sequence. > */ > sctp_cmd_t *sctp_next_cmd(sctp_cmd_seq_t *seq); > > -/* Dispose of a command sequence. */ > -void sctp_free_cmd_seq(sctp_cmd_seq_t *seq); > - > #endif /* __net_sctp_command_h__ */ > > --- linux-2.6.10-rc2-mm3-full/include/net/sctp/constants.h.old 2004-11-25 00:39:10.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/include/net/sctp/constants.h 2004-11-25 00:39:20.000000000 +0100 > @@ -155,10 +155,6 @@ > - (unsigned long)(c->chunk_hdr)\ > - sizeof(sctp_data_chunk_t))) > > -/* This is a table of printable names of sctp_param_t's. */ > -extern const char *sctp_param_tbl[]; > - > - > #define SCTP_MAX_ERROR_CAUSE SCTP_ERROR_NONEXIST_IP > #define SCTP_NUM_ERROR_CAUSE 10 > > --- linux-2.6.10-rc2-mm3-full/include/net/sctp/sctp.h.old 2004-11-25 00:41:46.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/include/net/sctp/sctp.h 2004-11-25 01:34:35.000000000 +0100 > @@ -162,17 +162,9 @@ > int sctp_rcv(struct sk_buff *skb); > void sctp_v4_err(struct sk_buff *skb, u32 info); > void sctp_hash_established(struct sctp_association *); > -void __sctp_hash_established(struct sctp_association *); > void sctp_unhash_established(struct sctp_association *); > -void __sctp_unhash_established(struct sctp_association *); > void sctp_hash_endpoint(struct sctp_endpoint *); > -void __sctp_hash_endpoint(struct sctp_endpoint *); > void sctp_unhash_endpoint(struct sctp_endpoint *); > -void __sctp_unhash_endpoint(struct sctp_endpoint *); > -struct sctp_association *__sctp_lookup_association( > - const union sctp_addr *, > - const union sctp_addr *, > - struct sctp_transport **); > struct sock *sctp_err_lookup(int family, struct sk_buff *, > struct sctphdr *, struct sctp_endpoint **, > struct sctp_association **, > @@ -310,8 +302,6 @@ > > int sctp_v6_init(void); > void sctp_v6_exit(void); > -void sctp_v6_err(struct sk_buff *skb, struct inet6_skb_parm *opt, > - int type, int code, int offset, __u32 info); > > #else /* #ifdef defined(CONFIG_IPV6) */ > > --- linux-2.6.10-rc2-mm3-full/include/net/sctp/sm.h.old 2004-11-25 01:42:52.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/include/net/sctp/sm.h 2004-11-25 03:14:05.000000000 +0100 > @@ -128,7 +128,6 @@ > sctp_state_fn_t sctp_sf_do_ecn_cwr; > sctp_state_fn_t sctp_sf_do_ecne; > sctp_state_fn_t sctp_sf_ootb; > -sctp_state_fn_t sctp_sf_shut_8_4_5; > sctp_state_fn_t sctp_sf_pdiscard; > sctp_state_fn_t sctp_sf_violation; > sctp_state_fn_t sctp_sf_discard_chunk; > @@ -138,7 +137,6 @@ > sctp_state_fn_t sctp_sf_unk_chunk; > sctp_state_fn_t sctp_sf_do_8_5_1_E_sa; > sctp_state_fn_t sctp_sf_cookie_echoed_err; > -sctp_state_fn_t sctp_sf_do_5_2_6_stale; > sctp_state_fn_t sctp_sf_do_asconf; > sctp_state_fn_t sctp_sf_do_asconf_ack; > sctp_state_fn_t sctp_sf_do_9_2_reshutack; > @@ -200,19 +198,10 @@ > struct sctp_chunk *sctp_make_cwr(const struct sctp_association *, > const __u32 lowest_tsn, > const struct sctp_chunk *); > -struct sctp_chunk *sctp_make_datafrag(struct sctp_association *, > - const struct sctp_sndrcvinfo *sinfo, > - int len, const __u8 *data, > - __u8 flags, __u16 ssn); > struct sctp_chunk * sctp_make_datafrag_empty(struct sctp_association *, > const struct sctp_sndrcvinfo *sinfo, > int len, const __u8 flags, > __u16 ssn); > -struct sctp_chunk *sctp_make_data(struct sctp_association *, > - const struct sctp_sndrcvinfo *sinfo, > - int len, const __u8 *data); > -struct sctp_chunk *sctp_make_data_empty(struct sctp_association *, > - const struct sctp_sndrcvinfo *, int len); > struct sctp_chunk *sctp_make_ecne(const struct sctp_association *, > const __u32); > struct sctp_chunk *sctp_make_sack(const struct sctp_association *); > @@ -246,17 +235,12 @@ > const void *payload, > size_t paylen); > > -struct sctp_chunk *sctp_make_asconf(struct sctp_association *asoc, > - union sctp_addr *addr, > - int vparam_len); > struct sctp_chunk *sctp_make_asconf_update_ip(struct sctp_association *, > union sctp_addr *, > struct sockaddr *, > int, __u16); > struct sctp_chunk *sctp_make_asconf_set_prim(struct sctp_association *asoc, > union sctp_addr *addr); > -struct sctp_chunk *sctp_make_asconf_ack(const struct sctp_association *asoc, > - __u32 serial, int vparam_len); > struct sctp_chunk *sctp_process_asconf(struct sctp_association *asoc, > struct sctp_chunk *asconf); > int sctp_process_asconf_ack(struct sctp_association *asoc, > @@ -277,71 +261,26 @@ > void *event_arg, > int gfp); > > -int sctp_side_effects(sctp_event_t event_type, sctp_subtype_t subtype, > - sctp_state_t state, > - struct sctp_endpoint *, > - struct sctp_association *asoc, > - void *event_arg, > - sctp_disposition_t status, > - sctp_cmd_seq_t *commands, > - int gfp); > - > /* 2nd level prototypes */ > -int sctp_cmd_interpreter(sctp_event_t, sctp_subtype_t, sctp_state_t, > - struct sctp_endpoint *, struct sctp_association *, > - void *event_arg, sctp_disposition_t, > - sctp_cmd_seq_t *retval, int gfp); > - > - > -int sctp_gen_sack(struct sctp_association *, int force, sctp_cmd_seq_t *); > void sctp_generate_t3_rtx_event(unsigned long peer); > void sctp_generate_heartbeat_event(unsigned long peer); > > -sctp_sackhdr_t *sctp_sm_pull_sack(struct sctp_chunk *); > -struct sctp_packet *sctp_abort_pkt_new(const struct sctp_endpoint *, > - const struct sctp_association *, > - struct sctp_chunk *chunk, > - const void *payload, > - size_t paylen); > -struct sctp_packet *sctp_ootb_pkt_new(const struct sctp_association *, > - const struct sctp_chunk *); > void sctp_ootb_pkt_free(struct sctp_packet *); > > -struct sctp_cookie_param * > -sctp_pack_cookie(const struct sctp_endpoint *, const struct sctp_association *, > - const struct sctp_chunk *, int *cookie_len, > - const __u8 *, int addrs_len); > struct sctp_association *sctp_unpack_cookie(const struct sctp_endpoint *, > const struct sctp_association *, > struct sctp_chunk *, int gfp, int *err, > struct sctp_chunk **err_chk_p); > int sctp_addip_addr_config(struct sctp_association *, sctp_param_t, > struct sockaddr_storage*, int); > -void sctp_send_stale_cookie_err(const struct sctp_endpoint *ep, > - const struct sctp_association *asoc, > - const struct sctp_chunk *chunk, > - sctp_cmd_seq_t *commands, > - struct sctp_chunk *err_chunk); > -int sctp_eat_data(const struct sctp_association *asoc, > - struct sctp_chunk *chunk, > - sctp_cmd_seq_t *commands); > > /* 3rd level prototypes */ > __u32 sctp_generate_tag(const struct sctp_endpoint *); > __u32 sctp_generate_tsn(const struct sctp_endpoint *); > > /* Extern declarations for major data structures. */ > -const sctp_sm_table_entry_t *sctp_chunk_event_lookup(sctp_cid_t, sctp_state_t); > -extern const sctp_sm_table_entry_t > -primitive_event_table[SCTP_NUM_PRIMITIVE_TYPES][SCTP_STATE_NUM_STATES]; > -extern const sctp_sm_table_entry_t > -other_event_table[SCTP_NUM_OTHER_TYPES][SCTP_STATE_NUM_STATES]; > -extern const sctp_sm_table_entry_t > -timeout_event_table[SCTP_NUM_TIMEOUT_TYPES][SCTP_STATE_NUM_STATES]; > extern sctp_timer_event_t *sctp_timer_events[SCTP_NUM_TIMEOUT_TYPES]; > > -/* These are some handy utility macros... */ > - > > /* Get the size of a DATA chunk payload. */ > static inline __u16 sctp_data_size(struct sctp_chunk *chunk) > --- linux-2.6.10-rc2-mm3-full/net/sctp/associola.c.old 2004-11-25 00:33:40.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/net/sctp/associola.c 2004-11-25 00:34:52.000000000 +0100 > @@ -66,33 +66,8 @@ > > /* 1st Level Abstractions. */ > > -/* Allocate and initialize a new association */ > -struct sctp_association *sctp_association_new(const struct sctp_endpoint *ep, > - const struct sock *sk, > - sctp_scope_t scope, int gfp) > -{ > - struct sctp_association *asoc; > - > - asoc = t_new(struct sctp_association, gfp); > - if (!asoc) > - goto fail; > - > - if (!sctp_association_init(asoc, ep, sk, scope, gfp)) > - goto fail_init; > - > - asoc->base.malloced = 1; > - SCTP_DBG_OBJCNT_INC(assoc); > - > - return asoc; > - > -fail_init: > - kfree(asoc); > -fail: > - return NULL; > -} > - > /* Initialize a new association from provided memory. */ > -struct sctp_association *sctp_association_init(struct sctp_association *asoc, > +static struct sctp_association *sctp_association_init(struct sctp_association *asoc, > const struct sctp_endpoint *ep, > const struct sock *sk, > sctp_scope_t scope, > @@ -296,6 +271,31 @@ > return NULL; > } > > +/* Allocate and initialize a new association */ > +struct sctp_association *sctp_association_new(const struct sctp_endpoint *ep, > + const struct sock *sk, > + sctp_scope_t scope, int gfp) > +{ > + struct sctp_association *asoc; > + > + asoc = t_new(struct sctp_association, gfp); > + if (!asoc) > + goto fail; > + > + if (!sctp_association_init(asoc, ep, sk, scope, gfp)) > + goto fail_init; > + > + asoc->base.malloced = 1; > + SCTP_DBG_OBJCNT_INC(assoc); > + > + return asoc; > + > +fail_init: > + kfree(asoc); > +fail: > + return NULL; > +} > + > /* Free this association if possible. There may still be users, so > * the actual deallocation may be delayed. > */ > @@ -714,18 +714,6 @@ > return retval; > } > > -/* Allocate 'num' TSNs by incrementing the association's TSN by num. */ > -__u32 sctp_association_get_tsn_block(struct sctp_association *asoc, int num) > -{ > - __u32 retval = asoc->next_tsn; > - > - asoc->next_tsn += num; > - asoc->unack_data += num; > - > - return retval; > -} > - > - > /* Compare two addresses to see if they match. Wildcard addresses > * only match themselves. > */ > @@ -760,14 +748,6 @@ > return chunk; > } > > -/* Use this function for the packet prepend callback when no ECNE > - * packet is desired (e.g. some packets don't like to be bundled). > - */ > -struct sctp_chunk *sctp_get_no_prepend(struct sctp_association *asoc) > -{ > - return NULL; > -} > - > /* > * Find which transport this TSN was sent on. > */ > --- linux-2.6.10-rc2-mm3-full/net/sctp/bind_addr.c.old 2004-11-25 00:35:13.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/net/sctp/bind_addr.c 2004-11-25 00:35:23.000000000 +0100 > @@ -104,23 +104,6 @@ > return error; > } > > -/* Create a new SCTP_bind_addr from nothing. */ > -struct sctp_bind_addr *sctp_bind_addr_new(int gfp) > -{ > - struct sctp_bind_addr *retval; > - > - retval = t_new(struct sctp_bind_addr, gfp); > - if (!retval) > - goto nomem; > - > - sctp_bind_addr_init(retval, 0); > - retval->malloced = 1; > - SCTP_DBG_OBJCNT_INC(bind_addr); > - > -nomem: > - return retval; > -} > - > /* Initialize the SCTP_bind_addr structure for either an endpoint or > * an association. > */ > --- linux-2.6.10-rc2-mm3-full/net/sctp/chunk.c.old 2004-11-25 00:36:15.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/net/sctp/chunk.c 2004-11-25 00:36:55.000000000 +0100 > @@ -51,7 +51,7 @@ > */ > > /* Initialize datamsg from memory. */ > -void sctp_datamsg_init(struct sctp_datamsg *msg) > +static void sctp_datamsg_init(struct sctp_datamsg *msg) > { > atomic_set(&msg->refcnt, 1); > msg->send_failed = 0; > @@ -62,7 +62,7 @@ > } > > /* Allocate and initialize datamsg. */ > -struct sctp_datamsg *sctp_datamsg_new(int gfp) > +static struct sctp_datamsg *sctp_datamsg_new(int gfp) > { > struct sctp_datamsg *msg; > msg = kmalloc(sizeof(struct sctp_datamsg), gfp); > @@ -124,7 +124,7 @@ > } > > /* Hold a reference. */ > -void sctp_datamsg_hold(struct sctp_datamsg *msg) > +static void sctp_datamsg_hold(struct sctp_datamsg *msg) > { > atomic_inc(&msg->refcnt); > } > @@ -151,7 +151,7 @@ > } > > /* Assign a chunk to this datamsg. */ > -void sctp_datamsg_assign(struct sctp_datamsg *msg, struct sctp_chunk *chunk) > +static void sctp_datamsg_assign(struct sctp_datamsg *msg, struct sctp_chunk *chunk) > { > sctp_datamsg_hold(msg); > chunk->msg = msg; > --- linux-2.6.10-rc2-mm3-full/net/sctp/command.c.old 2004-11-25 00:38:22.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/net/sctp/command.c 2004-11-25 00:38:55.000000000 +0100 > @@ -42,17 +42,6 @@ > #include > #include > > -/* Create a new sctp_command_sequence. */ > -sctp_cmd_seq_t *sctp_new_cmd_seq(int gfp) > -{ > - sctp_cmd_seq_t *retval = t_new(sctp_cmd_seq_t, gfp); > - > - if (retval) > - sctp_init_cmd_seq(retval); > - > - return retval; > -} > - > /* Initialize a block of memory as a command sequence. */ > int sctp_init_cmd_seq(sctp_cmd_seq_t *seq) > { > @@ -77,13 +66,6 @@ > return 0; > } > > -/* Rewind an sctp_cmd_seq_t to iterate from the start. */ > -int sctp_rewind_sequence(sctp_cmd_seq_t *seq) > -{ > - seq->next_cmd = 0; > - return 1; /* We always succeed. */ > -} > - > /* Return the next command structure in a sctp_cmd_seq. > * Returns NULL at the end of the sequence. > */ > @@ -97,8 +79,3 @@ > return retval; > } > > -/* Dispose of a command sequence. */ > -void sctp_free_cmd_seq(sctp_cmd_seq_t *seq) > -{ > - kfree(seq); > -} > --- linux-2.6.10-rc2-mm3-full/net/sctp/debug.c.old 2004-11-25 00:39:29.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/net/sctp/debug.c 2004-11-25 00:39:39.000000000 +0100 > @@ -98,23 +98,6 @@ > return "unknown chunk"; > } > > -/* These are printable form of variable-length parameters. */ > -const char *sctp_param_tbl[SCTP_PARAM_ECN_CAPABLE + 1] = { > - "", > - "PARAM_HEARTBEAT_INFO", > - "", > - "", > - "", > - "PARAM_IPV4_ADDRESS", > - "PARAM_IPV6_ADDRESS", > - "PARAM_STATE_COOKIE", > - "PARAM_UNRECOGNIZED_PARAMETERS", > - "PARAM_COOKIE_PRESERVATIVE", > - "", > - "PARAM_HOST_NAME_ADDRESS", > - "PARAM_SUPPORTED_ADDRESS_TYPES", > -}; > - > /* These are printable forms of the states. */ > const char *sctp_state_tbl[SCTP_STATE_NUM_STATES] = { > "STATE_EMPTY", > --- linux-2.6.10-rc2-mm3-full/net/sctp/endpointola.c.old 2004-11-25 00:40:01.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/net/sctp/endpointola.c 2004-11-25 00:41:19.000000000 +0100 > @@ -63,34 +63,11 @@ > /* Forward declarations for internal helpers. */ > static void sctp_endpoint_bh_rcv(struct sctp_endpoint *ep); > > -/* Create a sctp_endpoint with all that boring stuff initialized. > - * Returns NULL if there isn't enough memory. > - */ > -struct sctp_endpoint *sctp_endpoint_new(struct sock *sk, int gfp) > -{ > - struct sctp_endpoint *ep; > - > - /* Build a local endpoint. */ > - ep = t_new(struct sctp_endpoint, gfp); > - if (!ep) > - goto fail; > - if (!sctp_endpoint_init(ep, sk, gfp)) > - goto fail_init; > - ep->base.malloced = 1; > - SCTP_DBG_OBJCNT_INC(ep); > - return ep; > - > -fail_init: > - kfree(ep); > -fail: > - return NULL; > -} > - > /* > * Initialize the base fields of the endpoint structure. > */ > -struct sctp_endpoint *sctp_endpoint_init(struct sctp_endpoint *ep, > - struct sock *sk, int gfp) > +static struct sctp_endpoint *sctp_endpoint_init(struct sctp_endpoint *ep, > + struct sock *sk, int gfp) > { > struct sctp_opt *sp = sctp_sk(sk); > memset(ep, 0, sizeof(struct sctp_endpoint)); > @@ -160,6 +137,29 @@ > return ep; > } > > +/* Create a sctp_endpoint with all that boring stuff initialized. > + * Returns NULL if there isn't enough memory. > + */ > +struct sctp_endpoint *sctp_endpoint_new(struct sock *sk, int gfp) > +{ > + struct sctp_endpoint *ep; > + > + /* Build a local endpoint. */ > + ep = t_new(struct sctp_endpoint, gfp); > + if (!ep) > + goto fail; > + if (!sctp_endpoint_init(ep, sk, gfp)) > + goto fail_init; > + ep->base.malloced = 1; > + SCTP_DBG_OBJCNT_INC(ep); > + return ep; > + > +fail_init: > + kfree(ep); > +fail: > + return NULL; > +} > + > /* Add an association to an endpoint. */ > void sctp_endpoint_add_asoc(struct sctp_endpoint *ep, > struct sctp_association *asoc) > @@ -184,7 +184,7 @@ > } > > /* Final destructor for endpoint. */ > -void sctp_endpoint_destroy(struct sctp_endpoint *ep) > +static void sctp_endpoint_destroy(struct sctp_endpoint *ep) > { > SCTP_ASSERT(ep->base.dead, "Endpoint is not dead", return); > > @@ -257,7 +257,7 @@ > * We do a linear search of the associations for this endpoint. > * We return the matching transport address too. > */ > -struct sctp_association *__sctp_endpoint_lookup_assoc( > +static struct sctp_association *__sctp_endpoint_lookup_assoc( > const struct sctp_endpoint *ep, > const union sctp_addr *paddr, > struct sctp_transport **transport) > --- linux-2.6.10-rc2-mm3-full/net/sctp/input.c.old 2004-11-25 00:42:03.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/net/sctp/input.c 2004-11-25 00:46:22.000000000 +0100 > @@ -63,11 +63,15 @@ > > /* Forward declarations for internal helpers. */ > static int sctp_rcv_ootb(struct sk_buff *); > -struct sctp_association *__sctp_rcv_lookup(struct sk_buff *skb, > +static struct sctp_association *__sctp_rcv_lookup(struct sk_buff *skb, > const union sctp_addr *laddr, > const union sctp_addr *paddr, > struct sctp_transport **transportp); > -struct sctp_endpoint *__sctp_rcv_lookup_endpoint(const union sctp_addr *laddr); > +static struct sctp_endpoint *__sctp_rcv_lookup_endpoint(const union sctp_addr *laddr); > +static struct sctp_association *__sctp_lookup_association( > + const union sctp_addr *local, > + const union sctp_addr *peer, > + struct sctp_transport **pt); > > > /* Calculate the SCTP checksum of an SCTP packet. */ > @@ -522,7 +526,7 @@ > } > > /* Insert endpoint into the hash table. */ > -void __sctp_hash_endpoint(struct sctp_endpoint *ep) > +static void __sctp_hash_endpoint(struct sctp_endpoint *ep) > { > struct sctp_ep_common **epp; > struct sctp_ep_common *epb; > @@ -552,7 +556,7 @@ > } > > /* Remove endpoint from the hash table. */ > -void __sctp_unhash_endpoint(struct sctp_endpoint *ep) > +static void __sctp_unhash_endpoint(struct sctp_endpoint *ep) > { > struct sctp_hashbucket *head; > struct sctp_ep_common *epb; > @@ -584,7 +588,7 @@ > } > > /* Look up an endpoint. */ > -struct sctp_endpoint *__sctp_rcv_lookup_endpoint(const union sctp_addr *laddr) > +static struct sctp_endpoint *__sctp_rcv_lookup_endpoint(const union sctp_addr *laddr) > { > struct sctp_hashbucket *head; > struct sctp_ep_common *epb; > @@ -610,16 +614,8 @@ > return ep; > } > > -/* Add an association to the hash. Local BH-safe. */ > -void sctp_hash_established(struct sctp_association *asoc) > -{ > - sctp_local_bh_disable(); > - __sctp_hash_established(asoc); > - sctp_local_bh_enable(); > -} > - > /* Insert association into the hash table. */ > -void __sctp_hash_established(struct sctp_association *asoc) > +static void __sctp_hash_established(struct sctp_association *asoc) > { > struct sctp_ep_common **epp; > struct sctp_ep_common *epb; > @@ -642,16 +638,16 @@ > sctp_write_unlock(&head->lock); > } > > -/* Remove association from the hash table. Local BH-safe. */ > -void sctp_unhash_established(struct sctp_association *asoc) > +/* Add an association to the hash. Local BH-safe. */ > +void sctp_hash_established(struct sctp_association *asoc) > { > sctp_local_bh_disable(); > - __sctp_unhash_established(asoc); > + __sctp_hash_established(asoc); > sctp_local_bh_enable(); > } > > /* Remove association from the hash table. */ > -void __sctp_unhash_established(struct sctp_association *asoc) > +static void __sctp_unhash_established(struct sctp_association *asoc) > { > struct sctp_hashbucket *head; > struct sctp_ep_common *epb; > @@ -675,8 +671,16 @@ > sctp_write_unlock(&head->lock); > } > > +/* Remove association from the hash table. Local BH-safe. */ > +void sctp_unhash_established(struct sctp_association *asoc) > +{ > + sctp_local_bh_disable(); > + __sctp_unhash_established(asoc); > + sctp_local_bh_enable(); > +} > + > /* Look up an association. */ > -struct sctp_association *__sctp_lookup_association( > +static struct sctp_association *__sctp_lookup_association( > const union sctp_addr *local, > const union sctp_addr *peer, > struct sctp_transport **pt) > @@ -713,7 +717,7 @@ > } > > /* Look up an association. BH-safe. */ > -struct sctp_association *sctp_lookup_association(const union sctp_addr *laddr, > +static struct sctp_association *sctp_lookup_association(const union sctp_addr *laddr, > const union sctp_addr *paddr, > struct sctp_transport **transportp) > { > @@ -821,7 +825,7 @@ > } > > /* Lookup an association for an inbound skb. */ > -struct sctp_association *__sctp_rcv_lookup(struct sk_buff *skb, > +static struct sctp_association *__sctp_rcv_lookup(struct sk_buff *skb, > const union sctp_addr *paddr, > const union sctp_addr *laddr, > struct sctp_transport **transportp) > --- linux-2.6.10-rc2-mm3-full/net/sctp/ipv6.c.old 2004-11-25 01:34:44.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/net/sctp/ipv6.c 2004-11-25 01:35:57.000000000 +0100 > @@ -84,8 +84,8 @@ > }; > > /* ICMP error handler. */ > -void sctp_v6_err(struct sk_buff *skb, struct inet6_skb_parm *opt, > - int type, int code, int offset, __u32 info) > +static void sctp_v6_err(struct sk_buff *skb, struct inet6_skb_parm *opt, > + int type, int code, int offset, __u32 info) > { > struct inet6_dev *idev; > struct ipv6hdr *iph = (struct ipv6hdr *)skb->data; > @@ -188,9 +188,9 @@ > /* Returns the dst cache entry for the given source and destination ip > * addresses. > */ > -struct dst_entry *sctp_v6_get_dst(struct sctp_association *asoc, > - union sctp_addr *daddr, > - union sctp_addr *saddr) > +static struct dst_entry *sctp_v6_get_dst(struct sctp_association *asoc, > + union sctp_addr *daddr, > + union sctp_addr *saddr) > { > struct dst_entry *dst; > struct flowi fl; > @@ -251,8 +251,10 @@ > /* Fills in the source address(saddr) based on the destination address(daddr) > * and asoc's bind address list. > */ > -void sctp_v6_get_saddr(struct sctp_association *asoc, struct dst_entry *dst, > - union sctp_addr *daddr, union sctp_addr *saddr) > +static void sctp_v6_get_saddr(struct sctp_association *asoc, > + struct dst_entry *dst, > + union sctp_addr *daddr, > + union sctp_addr *saddr) > { > struct sctp_bind_addr *bp; > rwlock_t *addr_lock; > @@ -577,8 +579,8 @@ > } > > /* Create and initialize a new sk for the socket to be returned by accept(). */ > -struct sock *sctp_v6_create_accept_sk(struct sock *sk, > - struct sctp_association *asoc) > +static struct sock *sctp_v6_create_accept_sk(struct sock *sk, > + struct sctp_association *asoc) > { > struct inet_opt *inet = inet_sk(sk); > struct sock *newsk; > --- linux-2.6.10-rc2-mm3-full/net/sctp/inqueue.c.old 2004-11-25 00:46:50.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/net/sctp/inqueue.c 2004-11-25 00:46:57.000000000 +0100 > @@ -59,19 +59,6 @@ > queue->malloced = 0; > } > > -/* Create an initialized sctp_inq. */ > -struct sctp_inq *sctp_inq_new(void) > -{ > - struct sctp_inq *retval; > - > - retval = t_new(struct sctp_inq, GFP_ATOMIC); > - if (retval) { > - sctp_inq_init(retval); > - retval->malloced = 1; > - } > - return retval; > -} > - > /* Release the memory associated with an SCTP inqueue. */ > void sctp_inq_free(struct sctp_inq *queue) > { > --- linux-2.6.10-rc2-mm3-full/net/sctp/objcnt.c.old 2004-11-25 01:36:46.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/net/sctp/objcnt.c 2004-11-25 01:36:56.000000000 +0100 > @@ -62,7 +62,7 @@ > /* An array to make it easy to pretty print the debug information > * to the proc fs. > */ > -sctp_dbg_objcnt_entry_t sctp_dbg_objcnt[] = { > +static sctp_dbg_objcnt_entry_t sctp_dbg_objcnt[] = { > SCTP_DBG_OBJCNT_ENTRY(sock), > SCTP_DBG_OBJCNT_ENTRY(ep), > SCTP_DBG_OBJCNT_ENTRY(assoc), > --- linux-2.6.10-rc2-mm3-full/net/sctp/outqueue.c.old 2004-11-25 01:37:31.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/net/sctp/outqueue.c 2004-11-25 01:37:47.000000000 +0100 > @@ -190,19 +190,6 @@ > return 0; > } > > -/* Generate a new outqueue. */ > -struct sctp_outq *sctp_outq_new(struct sctp_association *asoc) > -{ > - struct sctp_outq *q; > - > - q = t_new(struct sctp_outq, GFP_KERNEL); > - if (q) { > - sctp_outq_init(asoc, q); > - q->malloced = 1; > - } > - return q; > -} > - > /* Initialize an existing sctp_outq. This does the boring stuff. > * You still need to define handlers if you really want to DO > * something with this structure... > @@ -362,7 +349,7 @@ > /* Insert a chunk into the sorted list based on the TSNs. The retransmit list > * and the abandoned list are in ascending order. > */ > -void sctp_insert_list(struct list_head *head, struct list_head *new) > +static void sctp_insert_list(struct list_head *head, struct list_head *new) > { > struct list_head *pos; > struct sctp_chunk *nchunk, *lchunk; > --- linux-2.6.10-rc2-mm3-full/net/sctp/proc.c.old 2004-11-25 01:38:01.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/net/sctp/proc.c 2004-11-25 01:38:10.000000000 +0100 > @@ -39,7 +39,7 @@ > #include > #include > > -struct snmp_mib sctp_snmp_list[] = { > +static struct snmp_mib sctp_snmp_list[] = { > SNMP_MIB_ITEM("SctpCurrEstab", SCTP_MIB_CURRESTAB), > SNMP_MIB_ITEM("SctpActiveEstabs", SCTP_MIB_ACTIVEESTABS), > SNMP_MIB_ITEM("SctpPassiveEstabs", SCTP_MIB_PASSIVEESTABS), > --- linux-2.6.10-rc2-mm3-full/net/sctp/protocol.c.old 2004-11-25 01:39:00.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/net/sctp/protocol.c 2004-11-25 01:41:45.000000000 +0100 > @@ -95,7 +95,7 @@ > } > > /* Set up the proc fs entry for the SCTP protocol. */ > -__init int sctp_proc_init(void) > +static __init int sctp_proc_init(void) > { > if (!proc_net_sctp) { > struct proc_dir_entry *ent; > @@ -124,7 +124,7 @@ > * Note: Do not make this __exit as it is used in the init error > * path. > */ > -void sctp_proc_exit(void) > +static void sctp_proc_exit(void) > { > sctp_snmp_proc_exit(); > sctp_eps_proc_exit(); > @@ -428,9 +428,9 @@ > * addresses. If an association is passed, trys to get a dst entry with a > * source address that matches an address in the bind address list. > */ > -struct dst_entry *sctp_v4_get_dst(struct sctp_association *asoc, > - union sctp_addr *daddr, > - union sctp_addr *saddr) > +static struct dst_entry *sctp_v4_get_dst(struct sctp_association *asoc, > + union sctp_addr *daddr, > + union sctp_addr *saddr) > { > struct rtable *rt; > struct flowi fl; > @@ -520,10 +520,10 @@ > /* For v4, the source address is cached in the route entry(dst). So no need > * to cache it separately and hence this is an empty routine. > */ > -void sctp_v4_get_saddr(struct sctp_association *asoc, > - struct dst_entry *dst, > - union sctp_addr *daddr, > - union sctp_addr *saddr) > +static void sctp_v4_get_saddr(struct sctp_association *asoc, > + struct dst_entry *dst, > + union sctp_addr *daddr, > + union sctp_addr *saddr) > { > struct rtable *rt = (struct rtable *)dst; > > @@ -547,8 +547,8 @@ > } > > /* Create and initialize a new sk for the socket returned by accept(). */ > -struct sock *sctp_v4_create_accept_sk(struct sock *sk, > - struct sctp_association *asoc) > +static struct sock *sctp_v4_create_accept_sk(struct sock *sk, > + struct sctp_association *asoc) > { > struct sock *newsk; > struct inet_opt *inet = inet_sk(sk); > @@ -639,7 +639,7 @@ > * Initialize the control inode/socket with a control endpoint data > * structure. This endpoint is reserved exclusively for the OOTB processing. > */ > -int sctp_ctl_sock_init(void) > +static int sctp_ctl_sock_init(void) > { > int err; > sa_family_t family; > @@ -808,7 +808,7 @@ > return ip_queue_xmit(skb, ipfragok); > } > > -struct sctp_af sctp_ipv4_specific; > +static struct sctp_af sctp_ipv4_specific; > > static struct sctp_pf sctp_pf_inet = { > .event_msgname = sctp_inet_event_msgname, > @@ -829,7 +829,7 @@ > }; > > /* Socket operations. */ > -struct proto_ops inet_seqpacket_ops = { > +static struct proto_ops inet_seqpacket_ops = { > .family = PF_INET, > .owner = THIS_MODULE, > .release = inet_release, /* Needs to be wrapped... */ > @@ -878,7 +878,7 @@ > }; > > /* IPv4 address related functions. */ > -struct sctp_af sctp_ipv4_specific = { > +static struct sctp_af sctp_ipv4_specific = { > .sctp_xmit = sctp_v4_xmit, > .setsockopt = ip_setsockopt, > .getsockopt = ip_getsockopt, > @@ -959,7 +959,7 @@ > } > > /* Initialize the universe into something sensible. */ > -__init int sctp_init(void) > +static __init int sctp_init(void) > { > int i; > int status = -EINVAL; > @@ -1196,7 +1196,7 @@ > } > > /* Exit handler for the SCTP protocol. */ > -__exit void sctp_exit(void) > +static __exit void sctp_exit(void) > { > /* BUG. This should probably do something useful like clean > * up all the remaining associations and all that memory. > --- linux-2.6.10-rc2-mm3-full/net/sctp/sm_make_chunk.c.old 2004-11-25 01:43:40.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/net/sctp/sm_make_chunk.c 2004-11-25 01:51:15.000000000 +0100 > @@ -67,6 +67,18 @@ > > extern kmem_cache_t *sctp_chunk_cachep; > > +static struct sctp_chunk *sctp_make_chunk(const struct sctp_association *asoc, > + __u8 type, __u8 flags, int paylen); > +static sctp_cookie_param_t *sctp_pack_cookie(const struct sctp_endpoint *ep, > + const struct sctp_association *asoc, > + const struct sctp_chunk *init_chunk, > + int *cookie_len, > + const __u8 *raw_addrs, int addrs_len); > +static int sctp_process_param(struct sctp_association *asoc, > + union sctp_params param, > + const union sctp_addr *peer_addr, > + int gfp); > + > /* What was the inbound interface for this chunk? */ > int sctp_chunk_iif(const struct sctp_chunk *chunk) > { > @@ -559,52 +571,6 @@ > return retval; > } > > -/* Make a DATA chunk for the given association. Populate the data > - * payload. > - */ > -struct sctp_chunk *sctp_make_datafrag(struct sctp_association *asoc, > - const struct sctp_sndrcvinfo *sinfo, > - int data_len, const __u8 *data, > - __u8 flags, __u16 ssn) > -{ > - struct sctp_chunk *retval; > - > - retval = sctp_make_datafrag_empty(asoc, sinfo, data_len, flags, ssn); > - if (retval) > - sctp_addto_chunk(retval, data_len, data); > - > - return retval; > -} > - > -/* Make a DATA chunk for the given association to ride on stream id > - * 'stream', with a payload id of 'payload', and a body of 'data'. > - */ > -struct sctp_chunk *sctp_make_data(struct sctp_association *asoc, > - const struct sctp_sndrcvinfo *sinfo, > - int data_len, const __u8 *data) > -{ > - struct sctp_chunk *retval = NULL; > - > - retval = sctp_make_data_empty(asoc, sinfo, data_len); > - if (retval) > - sctp_addto_chunk(retval, data_len, data); > - return retval; > -} > - > -/* Make a DATA chunk for the given association to ride on stream id > - * 'stream', with a payload id of 'payload', and a body big enough to > - * hold 'data_len' octets of data. We use this version when we need > - * to build the message AFTER allocating memory. > - */ > -struct sctp_chunk *sctp_make_data_empty(struct sctp_association *asoc, > - const struct sctp_sndrcvinfo *sinfo, > - int data_len) > -{ > - __u8 flags = SCTP_DATA_NOT_FRAG; > - > - return sctp_make_datafrag_empty(asoc, sinfo, data_len, flags, 0); > -} > - > /* Create a selective ackowledgement (SACK) for the given > * association. This reports on which TSN's we've seen to date, > * including duplicates and gaps. > @@ -933,7 +899,7 @@ > /* Create an Operation Error chunk with the specified space reserved. > * This routine can be used for containing multiple causes in the chunk. > */ > -struct sctp_chunk *sctp_make_op_error_space( > +static struct sctp_chunk *sctp_make_op_error_space( > const struct sctp_association *asoc, > const struct sctp_chunk *chunk, > size_t size) > @@ -1062,8 +1028,8 @@ > /* Create a new chunk, setting the type and flags headers from the > * arguments, reserving enough space for a 'paylen' byte payload. > */ > -struct sctp_chunk *sctp_make_chunk(const struct sctp_association *asoc, > - __u8 type, __u8 flags, int paylen) > +static struct sctp_chunk *sctp_make_chunk(const struct sctp_association *asoc, > + __u8 type, __u8 flags, int paylen) > { > struct sctp_chunk *retval; > sctp_chunkhdr_t *chunk_hdr; > @@ -1261,7 +1227,7 @@ > /* Build a cookie representing asoc. > * This INCLUDES the param header needed to put the cookie in the INIT ACK. > */ > -sctp_cookie_param_t *sctp_pack_cookie(const struct sctp_endpoint *ep, > +static sctp_cookie_param_t *sctp_pack_cookie(const struct sctp_endpoint *ep, > const struct sctp_association *asoc, > const struct sctp_chunk *init_chunk, > int *cookie_len, > @@ -1912,8 +1878,10 @@ > * work we do. In particular, we should not build transport > * structures for the addresses. > */ > -int sctp_process_param(struct sctp_association *asoc, union sctp_params param, > - const union sctp_addr *peer_addr, int gfp) > +static int sctp_process_param(struct sctp_association *asoc, > + union sctp_params param, > + const union sctp_addr *peer_addr, > + int gfp) > { > union sctp_addr addr; > int i; > @@ -2078,8 +2046,9 @@ > * > * Address Parameter and other parameter will not be wrapped in this function > */ > -struct sctp_chunk *sctp_make_asconf(struct sctp_association *asoc, > - union sctp_addr *addr, int vparam_len) > +static struct sctp_chunk *sctp_make_asconf(struct sctp_association *asoc, > + union sctp_addr *addr, > + int vparam_len) > { > sctp_addiphdr_t asconf; > struct sctp_chunk *retval; > @@ -2248,8 +2217,8 @@ > * > * Create an ASCONF_ACK chunk with enough space for the parameter responses. > */ > -struct sctp_chunk *sctp_make_asconf_ack(const struct sctp_association *asoc, > - __u32 serial, int vparam_len) > +static struct sctp_chunk *sctp_make_asconf_ack(const struct sctp_association *asoc, > + __u32 serial, int vparam_len) > { > sctp_addiphdr_t asconf; > struct sctp_chunk *retval; > --- linux-2.6.10-rc2-mm3-full/net/sctp/sm_sideeffect.c.old 2004-11-25 01:52:16.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/net/sctp/sm_sideeffect.c 2004-11-25 03:24:28.000000000 +0100 > @@ -55,6 +55,24 @@ > #include > #include > > +static int sctp_cmd_interpreter(sctp_event_t event_type, > + sctp_subtype_t subtype, > + sctp_state_t state, > + struct sctp_endpoint *ep, > + struct sctp_association *asoc, > + void *event_arg, > + sctp_disposition_t status, > + sctp_cmd_seq_t *commands, > + int gfp); > +static int sctp_side_effects(sctp_event_t event_type, sctp_subtype_t subtype, > + sctp_state_t state, > + struct sctp_endpoint *ep, > + struct sctp_association *asoc, > + void *event_arg, > + sctp_disposition_t status, > + sctp_cmd_seq_t *commands, > + int gfp); > + > /******************************************************************** > * Helper functions > ********************************************************************/ > @@ -134,8 +152,8 @@ > } > > /* Generate SACK if necessary. We call this at the end of a packet. */ > -int sctp_gen_sack(struct sctp_association *asoc, int force, > - sctp_cmd_seq_t *commands) > +static int sctp_gen_sack(struct sctp_association *asoc, int force, > + sctp_cmd_seq_t *commands) > { > __u32 ctsn, max_tsn_seen; > struct sctp_chunk *sack; > @@ -276,31 +294,31 @@ > sctp_association_put(asoc); > } > > -void sctp_generate_t1_cookie_event(unsigned long data) > +static void sctp_generate_t1_cookie_event(unsigned long data) > { > struct sctp_association *asoc = (struct sctp_association *) data; > sctp_generate_timeout_event(asoc, SCTP_EVENT_TIMEOUT_T1_COOKIE); > } > > -void sctp_generate_t1_init_event(unsigned long data) > +static void sctp_generate_t1_init_event(unsigned long data) > { > struct sctp_association *asoc = (struct sctp_association *) data; > sctp_generate_timeout_event(asoc, SCTP_EVENT_TIMEOUT_T1_INIT); > } > > -void sctp_generate_t2_shutdown_event(unsigned long data) > +static void sctp_generate_t2_shutdown_event(unsigned long data) > { > struct sctp_association *asoc = (struct sctp_association *) data; > sctp_generate_timeout_event(asoc, SCTP_EVENT_TIMEOUT_T2_SHUTDOWN); > } > > -void sctp_generate_t4_rto_event(unsigned long data) > +static void sctp_generate_t4_rto_event(unsigned long data) > { > struct sctp_association *asoc = (struct sctp_association *) data; > sctp_generate_timeout_event(asoc, SCTP_EVENT_TIMEOUT_T4_RTO); > } > > -void sctp_generate_t5_shutdown_guard_event(unsigned long data) > +static void sctp_generate_t5_shutdown_guard_event(unsigned long data) > { > struct sctp_association *asoc = (struct sctp_association *)data; > sctp_generate_timeout_event(asoc, > @@ -308,7 +326,7 @@ > > } /* sctp_generate_t5_shutdown_guard_event() */ > > -void sctp_generate_autoclose_event(unsigned long data) > +static void sctp_generate_autoclose_event(unsigned long data) > { > struct sctp_association *asoc = (struct sctp_association *) data; > sctp_generate_timeout_event(asoc, SCTP_EVENT_TIMEOUT_AUTOCLOSE); > @@ -353,7 +371,7 @@ > } > > /* Inject a SACK Timeout event into the state machine. */ > -void sctp_generate_sack_event(unsigned long data) > +static void sctp_generate_sack_event(unsigned long data) > { > struct sctp_association *asoc = (struct sctp_association *) data; > sctp_generate_timeout_event(asoc, SCTP_EVENT_TIMEOUT_SACK); > @@ -857,14 +875,14 @@ > /***************************************************************** > * This the master state function side effect processing function. > *****************************************************************/ > -int sctp_side_effects(sctp_event_t event_type, sctp_subtype_t subtype, > - sctp_state_t state, > - struct sctp_endpoint *ep, > - struct sctp_association *asoc, > - void *event_arg, > - sctp_disposition_t status, > - sctp_cmd_seq_t *commands, > - int gfp) > +static int sctp_side_effects(sctp_event_t event_type, sctp_subtype_t subtype, > + sctp_state_t state, > + struct sctp_endpoint *ep, > + struct sctp_association *asoc, > + void *event_arg, > + sctp_disposition_t status, > + sctp_cmd_seq_t *commands, > + int gfp) > { > int error; > > @@ -944,11 +962,15 @@ > ********************************************************************/ > > /* This is the side-effect interpreter. */ > -int sctp_cmd_interpreter(sctp_event_t event_type, sctp_subtype_t subtype, > - sctp_state_t state, struct sctp_endpoint *ep, > - struct sctp_association *asoc, void *event_arg, > - sctp_disposition_t status, sctp_cmd_seq_t *commands, > - int gfp) > +static int sctp_cmd_interpreter(sctp_event_t event_type, > + sctp_subtype_t subtype, > + sctp_state_t state, > + struct sctp_endpoint *ep, > + struct sctp_association *asoc, > + void *event_arg, > + sctp_disposition_t status, > + sctp_cmd_seq_t *commands, > + int gfp) > { > int error = 0; > int force; > --- linux-2.6.10-rc2-mm3-full/net/sctp/sm_statefuns.c.old 2004-11-25 01:56:27.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/net/sctp/sm_statefuns.c 2004-11-25 02:14:47.000000000 +0100 > @@ -65,6 +65,33 @@ > #include > #include > > +static struct sctp_packet *sctp_abort_pkt_new(const struct sctp_endpoint *ep, > + const struct sctp_association *asoc, > + struct sctp_chunk *chunk, > + const void *payload, > + size_t paylen); > +static int sctp_eat_data(const struct sctp_association *asoc, > + struct sctp_chunk *chunk, > + sctp_cmd_seq_t *commands); > +static struct sctp_packet *sctp_ootb_pkt_new(const struct sctp_association *asoc, > + const struct sctp_chunk *chunk); > +static void sctp_send_stale_cookie_err(const struct sctp_endpoint *ep, > + const struct sctp_association *asoc, > + const struct sctp_chunk *chunk, > + sctp_cmd_seq_t *commands, > + struct sctp_chunk *err_chunk); > +static sctp_disposition_t sctp_sf_do_5_2_6_stale(const struct sctp_endpoint *ep, > + const struct sctp_association *asoc, > + const sctp_subtype_t type, > + void *arg, > + sctp_cmd_seq_t *commands); > +static sctp_disposition_t sctp_sf_shut_8_4_5(const struct sctp_endpoint *ep, > + const struct sctp_association *asoc, > + const sctp_subtype_t type, > + void *arg, > + sctp_cmd_seq_t *commands); > +static struct sctp_sackhdr *sctp_sm_pull_sack(struct sctp_chunk *chunk); > + > /********************************************************** > * These are the state functions for handling chunk events. > **********************************************************/ > @@ -748,11 +775,11 @@ > } > > /* Generate and sendout a heartbeat packet. */ > -sctp_disposition_t sctp_sf_heartbeat(const struct sctp_endpoint *ep, > - const struct sctp_association *asoc, > - const sctp_subtype_t type, > - void *arg, > - sctp_cmd_seq_t *commands) > +static sctp_disposition_t sctp_sf_heartbeat(const struct sctp_endpoint *ep, > + const struct sctp_association *asoc, > + const sctp_subtype_t type, > + void *arg, > + sctp_cmd_seq_t *commands) > { > struct sctp_transport *transport = (struct sctp_transport *) arg; > struct sctp_chunk *reply; > @@ -1928,11 +1955,11 @@ > * > * The return value is the disposition of the chunk. > */ > -sctp_disposition_t sctp_sf_do_5_2_6_stale(const struct sctp_endpoint *ep, > - const struct sctp_association *asoc, > - const sctp_subtype_t type, > - void *arg, > - sctp_cmd_seq_t *commands) > +static sctp_disposition_t sctp_sf_do_5_2_6_stale(const struct sctp_endpoint *ep, > + const struct sctp_association *asoc, > + const sctp_subtype_t type, > + void *arg, > + sctp_cmd_seq_t *commands) > { > struct sctp_chunk *chunk = arg; > time_t stale; > @@ -2853,11 +2880,11 @@ > * > * The return value is the disposition of the chunk. > */ > -sctp_disposition_t sctp_sf_shut_8_4_5(const struct sctp_endpoint *ep, > - const struct sctp_association *asoc, > - const sctp_subtype_t type, > - void *arg, > - sctp_cmd_seq_t *commands) > +static sctp_disposition_t sctp_sf_shut_8_4_5(const struct sctp_endpoint *ep, > + const struct sctp_association *asoc, > + const sctp_subtype_t type, > + void *arg, > + sctp_cmd_seq_t *commands) > { > struct sctp_packet *packet = NULL; > struct sctp_chunk *chunk = arg; > @@ -4537,7 +4564,7 @@ > ********************************************************************/ > > /* Pull the SACK chunk based on the SACK header. */ > -struct sctp_sackhdr *sctp_sm_pull_sack(struct sctp_chunk *chunk) > +static struct sctp_sackhdr *sctp_sm_pull_sack(struct sctp_chunk *chunk) > { > struct sctp_sackhdr *sack; > unsigned int len; > @@ -4564,7 +4591,7 @@ > /* Create an ABORT packet to be sent as a response, with the specified > * error causes. > */ > -struct sctp_packet *sctp_abort_pkt_new(const struct sctp_endpoint *ep, > +static struct sctp_packet *sctp_abort_pkt_new(const struct sctp_endpoint *ep, > const struct sctp_association *asoc, > struct sctp_chunk *chunk, > const void *payload, > @@ -4600,8 +4627,8 @@ > } > > /* Allocate a packet for responding in the OOTB conditions. */ > -struct sctp_packet *sctp_ootb_pkt_new(const struct sctp_association *asoc, > - const struct sctp_chunk *chunk) > +static struct sctp_packet *sctp_ootb_pkt_new(const struct sctp_association *asoc, > + const struct sctp_chunk *chunk) > { > struct sctp_packet *packet; > struct sctp_transport *transport; > @@ -4664,11 +4691,11 @@ > } > > /* Send a stale cookie error when a invalid COOKIE ECHO chunk is found */ > -void sctp_send_stale_cookie_err(const struct sctp_endpoint *ep, > - const struct sctp_association *asoc, > - const struct sctp_chunk *chunk, > - sctp_cmd_seq_t *commands, > - struct sctp_chunk *err_chunk) > +static void sctp_send_stale_cookie_err(const struct sctp_endpoint *ep, > + const struct sctp_association *asoc, > + const struct sctp_chunk *chunk, > + sctp_cmd_seq_t *commands, > + struct sctp_chunk *err_chunk) > { > struct sctp_packet *packet; > > @@ -4694,9 +4721,9 @@ > > > /* Process a data chunk */ > -int sctp_eat_data(const struct sctp_association *asoc, > - struct sctp_chunk *chunk, > - sctp_cmd_seq_t *commands) > +static int sctp_eat_data(const struct sctp_association *asoc, > + struct sctp_chunk *chunk, > + sctp_cmd_seq_t *commands) > { > sctp_datahdr_t *data_hdr; > struct sctp_chunk *err; > --- linux-2.6.10-rc2-mm3-full/net/sctp/sm_statetable.c.old 2004-11-25 02:15:50.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/net/sctp/sm_statetable.c 2004-11-25 03:23:43.000000000 +0100 > @@ -50,6 +50,17 @@ > #include > #include > > +static const sctp_sm_table_entry_t > +primitive_event_table[SCTP_NUM_PRIMITIVE_TYPES][SCTP_STATE_NUM_STATES]; > +static const sctp_sm_table_entry_t > +other_event_table[SCTP_NUM_OTHER_TYPES][SCTP_STATE_NUM_STATES]; > +static const sctp_sm_table_entry_t > +timeout_event_table[SCTP_NUM_TIMEOUT_TYPES][SCTP_STATE_NUM_STATES]; > + > +static const sctp_sm_table_entry_t *sctp_chunk_event_lookup(sctp_cid_t cid, > + sctp_state_t state); > + > + > static const sctp_sm_table_entry_t bug = { > .fn = sctp_sf_bug, > .name = "sctp_sf_bug" > @@ -419,7 +430,7 @@ > * > * For base protocol (RFC 2960). > */ > -const sctp_sm_table_entry_t chunk_event_table[SCTP_NUM_BASE_CHUNK_TYPES][SCTP_STATE_NUM_STATES] = { > +static const sctp_sm_table_entry_t chunk_event_table[SCTP_NUM_BASE_CHUNK_TYPES][SCTP_STATE_NUM_STATES] = { > TYPE_SCTP_DATA, > TYPE_SCTP_INIT, > TYPE_SCTP_INIT_ACK, > @@ -482,7 +493,7 @@ > /* The primary index for this table is the chunk type. > * The secondary index for this table is the state. > */ > -const sctp_sm_table_entry_t addip_chunk_event_table[SCTP_NUM_ADDIP_CHUNK_TYPES][SCTP_STATE_NUM_STATES] = { > +static const sctp_sm_table_entry_t addip_chunk_event_table[SCTP_NUM_ADDIP_CHUNK_TYPES][SCTP_STATE_NUM_STATES] = { > TYPE_SCTP_ASCONF, > TYPE_SCTP_ASCONF_ACK, > }; /*state_fn_t addip_chunk_event_table[][] */ > @@ -511,7 +522,7 @@ > /* The primary index for this table is the chunk type. > * The secondary index for this table is the state. > */ > -const sctp_sm_table_entry_t prsctp_chunk_event_table[SCTP_NUM_PRSCTP_CHUNK_TYPES][SCTP_STATE_NUM_STATES] = { > +static const sctp_sm_table_entry_t prsctp_chunk_event_table[SCTP_NUM_PRSCTP_CHUNK_TYPES][SCTP_STATE_NUM_STATES] = { > TYPE_SCTP_FWD_TSN, > }; /*state_fn_t prsctp_chunk_event_table[][] */ > > @@ -684,7 +695,7 @@ > /* The primary index for this table is the primitive type. > * The secondary index for this table is the state. > */ > -const sctp_sm_table_entry_t primitive_event_table[SCTP_NUM_PRIMITIVE_TYPES][SCTP_STATE_NUM_STATES] = { > +static const sctp_sm_table_entry_t primitive_event_table[SCTP_NUM_PRIMITIVE_TYPES][SCTP_STATE_NUM_STATES] = { > TYPE_SCTP_PRIMITIVE_ASSOCIATE, > TYPE_SCTP_PRIMITIVE_SHUTDOWN, > TYPE_SCTP_PRIMITIVE_ABORT, > @@ -716,7 +727,7 @@ > {.fn = sctp_sf_ignore_other, .name = "sctp_sf_ignore_other"}, \ > } > > -const sctp_sm_table_entry_t other_event_table[SCTP_NUM_OTHER_TYPES][SCTP_STATE_NUM_STATES] = { > +static const sctp_sm_table_entry_t other_event_table[SCTP_NUM_OTHER_TYPES][SCTP_STATE_NUM_STATES] = { > TYPE_SCTP_OTHER_NO_PENDING_TSN, > }; > > @@ -931,7 +942,7 @@ > {.fn = sctp_sf_timer_ignore, .name = "sctp_sf_timer_ignore"}, \ > } > > -const sctp_sm_table_entry_t timeout_event_table[SCTP_NUM_TIMEOUT_TYPES][SCTP_STATE_NUM_STATES] = { > +static const sctp_sm_table_entry_t timeout_event_table[SCTP_NUM_TIMEOUT_TYPES][SCTP_STATE_NUM_STATES] = { > TYPE_SCTP_EVENT_TIMEOUT_NONE, > TYPE_SCTP_EVENT_TIMEOUT_T1_COOKIE, > TYPE_SCTP_EVENT_TIMEOUT_T1_INIT, > @@ -944,8 +955,8 @@ > TYPE_SCTP_EVENT_TIMEOUT_AUTOCLOSE, > }; > > -const sctp_sm_table_entry_t *sctp_chunk_event_lookup(sctp_cid_t cid, > - sctp_state_t state) > +static const sctp_sm_table_entry_t *sctp_chunk_event_lookup(sctp_cid_t cid, > + sctp_state_t state) > { > if (state > SCTP_STATE_MAX) > return &bug; > --- linux-2.6.10-rc2-mm3-full/net/sctp/socket.c.old 2004-11-25 02:19:43.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/net/sctp/socket.c 2004-11-25 02:20:25.000000000 +0100 > @@ -208,7 +208,7 @@ > * id are specified, the associations matching the address and the id should be > * the same. > */ > -struct sctp_transport *sctp_addr_id2transport(struct sock *sk, > +static struct sctp_transport *sctp_addr_id2transport(struct sock *sk, > struct sockaddr_storage *addr, > sctp_assoc_t id) > { > @@ -245,7 +245,7 @@ > * sockaddr_in6 [RFC 2553]), > * addr_len - the size of the address structure. > */ > -int sctp_bind(struct sock *sk, struct sockaddr *uaddr, int addr_len) > +static int sctp_bind(struct sock *sk, struct sockaddr *uaddr, int addr_len) > { > int retval = 0; > > --- linux-2.6.10-rc2-mm3-full/net/sctp/ssnmap.c.old 2004-11-25 02:20:54.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/net/sctp/ssnmap.c 2004-11-25 03:29:59.000000000 +0100 > @@ -42,6 +42,9 @@ > > #define MAX_KMALLOC_SIZE 131072 > > +static struct sctp_ssnmap *sctp_ssnmap_init(struct sctp_ssnmap *map, __u16 in, > + __u16 out); > + > /* Storage size needed for map includes 2 headers and then the > * specific needs of in or out streams. > */ > @@ -87,8 +90,8 @@ > > > /* Initialize a block of memory as a ssnmap. */ > -struct sctp_ssnmap *sctp_ssnmap_init(struct sctp_ssnmap *map, __u16 in, > - __u16 out) > +static struct sctp_ssnmap *sctp_ssnmap_init(struct sctp_ssnmap *map, __u16 in, > + __u16 out) > { > memset(map, 0x00, sctp_ssnmap_size(in, out)); > > --- linux-2.6.10-rc2-mm3-full/net/sctp/transport.c.old 2004-11-25 02:22:16.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/net/sctp/transport.c 2004-11-25 02:23:01.000000000 +0100 > @@ -54,34 +54,10 @@ > > /* 1st Level Abstractions. */ > > -/* Allocate and initialize a new transport. */ > -struct sctp_transport *sctp_transport_new(const union sctp_addr *addr, int gfp) > -{ > - struct sctp_transport *transport; > - > - transport = t_new(struct sctp_transport, gfp); > - if (!transport) > - goto fail; > - > - if (!sctp_transport_init(transport, addr, gfp)) > - goto fail_init; > - > - transport->malloced = 1; > - SCTP_DBG_OBJCNT_INC(transport); > - > - return transport; > - > -fail_init: > - kfree(transport); > - > -fail: > - return NULL; > -} > - > /* Initialize a new transport from provided memory. */ > -struct sctp_transport *sctp_transport_init(struct sctp_transport *peer, > - const union sctp_addr *addr, > - int gfp) > +static struct sctp_transport *sctp_transport_init(struct sctp_transport *peer, > + const union sctp_addr *addr, > + int gfp) > { > /* Copy in the address. */ > peer->ipaddr = *addr; > @@ -144,6 +120,30 @@ > return peer; > } > > +/* Allocate and initialize a new transport. */ > +struct sctp_transport *sctp_transport_new(const union sctp_addr *addr, int gfp) > +{ > + struct sctp_transport *transport; > + > + transport = t_new(struct sctp_transport, gfp); > + if (!transport) > + goto fail; > + > + if (!sctp_transport_init(transport, addr, gfp)) > + goto fail_init; > + > + transport->malloced = 1; > + SCTP_DBG_OBJCNT_INC(transport); > + > + return transport; > + > +fail_init: > + kfree(transport); > + > +fail: > + return NULL; > +} > + > /* This transport is no longer needed. Free up if possible, or > * delay until it last reference count. > */ > @@ -161,7 +161,7 @@ > /* Destroy the transport data structure. > * Assumes there are no more users of this structure. > */ > -void sctp_transport_destroy(struct sctp_transport *transport) > +static void sctp_transport_destroy(struct sctp_transport *transport) > { > SCTP_ASSERT(transport->dead, "Transport is not dead", return); > > --- linux-2.6.10-rc2-mm3-full/include/net/sctp/tsnmap.h.old 2004-11-25 02:23:58.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/include/net/sctp/tsnmap.h 2004-11-25 02:24:21.000000000 +0100 > @@ -120,12 +120,6 @@ > __u32 start; > }; > > -/* Create a new tsnmap. */ > -struct sctp_tsnmap *sctp_tsnmap_new(__u16 len, __u32 init_tsn, int gfp); > - > -/* Dispose of a tsnmap. */ > -void sctp_tsnmap_free(struct sctp_tsnmap *); > - > /* This macro assists in creation of external storage for variable length > * internal buffers. We double allocate so the overflow map works. > */ > @@ -210,14 +204,4 @@ > /* Is there a gap in the TSN map? */ > int sctp_tsnmap_has_gap(const struct sctp_tsnmap *); > > -/* Initialize a gap ack block interator from user-provided memory. */ > -void sctp_tsnmap_iter_init(const struct sctp_tsnmap *, > - struct sctp_tsnmap_iter *); > - > -/* Get the next gap ack blocks. We return 0 if there are no more > - * gap ack blocks. > - */ > -int sctp_tsnmap_next_gap_ack(const struct sctp_tsnmap *, > - struct sctp_tsnmap_iter *,__u16 *start, __u16 *end); > - > #endif /* __sctp_tsnmap_h__ */ > --- linux-2.6.10-rc2-mm3-full/net/sctp/tsnmap.c.old 2004-11-25 02:24:29.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/net/sctp/tsnmap.c 2004-11-25 02:25:39.000000000 +0100 > @@ -52,29 +52,6 @@ > int *started, __u16 *start, > int *ended, __u16 *end); > > -/* Create a new sctp_tsnmap. > - * Allocate room to store at least 'len' contiguous TSNs. > - */ > -struct sctp_tsnmap *sctp_tsnmap_new(__u16 len, __u32 initial_tsn, int gfp) > -{ > - struct sctp_tsnmap *retval; > - > - retval = kmalloc(sizeof(struct sctp_tsnmap) + > - sctp_tsnmap_storage_size(len), gfp); > - if (!retval) > - goto fail; > - > - if (!sctp_tsnmap_init(retval, len, initial_tsn)) > - goto fail_map; > - retval->malloced = 1; > - return retval; > - > -fail_map: > - kfree(retval); > -fail: > - return NULL; > -} > - > /* Initialize a block of memory as a tsnmap. */ > struct sctp_tsnmap *sctp_tsnmap_init(struct sctp_tsnmap *map, __u16 len, > __u32 initial_tsn) > @@ -168,16 +145,9 @@ > } > > > -/* Dispose of a tsnmap. */ > -void sctp_tsnmap_free(struct sctp_tsnmap *map) > -{ > - if (map->malloced) > - kfree(map); > -} > - > /* Initialize a Gap Ack Block iterator from memory being provided. */ > -void sctp_tsnmap_iter_init(const struct sctp_tsnmap *map, > - struct sctp_tsnmap_iter *iter) > +static void sctp_tsnmap_iter_init(const struct sctp_tsnmap *map, > + struct sctp_tsnmap_iter *iter) > { > /* Only start looking one past the Cumulative TSN Ack Point. */ > iter->start = map->cumulative_tsn_ack_point + 1; > @@ -186,8 +156,9 @@ > /* Get the next Gap Ack Blocks. Returns 0 if there was not another block > * to get. > */ > -int sctp_tsnmap_next_gap_ack(const struct sctp_tsnmap *map, > - struct sctp_tsnmap_iter *iter, __u16 *start, __u16 *end) > +static int sctp_tsnmap_next_gap_ack(const struct sctp_tsnmap *map, > + struct sctp_tsnmap_iter *iter, > + __u16 *start, __u16 *end) > { > int started, ended; > __u16 _start, _end, offset; > --- linux-2.6.10-rc2-mm3-full/include/net/sctp/ulpevent.h.old 2004-11-25 02:26:08.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/include/net/sctp/ulpevent.h 2004-11-25 02:26:19.000000000 +0100 > @@ -77,8 +77,6 @@ > return (struct sctp_ulpevent *)skb->cb; > } > > -struct sctp_ulpevent *sctp_ulpevent_new(int size, int flags, int gfp); > -void sctp_ulpevent_init(struct sctp_ulpevent *, int flags); > void sctp_ulpevent_free(struct sctp_ulpevent *); > int sctp_ulpevent_is_notification(const struct sctp_ulpevent *); > void sctp_queue_purge_ulpevents(struct sk_buff_head *list); > --- linux-2.6.10-rc2-mm3-full/net/sctp/ulpevent.c.old 2004-11-25 02:26:26.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/net/sctp/ulpevent.c 2004-11-25 02:27:38.000000000 +0100 > @@ -65,8 +65,17 @@ > */ > } > > +/* Initialize an ULP event from an given skb. */ > +static void sctp_ulpevent_init(struct sctp_ulpevent *event, int msg_flags) > +{ > + memset(event, 0, sizeof(struct sctp_ulpevent)); > + event->msg_flags = msg_flags; > +} > + > /* Create a new sctp_ulpevent. */ > -struct sctp_ulpevent *sctp_ulpevent_new(int size, int msg_flags, int gfp) > +static struct sctp_ulpevent *sctp_ulpevent_new(int size, > + int msg_flags, > + int gfp) > { > struct sctp_ulpevent *event; > struct sk_buff *skb; > @@ -84,13 +93,6 @@ > return NULL; > } > > -/* Initialize an ULP event from an given skb. */ > -void sctp_ulpevent_init(struct sctp_ulpevent *event, int msg_flags) > -{ > - memset(event, 0, sizeof(struct sctp_ulpevent)); > - event->msg_flags = msg_flags; > -} > - > /* Is this a MSG_NOTIFICATION? */ > int sctp_ulpevent_is_notification(const struct sctp_ulpevent *event) > { > --- linux-2.6.10-rc2-mm3-full/include/net/sctp/ulpqueue.h.old 2004-11-25 02:28:27.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/include/net/sctp/ulpqueue.h 2004-11-25 02:28:36.000000000 +0100 > @@ -57,7 +57,6 @@ > }; > > /* Prototypes. */ > -struct sctp_ulpq *sctp_ulpq_new(struct sctp_association *asoc, int gfp); > struct sctp_ulpq *sctp_ulpq_init(struct sctp_ulpq *, > struct sctp_association *); > void sctp_ulpq_free(struct sctp_ulpq *); > --- linux-2.6.10-rc2-mm3-full/net/sctp/ulpqueue.c.old 2004-11-25 02:28:44.000000000 +0100 > +++ linux-2.6.10-rc2-mm3-full/net/sctp/ulpqueue.c 2004-11-25 02:28:58.000000000 +0100 > @@ -56,25 +56,6 @@ > > /* 1st Level Abstractions */ > > -/* Create a new ULP queue. */ > -struct sctp_ulpq *sctp_ulpq_new(struct sctp_association *asoc, int gfp) > -{ > - struct sctp_ulpq *ulpq; > - > - ulpq = kmalloc(sizeof(struct sctp_ulpq), gfp); > - if (!ulpq) > - goto fail; > - if (!sctp_ulpq_init(ulpq, asoc)) > - goto fail_init; > - ulpq->malloced = 1; > - return ulpq; > - > -fail_init: > - kfree(ulpq); > -fail: > - return NULL; > -} > - > /* Initialize a ULP queue from a block of memory. */ > struct sctp_ulpq *sctp_ulpq_init(struct sctp_ulpq *ulpq, > struct sctp_association *asoc) > @@ -92,7 +73,7 @@ > > > /* Flush the reassembly and ordering queues. */ > -void sctp_ulpq_flush(struct sctp_ulpq *ulpq) > +static void sctp_ulpq_flush(struct sctp_ulpq *ulpq) > { > struct sk_buff *skb; > struct sctp_ulpevent *event; > > From jesse.brandeburg@intel.com Tue Dec 7 17:13:43 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 17:13:47 -0800 (PST) Received: from orsfmr003.jf.intel.com (fmr18.intel.com [134.134.136.17]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB81Dgvh020382 for ; Tue, 7 Dec 2004 17:13:43 -0800 Received: from orsfmr100.jf.intel.com (orsfmr100.jf.intel.com [10.7.209.16]) by orsfmr003.jf.intel.com (8.12.10/8.12.10/d: major-outer.mc,v 1.1 2004/09/17 17:50:56 root Exp $) with ESMTP id iB81DFcE006153; Wed, 8 Dec 2004 01:13:15 GMT Received: from orsmsxvs040.jf.intel.com (orsmsxvs040.jf.intel.com [192.168.65.206]) by orsfmr100.jf.intel.com (8.12.10/8.12.10/d: major-inner.mc,v 1.2 2004/09/17 18:05:01 root Exp $) with SMTP id iB81D8Zp007661; Wed, 8 Dec 2004 01:13:08 GMT Received: from orsmsx332.amr.corp.intel.com ([192.168.65.60]) by orsmsxvs040.jf.intel.com (SAVSMTP 3.1.2.35) with SMTP id M2004120717102922328 ; Tue, 07 Dec 2004 17:10:32 -0800 Received: from orsmsx408.amr.corp.intel.com ([192.168.65.52]) by orsmsx332.amr.corp.intel.com with Microsoft SMTPSVC(6.0.3790.0); Tue, 7 Dec 2004 17:10:27 -0800 X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Subject: RE: how to tune a pair of e1000 cards on intel e7501-based system? Date: Tue, 7 Dec 2004 17:10:26 -0800 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: how to tune a pair of e1000 cards on intel e7501-based system? Thread-Index: AcTb6WG4udppu+VYRkClljDSThg2tgA2HYsw From: "Brandeburg, Jesse" To: "Ray Lehtiniemi" , "Scott Feldman" Cc: X-OriginalArrivalTime: 08 Dec 2004 01:10:27.0208 (UTC) FILETIME=[B3D54080:01C4DCC2] X-Scanned-By: MIMEDefang 2.44 Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by oss.sgi.com id iB81Dgvh020382 X-archive-position: 12550 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jesse.brandeburg@intel.com Precedence: bulk X-list: netdev > > > What kind of numbers are you getting? > > i'm seeing about 100Kpps, with all settings at their defaults on the > 2.4.20 > kernel. > > basically, i have a couple of desktop PCs generating 480 streams > of UDP data at 50 packets per second. Packet size on the wire, including > 96 bits of IFG, is 128 bytes. these packets are forwarded through a user > process on the NexGen box to an echoer process which is also running on > the > traffic generator boxes. the echoer sends them back to the NexGen user > process, > which forwards them back to the generator process. timestamps are logged > for each packet at send, loop and recv points. > I'm not much of an expert, but one easy thing to try is to up your receive stack resources, as they were particularly low on 2.4 series kernels, leading to udp getting overrun pretty easily with gig nics. I think if you make the value go too high it just ignores it, so if you see no change, try 256kB instead. cat /proc/sys/net/core/rmem_default cat /proc/sys/net/core/rmem_max echo -n 512000 > /proc/sys/net/core/rmem_default echo -n 512000 > /proc/sys/net/core/rmem_max Jesse From hadi@cyberus.ca Tue Dec 7 20:28:01 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 20:28:08 -0800 (PST) Received: from mx03.cybersurf.com (mx03.cybersurf.com [209.197.145.106]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB84Rxlo027978 for ; Tue, 7 Dec 2004 20:28:00 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx03.cybersurf.com with esmtp (Exim 4.30) id 1CbtQS-0007v3-92 for netdev@oss.sgi.com; Tue, 07 Dec 2004 23:27:40 -0500 Received: from cpe0030ab124d2f-cm014500000962.cpe.net.cable.rogers.com ([24.103.99.32] helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1CbtQI-0007Z6-Jb; Tue, 07 Dec 2004 23:27:30 -0500 Subject: Re: Hard freeze with 2.6.10-rc3 and QoS, worked fine with 2.6.9 From: jamal Reply-To: hadi@cyberus.ca To: Patrick McHardy Cc: Thomas Graf , Andrew Morton , Thomas Cataldo , linux-kernel@vger.kernel.org, netdev@oss.sgi.com, "David S. Miller" In-Reply-To: <41B5E722.2080600@trash.net> References: <1102380430.6103.6.camel@buffy> <20041206224441.628e7885.akpm@osdl.org> <1102422544.1088.98.camel@jzny.localdomain> <41B5E188.5050800@trash.net> <20041207170748.GF1371@postel.suug.ch> <41B5E722.2080600@trash.net> Content-Type: text/plain Organization: jamalopolous Message-Id: <1102480044.1050.9.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 07 Dec 2004 23:27:25 -0500 Content-Transfer-Encoding: 7bit X-archive-position: 12551 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev On Tue, 2004-12-07 at 12:23, Patrick McHardy wrote: > Either one is fine with me, although I would prefer to see > the number of ifdefs in this area going down, not up :) You guys pick one or other or a mix. I run 4 base testcases for the policer typically: 1) Old kernel, uptodate TC - MUST pass 2) old kernel, old tc (trivial - expected to pass). 3) New Kernel, uptodate TC - MUST pass 4) New Kernel, uptodate TC - MUST pass (although trivial) Try both setting, dumping then deleting policies. If these tests pass, please push patch to Dave. cheers, jamal From hadi@cyberus.ca Tue Dec 7 20:32:28 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 20:32:33 -0800 (PST) Received: from mx03.cybersurf.com (mx03.cybersurf.com [209.197.145.106]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB84WSQx028360 for ; Tue, 7 Dec 2004 20:32:28 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx03.cybersurf.com with esmtp (Exim 4.30) id 1CbtUn-0001AA-1E for netdev@oss.sgi.com; Tue, 07 Dec 2004 23:32:09 -0500 Received: from cpe0030ab124d2f-cm014500000962.cpe.net.cable.rogers.com ([24.103.99.32] helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1CbtUh-0007vs-Cn; Tue, 07 Dec 2004 23:32:03 -0500 Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets From: jamal Reply-To: hadi@cyberus.ca To: Karsten Desler Cc: netdev@oss.sgi.com, linux-kernel@vger.kernel.org, "David S. Miller" , Robert Olsson , P@draigBrady.com In-Reply-To: <20041207211035.GA20286@quickstop.soohrt.org> References: <20041206205305.GA11970@soohrt.org> <20041207211035.GA20286@quickstop.soohrt.org> Content-Type: text/plain Organization: jamalopolous Message-Id: <1102480318.1050.16.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 07 Dec 2004 23:31:58 -0500 Content-Transfer-Encoding: 7bit X-archive-position: 12552 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev On Tue, 2004-12-07 at 16:10, Karsten Desler wrote: > Karsten Desler wrote: > > Current packetload on eth0 (and reversed on eth1): > > 115kpps tx > > 135kpps rx > > I totally forgot to mention: There are approximately 100k concurrent > flows. ;-> Aha. That would make a huge difference. I know of noone who has actually done this level of testing. I have tried upto about 50K flows myself in early 2.6.x and was eventually compute bound. Really try compiling out totaly iptables/netfilter - it will make a difference. You may also want to try something like LC trie algorithm that Robert and co are playing with to see if it makes a difference with this many flows. cheers, jamal From hadi@znyx.com Tue Dec 7 20:42:22 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 20:42:27 -0800 (PST) Received: from lotus.znyx.com (znx208-2-156-007.znyx.com [208.2.156.7]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB84gM8P029034 for ; Tue, 7 Dec 2004 20:42:22 -0800 Received: from [127.0.0.1] ([208.2.156.2]) by lotus.znyx.com (Lotus Domino Release 5.0.11) with ESMTP id 2004120720452509:52708 ; Tue, 7 Dec 2004 20:45:25 -0800 Subject: Re: Hard freeze with 2.6.10-rc3 and QoS, worked fine with 2.6.9 From: Jamal Hadi Salim Reply-To: hadi@znyx.com To: Patrick McHardy Cc: Thomas Graf , Andrew Morton , Thomas Cataldo , linux-kernel@vger.kernel.org, netdev@oss.sgi.com, "David S. Miller" In-Reply-To: <1102480044.1050.9.camel@jzny.localdomain> References: <1102380430.6103.6.camel@buffy> <20041206224441.628e7885.akpm@osdl.org> <1102422544.1088.98.camel@jzny.localdomain> <41B5E188.5050800@trash.net> <20041207170748.GF1371@postel.suug.ch> <41B5E722.2080600@trash.net> <1102480044.1050.9.camel@jzny.localdomain> Organization: ZNYX Networks Message-Id: <1102480913.1049.24.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 07 Dec 2004 23:41:53 -0500 X-MIMETrack: Itemize by SMTP Server on Lotus/Znyx(Release 5.0.11 |July 24, 2002) at 12/07/2004 08:45:25 PM, Serialize by Router on Lotus/Znyx(Release 5.0.11 |July 24, 2002) at 12/07/2004 08:45:30 PM, Serialize complete at 12/07/2004 08:45:30 PM Content-Transfer-Encoding: 7bit Content-Type: text/plain X-archive-position: 12553 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@znyx.com Precedence: bulk X-list: netdev BTW, old kernel in this case implies one that does not support tc actions at all. So pick something like 2.4.28. New is whatever 2.6.x with patch. Old tc is something that for example ships with redhat new tc is whatever one is patched. Supplementary tests are: in 2.6.x to compile the policer in two different ways a) via tc actions and b) using the old scheme which is understood by "old" tc. Repeat the tests i described earlier with b) pretending to be "old" kernel. Infact come to think of it i would also prefer to have the suplementary tests run as well. If you guys have no cycles, please pass the patch to me and i will test on the weekend. cheers, jamal On Tue, 2004-12-07 at 23:27, jamal wrote: > On Tue, 2004-12-07 at 12:23, Patrick McHardy wrote: > > > Either one is fine with me, although I would prefer to see > > the number of ifdefs in this area going down, not up :) > > You guys pick one or other or a mix. I run 4 base testcases for the > policer typically: > > 1) Old kernel, uptodate TC - MUST pass > 2) old kernel, old tc (trivial - expected to pass). > 3) New Kernel, uptodate TC - MUST pass > 4) New Kernel, uptodate TC - MUST pass (although trivial) > > Try both setting, dumping then deleting policies. > > If these tests pass, please push patch to Dave. > > cheers, > jamal > From kaber@trash.net Tue Dec 7 21:17:57 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 21:18:03 -0800 (PST) Received: from kaber.coreworks.de ([62.206.217.67]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB85Hu8C031042 for ; Tue, 7 Dec 2004 21:17:57 -0800 Received: from localhost ([127.0.0.1]) by kaber.coreworks.de with esmtp (Exim 4.34) id 1CbuCT-000158-Jy; Wed, 08 Dec 2004 06:17:17 +0100 Message-ID: <41B68E5D.2080009@trash.net> Date: Wed, 08 Dec 2004 06:17:17 +0100 From: Patrick McHardy User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.7.3) Gecko/20041008 Debian/1.7.3-5 X-Accept-Language: en MIME-Version: 1.0 To: hadi@znyx.com CC: Thomas Graf , Andrew Morton , Thomas Cataldo , linux-kernel@vger.kernel.org, netdev@oss.sgi.com, "David S. Miller" Subject: Re: Hard freeze with 2.6.10-rc3 and QoS, worked fine with 2.6.9 References: <1102380430.6103.6.camel@buffy> <20041206224441.628e7885.akpm@osdl.org> <1102422544.1088.98.camel@jzny.localdomain> <41B5E188.5050800@trash.net> <20041207170748.GF1371@postel.suug.ch> <41B5E722.2080600@trash.net> <1102480044.1050.9.camel@jzny.localdomain> <1102480913.1049.24.camel@jzny.localdomain> In-Reply-To: <1102480913.1049.24.camel@jzny.localdomain> Content-Type: multipart/mixed; boundary="------------040309070208000504050501" X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12554 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kaber@trash.net Precedence: bulk X-list: netdev This is a multi-part message in MIME format. --------------040309070208000504050501 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Jamal Hadi Salim wrote: >BTW, old kernel in this case implies one that does not support tc >actions at all. So pick something like 2.4.28. >New is whatever 2.6.x with patch. >Old tc is something that for example ships with redhat >new tc is whatever one is patched. > >Supplementary tests are: in 2.6.x to compile the policer >in two different ways a) via tc actions and b) using the old scheme >which is understood by "old" tc. Repeat the tests i described earlier >with b) pretending to be "old" kernel. > >Infact come to think of it i would also prefer to have the suplementary >tests run as well. >If you guys have no cycles, please pass the patch to me and i will test >on the weekend. > > I think these tests are a waste of time. struct tcf_police is not userspace-visible, so it's highly unlikely that the tc version matters. Why an old kernel needs to be tested is beyond me. For possible in-kernel breakage caused by the restructuring, without CONFIG_NET_CLS_ACT, struct tcf_police is only used in police.c, without any casts or assumptions about layout, so I can't see what could break. With CONFIG_NET_CLS_ACT, the only place where it is used outside of police.c is tcf_action_copy_stats, and this is exactly what this patch (tested) fixes. If you still want to do these test, please use the attached patch. Regards Patrick --------------040309070208000504050501 Content-Type: text/plain; name="x" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="x" ===== include/net/act_api.h 1.4 vs edited ===== --- 1.4/include/net/act_api.h 2004-11-06 01:33:12 +01:00 +++ edited/include/net/act_api.h 2004-12-07 17:53:37 +01:00 @@ -8,15 +8,23 @@ #include #include +#define tca_gen(name) \ +struct tcf_##name *next; \ + u32 index; \ + int refcnt; \ + int bindcnt; \ + u32 capab; \ + int action; \ + struct tcf_t tm; \ + struct gnet_stats_basic bstats; \ + struct gnet_stats_queue qstats; \ + struct gnet_stats_rate_est rate_est; \ + spinlock_t *stats_lock; \ + spinlock_t lock + struct tcf_police { - struct tcf_police *next; - int refcnt; -#ifdef CONFIG_NET_CLS_ACT - int bindcnt; -#endif - u32 index; - int action; + tca_gen(police); int result; u32 ewma_rate; u32 burst; @@ -24,33 +32,14 @@ u32 toks; u32 ptoks; psched_time_t t_c; - spinlock_t lock; struct qdisc_rate_table *R_tab; struct qdisc_rate_table *P_tab; - - struct gnet_stats_basic bstats; - struct gnet_stats_queue qstats; - struct gnet_stats_rate_est rate_est; - spinlock_t *stats_lock; }; #ifdef CONFIG_NET_CLS_ACT #define ACT_P_CREATED 1 #define ACT_P_DELETED 1 -#define tca_gen(name) \ -struct tcf_##name *next; \ - u32 index; \ - int refcnt; \ - int bindcnt; \ - u32 capab; \ - int action; \ - struct tcf_t tm; \ - struct gnet_stats_basic bstats; \ - struct gnet_stats_queue qstats; \ - struct gnet_stats_rate_est rate_est; \ - spinlock_t *stats_lock; \ - spinlock_t lock struct tcf_act_hdr { --------------040309070208000504050501-- From davem@davemloft.net Tue Dec 7 21:30:31 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 21:30:36 -0800 (PST) Received: from cheetah.davemloft.net (mail@adsl-63-197-226-105.dsl.snfc21.pacbell.net [63.197.226.105]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB85UVCs031807 for ; Tue, 7 Dec 2004 21:30:31 -0800 Received: from localhost ([127.0.0.1] helo=cheetah.davemloft.net ident=davem) by cheetah.davemloft.net with smtp (Exim 3.36 #1 (Debian)) id 1CbuN8-0001VE-00; Tue, 07 Dec 2004 21:28:18 -0800 Date: Tue, 7 Dec 2004 21:28:17 -0800 From: "David S. Miller" To: Mitchell Blank Jr Cc: kernel@linuxace.com, shemminger@osdl.org, linux-net@vger.kernel.org, linux-kernel@vger.kernel.org, netdev@oss.sgi.com Subject: Re: [PATCH] fix select() for SOCK_RAW sockets Message-Id: <20041207212817.1b74671b.davem@davemloft.net> In-Reply-To: <20041207150834.GA75700@gaz.sfgoth.com> References: <20041207003525.GA22933@linuxace.com> <20041207025218.GB61527@gaz.sfgoth.com> <20041207045302.GA23746@linuxace.com> <20041207054840.GD61527@gaz.sfgoth.com> <20041207150834.GA75700@gaz.sfgoth.com> X-Mailer: Sylpheed version 1.0.0beta3 (GTK+ 1.2.10; sparc-unknown-linux-gnu) X-Face: "_;p5u5aPsO,_Vsx"^v-pEq09'CU4&Dc1$fQExov$62l60cgCc%FnIwD=.UF^a>?5'9Kn[;433QFVV9M..2eN.@4ZWPGbdi<=?[:T>y?SD(R*-3It"Vj:)"dP Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12555 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: davem@davemloft.net Precedence: bulk X-list: netdev On Tue, 7 Dec 2004 07:08:34 -0800 Mitchell Blank Jr wrote: > Davem: I only tested that this doesn't break UDP; if it works for Phil and > Stephen can verify that it doesn't break his bad-checksum UDP tests then > please push it for 2.6.10. Looks good Mitchell, patch applied. Thanks. From davem@davemloft.net Tue Dec 7 21:33:23 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 21:33:29 -0800 (PST) Received: from cheetah.davemloft.net (mail@adsl-63-197-226-105.dsl.snfc21.pacbell.net [63.197.226.105]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB85XNIC032269 for ; Tue, 7 Dec 2004 21:33:23 -0800 Received: from localhost ([127.0.0.1] helo=cheetah.davemloft.net ident=davem) by cheetah.davemloft.net with smtp (Exim 3.36 #1 (Debian)) id 1CbuPd-0001Vp-00; Tue, 07 Dec 2004 21:30:53 -0800 Date: Tue, 7 Dec 2004 21:30:53 -0800 From: "David S. Miller" To: Patrick McHardy Cc: tgraf@suug.ch, hadi@cyberus.ca, akpm@osdl.org, tomc@compaqnet.fr, linux-kernel@vger.kernel.org, netdev@oss.sgi.com Subject: Re: Hard freeze with 2.6.10-rc3 and QoS, worked fine with 2.6.9 Message-Id: <20041207213053.6bb602c1.davem@davemloft.net> In-Reply-To: <41B5E722.2080600@trash.net> References: <1102380430.6103.6.camel@buffy> <20041206224441.628e7885.akpm@osdl.org> <1102422544.1088.98.camel@jzny.localdomain> <41B5E188.5050800@trash.net> <20041207170748.GF1371@postel.suug.ch> <41B5E722.2080600@trash.net> X-Mailer: Sylpheed version 1.0.0beta3 (GTK+ 1.2.10; sparc-unknown-linux-gnu) X-Face: "_;p5u5aPsO,_Vsx"^v-pEq09'CU4&Dc1$fQExov$62l60cgCc%FnIwD=.UF^a>?5'9Kn[;433QFVV9M..2eN.@4ZWPGbdi<=?[:T>y?SD(R*-3It"Vj:)"dP Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12556 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: davem@davemloft.net Precedence: bulk X-list: netdev On Tue, 07 Dec 2004 18:23:46 +0100 Patrick McHardy wrote: > Either one is fine with me, although I would prefer to see > the number of ifdefs in this area going down, not up :) I agree, therefore I applied Patrick's patch. From bunk@stusta.de Tue Dec 7 21:33:59 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 21:34:06 -0800 (PST) Received: from mailout.stusta.mhn.de (mailout.stusta.mhn.de [141.84.69.5]) by oss.sgi.com (8.13.0/8.13.0) with SMTP id iB85Xwq4032505 for ; Tue, 7 Dec 2004 21:33:58 -0800 Received: (qmail 13316 invoked from network); 8 Dec 2004 03:33:30 -0000 Received: from r063144.stusta.swh.mhn.de (10.150.63.144) by mailout.stusta.mhn.de with SMTP; 8 Dec 2004 03:33:30 -0000 Received: by r063144.stusta.swh.mhn.de (Postfix, from userid 1000) id A26C6BBA6A; Wed, 8 Dec 2004 04:33:26 +0100 (CET) Date: Wed, 8 Dec 2004 04:33:26 +0100 From: Adrian Bunk To: prism54-private@prism54.org, Andrew Morton Cc: netdev@oss.sgi.com, jgarzik@pobox.com, linux-net@vger.kernel.org Subject: [2.6 patch] prism54: small prismcompat cleanup (fwd) Message-ID: <20041208033326.GQ5496@stusta.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.6+20040907i X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12557 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: bunk@stusta.de Precedence: bulk X-list: netdev The patch forwarded below (slightly adopted for 2.6.10-rc2-mm4) still applies. Please apply. ----- Forwarded message from Adrian Bunk ----- Date: Sat, 30 Oct 2004 07:38:00 +0200 From: Adrian Bunk To: Margit Schubert-While Cc: prism54-private@prism54.org, netdev@oss.sgi.com, jgarzik@pobox.com, linux-net@vger.kernel.org Subject: [2.6 patch] prism54: small prismcompat cleanup - the FW_LOADER is already guaranteed through the Kconfig file - prism54_synchronize_irq is also #define'd to synchronize_irq in prismcompat24.h, so there's no need for it diffstat output: drivers/net/wireless/prism54/islpci_dev.c | 2 +- drivers/net/wireless/prism54/prismcompat.h | 6 ------ 2 files changed, 1 insertion(+), 7 deletions(-) Signed-off-by: Adrian Bunk --- linux-2.6.10-rc1-mm2-full/drivers/net/wireless/prism54/islpci_dev.c.old2 2004-10-30 07:23:07.000000000 +0200 +++ linux-2.6.10-rc1-mm2-full/drivers/net/wireless/prism54/islpci_dev.c 2004-10-30 07:23:19.000000000 +0200 @@ -420,7 +420,7 @@ * currently in progress by emptying the TX and RX queues. */ /* wait until interrupts have finished executing on other CPUs */ - prism54_synchronize_irq(priv->pdev->irq); + synchronize_irq(priv->pdev->irq); reg = readl(device_base + ISL38XX_CTRL_STAT_REG); reg &= ~(ISL38XX_CTRL_STAT_RESET | ISL38XX_CTRL_STAT_RAMBOOT); --- linux-2.6.10-rc2-mm4-full/drivers/net/wireless/prism54/prismcompat.h.old 2004-12-08 04:06:13.000000000 +0100 +++ linux-2.6.10-rc2-mm4-full/drivers/net/wireless/prism54/prismcompat.h 2004-12-08 04:06:25.000000000 +0100 @@ -34,16 +34,10 @@ #include #include -#if !defined(CONFIG_FW_LOADER) && !defined(CONFIG_FW_LOADER_MODULE) -#error Firmware Loading is not configured in the kernel ! -#endif - #ifndef __iomem #define __iomem #endif -#define prism54_synchronize_irq(irq) synchronize_irq(irq) - #define PRISM_FW_PDEV &priv->pdev->dev #endif /* _PRISM_COMPAT_H */ From davem@davemloft.net Tue Dec 7 21:34:43 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 21:34:49 -0800 (PST) Received: from cheetah.davemloft.net (mail@adsl-63-197-226-105.dsl.snfc21.pacbell.net [63.197.226.105]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB85YhFg000624 for ; Tue, 7 Dec 2004 21:34:43 -0800 Received: from localhost ([127.0.0.1] helo=cheetah.davemloft.net ident=davem) by cheetah.davemloft.net with smtp (Exim 3.36 #1 (Debian)) id 1CbuRG-0001Wk-00; Tue, 07 Dec 2004 21:32:34 -0800 Date: Tue, 7 Dec 2004 21:32:34 -0800 From: "David S. Miller" To: Thomas Graf Cc: netdev@oss.sgi.com Subject: Re: [PATCH] PKT_SCHED: validate policer configuration TLVs Message-Id: <20041207213234.257fd0d9.davem@davemloft.net> In-Reply-To: <20041207172349.GG1371@postel.suug.ch> References: <20041207172349.GG1371@postel.suug.ch> X-Mailer: Sylpheed version 1.0.0beta3 (GTK+ 1.2.10; sparc-unknown-linux-gnu) X-Face: "_;p5u5aPsO,_Vsx"^v-pEq09'CU4&Dc1$fQExov$62l60cgCc%FnIwD=.UF^a>?5'9Kn[;433QFVV9M..2eN.@4ZWPGbdi<=?[:T>y?SD(R*-3It"Vj:)"dP Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12558 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: davem@davemloft.net Precedence: bulk X-list: netdev On Tue, 7 Dec 2004 18:23:49 +0100 Thomas Graf wrote: > Adds TLV size sanity checks for policer configuration. Hmmm... > - if (tb[TCA_POLICE_RESULT-1]) > + if (tb[TCA_POLICE_RESULT-1]) { > + if (RTA_PAYLOAD(tb[TCA_POLICE_RESULT-1]) != sizeof(u32)) > + goto failure; > p->result = *(int*)RTA_DATA(tb[TCA_POLICE_RESULT-1]); > + } Either these things are int's or u32's, they cannot be both :-) I know that size wise it's identical, but at least make the code look consistent. From willy@w.ods.org Tue Dec 7 21:53:19 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 21:53:23 -0800 (PST) Received: from willy.net1.nerim.net (willy.net1.nerim.net [62.212.114.60]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB85rHM9001487 for ; Tue, 7 Dec 2004 21:53:18 -0800 Date: Wed, 8 Dec 2004 06:39:53 +0100 From: Willy Tarreau To: Karsten Desler Cc: P@draigBrady.com, "David S. Miller" , netdev@oss.sgi.com, linux-kernel@vger.kernel.org Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets Message-ID: <20041208053953.GC17946@alpha.home.local> References: <20041206205305.GA11970@soohrt.org> <20041206134849.498bfc93.davem@davemloft.net> <20041206224107.GA8529@soohrt.org> <41B58A58.8010007@draigBrady.com> <20041207112139.GA3610@soohrt.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20041207112139.GA3610@soohrt.org> User-Agent: Mutt/1.4i X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12559 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: willy@w.ods.org Precedence: bulk X-list: netdev On Tue, Dec 07, 2004 at 12:21:39PM +0100, Karsten Desler wrote: > > I also notice that a lot of time is spent allocating > > and freeing the packet buffers (and possible hidden > > time due to cache misses due to allocating on one > > CPU and freeing on another?). > > How many [RT]xDescriptors do you have configured by the way? > > 256. I increased them to 1024 shortly after the profiling run, but > didn't notice any change in the cpu usage (will try again with cyclesoak). Have you checked the interrupts rate ? I had an e1000 eating many CPU cycles because it would generate 50000 interrupts/s. Passing the module InterruptThrottleRate=5000 definitely calmed it down, and more than doubled the data rate. Regards Willy From buytenh@wantstofly.org Tue Dec 7 23:39:22 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 23:39:26 -0800 (PST) Received: from xi.wantstofly.org (alephnull.demon.nl [212.238.201.82]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB87dLgg005057 for ; Tue, 7 Dec 2004 23:39:22 -0800 Received: by xi.wantstofly.org (Postfix, from userid 500) id 217542B0ED; Wed, 8 Dec 2004 08:38:59 +0100 (MET) Date: Wed, 8 Dec 2004 08:38:58 +0100 From: Lennert Buytenhek To: Ben Greear Cc: Robert Olsson , hadi@cyberus.ca, netdev@oss.sgi.com Subject: Re: inter-packet gap in pktgen Message-ID: <20041208073858.GA4027@xi.wantstofly.org> References: <20041207222522.GA30266@xi.wantstofly.org> <41B632F3.1090104@candelatech.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <41B632F3.1090104@candelatech.com> User-Agent: Mutt/1.4.1i X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12560 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: buytenh@wantstofly.org Precedence: bulk X-list: netdev On Tue, Dec 07, 2004 at 02:47:15PM -0800, Ben Greear wrote: > >By tweaking the 'ipg' parameter I can generate pretty much any packet rate > >I want, as long as I set ipg=(1e9/rate)-496 instead of something possibly > >more straightforward. > > That 496 will also change with load on the system, at least on average. I > dealt with this by having a user-space app sample the rate and adjust the > ipg to keep the average rate where I want it. > > So, I'd suggest leaving the ipg as it is, and use external tools to get > the exact pps that you are looking for. Just because it's possible that way doesn't mean that it's the only way or even _the_ way of doing it. Another option is: next_tx = get_time_in_ns(); while (--count) { tx_packet(); next_tx += 1e9/intended_pps; nanospin(next_tx - get_time_in_ns()); } This should be relatively independent of system load. OK, I know, time to show some code. --L From yoshfuji@linux-ipv6.org Tue Dec 7 23:57:47 2004 Received: with ECARTIS (v1.0.0; list netdev); Tue, 07 Dec 2004 23:57:52 -0800 (PST) Received: from yue.st-paulia.net (yue.linux-ipv6.org [203.178.140.15]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB87vkeL006170 for ; Tue, 7 Dec 2004 23:57:47 -0800 Received: from localhost (localhost [127.0.0.1]) by yue.st-paulia.net (Postfix) with ESMTP id 4C3A833CE5; Wed, 8 Dec 2004 16:58:59 +0900 (JST) Date: Wed, 08 Dec 2004 16:58:50 +0900 (JST) Message-Id: <20041208.165850.97660819.yoshfuji@linux-ipv6.org> To: davem@davemloft.net Cc: shemminger@osdl.org, mitch@sfgoth.com, kernel@linuxace.com, linux-net@vger.kernel.org, linux-kernel@vger.kernel.org, netdev@oss.sgi.com, yoshfuji@linux-ipv6.org Subject: Re: [PATCH] fix select() for SOCK_RAW sockets (ipv6) From: YOSHIFUJI Hideaki / =?iso-2022-jp?B?GyRCNUhGIzFRTEAbKEI=?= In-Reply-To: <20041207104815.3f7a4684.davem@davemloft.net> References: <20041208.023530.26430801.yoshfuji@linux-ipv6.org> <20041207100140.781f4c00@dxpl.pdx.osdl.net> <20041207104815.3f7a4684.davem@davemloft.net> Organization: USAGI Project X-URL: http://www.yoshifuji.org/%7Ehideaki/ X-Fingerprint: 9022 65EB 1ECF 3AD1 0BDF 80D8 4807 F894 E062 0EEA X-PGP-Key-URL: http://www.yoshifuji.org/%7Ehideaki/hideaki@yoshifuji.org.asc X-Face: "5$Al-.M>NJ%a'@hhZdQm:."qn~PA^gq4o*>iCFToq*bAi#4FRtx}enhuQKz7fNqQz\BYU] $~O_5m-9'}MIs`XGwIEscw;e5b>n"B_?j/AkL~i/MEaZBLP X-Mailer: Mew version 2.2 on Emacs 20.7 / Mule 4.1 (AOI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12561 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: yoshfuji@linux-ipv6.org Precedence: bulk X-list: netdev In article <20041207104815.3f7a4684.davem@davemloft.net> (at Tue, 7 Dec 2004 10:48:15 -0800), "David S. Miller" says: > On Tue, 7 Dec 2004 10:01:40 -0800 > Stephen Hemminger wrote: > > > > Probably, we need to do the same for ipv6, don't we? > > > > diff -Nru a/net/ipv6/af_inet6.c b/net/ipv6/af_inet6.c > > --- a/net/ipv6/af_inet6.c 2004-12-07 10:02:50 -08:00 > > +++ b/net/ipv6/af_inet6.c 2004-12-07 10:02:50 -08:00 > > @@ -513,6 +513,27 @@ > > We didn't do the "UDP select() fix" on ipv6 because we don't > have the delayed checksumming optimization there. So the > ipv6 side really doesn't need this change. Ok, thanks for the explanation. --yoshfuji @ Strasbourg From yoshfuji@linux-ipv6.org Wed Dec 8 01:43:45 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 08 Dec 2004 01:43:51 -0800 (PST) Received: from yue.st-paulia.net (yue.linux-ipv6.org [203.178.140.15]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB89hiPI013225 for ; Wed, 8 Dec 2004 01:43:44 -0800 Received: from localhost (localhost [127.0.0.1]) by yue.st-paulia.net (Postfix) with ESMTP id F151933CE5; Wed, 8 Dec 2004 18:44:56 +0900 (JST) Date: Wed, 08 Dec 2004 10:44:56 +0100 (CET) Message-Id: <20041208.104456.103795781.yoshfuji@linux-ipv6.org> To: davem@davemloft.net Cc: netdev@oss.sgi.com Subject: [PATCH] [IPV6]: Fix check if we're router. From: YOSHIFUJI Hideaki / =?iso-2022-jp?B?GyRCNUhGIzFRTEAbKEI=?= Organization: USAGI Project X-URL: http://www.yoshifuji.org/%7Ehideaki/ X-Fingerprint: 9022 65EB 1ECF 3AD1 0BDF 80D8 4807 F894 E062 0EEA X-PGP-Key-URL: http://www.yoshifuji.org/%7Ehideaki/hideaki@yoshifuji.org.asc X-Face: "5$Al-.M>NJ%a'@hhZdQm:."qn~PA^gq4o*>iCFToq*bAi#4FRtx}enhuQKz7fNqQz\BYU] $~O_5m-9'}MIs`XGwIEscw;e5b>n"B_?j/AkL~i/MEaZBLP X-Mailer: Mew version 2.2 on Emacs 20.7 / Mule 4.1 (AOI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12562 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: yoshfuji@linux-ipv6.org Precedence: bulk X-list: netdev Hello. Fix the check if we're router. against 2.6.10-rc3, for 2.6.10 queue. Signed-off-by: Hideaki YOSHIFUJI ===== net/ipv6/ndisc.c 1.105 vs edited ===== --- 1.105/net/ipv6/ndisc.c 2004-11-10 15:57:03 +09:00 +++ edited/net/ipv6/ndisc.c 2004-12-08 18:34:45 +09:00 @@ -943,7 +943,7 @@ } /* Don't accept RS if we're not in router mode */ - if (!idev->cnf.forwarding || idev->cnf.accept_ra) + if (!idev->cnf.forwarding) goto out; /* -- Hideaki YOSHIFUJI @ USAGI Project GPG FP: 9022 65EB 1ECF 3AD1 0BDF 80D8 4807 F894 E062 0EEA From Robert.Olsson@data.slu.se Wed Dec 8 02:22:24 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 08 Dec 2004 02:22:29 -0800 (PST) Received: from mail1.slu.se (mail1.slu.se [130.238.96.11]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB8AMMuH015083 for ; Wed, 8 Dec 2004 02:22:23 -0800 Received: from robur.slu.se (robur.slu.se [130.238.98.12]) by mail1.slu.se (8.12.10/8.12.10) with ESMTP id iB8ALsKO001387; Wed, 8 Dec 2004 11:21:54 +0100 Received: by robur.slu.se (Postfix, from userid 1000) id CA3B7EC002; Wed, 8 Dec 2004 11:21:54 +0100 (CET) From: Robert Olsson MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <16822.54722.755218.745451@robur.slu.se> Date: Wed, 8 Dec 2004 11:21:54 +0100 To: Lennert Buytenhek Cc: Ben Greear , Robert Olsson , hadi@cyberus.ca, netdev@oss.sgi.com Subject: Re: inter-packet gap in pktgen In-Reply-To: <20041208073858.GA4027@xi.wantstofly.org> References: <20041207222522.GA30266@xi.wantstofly.org> <41B632F3.1090104@candelatech.com> <20041208073858.GA4027@xi.wantstofly.org> X-Mailer: VM 7.18 under Emacs 21.3.1 X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12563 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: Robert.Olsson@data.slu.se Precedence: bulk X-list: netdev Lennert Buytenhek writes: > > Another option is: > > next_tx = get_time_in_ns(); > while (--count) { > tx_packet(); > next_tx += 1e9/intended_pps; > nanospin(next_tx - get_time_in_ns()); > } Hello! I think this what Ben is doing with his userland app. Ev. adjusting the ipg delay in runtime? A kind of control system. Even the device stats could possible be read. > This should be relatively independent of system load. OK, I know, > time to show some code. Also an idea might be to have some kind of option to use pktgen w. existent qdisc/tc infrastructure for this type of tests. Ben did you try this? --ro From webvenza@libero.it Wed Dec 8 02:47:48 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 08 Dec 2004 02:47:55 -0800 (PST) Received: from gateway.milesteg.arr (venza@adsl-ull-208-134.44-151.net24.it [151.44.134.208]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB8AllcV022245 for ; Wed, 8 Dec 2004 02:47:48 -0800 Date: Wed, 8 Dec 2004 11:47:21 +0100 From: Daniele Venzano To: NetDev , Jeff Garzik Subject: [PATCH 0/5] sis900 printk and stack usage audit Message-ID: <20041208104721.GA31707@picchio.gall.it> Mail-Followup-To: NetDev , Jeff Garzik Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Operating-System: Debian GNU/Linux on kernel Linux 2.4.27-grsec X-Copyright: Forwarding or publishing without permission is prohibited. X-Truth: La vita e' una questione di culo, o ce l'hai o te lo fanno. X-GPG-Fingerprint: 642A A345 1CEF B6E3 925C 23CE DAB9 8764 25B3 57ED X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12564 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: webvenza@libero.it Precedence: bulk X-list: netdev This patchset brings the debug output of the sis900 driver up to date, making it actually useful. There is also a rework of the code to avoid unnede calls to sis630_set_eq and to remove repeated calls to pci_read_config_byte. All the patches are made against latest netdev-2.6 tree (linus' sis900 + altimata PHY patch) I've tested all patches and the runtime debug parameter and everything works as expected. As usual all patches are available from http://teg.homeunix.org/sis900.html Thanks. -- ----------------------------- Daniele Venzano Web: http://teg.homeunix.org From dev-null@krynn.se.axis.com Wed Dec 8 03:01:06 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 08 Dec 2004 03:01:13 -0800 (PST) Received: from krynn.se.axis.com ([212.209.10.221]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB8B15tU023973 for ; Wed, 8 Dec 2004 03:01:05 -0800 Received: from krynn.se.axis.com (localhost [127.0.0.1]) by krynn.se.axis.com (8.12.9/8.12.9/Debian-5local0.1) with ESMTP id iB8B0XAD012187 for ; Wed, 8 Dec 2004 12:00:33 +0100 Received: (from root@localhost) by krynn.se.axis.com (8.12.9/8.12.9/Debian-5local0.1) id iB8B0XLi012185; Wed, 8 Dec 2004 12:00:33 +0100 Message-Id: <200412081100.iB8B0XLi012185@krynn.se.axis.com> Content-Disposition: inline Content-Type: text/plain MIME-Version: 1.0 X-Mailer: MIME::Lite 2.117 (F2.6; A1.44; B2.12; Q2.03) Date: Wed, 8 Dec 2004 11:00:33 UT From: Return daemon S7KW8 To: netdev@oss.sgi.com Subject: Invalid address: ted.gulley Precedence: bulk X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by oss.sgi.com id iB8B15tU023973 X-archive-position: 12565 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: dev-null@axis.com Precedence: bulk X-list: netdev The addresses and are not valid. ted gulley has no known new address. From webvenza@libero.it Wed Dec 8 03:02:20 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 08 Dec 2004 03:02:26 -0800 (PST) Received: from gateway.milesteg.arr (venza@adsl-ull-208-134.44-151.net24.it [151.44.134.208]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB8B2I22025274 for ; Wed, 8 Dec 2004 03:02:19 -0800 Date: Wed, 8 Dec 2004 12:01:56 +0100 From: Daniele Venzano To: NetDev , Jeff Garzik Subject: [PATCH 1/5] sis900 printk and stack usage audit Message-ID: <20041208110156.GB31707@picchio.gall.it> Mail-Followup-To: NetDev , Jeff Garzik References: <20041208104721.GA31707@picchio.gall.it> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="XOIedfhf+7KOe/yw" Content-Disposition: inline In-Reply-To: <20041208104721.GA31707@picchio.gall.it> X-Operating-System: Debian GNU/Linux on kernel Linux 2.4.27-grsec X-Copyright: Forwarding or publishing without permission is prohibited. X-Truth: La vita e' una questione di culo, o ce l'hai o te lo fanno. X-GPG-Fingerprint: 642A A345 1CEF B6E3 925C 23CE DAB9 8764 25B3 57ED X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12566 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: webvenza@libero.it Precedence: bulk X-list: netdev --XOIedfhf+7KOe/yw Content-Type: multipart/mixed; boundary="huq684BweRXVnRxX" Content-Disposition: inline --huq684BweRXVnRxX Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Audit of current printk() calls Changed debug levels to 0,1,2,3 as follows: 0 No debug 1 load/probe/unload/suspend/resume stuff 2 rx/tx errors 3 rx/tx packets and every interrupt are logged (very verbose) Debug levels are incremental Removed double printing of version string in module_init and in sis900_probe Made the sis900_debug parameter modifiable at runtime Signed-off-by: Daniele Venzano -- ----------------------------- Daniele Venzano Web: http://teg.homeunix.org --huq684BweRXVnRxX Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="sis900-debug1.diff" Index: sis900.c =================================================================== --- a/drivers/net/sis900.c (revision 39) +++ b/drivers/net/sis900.c (revision 40) @@ -183,10 +183,10 @@ module_param(multicast_filter_limit, int, 0444); module_param(max_interrupt_work, int, 0444); -module_param(debug, int, 0444); +module_param(debug, int, 0644); MODULE_PARM_DESC(multicast_filter_limit, "SiS 900/7016 maximum number of filtered multicast addresses"); MODULE_PARM_DESC(max_interrupt_work, "SiS 900/7016 maximum events handled per interrupt"); -MODULE_PARM_DESC(debug, "SiS 900/7016 debug level (2-4)"); +MODULE_PARM_DESC(debug, "SiS 900/7016 debug level (0-3)"); static int sis900_open(struct net_device *net_dev); static int sis900_mii_probe (struct net_device * net_dev); @@ -236,7 +236,7 @@ /* check to see if we have sane EEPROM */ signature = (u16) read_eeprom(ioaddr, EEPROMSignature); if (signature == 0xffff || signature == 0x0000) { - printk (KERN_INFO "%s: Error EERPOM read %x\n", + printk (KERN_WARNING "%s: Error: EEPROM signature is %x\n", net_dev->name, signature); return 0; } @@ -269,7 +269,7 @@ if (!isa_bridge) { isa_bridge = pci_find_device(PCI_VENDOR_ID_SI, 0x0018, isa_bridge); if (!isa_bridge) { - printk("%s: Can not find ISA bridge\n", net_dev->name); + printk(KERN_WARNING "%s: Can not find ISA bridge\n", net_dev->name); return 0; } } @@ -390,13 +390,6 @@ u8 revision; char *card_name = card_names[pci_id->driver_data]; -/* when built into the kernel, we only print version if device is found */ -#ifndef MODULE - static int printed_version; - if (!printed_version++) - printk(version); -#endif - /* setup various bits in PCI command register */ ret = pci_enable_device(pci_dev); if(ret) return ret; @@ -554,7 +547,7 @@ continue; if ((mii_phy = kmalloc(sizeof(struct mii_phy), GFP_KERNEL)) == NULL) { - printk(KERN_INFO "Cannot allocate mem for struct mii_phy\n"); + printk(KERN_WARNING "Cannot allocate mem for struct mii_phy\n"); mii_phy = sis_priv->first_mii; while (mii_phy) { struct mii_phy *phy; @@ -1015,8 +1008,8 @@ outl((i << RFADDR_shift), ioaddr + rfcr); outl(w, ioaddr + rfdr); - if (sis900_debug > 2) { - printk(KERN_INFO "%s: Receive Filter Addrss[%d]=%x\n", + if (sis900_debug > 0) { + printk(KERN_DEBUG "%s: Receive Filter Addrss[%d]=%x\n", net_dev->name, i, inl(ioaddr + rfdr)); } } @@ -1053,8 +1046,8 @@ /* load Transmit Descriptor Register */ outl(sis_priv->tx_ring_dma, ioaddr + txdp); - if (sis900_debug > 2) - printk(KERN_INFO "%s: TX descriptor register loaded with: %8.8x\n", + if (sis900_debug > 0) + printk(KERN_DEBUG "%s: TX descriptor register loaded with: %8.8x\n", net_dev->name, inl(ioaddr + txdp)); } @@ -1107,8 +1100,8 @@ /* load Receive Descriptor Register */ outl(sis_priv->rx_ring_dma, ioaddr + rxdp); - if (sis900_debug > 2) - printk(KERN_INFO "%s: RX descriptor register loaded with: %8.8x\n", + if (sis900_debug > 0) + printk(KERN_DEBUG "%s: RX descriptor register loaded with: %8.8x\n", net_dev->name, inl(ioaddr + rxdp)); } @@ -1557,8 +1550,8 @@ net_dev->trans_start = jiffies; - if (sis900_debug > 3) - printk(KERN_INFO "%s: Queued Tx packet at %p size %d " + if (sis900_debug > 2) + printk(KERN_DEBUG "%s: Queued Tx packet at %p size %d " "to slot %d.\n", net_dev->name, skb->data, (int)skb->len, entry); @@ -1617,8 +1610,8 @@ } } while (1); - if (sis900_debug > 3) - printk(KERN_INFO "%s: exiting interrupt, " + if (sis900_debug > 2) + printk(KERN_DEBUG "%s: exiting interrupt, " "interrupt status = 0x%#8.8x.\n", net_dev->name, inl(ioaddr + isr)); @@ -1643,8 +1636,8 @@ unsigned int entry = sis_priv->cur_rx % NUM_RX_DESC; u32 rx_status = sis_priv->rx_ring[entry].cmdsts; - if (sis900_debug > 3) - printk(KERN_INFO "sis900_rx, cur_rx:%4.4d, dirty_rx:%4.4d " + if (sis900_debug > 2) + printk(KERN_DEBUG "sis900_rx, cur_rx:%4.4d, dirty_rx:%4.4d " "status:0x%8.8x\n", sis_priv->cur_rx, sis_priv->dirty_rx, rx_status); @@ -1655,8 +1648,8 @@ if (rx_status & (ABORT|OVERRUN|TOOLONG|RUNT|RXISERR|CRCERR|FAERR)) { /* corrupted packet received */ - if (sis900_debug > 3) - printk(KERN_INFO "%s: Corrupted packet " + if (sis900_debug > 1) + printk(KERN_DEBUG "%s: Corrupted packet " "received, buffer status = 0x%8.8x.\n", net_dev->name, rx_status); sis_priv->stats.rx_errors++; @@ -1793,8 +1786,8 @@ if (tx_status & (ABORT | UNDERRUN | OWCOLL)) { /* packet unsuccessfully transmitted */ - if (sis900_debug > 3) - printk(KERN_INFO "%s: Transmit " + if (sis900_debug > 1) + printk(KERN_DEBUG "%s: Transmit " "error, Tx status %8.8x.\n", net_dev->name, tx_status); sis_priv->stats.tx_errors++; @@ -2047,12 +2040,10 @@ case IF_PORT_AUI: /* AUI */ case IF_PORT_100BASEFX: /* 100BaseFx */ /* These Modes are not supported (are they?)*/ - printk(KERN_INFO "Not supported"); return -EOPNOTSUPP; break; default: - printk(KERN_INFO "Invalid"); return -EINVAL; } } --huq684BweRXVnRxX-- --XOIedfhf+7KOe/yw Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (GNU/Linux) iD8DBQFBtt8j2rmHZCWzV+0RAmGtAKCQJtoDkJz7VUY3Ncq/bRxVLpjcSwCgsEd2 xsqL2EXJ+V8uQT8Ln6mT5uE= =r8bb -----END PGP SIGNATURE----- --XOIedfhf+7KOe/yw-- From webvenza@libero.it Wed Dec 8 03:04:52 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 08 Dec 2004 03:04:58 -0800 (PST) Received: from gateway.milesteg.arr (venza@adsl-ull-208-134.44-151.net24.it [151.44.134.208]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB8B4oIZ026034 for ; Wed, 8 Dec 2004 03:04:51 -0800 Date: Wed, 8 Dec 2004 12:04:26 +0100 From: Daniele Venzano To: NetDev , Jeff Garzik Subject: [PATCH 2/5] sis900 printk and stack usage audit Message-ID: <20041208110426.GC31707@picchio.gall.it> Mail-Followup-To: NetDev , Jeff Garzik References: <20041208104721.GA31707@picchio.gall.it> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="uxuisgdDHaNETlh8" Content-Disposition: inline In-Reply-To: <20041208104721.GA31707@picchio.gall.it> X-Operating-System: Debian GNU/Linux on kernel Linux 2.4.27-grsec X-Copyright: Forwarding or publishing without permission is prohibited. X-Truth: La vita e' una questione di culo, o ce l'hai o te lo fanno. X-GPG-Fingerprint: 642A A345 1CEF B6E3 925C 23CE DAB9 8764 25B3 57ED X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12567 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: webvenza@libero.it Precedence: bulk X-list: netdev --uxuisgdDHaNETlh8 Content-Type: multipart/mixed; boundary="vOmOzSkFvhd7u8Ms" Content-Disposition: inline --vOmOzSkFvhd7u8Ms Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Added some initialization debug output Version bump Signed-off-by: Daniele Venzano -- ----------------------------- Daniele Venzano Web: http://teg.homeunix.org --vOmOzSkFvhd7u8Ms Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="sis900-debug2.diff" Index: sis900.c =================================================================== --- a/drivers/net/sis900.c (revision 40) +++ b/drivers/net/sis900.c (revision 41) @@ -1,6 +1,6 @@ /* sis900.c: A SiS 900/7016 PCI Fast Ethernet driver for Linux. Copyright 1999 Silicon Integrated System Corporation - Revision: 1.08.06 Sep. 24 2002 + Revision: 1.08.08 Nov. 28 2004 Modified from the driver which is originally written by Donald Becker. @@ -18,6 +18,7 @@ preliminary Rev. 1.0 Jan. 18, 1998 http://www.sis.com.tw/support/databook.htm + Rev 1.08.08 Nov. 28 2004 Daniele Venzano audit debug code and printk() calls Rev 1.08.07 Nov. 2 2003 Daniele Venzano add suspend/resume support Rev 1.08.06 Sep. 24 2002 Mufasa Yang bug fix for Tx timeout & add SiS963 support Rev 1.08.05 Jun. 6 2002 Mufasa Yang bug fix for read_eeprom & Tx descriptor over-boundary @@ -74,7 +75,7 @@ #include "sis900.h" #define SIS900_MODULE_NAME "sis900" -#define SIS900_DRV_VERSION "v1.08.07 11/02/2003" +#define SIS900_DRV_VERSION "v1.08.08 Nov. 28 2004" static char version[] __devinitdata = KERN_INFO "sis900.c: " SIS900_DRV_VERSION "\n"; @@ -107,8 +108,6 @@ }; MODULE_DEVICE_TABLE (pci, sis900_pci_tbl); -static void sis900_read_mode(struct net_device *net_dev, int *speed, int *duplex); - static struct mii_chip_info { const char * name; u16 phy_id0; @@ -216,6 +215,7 @@ static u16 sis900_reset_phy(struct net_device *net_dev, int phy_addr); static void sis900_auto_negotiate(struct net_device *net_dev, int phy_addr); static void sis900_set_mode (long ioaddr, int speed, int duplex); +static void sis900_read_mode(struct net_device *net_dev, int *speed, int *duplex); static struct ethtool_ops sis900_ethtool_ops; /** @@ -269,7 +269,7 @@ if (!isa_bridge) { isa_bridge = pci_find_device(PCI_VENDOR_ID_SI, 0x0018, isa_bridge); if (!isa_bridge) { - printk(KERN_WARNING "%s: Can not find ISA bridge\n", net_dev->name); + printk(KERN_WARNING "%s: Cannot find ISA bridge\n", net_dev->name); return 0; } } @@ -455,10 +455,15 @@ if (ret) goto err_unmap_rx; + pci_read_config_byte(pci_dev, PCI_CLASS_REVISION, &revision); + + if(sis900_debug > 0) + printk(KERN_DEBUG "%s: detected revision %2.2x," + "trying to get MAC address...\n", + net_dev->name, revision); + /* Get Mac address according to the chip revision */ - pci_read_config_byte(pci_dev, PCI_CLASS_REVISION, &revision); ret = 0; - if (revision == SIS630E_900_REV) ret = sis630e_get_mac_addr(pci_dev, net_dev); else if ((revision > 0x81) && (revision <= 0x90) ) @@ -469,6 +474,7 @@ ret = sis900_get_mac_addr(pci_dev, net_dev); if (ret == 0) { + printk(KERN_WARNING "%s: Cannot read MAC address.\n", net_dev->name); ret = -ENODEV; goto err_out_unregister; } @@ -479,6 +485,7 @@ /* probe for mii transceiver */ if (sis900_mii_probe(net_dev) == 0) { + printk(KERN_WARNING "%s: Error probing MII device.\n", net_dev->name); ret = -ENODEV; goto err_out_unregister; } @@ -542,9 +549,14 @@ for(i = 0; i < 2; i++) mii_status = mdio_read(net_dev, phy_addr, MII_STATUS); - if (mii_status == 0xffff || mii_status == 0x0000) + if (mii_status == 0xffff || mii_status == 0x0000) { /* the mii is not accessible, try next one */ + if (sis900_debug > 0) + printk(KERN_DEBUG "%s: MII at address %d" + " not accessible\n", + net_dev->name, phy_addr); continue; + } if ((mii_phy = kmalloc(sizeof(struct mii_phy), GFP_KERNEL)) == NULL) { printk(KERN_WARNING "Cannot allocate mem for struct mii_phy\n"); @@ -568,14 +580,15 @@ for (i = 0; mii_chip_table[i].phy_id1; i++) if ((mii_phy->phy_id0 == mii_chip_table[i].phy_id0 ) && - ((mii_phy->phy_id1 & 0xFFF0) == mii_chip_table[i].phy_id1)){ + ((mii_phy->phy_id1 & 0xFFF0) == mii_chip_table[i].phy_id1)) + { mii_phy->phy_types = mii_chip_table[i].phy_types; if (mii_chip_table[i].phy_types == MIX) - mii_phy->phy_types = - (mii_status & (MII_STAT_CAN_TX_FDX | MII_STAT_CAN_TX)) ? LAN : HOME; + mii_phy->phy_types = (mii_status & (MII_STAT_CAN_TX_FDX | MII_STAT_CAN_TX)) ? LAN : HOME; printk(KERN_INFO "%s: %s transceiver found at address %d.\n", - net_dev->name, mii_chip_table[i].name, - phy_addr); + net_dev->name, + mii_chip_table[i].name, + phy_addr); break; } --vOmOzSkFvhd7u8Ms-- --uxuisgdDHaNETlh8 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (GNU/Linux) iD8DBQFBtt+62rmHZCWzV+0RAmfHAJwPWQeSrRvdBpuPFZS1ldYrON4b4wCglYiG o7Gxt7+sQQUgYjVLkDhV2JU= =Ikf7 -----END PGP SIGNATURE----- --uxuisgdDHaNETlh8-- From webvenza@libero.it Wed Dec 8 03:06:34 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 08 Dec 2004 03:08:05 -0800 (PST) Received: from gateway.milesteg.arr (venza@adsl-ull-208-134.44-151.net24.it [151.44.134.208]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB8B6XKf026550 for ; Wed, 8 Dec 2004 03:06:33 -0800 Date: Wed, 8 Dec 2004 12:06:10 +0100 From: Daniele Venzano To: NetDev , Jeff Garzik Subject: [PATCH 3/5] sis900 printk and stack usage audit Message-ID: <20041208110610.GD31707@picchio.gall.it> Mail-Followup-To: NetDev , Jeff Garzik References: <20041208104721.GA31707@picchio.gall.it> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="hUH5gZbnpyIv7Mn4" Content-Disposition: inline In-Reply-To: <20041208104721.GA31707@picchio.gall.it> X-Operating-System: Debian GNU/Linux on kernel Linux 2.4.27-grsec X-Copyright: Forwarding or publishing without permission is prohibited. X-Truth: La vita e' una questione di culo, o ce l'hai o te lo fanno. X-GPG-Fingerprint: 642A A345 1CEF B6E3 925C 23CE DAB9 8764 25B3 57ED X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12568 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: webvenza@libero.it Precedence: bulk X-list: netdev --hUH5gZbnpyIv7Mn4 Content-Type: multipart/mixed; boundary="9crTWz/Z+Zyzu20v" Content-Disposition: inline --9crTWz/Z+Zyzu20v Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Chip revision is now a member of sis_priv structure Kill all calls to pci_read_config_byte but one Change the code to use sis_priv->chipset_rev Signed-off-by: Daniele Venzano -- ----------------------------- Daniele Venzano Web: http://teg.homeunix.org --9crTWz/Z+Zyzu20v Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="sis900-chipset-revision.diff" Index: sis900.c =================================================================== --- a/drivers/net/sis900.c (revision 41) +++ b/drivers/net/sis900.c (revision 42) @@ -173,6 +173,7 @@ unsigned int tx_full; /* The Tx queue is full. */ u8 host_bridge_rev; + u8 chipset_rev; u32 pci_state[16]; }; @@ -387,7 +388,6 @@ void *ring_space; long ioaddr; int i, ret; - u8 revision; char *card_name = card_names[pci_id->driver_data]; /* setup various bits in PCI command register */ @@ -455,20 +455,20 @@ if (ret) goto err_unmap_rx; - pci_read_config_byte(pci_dev, PCI_CLASS_REVISION, &revision); + pci_read_config_byte(pci_dev, PCI_CLASS_REVISION, &(sis_priv->chipset_rev)); if(sis900_debug > 0) printk(KERN_DEBUG "%s: detected revision %2.2x," "trying to get MAC address...\n", - net_dev->name, revision); + net_dev->name, sis_priv->chipset_rev); /* Get Mac address according to the chip revision */ ret = 0; - if (revision == SIS630E_900_REV) + if (sis_priv->chipset_rev == SIS630E_900_REV) ret = sis630e_get_mac_addr(pci_dev, net_dev); - else if ((revision > 0x81) && (revision <= 0x90) ) + else if ((sis_priv->chipset_rev > 0x81) && (sis_priv->chipset_rev <= 0x90)) ret = sis635_get_mac_addr(pci_dev, net_dev); - else if (revision == SIS96x_900_REV) + else if (sis_priv->chipset_rev == SIS96x_900_REV) ret = sis96x_get_mac_addr(pci_dev, net_dev); else ret = sis900_get_mac_addr(pci_dev, net_dev); @@ -480,7 +480,7 @@ } /* 630ET : set the mii access mode as software-mode */ - if (revision == SIS630ET_900_REV) + if (sis_priv->chipset_rev == SIS630ET_900_REV) outl(ACCESSMODE | inl(ioaddr + cr), ioaddr + cr); /* probe for mii transceiver */ @@ -535,7 +535,6 @@ u16 poll_bit = MII_STAT_LINK, status = 0; unsigned long timeout = jiffies + 5 * HZ; int phy_addr; - u8 revision; sis_priv->mii = NULL; @@ -632,8 +631,7 @@ } } - pci_read_config_byte(sis_priv->pci_dev, PCI_CLASS_REVISION, &revision); - if (revision == SIS630E_900_REV) { + if (sis_priv->chipset_rev == SIS630E_900_REV) { /* SiS 630E has some bugs on default value of PHY registers */ mdio_write(net_dev, sis_priv->cur_phy, MII_ANADV, 0x05e1); mdio_write(net_dev, sis_priv->cur_phy, MII_CONFIG1, 0x22); @@ -948,15 +946,13 @@ { struct sis900_private *sis_priv = net_dev->priv; long ioaddr = net_dev->base_addr; - u8 revision; int ret; /* Soft reset the chip. */ sis900_reset(net_dev); /* Equalizer workaround Rule */ - pci_read_config_byte(sis_priv->pci_dev, PCI_CLASS_REVISION, &revision); - sis630_set_eq(net_dev, revision); + sis630_set_eq(net_dev, sis_priv->chipset_rev); ret = request_irq(net_dev->irq, &sis900_interrupt, SA_SHIRQ, net_dev->name, net_dev); @@ -1224,7 +1220,6 @@ struct mii_phy *mii_phy = sis_priv->mii; static int next_tick = 5*HZ; u16 status; - u8 revision; if (!sis_priv->autong_complete){ int speed, duplex = 0; @@ -1232,9 +1227,7 @@ sis900_read_mode(net_dev, &speed, &duplex); if (duplex){ sis900_set_mode(net_dev->base_addr, speed, duplex); - pci_read_config_byte(sis_priv->pci_dev, - PCI_CLASS_REVISION, &revision); - sis630_set_eq(net_dev, revision); + sis630_set_eq(net_dev, sis_priv->chipset_rev); netif_start_queue(net_dev); } @@ -1268,9 +1261,7 @@ ((mii_phy->phy_id1 & 0xFFF0) == 0x8000)) sis900_reset_phy(net_dev, sis_priv->cur_phy); - pci_read_config_byte(sis_priv->pci_dev, - PCI_CLASS_REVISION, &revision); - sis630_set_eq(net_dev, revision); + sis630_set_eq(net_dev, sis_priv->chipset_rev); goto LookForLink; } @@ -2102,11 +2093,10 @@ u16 mc_filter[16] = {0}; /* 256/128 bits multicast hash table */ int i, table_entries; u32 rx_mode; - u8 revision; /* 635 Hash Table entires = 256(2^16) */ - pci_read_config_byte(sis_priv->pci_dev, PCI_CLASS_REVISION, &revision); - if((revision >= SIS635A_900_REV) || (revision == SIS900B_900_REV)) + if((sis_priv->chipset_rev >= SIS635A_900_REV) || + (sis_priv->chipset_rev == SIS900B_900_REV)) table_entries = 16; else table_entries = 8; @@ -2132,7 +2122,7 @@ mclist && i < net_dev->mc_count; i++, mclist = mclist->next) { unsigned int bit_nr = - sis900_mcast_bitnr(mclist->dmi_addr, revision); + sis900_mcast_bitnr(mclist->dmi_addr, sis_priv->chipset_rev); mc_filter[bit_nr >> 4] |= (1 << (bit_nr & 0xf)); } } @@ -2178,7 +2168,6 @@ long ioaddr = net_dev->base_addr; int i = 0; u32 status = TxRCMP | RxRCMP; - u8 revision; outl(0, ioaddr + ier); outl(0, ioaddr + imr); @@ -2191,8 +2180,8 @@ status ^= (inl(isr + ioaddr) & status); } - pci_read_config_byte(sis_priv->pci_dev, PCI_CLASS_REVISION, &revision); - if( (revision >= SIS635A_900_REV) || (revision == SIS900B_900_REV) ) + if( (sis_priv->chipset_rev >= SIS635A_900_REV) || + (sis_priv->chipset_rev == SIS900B_900_REV) ) outl(PESEL | RND_CNT, ioaddr + cfg); else outl(PESEL, ioaddr + cfg); --9crTWz/Z+Zyzu20v-- --hUH5gZbnpyIv7Mn4 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (GNU/Linux) iD8DBQFBtuAi2rmHZCWzV+0RAsOuAJ91SdqW9b0XjtWJKTsYPMw6nLq18wCgneuq 18Au9MtLx2C5hSy0CKy0aqg= =F0z5 -----END PGP SIGNATURE----- --hUH5gZbnpyIv7Mn4-- From webvenza@libero.it Wed Dec 8 03:08:59 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 08 Dec 2004 03:09:04 -0800 (PST) Received: from gateway.milesteg.arr (venza@adsl-ull-208-134.44-151.net24.it [151.44.134.208]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB8B8wXf026901 for ; Wed, 8 Dec 2004 03:08:58 -0800 Date: Wed, 8 Dec 2004 12:08:35 +0100 From: Daniele Venzano To: NetDev , Jeff Garzik Subject: [PATCH 4/5] sis900 printk and stack usage audit Message-ID: <20041208110835.GE31707@picchio.gall.it> Mail-Followup-To: NetDev , Jeff Garzik References: <20041208104721.GA31707@picchio.gall.it> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="jaoouwwPWoQSJZYp" Content-Disposition: inline In-Reply-To: <20041208104721.GA31707@picchio.gall.it> X-Operating-System: Debian GNU/Linux on kernel Linux 2.4.27-grsec X-Copyright: Forwarding or publishing without permission is prohibited. X-Truth: La vita e' una questione di culo, o ce l'hai o te lo fanno. X-GPG-Fingerprint: 642A A345 1CEF B6E3 925C 23CE DAB9 8764 25B3 57ED X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12569 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: webvenza@libero.it Precedence: bulk X-list: netdev --jaoouwwPWoQSJZYp Content-Type: multipart/mixed; boundary="B8ONY/mu/bqBak9m" Content-Disposition: inline --B8ONY/mu/bqBak9m Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Don't do useless calls of sis630_set_eq Kill now unneeded paramenter of sis630_set_eq Signed-off-by: Daniele Venzano -- ----------------------------- Daniele Venzano Web: http://teg.homeunix.org --B8ONY/mu/bqBak9m Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="sis900-sis630-set-eq.diff" Index: sis900.c =================================================================== --- a/drivers/net/sis900.c (revision 42) +++ b/drivers/net/sis900.c (revision 43) @@ -209,7 +209,7 @@ static u16 sis900_mcast_bitnr(u8 *addr, u8 revision); static void set_rx_mode(struct net_device *net_dev); static void sis900_reset(struct net_device *net_dev); -static void sis630_set_eq(struct net_device *net_dev, u8 revision); +static void sis630_set_eq(struct net_device *net_dev); static int sis900_set_config(struct net_device *dev, struct ifmap *map); static u16 sis900_default_phy(struct net_device * net_dev); static void sis900_set_capability( struct net_device *net_dev ,struct mii_phy *phy); @@ -952,7 +952,11 @@ sis900_reset(net_dev); /* Equalizer workaround Rule */ - sis630_set_eq(net_dev, sis_priv->chipset_rev); + if (sis_priv->chipset_rev == SIS630E_900_REV || + sis_priv->chipset_rev == SIS630EA1_900_REV || + sis_priv->chipset_rev == SIS630A_900_REV || + sis_priv->chipset_rev == SIS630ET_900_REV) + sis630_set_eq(net_dev); ret = request_irq(net_dev->irq, &sis900_interrupt, SA_SHIRQ, net_dev->name, net_dev); @@ -1141,16 +1145,12 @@ * max >= 15 --> set equalizer to max+5 or set equalizer to max+6 if max == min */ -static void sis630_set_eq(struct net_device *net_dev, u8 revision) +static void sis630_set_eq(struct net_device *net_dev) { struct sis900_private *sis_priv = net_dev->priv; u16 reg14h, eq_value=0, max_value=0, min_value=0; int i, maxcount=10; - if ( !(revision == SIS630E_900_REV || revision == SIS630EA1_900_REV || - revision == SIS630A_900_REV || revision == SIS630ET_900_REV) ) - return; - if (netif_carrier_ok(net_dev)) { reg14h = mdio_read(net_dev, sis_priv->cur_phy, MII_RESV); mdio_write(net_dev, sis_priv->cur_phy, MII_RESV, @@ -1166,8 +1166,9 @@ eq_value : min_value; } /* 630E rule to determine the equalizer value */ - if (revision == SIS630E_900_REV || revision == SIS630EA1_900_REV || - revision == SIS630ET_900_REV) { + if (sis_priv->chipset_rev == SIS630E_900_REV || + sis_priv->chipset_rev == SIS630EA1_900_REV || + sis_priv->chipset_rev == SIS630ET_900_REV) { if (max_value < 5) eq_value = max_value; else if (max_value >= 5 && max_value < 15) @@ -1178,7 +1179,7 @@ max_value+6 : max_value+5; } /* 630B0&B1 rule to determine the equalizer value */ - if (revision == SIS630A_900_REV && + if (sis_priv->chipset_rev == SIS630A_900_REV && (sis_priv->host_bridge_rev == SIS630B0 || sis_priv->host_bridge_rev == SIS630B1)) { if (max_value == 0) @@ -1193,7 +1194,7 @@ mdio_write(net_dev, sis_priv->cur_phy, MII_RESV, reg14h); } else { reg14h = mdio_read(net_dev, sis_priv->cur_phy, MII_RESV); - if (revision == SIS630A_900_REV && + if (sis_priv->chipset_rev == SIS630A_900_REV && (sis_priv->host_bridge_rev == SIS630B0 || sis_priv->host_bridge_rev == SIS630B1)) mdio_write(net_dev, sis_priv->cur_phy, MII_RESV, @@ -1227,7 +1228,11 @@ sis900_read_mode(net_dev, &speed, &duplex); if (duplex){ sis900_set_mode(net_dev->base_addr, speed, duplex); - sis630_set_eq(net_dev, sis_priv->chipset_rev); + if (sis_priv->chipset_rev == SIS630E_900_REV || + sis_priv->chipset_rev == SIS630EA1_900_REV || + sis_priv->chipset_rev == SIS630A_900_REV || + sis_priv->chipset_rev == SIS630ET_900_REV) + sis630_set_eq(net_dev); netif_start_queue(net_dev); } @@ -1260,9 +1265,13 @@ if ((mii_phy->phy_id0 == 0x001D) && ((mii_phy->phy_id1 & 0xFFF0) == 0x8000)) sis900_reset_phy(net_dev, sis_priv->cur_phy); + + if (sis_priv->chipset_rev == SIS630E_900_REV || + sis_priv->chipset_rev == SIS630EA1_900_REV || + sis_priv->chipset_rev == SIS630A_900_REV || + sis_priv->chipset_rev == SIS630ET_900_REV) + sis630_set_eq(net_dev); - sis630_set_eq(net_dev, sis_priv->chipset_rev); - goto LookForLink; } } --B8ONY/mu/bqBak9m-- --jaoouwwPWoQSJZYp Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (GNU/Linux) iD8DBQFBtuCz2rmHZCWzV+0RAnSJAJ9XXuoldG7DCKmKGrTD9nDQjt/rBgCfebjo 3d5CGqu9Q38MsjW/XKFiMvE= =Dp2S -----END PGP SIGNATURE----- --jaoouwwPWoQSJZYp-- From webvenza@libero.it Wed Dec 8 03:10:34 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 08 Dec 2004 03:10:37 -0800 (PST) Received: from gateway.milesteg.arr (venza@adsl-ull-208-134.44-151.net24.it [151.44.134.208]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB8BAXM9027551 for ; Wed, 8 Dec 2004 03:10:33 -0800 Date: Wed, 8 Dec 2004 12:10:10 +0100 From: Daniele Venzano To: NetDev , Jeff Garzik Subject: [PATCH 5/5] sis900 printk and stack usage audit Message-ID: <20041208111010.GF31707@picchio.gall.it> Mail-Followup-To: NetDev , Jeff Garzik References: <20041208104721.GA31707@picchio.gall.it> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="oplxJGu+Ee5xywIT" Content-Disposition: inline In-Reply-To: <20041208104721.GA31707@picchio.gall.it> X-Operating-System: Debian GNU/Linux on kernel Linux 2.4.27-grsec X-Copyright: Forwarding or publishing without permission is prohibited. X-Truth: La vita e' una questione di culo, o ce l'hai o te lo fanno. X-GPG-Fingerprint: 642A A345 1CEF B6E3 925C 23CE DAB9 8764 25B3 57ED X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12570 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: webvenza@libero.it Precedence: bulk X-list: netdev --oplxJGu+Ee5xywIT Content-Type: multipart/mixed; boundary="tSiBuZsJmMXpnp7T" Content-Disposition: inline --tSiBuZsJmMXpnp7T Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Change comment to reflect changes in parameters od sis630_set_eq Signed-off-by: Daniele Venzano -- ----------------------------- Daniele Venzano Web: http://teg.homeunix.org --tSiBuZsJmMXpnp7T Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="sis900-sis630-docs.diff" Index: sis900.c =================================================================== --- a/drivers/net/sis900.c (revision 43) +++ b/drivers/net/sis900.c (revision 44) @@ -1121,7 +1121,6 @@ /** * sis630_set_eq - set phy equalizer value for 630 LAN * @net_dev: the net device to set equalizer value - * @revision: 630 LAN revision number * * 630E equalizer workaround rule(Cyrus Huang 08/15) * PHY register 14h(Test) --tSiBuZsJmMXpnp7T-- --oplxJGu+Ee5xywIT Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (GNU/Linux) iD8DBQFBtuES2rmHZCWzV+0RAv3BAJ9iap/qRLxiEoc0yRXdy5kSS7RdHwCeNBdH 3X4qLtiWfkFz1ADLgpxnUJc= =5Kxh -----END PGP SIGNATURE----- --oplxJGu+Ee5xywIT-- From buytenh@wantstofly.org Wed Dec 8 03:16:13 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 08 Dec 2004 03:16:17 -0800 (PST) Received: from xi.wantstofly.org (alephnull.demon.nl [212.238.201.82]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB8BGCoJ028197 for ; Wed, 8 Dec 2004 03:16:12 -0800 Received: by xi.wantstofly.org (Postfix, from userid 500) id AA1F52B0ED; Wed, 8 Dec 2004 12:15:49 +0100 (MET) Date: Wed, 8 Dec 2004 12:15:49 +0100 From: Lennert Buytenhek To: Robert Olsson Cc: Ben Greear , hadi@cyberus.ca, netdev@oss.sgi.com Subject: Re: inter-packet gap in pktgen Message-ID: <20041208111549.GA5703@xi.wantstofly.org> References: <20041207222522.GA30266@xi.wantstofly.org> <41B632F3.1090104@candelatech.com> <20041208073858.GA4027@xi.wantstofly.org> <16822.54722.755218.745451@robur.slu.se> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <16822.54722.755218.745451@robur.slu.se> User-Agent: Mutt/1.4.1i X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12571 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: buytenh@wantstofly.org Precedence: bulk X-list: netdev On Wed, Dec 08, 2004 at 11:21:54AM +0100, Robert Olsson wrote: > > Another option is: > > > > next_tx = get_time_in_ns(); > > while (--count) { > > tx_packet(); > > next_tx += 1e9/intended_pps; > > nanospin(next_tx - get_time_in_ns()); > > } > > Hello! > > I think this what Ben is doing with his userland app. Ev. adjusting > the ipg delay in runtime? A kind of control system. I think what Ben is doing is just measuring the # of pps and then using a PI-controller or something like that to adjust the ipg. What I mean is something like this (warning: whitespace damaged.) But this gives me an IPG that is consistently too small. Are the nanospin() and pg_udelay() functions accurate? --L --- pktgen.c.04111.orig 2004-12-08 11:57:03.627392497 +0100 +++ pktgen.c 2004-12-08 12:11:35.777150214 +0100 @@ -2794,7 +2794,13 @@ pkt_dev->last_ok = 0; pkt_dev->next_tx_ns = getRelativeCurNs(); /* TODO */ } - pkt_dev->next_tx_ns = getRelativeCurNs() + pkt_dev->ipg; + if (now == 0) + now = getRelativeCurNs(); + pkt_dev->next_tx_ns += pkt_dev->ipg; + if (pkt_dev->next_tx_ns < now) { + pkt_dev->next_tx_ns = now; + pkt_dev->errors++; /* missed ipg deadline */ + } } else { /* Retry it next time */ From hadi@cyberus.ca Wed Dec 8 04:05:03 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 08 Dec 2004 04:05:10 -0800 (PST) Received: from mx02.cybersurf.com (mx02.cybersurf.com [209.197.145.105]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB8C52p1003289 for ; Wed, 8 Dec 2004 04:05:02 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx02.cybersurf.com with esmtp (Exim 4.30) id 1Cc0Ye-0000gM-ON for netdev@oss.sgi.com; Wed, 08 Dec 2004 07:04:36 -0500 Received: from cpe0030ab124d2f-cm014500000962.cpe.net.cable.rogers.com ([24.103.99.32] helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1Cc0Yb-0004qI-7U; Wed, 08 Dec 2004 07:04:33 -0500 Subject: Re: Hard freeze with 2.6.10-rc3 and QoS, worked fine with 2.6.9 From: jamal Reply-To: hadi@cyberus.ca To: "David S. Miller" Cc: Patrick McHardy , tgraf@suug.ch, akpm@osdl.org, tomc@compaqnet.fr, linux-kernel@vger.kernel.org, netdev@oss.sgi.com In-Reply-To: <20041207213053.6bb602c1.davem@davemloft.net> References: <1102380430.6103.6.camel@buffy> <20041206224441.628e7885.akpm@osdl.org> <1102422544.1088.98.camel@jzny.localdomain> <41B5E188.5050800@trash.net> <20041207170748.GF1371@postel.suug.ch> <41B5E722.2080600@trash.net> <20041207213053.6bb602c1.davem@davemloft.net> Content-Type: text/plain Organization: jamalopolous Message-Id: <1102507470.1051.27.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 08 Dec 2004 07:04:30 -0500 Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12572 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev I can almost guarantee that one or more of those tests i outlined would fail. So i would suggest a revert until the testing has been done. cheers, jamal On Wed, 2004-12-08 at 00:30, David S. Miller wrote: > On Tue, 07 Dec 2004 18:23:46 +0100 > Patrick McHardy wrote: > > > Either one is fine with me, although I would prefer to see > > the number of ifdefs in this area going down, not up :) > > I agree, therefore I applied Patrick's patch. > From hadi@cyberus.ca Wed Dec 8 04:32:20 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 08 Dec 2004 04:32:25 -0800 (PST) Received: from mx02.cybersurf.com (mx02.cybersurf.com [209.197.145.105]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB8CWKgk010308 for ; Wed, 8 Dec 2004 04:32:20 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx02.cybersurf.com with esmtp (Exim 4.30) id 1Cc0z5-000386-2m for netdev@oss.sgi.com; Wed, 08 Dec 2004 07:31:55 -0500 Received: from cpe0030ab124d2f-cm014500000962.cpe.net.cable.rogers.com ([24.103.99.32] helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1Cc0z3-0008JD-V2; Wed, 08 Dec 2004 07:31:54 -0500 Subject: Re: Hard freeze with 2.6.10-rc3 and QoS, worked fine with 2.6.9 From: jamal Reply-To: hadi@cyberus.ca To: Patrick McHardy Cc: Thomas Graf , Andrew Morton , Thomas Cataldo , linux-kernel@vger.kernel.org, netdev@oss.sgi.com, "David S. Miller" In-Reply-To: <41B68E5D.2080009@trash.net> References: <1102380430.6103.6.camel@buffy> <20041206224441.628e7885.akpm@osdl.org> <1102422544.1088.98.camel@jzny.localdomain> <41B5E188.5050800@trash.net> <20041207170748.GF1371@postel.suug.ch> <41B5E722.2080600@trash.net> <1102480044.1050.9.camel@jzny.localdomain> <1102480913.1049.24.camel@jzny.localdomain> <41B68E5D.2080009@trash.net> Content-Type: text/plain Organization: jamalopolous Message-Id: <1102509111.1051.54.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 08 Dec 2004 07:31:51 -0500 Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12573 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev On Wed, 2004-12-08 at 00:17, Patrick McHardy wrote: > I think these tests are a waste of time. struct tcf_police is not > userspace-visible, so it's highly unlikely that the tc version matters. > Why an old kernel needs to be tested is beyond me. Regression testing. You need both backward and forward compatibility. Old kernels must continue to work with new tc for the policer using the old syntax. new kernels must continue to work with old tc for policer management using old syntax. Policer existed before any tc action code was written and has a very different layout of the structure. User tools and classifiers (accessed from user tools) do touch that code. These kind of tests constitute about 50% or more of my testing. > For possible in-kernel > breakage caused by the restructuring, without CONFIG_NET_CLS_ACT, > struct tcf_police is only used in police.c, without any casts or > assumptions about layout, so I can't see what could break. With > CONFIG_NET_CLS_ACT, the only place where it is used outside of > police.c is tcf_action_copy_stats, and this is exactly what this patch > (tested) fixes. > > If you still want to do these test, please use the attached patch. No rush now that its in (I also dont have time or equipment at the moment). Lets hope no more freezes reported. When i get time i will look into it. cheers, jamal From mumasankar@novell.com Wed Dec 8 04:46:17 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 08 Dec 2004 04:46:24 -0800 (PST) Received: from lucius.provo.novell.com (lucius.provo.novell.com [137.65.81.172]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB8CkHsM014107 for ; Wed, 8 Dec 2004 04:46:17 -0800 Received: from INET-PRV1-MTA by lucius.provo.novell.com with Novell_GroupWise; Wed, 08 Dec 2004 05:45:50 -0700 Message-Id: X-Mailer: Novell GroupWise Internet Agent 6.5.3 Date: Wed, 08 Dec 2004 05:45:35 -0700 From: "Umasankar Mukkara" To: Subject: IPSEC MIB instrumentatoin ?? Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12574 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: mumasankar@novell.com Precedence: bulk X-list: netdev Hello, Native ipsec stack lacks proc file instrumentation for IPSEC statistics. Would a patch for this instrumentation be looked as a good one? Following can be the ipsec related counters in ESP4/AH4/ ESP6/AH6/IPCOMP -Packets Encrypted -Packets decypted -Packets discarded (Policy enforcement) -Replay packets etc etc. Or something closer to "draft-ietf-ipsec-monitor-mib-06.txt " which is an expired draft and can be found at http://www.tatsuyababa.com/internet-drafts/draft-ietf-ipsec-monitor-mib-06.txt - Uma. From ilya.pashkovsky@gmail.com Wed Dec 8 04:56:53 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 08 Dec 2004 04:56:59 -0800 (PST) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.198]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB8Cur7G019455 for ; Wed, 8 Dec 2004 04:56:53 -0800 Received: by rproxy.gmail.com with SMTP id b11so349543rne for ; Wed, 08 Dec 2004 04:56:31 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:mime-version:content-type:content-transfer-encoding; b=XLw0tmuPx2Cy2M90VDsljvg22KkL1PseAhNw1HSguhEu0xqZHx5aoP0xFM6RRnL3aPfRRW2n0854UGaXbFRvgzuJthBuFADd5FbiHrgg6xpR6Lz4S5kTT6tiKCxnnmyfsBH3r6TYQIDeZt9ETGOg7nld4FH95Xz/lhOxRRIR30Q= Received: by 10.38.162.30 with SMTP id k30mr22740rne; Wed, 08 Dec 2004 04:56:31 -0800 (PST) Received: by 10.38.149.15 with HTTP; Wed, 8 Dec 2004 04:56:31 -0800 (PST) Message-ID: Date: Wed, 8 Dec 2004 14:56:31 +0200 From: Ilya Pashkovsky Reply-To: Ilya Pashkovsky To: davem@redhat.com, netdev@oss.sgi.com Subject: [PATCH] sk_reuse fixes Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12575 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: ilya.pashkovsky@gmail.com Precedence: bulk X-list: netdev This fixes sk_reuse checks: 1) allow outgoing connections AND one listening socket bound to same source port. 2) remove > 1 check of a boolean variable http://puding.mine.nu/patches/patch-reuse-bool --- linux/net/ipv4/tcp_ipv4.c.orig 2004-12-07 14:54:12.597084704 +0200 +++ linux/net/ipv4/tcp_ipv4.c 2004-12-08 14:47:27.792827816 +0200 @@ -50,6 +50,8 @@ * YOSHIFUJI Hideaki @USAGI and: Support IPV6_V6ONLY socket option, which * Alexey Kuznetsov allow both IPv4 and IPv6 sockets to bind * a single port at the same time. + * Ilya Pashkovsky : fix TCP_LISTEN check on reuse + * remove (sk_reuse > 1) check in get_port */ #include @@ -184,7 +186,8 @@ static inline int tcp_bind_conflict(stru const u32 sk_rcv_saddr = tcp_v4_rcv_saddr(sk); struct sock *sk2; struct hlist_node *node; - int reuse = sk->sk_reuse; + unsigned char reuse = sk->sk_reuse; + unsigned char state = sk->sk_state; sk_for_each_bound(sk2, node, &tb->owners) { if (sk != sk2 && @@ -193,7 +196,7 @@ static inline int tcp_bind_conflict(stru !sk2->sk_bound_dev_if || sk->sk_bound_dev_if == sk2->sk_bound_dev_if)) { if (!reuse || !sk2->sk_reuse || - sk2->sk_state == TCP_LISTEN) { + (state == TCP_LISTEN && sk2->sk_state == TCP_LISTEN)) { const u32 sk2_rcv_saddr = tcp_v4_rcv_saddr(sk2); if (!sk2_rcv_saddr || !sk_rcv_saddr || sk2_rcv_saddr == sk_rcv_saddr) @@ -259,8 +262,14 @@ static int tcp_v4_get_port(struct sock * goto tb_not_found; tb_found: if (!hlist_empty(&tb->owners)) { - if (sk->sk_reuse > 1) - goto success; + + /* + * sk_reuse is boolean + * + *if (sk->sk_reuse > 1) + * goto success; + */ + if (tb->fastreuse > 0 && sk->sk_reuse && sk->sk_state != TCP_LISTEN) { goto success; From kdesler@soohrt.org Wed Dec 8 05:09:13 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 08 Dec 2004 05:09:17 -0800 (PST) Received: from quickstop.soohrt.org (quickstop.soohrt.org [81.2.155.147]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB8D9B3d020211 for ; Wed, 8 Dec 2004 05:09:12 -0800 Received: (qmail 5857 invoked by uid 1000); 8 Dec 2004 13:08:45 -0000 Date: Wed, 8 Dec 2004 14:08:45 +0100 From: Karsten Desler To: Willy Tarreau Cc: P@draigBrady.com, "David S. Miller" , netdev@oss.sgi.com, linux-kernel@vger.kernel.org Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets Message-ID: <20041208130845.GA5036@soohrt.org> References: <20041206205305.GA11970@soohrt.org> <20041206134849.498bfc93.davem@davemloft.net> <20041206224107.GA8529@soohrt.org> <41B58A58.8010007@draigBrady.com> <20041207112139.GA3610@soohrt.org> <20041208053953.GC17946@alpha.home.local> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: <20041208053953.GC17946@alpha.home.local> User-Agent: Mutt/1.5.6+20040722i X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12576 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kdesler@soohrt.org Precedence: bulk X-list: netdev * Willy Tarreau wrote: > On Tue, Dec 07, 2004 at 12:21:39PM +0100, Karsten Desler wrote: > > > > I also notice that a lot of time is spent allocating > > > and freeing the packet buffers (and possible hidden > > > time due to cache misses due to allocating on one > > > CPU and freeing on another?). > > > How many [RT]xDescriptors do you have configured by the way? > > > > 256. I increased them to 1024 shortly after the profiling run, but > > didn't notice any change in the cpu usage (will try again with cyclesoak). > > Have you checked the interrupts rate ? I had an e1000 eating many CPU cycles > because it would generate 50000 interrupts/s. Passing the module > InterruptThrottleRate=5000 definitely calmed it down, and more than doubled > the data rate. I was running mit ITR=3000, but as a test to see if NAPI works, I disabled ITR on eth0 bringing the int/s rate up to 50k. Is that normal? I always though NAPI was supposed to kick in way earlier. Anyways, I'm going to try different ITR settings to see if they make any difference. Cheers, Karsten From yoshfuji@linux-ipv6.org Wed Dec 8 05:12:58 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 08 Dec 2004 05:13:02 -0800 (PST) Received: from yue.st-paulia.net (yue.linux-ipv6.org [203.178.140.15]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB8DCv9G020762 for ; Wed, 8 Dec 2004 05:12:58 -0800 Received: from localhost (localhost [127.0.0.1]) by yue.st-paulia.net (Postfix) with ESMTP id 3484033CE5; Wed, 8 Dec 2004 22:14:10 +0900 (JST) Date: Wed, 08 Dec 2004 14:14:09 +0100 (CET) Message-Id: <20041208.141409.103464255.yoshfuji@linux-ipv6.org> To: ilya.pashkovsky@gmail.com Cc: davem@redhat.com, netdev@oss.sgi.com, yoshfuji@linux-ipv6.org Subject: Re: [PATCH] sk_reuse fixes From: YOSHIFUJI Hideaki / =?iso-2022-jp?B?GyRCNUhGIzFRTEAbKEI=?= In-Reply-To: References: Organization: USAGI Project X-URL: http://www.yoshifuji.org/%7Ehideaki/ X-Fingerprint: 9022 65EB 1ECF 3AD1 0BDF 80D8 4807 F894 E062 0EEA X-PGP-Key-URL: http://www.yoshifuji.org/%7Ehideaki/hideaki@yoshifuji.org.asc X-Face: "5$Al-.M>NJ%a'@hhZdQm:."qn~PA^gq4o*>iCFToq*bAi#4FRtx}enhuQKz7fNqQz\BYU] $~O_5m-9'}MIs`XGwIEscw;e5b>n"B_?j/AkL~i/MEaZBLP X-Mailer: Mew version 2.2 on Emacs 20.7 / Mule 4.1 (AOI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12577 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: yoshfuji@linux-ipv6.org Precedence: bulk X-list: netdev Hello. In article (at Wed, 8 Dec 2004 14:56:31 +0200), Ilya Pashkovsky says: > This fixes sk_reuse checks: > 1) allow outgoing connections AND one listening socket bound to same > source port. > 2) remove > 1 check of a boolean variable Two comments: 1. This (boolean variable) is not the case on Linux. Behavior is differentiated with sk_reuse > 1. 2. Please provide tcp_ipv6 part as well. --yoshfuji From ilya.pashkovsky@gmail.com Wed Dec 8 05:13:22 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 08 Dec 2004 05:13:28 -0800 (PST) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.204]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB8DDLfV020960 for ; Wed, 8 Dec 2004 05:13:22 -0800 Received: by rproxy.gmail.com with SMTP id b11so351434rne for ; Wed, 08 Dec 2004 05:13:00 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:mime-version:content-type:content-transfer-encoding; b=fNpDhJCnnO9sjtwJ9ifhKTCei5wtx6VM8+m/I/jDdUcLi4qcc+2XvjaZIU3lXbTzdOch9tlOM+oF6D+6JaOWNdhaOH8fJBaS2dCjoNAJm+a5tN4ZtR8cfvMPebPSORcXoWpTbKxyJxtqYyQNhFuNHjwsqNst4OInzcuj0XW59zU= Received: by 10.38.12.13 with SMTP id 13mr364259rnl; Wed, 08 Dec 2004 05:12:59 -0800 (PST) Received: by 10.38.149.15 with HTTP; Wed, 8 Dec 2004 05:12:59 -0800 (PST) Message-ID: Date: Wed, 8 Dec 2004 15:12:59 +0200 From: Ilya Pashkovsky Reply-To: Ilya Pashkovsky To: davem@redhat.com, netdev@oss.sgi.com Subject: [PATCH] fixed patch for sk_reuse Cc: linux-kernel@vger.kernel.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12578 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: ilya.pashkovsky@gmail.com Precedence: bulk X-list: netdev ah, it seems the sk_reuse > 1 is really used... do not try the previous patch. http://puding.mine.nu/patches/patch-reuse --- linux/net/ipv4/tcp_ipv4.c.orig2004-12-07 14:54:12.597084704 +0200 +++ linux/net/ipv4/tcp_ipv4.c2004-12-08 15:07:51.985722208 +0200 @@ -50,6 +50,7 @@ *YOSHIFUJI Hideaki @USAGI and:Support IPV6_V6ONLY socket option, which *Alexey Kuznetsovallow both IPv4 and IPv6 sockets to bind *a single port at the same time. + *Ilya Pashkovsky:fix TCP_LISTEN check on reuse */ #include @@ -184,7 +185,8 @@ static inline int tcp_bind_conflict(stru const u32 sk_rcv_saddr = tcp_v4_rcv_saddr(sk); struct sock *sk2; struct hlist_node *node; -int reuse = sk->sk_reuse; +unsigned char reuse = sk->sk_reuse; +unsigned char state = sk->sk_state; sk_for_each_bound(sk2, node, &tb->owners) { if (sk != sk2 && @@ -193,7 +195,7 @@ static inline int tcp_bind_conflict(stru !sk2->sk_bound_dev_if || sk->sk_bound_dev_if == sk2->sk_bound_dev_if)) { if (!reuse || !sk2->sk_reuse || - sk2->sk_state == TCP_LISTEN) { + (state == TCP_LISTEN && sk2->sk_state == TCP_LISTEN)) { const u32 sk2_rcv_saddr = tcp_v4_rcv_saddr(sk2); if (!sk2_rcv_saddr || !sk_rcv_saddr || sk2_rcv_saddr == sk_rcv_saddr) From kdesler@soohrt.org Wed Dec 8 05:27:24 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 08 Dec 2004 05:27:28 -0800 (PST) Received: from quickstop.soohrt.org (quickstop.soohrt.org [81.2.155.147]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB8DRMrj022561 for ; Wed, 8 Dec 2004 05:27:23 -0800 Received: (qmail 9640 invoked by uid 1000); 8 Dec 2004 13:26:57 -0000 Date: Wed, 8 Dec 2004 14:26:57 +0100 From: Karsten Desler To: jamal , Robert Olsson Cc: netdev@oss.sgi.com, linux-kernel@vger.kernel.org, "David S. Miller" , P@draigBrady.com Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets Message-ID: <20041208132657.GB5036@soohrt.org> References: <20041206205305.GA11970@soohrt.org> <20041207211035.GA20286@quickstop.soohrt.org> <1102480318.1050.16.camel@jzny.localdomain> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: <1102480318.1050.16.camel@jzny.localdomain> User-Agent: Mutt/1.5.6+20040722i X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12579 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kdesler@soohrt.org Precedence: bulk X-list: netdev * jamal wrote: > On Tue, 2004-12-07 at 16:10, Karsten Desler wrote: > > I totally forgot to mention: There are approximately 100k concurrent > > flows. > > ;-> Aha. That would make a huge difference. I know of noone > who has actually done this level of testing. I have tried upto about 50K > flows myself in early 2.6.x and was eventually compute bound. > Really try compiling out totaly iptables/netfilter - it will make a > difference. Unfortunately I need netfilter (for now, I haven't had time to look into replacing the rules with tc yet). > You may also want to try something like LC trie algorithm that Robert > and co are playing with to see if it makes a difference with this many > flows. On a scale of one to ten, one being "will crash on the first packet", five being "will allow moderate load, but is probably going to crash under high load" and ten being "rock stable" how would the patch be rated? The announcement doesn't look too promising: | Locking. | Not yet done. Also when looking at the profiles, fib_* isn't showing up at (not even near) the top. Testworthy none the less? ip route|wc -l: 40 profile: 4 fib_validate_source 0.0064 39 fib_semantic_match 0.1875 [...] 76 ipt_route_hook 1.5833 219 __kmalloc 1.7109 985 e1000_clean_tx_irq 1.8107 157 kmem_cache_alloc 1.9625 405 skb_release_data 2.5312 633 eth_type_trans 2.6375 394 handle_IRQ_event 3.5179 1017 __kfree_skb 3.9727 989 alloc_skb 4.1208 1645 ip_route_input 4.6733 5128 ipt_do_table 6.1635 1678 e1000_intr 6.9917 289 _read_unlock_bh 18.0625 616 _read_lock_bh 19.2500 418 _spin_lock 26.1250 2076 e1000_irq_enable 43.2500 881 _spin_unlock_irqrestore 55.0625 96895 default_idle 1513.9844 Cheers, Karsten From hadi@cyberus.ca Wed Dec 8 05:27:50 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 08 Dec 2004 05:27:54 -0800 (PST) Received: from mx01.cybersurf.com (mx01.cybersurf.com [209.197.145.104]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB8DRn6N022657 for ; Wed, 8 Dec 2004 05:27:50 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx01.cybersurf.com with esmtp (Exim 4.30) id 1Cc1qk-000146-NM for netdev@oss.sgi.com; Wed, 08 Dec 2004 08:27:22 -0500 Received: from cpe0030ab124d2f-cm014500000962.cpe.net.cable.rogers.com ([24.103.99.32] helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1Cc1qi-0007C4-Sm; Wed, 08 Dec 2004 08:27:21 -0500 Subject: Re: _High_ CPU usage while routing (mostly) small UDP packets From: jamal Reply-To: hadi@cyberus.ca To: Karsten Desler Cc: Willy Tarreau , P@draigBrady.com, "David S. Miller" , netdev@oss.sgi.com, linux-kernel@vger.kernel.org In-Reply-To: <20041208130845.GA5036@soohrt.org> References: <20041206205305.GA11970@soohrt.org> <20041206134849.498bfc93.davem@davemloft.net> <20041206224107.GA8529@soohrt.org> <41B58A58.8010007@draigBrady.com> <20041207112139.GA3610@soohrt.org> <20041208053953.GC17946@alpha.home.local> <20041208130845.GA5036@soohrt.org> Content-Type: text/plain Organization: jamalopolous Message-Id: <1102512438.1050.61.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 08 Dec 2004 08:27:18 -0500 Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12580 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev On Wed, 2004-12-08 at 08:08, Karsten Desler wrote: > I was running mit ITR=3000, but as a test to see if NAPI works, I > disabled ITR on eth0 bringing the int/s rate up to 50k. > Is that normal? I always though NAPI was supposed to kick in way earlier. > Anyways, I'm going to try different ITR settings to see if they make any > difference. > The one time you would need this ITR crap is when you are running low traffic (relatively speaking: which could mean anything below 100Kpps depending on h/ware). NAPI will consume a little more CPU otherwise - given it does an extra IO (if it kicks in and out on every packet or two). Granted you are doing some new types of tests, so you may be seeing some things we havent experienced before. Can you put a printk in ->open() of e1000 with #if napi is defined printk("%s: NAPI is on\n",dev->name); #endif This should print on ifconfig up and indicate whether NAPI is on or not. cheers, jamal From tgraf@suug.ch Wed Dec 8 06:32:18 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 08 Dec 2004 06:32:25 -0800 (PST) Received: from b.mx.projectdream.org (eth0-0.arisu.projectdream.org [194.158.4.191]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB8EWGbS026108 for ; Wed, 8 Dec 2004 06:32:17 -0800 Received: from postel.suug.ch (unknown [195.134.158.23]) (using TLSv1 with cipher EDH-RSA-DES-CBC3-SHA (168/168 bits)) (No client certificate requested) by b.mx.projectdream.org (Postfix) with ESMTP id AAEC1F; Wed, 8 Dec 2004 15:31:30 +0100 (CET) Received: by postel.suug.ch (Postfix, from userid 10001) id 9FFC61C0EA; Wed, 8 Dec 2004 15:32:12 +0100 (CET) Date: Wed, 8 Dec 2004 15:32:12 +0100 From: Thomas Graf To: jamal Cc: Patrick McHardy , Andrew Morton , Thomas Cataldo , linux-kernel@vger.kernel.org, netdev@oss.sgi.com, "David S. Miller" Subject: Re: Hard freeze with 2.6.10-rc3 and QoS, worked fine with 2.6.9 Message-ID: <20041208143212.GL1371@postel.suug.ch> References: <1102380430.6103.6.camel@buffy> <20041206224441.628e7885.akpm@osdl.org> <1102422544.1088.98.camel@jzny.localdomain> <41B5E188.5050800@trash.net> <20041207170748.GF1371@postel.suug.ch> <41B5E722.2080600@trash.net> <1102480044.1050.9.camel@jzny.localdomain> <1102480913.1049.24.camel@jzny.localdomain> <41B68E5D.2080009@trash.net> <1102509111.1051.54.camel@jzny.localdomain> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1102509111.1051.54.camel@jzny.localdomain> X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12581 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: tgraf@suug.ch Precedence: bulk X-list: netdev * jamal <1102509111.1051.54.camel@jzny.localdomain> 2004-12-08 07:31 > On Wed, 2004-12-08 at 00:17, Patrick McHardy wrote: > > > I think these tests are a waste of time. struct tcf_police is not > > userspace-visible, so it's highly unlikely that the tc version matters. > > Why an old kernel needs to be tested is beyond me. > > Regression testing. > You need both backward and forward compatibility. > Old kernels must continue to work with new tc for the policer using the > old syntax. > new kernels must continue to work with old tc for policer management > using old syntax. > Policer existed before any tc action code was written and has a very > different layout of the structure. User tools and classifiers (accessed > from user tools) do touch that code. > These kind of tests constitute about 50% or more of my testing. I invested some time to ease testing since this was primarly my fault by overlooking the special case of tcf_police. I've put together a small testsuite allowing to easly run tests for multiple versions of iproute2. It can be found at: http://people.suug.ch/~tgr/iproute2/tc-testsuite.tar.gz One simply extracts various iproute2 versions into iproute2/ and sets KERNEL_INCLUDE if needed for older versions. 'make compile' on the top level compiles all the versions. The tests are defined in tests/ and are simple shell scripts and get invoked for every iproute2 verison in iproute2 with $TC and $IP set to the version currently being tested. The output of every test run is stored in results/$TEST.$IPVERSION.out respectively .dmesg. 'make clean' removes all the results again. 'make liststests' lists all the available tests. 'make alltests' runs all the tests. I've run all the tests on my patch with the following kernels and iproute2 versions: - 2.6.10-rc2-bk13 (actions compiled in) - 2.6.10-rc2-bk13-no-act (old policer compiled in) - 2.4.28-rc1-bk1 - iproute2-2.6.9-tgr (with all my patches in) - iproute2-2.4.7 iproute-2.6.9 was sucessful with all kernels. I couldn't test with the old 2.4.7 iproute2 yet since the syntax has changed and I need to adopt the tests first. I will create better tests and run it on patrick's patch when I get home. From tgraf@suug.ch Wed Dec 8 06:59:28 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 08 Dec 2004 06:59:33 -0800 (PST) Received: from b.mx.projectdream.org (eth0-0.arisu.projectdream.org [194.158.4.191]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB8ExRm7028341 for ; Wed, 8 Dec 2004 06:59:27 -0800 Received: from postel.suug.ch (unknown [195.134.158.23]) (using TLSv1 with cipher EDH-RSA-DES-CBC3-SHA (168/168 bits)) (No client certificate requested) by b.mx.projectdream.org (Postfix) with ESMTP id 86C28F; Wed, 8 Dec 2004 15:58:42 +0100 (CET) Received: by postel.suug.ch (Postfix, from userid 10001) id 9ED481C0EA; Wed, 8 Dec 2004 15:59:25 +0100 (CET) Date: Wed, 8 Dec 2004 15:59:25 +0100 From: Thomas Graf To: jamal Cc: Patrick McHardy , Andrew Morton , Thomas Cataldo , linux-kernel@vger.kernel.org, netdev@oss.sgi.com, "David S. Miller" Subject: Re: Hard freeze with 2.6.10-rc3 and QoS, worked fine with 2.6.9 Message-ID: <20041208145925.GM1371@postel.suug.ch> References: <20041206224441.628e7885.akpm@osdl.org> <1102422544.1088.98.camel@jzny.localdomain> <41B5E188.5050800@trash.net> <20041207170748.GF1371@postel.suug.ch> <41B5E722.2080600@trash.net> <1102480044.1050.9.camel@jzny.localdomain> <1102480913.1049.24.camel@jzny.localdomain> <41B68E5D.2080009@trash.net> <1102509111.1051.54.camel@jzny.localdomain> <20041208143212.GL1371@postel.suug.ch> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20041208143212.GL1371@postel.suug.ch> X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12582 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: tgraf@suug.ch Precedence: bulk X-list: netdev * Thomas Graf <20041208143212.GL1371@postel.suug.ch> 2004-12-08 15:32 > iproute-2.6.9 was sucessful with all kernels. I couldn't test with the > old 2.4.7 iproute2 yet since the syntax has changed and I need to adopt > the tests first. I will create better tests and run it on patrick's > patch when I get home. I've updated the tests and all were successful: tc-2.6.9-tgr tc-2.4.7 2.6.10-rc2-bk13* Y Y 2.6.10-rc2-bk13-no-act* Y Y 2.4.28-rc1-bk1 Y Y * including tcf_police patch From hadi@cyberus.ca Wed Dec 8 07:05:44 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 08 Dec 2004 07:05:48 -0800 (PST) Received: from mx02.cybersurf.com (mx02.cybersurf.com [209.197.145.105]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB8F5hWl029001 for ; Wed, 8 Dec 2004 07:05:44 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx02.cybersurf.com with esmtp (Exim 4.30) id 1Cc3NW-0006r6-Q3 for netdev@oss.sgi.com; Wed, 08 Dec 2004 10:05:18 -0500 Received: from [216.209.86.2] (helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1Cc3NM-0005fu-B3; Wed, 08 Dec 2004 10:05:08 -0500 Subject: Re: Hard freeze with 2.6.10-rc3 and QoS, worked fine with 2.6.9 From: jamal Reply-To: hadi@cyberus.ca To: Thomas Graf Cc: Patrick McHardy , Andrew Morton , Thomas Cataldo , linux-kernel@vger.kernel.org, netdev@oss.sgi.com, "David S. Miller" In-Reply-To: <20041208143212.GL1371@postel.suug.ch> References: <1102380430.6103.6.camel@buffy> <20041206224441.628e7885.akpm@osdl.org> <1102422544.1088.98.camel@jzny.localdomain> <41B5E188.5050800@trash.net> <20041207170748.GF1371@postel.suug.ch> <41B5E722.2080600@trash.net> <1102480044.1050.9.camel@jzny.localdomain> <1102480913.1049.24.camel@jzny.localdomain> <41B68E5D.2080009@trash.net> <1102509111.1051.54.camel@jzny.localdomain> <20041208143212.GL1371@postel.suug.ch> Content-Type: text/plain Organization: jamalopolous Message-Id: <1102518304.1023.6.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 08 Dec 2004 10:05:05 -0500 Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12583 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev On Wed, 2004-12-08 at 09:32, Thomas Graf wrote: > I've put together a small testsuite allowing to easly run tests for > multiple versions of iproute2. It can be found at: > http://people.suug.ch/~tgr/iproute2/tc-testsuite.tar.gz > Good stuff. Hopefully we can run these tests everytime we make changes going forward. We cant compromise quality by handwaving on instinct. Famous last words: "that couldnt have possibly caused a bug down there". I will take a look at what you have and integrate my 20-30 testcases if they are not covered over time - or may be adpat what you have to follow how i do things or maybe i can send them to you and you can integrate them. > iproute-2.6.9 was sucessful with all kernels. I couldn't test with the > old 2.4.7 iproute2 yet since the syntax has changed and I need to adopt > the tests first. I will create better tests and run it on patrick's > patch when I get home. That would be appreaciated. cheers, jamal From hadi@cyberus.ca Wed Dec 8 07:07:14 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 08 Dec 2004 07:07:18 -0800 (PST) Received: from mx02.cybersurf.com (mx02.cybersurf.com [209.197.145.105]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB8F7EG4029475 for ; Wed, 8 Dec 2004 07:07:14 -0800 Received: from mail.cyberus.ca ([209.197.145.21]) by mx02.cybersurf.com with esmtp (Exim 4.30) id 1Cc3Oz-0007V8-GK for netdev@oss.sgi.com; Wed, 08 Dec 2004 10:06:49 -0500 Received: from [216.209.86.2] (helo=[10.0.0.9]) by mail.cyberus.ca with esmtp (Exim 4.20) id 1Cc3Oy-0005xA-Ez; Wed, 08 Dec 2004 10:06:48 -0500 Subject: Re: Hard freeze with 2.6.10-rc3 and QoS, worked fine with 2.6.9 From: jamal Reply-To: hadi@cyberus.ca To: Thomas Graf Cc: Patrick McHardy , Andrew Morton , Thomas Cataldo , linux-kernel@vger.kernel.org, netdev@oss.sgi.com, "David S. Miller" In-Reply-To: <20041208145925.GM1371@postel.suug.ch> References: <20041206224441.628e7885.akpm@osdl.org> <1102422544.1088.98.camel@jzny.localdomain> <41B5E188.5050800@trash.net> <20041207170748.GF1371@postel.suug.ch> <41B5E722.2080600@trash.net> <1102480044.1050.9.camel@jzny.localdomain> <1102480913.1049.24.camel@jzny.localdomain> <41B68E5D.2080009@trash.net> <1102509111.1051.54.camel@jzny.localdomain> <20041208143212.GL1371@postel.suug.ch> <20041208145925.GM1371@postel.suug.ch> Content-Type: text/plain Organization: jamalopolous Message-Id: <1102518405.1025.8.camel@jzny.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 08 Dec 2004 10:06:45 -0500 Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12584 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev Thanks for your efforts Thomas. cheers, jamal On Wed, 2004-12-08 at 09:59, Thomas Graf wrote: > * Thomas Graf <20041208143212.GL1371@postel.suug.ch> 2004-12-08 15:32 > > iproute-2.6.9 was sucessful with all kernels. I couldn't test with the > > old 2.4.7 iproute2 yet since the syntax has changed and I need to adopt > > the tests first. I will create better tests and run it on patrick's > > patch when I get home. > > I've updated the tests and all were successful: > > tc-2.6.9-tgr tc-2.4.7 > 2.6.10-rc2-bk13* Y Y > 2.6.10-rc2-bk13-no-act* Y Y > 2.4.28-rc1-bk1 Y Y > > * including tcf_police patch > From jmorris@redhat.com Wed Dec 8 08:56:34 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 08 Dec 2004 08:56:40 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com [66.187.233.31]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB8GuXRf005766 for ; Wed, 8 Dec 2004 08:56:34 -0800 Received: from int-mx1.corp.redhat.com (int-mx1.corp.redhat.com [172.16.52.254]) by mx1.redhat.com (8.12.11/8.12.11) with ESMTP id iB8Gu3K4021881; Wed, 8 Dec 2004 11:56:03 -0500 Received: from mail.boston.redhat.com (mail.boston.redhat.com [172.16.76.12]) by int-mx1.corp.redhat.com (8.11.6/8.11.6) with ESMTP id iB8Gu3r16911; Wed, 8 Dec 2004 11:56:03 -0500 Received: from thoron.boston.redhat.com (thoron.boston.redhat.com [172.16.80.63]) by mail.boston.redhat.com (8.12.8/8.12.8) with ESMTP id iB8Gu1ZV029721; Wed, 8 Dec 2004 11:56:01 -0500 Date: Wed, 8 Dec 2004 11:56:03 -0500 (EST) From: James Morris X-X-Sender: jmorris@thoron.boston.redhat.com To: Umasankar Mukkara cc: netdev@oss.sgi.com Subject: Re: IPSEC MIB instrumentatoin ?? In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12585 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jmorris@redhat.com Precedence: bulk X-list: netdev On Wed, 8 Dec 2004, Umasankar Mukkara wrote: > Or something closer to "draft-ietf-ipsec-monitor-mib-06.txt " which is > an expired draft and can be found at > > http://www.tatsuyababa.com/internet-drafts/draft-ietf-ipsec-monitor-mib-06.txt > Any idea what happened to the draft process? Is it likely to be re-issued and turned into an RFC? - James -- James Morris From kaber@trash.net Wed Dec 8 08:58:16 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 08 Dec 2004 08:58:21 -0800 (PST) Received: from kaber.coreworks.de ([62.206.217.67]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB8GwFGQ006097 for ; Wed, 8 Dec 2004 08:58:16 -0800 Received: from localhost ([127.0.0.1]) by kaber.coreworks.de with esmtp (Exim 4.34) id 1Cc57j-0002Ut-PZ; Wed, 08 Dec 2004 17:57:07 +0100 Message-ID: <41B73263.4040706@trash.net> Date: Wed, 08 Dec 2004 17:57:07 +0100 From: Patrick McHardy User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.7.3) Gecko/20041008 Debian/1.7.3-5 X-Accept-Language: en MIME-Version: 1.0 To: hadi@cyberus.ca CC: "David S. Miller" , tgraf@suug.ch, akpm@osdl.org, tomc@compaqnet.fr, linux-kernel@vger.kernel.org, netdev@oss.sgi.com Subject: Re: Hard freeze with 2.6.10-rc3 and QoS, worked fine with 2.6.9 References: <1102380430.6103.6.camel@buffy> <20041206224441.628e7885.akpm@osdl.org> <1102422544.1088.98.camel@jzny.localdomain> <41B5E188.5050800@trash.net> <20041207170748.GF1371@postel.suug.ch> <41B5E722.2080600@trash.net> <20041207213053.6bb602c1.davem@davemloft.net> <1102507470.1051.27.camel@jzny.localdomain> In-Reply-To: <1102507470.1051.27.camel@jzny.localdomain> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12586 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kaber@trash.net Precedence: bulk X-list: netdev jamal wrote: >I can almost guarantee that one or more of those tests i outlined would >fail. So i would suggest a revert until the testing has been done. > Please be more specific than an "almost guarantee" that "one or more tests" may fail when asking to revert a patch that fixes an easily triggerable crash. For example, point to the code that makes you think it might fail. From tgraf@suug.ch Wed Dec 8 09:30:31 2004 Received: with ECARTIS (v1.0.0; list netdev); Wed, 08 Dec 2004 09:30:38 -0800 (PST) Received: from b.mx.projectdream.org (eth0-0.arisu.projectdream.org [194.158.4.191]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id iB8HUUrn007635 for ; Wed, 8 Dec 2004 09:30:31 -0800 Received: from postel.suug.ch (unknown [195.134.158.23]) (using TLSv1 with cipher EDH-RSA-DES-CBC3-SHA (168/168 bits)) (No client certificate requested) by b.mx.projectdream.org (Postfix) with ESMTP id 1826AF; Wed, 8 Dec 2004 18:29:46 +0100 (CET) Received: by postel.suug.ch (Postfix, from userid 10001) id BA6EA1C0EA; Wed, 8 Dec 2004 18:30:28 +0100 (CET) Date: Wed, 8 Dec 2004 18:30:28 +0100 From: Thomas Graf To: jamal Cc: Patrick McHardy , Andrew Morton , Thomas Cataldo , linux-kernel@vger.kernel.org, netdev@oss.sgi.com, "David S. Miller" Subject: Re: Hard freeze with 2.6.10-rc3 and QoS, worked fine with 2.6.9 Message-ID: <20041208173028.GN1371@postel.suug.ch> References: <1102422544.1088.98.camel@jzny.localdomain> <41B5E188.5050800@trash.net> <20041207170748.GF1371@postel.suug.ch> <41B5E722.2080600@trash.net> <1102480044.1050.9.camel@jzny.localdomain> <1102480913.1049.24.camel@jzny.localdomain> <41B68E5D.2080009@trash.net> <1102509111.1051.54.camel@jzny.localdomain> <20041208143212.GL1371@postel.suug.ch> <1102518304.1023.6.camel@jzny.localdomain> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1102518304.1023.6.camel@jzny.localdomain> X-Virus-Scanned: ClamAV 0.80/533/Sat Oct 16 18:09:44 2004 clamav-milter version 0.80j on 127.0.0.1 X-Virus-Status: Clean X-archive-position: 12587 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: tgraf@suug.ch Precedence: bulk X-list: netdev * jamal <1102518304.1023.6.camel@jzny.localdomain> 2004-12-08 10:05 > On Wed, 2004-12-08 at 09:32, Thomas Graf wrote: > > > I've put together a small testsuite allowing to easly run tests for > > multiple versions of iproute2. It can be found at: > > http://people.suug.ch/~tgr/iproute2/tc-testsuite.tar.gz > > > > Good stuff. Hopefully we can