(12) United States Patent

Transcription

1 US B2 (12) Unitd Stts Ptnt Schffnggr (54) SYSTEMS, METHODS, AND COMPUTER PROGRAMI PRODUCTS PROVIDING FEEDBACK FOR NETWORK CONGESTION MANAGEMENT (75) Invntor: Richrd M. Schffnggr, Vinn (AT) (73) Assign: NtApp., Inc., Sunnyvl, CA (US) (*) Notic: Subjct to ny disclimr, th trm of this ptnt is xtndd or djustd undr 35 U.S.C. 154(b) by 363 dys. (21) Appl. No.: 13/351,372 (22) Fild: Jn. 17, 2012 (65) Prior Publiction Dt US 2013/O A1 Jul.18, 2013 (51) Int. Cl. H04L 2/80 ( ) H04L (2/825 ( ) HO4L (2/54 ( ) HO4L 12/70 ( ) (52) U.S. Cl. CPC... H04L 47/12 ( ); H04L 12/569 ( ); H04L 47/10 ( ); H04L 47/00 ( ); H04L 47/I 1 ( ); H04L 47/193 ( ); H04L 47/263 ( ) (58) Fild of Clssifiction Srch Non S ppliction fil for complt srch history. (56) Rfrncs Citd U.S. PATENT DOCUMENTS 2007/ A1 2/2007 Rudkin... TO9, /0267,070 A1* 10, 2008 Mnnl t l , / A1* 12/2008 Brisco t l , / A1 9, 2011 Attr t l. (10) Ptnt No.: US 9, B2 (45) Dt of Ptnt: My 26, 2015 FOREIGN PATENT DOCUMENTS WO WO2O A1 * 8, 2011 OTHER PUBLICATIONS Kuhldwind t l., Accurt ECN Fdbck in TCP, Jul. 4, 2011, IETF.* Kuhldwind t l., Accurt ECN Fdbck in TCP, Dc. 21, 2011, IETF.* Kuhldwind t l., Accurt ECN Fdbck in TCP, Nov. 16, 2011, TCPM-82. IETF Tipi.* M. Alizdh, A. Grnbrg, D. Mltz, J. Pdhy, P. Ptl, B. Prbhkr, S. Sngupt, nd M. Sridhrn, "Dt Cntr TCP (DCTCP), Jn. 2010, 12 pgs. M. Alizdh, A. Grnbrg, D. Mltz, J. Pdhy, P. Ptl, B. Prbhkr, S. Sngupt, nd M. Sridhrn, DCTCP. Efficint Pckt Trnsport for th Commoditizd Dt Cntr'. Jn. 2010, 15 pgs. (Continud) Primry Exminr Ayz Shikh Assistnt Exminr Ptr Chu (74) Attorny, Agnt, or Firm Hyns nd Boon, LLP (57) ABSTRACT A computr progrm product hving computr rdbl mdium tngibly rcording computr progrm logic for pro viding fdbck in ntwork, th computr progrm product including cod to rciv first dt nd scond dt ovr th ntwork t rciving dvic, cod to incrmnt first countr nd scond countr in rspons to th first dt nd scond dt, rspctivly, cod to gnrt plurlity offd bck signls rflcting stts of th first nd scond countrs using t lst thr bits, th bits dfining st of cod points mppd to th stts of th first nd scond countrs so tht ch individul cod point rprsnts diffrnt on of th stts nd ch on of th stts is rprsntd by on cod point, nd cod to trnsmit th plurlity offdbck signls to snding dvic in th ntwork. 20 Clims, 7 Drwing Shts 100

2 US 9, B2 Pg 2 (56) Rfrncs Citd OTHER PUBLICATIONS B. Brisco, A. Jcqut, T. Moncstr, nd A. Smith, "R-ECN: Add ing Accountbility for Cusing Congstion to TCP/IP, drft brisco-tsvwg-r-cn-tcp-09 (work in progrss), Oct. 2010, 51 pgs. M. Kihlwind nd R. Schffnggr, TCP Modifictions for Con gstion Exposur'. ConEx 81. IETF Qubc, Jul. 27, 2011, pp M. Kuhlwind nd R. Schffnggr, "Accurt ECN Fdbck in TCP drft-kuhlwind-conx-ccurt-cn-01 (work in progrss), Oct. 31, 2011, 23 pgs. M. Alizdh, A. Jvnmrd, B. Prbhkr, "Anlysis of DCTCP: Stbility, Convrgnc, nd Firnss, 14 pgs. B. Brisco, A. Jcqut, T. Moncstr, nd A. Smith, "R-ECN: A Frmwork for Adding Congstion Accountbility to TCP/IP, drft brisco-tsvwg-r-cn-tcp-09 (work in progrss), Oct. 2010, 41 pgs. * citd by xminr

3 U.S. Ptnt My 26, 2015 Sht 1 of 7 US 9, B Fig. 9 TABLE1 FIVE CONGESTION COUNTERVALUES NS, CWR, ANDECE FORMA THREE-BIT FIELD INACK MESSAGES THAT CAN BE USED TO CONVEY EIGHT CODE POINTS THREE HONESTY ACCOUNTING COUNTERVALUES

4

5 U.S. Ptnt My 26, 2015 Sht 3 of 7 US 9, B2 RECEIVER CFOs C-15 CF2s C-3s C-4s C=10s C=11 C=125 C-135 C=145 C-205 r r r P s r - CE21 5 sr P sr r s r r - r -D CE, dt > ACK -> ECTO, dt

6 U.S. Ptnt My 26, 2015 Sht 4 of 7 US 9, B2 Fig. 4 SENDER RECEIVER CFOs Cl=15 CE 15 Clg=1 C= 15 Clg=2 C= 15 Clg=3 ACKloss C=10 5 Cl=105 Clg=1 ACKloss C=10 5 Clg2 Cl=105 Clg=3 Cl=145 Cl=145 Clg=1 r r - r P sr r s r - r s r r Cl=145 Clg=2 r - s r -D CE, dt > ACK -> ECTO, dt

7 U.S. Ptnt My 26, 2015 Sht 5 Of 7 US 9, B2 Fig. 5 SENDER RECEIVER Cl=0s Cl=105 C=12 5 r r r --- C-135 r - r r --- ACK, Cl=3 s r ACK, C=4 r r r r P CI-35 CI-45 C-10s C=11 C=125 C-135 r C-14 ACK, C54 C C=215 C=145 ACK, C=1 C=21 5 r r s P r r s r r s r r r r r Ho CE, dt > ACK -> ECTO, dt

8 U.S. Ptnt My 26, 2015 Sht 6 of 7 US 9, B RECEIVING NETWORK CONGESTION INFORMATION AT ARECEIVER OVERA NETWORK 620 INCREMENTINGA FIRST COUNTER BASED ON THE CONGESTION INFORMATION RECEIVING HONESTY COUNTERADJUSTMENT 630 INDICATIONS FROMA SENDER OVER THENETWORK, WHERE THE RECEIVER IS IN A FEEDBACKLOOP WITH THE SENDER IN THE NETWORK 640 INCREMENTING ASECOND COUNTER BASED ON THE HONESTY COUNTERADJUSTMENT INDICATIONS 650 CONSTRUCTING A PLURALITY OF FEEDBACK RESPONSE MESSAGES, EACH OF THE FEEDBACKRESPONSE MESSAGESINCLUDING INFORMATION REGARDING THE FIRST OR THE SECOND COUNTER USING ONE OF NCODE POINTS, NBEING AN INTEGER, WHERE SOME OF THEN CODE POINTS REFERTO INFORMATIONASSOCIATED WITH THE FIRST COUNTER AND OTHERS OF THEN CODE POINTS REFERTO INFORMATIONASSOCATED WITH THE SECOND COUNTER, THEN CODE POINTSBEING DEFINED USINGA COMMON SET OF AT LEAST THREE BITS IN THE PLURALITY OF FEEDBACK RESPONSE MESSAGES 660 SENDING THE FEEDBACK MESSAGES TO THE SENDER Fig. 6

9 U.S. Ptnt My 26, 2015 Sht 7 Of 7 US 9, B2 700 RECEIVING FEEDBACK MESSAGES FROM THE RECEIVING DEVICE, THE FEEDBACK MESSAGES CARRYING INFORMATION INDICATINGA STATE OF A FIRST COUNTER AT THE RECEIVING DEVICE AND A STATE OFA 710 SECOND COUNTER AT THE RECEIVING DEVICE, WHERE EACH ONE OF THE FEEDBACK MESSAGES HAS AT LEAST THREE BITS DEFINING ONE OFNCODE POINTS, AFIRST SET OF THE CODE POINTS MAPPED TO THE STATE OF THE FIRST COUNTER ANDA SECOND SET OF THE CODE POINTS MAPPED TO THE STATE OF THE SECOND COUNTER 720 INCREMENTING ATHIRD COUNTER AT THE SENDING DEVICE IN RESPONSE TO THE RECEIVED FEEDBACKMESSAGES, THE THIRD COUNTER CORRESPONDING TO THE FIRST COUNTER INCREMENTINGA FOURTH COUNTER AT THE SENDING DEVICE IN RESPONSE TO THE RECEIVED FEEDBACK MESSAGES, THE 730 FOURTH COUNTER CORRESPONDING TO THE SECOND COUNTER MANAGING ANETWORKBEHAVIOR OF THE SENDING DEVICE IN 740 RESPONSE TO INCREMENTING THE THIRD AND FOURTH COUNTERS Fig

10 1. SYSTEMS, METHODS, AND COMPUTER PROGRAMI PRODUCTS PROVIDING FEEDBACK FOR NETWORK CONGESTION MANAGEMENT TECHNICAL FIELD US 9,042,227 B2 Th prsnt dscription rlts, in gnrl, to ntwork fdbck signling nd, mor spcificlly, to tchniqus using cod points providd by st of bits to prsnt ntwork 10 fdbck signling. BACKGROUND Communiction ntworks r oftn bndwidth-limitd. 15 Thus, dt snt from sndr to rcivr in givn com muniction ntwork my xprinc dly (or my vn b lost) s link btwn th sndr nd rcivr rchs its mximum throughput cpbility. This is tru for Locl Ar Ntworks (LANs), th public Intrnt, cloud computing/stor- 20 g ntworks, nd th lik. For instnc, in ntwork Storg nvironmnt, th dt snt ovr ntwork link to storg dvic t dt cntr or dt snt ovr ntwork link from storg dvic to host is Subjct to mximum throughput limits nd my xprinc dltrious ffcts s mximum 25 throughput is rchd. Hvy us of ntwork tht my ld to dly nd pckt loss my b rfrrd to in this xmpl s ntwork congstion. Som convntionl Solutions provid for congstion sig nling, whrin rcivr tht is md wr of congstion in 30 th ntwork (ithr by dtcting congstion itslf or bing informd of th congstion by ntwork dvic) snds fd bck to th sndr to mk th sndr wr of th ntwork congstion. Whn th sndr rcivs th fdbck tht indi cts congstion, th sndr my,.g., rduc its snding bit 35 rt to llvit th congstion in th ntwork. In mny dployd systms, singl sndr my hv simultnous sssions with multitud of rcivrs, nd th Sndr my djust bit rt for Som, ll, or non of th sssions s pproprit. Howvr, som lss honst usrs of th ntwork 40 my progrm rcivr not to rport congstion, thrby rducing th probbility tht sndr will djust bit rt of sssion with tht rcivr. Such dishonst ction my incrs th burdn on th othr rcivrs to llvit th ntwork congstion. Th following discussion provids 45 xmpls of convntionl ttmpts to provid ccurt con gstion fdbck s wll s to provid chck for honsty of rcivr. Explicit Congstion Notifiction (ECN) is dfind in th Intrnt Enginring Tsk Forc (IETF) stndrd givn in 50 Rqust For Commnts (RFC)3186 (whr RFC is n indi ction usd by Som stndrds bodis idntifying documnt tht dscribs tchnicl stndrd). Two bits, dfining four cod points, r plcd in n Intrnt Protocol (IP) hdr. Cod point 00 indicts tht dvic is not ECN cpbl: nd 10 both indict tht dvic is ECN cpbl; nd 11 is n indiction tht congstion is bing xprincd. For instnc, if routr in ntwork xprincs congs tion, th routr my insrt 11 into n IP hdr in pckt dstind for rcivr. Th rcivr dtcts th congstion 60 indiction nd snds fdbck pckt (.g., Trnsmission Control Protocol (TCP) pckt) furthr indicting th con gstion. RFC3186 dfins ECE flg in th TCP trnsport hdr, nd th rcivr my us th flg to indict th congstion stt in th ntwork. Upon rcipt of th fdbck 65 mssg, th Sndr my tk som ction to llvit th congstion, such s by rducing bit rt. Th sndr my 2 lso st Congstion Window Rducd (CWR) flg in n IP pckt to indict to th rcivr to stop mrking th congs tion flg in th fdbckpckts. Convntionl ECN provids only singl vlu for congstion, limiting its informtion vlu. ECN Nonc (RFC3540) ws proposd s n nhncmnt to convntionl ECN. ECN Nonc includs th concpt of rcivr honsty to prvnt scnrio whrin rcivr dlibrtly supprsss congstion fdbck in ordr to gin highr bndwidth ovr othr sssions in th ntwork. In ECN Nonc, th sndr snds signl ECT1 to incrs countr t th rcivr sid. Th rcivr constntly nnouncs th currnt vlu of this countr using th on-bit NS flg in th TCP trnsport hdr. Th sndr chcks if th countr vlu (NS bit) plus th ECE bit in th TCP trnsport hdr r sm or highr thn th count of th snt ECT1 cod points. If th rcivd countr is lss thn th xpctd vlu, ithr th pth or th rcivr is misbhving (.g., not proprly Supporting ECN or dlibrtly supprssing congs tion fdbck to th sndr). Othrwis, it is ssumd tht th pth nd th rcivr r supporting ECN Nonc. It is worth noting tht ftr th rcivr rcivs congstion indiction or ftr mssg loss, th rcivr cn not know th corrct vlu of th Nonc sum. If this vnt is not concld by th rcivr, th sndr synchronizs to th rcivr vlu. Oth rwis, th sndr my rst th sssion. In nothr proposd nhncmnt, clld Dt Cntr TCP (DCTCP) by Microsoft Corportion, congstion cod point chng in th IP hdr cuss th rcivr to immditly triggr n ACK to fdbck congstion informtion. Th pro posl is comptibl with ECN Nonc nd dos wy with th CWR flg. Som drwbcks of DCTCP includ tht lost ACKS rsult in loss of congstion informtion nd lso tht th systm my triggr n ACK for vry dt sgmnt. Anothr proposl from M. Kuhlwind builds upon DCTCP to us th CWR flg s shift bit for th lst congstion stt to protct ginst singl ACK loss. Howvr, two or mor lost ACKS my cus loss of congstion informtion. Anothr proposl clld r-ecn by British Tlcom uss thr bits (NS, CWR, ECE) in th TCP trnsport hdr s singl fild (ECI). ECI is usd to signl th currnt vlu of th rcivr sid congstion countr. Such proposl protcts ginst t lst two conscutiv ACK losss (sinc it uss thr bits), nd it provids highr lvl of dtil bout congstion lvl of th ntwork. Howvr, Such proposl is incomptibl with ECN Nonc bcus it uss ll thr bits (NS, CWR, ECE) for congstion indiction, lving non for honsty ccounting. Thus, sndr cnnot tll n honst rcivr from dishonst rcivr s with th ECN Nonc tchniqu. Thr is currntly no solution vilbl tht pro vids highr lvl of dtil bout lvl of congstion whil lso bing comptibl with ECN Nonc or othr honsty ccounting informtion. BRIEF DESCRIPTION OF THE DRAWINGS Th prsnt disclosur is bst undrstood from th follow ing dtild dscription whn rd with th ccompnying figurs. FIG. 1 is n illustrtion of n xmpl systm in which vrious mbodimnts my b implmntd. FIG. 2 is signl digrm illustrting procss for provid ing pproprit fdbck in th xmpl systm of FIG. 1 ccording to on mbodimnt. FIG. 3 is signl digrm illustrting ntwork congstion in th xmpl systm of FIG. 1 ccording to on mbodi mnt.

11 3 FIG. 4 is signl digrm illustrting procss for provid ing pproprit fdbck in th xmpl systm of FIG. 1 ccording to on mbodimnt. FIG. 5 is signl digrm illustrting procss for provid ing pproprit fdbck in th xmpl systm of FIG. 1 ccording to on mbodimnt. FIG. 6 is n illustrtion of n xmpl procss dptd ccording to on mbodimnt. FIG. 7 is n illustrtion of n xmpl procss dptd ccording to on mbodimnt. FIG. 8 is n illustrtion of n xmpl dvic dptd ccording to on mbodimnt. FIG.9 is n illustrtion of n xmpl rltionship btwn st of cod points nd two countrs, dptd ccording to on mbodimnt. SUMMARY Vrious mbodimnts includ systms, mthods, nd com putr progrm products providing ntwork fdbck. In on xmpl, n mbodimnt includs mnging ntwork con gstion using fdbck signls, whr th fdbck signls includ cod points rprsnting congstion informtion nd honsty ccounting informtion. On of th brodr forms of th prsnt disclosur involvs computr progrm product hving computr rdbl mdium tngibly rcording computr progrm logic for pro viding fdbck in ntwork, th computr progrm product including cod to rciv first dt nd scond dt ovr th ntwork t rciving dvic, cod to incrmnt first countr nd scond countr in rspons to th first dt nd scond dt, rspctivly, cod to gnrt plurlity offd bck signls rflcting stts of th first nd scond countrs using t lst thr bits, th bits dfining st of cod points mppd to th stts of th first nd scond countrs so tht ch individul cod point rprsnts diffrnt on of th stts nd ch on of th stts is rprsntd by on cod point, nd cod to trnsmit th plurlity offdbck signls to snding dvic in th ntwork. Anothr of th brodr forms of th prsnt disclosur involvs mthod includs rciving ntwork congstion informtion t rcivr ovr computr ntwork, rciving honsty countr djustmnt indictions from sndr ovr th ntwork, whr th rcivr is in fdbck loop with th sndr in th ntwork, incrmnting first countr bsd on th congstion informtion, incrmnting scond countr bsd on th honsty countr djustmnt indictions, con structing plurlity of fdbck rspons mssgs, ch of th fdbck rspons mssgs including informtion rgrding th first or th scond countr using on of N cod points, N bing n intgr, whr som of th N cod points rfr to informtion ssocitd with th first countr nd othrs of th N cod points rfr to informtion ssocitd with th scond countr, th N cod points bing dfind using common st of t lst thr bits in th plurlity of fdbck rspons mssgs, nd snding th fdbck ms sgs to th sndr ovr th computr ntwork. Anothr of th brodr forms of th prsnt disclosur involvs rcivr in ntwork nvironmnt includs t lst on port for communicting ovr computr ntwork, in which th rcivr is in fdbck loop with sndr ovr th ntwork, first countr for storing ntwork congstion infor mtion, scond countr for storing honsty ccounting informtion, procssor for incrmnting th first nd scond countrs nd for forming fdbck mssgs bsd on th first nd scond countrs, furthr in which ch fdbck mssg includs common st of Mbits dfining on of N US 9,042,227 B diffrnt cod points, whr M nd N r both intgrs lrgr thn on, whr first st of th N cod points rprsnts th ntwork congstion informtion nd whr scond st of th N cod points rprsnts th honsty ccounting informtion, nd trnsmission unit for snding th fdbck mssgs ovr th ntwork to th sndr. Anothr of th brodr forms of th prsnt disclosur involvs mthod prformd by snding dvic in fd bck loop with rciving dvic ovr computr ntwork, th mthod including rciving fdbck mssgs from th rciving dvic, th fdbck mssgs crrying inform tion indicting stt of first countr t th rciving dvic nd stt of scond countr t th rciving dvic, whr ch on of th fdbck mssgs hs t lst thr bits dfining on of N cod points, first st of th cod points mppd to th stt of th first countr nd scond st of th cod points mppd to th Stt of th scond countr, incr mnting third nd fourth countrs t th snding dvic in rspons to th rcivd fdbck mssgs, th third nd fourth countrs corrsponding to th first nd scond countrs, rspctivly, nd mnging ntwork bhvior of th snding dvic in rspons to incrmnting th third nd fourth countrs. DETAILED DESCRIPTION Th following disclosur provids mny diffrnt mbodi mnts, or xmpls, for implmnting diffrnt fturs of th invntion. Spcific xmpls of componnts nd rrng mnts r dscribd blow to simplify th prsnt disclosur. Ths r, of cours, mrly xmpls nd r not intndd to b limiting. In ddition, th prsnt disclosur my rpt rfrnc numrls nd/or lttrs in th vrious xmpls. This rptition is for th purpos of simplicity nd clrity nd dos not in itslf dictt rltionship btwn th vrious mbodimnts nd/or configurtions discussd. It is undrstood tht vrious mbodimnts my b impl mntd in ny kind of ntwork nvironmnt. For xmpl, som mbodimnts my b implmntd in TCP/IP nt work conncting multipl clints nd multipl srvrs. Othr mbodimnts my b implmntd in mor spcilizd nvi ronmnts, such s in Ntwork Attchd Storg (NAS), Storg Ar Ntwork (SAN), or ny othr ntwork storg configurtion. Vrious mbodimnts r not limitd to ny prticulr ntwork rchitctur or purpos nd my b dptd for us in wid-rnging rry of nvironmnts. Vrious mbodimnts disclosd hrin provid rcivr in ntwork fdbck loop with sndr. On xmpl includs sndr tht snds strming mdi to th rcivr. In yt nothr xmpl, sndr is dvic writing lrg mounts of dt to storg srvr (nd th storg srvr is th rcivr). Routrs nd/or switchs in th ntwork btwn th sndr nd th rcivr dtct congstion. Such routrs nd Switchs mrk pckts with congstion flg whn con gstion is dtctd. Th rcivr incrmnts countr whn it rcivs pckts mrkd with th congstion flg. An xmpl of countr is rgistr mintind by procssor t th rcivr. Th countr includs multipl bits, nd it uss thos bits to stor vlu. Th procssor cn incrmnt or dcrmnt th vlu t th countr s pproprit in this xmpl th procssor incrmnts th vlu in th countr upon rciving pckt from routr mrkd with congs tion indiction. Th rcivr snds fdbckpckts tht includ inform tion tht rflcts th stt of its congstion countr. A stt of th countr, in this xmpl, rfrs to th numbr stord in th

12 5 countr t givn tim. Th Stt of th countr my chng ovr tim, dpnding on congstion in th ntwork. Furthrmor, th sndr priodiclly snds indictions for th rcivr to updt its honsty countr, similr to ECN Nonc. Th rcivr, if it is oprting honstly, updts its honsty countr ccordingly nd, whn pproprit, snds fdbck to th sndr. Th fdbck indicts vlu (or stt) of th rcivr's honsty countr. As xplind in mor dtil blow, th sndr cn xmin this fdbck to discrn whthr th rcivr is bhving honstly. In contrst to Som convntionl pprochs dscribd in th bckground, th rcivr includs nhncd functionlity to llow it to snd both congstion informtion nd honsty ccounting informtion to th sndr using th fdbck loop. Th rcivr uss numbr of bits M in fdbck pckt hdr to rprsnt th congstion informtion nd th hon sty ccounting informtion. For instnc, if M is thr, thn thr bits provid ight totl cod points (i.., 2=8), which r vlus tht cn b rprsntd by th bits. A subst of th cod points cn b dvotd to th congstion informtion, whrs nothr subst of th cod points cn b dvotd to th honsty ccounting informtion. STbl 1 blow for n xmpl configurtion. Whil it is possibl tht som mbodimnts might tho rticlly dfin prticulr cod point to rprsnt n honsty ccounting vlu nd congstion vlu, most mbodimnts will dfin prticulr cod point s n honsty ccounting vlu or congstion vlu. Thus, mny mbodimnts includ fdbck pckts, whr ch individul pckt my crry on of th cod points to rprsnt n honsty ccount ing vlu or ntwork congstion vlu, but not both. Nv rthlss, common st of Mbits cn provid sufficint numbr of cod points to rprsnt th honsty ccounting vlus nd th ntwork congstion vlus. Furthrmor, som mbodimnts us vlu of M tht is lrg nough to provid thr or mor cod points for ch typ of informtion, thrby providing highr informtionl contnt thn simpl Zro or on. For instnc, whn M is thr, on xmpl dvots fiv cod points to congstion informtion nd thr cod points to honsty informtion. Howvr, th scop of mbodimnts is not limitd to ny prticulr vlu of M, nd th scop of mbodimnts cn includ ny pproprit vlu for M thr or lrgr. Furthr, th numbr of cod points dvotd to typ of informtion is not limitd to th xmpls bov, s ny pproprit numbr of cod points two or lrgr my b dvotd to prticulr typ of informtion. Th numbr of cod points provids for high lvl of informtion in th fdbck nd lso provids rsilincy whn on or mor fdbck pckts r lost. FIG. 1 is n illustrtion of n xmpl systm 100 in which vrious mbodimnts my b implmntd. Sndr 110 is coupld to rcivr 120 through ntwork 180. Th ntwork 180 my includ, for xmpl, locl r ntwork (LAN), wid r ntwork (WAN), th Intrnt, Fibr Chnnl fbric, or ny combintion of such intrconncts. Ech of th sndr 110 nd rcivr 120 my includ, for xmpl, prsonl computr (PC), srvr computr, storg srvr, worksttion, hndhld computing/communiction dvic or tblt, nd/or th lik. FIG. 1 shows only two computing componnts 110, 120, but th scop of mbodimnts cn includ ny pproprit numbr of sndrs nd/or rcivrs. Morovr, FIG. 1 shows two routrs 130, 135 for illustrtion only, nd it is undrstood tht in vrious mbodimnts nt work 180 my includ ny pproprit numbr of routrs nd Switchs. As mntiond bov, th rcivr 120 includs nhncd functionlity to kp trck of nd convy, honsty ccounting US 9,042,227 B informtion nd ntwork congstion informtion. Rcivr 120 cquirs honsty countr informtion nd congstion informtion vi IP pckts 140 nd updts its countrs ppropritly. Rcivr 120 provids fdbck s pckts 150 (.g., TCPACK pckts). Sndr 110 lso includs nhncd functionlity to kp trck of th fdbck from rcivr 120 nd to rct ccordingly. Sndr 110 nd rcivr 120 both includ computr rdbl cod, which whn xcutd, cuss sndr 110 or rcivr 120, rspctivly, to prform ccording to th signl digrm of FIGS In othr words, for ny of th ctions dscribd blow, th sndr 110 or th rcivr 120 my b pr-progrmmd to tk such ctions. Furthrmor, in mny mbodimnts, th vrious vlus (.g., numbr of cod points, siz of countrs, thrsholds, tc.) r pr-dtrmind nd progrmmd to th sndr 110 nd/or rcivr 120. FIG. 2 is signl digrm illustrting procss for provid ing pproprit fdbck in th systm 100 of FIG.1. On th right sid, th vlus in th rcivr's two countrs r shown. On th lft sid, th vlus in th sndr's complmntry countrs r shown. Th lgnd provids dscription of th prticulr signls btwn th sndr nd th rcivr. Th rcivr mintins two countrs to count CE mrks (congstion flgs from ntwork routr/switch) nd ECT1 mrks (flgs from th sndr to indict tht th honsty countr should b incrmntd) of th rcivd IP pckts. In this xmpl, th "CI countr is congstion countr t th rcivr, nd th E1 countr is th honsty ccounting countr t th rcivr. Th sndr includs complmntry countrs. Th fdbck from th rcivr cuss th sndr to updt its complmntry countrs. Using ths sndr-sid countrs, th sndr cn prform mor informd congstion control dcisions (such s in DCTCP) or IP-lvl signling (such s in r-ecn), nd dtct misbhving rcivrs (such s in ECN Nonc). Th prsnt mbodimnt utilizs th thr TCP ECN flgs s singl fild. Th thr bits of this fild provid ight diffrnt stts (on for ch cod point). Ech of th cod points of this fild r givn distinctivly diffrnt mning. Exmpl cod point vlus r providd in Tbl 1 (blow), nd th scop of mbodimnts is not limitd thrto. FIG. 9 is providd to illustrt th vrious rltionships of th vlus in Tbl 1. TABLE 1 NS CWR ECE CI (bs 5) E.1 (bs 3) O O O O O O 1 1 O 1 O 2 O O O 4 1 O 1 O 1 1 O InTbl 1, th ight cod points r th ight binry vlus (000, 001, ). Ech cod point indicts numbr in on of th countrs. For xmpl, th cod point 011 convys congstion informtion bcus it corrsponds to vlu of 3 t th CI countr. Similrly, th cod point 110 convys hon sty ccounting informtion bcus it corrsponds to vlu of 1 t th E1 countr. Whr thr bits r usd, s in Tbl 1, th xct ssign mnt of cod points my b diffrnt in othr mbodimnts. For instnc, th fiv Congstion Indiction (CI) cod points nd th thr ECN-Cpbl Trnsport 1 (ECT1 or E1 for

13 7 short) cod points my b rbitrrily mppd to ny of th ight vilbl vlus. Also, diffrnt distributions, such s six cod points for CI, nd two for E1, or four/four my b usd in som mbodimnts. Similrly, diffrnt mbodimnts my us numbr of bits highr thn thr. Such fturs my b pr-progrmmd into sndr nd/or rcivr. Th countrs thmslvs my b implmntd s modulo fiv nd modulo thr, rspctivly. Th modulo oprtion rfrs mximum numbr of vlus tht cn b incrmntd bfor th countr rolls ovr (wrps). Sinc fiv cod points r dvotd to congstion informtion in this xmpl, it my b pproprit to us modulo fiv countrfor th CI countr. Similrly, th E1 countr is implmntd s modulo thr countr. Only th lst digit of bs-5 (CI) or bs-3 (E1) countr is convyd ths countrs wrp t multipl of fiv or thr, rspctivly. In som mbodimnts, modulo oprtion is usd to convrt rgulr binry countr into bs5 or bs3 lst digit. Also, if diffrnt numbr of cod points is ssignd, th modulo oprtion cn b djustd ccordingly. In FIG. 2 th rcivr snds ACKs bck to th sndr, nd in this xmpl, th rcivr provids n ACK fdbck for vry two IP pckts rcivd. Ech ACK normlly crris th pproprit cod point to signl th stt of th CI countr. Whn th rcivr rcivs n ECT1 flg in th IP hdr, th rcivr will snd th pproprit cod point of th E1 countr in th nxt ACK. If both th E1 nd CI countrs chng in on intrvl, priority is givn to th CI countr so tht th nxt ACK will crry th cod point for th updtd CI countr, whil informtion convying th cod point for th E1 countr my b dlyd until Subsqunt ACK is snt. It might b possibl tht diffrnc btwn countr nd th lst signld cod point bcoms quit lrg. In Such cs, th rcivr my snd n ACK immditly, vn if n ACK ws snt in rspons to th immditly prcding IP pckt. Such ftur is dscribd in mor dtil blow with rspct to FIG. 5. A pr-dtrmind thrshold for such diffr nc my b progrmmd s pproprit for givn pplic tion t rcivr. In FIG. 2 th initil stts of th E1 nd CI countrs t th rcivr nd t th sndr r ll Zro. At tim point 210, th rcivr rcivs ECTO signl nd dos not chng th vlu of ny countr in rspons thrto. Th rcivr thn snds n ACK pckt to th sndr, whr th ACK crris cod point indicting tht th CI countr is t Zro. At tim point 225, th sndr rcivs th ACK pckt. Th nxt IP pckt hs n ECT1 flg nd is rcivd t tim point 215 cusing th rcivr to updt its E1 countr to vlu of on. In this xmpl, th rcivr dos not snd n ACK pckt for vry IP pckt nd, thus, wits until tim point 220 to snd th ACK pckt rporting chng in th E1 countr. Th sndr rcivs th ACK pckt t tim point 230 nd updts its corrsponding countr ppropritly. At tim point 235, th rcivr rcivs nothr ECT1 flg from th sndr nd updts its E1 countr. Howvr, it is not tim to snd n ACK t tim point 235. At tim point 240, th rcivr rcivs congstion indiction nd updts its CI countr. Th rcivr thn snds n ACK pckt tht crris cod point for th CI, rthr thn for th E1 countr, bcus in this xmpl th CI countrtks priority. Thus, th sndr dos not updt its complmntry honsty countr until tim point 250, whrs th sndr updts its complmntry congstion countr t tim point 245. Th xmpl of FIG. 2 shows scnrio whr congstion is light, no ACK pckts r lost, nd th rcivr cts hon stly. Th sndr mintins countr of th numbr of snt pckts with ECT1 (E1). A comprison with th rcivd E1 US 9,042,227 B countr (E1) nd th rcivd CI countr (CI) rvls if rcivr is oprting corrctly: ElisE1ss (E1+CI). Th xchng my go on indfinitly, nd th sndr should t ny tim b bl to vrify th oprtion of th ntwork nd th rcivr. If th sndr dtrmins t ny tim tht th rcivr or ntwork is not bhving proprly, th sndr my tk pproprit ction by,.g., nding or rstting th sssion. Th sndr my lso rct to congstion informtion in ny pproprit mnnr. For instnc, s th rcivr's CI countr incrmnts up, th sndr my bcom wr of this issu by nlyzing th rcivr's fdbck. Th sndr my thn djust bit rt of th sssion (or of nothr sssion), nd th sssion, or tk othr pproprit ction to llvit th con gstion. Howvr, in diffrnt scnrio whr th congstion is hvy, th sndr my hv mor difficulty dtrmining th stt of ithr of th rcivr's countrs. FIG. 3 shows scnrio ccording to on mbodimnt whrin ntwork con gstion is vry hvy, so tht ch on of th IP pckts shows congstion nd cuss th rcivr to incrmnt its CI countr. Furthrmor, ACKs r lost t tim points 305,310, which is likly vnt whn congstion is hvy. As rsult, th CI countr t th rcivr rolls ovr (or wrps) onc without th sndr bing wr. Thrfor, t tim point 315, th sndr my ssum tht th CI countr vlu is dciml two (Zro-two in Bs 5), whrs th CI countr vlu hs rolld ovr nd rlly should rflct vlu of dciml six (on-two whn counting in Bs 5). Vrious mbodimnts includ on or mor fturs to ddrss th issu illustrtd in FIG. 3. On xmpl ftur includs snding ch cod point twic. Such ftur llows for up to two conscutiv ACKs to b lost, in Som instncs, without ny impct of th fdbck signl. If both countrs chng simultnously, th rspctiv cod points r snt ltrntivly to llow t lst thr conscutiv ACK losss. FIG. 4 illustrts Such scnrio ccording to on mbodi mnt. In FIG. 4, th ntwork is vry congstd, nd vry IP pckt t th rcivr indicts congstion. At tim point 405. th rcivr snds n ACK indicting C1 countr vlu of dciml 1, nd th rcivr rpts th sm countr vlu t tim point 415, vn though nothr IP pckt indicting congstion is rcivd t tim point 410. Th rcivr snds subsqunt ACK t tim point 420, nd this ACK indicts th lst countr digit bing Zro, indicting dciml 5 (on-zro in Bs 5). Th ACK t tim point 425 lso indicts C1 lst digit vlu of Zro bcus ch countr vlu is snt twic. It is notd tht th ACKs snt t tim points 415 nd 420 r both lost bfor thy rch th sndr. Howvr, th sndr is bl to discrn th vlu of th countr bcus it did rciv th ACKs snt t tim points 405 nd 425, which r idnticl to th ACKs snt t tim points 415 nd 420, rspctivly. It is lso notd tht th rsilincy illustrtd in FIG. 4 is du to th fct tht th two lost ACKs hppn to strddl chng in th rportd C1 vlu. As n illustrtion, suppos th ACKs sntt tim points 420, 425 hd bn lost instd. In such scnrio, th sndr would hv missd both ACKs rporting C1 vlu of zro, nd th sndr might not b bl to discrn th corrct vlu of th C1 countr t th rcivr. Thus, th ftur illustrtd in FIG. 4 my dd rsilincy, but it my not b comprhnsiv. FIG. 4 lso shows th us of countr CI.g. Sinc th spc for communicting th informtion bck to th sndr in ACKS is limitd, instd of dirctly incrsing this countr, 'gug' (CI.g) is incrsd instd. Whn snding n ACK, th contnt of this gug (cppd by th mximum numbr tht cn b ncodd in th ACK,.g. 4 for CI, nd 2 for E1) is

14 copid to th ctul countr, nd CI.g. is rducd by th vlu tht ws copid ovr nd trnsmittd (unlss CI.g. ws Zro bfor). To void losing informtion, th rcivr snds n ACK t lst ftr fiv incoming congstion mrks (i.. whn CI.g xcds fiv). For rsilinc ginst lost ACKs, n indictor flg (CI.i) nsurs tht, whthr nothr congstion indic tion rrivs or not, scond ACK trnsmits th prvious countr vlu gin. Th sm countr/gug mthod is usd to count nd fdbck (using diffrnt mpping) th num br of incoming pckts mrkd ECT(1) (clld E1 in th lgorithm). As fwr cod points r vilbl for convying th E1 countr vlu, n immdit ACK my b triggrd whnvr th gug E1.g. xcds thrshold of 3. Th sndr rcivs th rcivr's countr vlus nd comprs thm with th loclly mintind countr. Any incrs of ths countrs is ddd to th sndrs intrnl countrs, yilding prcis numbr of CE-mrkd nd ECT(1) mrkd pckts. Architcturlly th countrs do not dcrs during TCP sssion. Howvr, ny ovrflow my b modulo 5 for CI nd modulo 3 for E1. Th ftur illustrtd with rspct to FIG. 4 is not limitd to th xct configurtion dscribd whrin ch cod point is snt twic. Vrious mbodimnts my rpt congstion vlu or n honsty ccounting vlu mor thn twic if pproprit. Howvr, s M incrss, th bnfit of such rptition dcrss. Anothr xmpl ftur is illustrtd in FIG. 5 ccording to on mbodimnt. In this xmpl, th rcivr limits th mximum diffrnc in th signld cod points for fw prcding ACKs. Whn th mximum diffrnc btwn th prcding ACK nd th corrsponding countr rchs th mximum diffrnc, th rcivr my snd n ACK, vn if it would othrwis b prmtur. In othr words, FIG. 5 illustrts n mbodimnt tht incrss snding rt for fdbck s pproprit. Th diffrnc should not b lrgr thn th numbr of cod points for prticulr typ of rportd informtion. Th mximum diffrnc my b mni fst s on or mor pr-dtrmind thrsholds (.g., Rltions 1 nd 2, blow) tht r progrmmd into th rcivr. Assuming tht (t) rprsnts th currnt tim point, nd (t 1), (t-2) nd (t-3) rprsnt tim points of th thr prcd ing ACKs, thn on mbodimnt oprts ccording to Rl tions 1 nd 2, givn blow, whrin whn ithr rltion is untru, th rcivr triggrs n ACK t tim point (t). CI, DCI, 2+2 (# cod points N-1)/2 (1) CID=CI, +4 (# cod points N-1) (2) In FIG. 5, th ntwork is xprincing hvy congstion, nd vry IP pckt rcivd by th rcivr indicts con gstion. Ordinrily th rcivr would snd n ACK for vry othr IP pckt, but in this xmpl, th rcivr oprts ccording to Rltions 1 nd 2 (bov). Assuming tht th numbr of CI cod points is fiv, s in th prvious xmpls, thn Rltion 1 bcoms untru t tim point 510. Thus, th rcivr snds n ACK t tim point 510, vn though th ACK would othrwis b prmtur. Furthr in th xmpl of FIG. 5, th ACKs snt t tim points 505 nd 510 r lost bfor thy rch th rcivr. Howvr, Rltions 1 nd 2 r formultd so tht ny two conscutiv ACKs (but not mor thn two) cn b lost, nd th sndr will still b bl to ccurtly know th stt of th countr t th rcivr. In othr words, in th xmpl of FIG. 5. Rltions 1 nd 2 nsur tht nough ACKs r snt so tht countr t th rcivr will not roll ovr mor thn onc for ch thr ACKs snt. US 9,042,227 B On disdvntg of th mbodimnt dscribd with rspct to FIG. 5 is tht such mbodimnt my incrs th numbr of ACKs snt in th fdbck loop, which my ctu lly incrs congstion. Howvr, th ddd rsilincy my outwigh tht disdvntg for Som pplictions. Whil th xmpl of FIG. 5 is shown with rspct to th congstion countr only, it is notd tht th scop of mbodi mnts my includ similr ftur for th honsty ccount ing countr s wll. Furthrmor, in mbodimnts whr M is lrgr thn 3 nd/or whr grtr rsilincy is dsird, Rl tions 1 nd 2 my b chngd ccordingly. Th mbodimnts dscribd with rspct to FIGS. 4 nd 5 r optionl nd my b implmntd in som css with no furthr sndr sid chngs. Such mbodimnts my b spcilly pplicbl to spcific nvironmnts with potntil xtrm congstion, Such s dt cntrs, virtul distributd dt cntrs, nd th lik. FIG. 6 is n illustrtion of n xmpl procss 600 dptd ccording to on mbodimnt. Th procss of FIG.6 my b prformd by,.g., th rcivr 120 of FIG. 1 to provid pproprit fdbck to sndr. At block 610, th rcivr rcivs ntwork congstion informtion ovr th ntwork. In on xmpl, th ntwork congstion informtion includs congstion mrkr plcd in n IP pckt by ntwork dvic,.g., routr or Switch. Howvr, ny pproprit ntwork congstion informtion my b dptd for us in Som mbodimnts. At block 620, th rcivr incrmnts first countr bsd on th congstion informtion. Exmpl countrs 820, 830 r shown in FIG. 8. Any pproprit countr my b usd in vrious mbodimnts, including, for instnc, trditionl countr or gug. At block 630, th rcivr rcivs honsty countr djust mnt indictions. On xmpl of honsty countr djust mnt informtion includs n ECT(1) flg in n IP hdr, though th scop of mbodimnts is not limitd thrto. Any pproprit countr djustmnt indiction my b dptd for us in procss 600. At block 640, th rcivr incrmnts scond countr bsd on th honsty countr djustmnt of block 630. In on xmpl, th countrs of blocks 620 nd 640 includ countrs tht wrp ftr spcific numbr of vlu chngs. For instnc, th mbodimnts bov mntion modulo 3 countr nd modulo 5 countr, though othr mbodimnts my includ diffrnt modulo divisors. Blocks 620 nd 640 my includ wrpping on or mor of th countrs ftr mxi mum vlu hs bn rchd. At block 650, th rcivr constructs fdbck mssgs. In this mbodimnt, th rcivr dos not construct ll of th fdbck mssgs t th sm tim, but rthr constructs thm on t tim s shown in FIGS In this xmpl, ch of th fdbck mssgs includs informtion rgrd ing th first countr or th scond countr. Ech of th fd bck mssgs includs on of N possibl cod points, whr th N cod points r dfind using common st of t lst thr bits. Ech on of th cod points rfrs to informtion ssocitd with th first countr or informtion ssocitd with th scond countr, nd ch individul stt of th countrs is mppd to on of th cod points. An xmpl is givn bov t Tbl 1, showing how thr bits cn b usd to dfin ight totl codpoints to mp ight totl stts of th two countrs. Thus, rthr thn hving on bit rprsnt stt of on countr nd nothr bit rprsnt ing stt of th othr countr, th xmpl mbodimnt uss ll thr bits in dscribing stts of two sprt countrs. Furthr, ch countr is rprsntd by t lst thr cod points, thrby providing high lvl of informtion contnt.

15 US 9,042,227 B2 11 At block 660, th rcivr snds th fdbck mssg to th sndr. In on xmpl, th rcivr plcs th bits in trnsport hdr in TCP ACK pckt. Howvr, vrious mbodimnts my us ny pproprit dt structur to con vy th fdbck to th sndr. 5 Th scop of mbodimnts is not limitd to th xct pro cdur shown in FIG. 6. For instnc, som ctions my b ddd, omittd, rrrngd, or modifid. In on xmpl, th procss 600 is rptd multipl tims during communic tion sssion to produc fdbck mssgs. In mny mbodi- 10 mnts, th blocks follow n ordr of 610, 620, 650, nd/or 630, 640, 650 so tht th ordr of th blocks in FIG. 6 is for s of illustrtion rthr thn for xplicitly dscribing n ordr. In on mbodimnt, th rcivr snds t lst two ms- 15 sgs with givn congstion (or honsty) cod point bfor Switching to diffrnt congstion (or honsty) codpoint, s illustrtd bov with rspct to FIG. 4. In nothr mbodi mnt, th rcivr djusts snding rt of fdbck in rspons to dtrmintion tht diffrnc btwn 20 countr nd prviously-snt pckt hs rchd thrshold, s illustrtd bov with rspct to FIG. 5. FIG. 7 is n illustrtion of n xmpl procss 700 dptd ccording to on mbodimnt. Th procss of FIG.7 my b prformd by,.g., th sndr 110 of FIG. 1 to rct ppro- 25 pritly to fdbck from rcivr. In block 710, th sndr rcivs fdbck mssgs from th rciving dvic, whr th fdbck mssgs convy informtion indicting stt of first countr t th rciving dvic nd stt of scond countr t th rciving dvic. 30 As in th xmpl bov, th first countr my rflct ntwork congstion informtion, nd th scond countr my rflct n honsty ccounting of th rcivr. In this xmpl mbodimnt, ch fdbck mssg indicts stt of th first countr or stt of th scond countr, but not both. 35 Ech on of th fdbck mssgs hs t lst thr bits dfining on of N cod points, first st of th cod points mppd to th stt of th first countr nd scond st of th cod points mppd to th stt of th scond countr. This concpt is xplind furthr bov t block At block 720, th sndr incrmnts third countr in rspons to th rcivd fdbck mssgs. Th third countr corrsponds to th first countr in this cs by trcking informtion in th fdbck tht provids stt of th first countr. 45 At block 730, th sndr incrmnts fourth countr in rspons to th rcivd fdbck mssgs. Th fourth countr corrsponds to th scond countr in this cs by trcking informtion in th fdbck tht provids stt of th scond countr. 50 At block 740, th sndr mngs its bhvior in rspons to incrmnting th third nd fourth countrs. In on xmpl, th third countr convys ntwork congstion informtion. In rspons to discrning high congstion stt, th sndr my djust bit rt of communiction sssion by, for xmpl, 55 dcrsing th bit rt. In scnrio whr th sndr is Supporting multipl communiction sssions, th sndr my dcrs bit rt of ch of th sssions similrly, my djust bit rts ccording to diffrnt prioritis of th sssions, nd/ or th lik. 60 In nothr xmpl, th sndr chcks th fourth countr for n ccounting of th honsty of th rcivr. At ny givn tim, Sum of th congstion countr nd th honsty ccounting countr t th sndr should b qul to or grtr thn th numbr of honsty countr incrmnting indictions 65 (.g., ECT(1) mssgs) snt from th sndr to th rcivr. If th bov-mntiond sum is not grtr, it my b n 12 indiction tht th rcivr is Supprssing congstion notifi ctions in ordr to rciv n unfir portion of ntwork bnd width. Accordingly, th sndr my sk to nforc ntwork policis in ny pproprit mnnr. Exmpls of nforcmnt includ rducing bit rt of communiction with th rcivr, rstting communiction sssion with th rcivr, bnning th rcivr, nd/or th lik. Th scop of mbodimnts is not limitd to th xct pro cdur shown in FIG. 7. For instnc, som ctions my b ddd, omittd, rrrngd, or modifid. In on xmpl, th procss 700 is rptd multipl tims during communic tion sssion so tht ntwork congstion is mngd through out th sssion. Furthrmor, th mbodimnts dscribd bov with rspct to FIGS. 6 nd 7 rfr to honsty informtion (.g., ECN Nonc informtion) nd congstion informtion (.g., CE informtion in ECN). Howvr, th scop of mbodi mnts ncompsss systms nd mthods tht trck othr rlvnt informtion using countrs nd fdbck. In othr words, th concpt of convying multitud of countrs using cod points to sv bits in protocol hdr my hv mor widsprd pplicbility outsid of ntwork congstion mn gmnt nd lso outsid of TCP/IP. FIG. 8 is n illustrtion of n xmpl dvic 800 ccording to on mbodimnt. In this xmpl, dvic 800 my b sndr or rcivr. Dvic 800 includs port 850 for intr fcing with th ntwork. Trnsmission nd rciving unit 840 is in communiction with th port 850 for snding nd rciv ing dt ovr th ntwork. Procssor 810 incrmnts th countrs 820, 830. For instnc, in th cs of rcivr, procssor 810 incrmnts countrs 820, 830 in rspons to ntwork congstion infor mtion nd honsty ccounting indictions from th sndr. In th cs of sndr, procssor 810 incrmnts countrs 820, 830 in rspons to th fdbck from th rcivr. Pro cssor 810 my lso form fdbck mssgs (s in block 650 of FIG. 6) nd/or chng ntwork bhvior (s in block 740 of FIG. 7). Embodimnts of th prsnt disclosur cn tk th form of computr progrm product ccssibl from tngibl com putr-usbl or computr-rdbl mdium providing pro grm cod for us by or in connction with computr or ny instruction xcution systm. For th purposs of this dscription, tngibl computr-usbl or computr-rd bl mdium cn b ny pprtus tht cn stor th progrm for us by or in connction with th instruction xcution systm, pprtus, or dvic. Th mdium cn b n lc tronic, mgntic, opticl, lctromgntic, infrrd, or smiconductor Systm (or pprtus or dvic). In som mbodimnts, on or mor procssors (such s procssor 810 of FIG. 8) running in on or mor of dvics (such s dvic 800) xcut cod to implmnt th ctions shown in FIGS Vrious mbodimnts my includ on or mor dvntgs ovr convntionl systms. For instnc, vrious mbodi mnts convy both honsty ccounting informtion nd con gstion informtion from rcivr to sndr without dd ing dditionl bits to th TCP trnsport hdr ovr thn dfind in ECN currntly. Furthrmor, th informtionl contnt providd by som mbodimnts in th fdbck ms sgs cn b quit high by llocting thr or mor cod points to ch typ of informtion. By contrst, som convn tionl tchniqus my provid only on bit of prticulr typ of informtion or, if providing multipl bits of prticulr typ of informtion, omitting othr typs of informtion. Th forgoing outlins fturs of svrl mbodimnts so tht thos skilld in th rt my bttr undrstnd th spcts

16 13 of th prsnt disclosur. Thos skilld in th rt should pprcit tht thy my rdily us th prsnt disclosur s bsis for dsigning or modifying othr procsss nd struc turs for crrying out th sm purposs nd/or chiving th sm dvntgs of th mbodimnts introducd hrin. Thos skilld in th rt should lso rliz tht such quiv lnt constructions do not dprt from th spirit nd scop of th prsnt disclosur, nd tht thy my mk vrious chngs, Substitutions, nd ltrtions hrin without dprt ing from th spirit nd scop of th prsnt disclosur. Wht is climd is: 1. A non-trnsitory computr-rdbl mdium, th com putr-rdbl mdium comprising: cod to rciv, t rciving dvic vi ntwork, nt work congstion informtion; cod to rciv, t th rciving dvic from snding dvic vi th ntwork, honsty ccounting informtion, whrin th rciving dvic is in fdbck loop with th snding dvic; cod to incrmnt first countr nd scond countr bsd on th ntwork congstion informtion nd hon sty ccounting informtion, rspctivly; cod to gnrt plurlity of fdbck signls rflcting stts of th first nd scond countrs using common st of t lst thr bits in th plurlity of fdbck signls, th common st of t lst thr bits dfining first st of cod points mppd to -stts of th first countr nd scond st of cod points mppd to stts of th scond countr such tht ch individul cod point rprsnts diffrnt on of th stts of th first or scond countrs nd ch on of th stts of th first nd scond countrs is rprsntd by on cod point; nd cod to trnsmit th plurlity of fdbck signls to th snding dvic in th ntwork. 2. Th computr-rdbl mdium of clim 1 in which th cod to trnsmit compriss: cod to trnsmit ch fdbck signl t lst twic. 3. Th computr-rdbl mdium of clim 1 in which th cod to trnsmit compriss: cod to incrs snding rt of th fdbck signls in rspons to dtrmintion tht diffrnc btwn vlu of th first or th scond countr nd cod point of prviously-snt fdbck signl hs rchd thrshold. 4. A mthod comprising: rciving ntwork congstion informtion t rcivr ovr computr ntwork; rciving honsty countr djustmnt indictions from sndr ovr th computr ntwork, whr th rcivr is in fdbck loop with th sndr in th computr ntwork; incrmnting first countr bsd on th congstion infor mtion; incrmnting scond countr bsd on th honsty countr djustmnt indictions; constructing plurlity offdbck mssgs, ch of th fdbck mssgs including informtion rflcting stt of th first or th scond countr using on of N cod points, N bing n intgr grtr thn on, whr first st of th N cod points rflct stts of th first countr nd scond st of th N cod points rflct stts of th scond countr, th N cod points bing dfind using common st of t lst thr bits in th plurlity of fdbck mssgs; nd snding th fdbck mssgs to th sndr ovr th com putr ntwork. US 9,042,227 B Th mthod of clim 4 in which numbr of bits quls thr nd N quls ight, furthr in which fiv of th N cod points r usd for rflcting Stts of th first countr nd thr of th N cod points r usd for rflcting stts of th scond countr. 6. Th mthod of clim 4 in which snding th fdbck mssgs to th sndr compriss: clculting diffrnc btwn cod point of prvi ously snt fdbck mssg nd on of th first or scond countrs; nd djusting frquncy of snding th fdbck mssgs in rspons to th clcultd diffrnc. 7. Th mthod of clim 4 in which snding th fdbck mssgs to th sndr compriss: snding numbr P of fdbck mssgs with givn cod point bfor snding Subsqunt fdbck ms sg with diffrnt cod point vn if th first countr incrmnts bfor th Pfdbck mssgs r snt, nd whr P is n intgr lrgr thn on. 8. Th mthod of clim 4 in which ch of th fdbck mssgs compriss Trnsmission Control Protocol (TCP) ACK pckt. 9. Th mthod of clim 4 in which th ntwork congstion informtion is rcivd from ntwork dvic othr thn th sndr. 10. Th mthod of clim 4 in which th first countr hs diffrnt modulo divisor thn modulo divisor of th scond COuntr. 11. Th mthod of clim 4 in which constructing plurlity offdbck mssgs compriss: ssigning priority to first on or mor of th N cod points, rfrring to informtion ssocitd with th first countr, ovr scond on or mor of th N cod points rfrring to informtion ssocitd with th scond COuntr. 12. Th mthod of clim 4 in which th N cod points r dfind so tht, for givn on of th fdbck mssgs, th rcivr snds ithr cod point for honsty ccounting informtion or cod point for ntwork congstion, but not both. 13. A rcivr in ntwork nvironmnt compriss: t lst on port for communicting ovr computr nt work, in which th rcivr is in fdbck loop with sndr ovr th computr ntwork; first countr, scond countr; procssor for: rciving ntwork congstion informtion; incrmnting th first countr bsd on th ntwork con gstion informtion; rciving, from th sndr, honsty ccounting inform tion; incrmnting th scond countr bsd on th honsty ccounting informtion; nd forming fdbck mssgs bsd on stts of th first nd scond countrs, furthr in which ch of th fdbck mssgs includs informtion rflcting stt of th first or th scond countr using common st of M bits in ch of th fdbck mssgs, th common st of Mbits dfining on of N diffrnt cod points, whr M nd N r both intgrs lrgr thn on, whr first st of th N cod points rflcts stts of th first countr nd whr scond st of th N cod points rflcts stts of th scond countr; nd trnsmission unit for snding th fdbck mssgs ovr th computr ntwork to th sndr.

17 US 9,042,227 B Th rcivr of clim 13 in which th procssor is configurd to djust frquncy of snding th fdbck mssgs in rspons to diffrnc btwn cod point in prviously-snt fdbck mssg nd vlu of th first COuntr. 15. Th rcivr of clim 13 in which th procssor is configurd to snd numbr P offdbck mssgs with givn cod point bfor snding Subsqunt fdbck ms sg with diffrnt cod point Th rcivr of clim 13 in which th first countr hs diffrnt modulo divisor thn modulo divisor of th scond COuntr. 17. A mthod prformd by snding dvic in fdbck loop with rciving dvic ovr computr ntwork, th 15 mthod comprising: rciving fdbck mssgs from th rciving dvic, th fdbck mssgs crrying informtion indicting stt of first countr t th rciving dvic nd stt of scond countr t th rciving dvic, whr ch 20 on of th fdbck mssgs hs t lst thr bits dfining on of N cod points, first st of th cod points mppd to th stt of th first countr nd scond st of th cod points mppd to th stt of th scond countr; 16 incrmnting third nd fourth countrs t th snding dvic in rspons to th rcivd fdbck mssgs, th third nd fourth countrs corrsponding to th first nd scond countrs, rspctivly; nd mnging ntwork bhvior of th snding dvic in rspons to incrmnting th third nd fourth countrs. 18. Th mthod of clim 17 in which th first nd third countrs rflct ntwork congstion informtion, furthr in which mnging ntwork bhvior compriss djusting bit rt of communiction sssion in rspons to stt of th third countr. 19. Th mthod of clim 17 in which th scond nd fourth countrs rflct n honsty ccounting of th rciving dvic, furthr in which mnging ntwork bhvior of th snding dvic compriss compring cod point vlu for honsty informtion in fdbck pckt with sum of th fourth countr nd numbr of countr-incrmnting ms sgs snt by th snding dvic. 20. Th mthod of clim 19 furthr comprising: tking n nforcmnt ction in rspons to dtrmining tht th sum of th fourth countr nd numbr of countr-incrmnting mssgs snt by th snding dvic is not th sm s, or grtr thn, th cod point vlu for honsty informtion in th fdbck pckt. k k k k k