Successfully reported this slideshow.
Methodologies for
         Networking Research


17 October 2008
   CS5229, Semester 1, 2008/09
                          ...
Measurementquot;
      V. Paxson. quot;End-to-end Internet packet dynamics,”quot;

      J. Padhye, V. Firoiu, D. Towesley...
“Reality Check”
      Are our assumptions reasonable? Is our
      mathematical model a good estimation of the real
      ...
e.g., from Paxson’s studyquot;

   1. packet losses are busrtyquot;
   2. OTT != RTT/2


17 October 2008
   CS5229, Semest...
17 October 2008
   CS5229, Semester 1, 2008/09
                                             
   5
                        ...
17 October 2008
   CS5229, Semester 1, 2008/09
                                             
   6
                        ...
Experimentationquot;
                                                          

      e.g., V. Jacobson. “Congestion Cont...
Deal with implementation
 issues
 Sometimes unforeseen complexities (e.g. own research
 experience in Unreliable TCP) 



...
Understand the Behavior
   of Systems
 Some systems are too complex to understand with
 “thought experiments” alone.




1...
17 October 2008
   CS5229, Semester 1, 2008/09
                                             
   10
                       ...
17 October 2008
   CS5229, Semester 1, 2008/09
                                             
   11
                       ...
Analysisquot;
      D. Chiu and R. Jain, quot;Analysis of the increase and decrease
      algorithms for congestion avoida...
Explore with Complete
 Control
 We can understand the basic forces that affect the
 system. e.g. TCP throughput is inverse...
Simplify complex systemsquot;

   If too simplified, important behavior could be missed
   (TCP throughput without timeout)...
Simulationquot;
          K. Fall and S. Floyd, quot;Simulation-based comparison of
          Tahoe, Reno, and SACK TCP,qu...
Check Correctness of
 Analysisquot;

 If simulation uses the same assumptions/model as the
 analysis, this simply verifies ...
Check Correctness of
 Analysisquot;

 Simulation can relax some assumptions, use more
 complex models, etc. to test the li...
Explore Complex Systemsquot;

 Some systems are too difficult/impossible to analyzed
 e.g. Internet 




17 October 2008
  ...
Helps Develop Intuitionquot;




17 October 2008
   CS5229, Semester 1, 2008/09
                                          ...
Measurementquot;
    Experimentationquot;                             }   Real World




    Analysisquot;
    Simulationq...
“Difficulties in Simulating
      the Internet” quot;
     Sally Floyd, Van Paxsonquot;
                              
 ACM...
Why is Internet hard to
             simulate?


17 October 2008
   CS5229, Semester 1, 2008/09
                          ...
1.!
                   Internet is diverse

17 October 2008
        CS5229, Semester 1, 2008/09
                          ...
End-hosts: phones,
      desktops, servers, iPod, Wii



17 October 2008
   CS5229, Semester 1, 2008/09
                  ...
Links: Ethernet, WiFi,
      Satellite, Dial-up, 3G



17 October 2008
   CS5229, Semester 1, 2008/09
                    ...
Transport: TCP variants,
      UDP, DCCP



17 October 2008
   CS5229, Semester 1, 2008/09
                               ...
Applications: games,
      videos, web, ftp, bittorrent 



17 October 2008
   CS5229, Semester 1, 2008/09
               ...
2.!
                   Internet is huge

17 October 2008
       CS5229, Semester 1, 2008/09
                              ...
570,937,778

                       Number of Hosts as of July 2008
           http://www.isc.org/index.pl?/ops/ds/host-co...
3.!
              Internet is changing

17 October 2008
    CS5229, Semester 1, 2008/09
                                  ...
http://www.isc.org/ds/
17 October 2008
     CS5229, Semester 1, 2008/09
                                               
  ...
17 October 2008
          CS5229, Semester 1, 2008/09
                                                    
   32
         ...
Median File Transfer
     Time
                              Size
 March 1998
                  10.9 kB
December 1998
    ...
Why is Internet hard to
      simulate?quot;

      1. Heterogeneous quot;
      2. Huge quot;
      3. Changing
17 Octobe...
Suppose you come up
      with the greatest
      BitTorrent
      improvement ever..

17 October 2008
   CS5229, Semester...
You want to simulate it
      to make sure it works
      before you release it
      (and call the press)

17 October 200...
What Internet topology
      should you use in your
      simulation?
      How end hosts are connected? What are the
    ...
Topology changes constantlyquot;
 Companies keep info secretsquot;
 Routes may changequot;
 Routes may be asymmetric

17 O...
You will need to simulate over
      a wide range of connectivity
      and link properties



17 October 2008
   CS5229, ...
Suppose you come up
      with the greatest TCP
      optimization ever..

17 October 2008
   CS5229, Semester 1, 2008/09
...
You want to know if it
      is fair to existing TCP
      versions before you
      write your SIGCOMM
      paper..
17 O...
Which TCP version to
      use?


17 October 2008
   CS5229, Semester 1, 2008/09
                                         ...
Using “fingerprinting”,
      831 different TCP
      implementations and
      versions are identified.

17 October 2008
  ...
Which to use? Which
      to ignore? 


17 October 2008
   CS5229, Semester 1, 2008/09
                                   ...
What applications to run?
      What type of traffic to
      generate?quot;

      Telnet? FTP? Web? BitTorrent?
      Sky...
How congested should
        the network be? 



17 October 2008
   CS5229, Semester 1, 2008/09
                          ...
How congested should
      the network be? 




17 October 2008
   CS5229, Semester 1, 2008/09
                           ...
Example from Sally Floyd:
RED vs DropTail


17 October 2008
   CS5229, Semester 1, 2008/09
                               ...
17 October 2008
   CS5229, Semester 1, 2008/09
                                             
   49
                       ...
17 October 2008
   CS5229, Semester 1, 2008/09
                                             
   50
                       ...
Example from Sally Floyd:
Using TFRC for VoIP


17 October 2008
   CS5229, Semester 1, 2008/09
                           ...
17 October 2008
   CS5229, Semester 1, 2008/09
                                             
   52
                       ...
17 October 2008
   CS5229, Semester 1, 2008/09
                                             
   53
                       ...
We can focus our
     simulation on dominant
     technology/application
     today..

17 October 2008
   CS5229, Semester...
TCP: NewReno SACKSquot;
     OS: Windows Linuxquot;
     Applications: Web, FTP

17 October 2008
   CS5229, Semester 1, 20...
What about tomorrow?


17 October 2008
   CS5229, Semester 1, 2008/09
                                             
   56
...
WiMax? quot;
     Sensors? quot;
     Virtual World?quot;
     DCCP?

17 October 2008
   CS5229, Semester 1, 2008/09
     ...
10 years ago, you came
    up with a router
    mechanism to improve
    TCP Reno.. quot;

    No one cares today.
17 Octo...
How to verify the
    simulator itself?


17 October 2008
   CS5229, Semester 1, 2008/09
                                 ...
So, how?


17 October 2008
   CS5229, Semester 1, 2008/09
                                             
   60
            ...
Looking for
                    Invariants


17 October 2008
     CS5229, Semester 1, 2008/09
                            ...
1. Diurnal Patterns


17 October 2008
    CS5229, Semester 1, 2008/09
                                              
   62...
17 October 2008
   CS5229, Semester 1, 2008/09
                                             
   63
                       ...
hour #constrained
---- ------------
  00   139   2.5%    -----------------------------------------------------X
  01   144...
2. Self-Similar Traffic


17 October 2008
   CS5229, Semester 1, 2008/09
                                             
   6...
The traffic is bursty
       regardless of time scale


17 October 2008
   CS5229, Semester 1, 2008/09
                    ...
Wikipedia

17 October 2008
   CS5229, Semester 1, 2008/09
                                             
   67
            ...
3. Poisson Session
                         Arrival


17 October 2008
        CS5229, Semester 1, 2008/09
                ...
Remote logins, starting
        FTP, beginning of web
             surfing etc.

17 October 2008
   CS5229, Semester 1, 200...
(so are dead light bulbs,
       spelling mistakes, etc.)


17 October 2008
   CS5229, Semester 1, 2008/09
               ...
4. Log-normal Duration


17 October 2008
   CS5229, Semester 1, 2008/09
                                             
   7...
5. Heavy Tail
                   Distributions


17 October 2008
     CS5229, Semester 1, 2008/09
                        ...
Self-Similarity in World Wide Web Traffic: Evidence and
        Possible Causes, by Mark E. Crovella and Azer Bestavros
17 ...
1. Looking for
                     Invariants


17 October 2008
       CS5229, Semester 1, 2008/09
                      ...
2. Explore
          Parameter Space


17 October 2008
   CS5229, Semester 1, 2008/09
                                    ...
Change one parameter,quot;
                        fix the rest



17 October 2008
         CS5229, Semester 1, 2008/09
   ...
Explore a wide range of values




17 October 2008
   CS5229, Semester 1, 2008/09
                                        ...
3. Use Traces


17 October 2008
      CS5229, Semester 1, 2008/09
                                                
   78
 ...
e.g. collects traces of web
      sessions, video files, VoIP traffic



17 October 2008
   CS5229, Semester 1, 2008/09
    ...
Use it to simulate the traffic
                      source



17 October 2008
    CS5229, Semester 1, 2008/09
            ...
But must be careful about traffic
       shaping and user/application
               adaptation. 



17 October 2008
   CS5...
e.g. traces collected during non-
congested time should not be use to
   simulate congested networks. 



17 October 2008
...
4. publish simulator script for
                  others to verify



17 October 2008
    CS5229, Semester 1, 2008/09
    ...
Conclusion


17 October 2008
     CS5229, Semester 1, 2008/09
                                               
   84
      ...
Simulation is useful but needs to
               do it properly



17 October 2008
   CS5229, Semester 1, 2008/09
        ...
Be careful about your simulation
   model: you want it to be as simple
      as possible, but not simpler.



17 October 2...
Be careful about your conclusion:
      “A is 13.5% better than B” is
            probably useless.



17 October 2008
   ...
“A is 13.5% better than B under
             these environment”quot;
          is better but not general



17 October 200...
Not really for quantitative results,
             but more for



17 October 2008
   CS5229, Semester 1, 2008/09
         ...
understanding the dynamics,quot;
illustrate a point,quot;
explore unexpected behavior.



17 October 2008
   CS5229, Semes...
Nächste SlideShare
Wird geladen in …5
×

CS5229 Lecture 9: Simulating the Internet

663 Aufrufe

Veröffentlicht am

Veröffentlicht in: Technologie
  • Als Erste(r) kommentieren

  • Gehören Sie zu den Ersten, denen das gefällt!

CS5229 Lecture 9: Simulating the Internet

  1. 1. Methodologies for Networking Research 17 October 2008 CS5229, Semester 1, 2008/09 1
  2. 2. Measurementquot; V. Paxson. quot;End-to-end Internet packet dynamics,”quot; J. Padhye, V. Firoiu, D. Towesley, and J. Kurose quot;Modeling TCP Throughput: A Simple Model and its Empirical Validation,” 17 October 2008 CS5229, Semester 1, 2008/09 2
  3. 3. “Reality Check” Are our assumptions reasonable? Is our mathematical model a good estimation of the real world? 17 October 2008 CS5229, Semester 1, 2008/09 3
  4. 4. e.g., from Paxson’s studyquot; 1. packet losses are busrtyquot; 2. OTT != RTT/2 17 October 2008 CS5229, Semester 1, 2008/09 4
  5. 5. 17 October 2008 CS5229, Semester 1, 2008/09 5
  6. 6. 17 October 2008 CS5229, Semester 1, 2008/09 6
  7. 7. Experimentationquot; 
 e.g., V. Jacobson. “Congestion Control and Avoidancequot; 17 October 2008 CS5229, Semester 1, 2008/09 7
  8. 8. Deal with implementation issues Sometimes unforeseen complexities (e.g. own research experience in Unreliable TCP) 17 October 2008 CS5229, Semester 1, 2008/09 8
  9. 9. Understand the Behavior of Systems Some systems are too complex to understand with “thought experiments” alone. 17 October 2008 CS5229, Semester 1, 2008/09 9
  10. 10. 17 October 2008 CS5229, Semester 1, 2008/09 10
  11. 11. 17 October 2008 CS5229, Semester 1, 2008/09 11
  12. 12. Analysisquot; D. Chiu and R. Jain, quot;Analysis of the increase and decrease algorithms for congestion avoidance in computer networks,”quot; J. Padhye, V. Firoiu, D. Towesley, and J. Kurose quot;Modeling 
 TCP Throughput: A Simple Model and its Empirical Validation,” 17 October 2008 CS5229, Semester 1, 2008/09 12
  13. 13. Explore with Complete Control We can understand the basic forces that affect the system. e.g. TCP throughput is inversely propotional to √p 17 October 2008 CS5229, Semester 1, 2008/09 13
  14. 14. Simplify complex systemsquot; If too simplified, important behavior could be missed (TCP throughput without timeout) 17 October 2008 CS5229, Semester 1, 2008/09 14
  15. 15. Simulationquot; K. Fall and S. Floyd, quot;Simulation-based comparison of Tahoe, Reno, and SACK TCP,quot;quot; S. Floyd, K. Fall, quot;Promoting the Use of End-to-End Congestion Control in the Internet,”quot; S. Floyd, V. Jacobson, quot;Random Early Detection Gateways for Congestion Avoidance,quot; 17 October 2008 CS5229, Semester 1, 2008/09 15
  16. 16. Check Correctness of Analysisquot; If simulation uses the same assumptions/model as the analysis, this simply verifies the correctness of the mathematical derivations. 17 October 2008 CS5229, Semester 1, 2008/09 16
  17. 17. Check Correctness of Analysisquot; Simulation can relax some assumptions, use more complex models, etc. to test the limits of analysis. (Real measurement/experiments still needed to check the usefulness of analysis results) 17 October 2008 CS5229, Semester 1, 2008/09 17
  18. 18. Explore Complex Systemsquot; Some systems are too difficult/impossible to analyzed e.g. Internet 17 October 2008 CS5229, Semester 1, 2008/09 18
  19. 19. Helps Develop Intuitionquot; 17 October 2008 CS5229, Semester 1, 2008/09 19
  20. 20. Measurementquot; Experimentationquot; } Real World Analysisquot; Simulationquot; } Abstract Model 17 October 2008 CS5229, Semester 1, 2008/09 20
  21. 21. “Difficulties in Simulating the Internet” quot; Sally Floyd, Van Paxsonquot; ACM/IEEE TON, 9(4) August 2001
  22. 22. Why is Internet hard to simulate? 17 October 2008 CS5229, Semester 1, 2008/09 22
  23. 23. 1.! Internet is diverse 17 October 2008 CS5229, Semester 1, 2008/09 23
  24. 24. End-hosts: phones, desktops, servers, iPod, Wii 17 October 2008 CS5229, Semester 1, 2008/09 24
  25. 25. Links: Ethernet, WiFi, Satellite, Dial-up, 3G 17 October 2008 CS5229, Semester 1, 2008/09 25
  26. 26. Transport: TCP variants, UDP, DCCP 17 October 2008 CS5229, Semester 1, 2008/09 26
  27. 27. Applications: games, videos, web, ftp, bittorrent 17 October 2008 CS5229, Semester 1, 2008/09 27
  28. 28. 2.! Internet is huge 17 October 2008 CS5229, Semester 1, 2008/09 28
  29. 29. 570,937,778
 Number of Hosts as of July 2008 http://www.isc.org/index.pl?/ops/ds/host-count-history.php 17 October 2008 CS5229, Semester 1, 2008/09 29
  30. 30. 3.! Internet is changing 17 October 2008 CS5229, Semester 1, 2008/09 30
  31. 31. http://www.isc.org/ds/ 17 October 2008 CS5229, Semester 1, 2008/09 31
  32. 32. 17 October 2008 CS5229, Semester 1, 2008/09 32 http://www.dtc.umn.edu/mints/
  33. 33. Median File Transfer Time Size March 1998 10.9 kB December 1998 5.6 kB December 1999 10.9 kB June 2000 62 kB November 2000 10 kB Measurement at LBNL: Statistical property of Internet changes as well.
  34. 34. Why is Internet hard to simulate?quot; 1. Heterogeneous quot; 2. Huge quot; 3. Changing 17 October 2008 CS5229, Semester 1, 2008/09 34
  35. 35. Suppose you come up with the greatest BitTorrent improvement ever.. 17 October 2008 CS5229, Semester 1, 2008/09 35
  36. 36. You want to simulate it to make sure it works before you release it (and call the press) 17 October 2008 CS5229, Semester 1, 2008/09 36
  37. 37. What Internet topology should you use in your simulation? How end hosts are connected? What are the properties of the links? 17 October 2008 CS5229, Semester 1, 2008/09 37
  38. 38. Topology changes constantlyquot; Companies keep info secretsquot; Routes may changequot; Routes may be asymmetric 17 October 2008 CS5229, Semester 1, 2008/09 38
  39. 39. You will need to simulate over a wide range of connectivity and link properties 17 October 2008 CS5229, Semester 1, 2008/09 39
  40. 40. Suppose you come up with the greatest TCP optimization ever.. 17 October 2008 CS5229, Semester 1, 2008/09 40
  41. 41. You want to know if it is fair to existing TCP versions before you write your SIGCOMM paper.. 17 October 2008 CS5229, Semester 1, 2008/09 41
  42. 42. Which TCP version to use? 17 October 2008 CS5229, Semester 1, 2008/09 42
  43. 43. Using “fingerprinting”, 831 different TCP implementations and versions are identified. 17 October 2008 CS5229, Semester 1, 2008/09 43
  44. 44. Which to use? Which to ignore? 17 October 2008 CS5229, Semester 1, 2008/09 44
  45. 45. What applications to run? What type of traffic to generate?quot; Telnet? FTP? Web? BitTorrent? Skype? 17 October 2008 CS5229, Semester 1, 2008/09 45
  46. 46. How congested should the network be? 17 October 2008 CS5229, Semester 1, 2008/09 46
  47. 47. How congested should the network be? 17 October 2008 CS5229, Semester 1, 2008/09 47
  48. 48. Example from Sally Floyd: RED vs DropTail 17 October 2008 CS5229, Semester 1, 2008/09 48
  49. 49. 17 October 2008 CS5229, Semester 1, 2008/09 49
  50. 50. 17 October 2008 CS5229, Semester 1, 2008/09 50
  51. 51. Example from Sally Floyd: Using TFRC for VoIP 17 October 2008 CS5229, Semester 1, 2008/09 51
  52. 52. 17 October 2008 CS5229, Semester 1, 2008/09 52
  53. 53. 17 October 2008 CS5229, Semester 1, 2008/09 53
  54. 54. We can focus our simulation on dominant technology/application today.. 17 October 2008 CS5229, Semester 1, 2008/09 54
  55. 55. TCP: NewReno SACKSquot; OS: Windows Linuxquot; Applications: Web, FTP 17 October 2008 CS5229, Semester 1, 2008/09 55
  56. 56. What about tomorrow? 17 October 2008 CS5229, Semester 1, 2008/09 56
  57. 57. WiMax? quot; Sensors? quot; Virtual World?quot; DCCP? 17 October 2008 CS5229, Semester 1, 2008/09 57
  58. 58. 10 years ago, you came up with a router mechanism to improve TCP Reno.. quot; No one cares today. 17 October 2008 CS5229, Semester 1, 2008/09 58
  59. 59. How to verify the simulator itself? 17 October 2008 CS5229, Semester 1, 2008/09 59
  60. 60. So, how? 17 October 2008 CS5229, Semester 1, 2008/09 60
  61. 61. Looking for Invariants 17 October 2008 CS5229, Semester 1, 2008/09 61
  62. 62. 1. Diurnal Patterns 17 October 2008 CS5229, Semester 1, 2008/09 62
  63. 63. 17 October 2008 CS5229, Semester 1, 2008/09 63
  64. 64. hour #constrained ---- ------------ 00 139 2.5% -----------------------------------------------------X 01 144 2.6% ------------------------------------------------------X 02 146 2.6% -------------------------------------------------------X 03 140 2.5% -----------------------------------------------------X 04 119 2.1% ---------------------------------------------X 05 89 1.6% ----------------------------------X 06 69 1.2% --------------------------X 07 55 1.0% ---------------------X 08 45 0.8% -----------------X 09 40 0.7% ---------------X 10 40 0.7% ---------------X 11 42 0.8% ----------------X 12 51 0.9% -------------------X 13 57 1.0% ---------------------X 14 68 1.2% --------------------------X 15 75 1.3% ----------------------------X 16 77 1.4% -----------------------------X 17 92 1.6% -----------------------------------X 18 98 1.8% -------------------------------------X 19 105 1.9% ----------------------------------------X 20 108 1.9% -----------------------------------------X 21 113 2.0% -------------------------------------------X 22 124 2.2% -----------------------------------------------X 23 134 2.4% ---------------------------------------------------X U Waterloo Data 24 Oct 2007 17 October 2008 CS5229, Semester 1, 2008/09 64
  65. 65. 2. Self-Similar Traffic 17 October 2008 CS5229, Semester 1, 2008/09 65
  66. 66. The traffic is bursty regardless of time scale 17 October 2008 CS5229, Semester 1, 2008/09 66
  67. 67. Wikipedia 17 October 2008 CS5229, Semester 1, 2008/09 67
  68. 68. 3. Poisson Session Arrival 17 October 2008 CS5229, Semester 1, 2008/09 68
  69. 69. Remote logins, starting FTP, beginning of web surfing etc. 17 October 2008 CS5229, Semester 1, 2008/09 69
  70. 70. (so are dead light bulbs, spelling mistakes, etc.) 17 October 2008 CS5229, Semester 1, 2008/09 70
  71. 71. 4. Log-normal Duration 17 October 2008 CS5229, Semester 1, 2008/09 71
  72. 72. 5. Heavy Tail Distributions 17 October 2008 CS5229, Semester 1, 2008/09 72
  73. 73. Self-Similarity in World Wide Web Traffic: Evidence and Possible Causes, by Mark E. Crovella and Azer Bestavros 17 October 2008 CS5229, Semester 1, 2008/09 73
  74. 74. 1. Looking for Invariants 17 October 2008 CS5229, Semester 1, 2008/09 74
  75. 75. 2. Explore Parameter Space 17 October 2008 CS5229, Semester 1, 2008/09 75
  76. 76. Change one parameter,quot; fix the rest 17 October 2008 CS5229, Semester 1, 2008/09 76
  77. 77. Explore a wide range of values 17 October 2008 CS5229, Semester 1, 2008/09 77
  78. 78. 3. Use Traces 17 October 2008 CS5229, Semester 1, 2008/09 78
  79. 79. e.g. collects traces of web sessions, video files, VoIP traffic 17 October 2008 CS5229, Semester 1, 2008/09 79
  80. 80. Use it to simulate the traffic source 17 October 2008 CS5229, Semester 1, 2008/09 80
  81. 81. But must be careful about traffic shaping and user/application adaptation. 17 October 2008 CS5229, Semester 1, 2008/09 81
  82. 82. e.g. traces collected during non- congested time should not be use to simulate congested networks. 17 October 2008 CS5229, Semester 1, 2008/09 82
  83. 83. 4. publish simulator script for others to verify 17 October 2008 CS5229, Semester 1, 2008/09 83
  84. 84. Conclusion 17 October 2008 CS5229, Semester 1, 2008/09 84
  85. 85. Simulation is useful but needs to do it properly 17 October 2008 CS5229, Semester 1, 2008/09 85
  86. 86. Be careful about your simulation model: you want it to be as simple as possible, but not simpler. 17 October 2008 CS5229, Semester 1, 2008/09 86
  87. 87. Be careful about your conclusion: “A is 13.5% better than B” is probably useless. 17 October 2008 CS5229, Semester 1, 2008/09 87
  88. 88. “A is 13.5% better than B under these environment”quot; is better but not general 17 October 2008 CS5229, Semester 1, 2008/09 88
  89. 89. Not really for quantitative results, but more for 17 October 2008 CS5229, Semester 1, 2008/09 89
  90. 90. understanding the dynamics,quot; illustrate a point,quot; explore unexpected behavior. 17 October 2008 CS5229, Semester 1, 2008/09 90

×