An Empirical Mixture Model for Large-Scale RTT Measurements

Transcription

1 1 An Empirical Mixture Model for Large-Scale RTT Measurements Romain Fontugne 1,2 Johan Mazel 1,2 Kensuke Fukuda 1,3 1 National Institute of Informatics 2 JFLI 3 Sokendai June 9, 2015

2 Introduction RTT: Round Trip Time Time to send a packet and receive its acknowledgment Key indicator of network conditions Important for server selection, overlay network, geolocation...

3 3 Motivations Monitor Internet-wide delays over time Measure millions of hosts RTTs Assess network performance at large-scale Report significant RTT fluctuations

4 4 RTT Estimation RTT from passive measurements Measure traffic at backbone network RTT estimation from TCP traffic Advantages Non-Intrusive (Ping the entire IP space) Monitor RTT experienced by Internet users

5 5 RTT Estimation in the network A ISP Measurement point Internet B

6 RTT Estimation in the network A ISP Measurement point Internet B A B Based on Karn s algorithm For a certain host A Compute delay samples δ = θ ACK θ SEQ Ignore retransmitted packets θseq θack } δ Measurement point 5

7 6 Problem: Understanding RTT from numerous hosts? ISP Internet

8 6 Problem: Understanding RTT from numerous hosts? ISP Internet #hosts RTT IN RTT (ms) median #hosts RTT OUT RTT (ms) median Multimodal distributions Median value is misleading! (Don t use it!)

9 Monitoring RTT Find and monitor typical RTTs #hosts RTT (ms) median RTTOUT Identify usual RTTs experienced by Internet hosts Characterize, and monitor spatial and temporal dynamics of RTTs Detect abnormal RTTs fluctuations for both host population or specific hosts

10 Proposed Model Day d Day d' Mixture Model Temporal Tracking Graph Construction B A B C D E A A' A' T = A' B' C' D' RTT B C D E d B' C' D' d' 1. Uncover the daily RTT distributions using a mixture model 2. Link RTT distributions from similar sub-population of IPs across time 3. Formalize RTTs time evolution in a graph for further systematical analysis

11 Mixture Model Identify RTT sub-populations: Unknown number of mixed components Dirichlet process mixture model log-normal distribution Obtain the mean and std. deviation of typical RTTs (µ, σ) # hosts median RT TOUT RT TIN RTT (ms) 25 # hosts pdf pdf median RTTOUT RTTIN RTT (ms)

12 10 Temporal Tracking Number of components from d and d might differ IPs from E moved to D? or they are not active in day d?

13 Temporal Tracking (cont.) Connect distributions from different days: See components as probability density functions Compute probability of IPs to fall in A and A, A and B,... Transition matrix from day d to day d

14 Graph Construction Day d Day d' Mixture Model Temporal Tracking Graph Construction B A B C D E A A' A' T = A' B' C' D' RTT B C D E d B' C' D' d' Graph from transition matrix 1. Nodes: identified modes / typical RTTs for one day 2. Edges: relate modes with similar IPs

15 Evaluation Dataset: MAWI Archive transit link between the WIDE network (ASN2500) and the Internet 15 minutes of traffic everyday from Jan to Mar traces (pcap files) RTT estimates from 12 millions unique IP addresses Separate RTTs to hosts inside the WIDE network (RTT IN ) and outside (RTT OUT )

16 Longitudinal Study cdf RT T 0.2 OUT RT T IN # components (a) CDF of the number of identified modes per day. cdf RT T 0.2 OUT RT T IN µ (b) CDF of 10 µ, the modeled RTT of the identified modes (milliseconds). RTT OUT contains more components and higher RTTs RTT IN = 500ms ( ) satellite link!

17 Geolocation Example: RTT OUT from 2014/03/21 to 2014/03/31 Cluster modes using community mining (Louvain algorithm)

18 Geolocation Example: RTT OUT from 2014/03/21 to 2014/03/31 Cluster modes using community mining (Louvain algorithm) JP KR US CA EU CN RTT C5 8% 73% 3% 289 ms C4 87% 4% 175 ms C3 73% 11% 149 ms C2 91% 108 ms C1 97% 44 ms C0 98% 19 ms Table: Hosts geolocation breakdown using Maxmind Geo-IP 15

19 16 Application 1 Look at communities RTT fluctuations: µ /01 01/08 01/15 01/22 01/29 02/05 02/12 02/19 02/26 03/05 03/12 03/19 03/26 04/02 04/09 04/16 04/23 04/30 Date RTT fluctuation = avg. RTT day2 - avg. RTT day1 (normalized) 0 means the RTTs are stable if deviate from 0 means RTTs of numerous hosts have changed RTT fluctuation depicts important RTT changes

20 RTT fluctuations & BGP updates? BGP Route Information Base (RIB) from Route Views Project Ratio of IPs affected by a BGP route vs. RTT fluctuations: abs(rt T fluctuation) BGP affected host ratio 66% of the BGP updates affecting > 15% clustered IPs exhibit > 0.15 RTT fluctuations (similar to Rimondini et al. PAM 14)

21 RTT fluctuations & network congestion? Assuming MAWI throughput is proportional to network congestion Compare MAWI throughput and RTT fluctuations abs(rt T fluctuation) Mbps higher RTT fluctuations when average throughput is higher than 500 Mbps

22 RTT fluctuations: Example Tohoku earthquake (2011/03/ /03/14) µ /07 03/09 03/11 03/13 Date March : 20ms RTT increase for all hosts outside of Japan RTT inside Japan are unchanged Impact of damaged trans-pacific links and intra-as route change

23 Consistency check 14 Application Single Host RTT 10 µ /01 01/08 01/15 01/22 01/29 02/05 02/12 02/19 02/26 03/05 03/12 03/19 03/26 04/02 04/09 04/16 04/23 04/30 Date Verify if a host is consistent with its cluster Compare the host RTTs with the identified RTT distributions Take into account the RTT variance Consistency score: 1 means the host behaves like other hosts 0 means the host deviates from other hosts

24 Proposed model gives more insights than simple RTT analysis Consistency check: Examples Consistency Anomaly RTT BGP Update RTT Consistency RTT Consistency 100 Feb Feb Mar Mar Mar Mar Feb Feb Mar Mar Mar Mar (c) TLD DNS server affected by route change (d) Amazon host experiencing suspicious RTT peak RTT Consistency RTT Consistency Mar Mar Mar Mar Mar Mar Mar Mar Jan Jan Jan Jan Feb Feb (e) RTT fluctuation during the Tohoku earthquake (f) Suspicious RTT fluctuations identified with the consistency check

25 Discussions Implementation Low memory usage and computational complexity Suitable to sampled traffic Accuracy decreases with distance (Tokyo Osaka, FR = UK) Empirical Approach Monitor RTT experienced by Internet users Cluster IP based on RTT values (not AS) Possible applications DDoS detection? BGP hijack? Difficult to evaluate

26 Conclusions Proposed mixture model Coarse view of numerous hosts RTTs Track typical RTTs time evolution Formalize RTT dynamics in a graph Applications Provides insights into the delays experienced by a large population of IP hosts Reference to find hosts deviating from their population R. Fontugne, J. Mazel, K. Fukuda. An Empirical Mixture Model for Large-Scale RTT Measurements, INFOCOM