Abstract
Network monitoring is indispensable for maintaining and managing networks efficiently. With increasing network traffic in the ISP, enterprise and cloud environments, it is challenging to provide low overhead monitoring services without sacrificing accuracy. In this dissertation, we present techniques to enable measurement systems and services to have (1) high measurement accuracy, and (2) low measurement overhead. In the context of active measurements, shared active measurement services have been proposed to provide a common and safe environment to conduct measurements. By adapting to user measurement requests, we present solutions to (1) selectively use inference mechanisms, and (2) schedule active measurements in a non-interfering manner. These techniques reduce the measurement overhead costs and improve the accuracy for an active measurement service.^ In the context of passive flow based measurements systems, this dissertation introduces Pegasus, a monitoring system that leverages co-located compute and storage devices to support aggregation queries. Using Pegasus, we present IFA (Iterative Feedback Aggregator), a technique to accurately detect global icebergs and network anomalies at a low communication cost. Finally, we present ALE (Approximate Latency Estimator), a scalable and low-overhead technique to estimate TCP round trip times at high data rates for troubleshooting network performance problems.^
