PinDrop

Established: May 28, 2015

The PinDrop project focuses on building the substrate for supporting high-quality real-time streaming over wired and wireless networks.

Context

Real-time streaming across the wide-area network (WAN) is key to several existing and anticipated services, including voice and video conferencing (e.g., Skype, Viber), remote console access (e.g., remote desktop, VNC), remote application streaming (e.g., Azure RemoteApp), and cloud-based gaming. A common characteristic of these services is bidirectional interactive communication, between two or more users or between a user and a remote service such as a cloud-hosted application or game. Therefore, these applications place stringent demands on the network not only in terms of bandwidth but also latency, jitter, packet loss, and availability. For example, VoIP would require an end-to-end round-trip time (RTT) of no more than 300 ms, otherwise users will notice a lag. With cloud-based gaming, the latency requirement is even more stringent, with an RTT of no more than 60 ms needed in some cases.

As against these demands of real-time streaming applications, the Internet largely remains a best-effort network that provides undifferentiated service, with no guarantees on latency, packet loss, or even bandwidth. While there have been many research proposals and even some standardization activity aimed at providing real-time guarantees or preferential treatment for real-time flows, these have received limited traction and seen inconsistent deployment. There are also various technical and policy impediments, even where these mechanisms have been deployed. For example, the differentiated services code point (DSCP) contained in the IP header for DiffServ is typically not preserved across ISP boundaries, limiting its usefulness as a mechanism for prioritization. Furthermore, similar prioritization mechanism called wireless multimedia extensions, which is part of the 802.11e standard for WiFi, is of little utility, say when the link is lossy due to wireless interference.

The result is that even as real-time streaming applications have moved into the mainstream over the past decade, it is not uncommon for these to suffer a noticeable degradation in performance. Indeed, the reliability of VoIP is a far cry from the five 9s service provided by traditional telephony. Thus, there is a clear need to bridge the gap between the needs of real-time streaming and the best-effort service provided by the network.

For the first time, we have applications such as Skype that exercise a wide swathe of the Internet through peer-to-peer paths, in a manner that few cloud-to-client services do. The geo-distributed footprint of users of these applications presents us with an exciting opportunity to derive Internet-wide insights and use these to improve performance.

  • The goal of the PinDrop project is to devise a comprehensive network substrate to support high-quality streaming for a range of applications across heterogeneous networks. We seek solutions that could be deployed on the Internet as it stands and also those that involve carefully-designed and local modifications (e.g., within an enterprise) to improve performance.

  • Our research agenda in the PinDrop project encompasses the following:

    1. Leveraging Geo-distributed Data: A real-time streaming service may already have a number of users connecting from diverse locations and communicating via myriad network paths. We are investigating how knowledge of the network conditions experienced by these users could be tapped to obtain visibility into the state of the network on a continual basis.
    2. Leveraging Network Path Redundancy: While the Internet, for the most part, only provides best-effort service, even for real-time streaming, what it often offers is a choice of paths between a communicating pair. We are looking at how insights on the state of the network provide the opportunity to make smart routing choices, to route around hotspots and poorly-performing sections of the network. Further, we could also pick more than one path to use simultaneously. We are interested in developing a principled approach to leverage such path redundancy.
    3. Leveraging Network Control: In certain settings, it may be feasible to customize the network to improve real-time streaming performance. We are looking at specific opportunities for such customization, with particular focus on the enterprise setting.

  • Here are some specific projects that we are pursuing under the PinDrop umbrella:

    Real-time Network Quality Map

    Vagaries in performance, along with best-effort service, make the Internet deficient in supporting high-quality real-time streaming, unlike traditional telephony. To be cognizant of network hotspots responsible for the vagaries, we seek to create a “network quality map” that indicates the state of the different parts of the Internet in real-time. By leveraging the global footprint of a service such as Skype,  we obtain performance information for a large number of network paths through end-to-end measurements between clients. With these end-to-end measurements, coupled with network routing information, we perform network tomography to glean information on the internal state of the network.

    Our research is distinguished from prior work in terms of the scale and distribution of the end-points (hundreds of millions of vantage points), and the heterogeneity of end-to-end paths (direct paths, relayed paths, etc.).

    tomography

    Robust Multi-Link Streaming over Wireless

    WiFi networks have been patterned after Ethernet networks that preceded them, with a device “plugging into” (associating with) a single port (access point) for connectivity. Such a choice, however, is not only suboptimal from a performance viewpoint, it is also unnecessary, for the device often has multiple access points in the vicinity. The wireless links to these access points tend to be weakly correlated. In DiversiFi, we have devised techniques to leverage this diversity of wireless links to not only improve streaming performance but to do so in a manner that is deployable and imposes little overhead, thereby ensuring coexistence with other traffic. Our results are promising, showing a more than 2x improvement in the poor call rate for VoIP streams.

    diversifi-arch-middlebox

    Enterprise-Scale Network Optimization

    Real-time streaming applications for communication and collaboration are growing in importance in enterprises. The enterprise setting is unique in that both the end-points and the network are under a single administrative control. This provides the opportunity to have both the end-points and the network work in unison to ensure a high-quality streaming experience for users. Our research is looking at how problem diagnostics, network optimization, and end-point adaptation can be performed in a holistic manner in enterprise networks.

    Robust Bandwidth Estimation

    Real-time streaming over inherently variable network conditions calls for constant adaptation. In particular, there is the need to estimate the bandwidth available to the stream on a continual basis. The objective is not only to estimate bandwidth but to do so in a manner that keeps the end-to-end latency within bounds. Key to our approach is a recognition that in the presence of wireless links, the traditional end-to-end approach to bandwidth estimation needs to be augmented with wireless-specific signals. To this end, we have developed Kwikr, which employs a suite of detectors for various conditions of interest on a Wi-Fi link (e.g., congestion, fluctuating link quality, handoffs) and delivers appropriate “hints” to applications such as Skype, to enable more effective bandwidth adaptation. Kwikr received the best corporate demo award (opens in new tab)at COMSNETS 2017. NEW: a paper on Kwikr will appear at ACM CoNEXT 2017 and will be made available on this page soon.

    kwikr

People

Portrait of Ranveer Chandra

Ranveer Chandra

Managing Director, Research for Industry

Portrait of Venkat Padmanabhan

Venkat Padmanabhan

Managing Director, Microsoft Research India

Portrait of Vishnu Navda

Vishnu Navda

Principal Applied Science Manager Content Services - WebXT

Portrait of Ganesh Ananthanarayanan

Ganesh Ananthanarayanan

Senior Principal Researcher