Ing Lab 香港科技大學 THE HONG KONG UNIVERSITY OF HKUST SCIENCE AND TECHNOLOGY Information-Agnostic Flow Scheduling for Commodity Data Centers Wei bai with Li Chen, Kai chen Dongsu Han Chen Tian hao Wang SING Group HKUsT SJTU June 1st, 2015
Wei Bai with Li Chen, Kai Chen, Dongsu Han, Chen Tian, Hao Wang SING Group @ HKUST Information-Agnostic Flow Scheduling for Commodity Data Centers SJTU, June 1st, 2015 1
Data Centers
Data Centers 2
This talk is about the transport inside the data center
This talk is about the transport inside the data center 3
Data Center Networks High bandwidth Low latency Commodity switches Single administrative domain
Data Center Networks 4 • High bandwidth • Low latency • Commodity switches • Single administrative domain
Data Center transport Cloud applications Desire low latency for short messages Goal: Minimize flow completion time(FCT) Many flow scheduling proposals search ae SQL mEmCACHED
Data Center Transport • Cloud applications – Desire low latency for short messages • Goal: Minimize flow completion time (FCT) – Many flow scheduling proposals… 5
The state-of-the-art Solutions PDQ [SIGCOMM,121 fAbric[SIGCOMM'13 PASE [ SIGCOMM'141 All assume prior knowledge of flow size information to approximate ideal preemptive Shortest Job First(SJF) with customized network elements
The State-of-the-art Solutions • PDQ [SIGCOMM’12] • pFabric [SIGCOMM’13] • PASE [SIGCOMM’14] • … All assume prior knowledge of flow size information to approximate ideal preemptive Shortest Job First (SJF) with customized network elements 6
The state-of-the-art Solutions PDQ [SIGCOMM,121 fAbric[SIGCOMM'13 PASE [ SIGCOMM'141 All assume prior knowledge of ftow size informatiorto approximate ideal preemptive Shortest Job First(SJF) with customized network elements Not feasible for some applications
The State-of-the-art Solutions • PDQ [SIGCOMM’12] • pFabric [SIGCOMM’13] • PASE [SIGCOMM’14] • … All assume prior knowledge of flow size information to approximate ideal preemptive Shortest Job First (SJF) with customized network elements Not feasible for some applications 7
The state-of-the-art Solutions PDQ [SIGCOMM,121 fAbric[SIGCOMM'13 PASE [ SIGCOMM'141 All assume prior knowledge of ftow size imformrationtto approximate ideal preemptive Shortest Job First(SJF) with customized network elements Hard to deploy in practice
The State-of-the-art Solutions • PDQ [SIGCOMM’12] • pFabric [SIGCOMM’13] • PASE [SIGCOMM’14] • … All assume prior knowledge of flow size information to approximate ideal preemptive Shortest Job First (SJF) with customized network elements Hard to deploy in practice 8
Question Without prior knowledge of flow size information, how to minimize fct in commodity data centers?
Question Without prior knowledge of flow size information, how to minimize FCT in commodity data centers? 9
Design goal 1 Without prior knowledge of flow size information, how to minimize fct in commodity data centers? Information-agnostic: not assume a priori knowledge of flow size information available from the applications
Design Goal 1 Without prior knowledge of flow size information, how to minimize FCT in commodity data centers? Information-agnostic: not assume a priori knowledge of flow size information available from the applications 10