1626 IEEE COMMUNICATIONS SURVEYS TUTORIALS.VOL.16.NO.3.THIRD QUARTER 2014 OpenFlow protocol version 1.0,unless stated otherwise.This V.SDN APPLICATIONS table also provides a brief overview of the listed controllers. Software-defined networking has applications in a wide va- Included in Table III are also two special purpose controller riety of networked environments.By decoupling the control- implementations:Flowvisor [48,mentioned previously,and and data planes,programmable networks enable customized RouteFlow [66].The former acts as a transparent proxy be- control,an opportunity to eliminate middleboxes,as well tween OpenFlow switches and multiple OpenFlow controllers. as simplified development and deployment of new network It is able to create network slices and can delegate control of services and protocols.Below,we examine different envi- each slice to a different controller,also promoting isolation ronments for which SDN solutions have been proposed or between slices.RouteFlow,on the other hand,is an open implemented. source project to provide virtualized IP routing over OpenFlow capable hardware.It is composed of an OpenFlow Controller application,an independent server,and a virtual network A.Enterprise Networks environment that reproduces the connectivity of a physical Enterprises often run large networks,while also having infrastructure and runs IP routing engines.The routing engines strict security and performance requirements.Furthermore, generate the forwarding information base(FIB)into the Linux different enterprise environments can have very different re- IP tables according to the routing protocols configured (e.g.. quirements,characteristics,and user population,For example, OSPF,BGP).An extension of RouteFlow is presented in [67]. University networks can be considered a special case of which discusses Routing Control Platforms (RCPs)in the enterprise networks:in such an environment,many of the context of OpenFlow/SDN.They proposed a controller-centric connecting devices are temporary and not controlled by the networking model along with a prototype implementation of University,further challenging security and resource alloca- an autonomous-system-wide abstract BGP routing service. tion.Additionally,Universities must often provide support for research testbeds and experimental protocols. Adequate management is critically important in Enterprise E.Code Verification and Debugging environments,and SDN can be used to programmatically enforce and adjust network policies as well as help monitor Verification and debugging tools are vital resources for network activity and tune network performance. traditional software development and are no less important for SDN.Indeed,for the idea of portable network "apps"to Additionally,SDN can be used to simplify the network by ridding it from middleboxes and integrating their functionality be successful,network behavior must be thoroughly tested and within the network controller.Some notable examples of verified. middlebox functionality that has been implemented using NICE [68]is an automated testing tool used to help uncover SDN include NAT,firewalls,load balancers [74][75],and bugs in OpenFlow programs through model checking and network access control [76.In the case of more complex symbolic execution. middleboxes with functionalities that cannot be directly im- Anteater [69]takes a different approach by attempting to plemented without performance degradation(e.g.,deep packet check network invariants that exist in the data plane,such as inspection),SDN can be used to provide unified control and connectivity or consistency.The main benefit of this approach management77. is that it is protocol-agnostic;it will also catch errors that The work presented in [78]addresses the issues related result from faulty switch firmware or inconsistencies with the to consistent network updates.Configuration changes are control plane communication.VeriFlow [70]has a similar a common source of instability in networks and can lead goal,but goes further by proposing a real-time verification to outages,security flaws,and performance disruptions.In tool that resides between the controller and the forwarding [78],a set of high-level abstractions are proposed that allow elements.This adds the potential benefit of being able to halt network administrators to update the entire network,guaran- bad rules that will cause anomalous behavior before they reach teeing that every packet traversing the network is processed the network. by exactly one consistent global network configuration.To Other efforts proposed debugging tools that provide insights support these abstractions,several OpenFlow-based update gleaned from control plane traffic.OFRewind [71]allows mechanisms were developed. network events (control and data)to be recorded at different As discussed in earlier sections,OpenFlow evolved from granularities and later replayed to reproduce a specific sce- Ethane [20],a network architecture designed specifically to nario,granting the opportunity to localize and troubleshoot the address the issues faced by enterprise networks. events that caused the network anomaly.ndb [72]implements breakpoints and packet-backtraces for SDN.Just as with the popular software debugger gdb,users can pinpoint events that B.Data Centers lead to error by pausing execution at a breakpoint,or,using Data centers have evolved at an amazing pace in recent a packet backtrace,show the sequence of forwarding actions years,constantly attempting to meet increasingly higher and seen by that packet.STS [73]is a software-defined network rapidly changing demand.Careful traffic management and troubleshooting simulator.It is written in python and depends policy enforcement is critical when operating at such large on POX.It simulates the devices in a given network allowing scales,especially when any service disruption or additional for testing cases and identifying the set of inputs that generates delay may lead to massive productivity and/or profit loss.Due a given error. to the challenges of engineering networks of this scale and1626 IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. 16, NO. 3, THIRD QUARTER 2014 OpenFlow protocol version 1.0, unless stated otherwise. This table also provides a brief overview of the listed controllers. Included in Table III are also two special purpose controller implementations: Flowvisor [48], mentioned previously, and RouteFlow [66]. The former acts as a transparent proxy be￾tween OpenFlow switches and multiple OpenFlow controllers. It is able to create network slices and can delegate control of each slice to a different controller, also promoting isolation between slices. RouteFlow, on the other hand, is an open source project to provide virtualized IP routing over OpenFlow capable hardware. It is composed of an OpenFlow Controller application, an independent server, and a virtual network environment that reproduces the connectivity of a physical infrastructure and runs IP routing engines. The routing engines generate the forwarding information base (FIB) into the Linux IP tables according to the routing protocols configured (e.g., OSPF, BGP). An extension of RouteFlow is presented in [67], which discusses Routing Control Platforms (RCPs) in the context of OpenFlow/SDN. They proposed a controller-centric networking model along with a prototype implementation of an autonomous-system-wide abstract BGP routing service. E. Code Verification and Debugging Verification and debugging tools are vital resources for traditional software development and are no less important for SDN. Indeed, for the idea of portable network “apps” to be successful, network behavior must be thoroughly tested and verified. NICE [68] is an automated testing tool used to help uncover bugs in OpenFlow programs through model checking and symbolic execution. Anteater [69] takes a different approach by attempting to check network invariants that exist in the data plane, such as connectivity or consistency. The main benefit of this approach is that it is protocol-agnostic; it will also catch errors that result from faulty switch firmware or inconsistencies with the control plane communication. VeriFlow [70] has a similar goal, but goes further by proposing a real-time verification tool that resides between the controller and the forwarding elements. This adds the potential benefit of being able to halt bad rules that will cause anomalous behavior before they reach the network. Other efforts proposed debugging tools that provide insights gleaned from control plane traffic. OFRewind [71] allows network events (control and data) to be recorded at different granularities and later replayed to reproduce a specific sce￾nario, granting the opportunity to localize and troubleshoot the events that caused the network anomaly. ndb [72] implements breakpoints and packet-backtraces for SDN. Just as with the popular software debugger gdb, users can pinpoint events that lead to error by pausing execution at a breakpoint, or, using a packet backtrace, show the sequence of forwarding actions seen by that packet. STS [73] is a software-defined network troubleshooting simulator. It is written in python and depends on POX. It simulates the devices in a given network allowing for testing cases and identifying the set of inputs that generates a given error. V. SDN APPLICATIONS Software-defined networking has applications in a wide va￾riety of networked environments. By decoupling the control– and data planes, programmable networks enable customized control, an opportunity to eliminate middleboxes, as well as simplified development and deployment of new network services and protocols. Below, we examine different envi￾ronments for which SDN solutions have been proposed or implemented. A. Enterprise Networks Enterprises often run large networks, while also having strict security and performance requirements. Furthermore, different enterprise environments can have very different re￾quirements, characteristics, and user population, For example, University networks can be considered a special case of enterprise networks: in such an environment, many of the connecting devices are temporary and not controlled by the University, further challenging security and resource alloca￾tion. Additionally, Universities must often provide support for research testbeds and experimental protocols. Adequate management is critically important in Enterprise environments, and SDN can be used to programmatically enforce and adjust network policies as well as help monitor network activity and tune network performance. Additionally, SDN can be used to simplify the network by ridding it from middleboxes and integrating their functionality within the network controller. Some notable examples of middlebox functionality that has been implemented using SDN include NAT, firewalls, load balancers [74] [75], and network access control [76]. In the case of more complex middleboxes with functionalities that cannot be directly im￾plemented without performance degradation (e.g., deep packet inspection), SDN can be used to provide unified control and management[77]. The work presented in [78] addresses the issues related to consistent network updates. Configuration changes are a common source of instability in networks and can lead to outages, security flaws, and performance disruptions. In [78], a set of high-level abstractions are proposed that allow network administrators to update the entire network, guaran￾teeing that every packet traversing the network is processed by exactly one consistent global network configuration. To support these abstractions, several OpenFlow-based update mechanisms were developed. As discussed in earlier sections, OpenFlow evolved from Ethane [20], a network architecture designed specifically to address the issues faced by enterprise networks. B. Data Centers Data centers have evolved at an amazing pace in recent years, constantly attempting to meet increasingly higher and rapidly changing demand. Careful traffic management and policy enforcement is critical when operating at such large scales, especially when any service disruption or additional delay may lead to massive productivity and/or profit loss. Due to the challenges of engineering networks of this scale and