Openflow Controller Comparison Essay

OpenFlow's architecture is inefficient, and caps performance while sucking unnecessary power.

That's the conclusion of a bunch of Comp. Sci boffins from researchers at Australian brain box Data61 and Sydney University, who assessed four major OpenFlow controllers – NOX, Maestro, Floodlight and Beacon. Their paper is at Arxiv.

Poor old OpenDaylight was also tested but not reported: “the performance [was] too low to provide any insightful comparison”.

To cut to the chase: none of the controllers tested got anywhere close to line speed, whether running on a network processor (based on Tilera chips), or on a Xeon E5-2450-based server (more on the configurations later).

On the CBench software defined networking (SDN) controller benchmark, the best the Tilera setup achieved was just under five million requests per second, compared to a line rate of 29 million requests per second.

It seems that Intel's long years of work understanding packet processing is paying off: on the x86 setup, Beacon was able to hit 20 million requests per second; the maximum of the other controllers was 7 million requests per second.

Line rate? Forget it: the CSIRO/Data61/Sydney Uni benchmark results

Since SDN controllers have to deal with traffic as flows (meaning they have to remember MAC addresses so as to track conversations, compared to an Ethernet switch that only has to know which port it's forwarding traffic to), network scalability is also a big problem.

None of the controllers stayed near their peak performance with 10 million unique MAC addresses in the benchmark, and the Java-based controllers (Beacon and Floodlight) stagger to a near-complete halt at that scale.

The problem, the paper says, is that OpenFlow itself has architectural inefficiencies. The authors pick out serialisation; I/O threading; and, “the key data structure of the learning switch application: the hash table”.

Serialisation is by far the biggest overhead: even the most efficient controller “spends a fifth of his time in packet serialisation … this limitation is inherent to the object oriented design principle of these controllers. They all treat each single packet as an individual object, a limitation that induces an unaffordable per-packet overhead.”

The authors' proposal is for a new SDN controller design: “treat arriving packets with pre-allocated buffers rather than new objects,” they write. Controllers should also “be aware of the hardware characteristics to limit cache misses on multi-core platforms, or exploit the network-on-chip on many-core platforms”. ®

Sponsored: Unleash the potential of all-flash storage in your Data Center with Huawei

  • 1.

    Hakiri, A., Gokhale, A., Berthou, P., & Gayraud, T. (2014). Software-defined networking: Challenges and research opportunities for future internet. Computer Networks,75(A), 453–471. doi:10.1016/j.comnet.2014.10.015.CrossRefGoogle Scholar

  • 2.

    Turner, J., & McKeown, N. (2007). Can Overlay hosting services make IP ossification irrelevant? In Proceedings PRESTO: Workshop on programmable routers for the extensible service of tomorrow, Princeton, NJ.Google Scholar

  • 3.

    Astuto, B. N., Mendonca, M., Nguyen, X. N., Obraczka, K., & Turletti, T. (2014). A survey of software-defined networking: Past, present, and future of programmable networks. IEEE Communications Surveys and Tutorials,16(3), 1617–1634.CrossRefGoogle Scholar

  • 4.

    Software-Defined Networking (SDN) Definition—Open networking foundation, open networking foundation (online). https://www.opennetworking.org/sdn-resources/sdndefinition.

  • 5.

    Kreutz, D., Ramos, F. M. V., Verissimo, P., Rothenberg, C. E., Azodolmolky, S., & Uhlig, S. (2015). Software-defined networking: A comprehensive survey. Proceedings of the IEEE,103(1), 14–76.CrossRefGoogle Scholar

  • 6.

    SDN Resources, SDxCentral, (2015). (online) https://www.sdxcentral.com/resources/sdn/.

  • 7.

    Lara, A., Kolasani, A., & Ramamurthy, B. (2014). Network innovation using openflow: A survey. IEEE Communication Surveys and Tutorials,16(1), 493–512.CrossRefGoogle Scholar

  • 8.

    McKeown, N., Anderson, T., Balakrishnan, H., Parulkar, G., Peterson, L., Rexford, J., et al. (2008). OpenFlow: Enabling innovation in campus networks. ACM SIGCOMM Computer Communication Review,38(2), 69–74.CrossRefGoogle Scholar

  • 9.

    Doria, A., Salim, J. H., Haas, R., Khosravi, H., Wang, W., Dong, L., Gopal, R., & Halpern, J. (2010). Forwarding and control element separation (ForCES) protocol specification. Internet Engineering Task Force (online). Available: http://www.ietf.org/rfc/rfc5810.txt.

  • 10.

    Pfaff, B., & Davie, B. (2013). The Open vSwitch database management protocol, RFC 7047 (Informational), internet engineering task force (online). Available: http://www.ietf.org/rfc/rfc7047.txt.

  • 11.

    Song, H. (2013). Protocol-oblivious forwarding: Unleash the power of SDN through a future-proof forwarding plane. In Proceedings of the second ACM SIGCOMM workshop on hot topics in software defined networking. New York, NY, USA: ACM, pp. 127–132.Google Scholar

  • 12.

    Smith, M., Dvorkin, M., Laribi, Y., Pandey, V., Garg, P., & Weidenbacher, N. (2014). OpFlex control protocol, internet draft, internet engineering task force (online). http://tools.ietf.org/html/draft-smith-opflex-00.

  • 13.

    Bianchi, G., Bonola, M., Capone, A., & Cascone, C. (2014). OpenState: Programming platform-independent stateful OpenFlow applications inside the switch. SIGCOMM Computer Communication Review,44(2), 44–51.CrossRefGoogle Scholar

  • 14.

    OpenFlow Current Deployment, OpenFlow. (2011). (online). http://archive.openflow.org/wp/current-deployments/.

  • 15.

    Richardson, L., & Ruby, S. (2008). RESTful web services. Sebastopol: O’Reilly Media, Inc.Google Scholar

  • 16.

    Foster, N., Harrison, R., Freedman, M. J., Monsanto, C., Rexford, J., Story, A., & Walker, D. (2011). Frenetic: A network programming language. SIGPLAN Notices, 46(9), 279–291.CrossRefMATHGoogle Scholar

  • 17.

    Voellmy, A., & Hudak, P. (2011). Nettle: Taking the sting out of programming network routers. In Proceedings of the 13th international conference on practical aspects of declarative languages, ser. PADL’11. Berlin, Heidelberg: Springer, pp. 235–249.Google Scholar

  • 18.

    Monsanto, C., Foster, N., Harrison, R., & Walker, D. (2012). A compiler and run-time system for network programming languages. SIGPLAN Notices,47(1), 217–230.CrossRefGoogle Scholar

  • 19.

    Voellmy, A., Kim, H., & Feamster, N. (2012). Procera: A language for highlevel reactive network control. In Proceedings of the first workshop on hot topics in software defined networks, ser. HotSDN’12. New York, NY, USA: ACM, pp. 43–48.Google Scholar

  • 20.

    Monsanto, C., Reich, J., Foster, N., Rexford, J., & Walker, D. (2013). Composing software-defined networks. In Proceedings of the 10th USENIX conference on Networked Systems Design and Implementation, ser. nsdi’13. Berkeley, CA, USA: USENIX Association, pp. 1–14.Google Scholar

  • 21.

    Anderson, C. J., Foster, N., Guha, A., Jeannin, J.-B., Kozen, D., Schlesinger, C., et al. (2014). NetKAT: Semantic foundations for networks. SIGPLAN Notices,49(1), 113–126.MATHGoogle Scholar

  • 22.

    OpenFlow Switch Specification, Version 1.0.0 (Wire Protocol 0x01), December 2009 (online). http://www.openflow.org/documents/openflow-spec-v1.0.0.pdf.

  • 23.

    OpenFlow Switch Specification, Version 1.4.0 (Wire Protocol 0x05), October 2013 (online). https://www.opennetworking.org/images/stories/downloads/sdn-resources/onf-specifications/openflow/openflow-spec-v1.4.0.pdf.

  • 24.

    Home—Open Networking Foundation, 2013 (online). https://www.opennetworking.org/index.php?lang=en.

  • 25.

    Bholebawa, I. Z., Jha, R. K., & Dalal, U. D. (2016). Performance analysis of proposed openflow-based network architecture using mininet. Wireless Personal Communication,86(2), 943–958. doi:10.1007/s11277-015-2963-4.CrossRefGoogle Scholar

  • 26.

    Liu, Y., Li, Y., Wang, Y., & Yuan, J. (2015). Optimal scheduling for multi-flow update in software-defined networks. Journal of Network and Computer Applications,54(C), 11–19.CrossRefGoogle Scholar

  • 27.

    Metter, C., Seufert, M., Wamser, F., Zinner T., & Tran-Gia, P. (2017). Analytic model for SDN controller traffic and switch table occupancy. In IEEE 12th international conference on network and service management (CNSM), pp. 109–117. doi: 10.1109/CNSM.2016.7818406.

  • 28.

    Gude, N., Koponen, T., Pettit, J., Pfaff, B., Casado, M., McKeown, N., & Shenker, S. (2008). NOX: Towards an operating system for networks. ACM SIGCOMM Computer Communication Review, 38(3), 105–110.CrossRefGoogle Scholar

  • 29.

    McCauley, M. (2009). POX (online). http://www.noxrepo.org/.

  • 30.

    Floodlight is a Java-based OpenFlow controller. (2012). (online). http://floodlight.openflowhub.org/.

  • 31.

    Saikia, D. (2013). MuL OpenFlow controller (online). http://sourceforge.net/projects/mul/.

  • 32.

    Takamiya, Y., & Karanatsios, N. (2012). Trema OpenFlow controller framework (online). https://github.com/trema/trema.

  • 33.

    Erickson, D. (2013). The Beacon OpenFlow controller. In Proceedings of the second ACM SIGCOMM workshop on Hot topics in software defined networking, ser. HotSDN’13. New York, NY, USA: ACM, pp. 13–18.Google Scholar

  • 34.

    Cai, Z., Cox, A. L. & Ng, T. S. E. (2011). Maestro: A system for scalable openflow control, Rice University, Tech. Rep.Google Scholar

  • 35.

    Nippon Telegraph and Telephone Corporation. (2012). Ryu network operating system (online). http://osrg.github.com/ryu/.

  • 36.

    Naning, H. S., Munadi R., & Effendy, M. Z. (2017). SDN controller placement design: For large scale production network. In IEEE Asia Pacific conference on wireless and mobile (APWiMob), pp. 74–79. doi:10.1109/APWiMob.2016.7811452.

  • 37.

    Lantz, B., Heller B., & McKeown, N. (2010). A network in a laptop: Rapid prototyping for software-defined networks. In Hotnets-IX proceedings of the 9th ACM SIGCOMM workshop on hot topics in networks, New York, NY, USA.Google Scholar

  • 38.

    Team, T. M. (2012). Mininet: An instant virtual network on your laptop (or Other PC) (online). http://www.mininet.org.

  • 39.

    McCauley, M. (2009). About POX | NOXRepo (online). http://www.noxrepo.org/pox/about-pox/.

  • 40.

    McCauley, M. (2013). Noxrepo/pox. Github (online). https://github.com/noxrepo/pox.

  • 41.

    McCauley, M. (2014). POX Wiki—Open Netwroking Lab—confluence (online). https://openflow.stanford.edu/display/ONL/POX+Wiki.

  • 42.

    Big Switch Networks, Inc. | The Leader in Open Software Defined Networking. (2015). Big switch networks (online). http://www.bigswitch.com/.

  • 43.

    Floodlight OpenFlow Controller—Project Floodlight. (2015). Big switch network (online). http://www.projectfloodlight.org/floodlight/.

  • 44.

    Architecture—Floodlight Controller—Project Floodlight. (2012). Big Switch Network (online). https://floodlight.atlassian.net/wiki/display/floodlightcontroller/Architecture.

  • 45.

    Jha, R. K., Kharga, P., Bholebawa, I. Z., Satyarthi, S., Gupta, A., & Kumari, S. (2014). OpenFlow technology: A journey of simulation tools. International Journal of Computer Network and Information Security (IJCNIS),6(11), 49–55.CrossRefGoogle Scholar

  • 46.

    Blial, O., Mamoun, M. B., & Benaini, R. (2016). An overview on SDN architectures with multiple controllers. Journal of Computer Networks and Communications. doi:10.1155/2016/9396525.Google Scholar

  • 47.

    Ma, Y. W., Chen, J. L., Tsai, Y. H., Cheng, K. H., & Hung, W. C. (2016). Load-balancing multiple controllers mechanism for software-defined networking. Wireless Personal Communications. doi:10.1007/s11277-016-3790-y.Google Scholar

  • 0 comments

    Leave a Reply

    Your email address will not be published. Required fields are marked *