g2k16 Hackathon Report: Alexander Bluhm on mbufs and more

Alexander Bluhm (bluhm@) contributed our next report (which even includes a picture):

Big plans what to do at a hackathon don't work. There is alwayssomething unexpected that requires your attention. So I was expectingthe unexpected. The TCP send performance has dropped to a very lowthroughput in some environments. It was pretty clear that it wasrelated with claudio@'s change to speed up TCP by using large mbufsinstead of chaining small ones. Mbufs are used inside the kernelto hold network data. Using a lot of them requires many allocationsand frees. This can be avoided by using larger mbufs. But why didit get slower? Together with mikeb@ we found out that it was relatedto the mbuf space limit in the socket buffer. One large mbuf filledthe send buffer, so no new mbufs could be inserted until TCP receivedthe acknowledgements for everything. So the sliding window algorithmwith mbufs cycling through the socket buffer did not work anymore.After identifying the problem, the fix was easy, just increase thedefault socket buffer mbuf size limit.