Genode 16.11 revisits low-level protocols

Following the feature-rich release in August, with the new version 16.11, Genode's developers took the chance to work on long-standing architectural topics, most prominently the low-level interplay between parent and child components. Besides this low-level work, the release features much improved virtual-networking capabilities. Originally introduced in the previous version, Genode's network-routing mechanism has become more versatile and easier to use. Further topics include the added support for smart cards, kernel improvements of the NOVA hypervisor, and a virtual file system for generating time-based passcodes.The efficient interaction between user-level components is one of the most important aspects of microkernel-based systems like Genode. The design space for this interplay is huge and there is no widely accepted consensus about the "right" way. The options include message passing between independent threads, the migration of threads between address spaces, shared memory, and various flavours of asynchronous communication.When the Genode project originally emerged from the L4 community, it was somehow preoccupied with the idea that synchronous IPC is the best way to go. After all, the sole reliance on unbuffered synchronous IPC was widely regarded as the key for L4's excellent performance. Over the years, however, the mindset of the Genode developers shifted away from this position. Whereas synchronous IPC was found to be a perfect match for some use cases, it needlessly complicated others. It turns out that any IPC mechanism is ultimately a trade-off between low latency, throughput, simplicity, and scalability. Finding a single sweet spot that fits well for all parts of an operating system seems futile. Given this realization and countless experiments, Genode's inter-component protocols were gradually shaped towards the combination of synchronous IPC where low-latency remote procedure calls are desired, asynchronous notifications, and shared memory. That said, Genode's most fundamental inter-component communication protocol - the interplay between parent and child components to establish communication sessions between clients and servers - remained unchanged since the very first version. The current release reconsiders the architectural decisions made in the early days and applies Genode's modern design principles to these low-level protocols. The release documentation contrasts the original design that was solely based on synchronous IPC with the new way. Even though the new version overcomes long-standing limitations of the original design, at the first glance, it gives the impression to be more complicated and expensive in terms of the number of context switches. Interestingly, however, the change has no measurable effect on the performance of even the most dynamic system scenarios. The apparent reason is that the parent-child interactions make up a minuscule part of the overall execution time in real-world scenarios.Even though the architectural work mentioned above is fundamental to the Genode system as a whole, it is barely visible to users of the framework. With respect to user-visible changes, the most prominent improvement is the vastly improved infrastructure for virtual networking, which is covered in great detail in the release documentation. Further topics are the added support for using smart cards, a new VFS plugin for generating time-based passcodes, and updated versions of VirtualBox 4 and 5 running of top of NOVA. Speaking of NOVA, the release improves this kernel in several respects, in particular by adding support for asynchronous map operations. Each of the topics is covered in more depth in the release documentation.