LWN.net

Greg Kroah-Hartman has released a full set of stable kernels: 4.16.7, 4.14.39, 4.9.98, 4.4.131, and 3.18.108. All of them contain important fixesand users should update.

Containers are, of course, all the rage these days; in fact, during his2018 Legal andLicensing Workshop (LLW) talk, Dirk Hohndelsaid with a grin that he hears "containers may take off". But, whilecontainers are easy to set up and use, license compliance for containers is "incrediblyhard". He has been spending "way too much time" thinking about containercompliance recently and, beyond the standard "let's go shopping" solutionto hard problems, has come up with some ideas.Hohndel is a longtime member of the FOSS community who is now the chiefopen source officer at VMware—a company that ships some container images.

Google has announcedthe open-sourcing of gVisor, a sandboxed container runtime."gVisor is more lightweight than a VM while maintaining a similarlevel of isolation. The core of gVisor is a kernel that runs as a normal,unprivileged process that supports most Linux system calls. This kernel iswritten in Go, which was chosen for its memory- and type-safety. Just likewithin a VM, an application running in a gVisor sandbox gets its own kerneland set of virtualized devices, distinct from the host and othersandboxes."

Security updates have been issued by Debian (kernel), Fedora (haproxy), openSUSE (flac, GraphicsMagick, and quassel), Oracle (kernel), Red Hat (python-paramiko and redhat-virtualization-host), and SUSE (corosync).

There is a new initiative in the Fedora community based on what used to becalled "Fedora Atomic Workstation". From thiswhitepaper [PDF]: "The descriptive name for this product is ​image-mode container-based Fedora Workstation based on rpm-ostree, which isclear but terrible for branding. Therefore, we call it Team Silverblue.The long-term goal for this effort is to transform Fedora Workstation intoan image-based system where applications are separate from the OS andupdates are atomic."

Version 8.1 of the GCC compiler suite is out."Are you tired of your existing compilers?Want fresh new language features and better optimizations?Make your day with the new GCC 8.1!" See this page for a completelist of changes in this release.

Martin Pitt describes hisexperience running a fully free-software Android phone."I previously used Opera as a web browser, because it is relativelylightweight (important on my previous phone) and the really good builtin adblocker. But these days Firefox is really fast and good enough, so Ireplaced it with Fennec, which is more or less Firefox with some non-freebits removed. After installing uBlock Origin I’ve never lookedback."

Christoph Lameter works in a different computing environment than most ofus; he supports high-volume trading applications that need every bit ofperformance that the fastest hardware can give them. Even then, it seemsthat isn't fast enough. In a memory-management-track session at the 2018Linux Storage, Filesystem, and Memory-Management Summit, Lameter describedsome of the problems he has encountered and approaches he is considering toaddress them.

Allocating chunks of memory that are both large and physically contiguoushas long been a difficult thing to do in the kernel. But there are times wherethere is no alternative. Two sessions in the memory-management track ofthe 2018 Linux Storage, Filesystem, and Memory-Management Summit exploredways of making those allocations more reliable. It turns out that some usecases have a rather larger value of "large" than others.

Memory control groups allow the system administrator to impose memory-uselimits on the members of control groups. In many ways, these limits behavelike the overall limit on available memory, but there are also somedifferences. The behavior of the memory controller also changed with theadvent of the version-2 control-group API, creating problems for at leastone significant user. Three sessions held in the memory-management track ofthe Linux Storage, Filesystem, and Memory-Management Summit explored someof these problems.

The recent fsync() woesexperienced by PostgreSQL led to a session on the firstday (April 23) of the 2018 Linux Storage, Filesystem, and Memory-Management Summit (LSFMM). Those problemsalso led to a second-day session with PostgreSQL developer Andres Freund who gave anoverview of how PostgreSQL does I/O and where that ran aground on someassumptions that had been made. The session led to a fair amount ofdiscussion with the filesystem-track developers; real solutions seem to bein the offing.

One of the core jobs of the memory-management subsystem is to make memoryavailable to other parts of the kernel when the need arises. Thememory-management track of the 2018 Linux Storage, Filesystem, andMemory-Management Summit hosted a pair of sessions on new or improvedallocation functions for the kernel covering the slab allocators andprotectable memory.

Security updates have been issued by Fedora (cups-filters, ghostscript, glusterfs, PackageKit, qpdf, and xen), Mageia (anki, libofx, ming, sox, webkit2, and xdg-user-dirs), Oracle (corosync, java-1.7.0-openjdk, and pcs), Red Hat (java-1.7.0-openjdk), Scientific Linux (corosync, firefox, gcc, glibc, golang, java-1.7.0-openjdk, java-1.8.0-openjdk, kernel, krb5, librelp, libvncserver, libvorbis, ntp, openssh, openssl, PackageKit, patch, pcs, policycoreutils, qemu-kvm, and xdg-user-dirs), Slackware (libwmf and mozilla), and Ubuntu (apache2, ghostscript, mysql-5.7, wavpack, and webkit2gtk).

The Fedora 28 release has been announced."The headline feature for Fedora 28 Server is the inclusion of thenew Modular repository. This lets you select between different versions ofsoftware like NodeJS or Django, so you can chose the stack you need foryour software." Some users will also appreciate that proprietaryblobs (such as the NVIDIA drivers) are now easier to obtain and install.

Matthew "Willy" Wilcox has been doing a fair amount of work in thememory-management area recently. He showed up at the 2018 Linux Storage,Filesystem, and Memory-Management Summit with a list of discussion topicsrelated to that work; it was enough to fill a plenary session with somespillover into the memory-management track the next day. Some of histopics were fairly straightforward; others look to be somewhat moreinvolved.

The kernel's memory-management subsystem has to manage a great deal ofconcurrency; that leads to an ongoing series of locking challenges thatsometimes seem intractable. Two recurring locking issues — the LRU locksand the mmap_sem lock — were the topic of sessions held during thememory-management track of the 2018 Linux Storage, Filesystem, andMemory-Management Summit. In both cases, it quickly became clear that,while some interesting ideas are being pursued, easysolutions are not on offer.

When kernel code needs to work directly with user-space pages, it oftencalls get_user_pages()(or one of several variants) to fault those pages into RAM and pin themthere. This function is not entirely easy to use, though, and recentchanges have made it harder to use safely. Jan Kara and Dan Williams led aplenary session at the 2018 Linux Storage, Filesystem, andMemory-Management Summit to discuss potential solutions, but it is notentirely clear that any were found.

Stable kernels 4.16.6, 4.14.38, 4.9.97, 4.4.130, and 3.18.107 have been released. They all containimportant fixes throughout the tree and users should upgrade.

Security updates have been issued by Arch Linux (drupal), Debian (chromium-browser, gunicorn, libvorbis, openjdk-8, roundcube, sdl-image1.2, slurm-llnl, and tor), Fedora (boost, cups-filters, ghostscript, gsoap, memcached, mod_http2, and qpdf), openSUSE (Chromium and mysql-community-server), and Red Hat (glusterfs, OpenShift Container Platform 3.1, OpenShift Container Platform 3.2, OpenShift Container Platform 3.3, OpenShift Container Platform 3.4, OpenShift Container Platform 3.5, OpenShift Container Platform 3.6, OpenShift Container Platform 3.7, OpenShift Container Platform 3.8, OpenShift Container Platform 3.9, and openvswitch).

The 4.17-rc3 kernel prepatch is out."And by now, I think we've fixed all the nastiest fall-out from themerge window. In particular, the PTI large-page fallout that hit somepeople with particular configurations should all be good."

The memory-management subsystem is maintained by a small but dedicatedgroup of developers. How healthy is that development community? MichalHocko raised that question during the memory-management track at the 2018Linux Storage, Filesystem, and Memory-Management Summit. Hocko is worried,but it appears that his concerns are not universally felt.

The non-uniform memory architecture (NUMA) was designed around the ideathat there are two types of memory on complex systems: local (faster) andremote (slower). During the memory-management track of the 2018 LinuxStorage, Filesystem, and Memory-Management Summit, Anshuman Khandualasserted that the situation has since become rather more complicated.Perhaps, he said, the time has come to rethink how we view NUMA systems.

Storage devices are in a period of extensive change. As theyget faster and become byte-addressable by the CPU, they tend to lookincreasingly like ordinary memory. But they aren't memory, so it stillisn't clear what the best model for accessing them should be. AdamManzanares led a session during the memory-management track of the 2018Linux Storage, Filesystem, and Memory-Management Summit, where his proposalof a new access mechanism ran into some skepticism.

Heterogeneous memory management (HMM) is a relatively new kernel subsystemthat allows the system to manage peripherals (such as graphics processors)that have their own memory-management units. In two sessions during thememory-management track of the 2018 Linux Storage,Filesystem, and Memory-Management Summit, HMM creator Jérôme Glisseprovided an update on the status of this subsystem and where it is going,along with a more detailed look at the memory-management unit (MMU)notifiers mechanism on which it depends.

At the 2018 Linux Storage, Filesystem, and Memory Management Summit, TedTs'o introduced an integrity feature akin to dm-verity that targets Android,at least to start with. It is meant to protect the integrity of files onthe system so that any tampering would be detectable. Theinitial use case would be for a certain special type of Android file, but othersystems may find uses for it as well.

Security updates have been issued by Debian (wordpress), Fedora (boost), openSUSE (perl and zsh), Oracle (kernel), Red Hat (apr), and Slackware (openvpn).

Ubuntu 18.04, a long-term-support release, is out."Codenamed 'Bionic Beaver', 18.04 LTS continues Ubuntu's proud traditionof integrating the latest and greatest open source technologies into ahigh-quality, easy-to-use Linux distribution. The team has been hard atwork through this cycle, introducing new features and fixing bugs."It features a 4.15 kernel, a new GNOME-based desktop environment, andmore. See therelease notes and this overview for details.

Christian Schaller looksforward to the Fedora 28 release (which will evidently be the first on-time Fedora release ever)."The Spectre/Meltdown situation did hammer home to a lot of peoplethe need to have firmware updates easily available and easy to update. Wecreated the Linux Vendor Firmware service for Fedora Workstation users withthat in mind and it was great to see the service paying off for many Linuxusers, not only on Fedora, but also on other distributions who startedusing the service we provided. I would like to call out to Dell who was acritical partner for the Linux Vendor Firmware effort from day 1 and thustheir users got the most benefit from it when Spectre and Meltdownhit. Spectre and Meltdown also helped get a lot of other vendors off thefence or to accelerate their efforts to support LVFS and Richard Hughes andPeter Jones have been working closely with a lot of new vendors during thiscycle to get support for their hardware and devices into LVFS."

The page structure is one of themost complex in the kernel due to the need to cram the maximum amount ofinformation into as little space as possible. Each field is so heavilyoverloaded that developers prefer to avoid making changes to structpage if they can avoid it. That didn't deter Jérôme Glisse from proposing a significant changeduring two plenary sessions atthe 2018 Linux Storage, Filesystem, and Memory-Management Summit, though.There are some interesting benefits on offer, but getting there will not bea simple task.

Stable kernels 4.16.5 and 4.14.37 have been released. They both containimportant fixes and users should upgrade.

Security updates have been issued by Debian (drupal7, gcc-4.9-backport, ghostscript, and openslp-dfsg), Fedora (anki, composer, perl, and perl-Module-CoreList), Red Hat (kernel and rh-mysql56-mysql), and SUSE (kernel, kvm, and zsh).

The LWN.net Weekly Edition for April 26, 2018 is available.

Once a niche feature, memory encryption is becoming mainstream with supportin both Intel and AMD processors, Kirill Shutemov said at the beginning ofhis session during the memory-management track of the 2018 Linux Storage,Filesystem, and Memory-Management Summit. Memory encryption can harden thesystem against attack, but it also presents some interesting challenges forthe kernel.

Security updates have been issued by Debian (lucene-solr and psensor), Oracle (librelp and PackageKit), Red Hat (kernel, librelp, and PackageKit), Scientific Linux (librelp), and Ubuntu (mysql-5.5 and packagekit).

After a session at last year's LinuxStorage, Filesystem, and Memory Management Summit (LSFMM), Jeff Layton was able tomake some improvements to block-layer errorhandling. Those changes, which added a newerrseq_t type to hold an error number and sequence number, seemedto help and were well received—except by the PostgreSQLdevelopers. So Layton led a session at the 2018 LSFMM to discuss waysto improve things further; it would be followed later in the week with asession by one of the PostgreSQL developers to look at the specifics of theproblem from their perspective.

Using the kernel thread (kthread) freezer has been a longtime problemfor a variety of reasons. It is meant as a way to suspend kthreads on theway toward system suspend, but in practice has proved problematic to thepoint that it came up at both the 2015 and2016 Kernel Summits (as well as on themailing lists over the years); the intent is to tryto remove the kthread freezer entirely. To that end, Luis Rodriguez led adiscussion in the filesystem track of the 2018 Linux Storage, Filesystem,and Memory-Management Summit on the problems and possible solutions.

Dave Hansen did much of the work to get kernel page-table isolation(PTI) into the kernel in response to the Meltdown CPU vulnerability. In thememory-management track of the2018 Linux Storage, Filesystem, and Memory-Management Summit, he ran adiscussion on how PTI came about, what the costs are, and what can be doneto minimize its performance impact.

Ever since kernel page-table isolation(PTI) was introduced as a mitigation for the Meltdown CPU vulnerability, users have worried about how it affects theperformance of their systems. Most of that concern has been directedtoward its impact on computing performance, but I/O performance alsomatters. At the 2018 Linux Storage, Filesystem, and Memory-ManagementSummit, Ming Lei presented some preliminary work he has done to try toquantify how severely PTI affects block I/O operations.

Stable kernels 4.16.4, 4.14.36, 4.9.96, 4.4.129, and 3.18.106 have been released. All of themcontain important fixes and users should update.

Security updates have been issued by Arch Linux (roundcubemail, xfig, and zsh), Debian (linux-tools), Fedora (java-1.8.0-openjdk and mingw-libid3tag), Gentoo (chromium), openSUSE (hdf5, ocaml, PackageKit, phpMyAdmin, salt, and virtualbox), Oracle (patch), Red Hat (java-1.6.0-sun, java-1.7.0-oracle, java-1.8.0-oracle, patch, and python-paramiko), Scientific Linux (patch), SUSE (kernel and PackageKit), and Ubuntu (linux, linux-aws, linux-kvm, linux-raspi2, linux-snapdragon, linux, linux-raspi2, linux-azure, linux-euclid, linux-hwe, linux-gcp, linux-oem, linux-lts-xenial, linux-aws, and mysql-5.5, mysql-5.7).

It is a good thing that strong coffee was served at the 2018 Linux Storage,Filesystem, and Memory-Management Summit; full awareness was required fromthe first session, in which Josef Bacik discussed some issues that havearisen in the interaction between filesystems and the memory-managementsubsystem. Filesystems cache a lot of data from files, but also a lot ofmetadata about those files. It turns out, though, that management of thecached metadata does not work as well as one might like.

At the 2018 Legal andLicensing Workshop (LLW), which is a yearly gathering of lawyers and technical folks organized by the Free Software FoundationEurope (FSFE), attendees got more details on a recent hearing in a German GPLenforcement case. Marcus von Welser is a lawyer who represented thedefendant, Geniatech, in a case that was brought by PatrickMcHardy. In the presentation, von Welser was joined by Armijn Hemel, who helped Geniatech in its compliance efforts. The hearingwas of interest for a number of reasons, not least because McHardywithdrew his request for an injunction once it became clear that the judgewas leaning infavor of the defendants—effectively stopping this case dead in its tracks.

Daniel Vetter looks atsome kernel-development statistics, with a focus on patches written bythe maintainers who commit them. "Naively extrapolating the relative trend predicts that around the year 2025 large numbers of kernel maintainers will do nothing else than be the bottleneck, preventing everyone else from getting their work merged and not contributing anything of their own. The kernel community imploding under its own bureaucratic weight being the likely outcome of that.This is a huge contrast to the 'everything is getting better, bigger, andthe kernel community is very healthy' fanfare touted at keynotes and theyearly kernel report. In my opinion, the kernel community is very much notlooking like it is coping with its growth well and an overall healthycommunity."

Each kernel development cycle includes a vast number of changes that arenot intended to change visible behavior and which, as a result, gounnoticed by most users and developers. One such change in 4.17 is arewiring of how system-call implementations are invoked within the kernel.The change is interesting, though, and provides an opportunity to look atthe macro magic that handles system-call definitions.

Security updates have been issued by Debian (gunicorn, libreoffice, libsdl2-image, ruby1.8, and ruby1.9.1), Fedora (java-1.8.0-openjdk, jgraphx, memcached, nghttp2, perl, perl-Module-CoreList, and roundcubemail), Gentoo (clamav, librelp, mbedtls, quagga, tenshi, and unadf), Mageia (freeplane, libcdio, libtiff, thunderbird, and zsh), openSUSE (cfitsio, chromium, mbedtls, and nextcloud), and Red Hat (chromium-browser, kernel, and rh-perl524-perl).

The 4.17-rc2 kernel prepatch is out."We've still got some known fallout from the merge window, but itshouldn't affect most normal configurations, so go out and test."

The first article in this series describedthe interface to the "rhashtable"resizable hash-table abstraction in Linux 4.15. While a knowledge ofthe interface can result in successful use of rhashtables, it oftenhelps to understand what is going on "under the hood", particularly whenthose details leak out through the interface, as is occasionally thecase with rhashtable. The centerpiece for understanding theimplementation is knowing exactly how the table is resized. So thisfollow-on article will explain that operation; it will also present theconfiguration parameters that were skimmed over last time and discusshow they affect the implementation.

Version 4.0 of the FFmpegmultimedia toolkit is out. There is a long list of new filters, formats,and more; see the announcement for details.

The furor over the Meltdown and Spectre vulnerabilities has calmed a bit —for now, at least — but that does not mean that developers have stoppedworrying about them. Spectre variant 1 (the bounds-check bypassvulnerability) has been of particular concern because, while the kernel isthought to contain numerous vulnerable spots, nobody really knows how tofind them all. As a result, the defenses that have been developed forvariant 1 have only been deployed in a few places. Recently, though,Dan Carpenter has enhanced the smatch tool to enable it to find possiblyvulnerable code in the kernel.

Greg Kroah-Hartman has released stable kernel 4.9.95 with important fixes throughout thetree. Users should update.