LWN.net

The LWN.net Weekly Edition for June 28, 2018 is available.

While there has been quite a bit of work on various aspects of networkingperformance, including bufferbloat reduction, queue management, and more,much of that work has been oriented toward the needs of high-end users.But there is more to the Internet than data centers and high-speed links.A large number of Internet-connected devices can be found behindconsumer-level routers on relatively slow broadband links. For some time,a group of developers has been working on the "Common Applications KeptEnhanced" (CAKE) queuing discipline, which is aimed directly at the needsof those users.

Collabora introducesdebos, a tool to build customized images for Debian and derivatives. "As opposite to debootstrap and other tools, debos doesn't need to be run as root for making actions that require root privileges in the images. debos uses fakemachine a library that setups qemu-system allowing you to work in the image with root privileges and to create images for all the architectures supported by qemu user."

Python 3 adoption has clearly picked up over the last few years, thoughthere is still a long way to go. Big Python-using companies tend to have awhole lot of Python 2.7 code running on their infrastructure and Facebook isno exception. But Jason Fried came to PyCon 2018 to describe what hashappened at the company over the last four years or so—it has gone fromusing almost no Python 3 to it becoming the dominant version of Pythonin the company. He was instrumental in helping to make that happen and his talk [YouTube video]may provide other organizations with some ideas on how to tackle theirmigration.

Security updates have been issued by Arch Linux (cantata and qutebrowser), Debian (imagemagick, php5, and redis), Fedora (cri-o and libgxps), Oracle (glibc, kernel, libvirt, samba, samba4, sssd and ding-libs, and zsh), Red Hat (ansible, dpdk, kernel, kernel-alt, kernel-rt, libvirt, pki-core, podman, qemu-kvm, and qemu-kvm-rhev), Scientific Linux (kernel, libvirt, pki-core, and qemu-kvm), SUSE (firefox, gcc43, and kernel), and Ubuntu (openssl).

One of the new features merged for the 4.18 kernel is a new polling interface using the asynchronousI/O mechanism. As part of this work, the internal implementation of howthe various polling-related system calls (poll(),select(), and epoll_wait()) work was significantlychanged. The reporting of a significant performance regression has now putall of that work into doubt, though. While it could be reverted, the morelikely outcome would appear to be another set of changes to how pollingworks in the kernel.

Mozilla has announcedthe release of Firefox 61. Key highlights include the ability to easilyadd custom search engines, speedier response times when switching betweentabs, retained display lists, an accessibility Tools Inspector, andWebExtension Tab Management. See the releasenotes for additional information.

The combination of an "unsuspecting library employee" and a bunch of boredchildren has created a popular program using the Raspberry Pi and othertools to teachcoding to kids. Qumisha Goss is a librarian at the Parkman branch of theDetroit Public Library; she started the "Parkman Coders" program and came to PyCon 2018 in Cleveland, Ohio to tell the assembled Pythonistas all about it. She also hadsome thoughts on ways to make the Python community a more diverse place,along with some concerns for her students that are much bigger thanthe diversity topic.

Security updates have been issued by Slackware (firefox), SUSE (gpg2 and zlib), and Ubuntu (openssl, openssl1.0).

The latest set of stable kernel updates consists of4.17.3,4.16.18,4.14.52, and4.9.110. Each contains a fair number ofimportant updates. Note that 4.16.18 is the end of the line for the 4.16series.

As attackers have lost the easy ability to execute code stored in writablememory, they have increasingly turned to return-orientedprogramming (ROP) and related techniques to compromise vulnerablesystems. ROP attacks use the code that is present in the program underattack and are hard to defend against in software. In response, hardwarevendors are developing ways to defeat ROP-like techniques at a lowerlevel. One of the results is Intel's Control-FlowEnforcement Technology (CET) [PDF], which adds two mechanisms (shadowstacks and indirect-branch tracking) that are intended to resist theseattacks. Yu-cheng Yu recently posted a set of patches showing how this technology is to be used to defend Linuxsystems.

SUSE has announcedthe release of SUSE Linux Enterprise 15, SUSE Manager 3.2, and SUSE LinuxEnterprise High Performance Computing 15. "SUSE Linux Enterprise 15 is a modern, modular operating system that helps simplify multimodal IT, makes traditional IT infrastructure more efficient and provides an engaging platform for developers. As a result, customers can easily deploy and transition business-critical workloads across on-premise and public cloud environments."

Security updates have been issued by CentOS (git), Debian (bouncycastle and lava-server), Fedora (ansible, epiphany, kernel, kernel-tools, matrix-synapse, mingw-podofo, pass, podofo, python-prometheus_client, redis, rubygem-sinatra, and thunderbird-enigmail), Gentoo (file and pnp4nagios), Mageia (file, glibc, kernel, librsvg, and libvorbis), openSUSE (go1.9, mariadb, phpMyAdmin, and redis), and SUSE (firefox, kernel modules packages, and python).

Systemd v239 has been released with a long list of changes; click below forthe full set. "A new system.conf setting NoNewPrivileges= is now available which may be used to turn off acquisition of new privileges system-wide (i.e. set Linux' PR_SET_NO_NEW_PRIVS for PID 1 itself, and thus also for all its children). Note that turning this option on means setuid binaries and file system capabilities lose their special powers. While turning on this option is a big step towards a more secure system, doing so is likely to break numerous pre-existing UNIX tools, in particular su and sudo."

Version 5.28.0 of the Perl language has been released."Perl 5.28.0 represents approximately 13 months of development since Perl5.26.0 and contains approximately 730,000 lines of changes across 2,200files from 77 authors". The full list of changes can be found overhere; some highlights include Unicode 10.0 support, string- andnumber-specific bitwise operators, a change to more secure hash functions,and safer in-place editing.

The second 4.18 kernel prepatch is out fortesting. "Anyway, it's early in the rc series yet, but things lookfairly normal."

A new archive of linux-kernel mailing list (LKML) posts going back to 1998 is now available at lore.kernel.org. It is based on public-inbox (which we looked at back in February. Among other things, public-inbox allows retrieving the entire archive via Git: "Git clone URLs are provided at the bottom of each page. Note, that due to its volume, the LKML archive is sharded into multiple repositories, each roughly 1GB in size. In addition to cloning from lore.kernel.org, you may also access these repositories on git.kernel.org." The full announcement, which includes information about a new Patchwork instance as well as ways to link into the new archive, can be found on kernel.org.

Security updates have been issued by Debian (php-horde-image), openSUSE (kernel), Scientific Linux (git), SUSE (bluez, kernel, mariadb, and mariadb, mariadb-connector-c, xtrabackup), and Ubuntu (openjdk-7).

On his blog, James Bottomley looks at the value proposition for various types of cloud deployments. In particular, he compares the vertical and horizontal attack profile (VAP and HAP) of four different models: separate servers, separate logins on a single server, virtual machines, and containers. He finds the container story to be compelling: "The total VAP here is identical to that of physical infrastructure. However, the Tenant component is much smaller (the kernel accounting for around 50% of all vulnerabilities). It is this reduction in the Tenant VAP that makes containers so appealing: the CSP [cloud service provider] is now responsible for monitoring and remediating about half of the physical system VAP which is a great improvement for the Tenant. Plus when the CSP remediates on the host, every container benefits at once, which is much better than having to crack open every virtual machine image to do it. Best of all, the Tenant images don’t have to be modified to benefit from these fixes, simply running on an updated CSP host is enough. However, the cost for this is that the HAP is the entire linux kernel syscall interface meaning the HAP is much larger than then hypervisor virtual infrastructure case because the latter benefits from interface narrowing to only the hypercalls (qualitatively, assuming the hypercall interface is ~30 calls and the syscall interface is ~300 calls, then the HAP is 10x larger in the container case than the hypervisor case); however, thanks to protections from the kernel namespace code, the HAP is less than the shared login server case. Best of all, from the Tenant point of view, this entire HAP cost is borne by the CSP, which makes this an incredible deal: not only does the Tenant get a significant reduction in their VAP but the CSP is hugely motivated to keep on top of all vulnerabilities in their part of the VAP and remediate very fast because of the business implications of a successful horizontal attack."

Security updates have been issued by openSUSE (cobbler and matrix-synapse), Oracle (git), Red Hat (git), SUSE (java-1_7_1-ibm, nagios-nrpe, and ntp), and Ubuntu (AMD microcode).

The LWN.net Weekly Edition for June 21, 2018 is available.

Stable kernels 4.16.17 and 4.14.51 have been released with lots of fixesthroughout the tree. Users should upgrade.

<p>A two-part session at the 2018 Python Language Summit tackled the coredeveloper diversity problem from two different angles. Victor Stinneroutlined some work he has been doing to mentor new developers on their pathtoward joining the core development ranks; he has also been trying todocument that path. Mariatta Wijaya gave a very personal talk thatdescribed the diversity problem while also providing some concrete actionitems that the project and individuals could take to help make Python morewelcoming to minorities.

In a session with a title that used a common misquote of RodneyKing ("can't we all just get along?"), severalPython developers wanted to discuss an incident that had recently occurredon the python-dev mailing list. A rude posting to the list led to a thread thatgot somewhat out of control. Some short tempers among the members of thePython developer community likely escalated things unnecessarily. Theincident in question was brought up as something of an object lesson;people should take some time to simmer down before firing off that quick,but perhaps needlessly confrontational, reply.

The "PEP 572 mess" was the topic of a 2018 Python Language Summit sessionled by benevolent dictator for life (BDFL) Guido van Rossum. PEP 572 seeks to addassignment expressions (or "inline assignments") to the language, but ithas seen a prolonged discussion over multiple huge threads on the python-dev mailing list—evenafter multiple rounds on python-ideas.Those threads were often contentious and were clearly voluminous to thepoint where many probably just tuned them out.At the summit, Van Rossum gave an overview of thefeature proposal, which he seems inclined toward accepting, but he alsowanted to discuss how to avoid this kind of thread explosion in the future.

Matthew Miller looks at how Red Hat's acquisition of CoreOS will affect theFedora project. "This isn’t the place for technical details — see“what next?” at the bottom of this message for more. I expect that over thenext year or so, Fedora Atomic Host will be replaced by a new thingcombining the best from Container Linux and Project Atomic. Thisnew thing will be “Fedora CoreOS” and serve as the upstream to RedHat CoreOS."

Security updates have been issued by Arch Linux (pass), Debian (xen), Fedora (chromium, cobbler, gnupg, kernel, LibRaw, mariadb, mingw-libtiff, nikto, and timidity++), Gentoo (chromium, curl, and transmission), Mageia (gnupg, gnupg2, librsvg, poppler, roundcubemail, and xdg-utils), Red Hat (ansible and glusterfs), Slackware (gnupg), SUSE (cobbler, dwr, java-1_8_0-ibm, kernel, microcode_ctl, pam-modules, salt, slf4j, and SMS3.1), and Ubuntu (libgcrypt11, libgcrypt11, libgcrypt20, and mozjs52).

Security updates have been issued by Arch Linux (libgcrypt), Fedora (bouncycastle, nodejs, and perl-Archive-Tar), openSUSE (aubio), and Red Hat (chromium-browser, glibc, kernel, kernel-rt, libvirt, pcs, samba, samba4, sssd and ding-libs, and zsh).

This article describes our findings that connected TCP small queues (TSQ)with the behavior of advanced WiFi protocols and, in the process, solved athroughput regression. The resulting patch is already in the mainline tree, so beforecontinuing, please make sure your kernel is updated. Beyond the fix, it isdelightful to travel through history to see how we discovered the problem,how it was tackled, and how it was patched.Subscribers can read on for the full story by guest authors Carlo Grazia and Natale Patriciello.

Security updates have been issued by CentOS (kernel), Debian (libgcrypt20, redis, and strongswan), Fedora (epiphany, freedink-dfarc, gnupg, LibRaw, nodejs-JSV, nodejs-uri-js, singularity, strongswan, and webkit2gtk3), Mageia (flash-player-plugin, freedink-dfarc, and imagemagick), openSUSE (enigmail, gpg2, java-1_7_0-openjdk, java-1_8_0-openjdk, poppler, postgresql96, python-python-gnupg, and samba), Oracle (kernel), SUSE (gpg2 and xen), and Ubuntu (gnupg and webkit2gtk).

By the time that Linus Torvalds released 4.18-rc1 and closed the merge window for this development cycle, 11,594 non-merge changesets hadfound their way into the mainline kernel repository. Nearly 4,500 of thosewere pulled after last week's summary waswritten. Thus, in terms of commit traffic, 4.18 looks to be quite similarto its predecessors. As usual, the entry of significant new features hasslowed toward the end of the merge window, but there are still someimportant changes on the list.

The stable update machine continues to crank out releases:4.17.2,4.16.16,4.14.50,4.9.109, and4.4.138are all available with another set of important fixes.

The first 4.18 prepatch is out, and themerge window has closed for this development cycle. "You may think it's stillSaturday for me, and that I should give you one more day of merge window tosend in some last-minute pull requests, but I know better. I'm in Japan,and it's Sunday here."

It's been a little over one year since we last covered Debian's reproducible buildsproject. The effort has not stopped in the interim; progress continuesto be made, the message has sharpened up, and word is spreading. ChrisLamb, speaking about this at FLOSS UK in a talk called "You may thinkyou're not a target: a tale of three developers", hinted that the end may be starting to come into sight.

Security updates have been issued by CentOS (plexus-archiver), Fedora (chromium, kernel, and plexus-archiver), Mageia (firefox, gifsicle, jasper, leptonica, patch, perl-DBD-mysql, qt3, and scummvm), openSUSE (opencv), Oracle (kernel), Red Hat (kernel), Scientific Linux (kernel), SUSE (gpg2, nautilus, and postgresql96), and Ubuntu (gnupg2 and linux-raspi2).

Kees Cook describesthe security-oriented changes included in the 4.17 kernel release."It was possible that old memory contents would live in a newprocess’s kernel stack. While normally not visible, “uninitialized” memoryread flaws or read overflows could expose these contents (especially stuff“deeper” in the stack that may never get overwritten for the life of theprocess). To avoid this, I made sure that new stacks were alwayszeroed. Oddly, this “priming” of the cache appeared to actually improveperformance, though it was mostly in the noise."

Ars technica has thestory of a set of Docker images containing cryptocurrency miners thatpersisted on Docker Hub for the better part of a year — after beingdiscovered. "Neither the Docker Hub account nor the malicious images it submitted were takendown. Over the coming months, the account went on to submit 14 moremalicious images. The submissions were publicly called out two more times,once in January by security firm Sysdig and again in May by securitycompany Fortinet. Eight days after last month's report, Docker Hub finallyremoved the images."

Security updates have been issued by Arch Linux (chromium and gnupg), Debian (spip), Fedora (pdns-recursor), Gentoo (adobe-flash, burp, quassel, and wget), openSUSE (bouncycastle and taglib), Oracle (kernel), SUSE (java-1_7_0-openjdk, java-1_8_0-openjdk, poppler, and samba), and Ubuntu (file, perl, and ruby1.9.1, ruby2.0, ruby2.3).

The LWN.net Weekly Edition for June 14, 2018 is available.

In a short session at the 2018 Python Language Summit, Steve Dower broughtup the shortcomings of Python virtual environments,which are meant to create isolated installations of the language and itsmodules. He said his presentation was "co-written with Twitter" and,indeed, most of his slides were of tweets. At the end, he also slipped in anannouncement of his plans for hosting a core development sprint in September.

The XArray data structure was the topic ofthe final filesystem track session at the 2018 Linux Storage, Filesystem,and Memory-Management Summit (LSFMM). XArray is a new API for the kernel'sradix-tree data structure; the session wasled by Matthew Wilcox, who created XArray. When asked by Dave Chinner ifthe session was intended to be a live review of the patches, Wilcoxadmitted with a grin that it might be "the only way to get a review on thisdamn patch set".

While the 2018 Linux Storage, Filesystem, and Memory-Management Summit(LSFMM) filesystem track session was advertised as being a filesystem test suite "bakeoff", it actually focused on how to make the existing test suites moreaccessible. Kent Overstreet said that he has learned over theyears that various filesystem developers have their own scripts for testingusing QEMU and other tools. He and Ted Ts'o put the session together totry to share some of that information (and code) more widely.

Stable kernels 4.9.108, 4.4.137, and 3.18.113 have been released. As usual, theyall contain important fixes and users should upgrade.

In the filesystem track at the 2018 Linux Storage, Filesystem, andMemory-Management Summit (LSFMM), Al Viro discussed some problems he hasrecently spotted in the implementation of rmdir().He covered some of the history of that implementation and how things got towhere they are now. He also described areas that needed to be checkedbecause the problem may be present in different places in multiple filesystems.

Security updates have been issued by Debian (plexus-archiver), Oracle (plexus-archiver), Red Hat (plexus-archiver and rh-maven33-plexus-archiver and rh-maven35-plexus-archiver), Scientific Linux (plexus-archiver), SUSE (pdns, poppler, ucode-intel, wpa_supplicant, and xen), and Ubuntu (bind9, firefox, and linux-azure).

One of the larger features added to Python over the last few releases issupport for static typing in the language. Static type-checking and toolsto support it show up frequentlyas topics at the Python LanguageSummit (PLS) and this year was no exception. Mypy developers Jukka Lehtosalo and IvanLevkivskyi gave an update on static typing at PLS 2018.

One of the many longstanding — though unwritten — rules of kerneldevelopment is that infrastructure is not merged until at least one userfor that infrastructure exists. That helps developers evaluate potentialinterfaces and be sure that the proposed addition is truly needed. A bigexception to this rule was made when the heterogeneous memory management(HMM) code was merged, though. One of the reasons for the lack of users inthis case turns out to be that many of the use cases are proprietary; thathas led to some disagreements over the GPL-only status of an exportedkernel symbol.

Stable kernels 4.17.1, 4.16.15, and 4.14.49 have been released. They all containimportant fixes and users should upgrade.

Security updates have been issued by Debian (perl), Red Hat (kernel), SUSE (slurm), and Ubuntu (gnupg, gnupg2, imagemagick, kernel, libvirt, linux, linux-aws, linux-gcp, linux-kvm, linux, linux-aws, linux-kvm, linux-raspi2, linux-snapdragon, linux, linux-raspi2, linux-azure, linux-hwe, linux-gcp, linux-oem, linux-lts-trusty, linux-lts-xenial, linux-aws, and qemu).

The kernel's handling of I/O errors was the topic of a discussion led byMatthew Wilcox at the 2018 Linux Storage, Filesystem, and Memory-ManagementSummit (LSFMM) in a combined storage and filesystem track session. At the start, he asked: "how is our error handling andwhat do we plan to do about it?" That led to a discussion between thedevelopers present on the kinds of errors that can occur and onways to handle them.