LWN.net

There are currently a number of sites bouncing emails from LWN.net due to analleged listing in the dnsbl.njabl.org blacklist. The only problem is thatthis blacklist hasbeen offline since 2013. That domain has been taken over by somebodyelse; checking web content at that address is not recommended as it is, toput it lightly, non-technical. If you are not getting expected email fromLWN, you might want to look into whether your local setup is still tryingto use that old, discontinued blacklist.

Antonio Cuni writesabout recent work to support running Python code with low latencyrequirements under PyPy. "As we said, the total cost of memorymanagement is less on PyPy than on CPython, and it's one of the reasons whyPyPy is so fast. However, one big disadvantage is that while on CPython thecost of memory management is spread all over the execution of the program,on PyPy it is concentrated into GC runs, causing observable pauses whichinterrupt the execution of the user program. To avoid excessively longpauses, the PyPy GC has been using an incremental strategy since 2013. TheGC runs as a series of 'steps', letting the user program to progressbetween each step."

The LWN.net Weekly Edition for January 3, 2019 is available.

The January 3 LWN.net Weekly Edition will be our first for 2019, markingour return after an all-too-short holiday period. Years ago, we made theill-considered decision to post some predictions at the beginning of theyear and, like many mistakes, that decision has persisted and become anannual tradition. We fully expect 2019 to be an event-filled year, withboth ups and downs; read on for some wild guesses as to what some of thoseevents may look like.

The Internet Archive (IA) has been around forover 20 years now; many will know it for its Wayback Machine, which is an archive ofold versions of web pages, but IA is much more than just that. Tracey Jaquith said that sheand her IA colleague David Van Duzer would relate a "love/hate, longadventure story—mostly love" about the migration of parts of IA toKubernetes. It is an ongoing process, but they learned a lot along theway, so they wanted to share some of that with attendees of KubeCon +CloudNativeCon North America 2018.

The linux-kernel mailing list carries the sad news that Shaohua Li, atalented contributor to much of the core kernel and the maintainer of theMD RAID subsystem, passed away over the holidays. Thank you for your work,Shaohua, you will certainly be missed.

Dylan O'Mahony, the cloud architecture manager for Bose,opened a presentation atKubeCon +CloudNativeCon North America 2018 by noting that many attendees may bewondering why a "50-year-old audio company" would be part of a presentationon Kubernetes. It turns out that Bose was looking for ways to support itssmart-speaker products and found the existing solutions to be lacking.Bose partnered with Connected, "a product development company fromToronto", to use Kubernetes as part ofthat solution, so O'Mahony and David Doyle from Connected were at theconference to describe the prototype that they built.

Security updates have been issued by Debian (thunderbird), Fedora (terminology), openSUSE (GraphicsMagick), and Red Hat (rh-perl526-perl).

Cloud computing services that run customer code in short-lived processesare often called "serverless". Butunder the hood, virtual machines (VMs) are usually launched to run that isolatedcode on demand. The boot times for these VMs can be slow. This is the causeof noticeable start-up latency in a serverless platform like Amazon WebServices (AWS) Lambda. Toaddress the start-up latency, AWS developed Firecracker, a lightweightvirtual machine monitor (VMM), which it recentlyreleasedas open-source software. Firecracker emulates a minimal device modelto launch Linux guest VMs more quickly. It's an interesting exploration ofimproving security and hardware utilization by using a minimal VMM built withalmost no legacy emulation.

Security updates have been issued by Mageia (graphicsmagick, poppler, python, and python-lxml) and openSUSE (GraphicsMagick).

When the 4.20 kernel was released onDecember 23, Linus Torvalds indicated that he would try to keep to thenormal merge window schedule despite the presence of the holidays in themiddle of it. Thus far, he seems to be trying to live up to that;just over 8,700 changesets have been merged for the next release, whichseems likely to be called 5.0. A number of long-awaited features arefinally landing in the kernel with this release.

The New York Times reportsthe death of Dr. Lawrence G. Roberts, who was heavily involved inArpanet. "Dr. Roberts was considered the decisive force behind packetswitching, the technology that breaks data into discrete bundles that arethen sent along various paths around a network and reassembled at theirdestination. He decided to use packet switching as the underlyingtechnology of the Arpanet; it remains central to the function of theinternet." (Thanks to Paul Wise.)

Security updates have been issued by Arch Linux (go, go-pie, and webkit2gtk), Debian (c3p0, debian-security-support, libextractor, and tar), Fedora (electron-cash, leptonica, LibRaw, mingw-leptonica, mingw-openjpeg2, mingw-poppler, nettle, openjpeg2, php-pear, sqlite, and vcftools), Gentoo (GKSu and rust), Mageia (keepalived and libtiff), openSUSE (containerd, docker, go, go, GraphicsMagick, libraw, mozilla-nspr and mozilla-nss, netatalk, polkit, wireshark, and xen), and SUSE (containerd, docker, go, libqt5-qtbase, mailman, wireshark, and xen).

The4.19.13,4.14.91, and4.9.148stable kernels have all been released; each contains another set ofimportant fixes.

Most processors spend a great deal of their time doing nothing, waitingfor devices and timer interrupts. In these cases, they can switch to idlemodes that shut down parts of their internal circuitry, especially stoppingcertain clocks. This lowers power consumption significantlyand avoids draining device batteries. There are usuallya number of idle modes available; the deeper the mode is, the less power theprocessor needs. The tradeoff is that the cost of switching to and fromdeeper modes ishigher; it takes more time and the content of some caches is also lost. In theLinux kernel, the cpuidle subsystem has the task of predicting which choicewill be the most appropriate. Recently, Rafael Wysocki proposeda new governor for systems with tickless operation enabled that isexpected to be more accurate than the existing menu governor.

Security updates have been issued by Debian (libphp-phpmailer), Fedora (mosquitto and tinc), and Mageia (ruby-i18n and tcpdump).

Kees Cook summarizesthe security-related improvements in the 4.20 kernel."Enabling CONFIG_GCC_PLUGIN_STACKLEAK=y means almost alluninitialized variable flaws go away, with only a very minor performancehit (it appears to be under 1% for most workloads). It’s still possiblethat, within a single syscall, a later buggy function call could use'uninitialized' bytes from the stack from an earlier function. Fixing thiswill need compiler support for pre-initialization (this is underdevelopment already for Clang, for example), but that may have largerperformance implications."

Security updates have been issued by Debian (ghostscript, graphicsmagick, libarchive, libsndfile, libvncserver, ruby-sanitize, and wireshark), Fedora (mosquitto and tinc), Mageia (monit, sqlite3, and thunderbird), and SUSE (openssl).

Security updates have been issued by Debian (libextractor and nagios3) and Fedora (adplug, mingw-podofo, and podofo).

Security updates have been issued by CentOS (firefox), Debian (ghostscript, libarchive, openjpeg2, and sqlite3), Fedora (krb5, mariadb, mariadb-connector-c, mingw-openjpeg2, openjpeg2, phpMyAdmin, python-lxml, spatialite-tools, sqlite, and squid), Mageia (kernel), openSUSE (bluez, git, go1.10, libnettle, libqt5-qtbase, ovmf, pdns, perl, tcpdump, tiff, tryton, and yast2-rmt), Slackware (netatalk), and SUSE (buildah, caasp-cli, caasp-dex, cni-plugins, container-feeder, containerd-kubic, cri-o, cri-tools, docker-kubic, docker-runc-kubic, etcd, flannel, golang-github-docker-libnetwork-kubic, helm, kubernetes, kubernetes-dns, libcontainers-storage, podman, runc, skopeo, umoci, firefox, nspr, nss, netatalk, and qemu).

Linus has released 4.20 as expected."Let's face it, last week wasn't quite as quiet as I would have hopedfor, but there really doesn't seem to be any point to delay 4.20because everybody is already taking a break." Some of the headlinefeatures in 4.20 includenetwork flow dissectors in BPF,the tapriotraffic scheduler,peer-to-peer DMA support in the PCI layer,C-SKY architecture support,the pressure-stall instrumentation mechanism,the XArray data structure,and much more. The KernelNewbies 4.20 page iscoming together with more information.

This year's holiday gifts will include the 4.20 kernel; that can only meanthat it is time for another look at where the code going into this releasehas come from. This development cycle was typically busy and brought a lotof new code into the kernel. There are some new faces showing up in thestatistics this time around, but not a lot of surprises otherwise.

Greg Kroah-Hartman has announced the release of five new stable kernels: 4.19.12, 4.14.90, 4.9.147, 4.4.169, and 3.18.131. As usual, these contain importantfixes throughout the tree; users of those series should upgrade.

Security updates have been issued by Debian (libapache-mod-jk, libav, and netatalk), Fedora (kernel-headers, kernel-tools, and phpMyAdmin), Gentoo (go), Mageia (netty, jctools, php, and phpmyadmin), openSUSE (keepalived), Scientific Linux (ntp), SUSE (enigmail, libqt5-qtbase, mariadb, netatalk, and yast2-rmt), and Ubuntu (kernel, linux, linux-aws, linux-azure, linux-gcp, linux-kvm, linux-raspi2, linux, linux-aws, linux-kvm, linux-raspi2, linux-snapdragon, linux-azure, linux-hwe, linux-aws-hwe, linux-azure, linux-gcp, linux-lts-trusty, linux-lts-xenial, linux-aws, and linux-raspi2).

The kernel's live-patching (KLP) mechanism can apply a wide variety offixes to a running kernel but, at a first glance, the sort of highlyintrusive changes needed to address vulnerabilities like Meltdown or L1TFwould not seem like likely candidates for live patches.The most notable obstacles are the requiredmodifications of global semantics on a running system, as well as theneed for live patching the kernel's entry code. However, we at the SUSE livepatching team started working on proof-of-concept live patches for thesevulnerabilities as afun project and have been able to overcome these hurdles. The techniques we developed are generic and might become handy again whenfixing future vulnerabilities.

Security updates have been issued by CentOS (ntp), Debian (openssl1.0), openSUSE (salt), Oracle (firefox, ghostscript, and ntp), Red Hat (ntp), and SUSE (bluez, git, libnettle, ovmf, and tiff).

The LWN.net Weekly Edition for December 20, 2018 is available.

The December 20 LWN.net Weekly Edition is the final one for the year; asusual, we will be taking the last week of the year off for a brief rest.LWN, which is about to conclude its 21st year of publication, has had thetime to build up some traditions, one of which is a year-end retrospectivethat evaluates the predictions we made backin January. As usual, some of those predictions aged rather better thanothers; read on for our report card.

Stable kernel 4.19.11 has been releasedwith some important fixes; users should upgrade.

A year-old bug in Kubernetes was thetopic of a talk given by Michelle Au and Jan Šafránek at KubeCon+ CloudNativeCon North America, which was held mid-December inSeattle. In the talk, they looked at the details of the bug and theresponse from the Kubernetes product security team (PST). While the bug was fairly straightforward, it wassurprisingly hard to fix. The whole process also provided experience thatwill help improve vulnerability handling in the future.

Security updates have been issued by CentOS (ghostscript), Fedora (ansible and wireshark), openSUSE (go1.11, pdns, and pdns-recursor), Oracle (firefox), Red Hat (java-1.8.0-ibm), Scientific Linux (firefox), and SUSE (crash, libqt5-qtbase, perl, and qemu).

Back in late October, when we looked in onthe Python governance question, which came about due to the resignation of Guido van Rossum, things seemedto be mostly set for a vote in late November. There were six PythonEnhancement Proposals (PEPs) under consideration that would be ranked byvoters in a two-week period ending December 1; instant-runoffvoting would be used to determine the winner. In the interim, though,much of that changed; the voting period, winner-determination mechanism,and number of PEPs under consideration are all different. But the votingconcluded on December 16 and a winnerhas been declared; PEP 8016 ("TheSteering Council Model"), which was added to the mix in early November, cameout on top.

HardenedBSD has releasedversion 12 of its security-enhanced fork of FreeBSD. Improvements inthis release include Non-Cross-DSO Control-Flow Integrity (CFI) forapplications on amd64 and arm64; jailed bhyve; per-jail toggles forunprivileged process debugging; Spectre v2 mitigation with retpolineapplied to the entirety of base and ports; Symmetric Multi-Threading (SMT)disabled by default; and more.

Security updates have been issued by Debian (libapache-mod-jk and sleuthkit), Fedora (kernel, kernel-headers, mbedtls, php, php-symfony, php-symfony3, php-symfony4, and wireshark), openSUSE (pdns, pdns-recursor, and salt), Oracle (firefox and ghostscript), Red Hat (ansible, firefox, ghostscript, and kernel), Scientific Linux (firefox and ghostscript), and SUSE (ovmf).

Greg Kroah-Hartman has released stable kernel 4.4.168. As usual, there are important fixesand users should upgrade.

Gustavo Padovan notesan important milestone in Linux graphics development: "The dreamfinally came true in 2018 with the release of the Google Pixel 3, the firstAndroid phone running with the mainline graphics stack. A feat that wasdeemed impossible 10 years ago is now a reality thanks to a lot of hardwork from the entire community."

Stable kernel 3.18.130 has been releasedwith important fixes; users should upgrade.

Security updates have been issued by Debian (php5, poppler, and samba), Fedora (firefox, mbedtls, nbdkit, pdns-recursor, php, php-symfony, php-symfony3, and php-symfony4), Gentoo (CouchDB, scala, and spamassassin), Mageia (firefox, libwpd, nss, and thunderbird), openSUSE (Chromium, cups, ghostscript, kernel, openvswitch, phpMyAdmin, qemu, and tcpdump), Red Hat (RHGS WA), and SUSE (ansible, openldap2, openvswitch, qemu, and tcpdump).

Linus has released 4.20-rc7, saying:"The plan remains the same: if everything continues normally, I'llrelease 4.20 just before christmas, and then just have a moreleisurely merge window than normal."On the stable side,4.19.10,4.14.89,and 4.9.146 are out with a new set ofimportant fixes.

Indirect function calls — calls to a function whose address is stored in apointer variable — have never been blindingly fast, but the Spectrehardware vulnerabilities have made things far worse. The indirect branchpredictor used to speed up indirect calls in the CPU can no longer beused, and performance has suffered accordingly. The "retpoline"mechanism was a brilliant hack that proved faster than the hardware-based solutionsthat were tried at the beginning. While retpolines took a lot of the painout of Spectre mitigation, experience over the last year has made it clearthat they still hurt. It is thus not surprising that developers have beenlooking for alternatives to retpolines; several of them have shown up onthe kernel lists recently.

Security updates have been issued by CentOS (ghostscript, git, java-1.7.0-openjdk, java-11-openjdk, kernel, NetworkManager, python-paramiko, ruby, sos-collector, thunderbird, and xorg-x11-server), Debian (gcc-4.9), and SUSE (amanda, ntfs-3g_ntfsprogs, and tiff).

The Linux kernel is generally seen as a poor fit for safety-criticalsystems; it was never designed to provide realtime response guarantees orto be certifiable for such uses. But the systems that can be usedin such settings lack the features needed to support complex applications.This problem is often solved by deploying a mix of computers runningdifferent operating systems. But what if you want to support a mixture oftasks, some safety-critical and some not, on the same system? At a talkgiven at LinuxLab 2018, ClaudioScordino described an effort to support this type of mixed-criticalitysystem.

Greg Kroah-Hartman has released stable kernels 4.19.9, 4.14.88, 4.9.145, 4.4.167, and 3.18.129. They all contain important fixes andusers should upgrade.

Security updates have been issued by Debian (firefox-esr), Fedora (singularity), openSUSE (compat-openssl098, cups, firefox, mozilla-nss, and xen), and SUSE (cups, exiv2, ghostscript, and git).

The LWN.net Weekly Edition for December 13, 2018 is available.

In the RDMA microconference of the 2018 Linux Plumbers Conference (LPC),John Hubbard, Dan Williams, and Matthew Wilcox led a discussion on theproblems surrounding get_user_pages() (and friends) and theinteraction with DMA. It is not the first time the topic has come up,there was also a discussion about it at theLinux Storage, Filesystem, and Memory-Management Summit back in April. Ina nutshell, the problem is that multiple parts of the kernel think theyhave responsibility for the same chunk of memory, but they do notcoordinate their activities; as might be guessed, mayhem can sometimes ensue.

The x32 subarchitectureis a software variant of x86-64; it runs the processor in the 64-bit mode,but uses 32-bit pointers and arithmetic. The idea is to get the advantagesof x86-64 without the extra memory usage that goes along with it. Itseems, though, that x32 is not much appreciated; few distributions supportit and the number of users appears to be small. So now Andy Lutomirski isproposingits eventual removal:I propose that we make CONFIG_X86_X32 depend on BROKEN for a releaseor two and then remove all the code if no one complains. If anyonewants to re-add it, IMO they're welcome to do so, but they need to doit in a way that is maintainable.If there are x32 users out there, now would be a good time for them tospeak up.

Security updates have been issued by Arch Linux (chromium, firefox, lib32-openssl, lib32-openssl-1.0, openssl, openssl-1.0, texlive-bin, and wireshark-cli), Fedora (perl), openSUSE (pdns), Oracle (kernel), Red Hat (kernel), Slackware (mozilla), SUSE (kernel, postgresql10, qemu, and xen), and Ubuntu (firefox, freerdp, freerdp2, pixman, and poppler).

Git 2.20.0 is out. Changes include interdiff generation support in gitformat-patch, an improved ability to cope with corrupted patches ingit am, a number of performance and usability improvements, and more.

The Mozilla Blog takesa look at the Contextual Feature Recommender (CFR) in Firefox64. "Aimed at people who are looking to get more out of their onlineexperience or ways to level up. CFR is a system that proactively recommendsFirefox features and add-ons based on how you use the web. For example, ifyou open multiple tabs and repeatedly use these tabs, we may offer afeature called “Pinned Tabs” and explain how it works. Firefox curates thesuggested features and notifies you. With today’s release, we will start torollout with three recommended extensions which include: FacebookContainer, Enhancer for YouTube and To Google Translate. This feature isavailable for US users in regular browsing mode only. They will not appearin Private Browsing mode. Also, Mozilla does NOT receive a copy of yourbrowser history. The entire process happens locally in your copy ofFirefox." The releasenotes contain more details about this release.