LWN.net

Stable kernels 4.17.10, 4.14.58, 4.9.115, and 4.4.144 have been released. They all containimportant fixes throughout the tree and users should upgrade.

Security updates have been issued by Debian (ant, evolution-data-server, libarchive-zip-perl, mailman, resiprocate, slurm-llnl, and sympa), Mageia (firmware, kernel, microcode, and wesnoth), openSUSE (Chromium), Oracle (openslp and thunderbird), Red Hat (java-1.7.0-oracle, java-1.8.0-oracle, kernel, qemu-kvm-rhev, and thunderbird), SUSE (kernel, nautilus, and xen), and Ubuntu (ant and clamav).

Peter Hutterer writesabout why libinput exists. It turns out that, like most otherhardware, input devices have no end of obnoxious quirks to deal with."All this is just handling features that users have come toexpect. Examples for non-features that you'll have to implement: on someLenovo series (*50 and newer) you will get a pointer jump after a series ofof events that only have pressure information. You'll have to detect anddiscard that jump. The HP Pavilion DM4 touchpad has random jumps in theslot data. Synaptics PS/2 touchpads may 'randomly' end touches and restartthem on the next event frame 10ms later. If you don't handle that you'llget ghost taps. And so on and so forth."

The 4.18-rc6 kernel prepatch came out onJuly 22, right on schedule. That is a sign that this development cycle is approachingits conclusion, so the time has come for a look at some statistics for howthings went this time around. It was another fairly ordinary releasecycle for the most part, but with a couple of distinctive features.

Here is theEconomist's take on the state of the Python language and community."Mr Van Rossum, though delighted by this enthusiasm for his software,has come to find the rigours of supervising it, in his role as 'benevolentdictator for life', unbearable. He fears he has become something of anidol. 'I’m uncomfortable with that fame,' he says, sounding uncannily likeBrian trying to drive away the crowds of disciples. 'Sometimes I feel likeeverything I say or do is seen as a very powerful force.' On July 12th heresigned, leaving the Pythonistas to manage themselves."

<p>Random number generation in the kernel has garnered a lot of attention overthe years. The tensions between the need for cryptographic-strength randomnumbers versus getting strong random numbers more quickly—along with the needto avoid regressions—has led to something of a patchwork of APIs. While itis widely agreed that waiting for a properly initialized random numbergenerator (RNG) before producing random numbers is the proper course,opinions differ on what "properly" means exactly. Beyond that, waiting,especially early in the boot process, can be problematic as well. Onesolution would be to trust the RNG instructions provided by most modernprocessors, but that comes with worries of its own.

Security updates have been issued by Debian (network-manager-vpnc), Fedora (haproxy, mailman, and NetworkManager-vpnc), Mageia (clamav, ffmpeg, rust, thunderbird, and wireshark), Oracle (java-1.8.0-openjdk and openslp), Red Hat (rh-ror42-rubygem-sprockets and rh-ror50-rubygem-sprockets), Scientific Linux (java-1.8.0-openjdk and openslp), SUSE (ImageMagick, libofx, php53, and python-dulwich), and Ubuntu (linux, linux-aws, linux-azure, linux-gcp, linux-kvm, linux-oem, linux-hwe, linux-azure, linux-gcp, mutt, and python-cryptography).

NetBSD 8.0 has been released.This version features USB stack rework with USB3 support added, anin-kernel audio mixer, reproducible builds, full userland debuginformation, and much more.

Security updates have been issued by Arch Linux (apache, networkmanager-vpnc, and znc), Debian (gosa, opencv, and slurm-llnl), Fedora (evolution, evolution-data-server, evolution-ews, gnome-bluetooth, libtomcrypt, podman, python-cryptography, and rust), Gentoo (passenger), Red Hat (java-1.8.0-openjdk and openslp), Slackware (php), SUSE (openssl-1_1, procps, python, rsyslog, rubygem-passenger, and xen), and Ubuntu (mutt).

The sixth 4.18 kernel prepatch is out fortesting. "So this was the week when the other shoe dropped ... The reason thetwo previous rc releases were so nice and small was that David hadn'tsent me much networking fixes, and they came in this week.That said, it's not really a huge rc this week either, so it's allgood."

Greg Kroah-Hartman has released five new stable kernels: 4.17.9, 4.14.57, 4.9.114, 4.4.143, and 3.18.116. As usual, they contain importantchanges throughout the kernel tree; users of those series should upgrade.

Over on the Facebook code site, Daniel Xu announces the release of oomd under the GPLv2. Oomd is a user-space "out of memory" killer that was mentioned in our recent article on the block I/O latency controller and it uses the pressure stall information covered in an even more recent article."Oomd constantly monitors PSI [Pressure Stall Information] metrics to assess whether a system is under unrecoverable load. PSI alone is insufficient, so oomd also monitors the system holistically. This is in contrast to Linux’s OOM killer, which focuses primarily on the kernel’s concerns. Since OOM detection criteria can vary depending on workload, the plugin system supports customization to both the detection and process kill strategies.Thanks to this new ability to monitor key system resource indicators, oomd is able to take corrective action in userspace before a system-wide OOM occurs. Corrective action is configured via a flexible plugin system that is capable of executing custom code. Thus, in addition to oomd’s default process SIGKILL behavior, application developers can customize their plugin with alternate strategies, such as sending a 'back off' RPC to the main workload or dumping system logs to a remote service."

Security updates have been issued by Debian (dnsmasq, linux-base, and openjpeg2), Fedora (libgit2, libtomcrypt, openslp, and perl-Archive-Zip), and openSUSE (gdk-pixbuf, libopenmpt, mercurial, perl, php7, polkit, and rsyslog).

The kernel supports two different "SCSI generic" pseudo-devices, each ofwhich allows user space to send arbitrary commands to a SCSI-attacheddevice. Both SCSI-generic implementations have proved to have securityissues in the past as a result of the way their API was designed. In thecase of one of those drivers, these problems seem almost certain to lead to theremoval of a significant chunk of functionality in the 4.19 developmentcycle.

Greg Kroah-Hartman has released the 4.4.142stable kernel. It is not an essential upgrade, "but a number ofbuild problems with perf are now resolved, and an x86 issue that some people might have hitis now handled properly. If those were problems for you, pleaseupgrade."

Security updates have been issued by Debian (ant, gpac, linux-4.9, linux-latest-4.9, taglib, vlc, and znc), Fedora (ceph), Red Hat (fluentd and qemu-kvm-rhev), Slackware (httpd), and SUSE (e2fsprogs, glibc, libgcrypt, mercurial, openssh, perl, rubygem-sprockets, shadow, and wireshark).

The LWN.net Weekly Edition for July 19, 2018 is available.

<p>Deep-learning applications typically rely on a trained neural net toaccomplish their goal (e.g. photo recognition, automatic translation, orplaying go). That neural net uses what is essentially a large collection ofweighting numbers that have been empirically determined as part of its training (which generally uses a huge set of training data). Afree-software application could use those weights, but there are a number of barriers for users who might want to tweak them for variousreasons. A discussion on the debian-devel mailing list recently looked atwhether these deep-learning applications can ever truly be considered "free" (as infreedom) because of these pre-computed weights—and the difficultiesinherent in changingthem.

Over the last few months, it became clear that the battle over PEP 572 wouldbe consequential; its scale and vehemence was largely unprecedented in thehistory of Python. The announcement by Guido van Rossum thathe was stepping down from his role as benevolent dictator for life (BDFL),due in part to that battle,underscored the importance of it. While the Python project charts its course in the wake of hisresignation, it makes sense to catch up on where things stand with thiscontentious PEP that has now been accepted for Python 3.8.

Stable kernel 4.17.8 has been released.This fixes the issue with i386 systems that was present in the 4.17.7 kernel.

Security updates have been issued by Arch Linux (curl, lib32-curl, lib32-libcurl-compat, lib32-libcurl-gnutls, libcurl-compat, and libcurl-gnutls), Debian (blender, ffmpeg, and wordpress), Fedora (curl), Gentoo (tqdm), Oracle (kernel), Slackware (mutt), SUSE (xen), and Ubuntu (policykit-1).

In order to actually do anything, a kernel module must gain access tofunctions and data structures in the rest of the kernel. Enabling andcontrolling that access is the job of the symbol-export mechanism. Whilethe enabling certainly happens, the control part is not quite so clear;many developers view the nearly 30,000 symbols in current kernels that areavailable to all modules as being far too many. The symbolnamespaces patch set from Martijn Coenen doesn't reduce that number,but it does provide a mechanism that might help to impose some order onexported symbols in general.

Stable kernels 4.17.7, 4.14.56, 4.9.113, and 4.4.141 have been released. The 4.17.7 kernel is broken for i386 systems. "I did this release anyway with this known problem as there is a fix in here for x86-64 systems that was nasty to track down and was affecting people. Given that the huge majority of systems are NOT i386, I felt this was a safe release to do at this point in time." Beyond that, these kernels all contain the usual set of important fixes.

Security updates have been issued by Arch Linux (thunderbird), Debian (ruby-rack-protection), Fedora (firefox and soundtouch), Red Hat (kernel), Scientific Linux (gnupg2), SUSE (perl and python-paramiko), and Ubuntu (policykit-1).

The recent announcement by Guido van Rossumthat he was stepping away from his "benevolent dictator for life" (BDFL) role for Python was met with somesurprise, but not much shock, at least in the core-developer community.Van Rossum has been telegraphing some kind of change, at some unspecifiedpoint, for several years now, though the proximate cause (the "PEP 572 mess") isunfortunate. In the meantime, though, the project needs to figure outhow to govern itself moving forward—Van Rossum did not appoint a successorand has left the governance question up to the core developers.

Security updates have been issued by CentOS (firefox, gnupg2, kernel, python, and qemu-kvm), Debian (389-ds-base, cups, imagemagick, kernel, mailman, ruby2.1, sssd, thunderbird, and znc), Fedora (glpi, hadoop, kernel, rubygem-sprockets, singularity, thunderbird, wordpress, xapian-core, and xen), Mageia (cantata and flash-player-plugin), openSUSE (exiv2, libvorbis, nodejs6, nodejs8, openslp, singularity, slurm, and tiff), and SUSE (kernel-azure and openssl).

The 4.18-rc5 kernel prepatch has beenreleased. "For some reason this week actually felt very busy, butthe rc5 numbers show otherwise. It's all small and calm, and things areprogressing nicely."

All underutilized systems are essentially the same, but each overutilizedsystem tends to be overloaded in its own way. If one's goal is tomaximize the use of the available computing resources, overutilizationtends not to be too far away, but when it happens, it can be hard to tellwhere the problem is. Sometimes, even the fact that there is a problem atall is not immediately apparent. Thepressure-stall information patch set from Johannes Weiner may make lifeeasier for system administrators by exposing more information about the real utilizationstate of the system.

Security updates have been issued by Debian (cinnamon), Fedora (docker, firefox, jetty, and knot-resolver), Oracle (gnupg2), Scientific Linux (gnupg2), SUSE (gdk-pixbuf, java-1_8_0-openjdk, libopenmpt, php7, and rsyslog), and Ubuntu (dns-root-data, dnsmasq, and thunderbird).

Python creator and Benevolent Dictator for Life Guido van Rossum has decided,in the wake of the difficult PEP 572discussion, to step down from his leadership of the project. "Now that PEP 572 is done, I don't ever want to have to fight so hard for aPEP and find that so many people despise my decisions.I would like to remove myself entirely from the decision process. I'llstill be there for a while as an ordinary core dev, and I'll still beavailable to mentor people -- possibly more available. But I'm basicallygiving myself a permanent vacation from being BDFL, and you all will be onyour own."

Mounting filesystems is a complicated business. The kernel supports a widevariety of filesystem types, and each has its own, often extensive set of options. As a result, the mount()system call is complex, and the list of mountoptions is a rather long read. But even with all of that complexity,mount() does not do everything that users would like. Forexample, the options for a mount operation must all fit within a single4096-byte page — the fact that this is a problem for some users isillustrative in its own right. Theproblems with mount() have come up at various meetings, includingat the 2018 Linux Storage, Filesystem, andMemory-Management Summit. A setof patches implementing a new approach is getting closer to beingready, but it features some complexity of its own and there are someremaining concerns about the proposed system-call API.

Security updates have been issued by Arch Linux (qutebrowser), CentOS (firefox), Debian (ruby-sprockets), Fedora (botan2, git-annex, kernel, kernel-tools, and visualboyadvance-m), Mageia (chromium-browser-stable, graphviz, mailman, nikto, perl-Archive-Zip, redis, and w3m), openSUSE (nextcloud), Oracle (gnupg2), Red Hat (flash-plugin, gnupg2, and kernel), Slackware (bind and curl), SUSE (java-1_8_0-openjdk, php7, rsyslog, slurm, and ucode-intel), and Ubuntu (cups, libpng, and libpng, libpng1.6).

The LWN.net Weekly Edition for July 12, 2018 is available.

The compromise of the Gentoo's GitHubmirror was certainly embarrassing, but its overall impact on Gentoo userswas likely fairly limited. Gentoo and GitHub respondedquickly and forcefully to the breach, which greatly limited the damagethat could be done; the fact that it was a mirror and not the master copyof Gentoo's repositories made it relatively straightforward to recoverfrom. But the black eye that it gave the project has led some to consider waysto make it even harder for an attacker to add malicious content toGentoo—even if the distribution's own infrastructure were to becompromised.

Greg Kroah-Hartman has released stable kernels 4.17.6, 4.14.55, 4.9.112, 4.4.140, and 3.18.115. As usual, they contain importantfixes and users should upgrade.

<p>A recent query about the status of network security (TLS settings inparticular) in Emacs led to a long thread in the emacs-devel mailing list. That threadtouched on a number of different areas, including using OpenSSL (or otherTLS libraries) rather thanGnuTLS, what kinds of problems should lead to complaints out of the box, what settings should be the default, and when those settings couldchange for Emacs so as not to discombobulate users. The latter issue isone that lots of projects struggle with: what kinds of changes areappropriate for a bug-fix release versus a feature release. For Emacs, itslengthy development cycle, coupled with the perceived urgency ofsecurity changes, makes that question even more difficult.

Security updates have been issued by Debian (cups), Oracle (kernel and qemu-kvm), Red Hat (ansible, kernel, kernel-rt, and qemu-kvm), Scientific Linux (kernel and qemu-kvm), Slackware (thunderbird), and Ubuntu (curl, firefox, imagemagick, and xapian-core).

Here's areport in Sensors Tech Forum on the discovery of a set of hostilepackages in the Arch Linux AUR repository system. AUR containsuser-contributed packages, of course; it's not a part of the Arch distributionitself. "The security investigation shows that shows that amalicious user with the nick name xeactor modified in June 7 an orphanedpackage (software without an active maintainer) called acroread. Thechanges included a curl script that downloads and runs a script from aremote site. This installs a persistent software that reconfigures systemdin order to start periodically. While it appears that they are not aserious threat to the security of the infected hosts, the scripts can bemanipulated at any time to include arbitrary code. Two other packages weremodified in the same manner." Thisthread in the aur-general list shows the timeline of the discovery andresponse.

In many ways, Spectre variant 1 (the bounds-check bypass vulnerability) isthe ugliest of the Meltdown/Spectre set, despite being relatively difficultto exploit. Any given code base could be filled with V1 problems, but theyare difficult to find and defend against. Static analysis can help, butthe available tools are few, mostly proprietary, and prone to falsepositives. There is also a lack of efficient, architecture-independentways of addressing Spectre V1 in user-space code. As a result, only alimited effort (at most) to find and fix Spectre V1 vulnerabilities hasbeen made in most projects. An effort to add some defenses to GCC may helpto make this situation better, but it comes at a cost of its own.

Security updates have been issued by Debian (ruby-sprockets), Red Hat (ansible and rh-git29-git), Scientific Linux (firefox), SUSE (ceph), and Ubuntu (libjpeg-turbo, ntp, and openslp-dfsg).

In the 4.18 kernel, a new feature was merged to allow infrared (IR)decoding to be done using BPF. Infrared remotes use many differentencodings; if a decoder were to be written for each, we would end up withhundreds of decoders in the kernel. So, currently, the kernel only supportsthe most widely used protocols. Alternatively, the lirc daemon canbe run to decode IR. Decoding IR can usually be expressed in a few lines ofcode, so a more lightweight solution without many kernel-to-userspacecontext switches would be preferable. This article will explain how IRmessages are encoded, the structure of a BPF program, and how a BPF programcan maintain state between invocations. It concludes with a look at thesteps that are taken to end up with a button event, such as a volume-up keyevent.

Security updates have been issued by Debian (bouncycastle and ca-certificates), Fedora (cantata, cinnamon, php-symfony3, and transifex-client), openSUSE (ghostscript, openssl, openvpn, php7, rubygem-yard, thunderbird, ucode-intel, and unzip), and SUSE (libqt4, nodejs8, and openslp).

The 4.18-rc4 kernel prepatch has beenreleased. "Things look pretty normal here, and size-wise this looksgood too, so it's another of those 'solid progress to release'weeks. Boring is good."

The 4.17.5 and 4.14.54 stable kernels have been released withyet another set of important fixes.

For those with a significant chunk of spare time and nothing better to do:Swapnil Bhartiya interviewed LWN editor Jonathan Corbet in February has now posted the resulting video onthe Patreon site.

Security updates have been issued by Debian (dokuwiki, libsoup2.4, mercurial, php7.0, and phpmyadmin), Fedora (ant, gnupg, libgit2, and libsoup), openSUSE (cairo, git-annex, postgresql95, and zsh), Scientific Linux (firefox), Slackware (mozilla), SUSE (nodejs6 and rubygem-yard), and Ubuntu (AMD microcode, devscripts, and firefox).

Large data centers routinely use control groups to balance the use of theavailable computing resources among competing users. Block I/O bandwidthcan be one of the most important resources for certain types of workloads,but the kernel's I/O controller is not a complete solution to the problem.The upcoming block I/O latency controllerlooks set to fill that gap in the near future, at least for some classes ofusers.

Security updates have been issued by Oracle (firefox), SUSE (exiv2, ghostscript, libvorbis, openssl, openvpn, php7, tiff, and unzip), and Ubuntu (libarchive-zip-perl and php7.2).

The LWN.net Weekly Edition for July 5, 2018 is available.

LWN reported on June 29 that Gentoo'sGitHub mirror had been compromised. Gentoo now considers the incidentresolved and the full report isavailable. "An unknown entity gained control of an admin account for the Gentoo GitHub Organization and removed all access to the organization (and its repositories) from Gentoo developers. They then proceeded to make various changes to content. Gentoo Developers & Infrastructure escalated to GitHub support and the Gentoo Organization was frozen by GitHub staff. Gentoo has regained control of the Gentoo GitHub Organization and has reverted the bad commits and defaced content."