LWN.net

Security updates have been issued by Debian (busybox, graphicsmagick, and libmspack), Fedora (pam_yubico), Scientific Linux (openslp), Slackware (lftp), SUSE (cups, libtirpc, and thunderbird), and Ubuntu (clamav).

Whenever one is engaged in large-scale changes to a software project, it isnice to have some assurance that regressions are not being introduced inthe process. Test suites can be helpful in that regard. But while thetesting of low-level components can be relatively straightforward, testingat the user-interface level can be harder. Web applications, which mustalso interact with web browsers, can be especially challenging in thisregard. While working on just this sort ofproject, your editor finally got around to looking at SeleniumWebDriver as a potential source of help for the testing problem.

Security updates have been issued by Debian (busybox and mutt), Fedora (bibutils and wireshark), openSUSE (glibc and rsyslog), Slackware (blueman), SUSE (cups, ovmf, and polkit), and Ubuntu (bouncycastle, libmspack, and python-django).

The LWN.net Weekly Edition for August 2, 2018 is available.

<p>Memory allocation for applications is a bit of a balancing act betweenvarious factors including CPU performance, memory efficiency, and how thememory is actually being allocated and deallocated by the application. Different programs may have diverse needs,but it is often the kind of workload that the application is expected to handle thatdetermines which memory allocator performs best. That argues for adiversity of memory allocators (and allocation strategies) but, on theother hand, thatcomplicates things for Linux distributions. As a result, Fedora isdiscussing ways to rein in the spread of allocators used by its packages.

The OpenWrt community has announcedthe first release of the OpenWrt 18.06 stable version series. "Itincorporates over 4000 commits since branching the previous LEDE 17.01release and has been under development for well over a year. With thisrelease, the re-merged OpenWrt project attempts to define a baseline for future development based on the technological modernizationand refined release processes done by the former LEDE project."

A PEP that has been around for a while, without being either accepted orrejected, was reintroduced recently on the python-ideas mailing list.PEP 505("None-aware operators") would provide some syntactic sugar, in the form ofnew operators, to handle cases where variables might be the special None value. It is a featurethat other languages support, but has generally raised concerns about being"un-Pythonic" over the years. At this point, though, the Python projectstill needs to figure out how it will begoverned—and how PEPs can be accepted or rejected.

Security updates have been issued by Debian (ruby2.3), Fedora (java-1.8.0-openjdk, java-openjdk, poppler, python-cryptography, and zziplib), Oracle (openslp), Red Hat (Red Hat Virtualization), and SUSE (kernel).

Version 2.28 of the GNU C Library is out. Changes include support forIntel's "Control-flow Enforcement Technology", Unicode 11.0.0 support, awrapper for statx(), ISO Cthreads support, several security fixes, and more.

The O'Reilly Open SourceConference (OSCON) returned to Portland, Oregon this July for the 20th convocation of this venerable gathering. While some of theprogram focused on retrospectives, there were also talks and tutorials onmultiple technical topics and open-source community management. To give youa feel for the whole conference, we will explore it in a two-part article. This installment will cover a retrospective of opensource and some presentations on releasing projects as open source at yourorganization. A second article will include a few of the technicaltopics at the conference.

LWN has been running articles for years to the effect that the end ofPython 2 is nigh and that code should be ported to Python 3immediately. So, naturally, one might expect that our own site code, written in Python, had beenforward-ported long ago. Strangely enough, that didn't actually happen.It has mostly happened now, though. In the process of doing thiswork, your editor has noticed a few things that don't necessarily appear inthe numerous porting guides circulating on the net.

For those waiting on the edges of their seats for the release of the 4.18kernel: it looks like Linus will be pushing it back one week (toAugust 12) in response to some late-discovered problems. "I _prefer_ justthe regular cadence of releases, but when I have a reason to delay, I'll delay."

Security updates have been issued by Debian (network-manager-vpnc), Fedora (wireshark), Oracle (java-1.7.0-openjdk and yum-utils), Red Hat (chromium-browser, java-1.7.0-openjdk, memcached, qemu-kvm-rhev, and yum-utils), Scientific Linux (java-1.7.0-openjdk and yum-utils), Slackware (file and seamonkey), SUSE (gdk-pixbuf, libcgroup, libcgroup1, libvirt, and sssd), and Ubuntu (mysql-5.5 and mysql-5.5, mysql-5.7).

One might think that memory allocation during system startup should not bedifficult: almost all of memory is free, there is no concurrency,and there are no background tasks that will compete for memory. Even so,boot-time memory management is a tricky task. Physical memory is notnecessarily contiguous, its extents change from system to system, andthe detection of those extents may be not trivial. With NUMA thingsare even more complex because, in order to satisfy allocationlocality, the exact memory topology must be determined.To cope with this, sophisticated mechanisms for memory management arerequired even during the earliest stages of the boot process.<p>Read on for a history of the evolution of the kernel's early-boot memoryallocator, contributed by Mike Rapoport.

Security updates have been issued by Arch Linux (libextractor and wesnoth), Debian (ffmpeg, fuse, libidn, mercurial, openssl, policykit-1, tomcat7, tomcat8, wireshark, and wordpress), Fedora (java-1.8.0-openjdk, java-openjdk, libpng10, php, sox, and suricata), Gentoo (curl and znc), openSUSE (bouncycastle, Chromium, cinnamon, e2fsprogs, ImageMagick, kernel, libgcrypt, mercurial, openssh, openssl-1_0_0, openssl-1_1, python, qutebrowser, rubygem-sprockets, shadow, and xen), Slackware (kernel), and SUSE (java-10-openjdk, kernel, libgcrypt, libvirt, mutt, and xen).

The 4.18-rc7 kernel prepatch is out fortesting. "So unless something odd happens, this should be the lastrc for 4.18".

Daniel Stone reflectson the completion of freedesktop.org's move to a GitLab-basedinfrastructure. "We’ve spent the past couple of years paying downour technical debt, and the community equivalent thereof. Ourinfrastructure is much less error-prone than it was: we’ve gone fromfighting fires to being able to prepare the new GitLab infrastructure andspend time shepherding projects through it. Now that we have a fair fewprojects on GitLab and they’ve been able to serve themselves, we’ve beenable to take some time for community issues."

Bradley Kuhn noteswith sadness the passing of Gervase Markham. "Gerv's time withus was too short. In response, I suggest that we look at his life and workand learn from his example. Gerv set aside his illness for as long aspossible to continue good work in FLOSS. If he can do that, we can all beinspired by him to set aside virtually any problem to work hard, together,for important outcomes that are bigger than us all."

There is a new set of stable kernel updates available:4.17.11,4.14.59,4.9.116,4.4.145, and3.18.117.Each contains another collection of important fixes.

The kernel's out-of-memory (OOM) killer is summoned when the system runsshort of free memory and is unable to proceed without killing one or moreprocesses. As might be expected, the policy decisions around whichprocesses should be targeted have engendered controversy for as long as theOOM killer has existed. The 4.19 development cycle is likely to includea new OOM-killer implementation that targets control groups rather thanindividual processes, but it turns out that there is significantdisagreement over how the OOM killer and control groups should interact.

Over on the Freedom to Tinker blog, Vitaly Shmatikov reports on some research he and others have been doing on machine-learning models—and what can be hidden inside them."Federated learning, where models are crowd-sourced from hundreds or even millions of users, is an even juicier target. In a recent paper [PDF], we show that a single malicious participant in federated learning can completely replace the joint model with another one that has the same accuracy but also incorporates backdoor functionality. For example, it can intentionally misclassify images with certain features or suggest adversary-chosen words to complete certain sentences.When training ML [machine learning] models, it is not enough to ask if the model has learned its task well. Creators of ML models must ask what else their models have learned. Are they memorizing and leaking their training data? Are they discovering privacy-violating features that have nothing to do with their learning tasks? Are they hiding backdoor functionality? We need least-privilege ML models that learn only what they need for their task – and nothing more."

This paper fromfour Graz University of Technology researchers [PDF] describes amechanism they have developed to exploit the Spectre V1 vulnerabilityover the net, with no local code execution required. "We show thatmemory access latency, in general, can be reflected in the latency ofnetwork requests. Hence, we demonstrate that it is possible for an attackerto distinguish cache hits and misses on specific cache lines remotely, bymeasuring and averaging over a larger number of measurements. Based onthis, we implemented the first access-driven remote cache attack, a remotevariant of Evict+ Reload called Thrash+Reload. Our remote Thrash+Reloadattack is a significant leap forward from previous remote cache timingattacks on cryptographic algorithms. We facilitate this technique toretrofit existing Spectre attacks to our network-based scenario. ThisNetSpectre variant is able to leak 15 bits per hour from a vulnerabletarget system." Other attacks described in the paper are able toachieve higher rates.

Security updates have been issued by CentOS (java-1.8.0-openjdk and thunderbird), Debian (busybox, chromium-browser, intel-microcode, mailman, and vim-syntastic), Fedora (NetworkManager-vpnc), SUSE (exempi, java-1_8_0-ibm, libofx, libsndfile, microcode_ctl, ntfs-3g, ovmf, rpm, util-linux, webkit2gtk3, and xen), and Ubuntu (clamav and evolution-data-server).

Patrick Volkerding, who is the founder and benevolent dictator for life of the Slackware Linux distribution, posted a note at LinuxQuestions.org detailing some financial problems. It appears they mostly stem from a deal that he made with the Slackware Store that has gone badly awry."Still not sure how to move forward, but I have some hope that the community might think that my work is and has been worth supporting. If at all possible I'd like to get away from replicating physical media which seems to be a lost cause. T-shirts? Well, maybe, but I don't see that providing a reasonable income either. I'm wondering how Patreon would do. It would at least be better than nothing, which is where I am now.Through all of this I have continued to work hard towards getting Slackware 15.0 released because I believe it will be by far the best release we've ever had, and because I'm dedicated to my work and the community that uses it. I've never really been in this for the money. " Note that there is at least one person out there soliciting Bitcoin who is not affiliated with Volkerding, in what looks like a scam of some sort; it is particularly sad because that is similar to what he alleges has happened with Slackware Store as well. No word, yet, on how to go about helping out. [Thanks to Ken Dawson for a heads-up about this.][Update: Volkerding has posted his PayPal link for donations.]

Security updates have been issued by Arch Linux (jenkins), CentOS (java-1.8.0-openjdk, openslp, and thunderbird), Fedora (dcraw and httpd), Oracle (java-1.8.0-openjdk and thunderbird), Red Hat (procps), Scientific Linux (thunderbird), SUSE (kernel), and Ubuntu (clamav and tomcat7, tomcat8).

The LWN.net Weekly Edition for July 26, 2018 is available.

Patents and open-source projects are always a messy combination it seems.A recent discussion on the pgsql-hackers mailing list highlights some ofthe problems that can result even when a patent holder wants to make theirpatents available to a project like PostgreSQL. Software patents are aminefield in many ways—often projects want to just avoid the problemsentirely by staying completely away from code known to be covered by patents.

The kernel has a range of mechanisms for notifying user space whensomething of interest happens. These include dnotify and inotify for filesystem events,signals, poll(), tracepoints, uevents, and more. One might think thatthere would be little need for yet another, but there are still events ofinterest that user space can only learn about by polling. In an attempt tofix this problem, David Howells, not content with his recent attempt to add seven new system calls for filesystemmounting, has put forward a proposal for ageneral-purpose event notification mechanism for Linux.

GCC has a lot of command-lineoptions—so many, in fact, that its build process does a fair amount ofprocessing using AWK to generate theoption-parsing code for the compiler. But some find the AWK code to bedifficult to work with. A recent post to the GCC mailing list proposes replacing AWK withPython in the hopes of more maintainable option-parsing generation in thefuture.

Stable kernels 4.17.10, 4.14.58, 4.9.115, and 4.4.144 have been released. They all containimportant fixes throughout the tree and users should upgrade.

Security updates have been issued by Debian (ant, evolution-data-server, libarchive-zip-perl, mailman, resiprocate, slurm-llnl, and sympa), Mageia (firmware, kernel, microcode, and wesnoth), openSUSE (Chromium), Oracle (openslp and thunderbird), Red Hat (java-1.7.0-oracle, java-1.8.0-oracle, kernel, qemu-kvm-rhev, and thunderbird), SUSE (kernel, nautilus, and xen), and Ubuntu (ant and clamav).

Peter Hutterer writesabout why libinput exists. It turns out that, like most otherhardware, input devices have no end of obnoxious quirks to deal with."All this is just handling features that users have come toexpect. Examples for non-features that you'll have to implement: on someLenovo series (*50 and newer) you will get a pointer jump after a series ofof events that only have pressure information. You'll have to detect anddiscard that jump. The HP Pavilion DM4 touchpad has random jumps in theslot data. Synaptics PS/2 touchpads may 'randomly' end touches and restartthem on the next event frame 10ms later. If you don't handle that you'llget ghost taps. And so on and so forth."

The 4.18-rc6 kernel prepatch came out onJuly 22, right on schedule. That is a sign that this development cycle is approachingits conclusion, so the time has come for a look at some statistics for howthings went this time around. It was another fairly ordinary releasecycle for the most part, but with a couple of distinctive features.

Here is theEconomist's take on the state of the Python language and community."Mr Van Rossum, though delighted by this enthusiasm for his software,has come to find the rigours of supervising it, in his role as 'benevolentdictator for life', unbearable. He fears he has become something of anidol. 'I’m uncomfortable with that fame,' he says, sounding uncannily likeBrian trying to drive away the crowds of disciples. 'Sometimes I feel likeeverything I say or do is seen as a very powerful force.' On July 12th heresigned, leaving the Pythonistas to manage themselves."

<p>Random number generation in the kernel has garnered a lot of attention overthe years. The tensions between the need for cryptographic-strength randomnumbers versus getting strong random numbers more quickly—along with the needto avoid regressions—has led to something of a patchwork of APIs. While itis widely agreed that waiting for a properly initialized random numbergenerator (RNG) before producing random numbers is the proper course,opinions differ on what "properly" means exactly. Beyond that, waiting,especially early in the boot process, can be problematic as well. Onesolution would be to trust the RNG instructions provided by most modernprocessors, but that comes with worries of its own.

Security updates have been issued by Debian (network-manager-vpnc), Fedora (haproxy, mailman, and NetworkManager-vpnc), Mageia (clamav, ffmpeg, rust, thunderbird, and wireshark), Oracle (java-1.8.0-openjdk and openslp), Red Hat (rh-ror42-rubygem-sprockets and rh-ror50-rubygem-sprockets), Scientific Linux (java-1.8.0-openjdk and openslp), SUSE (ImageMagick, libofx, php53, and python-dulwich), and Ubuntu (linux, linux-aws, linux-azure, linux-gcp, linux-kvm, linux-oem, linux-hwe, linux-azure, linux-gcp, mutt, and python-cryptography).

NetBSD 8.0 has been released.This version features USB stack rework with USB3 support added, anin-kernel audio mixer, reproducible builds, full userland debuginformation, and much more.

Security updates have been issued by Arch Linux (apache, networkmanager-vpnc, and znc), Debian (gosa, opencv, and slurm-llnl), Fedora (evolution, evolution-data-server, evolution-ews, gnome-bluetooth, libtomcrypt, podman, python-cryptography, and rust), Gentoo (passenger), Red Hat (java-1.8.0-openjdk and openslp), Slackware (php), SUSE (openssl-1_1, procps, python, rsyslog, rubygem-passenger, and xen), and Ubuntu (mutt).

The sixth 4.18 kernel prepatch is out fortesting. "So this was the week when the other shoe dropped ... The reason thetwo previous rc releases were so nice and small was that David hadn'tsent me much networking fixes, and they came in this week.That said, it's not really a huge rc this week either, so it's allgood."

Greg Kroah-Hartman has released five new stable kernels: 4.17.9, 4.14.57, 4.9.114, 4.4.143, and 3.18.116. As usual, they contain importantchanges throughout the kernel tree; users of those series should upgrade.

Over on the Facebook code site, Daniel Xu announces the release of oomd under the GPLv2. Oomd is a user-space "out of memory" killer that was mentioned in our recent article on the block I/O latency controller and it uses the pressure stall information covered in an even more recent article."Oomd constantly monitors PSI [Pressure Stall Information] metrics to assess whether a system is under unrecoverable load. PSI alone is insufficient, so oomd also monitors the system holistically. This is in contrast to Linux’s OOM killer, which focuses primarily on the kernel’s concerns. Since OOM detection criteria can vary depending on workload, the plugin system supports customization to both the detection and process kill strategies.Thanks to this new ability to monitor key system resource indicators, oomd is able to take corrective action in userspace before a system-wide OOM occurs. Corrective action is configured via a flexible plugin system that is capable of executing custom code. Thus, in addition to oomd’s default process SIGKILL behavior, application developers can customize their plugin with alternate strategies, such as sending a 'back off' RPC to the main workload or dumping system logs to a remote service."

Security updates have been issued by Debian (dnsmasq, linux-base, and openjpeg2), Fedora (libgit2, libtomcrypt, openslp, and perl-Archive-Zip), and openSUSE (gdk-pixbuf, libopenmpt, mercurial, perl, php7, polkit, and rsyslog).

The kernel supports two different "SCSI generic" pseudo-devices, each ofwhich allows user space to send arbitrary commands to a SCSI-attacheddevice. Both SCSI-generic implementations have proved to have securityissues in the past as a result of the way their API was designed. In thecase of one of those drivers, these problems seem almost certain to lead to theremoval of a significant chunk of functionality in the 4.19 developmentcycle.

Greg Kroah-Hartman has released the 4.4.142stable kernel. It is not an essential upgrade, "but a number ofbuild problems with perf are now resolved, and an x86 issue that some people might have hitis now handled properly. If those were problems for you, pleaseupgrade."

Security updates have been issued by Debian (ant, gpac, linux-4.9, linux-latest-4.9, taglib, vlc, and znc), Fedora (ceph), Red Hat (fluentd and qemu-kvm-rhev), Slackware (httpd), and SUSE (e2fsprogs, glibc, libgcrypt, mercurial, openssh, perl, rubygem-sprockets, shadow, and wireshark).

The LWN.net Weekly Edition for July 19, 2018 is available.

<p>Deep-learning applications typically rely on a trained neural net toaccomplish their goal (e.g. photo recognition, automatic translation, orplaying go). That neural net uses what is essentially a large collection ofweighting numbers that have been empirically determined as part of its training (which generally uses a huge set of training data). Afree-software application could use those weights, but there are a number of barriers for users who might want to tweak them for variousreasons. A discussion on the debian-devel mailing list recently looked atwhether these deep-learning applications can ever truly be considered "free" (as infreedom) because of these pre-computed weights—and the difficultiesinherent in changingthem.

Over the last few months, it became clear that the battle over PEP 572 wouldbe consequential; its scale and vehemence was largely unprecedented in thehistory of Python. The announcement by Guido van Rossum thathe was stepping down from his role as benevolent dictator for life (BDFL),due in part to that battle,underscored the importance of it. While the Python project charts its course in the wake of hisresignation, it makes sense to catch up on where things stand with thiscontentious PEP that has now been accepted for Python 3.8.

Stable kernel 4.17.8 has been released.This fixes the issue with i386 systems that was present in the 4.17.7 kernel.

Security updates have been issued by Arch Linux (curl, lib32-curl, lib32-libcurl-compat, lib32-libcurl-gnutls, libcurl-compat, and libcurl-gnutls), Debian (blender, ffmpeg, and wordpress), Fedora (curl), Gentoo (tqdm), Oracle (kernel), Slackware (mutt), SUSE (xen), and Ubuntu (policykit-1).