LWN.net

A new archive of linux-kernel mailing list (LKML) posts going back to 1998 is now available at lore.kernel.org. It is based on public-inbox (which we looked at back in February. Among other things, public-inbox allows retrieving the entire archive via Git: "Git clone URLs are provided at the bottom of each page. Note, that due to its volume, the LKML archive is sharded into multiple repositories, each roughly 1GB in size. In addition to cloning from lore.kernel.org, you may also access these repositories on git.kernel.org." The full announcement, which includes information about a new Patchwork instance as well as ways to link into the new archive, can be found on kernel.org.

Security updates have been issued by Debian (php-horde-image), openSUSE (kernel), Scientific Linux (git), SUSE (bluez, kernel, mariadb, and mariadb, mariadb-connector-c, xtrabackup), and Ubuntu (openjdk-7).

On his blog, James Bottomley looks at the value proposition for various types of cloud deployments. In particular, he compares the vertical and horizontal attack profile (VAP and HAP) of four different models: separate servers, separate logins on a single server, virtual machines, and containers. He finds the container story to be compelling: "The total VAP here is identical to that of physical infrastructure. However, the Tenant component is much smaller (the kernel accounting for around 50% of all vulnerabilities). It is this reduction in the Tenant VAP that makes containers so appealing: the CSP [cloud service provider] is now responsible for monitoring and remediating about half of the physical system VAP which is a great improvement for the Tenant. Plus when the CSP remediates on the host, every container benefits at once, which is much better than having to crack open every virtual machine image to do it. Best of all, the Tenant images don’t have to be modified to benefit from these fixes, simply running on an updated CSP host is enough. However, the cost for this is that the HAP is the entire linux kernel syscall interface meaning the HAP is much larger than then hypervisor virtual infrastructure case because the latter benefits from interface narrowing to only the hypercalls (qualitatively, assuming the hypercall interface is ~30 calls and the syscall interface is ~300 calls, then the HAP is 10x larger in the container case than the hypervisor case); however, thanks to protections from the kernel namespace code, the HAP is less than the shared login server case. Best of all, from the Tenant point of view, this entire HAP cost is borne by the CSP, which makes this an incredible deal: not only does the Tenant get a significant reduction in their VAP but the CSP is hugely motivated to keep on top of all vulnerabilities in their part of the VAP and remediate very fast because of the business implications of a successful horizontal attack."

Security updates have been issued by openSUSE (cobbler and matrix-synapse), Oracle (git), Red Hat (git), SUSE (java-1_7_1-ibm, nagios-nrpe, and ntp), and Ubuntu (AMD microcode).

The LWN.net Weekly Edition for June 21, 2018 is available.

Stable kernels 4.16.17 and 4.14.51 have been released with lots of fixesthroughout the tree. Users should upgrade.

<p>A two-part session at the 2018 Python Language Summit tackled the coredeveloper diversity problem from two different angles. Victor Stinneroutlined some work he has been doing to mentor new developers on their pathtoward joining the core development ranks; he has also been trying todocument that path. Mariatta Wijaya gave a very personal talk thatdescribed the diversity problem while also providing some concrete actionitems that the project and individuals could take to help make Python morewelcoming to minorities.

In a session with a title that used a common misquote of RodneyKing ("can't we all just get along?"), severalPython developers wanted to discuss an incident that had recently occurredon the python-dev mailing list. A rude posting to the list led to a thread thatgot somewhat out of control. Some short tempers among the members of thePython developer community likely escalated things unnecessarily. Theincident in question was brought up as something of an object lesson;people should take some time to simmer down before firing off that quick,but perhaps needlessly confrontational, reply.

The "PEP 572 mess" was the topic of a 2018 Python Language Summit sessionled by benevolent dictator for life (BDFL) Guido van Rossum. PEP 572 seeks to addassignment expressions (or "inline assignments") to the language, but ithas seen a prolonged discussion over multiple huge threads on the python-dev mailing list—evenafter multiple rounds on python-ideas.Those threads were often contentious and were clearly voluminous to thepoint where many probably just tuned them out.At the summit, Van Rossum gave an overview of thefeature proposal, which he seems inclined toward accepting, but he alsowanted to discuss how to avoid this kind of thread explosion in the future.

Matthew Miller looks at how Red Hat's acquisition of CoreOS will affect theFedora project. "This isn’t the place for technical details — see“what next?” at the bottom of this message for more. I expect that over thenext year or so, Fedora Atomic Host will be replaced by a new thingcombining the best from Container Linux and Project Atomic. Thisnew thing will be “Fedora CoreOS” and serve as the upstream to RedHat CoreOS."

Security updates have been issued by Arch Linux (pass), Debian (xen), Fedora (chromium, cobbler, gnupg, kernel, LibRaw, mariadb, mingw-libtiff, nikto, and timidity++), Gentoo (chromium, curl, and transmission), Mageia (gnupg, gnupg2, librsvg, poppler, roundcubemail, and xdg-utils), Red Hat (ansible and glusterfs), Slackware (gnupg), SUSE (cobbler, dwr, java-1_8_0-ibm, kernel, microcode_ctl, pam-modules, salt, slf4j, and SMS3.1), and Ubuntu (libgcrypt11, libgcrypt11, libgcrypt20, and mozjs52).

Security updates have been issued by Arch Linux (libgcrypt), Fedora (bouncycastle, nodejs, and perl-Archive-Tar), openSUSE (aubio), and Red Hat (chromium-browser, glibc, kernel, kernel-rt, libvirt, pcs, samba, samba4, sssd and ding-libs, and zsh).

This article describes our findings that connected TCP small queues (TSQ)with the behavior of advanced WiFi protocols and, in the process, solved athroughput regression. The resulting patch is already in the mainline tree, so beforecontinuing, please make sure your kernel is updated. Beyond the fix, it isdelightful to travel through history to see how we discovered the problem,how it was tackled, and how it was patched.Subscribers can read on for the full story by guest authors Carlo Grazia and Natale Patriciello.

Security updates have been issued by CentOS (kernel), Debian (libgcrypt20, redis, and strongswan), Fedora (epiphany, freedink-dfarc, gnupg, LibRaw, nodejs-JSV, nodejs-uri-js, singularity, strongswan, and webkit2gtk3), Mageia (flash-player-plugin, freedink-dfarc, and imagemagick), openSUSE (enigmail, gpg2, java-1_7_0-openjdk, java-1_8_0-openjdk, poppler, postgresql96, python-python-gnupg, and samba), Oracle (kernel), SUSE (gpg2 and xen), and Ubuntu (gnupg and webkit2gtk).

By the time that Linus Torvalds released 4.18-rc1 and closed the merge window for this development cycle, 11,594 non-merge changesets hadfound their way into the mainline kernel repository. Nearly 4,500 of thosewere pulled after last week's summary waswritten. Thus, in terms of commit traffic, 4.18 looks to be quite similarto its predecessors. As usual, the entry of significant new features hasslowed toward the end of the merge window, but there are still someimportant changes on the list.

The stable update machine continues to crank out releases:4.17.2,4.16.16,4.14.50,4.9.109, and4.4.138are all available with another set of important fixes.

The first 4.18 prepatch is out, and themerge window has closed for this development cycle. "You may think it's stillSaturday for me, and that I should give you one more day of merge window tosend in some last-minute pull requests, but I know better. I'm in Japan,and it's Sunday here."

It's been a little over one year since we last covered Debian's reproducible buildsproject. The effort has not stopped in the interim; progress continuesto be made, the message has sharpened up, and word is spreading. ChrisLamb, speaking about this at FLOSS UK in a talk called "You may thinkyou're not a target: a tale of three developers", hinted that the end may be starting to come into sight.

Security updates have been issued by CentOS (plexus-archiver), Fedora (chromium, kernel, and plexus-archiver), Mageia (firefox, gifsicle, jasper, leptonica, patch, perl-DBD-mysql, qt3, and scummvm), openSUSE (opencv), Oracle (kernel), Red Hat (kernel), Scientific Linux (kernel), SUSE (gpg2, nautilus, and postgresql96), and Ubuntu (gnupg2 and linux-raspi2).

Kees Cook describesthe security-oriented changes included in the 4.17 kernel release."It was possible that old memory contents would live in a newprocess’s kernel stack. While normally not visible, “uninitialized” memoryread flaws or read overflows could expose these contents (especially stuff“deeper” in the stack that may never get overwritten for the life of theprocess). To avoid this, I made sure that new stacks were alwayszeroed. Oddly, this “priming” of the cache appeared to actually improveperformance, though it was mostly in the noise."

Ars technica has thestory of a set of Docker images containing cryptocurrency miners thatpersisted on Docker Hub for the better part of a year — after beingdiscovered. "Neither the Docker Hub account nor the malicious images it submitted were takendown. Over the coming months, the account went on to submit 14 moremalicious images. The submissions were publicly called out two more times,once in January by security firm Sysdig and again in May by securitycompany Fortinet. Eight days after last month's report, Docker Hub finallyremoved the images."

Security updates have been issued by Arch Linux (chromium and gnupg), Debian (spip), Fedora (pdns-recursor), Gentoo (adobe-flash, burp, quassel, and wget), openSUSE (bouncycastle and taglib), Oracle (kernel), SUSE (java-1_7_0-openjdk, java-1_8_0-openjdk, poppler, and samba), and Ubuntu (file, perl, and ruby1.9.1, ruby2.0, ruby2.3).

The LWN.net Weekly Edition for June 14, 2018 is available.

In a short session at the 2018 Python Language Summit, Steve Dower broughtup the shortcomings of Python virtual environments,which are meant to create isolated installations of the language and itsmodules. He said his presentation was "co-written with Twitter" and,indeed, most of his slides were of tweets. At the end, he also slipped in anannouncement of his plans for hosting a core development sprint in September.

The XArray data structure was the topic ofthe final filesystem track session at the 2018 Linux Storage, Filesystem,and Memory-Management Summit (LSFMM). XArray is a new API for the kernel'sradix-tree data structure; the session wasled by Matthew Wilcox, who created XArray. When asked by Dave Chinner ifthe session was intended to be a live review of the patches, Wilcoxadmitted with a grin that it might be "the only way to get a review on thisdamn patch set".

While the 2018 Linux Storage, Filesystem, and Memory-Management Summit(LSFMM) filesystem track session was advertised as being a filesystem test suite "bakeoff", it actually focused on how to make the existing test suites moreaccessible. Kent Overstreet said that he has learned over theyears that various filesystem developers have their own scripts for testingusing QEMU and other tools. He and Ted Ts'o put the session together totry to share some of that information (and code) more widely.

Stable kernels 4.9.108, 4.4.137, and 3.18.113 have been released. As usual, theyall contain important fixes and users should upgrade.

In the filesystem track at the 2018 Linux Storage, Filesystem, andMemory-Management Summit (LSFMM), Al Viro discussed some problems he hasrecently spotted in the implementation of rmdir().He covered some of the history of that implementation and how things got towhere they are now. He also described areas that needed to be checkedbecause the problem may be present in different places in multiple filesystems.

Security updates have been issued by Debian (plexus-archiver), Oracle (plexus-archiver), Red Hat (plexus-archiver and rh-maven33-plexus-archiver and rh-maven35-plexus-archiver), Scientific Linux (plexus-archiver), SUSE (pdns, poppler, ucode-intel, wpa_supplicant, and xen), and Ubuntu (bind9, firefox, and linux-azure).

One of the larger features added to Python over the last few releases issupport for static typing in the language. Static type-checking and toolsto support it show up frequentlyas topics at the Python LanguageSummit (PLS) and this year was no exception. Mypy developers Jukka Lehtosalo and IvanLevkivskyi gave an update on static typing at PLS 2018.

One of the many longstanding — though unwritten — rules of kerneldevelopment is that infrastructure is not merged until at least one userfor that infrastructure exists. That helps developers evaluate potentialinterfaces and be sure that the proposed addition is truly needed. A bigexception to this rule was made when the heterogeneous memory management(HMM) code was merged, though. One of the reasons for the lack of users inthis case turns out to be that many of the use cases are proprietary; thathas led to some disagreements over the GPL-only status of an exportedkernel symbol.

Stable kernels 4.17.1, 4.16.15, and 4.14.49 have been released. They all containimportant fixes and users should upgrade.

Security updates have been issued by Debian (perl), Red Hat (kernel), SUSE (slurm), and Ubuntu (gnupg, gnupg2, imagemagick, kernel, libvirt, linux, linux-aws, linux-gcp, linux-kvm, linux, linux-aws, linux-kvm, linux-raspi2, linux-snapdragon, linux, linux-raspi2, linux-azure, linux-hwe, linux-gcp, linux-oem, linux-lts-trusty, linux-lts-xenial, linux-aws, and qemu).

The kernel's handling of I/O errors was the topic of a discussion led byMatthew Wilcox at the 2018 Linux Storage, Filesystem, and Memory-ManagementSummit (LSFMM) in a combined storage and filesystem track session. At the start, he asked: "how is our error handling andwhat do we plan to do about it?" That led to a discussion between thedevelopers present on the kinds of errors that can occur and onways to handle them.

LWN reviewed CopperheadOS, asecurity-enhanced Android distribution, in 2016. Unfortunately, thecompany behind CopperheadOS appears to have run into internal trouble; wedon't dare venture a guess as to the specifics, even after watching thesituation for a few days, beyond the fact that thereis clearly a dispute between the founders. ThisReddit post is apparently a letter to co-founder Daniel Micayessentially kicking him out of the company. Users of CopperheadOS may wantto be considering alternatives.

Netconf, the Linux kernel networking development conference, has providedcoverage of this year's event, which was held in Boston, MA, May 31-June 1.Day 1 looks at the following sessions:

Python 2.7 will reach itsend of life in less than two years—at least for the core developmentteam.Linux distributions need to figure out how to handle the transition given that many of their users are still usingthat version of the language—and may still be well beyond the end-of-lifedate. Petr Viktorin and Matthias Klose led a session at the 2018 PythonLanguage Summit to discuss distributions' approaches to deprecatingPython 2.

We now have less than 20 years to wait until the time_t value usedon 32-bit systems will overflow and create time-related mayhem across theplanet. The grand plan for solving thisproblem was posted over three years ago now; progress since then hasseemed slow. But quite a bit of work has happened deep inside the kerneland, in 4.18, some of the first work that will be visible to user space hasbeen merged. The year-2038 problem is not yet solved, but things aremoving in that direction.

Security updates have been issued by Arch Linux (chromium, firefox, flashplugin, krb5, and p7zip), Debian (firefox-esr, gnupg, gnupg1, gnupg2, libvncserver, and openjdk-7), Fedora (batik, qt3, remctl, and slurm), openSUSE (curl, glibc, ImageMagick, kernel-firmware, libvirt, libvorbis, MozillaFirefox, mozilla-nss, mupdf, prosody, qemu, slf4j, and xen), Red Hat (chromium-browser, flash-plugin, and Red Hat Virtualization), Slackware (gnupg2), and SUSE (libvirt, mailman, and xen).

The 2018 Linux AudioConference has just concluded in Berlin. A substantial set of videos of talksfrom the event has already been published, with the rest slated to appearin the near future.

Version 2.0.0 of the systemd-free Debian-based Devuan distribution isavailable. There are some releasenotes available, but there is little information on what has changedsince the 1.0 release.

Here's adetailed update from the postmarketOS project on its first year.PostmarketOS is building an Android distribution aimed at keeping olderdevices working in a supported mode; much of this work involves gettingmainline kernels working on various handsets."You might remember @bshah's photo of the Nexus 5 running mainlinewith a flipped and distorted screen from December. @flto continued hiswork: the display works without problems now. But it gets even better: thetouch screen is working, 3D acceleration is enabled with the open sourcefreedreno userspace driver, Wi-Fi works, and the best part is that@MartijnBraam was able to send SMS and initialize a call via command lineas well as getting the connectivity signal from the modem through oFonodisplayed in Plasma Mobile (#1502). All of that without proprietaryuserspace blobs!"

As of this writing, 7,515 non-merge changesets have been pulled into themainline repository for the 4.18 merge window. Things are clearly off to astrong start. The changes pulled this time around include more than theusual number of interestingnew features; read on for the details.

Security updates have been issued by Arch Linux (radare2), Debian (jruby), Fedora (elfutils and wireless-tools), openSUSE (glibc, mariadb, and xdg-utils), Oracle (kernel), Red Hat (chromium-browser and java-1.7.1-ibm), SUSE (ceph, icu, kernel-firmware, memcached, and xen), and Ubuntu (unbound).

Greg Kroah-Hartman has announced the release of the 4.9.107 and 4.4.136 stable kernels. As usual, theycontain fixes throughout the tree; users of those kernel series should upgrade.

Security updates have been issued by Debian (memcached), Fedora (java-1.8.0-openjdk-aarch32, sqlite, and xen), Mageia (corosync, gimp, qtpass, and SDL_image), openSUSE (zziplib), Slackware (mozilla), SUSE (git and libvorbis), and Ubuntu (liblouis).

Peter Hutterer writesabout the behavior of trackpoint devices in great detail."Trackpoints have built-in calibration procedures to find and settheir own center-point. Without that you'll get the trackpoint eventuallybeing ever so slightly off center over time, causing a mouse pointer thatjust wanders off the screen, possibly into the woods, without theobligatory red cape and basket full of whatever grandma eats when she'ssick."

The LWN.net Weekly Edition for June 7, 2018 is available.

The Fedora Project is running an election for members of the FedoraEngineering Steering Committee (FESCo). Interviews with the candidateshave been posted:JustinForbes, PetrÅ abata, StephenGallagher, RandyBarlow, and Till Maas.

At the 2018 Linux Storage, Filesystem, and Memory-Management Summit(LSFMM), Jaegeuk Kim described some current issues for flash storage,especially with regard to Android. Kim is the F2FS developer andmaintainer, and the filesystem-track session was ostensibly about thatfilesystem. In the end, though, the talk did not focus on F2FS and insteadranged over a number of problem areas for Android flash storage.