LWN.net

The XArray data structure was the topic ofthe final filesystem track session at the 2018 Linux Storage, Filesystem,and Memory-Management Summit (LSFMM). XArray is a new API for the kernel'sradix-tree data structure; the session wasled by Matthew Wilcox, who created XArray. When asked by Dave Chinner ifthe session was intended to be a live review of the patches, Wilcoxadmitted with a grin that it might be "the only way to get a review on thisdamn patch set".

While the 2018 Linux Storage, Filesystem, and Memory-Management Summit(LSFMM) filesystem track session was advertised as being a filesystem test suite "bakeoff", it actually focused on how to make the existing test suites moreaccessible. Kent Overstreet said that he has learned over theyears that various filesystem developers have their own scripts for testingusing QEMU and other tools. He and Ted Ts'o put the session together totry to share some of that information (and code) more widely.

Stable kernels 4.9.108, 4.4.137, and 3.18.113 have been released. As usual, theyall contain important fixes and users should upgrade.

In the filesystem track at the 2018 Linux Storage, Filesystem, andMemory-Management Summit (LSFMM), Al Viro discussed some problems he hasrecently spotted in the implementation of rmdir().He covered some of the history of that implementation and how things got towhere they are now. He also described areas that needed to be checkedbecause the problem may be present in different places in multiple filesystems.

Security updates have been issued by Debian (plexus-archiver), Oracle (plexus-archiver), Red Hat (plexus-archiver and rh-maven33-plexus-archiver and rh-maven35-plexus-archiver), Scientific Linux (plexus-archiver), SUSE (pdns, poppler, ucode-intel, wpa_supplicant, and xen), and Ubuntu (bind9, firefox, and linux-azure).

One of the larger features added to Python over the last few releases issupport for static typing in the language. Static type-checking and toolsto support it show up frequentlyas topics at the Python LanguageSummit (PLS) and this year was no exception. Mypy developers Jukka Lehtosalo and IvanLevkivskyi gave an update on static typing at PLS 2018.

One of the many longstanding — though unwritten — rules of kerneldevelopment is that infrastructure is not merged until at least one userfor that infrastructure exists. That helps developers evaluate potentialinterfaces and be sure that the proposed addition is truly needed. A bigexception to this rule was made when the heterogeneous memory management(HMM) code was merged, though. One of the reasons for the lack of users inthis case turns out to be that many of the use cases are proprietary; thathas led to some disagreements over the GPL-only status of an exportedkernel symbol.

Stable kernels 4.17.1, 4.16.15, and 4.14.49 have been released. They all containimportant fixes and users should upgrade.

Security updates have been issued by Debian (perl), Red Hat (kernel), SUSE (slurm), and Ubuntu (gnupg, gnupg2, imagemagick, kernel, libvirt, linux, linux-aws, linux-gcp, linux-kvm, linux, linux-aws, linux-kvm, linux-raspi2, linux-snapdragon, linux, linux-raspi2, linux-azure, linux-hwe, linux-gcp, linux-oem, linux-lts-trusty, linux-lts-xenial, linux-aws, and qemu).

The kernel's handling of I/O errors was the topic of a discussion led byMatthew Wilcox at the 2018 Linux Storage, Filesystem, and Memory-ManagementSummit (LSFMM) in a combined storage and filesystem track session. At the start, he asked: "how is our error handling andwhat do we plan to do about it?" That led to a discussion between thedevelopers present on the kinds of errors that can occur and onways to handle them.

LWN reviewed CopperheadOS, asecurity-enhanced Android distribution, in 2016. Unfortunately, thecompany behind CopperheadOS appears to have run into internal trouble; wedon't dare venture a guess as to the specifics, even after watching thesituation for a few days, beyond the fact that thereis clearly a dispute between the founders. ThisReddit post is apparently a letter to co-founder Daniel Micayessentially kicking him out of the company. Users of CopperheadOS may wantto be considering alternatives.

Netconf, the Linux kernel networking development conference, has providedcoverage of this year's event, which was held in Boston, MA, May 31-June 1.Day 1 looks at the following sessions:

Python 2.7 will reach itsend of life in less than two years—at least for the core developmentteam.Linux distributions need to figure out how to handle the transition given that many of their users are still usingthat version of the language—and may still be well beyond the end-of-lifedate. Petr Viktorin and Matthias Klose led a session at the 2018 PythonLanguage Summit to discuss distributions' approaches to deprecatingPython 2.

We now have less than 20 years to wait until the time_t value usedon 32-bit systems will overflow and create time-related mayhem across theplanet. The grand plan for solving thisproblem was posted over three years ago now; progress since then hasseemed slow. But quite a bit of work has happened deep inside the kerneland, in 4.18, some of the first work that will be visible to user space hasbeen merged. The year-2038 problem is not yet solved, but things aremoving in that direction.

Security updates have been issued by Arch Linux (chromium, firefox, flashplugin, krb5, and p7zip), Debian (firefox-esr, gnupg, gnupg1, gnupg2, libvncserver, and openjdk-7), Fedora (batik, qt3, remctl, and slurm), openSUSE (curl, glibc, ImageMagick, kernel-firmware, libvirt, libvorbis, MozillaFirefox, mozilla-nss, mupdf, prosody, qemu, slf4j, and xen), Red Hat (chromium-browser, flash-plugin, and Red Hat Virtualization), Slackware (gnupg2), and SUSE (libvirt, mailman, and xen).

The 2018 Linux AudioConference has just concluded in Berlin. A substantial set of videos of talksfrom the event has already been published, with the rest slated to appearin the near future.

Version 2.0.0 of the systemd-free Debian-based Devuan distribution isavailable. There are some releasenotes available, but there is little information on what has changedsince the 1.0 release.

Here's adetailed update from the postmarketOS project on its first year.PostmarketOS is building an Android distribution aimed at keeping olderdevices working in a supported mode; much of this work involves gettingmainline kernels working on various handsets."You might remember @bshah's photo of the Nexus 5 running mainlinewith a flipped and distorted screen from December. @flto continued hiswork: the display works without problems now. But it gets even better: thetouch screen is working, 3D acceleration is enabled with the open sourcefreedreno userspace driver, Wi-Fi works, and the best part is that@MartijnBraam was able to send SMS and initialize a call via command lineas well as getting the connectivity signal from the modem through oFonodisplayed in Plasma Mobile (#1502). All of that without proprietaryuserspace blobs!"

As of this writing, 7,515 non-merge changesets have been pulled into themainline repository for the 4.18 merge window. Things are clearly off to astrong start. The changes pulled this time around include more than theusual number of interestingnew features; read on for the details.

Security updates have been issued by Arch Linux (radare2), Debian (jruby), Fedora (elfutils and wireless-tools), openSUSE (glibc, mariadb, and xdg-utils), Oracle (kernel), Red Hat (chromium-browser and java-1.7.1-ibm), SUSE (ceph, icu, kernel-firmware, memcached, and xen), and Ubuntu (unbound).

Greg Kroah-Hartman has announced the release of the 4.9.107 and 4.4.136 stable kernels. As usual, theycontain fixes throughout the tree; users of those kernel series should upgrade.

Security updates have been issued by Debian (memcached), Fedora (java-1.8.0-openjdk-aarch32, sqlite, and xen), Mageia (corosync, gimp, qtpass, and SDL_image), openSUSE (zziplib), Slackware (mozilla), SUSE (git and libvorbis), and Ubuntu (liblouis).

Peter Hutterer writesabout the behavior of trackpoint devices in great detail."Trackpoints have built-in calibration procedures to find and settheir own center-point. Without that you'll get the trackpoint eventuallybeing ever so slightly off center over time, causing a mouse pointer thatjust wanders off the screen, possibly into the woods, without theobligatory red cape and basket full of whatever grandma eats when she'ssick."

The LWN.net Weekly Edition for June 7, 2018 is available.

The Fedora Project is running an election for members of the FedoraEngineering Steering Committee (FESCo). Interviews with the candidateshave been posted:JustinForbes, PetrÅ abata, StephenGallagher, RandyBarlow, and Till Maas.

At the 2018 Linux Storage, Filesystem, and Memory-Management Summit(LSFMM), Jaegeuk Kim described some current issues for flash storage,especially with regard to Android. Kim is the F2FS developer andmaintainer, and the filesystem-track session was ostensibly about thatfilesystem. In the end, though, the talk did not focus on F2FS and insteadranged over a number of problem areas for Android flash storage.

At the 2018 Linux Storage, Filesystem, and Memory-Management Summit(LSFMM), Boaz Harrosh presented his zero-copy user-mode filesystem (ZUFS).It is both a filesystem in its own right and a framework similar to FUSE forimplementing filesystems in user space. It is geared toward extremely low latency andhigh performance, particularly for systems using persistent memory.

Security updates have been issued by Arch Linux (git), Fedora (php-symfony, php-symfony4, and thunderbird-enigmail), Mageia (glpi and libreoffice), openSUSE (dpdk-thunderxdpdk, git, and ocaml), SUSE (glibc, libvorbis, and zziplib), and Ubuntu (elfutils, git, and procps).

The kernel's staging tree is meant to be a path by which substandard codecan attract increased developer attention, be improved, and eventually findits way into the mainline kernel. Not every module graduates from staging; some are simply removed after it becomes clear that nobody caresabout them. It is rare, though, for a project that is actively developedand widely used to be removed from the staging tree, but that may be aboutto happen with the Lustre filesystem.

<p>Python is famous for being a "batteries included" language—its standardlibrary provides a versatile set of modules with the language—butthere may be times when some of those batteries have reached their end of life. Atthe 2018 Python Language Summit, Christian Heimes wanted to suggest a fewbatteries that may have outlived their usefulness and to discuss how theprocess of retiring standard library modules should work.

Stable kernels 4.16.14, 4.14.48, and 4.9.106 have been released. The 4.16 and 4.14kernels contain the usual set of important fixes. "Not everyone whouses 4.9 needs to upgrade here, it's a big sync of the objtool codebase tomake future maintenance of 4.9.y easier over time, that's all. But ofcourse, updating and verifying that nothing broke is always appreciated:)"

Security updates have been issued by Fedora (chromium, php-symfony3, and wireshark), Mageia (chromium-browser-stable, mariadb, and python3), openSUSE (kernel modules), SUSE (gcc43, oracleasm kmp, and xdg-utils), and Ubuntu (apport and liblouis).

At the 2017 Linux Storage, Filesystem, and Memory-Management Summit(LSFMM), Amir Goldstein presented his workon adding a superblock watch mechanism to provide a scalable way to notifyapplicationsof changes in a filesystem. At the 2018 edition of LSFMM, he was back todiscuss adding NTFS-like changejournals to the kernel in support of backup solutions of varioussorts. As a second topic for the session, he also wanted to discuss doingmore performance-regression testing for filesystems.

If you use Python, there's a good chance you have heard of IPython, which provides an enhanced read-eval-printloop (REPL) for Python. But there is more to IPython than just a moreconvenient REPL. Today's IPython comes with integrated libraries that turnit into an assistant for several advanced computing tasks. We will look attwo of those tasks, using multiple languages and distributed computing, inthis article.

Security updates have been issued by CentOS (procps, xmlrpc, and xmlrpc3), Debian (batik, prosody, redmine, wireshark, and zookeeper), Fedora (jasper, kernel, poppler, and xmlrpc), Mageia (git and wireshark), Red Hat (rh-java-common-xmlrpc), Slackware (git), SUSE (bzr, dpdk-thunderxdpdk, and ocaml), and Ubuntu (exempi).

Here's thepress release announcing Microsoft's agreement to acquire GitHub for amere $7.5 billion. "GitHub will retain its developer-firstethos and will operate independently to provide an open platform for alldevelopers in all industries. Developers will continue to be able to usethe programming languages, tools and operating systems of their choice fortheir projects — and will still be able to deploy their code to anyoperating system, any cloud and any device."

Linus has released the 4.17 kernel, whichwill indeed be called "4.17"."No, I didn't call it 5.0, even though all the git object countnumerology was in place for that. It will happen in the not _too_distant future, and I'm told all the release scripts on kernel.org areready for it, but I didn't feel there was any real reason for it."Headline features in this release includeimproved load estimation in the CPUscheduler,rawBPF tracepoints,lazytime support in the XFS filesystem,full in-kernel TLS protocol support,histogram triggers for tracing,mitigations for the latest Spectre variants,and, of course, the removal of support for eight unloved processorarchitectures.

There has been a lot of work in recent years to use BPF to push policydecisions into the kernel. But sometimes, it seems, what is really wantedis a way for a BPF program to punt a decision back to user space. That isthe objective behind this patch set givingthe securecomputing (seccomp) mechanism a way to pass complex decisions to a user-space helper program.

The Mozilla blog has anarticle describing the addition of DNS over HTTPS (DoH) as an optionalfeature in the Firefox browser. "DoH support has been added toFirefox 62 to improve the way Firefox interacts with DNS. DoH usesencrypted networking to obtain DNS information from a server that isconfigured within Firefox. This means that DNS requests sent to the DoHcloud server are encrypted while old style DNS requests are notprotected." The configured server is hosted by Cloudflare, whichhas posted thisprivacy agreement about the service.

Security updates have been issued by Debian (kernel, procps, and tiff), Fedora (ca-certificates, chromium, and git), Mageia (kernel, kernel-linus, kernel-tmb, and libvirt), openSUSE (chromium and xen), Oracle (procps, xmlrpc, and xmlrpc3), Red Hat (xmlrpc and xmlrpc3), Scientific Linux (procps, xmlrpc, and xmlrpc3), SUSE (HA kernel modules and kernel), and Ubuntu (libytnef and python-oslo.middleware).

The 4.17 kernel appears to be on track for a June 3 release, barring anunlikely last-minute surprise. So the time has come for the usual look atsome development statistics for this cycle. While 4.17 is a normal cyclefor the most part, it does have one characteristic of note: it is the thirdkernel release ever to be smaller (in terms of lines of code) than itspredecessor.

Security updates have been issued by CentOS (389-ds-base, corosync, firefox, java-1.7.0-openjdk, java-1.8.0-openjdk, kernel, librelp, libvirt, libvncserver, libvorbis, PackageKit, patch, pcs, and qemu-kvm), Fedora (asterisk, ca-certificates, gifsicle, ncurses, nodejs-base64-url, nodejs-mixin-deep, and wireshark), Mageia (thunderbird), Red Hat (procps), SUSE (curl, kvm, and libvirt), and Ubuntu (apport, haproxy, and tomcat7, tomcat8).

The LWN.net Weekly Edition for May 31, 2018 is available.

The4.14.47,4.9.105,4.4.135 and 3.18.112 stable updates have been released;each contains a single commit reverting a networking patch "thatshould not have gotten backported".

In February, the bpfilter mechanism wasfirst posted to the mailing lists. Bpfilter is meant to be a replacementfor the current in-kernel firewall/packet-filtering code. It provideslittle functionality itself; instead, it creates a set of hooks that canrun BPF programs to make the packet-filtering decisions. A version of that patch set has been mergedinto the net-next tree for 4.18. It will not be replacing any existingpacket filters in its current form, but it does feature a significantchange to one of its more controversial features: the new user-mode helpermechanism.

The advent of user namespaces and container technology has made it possibleto extend more root-like powers to unprivileged users in a (we hope) safeway. One remaining sticking point is the mounting of filesystems, whichhas long been fraught with security problems. Work has been proceeding toallow such mounts for years, and it has gotten a little closer with theposting of a patch series intended for the 4.18 kernel. But, as anunrelated discussion has made clear, truly safe unprivileged filesystemmounting is still a rather distant prospect — at least, if one wants to doit in the kernel.

Security updates have been issued by Arch Linux (strongswan, wireshark-cli, wireshark-common, wireshark-gtk, and wireshark-qt), CentOS (libvirt, procps-ng, and thunderbird), Debian (apache2, git, and qemu), Gentoo (beep, git, and procps), Mageia (mariadb, microcode, python, virtualbox, and webkit2), openSUSE (ceph, pdns, and perl-DBD-mysql), Red Hat (kernel), SUSE (HA kernel modules, libmikmod, ntp, and tiff), and Ubuntu (nvidia-graphics-drivers-384).

There is a set of larger-than-usual stable kernel updates available:4.16.13 (272 patches),4.14.45 (496 patches),4.9.104 (329 patches),4.4.134 (268 patches), and3.18.111 (185 patches). Needless to say,each one contains a fair number of important fixes. 4.14 users who wantperf to work will want to go directly to 4.14.46, which fixes a regression in 4.14.45.

Git versions v2.17.1, v2.13.7, v2.14.4, v2.15.2 and v2.16.4 have all beenreleased with fixes to a couple of security issues. The nastier of the two(CVE-2018-11235) enables arbitrary code execution controlled by a hostilerepository. See thisMicrosoft blog entry for more details — after updating.

Stratis is a new localstorage-management solution for Linux. It can be compared to ZFS, Btrfs, or LVM. Its focus is on simplicity of concepts and ease of use,while giving users access to advanced storage features. Internally,Stratis's implementation favors tight integration of existing components instead of the fully-integrated, in-kernel approach that ZFS andBtrfs use. This has benefits and drawbacks for Stratis, but also greatlydecreases the overall time needed to develop a useful and stable initialversion, which can then be a base for further improvement in laterversions. Subscribers can read on for an introduction to Stratis, by guestauthor (and Stratis team lead at Red Hat) Andy Grover.