LWN.net

<p>Finding a way for applications to do atomic writes to files, so that eitherthe old or new data is present after a crash and not a combination of thetwo, was the topic of a session led by Christoph Hellwig at the 2019 Linux Storage, Filesystem, andMemory-Management Summit (LSFMM).Application developers hate the fact that when they update files in place,a crash can leave them with old or new data—or sometimes a combination ofboth. He discussed some implementation ideasthat he has for atomic writes for XFS and wanted to see what the otherfilesystem developers thought about it.

Ted Ts'o led a discussion on storage testing and, in particular, on hisexperience getting blktests running for his testenvironment,in a combined storage and filesystem session at the 2019 Linux Storage,Filesystem, and Memory-Management Summit. He has been adding more testingto his automated test platform, including blktests, and he would like tosee more people running storage tests. The idea of his session was to seewhat could be done to help that cause.

Debian Linux and its family of derivatives (such as Ubuntu) are partlycharacterized by their use of .deb as the packaging format.Packages in this format are produced not only by the distributions themselves,but also by independent software vendors. The last major change of the formatinternals happened back in 1995. However, a discussion of possiblechanges has been brought up recently on the debian-devel mailing list by AdamBorowski.

Security updates have been issued by Arch Linux (firefox and thunderbird), Debian (sox and vcftools), Fedora (safelease and sharpziplib), openSUSE (chromium, evolution, graphviz, nmap, systemd, transfig, and ucode-intel), Red Hat (pacemaker), SUSE (curl, libvirt, openssl, php7, php72, and systemd), and Ubuntu (gnome-desktop3, keepalived, and samba).

<p>The stable tree was the topic for a plenary session led by Sasha Levin atthe 2019 Linux Storage, Filesystem, and Memory-Management Summit (LSFMM).One of the main areas that needs attention is testing, according to Levin.He wanted to discuss how to do more and better testing as well as toaddress any concerns that attendees might have with regard to the stable tree.

The physical memory in a computer system is a precious resource, so alot of effort has been put into managing it effectively. This task is mademore difficult by the complexity of the memory architecture on contemporarysystems. There are severallayers of abstraction that deal with the details of how physical memoryis laid out; one of those is simply called the "memory model". Thereare three models supported in the kernel, but one of them is on its wayout. As a way of understanding this change, this article willtake a closer look at the evolution of the kernel's memory models,their current state, and their possible future.

Security updates have been issued by Debian (curl, jackson-databind, minissdpd, php5, thunderbird, wireshark, and wpa), Fedora (curl, drupal7, firefox, kernel, libmediainfo, mediaconch, mediainfo, mod_http2, mupdf, rust, and singularity), openSUSE (containerd, docker, docker-runc, go, go1.11, go1.12, golang-github-docker-libnetwork), Oracle (firefox and libvirt), Scientific Linux (firefox and libvirt), and SUSE (bluez, curl, gnutls, java-1_7_1-ibm, libu2f-host, libvirt, python3, screen, and xen).

The second 5.2 kernel prepatch is out fortesting. "Fairly normal rc2, no real highlights - I thinkmost of the diff is the SPDX updates.Who am I kidding? The highlight of the week was clearly Finlandwinning the ice hockey world championships." The codename has beenchanged to "Golden Lions".

The latest set of stable kernel updates is5.1.5,5.0.19,4.19.46,4.14.122, and4.9.179.Each contains yet another set of important fixes.

Several new system calls have been proposed for addition to the kernel in anear-future release. A few of those, in particular, focus on memory-managementtasks. Read on for a look at process_vm_mmap() (for zero-copydata transfer between processes), and two new APIs for advising the kernelabout memory use in a different process.

Security updates have been issued by Debian (zookeeper), Fedora (kernel, singularity, and thunderbird), openSUSE (java-1_8_0-openjdk), Oracle (curl), Red Hat (firefox, libvirt, and virt:rhel), SUSE (php5, python-Jinja2, python-Pillow, and sysstat), and Ubuntu (MariaDB).

The linux-kernel mailing list has recently seen more than the usual amountof traffic proposing new system calls. LWN is endeavoring to catch up withthat stream, starting with a couple of proposals for the management of filedescriptors. pidfd_open() is a new way to create a "pidfd" filedescriptor that refers to a process in the system, whileclose_range() is an efficient way to close many open descriptorswith a single call.

Security updates have been issued by Debian (ffmpeg and firefox-esr), openSUSE (bzip2, chromium, and GraphicsMagick), Slackware (curl), SUSE (ucode-intel), and Ubuntu (curl and intel-microcode).

The LWN.net Weekly Edition for May 23, 2019 is available.

One thing that is known about using transparent huge pages (THPs) forfilesystems is that it is a hard problem to solve, but is there a solid firststep that could be taken toward that goal? That is the question Song Liu asked toopen his combined filesystem and memory-management session atthe 2019 Linux Storage, Filesystem, and Memory-Management Summit (LSFMM).His employer, Facebook, has a solid use case for using THPs on files inthe page cache, which may provide a starting point.

<p>Amir Goldstein has a use case for a feature that could be called a "lazyfile reflink", he said, though it might also be described as "VFS-levelsnapshots". He went through the use case, looking for suggestions, in asession at the 2019 Linux Storage, Filesystem, and Memory-Management Summit(LSFMM). He has already implemented parts of the solution, but would liketo get something upstream, which would mean shifting from thestacked-filesystem approach he has taken so far.

The openSUSE project has announcedthe release of openSUSE Leap 15.1. "Leap releases are scalable and both the desktop and server are equally important for professional’s workloads, which is reflected in the installation menu as well as the amount of packages Leap offers and hardware it supports. Leap is well suited and prepared for usage as a Virtual Machine (VM) or container guest, allowing professional users to efficiently run network services no matter whether it’s a single server or a data center."

Stable kernels 5.1.4, 5.0.18, 4.19.45, 4.14.121, and 4.9.178 have been released. They all containimportant fixes and users should upgrade.

Security updates have been issued by CentOS (ruby and wget), Debian (proftpd-dfsg), Fedora (firefox, mupdf, nss, and wavpack), openSUSE (evolution, GraphicsMagick, graphviz, libxslt, openssl-1_0_0, ovmf, and sqlite3), Red Hat (dotnet, python27-python and python27-python-jinja2, and rh-mariadb102-mariadb and rh-mariadb102-galera), Slackware (mozilla), SUSE (gnutls, java-1_7_1-ibm, and java-1_8_0-ibm), and Ubuntu (curl, firefox, php5, and webkit2gtk).

Version8.5 of the Tor Browser is out. "Tor Browser 8.5 is the firststable release for Android. Since we released the first alpha version inSeptember, we've been hard at work making sure we can provide theprotections users are already enjoying on desktop to the Androidplatform. Mobile browsing is increasing around the world, and in someparts, it is commonly the only way people access the internet. In thesesame areas, there is often heavy surveillance and censorship online, so wemade it a priority to reach these users."

The cost of fsync()is well known to filesystem developers, which is why there are efforts to providecheaper alternatives. Ric Wheeler wanted to discuss the longstanding idea ofadding an asynchronous version of fsync() in a filesystem sessionat the 2019 Linux Storage, Filesystem, and Memory-Management Summit(LSFMM). It turns out that what he wants may already be available via the new io_uring interface.

The Mozilla blog takesa look at the Firefox 67 release. "Today’s new Firefox releasecontinues to bring fast and private together right at the crossroads ofperformance and security. It includes improvements that continue to keepFirefox fast while giving you more control and assurance through newfeatures that your personal information is safe while you’re online withus." See the releasenotes for more information.

The "guarantees" that existing filesystems make with regard to persistencein the face of a system crash was the subject of a session led by AmirGoldstein at the 2019 Linux Storage, Filesystem, and Memory-ManagementSummit (LSFMM). The problem is that filesystem developers are not willingto make much in the way of guarantees unless applications call fsync()—somethingthat is not popular with application developers, who want a cheaper option.

The kernel mailing lists carry the sad newsthat longtime kernel contributor and subsystem maintainer MartinSchwidefsky has been killed in an accident. "Martin was the most significant contributor to the initial s390 portof the Linux Kernel and later the maintainer of the s390 architecturebackend. His technical expertise as well as his mentoring skills wereoutstanding. Martin was well known for his positive mindset and hiswillingness to help.He will be greatly missed."

The relationship between SUSE and the openSUSE community is currently underdiscussion as the community considers different options for how it wantsto be organized and governed in the future. Among the options underconsideration is the possibility of openSUSE setting up an entirely independent foundation, asit seeks greater autonomy and control over its own future and operations.

Security updates have been issued by Debian (drupal7 and jackson-databind), Fedora (checkstyle and gradle), openSUSE (qemu and xen), SUSE (ffmpeg, kvm, and ucode-intel), and Ubuntu (libraw and python-urllib3).

Damien Le Moal and Naohiro Aota led a combined storage and filesystemsession at the 2019 Linux Storage, Filesystem, and Memory-Management Summit(LSFMM) on filesystem work that has been done for zoned block devices.These devices have multiple zones with different characteristics; usuallythere are zones that can only be written in sequential order as well asconventional zones that can be written in random order. The genesis of zonedblock devices is shingledmagnetic recording (SMR) devices, which were created to increase thecapacity of hard disks, but at the cost of some flexibility.

In a combined filesystem and storage session at the 2019 Linux Storage,Filesystem, and Memory-Management Summit, Avri Altman wanted to discuss the"turbo-write" mode that is coming for UniversalFlash Storage (UFS) devices. He wanted to introduce this new featureto assembled developers and to get some opinions on how to support thismode in the kernel.

Version 3.4 of the Bison parser generator is out. "A particular focus was put on improving the diagnostics, which are nowcolored by default, and accurate with multibyte input. Their format wasalso changed, and is now similar to GCC 9's diagnostics."

By the time Linus Torvalds released the 5.2-rc1 kernelprepatch and closed the merge window for this development cycle, 12,064 non-merge changesetshad been pulled into the mainline repository — about 3,700 since our summary of the first "half" was written. Thus, aspredicted, the rate of change did slow during the latter part of the mergewindow. That does not mean that no significant changes have been merged,though; read on for a summary of what else has been merged for 5.2.

Security updates have been issued by Debian (cups-filters, dhcpcd5, faad2, ghostscript, graphicsmagick, jruby, lemonldap-ng, and libspring-security-2.0-java), Fedora (gnome-desktop3, java-1.8.0-openjdk-aarch32, libu2f-host, samba, sqlite, webkit2gtk3, xen, and ytnef), Mageia (docker, flash-player-plugin, freeradius, libsndfile, libxslt, mariadb, netpbm, python-jinja2, tomcat-native, and virtualbox), openSUSE (kernel and ucode-intel), and SUSE (kernel, kvm, libvirt, nmap, and transfig).

Wired looksat the security issues stemming from the complexity of the Bluetoothstandard. "Bluetooth has certainly been investigated to a degree, butresearchers say that the lack of intense scrutiny historically stems againfrom just how involved it is to even read the standard, much lessunderstand how it works and all the possible implementations. On the plusside, this has created a sort of security through obscurity, in whichattackers have also found it easier to develop attacks against otherprotocols and systems rather than taking the time to work out how to messwith Bluetooth."

Linus has released the 5.2-rc1 kernelprepatch and closed the merge window for this development cycle."Nothing particularly odd going on this merge window. I had sometravel in the middle of it, but to offset that I had a new fastertest-build setup, and most of the pull requests came in early (thank you)so my travels didn't actually end up affecting the merge window all thatmuch."

The ever-increasing complexity of the software stacks we work with has giventesting an important role. There was a recent intersection between theautomated testing being done by the Yocto Project (YP) and a bug introduced into the Linux kernel that gives some insight into what thefuture holds and the potential available with this kind of testing.

Six new stable kernels have been released: 5.1.3, 5.0.17,4.19.44, 4.14.120, 4.9.177, and 4.4.180. As usual, they contain importantfixes throughout the kernel tree; users should upgrade.

Security updates have been issued by Debian (jquery), Fedora (kernel-headers, php-typo3-phar-stream-wrapper, and python3), openSUSE (qemu, ucode-intel, and xen), Red Hat (chromium-browser, java-1.8.0-ibm, and rh-python35-python-jinja2), SUSE (containerd, docker, docker-runc, go, go1.11, go1.12, golang-github-docker-libnetwork, evolution, graphviz, kernel, qemu, and systemd), and Ubuntu (libmediainfo, libvirt, and Wireshark).

Over the past four years, LWN has covered the PythonLanguage Summit, but this year the Python Software Foundation (PSF) electedto go in a different direction, with coverage by A. Jesse Jiryu Davis onthe PSF blog. Those reports are being gathered on a summitpage; as of this writing there are two reports up with plenty more tocome. "The Python Language Summit is a small gathering of Pythonlanguage implementers, both the core developers of CPython and alternativePythons, held on the first day of PyCon. The summit features shortpresentations from Python developers and community members, followed bylonger discussions. The 2019 summit is the first held since Guido vanRossum stepped down as Benevolent Dictator for Life, replaced by afive-member Steering Council."

Even with radiators and fans, a system's CPUs can overheat. When thathappens, the kernel's thermal governor will cap the maximum frequency ofthat CPU to allow it to cool. The scheduler, however, is not aware that the CPU'scapacity has changed; it may schedule more work than optimal in the currentconditions, leading to a performance degradation. Recently, TharaGopinath didsome research and posted a patch set to address this problem. Thesolution adds an interface to inform the scheduler about thermal events sothat it can assign tasks better and thus improve the overall systemperformance.

Greg Kroah-Hartman has announced the release of the 3.18.140 stable kernel. "Note, this is the LAST 3.18.y release that I will be doing onkernel.org. I know it has been marked as End-of-Life for quite sometime, but I have kept it alive due to a few million phones out there inthe wild that depend on it, and can not move to a new kernel base due tothem being stuck with a SoC vendor that does not work upstream.But, this does not mean the tree is dead, oh no, if only it were thateasy..." He and others will be updating the kernel in the AndroidOpen Source Project (AOSP) tree.

Security updates have been issued by CentOS (freeradius, kernel, libvirt, and qemu-kvm), Debian (intel-microcode, linux-4.9, and samba), Fedora (kernel, kernel-headers, memcached, microcode_ctl, php-pecl-imagick, and samba), Mageia (kernel, kernel-linus, kernel-tmb, and microcode), openSUSE (389-ds, bzip2, jakarta-commons-fileupload, kernel, and pacemaker), Red Hat (flash-plugin and ruby), Scientific Linux (kernel, libvirt, qemu-kvm, and ruby), Slackware (rdesktop), and Ubuntu (libvirt).

The LWN.net Weekly Edition for May 16, 2019 is available.

Over the past year, Python has moved on from the benevolent dictator forlife (BDFL) governance model since Guido van Rossum steppeddown from that role. In February, a new steering council was elected based onthe governance model that was adopted inDecember. At PyCon 2019 inCleveland, Ohio, the five members of the steering council took the stage for akeynote panel that was moderated by Python Software Foundation (PSF)executive director Ewa Jodlowska.

We contemplated putting together an LWN article on the "microarchitecturaldata sampling" (MDS) vulnerabilities, as we've done for pastspeculative-execution issues. But the truth of the matter is that it'sreally more of the same, and there is a lot of material out there on thenet already. So, for those who would like to learn more, here's a list ofresources.

<p>Amir Goldstein led a discussion on things that the two major networkfilesystems for Linux, Samba and NFS, could cooperate on at the end of dayone of the 2019 Linux Storage, Filesystem, and Memory-Management Summit. Inparticular, are there needs that both filesystems have that the kernel isnot currently providing? He had some ideas of areas that might be tackled,but was looking for feedback from the assembled filesystem developers.

Michael Crosby is one of the most influential developers working on Dockercontainers today, helping to lead development of containerd as well as serving as the Open Container Initiative (OCI)Technical Oversight Chair. At DockerCon 19, Crosby led astanding-room-only session, outlining the past, present and — moreimportantly — the future of Docker as a container technology. The earlyhistory of Docker is closely tied with Linux and, as it turns out, so too isDocker's future.

Security updates have been issued by Debian (drupal7, intel-microcode, kernel, and lemonldap-ng), Red Hat (kernel, kernel-rt, libvirt, qemu-kvm, qemu-kvm-rhev, redhat-virtualization-host, rhvm-appliance, vdsm, virt:rhel, and wget), Scientific Linux (wget), SUSE (containerd, docker, docker-runc, go, go1.11, go1.12, golang-github-docker-libnetwork, kernel, libxslt, microcode_ctl, qemu, ucode-intel, and xen), and Ubuntu (intel-microcode, kernel, linux, linux-aws, linux-azure, linux-gcp, linux-kvm, linux-raspi2, linux, linux-aws, linux-kvm, linux-raspi2, linux-snapdragon, linux-hwe, linux-azure, linux-hwe, linux-azure, linux-gcp, linux-oracle, linux-lts-trusty, linux-lts-xenial, linux-raspi2, linux-snapdragon, qemu, and samba).

Here's a blog post from"Brent" on how PHP deserves another look. "Today I want to look at the bright side: let's focus on the things that have changed and ways to write clean and maintainable PHP code. I want to ask you to set aside any prejudice for just a few minutes.Afterwards you're free to think exactly the same about PHP as you did before. Though chances are you will be surprised by some of the improvements made to PHP in the last few years."

A new filesystem aimed at sharing host filesystems with KVM guests, virtio-fs, was the topic of a session led by Miklos Szeredi at the 2019 Linux Storage,Filesystem, and Memory-Management Summit. The existing solution, which isbased on the 9P filesystem from Plan 9, has someshortcomings, he said. Virtio-fs is a prototype that uses the Filesystem inUserspace (FUSE) interface.

This round of kernel updates address a speculative-execution vulnerability foundin all Intel processors made since 2011. Greg Kroah-Hartman says in the 5.1.2 kernel patch: "Note, this release,and the other stable releases that are all being released right now at thesame time, just went out all contain patches that have only seen the"public eye" for about 5 minutes. So be forwarned, they might breakthings, they might not build, but hopefully they fix things. Odds are wewill be fixing a number of small things in this area for the next few weeksas things shake out on real hardware and workloads." In addition to5.1.2, stable kernels 5.0.16, 4.19.43, 4.14.119, and 4.9.176 are available. More information may befound in the Xensecurity advisory and this new in-kerneldocumentation.

The curtain has finally been lifted on the latest set ofspeculative-execution vulnerabilities. This one has the delightful name ofZombieLoad; it is also known as"microarchitectural data sampling", but what's the fun in that? Various x86processors stash data into hidden buffers that can, in some cases, berevealed via speculative execution. Exploits appear to be relativelyhard. See this pagefrom the kernel documentation for a fairly detailed description of theproblem, and thispage for mitigation information.