LWN.net

A kernel debugger that allows Python scripts to access data structures ina running kernel was the topic of Omar Sandoval's plenary session at the2019 Linux Storage, Filesystem, and Memory-Management Summit (LSFMM). Inhis day job at Facebook, Sandoval does a fair amount of kernel debuggingand he found the existing tools to be lacking. That led him to build drgn, which is a debugger builtinto a Python library.

In a followup to his earlier session on dyingcontrol groups, Roman Gushchin wanted to talk about problems with theshrinkers and filesystem caches in a combined filesystem andmemory-management session at the 2019 Linux Storage, Filesystem, and Memory-Management Summit (LSFMM).Specifically, for control groups that share the same underlying filesystem,the shrinkers are not able to reclaim memory from the VFS caches after acontrol group dies, at least under slight to moderate memory pressure. Hewanted to discuss how to reclaim that memory without major performanceimpacts.

Version 1.0 of the GParted GNOME Partition Editor has been released. "The GParted 1.0.0 release includes a significant undertaking to migratethe code base from gtkmm2 to gtkmm3 (our GTK3 port)."

Version 4.2.0of the Krita paint tool is out. "New in Krita 4.2.0 is updatedsupport for drawing tablets, support for HDR monitors on Windows, animproved color palette docker, scripting API for animation, color gamutmasking, improved selection handling, much nicer handling of theinteraction between opacity and flow and much, much, much more" Seethe releasenotes for more details.

Kees Cook reviewsthe security-related enhancements in the 5.1 kernel release."Now /proc/$pid can be opened and used as an argument for sendingsignals with the new pidfd_send_signal() syscall. This handle will onlyrefer to the original process at the time the open() happened, and not toany later 'reused' pid if the process dies and a new process is assignedthe same pid. Using this method, it’s now possible to racelessly sendsignals to exactly the intended process without having to worry about pidreuse. (BTW, this commit wins the 2019 award for Most Well DocumentedCommit Log Justification.)"

<p>In a filesystem session on the third day of the 2019 Linux Storage,Filesystem, and Memory-Management Summit (LSFMM), Steve French wanted totalk about copy operations. Much of the development work that has gone onin the Linux filesystem world over the last few years has been related tothe performance ofcopying files, at least indirectly, he said. There are still painpoints around copy operations, however, so he would like to see those getaddressed.

Security updates have been issued by Arch Linux (webkit2gtk), Debian (kernel and libav), Fedora (c3p0 and community-mysql), Scientific Linux (pacemaker), SUSE (axis, libtasn1, NetworkManager, sles12sp3-docker-image, sles12sp4-image, system-user-root, and xen), and Ubuntu (freerdp, GNU Screen, keepalived, and thunderbird).

<p>Finding a way for applications to do atomic writes to files, so that eitherthe old or new data is present after a crash and not a combination of thetwo, was the topic of a session led by Christoph Hellwig at the 2019 Linux Storage, Filesystem, andMemory-Management Summit (LSFMM).Application developers hate the fact that when they update files in place,a crash can leave them with old or new data—or sometimes a combination ofboth. He discussed some implementation ideasthat he has for atomic writes for XFS and wanted to see what the otherfilesystem developers thought about it.

Ted Ts'o led a discussion on storage testing and, in particular, on hisexperience getting blktests running for his testenvironment,in a combined storage and filesystem session at the 2019 Linux Storage,Filesystem, and Memory-Management Summit. He has been adding more testingto his automated test platform, including blktests, and he would like tosee more people running storage tests. The idea of his session was to seewhat could be done to help that cause.

Debian Linux and its family of derivatives (such as Ubuntu) are partlycharacterized by their use of .deb as the packaging format.Packages in this format are produced not only by the distributions themselves,but also by independent software vendors. The last major change of the formatinternals happened back in 1995. However, a discussion of possiblechanges has been brought up recently on the debian-devel mailing list by AdamBorowski.

Security updates have been issued by Arch Linux (firefox and thunderbird), Debian (sox and vcftools), Fedora (safelease and sharpziplib), openSUSE (chromium, evolution, graphviz, nmap, systemd, transfig, and ucode-intel), Red Hat (pacemaker), SUSE (curl, libvirt, openssl, php7, php72, and systemd), and Ubuntu (gnome-desktop3, keepalived, and samba).

<p>The stable tree was the topic for a plenary session led by Sasha Levin atthe 2019 Linux Storage, Filesystem, and Memory-Management Summit (LSFMM).One of the main areas that needs attention is testing, according to Levin.He wanted to discuss how to do more and better testing as well as toaddress any concerns that attendees might have with regard to the stable tree.

The physical memory in a computer system is a precious resource, so alot of effort has been put into managing it effectively. This task is mademore difficult by the complexity of the memory architecture on contemporarysystems. There are severallayers of abstraction that deal with the details of how physical memoryis laid out; one of those is simply called the "memory model". Thereare three models supported in the kernel, but one of them is on its wayout. As a way of understanding this change, this article willtake a closer look at the evolution of the kernel's memory models,their current state, and their possible future.

Security updates have been issued by Debian (curl, jackson-databind, minissdpd, php5, thunderbird, wireshark, and wpa), Fedora (curl, drupal7, firefox, kernel, libmediainfo, mediaconch, mediainfo, mod_http2, mupdf, rust, and singularity), openSUSE (containerd, docker, docker-runc, go, go1.11, go1.12, golang-github-docker-libnetwork), Oracle (firefox and libvirt), Scientific Linux (firefox and libvirt), and SUSE (bluez, curl, gnutls, java-1_7_1-ibm, libu2f-host, libvirt, python3, screen, and xen).

The second 5.2 kernel prepatch is out fortesting. "Fairly normal rc2, no real highlights - I thinkmost of the diff is the SPDX updates.Who am I kidding? The highlight of the week was clearly Finlandwinning the ice hockey world championships." The codename has beenchanged to "Golden Lions".

The latest set of stable kernel updates is5.1.5,5.0.19,4.19.46,4.14.122, and4.9.179.Each contains yet another set of important fixes.

Several new system calls have been proposed for addition to the kernel in anear-future release. A few of those, in particular, focus on memory-managementtasks. Read on for a look at process_vm_mmap() (for zero-copydata transfer between processes), and two new APIs for advising the kernelabout memory use in a different process.

Security updates have been issued by Debian (zookeeper), Fedora (kernel, singularity, and thunderbird), openSUSE (java-1_8_0-openjdk), Oracle (curl), Red Hat (firefox, libvirt, and virt:rhel), SUSE (php5, python-Jinja2, python-Pillow, and sysstat), and Ubuntu (MariaDB).

The linux-kernel mailing list has recently seen more than the usual amountof traffic proposing new system calls. LWN is endeavoring to catch up withthat stream, starting with a couple of proposals for the management of filedescriptors. pidfd_open() is a new way to create a "pidfd" filedescriptor that refers to a process in the system, whileclose_range() is an efficient way to close many open descriptorswith a single call.

Security updates have been issued by Debian (ffmpeg and firefox-esr), openSUSE (bzip2, chromium, and GraphicsMagick), Slackware (curl), SUSE (ucode-intel), and Ubuntu (curl and intel-microcode).

The LWN.net Weekly Edition for May 23, 2019 is available.

One thing that is known about using transparent huge pages (THPs) forfilesystems is that it is a hard problem to solve, but is there a solid firststep that could be taken toward that goal? That is the question Song Liu asked toopen his combined filesystem and memory-management session atthe 2019 Linux Storage, Filesystem, and Memory-Management Summit (LSFMM).His employer, Facebook, has a solid use case for using THPs on files inthe page cache, which may provide a starting point.

<p>Amir Goldstein has a use case for a feature that could be called a "lazyfile reflink", he said, though it might also be described as "VFS-levelsnapshots". He went through the use case, looking for suggestions, in asession at the 2019 Linux Storage, Filesystem, and Memory-Management Summit(LSFMM). He has already implemented parts of the solution, but would liketo get something upstream, which would mean shifting from thestacked-filesystem approach he has taken so far.

The openSUSE project has announcedthe release of openSUSE Leap 15.1. "Leap releases are scalable and both the desktop and server are equally important for professional’s workloads, which is reflected in the installation menu as well as the amount of packages Leap offers and hardware it supports. Leap is well suited and prepared for usage as a Virtual Machine (VM) or container guest, allowing professional users to efficiently run network services no matter whether it’s a single server or a data center."

Stable kernels 5.1.4, 5.0.18, 4.19.45, 4.14.121, and 4.9.178 have been released. They all containimportant fixes and users should upgrade.

Security updates have been issued by CentOS (ruby and wget), Debian (proftpd-dfsg), Fedora (firefox, mupdf, nss, and wavpack), openSUSE (evolution, GraphicsMagick, graphviz, libxslt, openssl-1_0_0, ovmf, and sqlite3), Red Hat (dotnet, python27-python and python27-python-jinja2, and rh-mariadb102-mariadb and rh-mariadb102-galera), Slackware (mozilla), SUSE (gnutls, java-1_7_1-ibm, and java-1_8_0-ibm), and Ubuntu (curl, firefox, php5, and webkit2gtk).

Version8.5 of the Tor Browser is out. "Tor Browser 8.5 is the firststable release for Android. Since we released the first alpha version inSeptember, we've been hard at work making sure we can provide theprotections users are already enjoying on desktop to the Androidplatform. Mobile browsing is increasing around the world, and in someparts, it is commonly the only way people access the internet. In thesesame areas, there is often heavy surveillance and censorship online, so wemade it a priority to reach these users."

The cost of fsync()is well known to filesystem developers, which is why there are efforts to providecheaper alternatives. Ric Wheeler wanted to discuss the longstanding idea ofadding an asynchronous version of fsync() in a filesystem sessionat the 2019 Linux Storage, Filesystem, and Memory-Management Summit(LSFMM). It turns out that what he wants may already be available via the new io_uring interface.

The Mozilla blog takesa look at the Firefox 67 release. "Today’s new Firefox releasecontinues to bring fast and private together right at the crossroads ofperformance and security. It includes improvements that continue to keepFirefox fast while giving you more control and assurance through newfeatures that your personal information is safe while you’re online withus." See the releasenotes for more information.

The "guarantees" that existing filesystems make with regard to persistencein the face of a system crash was the subject of a session led by AmirGoldstein at the 2019 Linux Storage, Filesystem, and Memory-ManagementSummit (LSFMM). The problem is that filesystem developers are not willingto make much in the way of guarantees unless applications call fsync()—somethingthat is not popular with application developers, who want a cheaper option.

The kernel mailing lists carry the sad newsthat longtime kernel contributor and subsystem maintainer MartinSchwidefsky has been killed in an accident. "Martin was the most significant contributor to the initial s390 portof the Linux Kernel and later the maintainer of the s390 architecturebackend. His technical expertise as well as his mentoring skills wereoutstanding. Martin was well known for his positive mindset and hiswillingness to help.He will be greatly missed."

The relationship between SUSE and the openSUSE community is currently underdiscussion as the community considers different options for how it wantsto be organized and governed in the future. Among the options underconsideration is the possibility of openSUSE setting up an entirely independent foundation, asit seeks greater autonomy and control over its own future and operations.

Security updates have been issued by Debian (drupal7 and jackson-databind), Fedora (checkstyle and gradle), openSUSE (qemu and xen), SUSE (ffmpeg, kvm, and ucode-intel), and Ubuntu (libraw and python-urllib3).

Damien Le Moal and Naohiro Aota led a combined storage and filesystemsession at the 2019 Linux Storage, Filesystem, and Memory-Management Summit(LSFMM) on filesystem work that has been done for zoned block devices.These devices have multiple zones with different characteristics; usuallythere are zones that can only be written in sequential order as well asconventional zones that can be written in random order. The genesis of zonedblock devices is shingledmagnetic recording (SMR) devices, which were created to increase thecapacity of hard disks, but at the cost of some flexibility.

In a combined filesystem and storage session at the 2019 Linux Storage,Filesystem, and Memory-Management Summit, Avri Altman wanted to discuss the"turbo-write" mode that is coming for UniversalFlash Storage (UFS) devices. He wanted to introduce this new featureto assembled developers and to get some opinions on how to support thismode in the kernel.

Version 3.4 of the Bison parser generator is out. "A particular focus was put on improving the diagnostics, which are nowcolored by default, and accurate with multibyte input. Their format wasalso changed, and is now similar to GCC 9's diagnostics."

By the time Linus Torvalds released the 5.2-rc1 kernelprepatch and closed the merge window for this development cycle, 12,064 non-merge changesetshad been pulled into the mainline repository — about 3,700 since our summary of the first "half" was written. Thus, aspredicted, the rate of change did slow during the latter part of the mergewindow. That does not mean that no significant changes have been merged,though; read on for a summary of what else has been merged for 5.2.

Security updates have been issued by Debian (cups-filters, dhcpcd5, faad2, ghostscript, graphicsmagick, jruby, lemonldap-ng, and libspring-security-2.0-java), Fedora (gnome-desktop3, java-1.8.0-openjdk-aarch32, libu2f-host, samba, sqlite, webkit2gtk3, xen, and ytnef), Mageia (docker, flash-player-plugin, freeradius, libsndfile, libxslt, mariadb, netpbm, python-jinja2, tomcat-native, and virtualbox), openSUSE (kernel and ucode-intel), and SUSE (kernel, kvm, libvirt, nmap, and transfig).

Wired looksat the security issues stemming from the complexity of the Bluetoothstandard. "Bluetooth has certainly been investigated to a degree, butresearchers say that the lack of intense scrutiny historically stems againfrom just how involved it is to even read the standard, much lessunderstand how it works and all the possible implementations. On the plusside, this has created a sort of security through obscurity, in whichattackers have also found it easier to develop attacks against otherprotocols and systems rather than taking the time to work out how to messwith Bluetooth."

Linus has released the 5.2-rc1 kernelprepatch and closed the merge window for this development cycle."Nothing particularly odd going on this merge window. I had sometravel in the middle of it, but to offset that I had a new fastertest-build setup, and most of the pull requests came in early (thank you)so my travels didn't actually end up affecting the merge window all thatmuch."

The ever-increasing complexity of the software stacks we work with has giventesting an important role. There was a recent intersection between theautomated testing being done by the Yocto Project (YP) and a bug introduced into the Linux kernel that gives some insight into what thefuture holds and the potential available with this kind of testing.

Six new stable kernels have been released: 5.1.3, 5.0.17,4.19.44, 4.14.120, 4.9.177, and 4.4.180. As usual, they contain importantfixes throughout the kernel tree; users should upgrade.

Security updates have been issued by Debian (jquery), Fedora (kernel-headers, php-typo3-phar-stream-wrapper, and python3), openSUSE (qemu, ucode-intel, and xen), Red Hat (chromium-browser, java-1.8.0-ibm, and rh-python35-python-jinja2), SUSE (containerd, docker, docker-runc, go, go1.11, go1.12, golang-github-docker-libnetwork, evolution, graphviz, kernel, qemu, and systemd), and Ubuntu (libmediainfo, libvirt, and Wireshark).

Over the past four years, LWN has covered the PythonLanguage Summit, but this year the Python Software Foundation (PSF) electedto go in a different direction, with coverage by A. Jesse Jiryu Davis onthe PSF blog. Those reports are being gathered on a summitpage; as of this writing there are two reports up with plenty more tocome. "The Python Language Summit is a small gathering of Pythonlanguage implementers, both the core developers of CPython and alternativePythons, held on the first day of PyCon. The summit features shortpresentations from Python developers and community members, followed bylonger discussions. The 2019 summit is the first held since Guido vanRossum stepped down as Benevolent Dictator for Life, replaced by afive-member Steering Council."

Even with radiators and fans, a system's CPUs can overheat. When thathappens, the kernel's thermal governor will cap the maximum frequency ofthat CPU to allow it to cool. The scheduler, however, is not aware that the CPU'scapacity has changed; it may schedule more work than optimal in the currentconditions, leading to a performance degradation. Recently, TharaGopinath didsome research and posted a patch set to address this problem. Thesolution adds an interface to inform the scheduler about thermal events sothat it can assign tasks better and thus improve the overall systemperformance.

Greg Kroah-Hartman has announced the release of the 3.18.140 stable kernel. "Note, this is the LAST 3.18.y release that I will be doing onkernel.org. I know it has been marked as End-of-Life for quite sometime, but I have kept it alive due to a few million phones out there inthe wild that depend on it, and can not move to a new kernel base due tothem being stuck with a SoC vendor that does not work upstream.But, this does not mean the tree is dead, oh no, if only it were thateasy..." He and others will be updating the kernel in the AndroidOpen Source Project (AOSP) tree.

Security updates have been issued by CentOS (freeradius, kernel, libvirt, and qemu-kvm), Debian (intel-microcode, linux-4.9, and samba), Fedora (kernel, kernel-headers, memcached, microcode_ctl, php-pecl-imagick, and samba), Mageia (kernel, kernel-linus, kernel-tmb, and microcode), openSUSE (389-ds, bzip2, jakarta-commons-fileupload, kernel, and pacemaker), Red Hat (flash-plugin and ruby), Scientific Linux (kernel, libvirt, qemu-kvm, and ruby), Slackware (rdesktop), and Ubuntu (libvirt).

The LWN.net Weekly Edition for May 16, 2019 is available.

Over the past year, Python has moved on from the benevolent dictator forlife (BDFL) governance model since Guido van Rossum steppeddown from that role. In February, a new steering council was elected based onthe governance model that was adopted inDecember. At PyCon 2019 inCleveland, Ohio, the five members of the steering council took the stage for akeynote panel that was moderated by Python Software Foundation (PSF)executive director Ewa Jodlowska.

We contemplated putting together an LWN article on the "microarchitecturaldata sampling" (MDS) vulnerabilities, as we've done for pastspeculative-execution issues. But the truth of the matter is that it'sreally more of the same, and there is a lot of material out there on thenet already. So, for those who would like to learn more, here's a list ofresources.