Linus Torvalds has released the 6.5-rc7 kernelprepatch, which looks to be the final release candidate before the likelyrelease of Linux 6.5 next Sunday. Torvalds released it a little earlierthan usual due to some travel; overall things look to be in good shape:
It is fair to say that the DNF packagemanager is not the favorite tool of many Fedora users. It was broughtin as a replacement for Yum but got off to arather rocky start; DNF hasstabilized over the years, though and the complaints have subsided. That can onlymean one thing: it must be time to throw it away and start over from thebeginning. The replacement, called DNF5, was slated to be a part of theFedora39 release, due in October, but that is not going to happen.
SUSE's long story of corporate ownership is gaining a new chapter; thecompany has announcedthat its majority shareholder (Marcel LUX III SARL) will be acquiring theremaining shares, and will take the company private and off of the stockexchange. "SUSE's Management Board and Supervisory Board support thestrategic opportunity from delisting of the company as it will allow SUSEto focus fully on its operational priorities and execution of its long-termstrategy."
In its default configuration, the Linux kernel will allow processes toallocate more memory than the system can actually provide; this policyenables better utilization of physical memory and works just fine - most ofthe time. On occasions, though, the kernel may find itself unable toprovide memory that processes may think already belongs to them. If thesituation gets bad enough, the only solution (short of rebooting) is todeclare a sort of memory bankruptcy and write off some of the kernel'sdebts by killing one or more processes. Over the years, a great deal ofeffort has gone into heuristics to select the processes that the user isleast likely to miss. This problem is still clearly not solved toeverybody's satisfaction, though, so it was only a matter of time beforesomebody introduced a way to select the out-of-memory (OOM) victim usingBPF.
Security updates have been issued by Debian (open-vm-tools, openjdk-11, and openssh), Fedora (librsvg2, llhttp, opensc, and rust), Oracle (.NET 6.0, .NET 7.0, iperf3, microcode_ctl, postgresql:10, and python-requests), SUSE (openssl-1_0_0, perl-Cpanel-JSON-XS, postgresql12, and postgresql15), and Ubuntu (ceph, haproxy, heat, libpod, and postgresql-12, postgresql-14, postgresql-15).
Readers have been pointing us to HashiCorp's announcementthat it is moving to its own "Business Source License" for some of its(formerly) open-source products. Like other companies (example) that have taken this path, HashiCorpis removing the freedom to use its products commercially in ways that itsees as competitive. This is, in a real sense, an old and tiresome story.The lessons to be drawn from this change are old as well. One is to bewareof depending on any platform, free or proprietary, that is controlled by asingle company. It is a rare company that will not try to take advantageof that control at some point.The other is to beware of contributor license agreements. HashiCorp'sagreement usedto read that it existed "to ensure that our projects remain licensedunder Free and Open Source licenses"; the current version doesn't say thatanymore. But both versions give HashiCorp the right to play exactly thiskind of game with any code contributed by outsiders. Developers who werecontributing to a free-software project will now have their code used in arather more proprietary setting. When a company is given the right to takesomebody else's code proprietary, many of them will eventually make use ofthat right.
The call for topics for the LinuxKernelMaintainers Summit went out on August15; one proposed topic hasgenerated some interesting discussion about security-bug reporting for thekernel. A recent patchto the kernel's documentation about how to report security bugs recommendsavoiding posting to the linux-distrosmailing list because its goals and rules do not mesh well with kernelsecurity practices. That led Jiri Kosina to suggesta discussion on security reporting, especially with regard to Linuxdistributions.
Security updates have been issued by Debian (datatables.js and openssl), Fedora (ghostscript, java-11-openjdk, java-latest-openjdk, microcode_ctl, and xen), Red Hat (redhat-ds:11), SUSE (java-1_8_0-openj9, kernel, krb5, pcre2, and perl-HTTP-Tiny), and Ubuntu (gstreamer1.0, mysql-8.0, tiff, and webkit2gtk).
"Subinterpreters", which are separate Python interpreters running in thesame process that can becreated usingthe C API, have been a part of Python since the previous century(version1.5 in1997), but they are largely unknown and unused.Eric Snow has been on something of a quest, since 2015 or so, to bring better multicore processing to Python byway of subinterpreters (or "multiple interpreters"). He has made it partof the way there, with the adoption of a separate global interpreter lock (GIL) for eachsubinterpreter, whichwas added for Python3.12. Back in April, Snow gave a talk (YouTube video) atPyCon about multiple interpreters, their status, and his plans for thefeature in the future.
Version5.0 ("Daedalus") of the Debian-based Devuan distribution has beenreleased. "This is the result of many months of painstaking work by theTeam and detailed testing by the wider Devuan community." Theannouncement lists a couple of new features but mostly defers to theDebian12 ("bookworm") release notes.
Security updates have been issued by Debian (samba), Red Hat (.NET 6.0, .NET 7.0, rh-dotnet60-dotnet, rust, rust-toolset-1.66-rust, and rust-toolset:rhel8), and SUSE (kernel and opensuse-welcome).
The Linux fast user-space mutex ("futex") subsystem debuted with the 2.6.0kernel; it provides a mechanism that can be used to implement user-spacelocking. Since futexes avoid calling into the kernel whenever possible,they can indeed be fast, especially in the uncontended case. The API usedto access futexes has never been seen as one of Linux's strongest points,though, so there has long been a desire to improve it. This patchseries from Peter Zijlstra shows what the future of futexes may looklike.
LWN recently covered a discussion onfile-position locking that demonstrated the hazards that can resultfrom unexpected concurrency. It turns out that this discussion had not yetfully run its course. Since that article was written, additional changesintended to address a performance regression evolved into a core virtualfilesystem (VFS) layer API change to carry out some much-delayed housecleaning.
Greg Kroah-Hartman has announced the release of the6.4.10, 6.1.45, 5.10.190, 5.4.253, 4.19.291, and 4.14.322 stable kernels. Note that 5.15.126was also inthe review process for this batch, but has not (yet) been released. Meanwhile, the rest of the batch all have important fixes throughoutthe kernel tree, as usual.Update: the 5.15.126 announcementhas now gone out as well.
The Open Enterprise Linux Associationhas announced itsexistence. It is a collaboration between CIQ (Rocky Linux), Oracle,and SUSE to provide an RHEL-compatible distribution.
It is the kernel's business to know when a process's memory has beenwritten to; among other things, this knowledge is needed to determine whichpages can be immediately reclaimed or to properly write dirty pages to backing store.Sometimes, though, user space also needs access to this information in areliable and fast manner. Thispatch series from Muhammad Usama Anjum adds a new ioctl() callfor this purpose; using it requires repurposing an existing system call inan unusual way, though.
OpenSSH 9.4 has been released. Changes this time include the ability toforward Unix-domain sockets, a tags mechanism for more flexibleconfiguration, and more.
The global interpreter lock (GIL) has been a part of CPython since thebeginning-nearly-butthat seems likely to change over the next five or so years. As we described last week, thePython steering council has announcedits intention to start moving toward a no-GILCPython, potentially as soon as Python3.13 in October2024for the preliminaries. The no-GIL version of CPython comes from SamGross, who introducedit as a proof-of-concept nearly two yearsago; now, the idea has been formalized in a Python Enhancement Proposal(PEP) that describes no-GIL mode and how it interacts with the rest of thePython ecosystem.
Getting a stack trace of a running program is useful in a variety ofscenarios: tracing, profiling, debugging, performance tuning, and more.There are existing mechanisms to get stack traces, but there are somedownsides to them; the "Simple Frame" (SFrame) stack-trace format cameabout to address the shortcomings in the other techniques. Back in May,Steve Rostedt and Indu Bhagat gave a talk aboutSFrame support in the kernel as part of LSFMM+BPF; a few days later, Bhagat gavea more general talk about SFrame (YouTube video)at OpenSource Summit North America in Vancouver. That second talk helped fillin some other aspects of SFrame and the overall stack-tracing picture.
The6.4.9,6.1.44,5.15.125,5.10.189,5.4.252,4.19.290, and4.14.321stable kernel updates have all been released; they are dominated by fixesfor the latest round ofspeculative-execution vulnerabilities.Do note the warning attached to each of these releases:
Security updates have been issued by Debian (libhtmlcleaner-java and thunderbird), Red Hat (dbus, kernel, kernel-rt, kpatch-patch, and thunderbird), Scientific Linux (thunderbird), SUSE (chromium, gstreamer-plugins-bad, gstreamer-plugins-base, gstreamer-plugins-good, gstreamer-plugins-ugly, kernel-firmware, libqt5-qtbase, libqt5-qtsvg, librsvg, pcre2, perl-Net-Netmask, qt6-base, and thunderbird), and Ubuntu (firefox).
The Linux Containers project hasannounced the addition ofIncus, which is a fork of LXD5.16 started by Aleksa Sarai. Incus was created in response to Canonical's removal of LXD from LinuxContainers.
Return-orientedprogramming (ROP) has, for some years now, been a valuable tool forthose who would subvert a system's security. It is thus not surprisingthat a lot of effort has gone into thwarting ROP attacks, which depend oncorrupting the call stack with a carefully chosen set of return addresses,at both the hardware and software levels. One result of this work isshadow stacks, which can detect corruption of the call stack, allowing theoperating system to react accordingly. The 64-bit Arm implementation ofshadow stacks is called "guarded control stack" (GCS); patches implementingsupport for this feature are currently under discussion.
Security updates have been issued by Debian (burp, chromium, ghostscript, openimageio, pdfcrack, python-werkzeug, thunderbird, and webkit2gtk), Fedora (amanda, libopenmpt, llhttp, samba, seamonkey, and xen), Red Hat (thunderbird), Slackware (mozilla and samba), and SUSE (perl-Net-Netmask, python-Django1, trytond, and virtualbox).
Bram Moolenaar, the creator of the vim editor, passedaway on August3. "Bram dedicated a large part of his life toVIM and he was very proud of the VIM community that you are all partof." He will be missed.
The big kernel lock (BKL) is a distant memory now but, for years, it wasone of the more intractable problems faced by the kernel developmentcommunity. The end of the BKL does not mean that the kernel is withoutproblematic locks, however. In recent times, some attention has been paidto the software-interrupt (or "bottom half") lock, which can create latencyproblems, especially on realtime systems. Frederic Weisbecker is taking anew tack in his campaign to cut this lock down to size, with an approachbased on how the BKL was eventually removed.
Security updates have been issued by CentOS (bind and kernel), Debian (cjose, firefox-esr, ntpsec, and python-django), Fedora (chromium, firefox, librsvg2, and webkitgtk), Red Hat (firefox), Scientific Linux (firefox and openssh), SUSE (go1.20, ImageMagick, javapackages-tools, javassist, mysql-connector-java, protobuf, python-python-gflags, kernel, openssl-1_1, pipewire, python-pip, and xtrans), and Ubuntu (cargo, rust-cargo, cpio, poppler, and xmltooling).
The kernel community has never had a smooth relationship with the purveyorsof proprietary kernel modules. Developers tend to strongly dislike thosemodules, which cannot be debugged or fixed by anybody other than theircreator, and many see them as a violation of the kernel's license and theircopyrights on the code. Nonetheless, proprietary modules are tolerated,within bounds. A recent patch from Christoph Hellwig suggests that thosebounds are about to be tightened slightly, in a somewhat surprising way.
Security updates have been issued by Debian (linux-5.10), Red Hat (.NET 6.0 and iperf3), Slackware (openssl), SUSE (kernel, mariadb, poppler, and python-Django), and Ubuntu (gst-plugins-base1.0, gst-plugins-good1.0, maradns, openjdk-20, and vim).
The Python global interpreter lock (GIL) has long been a barrier toincreasing the performance of programs by using multiple threads-the GILserializes access to the interpreter's virtual machine such that only one threadcan be executing Python code at any given time. There are other mechanismsto provide concurrency for the language, but the specter of the GIL-and its reality aswell-have often been cited as a major negative for Python. Back in October2021, Sam Gross introduceda proof-of-concept, no-GIL version of thelanguage. It was met with a lot of excitement at the time, butseemed to languish to a certain extent for more than a year; now, the PythonSteering Council has announced its intent to accept theno-GIL feature. It will still be some time before it lands in areleased Python version-and there is the possibility that it all has to berolled back at some point-but there are several companies backing theeffort, which gives it all a good chance to succeed.
Google's Project Zero has spent some time studying the Arm memory taggingextension (MTE), support for which wasmerged into the 5.10 kernel, and postedthe results:
LWN.net