LWN.net

Just over 7,500 non-merge changesets have been pulled into the mainlinerepository since the opening of the 5.8 merge window — not a small amountof work for just four days. The early pulls are dominated by thenetworking and graphics trees, but there is a lot of other material inthere as well. Read on for a summary of what entered the kernel in thefirst part of this development cycle.

Security updates have been issued by CentOS (bind, firefox, and freerdp), Debian (netqmail and python-django), Fedora (cacti, cacti-spine, dbus, firefox, gjs, mbedtls, mozjs68, and perl), Oracle (freerdp and kernel), Scientific Linux (bind and firefox), Slackware (mozilla), SUSE (krb5-appl, libcroco, libexif, libreoffice, libxml2, qemu, transfig, and vim), and Ubuntu (firefox, freerdp, and python-django).

Recently, the DMA-BUF heapsinterface was added to the 5.6 kernel. Thisinterface is similar to ION,which has been used for years by Android vendors. However, in trying to move vendors touse DMA-BUF heaps, we have begun to see how the DMA API modeldoesn't fit well for modern mobile devices. Additionally, the lack of clearguidance in how to handle cache operations efficiently, results in vendorsusing custom device-specific optimizations that aren't generic enough foran upstream solution. This article will describe the nature of theproblem; the upcoming second installment will look at the path toward asolution.

Security updates have been issued by Debian (firefox-esr), Fedora (firefox and prboom-plus), Oracle (bind), Red Hat (firefox), and SUSE (osc).

The LWN.net Weekly Edition for June 4, 2020 is available.

The PHP language is widely used in solving some of the most interestingtechnical problems on the web. But for a language with widespread use, itis unique — or at least an outlier — in the way it's governed compared toother open-source projects. Unlike others, PHP governance has grown intosomething fairly democratic for a project its size, allowing almost anyoneto bring an idea to the table. If it's popular enough, that idea can findits way into a future release. That is, of course, as long as there is adeveloper to put in the work to make it happen.

The FreeNAS distribution implements network-attached storage on top of theZFS filesystem; it was reviewed here backin 2015. FreeNAS has always been based on FreeBSD, but now iXsystems, thecompany behind this system, has announceda new version, called TrueNAS SCALE, that will be based on Debian."Linux is a key requirement to achieve some of the SCALE projectgoals". More information about those goals will evidently beforthcoming in the future.

In the kernel graphics world, there has been a longstanding "line in the sand" that disallows mergingkernel drivers without a corresponding free-software user-space driver. The idea is thatnot having a way to test the full functionality means that the kerneldevelopers cannot verify the proper functioning and security of thedriver; changes to the kernel driver may lead to unforeseen (anduntestable) problems on the user-space side. More recently, though, wehave seen other types of devices with complex drivers, but no useful freeuser-space piece, that have been proposed for inclusion into the kernel;at least one was merged, but the tide has perhaps turned against those typesof drivers at this point—or some of them, anyway.

Stable kernels 5.6.16, 5.4.44, 4.19.126, 4.14.183, 4.9.226, and 4.4.226 have been released. They all containimportant fixes and users should upgrade.

Security updates have been issued by Fedora (java-11-openjdk, perl-Email-MIME, perl-Email-MIME-ContentType, and slurm), openSUSE (imapfilter, mailman, and python-rpyc), Red Hat (bind and firefox), SUSE (evolution-data-server, python, qemu, and w3m), and Ubuntu (python-django).

Devuan Beowulf 3.0.0 has been released. This version is based on Debian10.4 Buster, with eudev and elogind to replace aspects of systemd. Optionalalternatives runit and openrc are also available.

The 5.7 kernel was released onMay 31. By all appearances this was a normal development cycle,unaffected by the troubles in the wider world. Still, there are things tobe learned by looking at where the code came from this time around. Readon for LWN's traditional look at who contributed to 5.7, who supported thatwork, and the paths by which it got into the mainline.

Firefox 77.0 has been released. Among the new things in this release, LWNreaders may be most interested in the new about:certificate pagewhere you can view and manage web certificates. See the releasenotes for details.

Security updates have been issued by Arch Linux (ant, bind, freerdp, and unbound), CentOS (bind, freerdp, and git), Debian (python-httplib2), Fedora (ant, kernel, sqlite, and sympa), openSUSE (java-11-openjdk and qemu), Oracle (bind), Red Hat (freerdp), Scientific Linux (python-pip and python-virtualenv), Slackware (firefox), SUSE (qemu), and Ubuntu (Apache Ant, ca-certificates, flask, and freerdp2).

The FSGSBASEpatch series is up to its thirteenth version as of late May. Itenables some "new" instructions for the x86 architecture, opening the way for a number ofsignificant performance improvements. One might think that such a patchseries would be a shoo-in, but FSGSBASE has had a troubled history;meanwhile, the delays in getting it merged may have led to a number ofusers installing root holes on their Linux systems in the hope of improvingsecurity.

Security updates have been issued by Debian (bind9, dosfstools, gst-plugins-good0.10, gst-plugins-ugly0.10, json-c, php-horde, php-horde-gollem, salt, and sane-backends), Fedora (drupal7, marked, NetworkManager, and wireshark), Mageia (gdb, jasper, and json-c), openSUSE (freetds, jasper, libmspack, mariadb-connector-c, sysstat, and trousers), Red Hat (bind), Scientific Linux (bind and freerdp), and SUSE (file-roller and java-11-openjdk).

Linus has released the 5.7 kernel right onschedule. Headline features in 5.7 includex86 split-lock detection,thermal-pressure management,frequency invariance in the load-trackingcode,coexistence between BPF and realtimepreemption,support for BPF security hook programs (formerly called the KRSI security module),a new, Microsoft-blessed exFAT filesystem implementation, and more.The final patch to be merged was this one deprecatingthe long-standing 80-column limit for kernel source.See the KernelNewbies 5.7 page forlots of details.

The Linux deadline scheduler supports realtime systems whereapplications need tobe sure of getting their work done within a specific period of time. Itallocates CPU time to deadline tasks in such a way as to ensure that eachtask's specific timing constraints are met.However, the currentimplementation does not work well on asymmetric CPU configurations like Arm'sbig.LITTLE. Dietmar Eggemann recently posteda patch set to address this problem by adding the notion of CPUcapacity to the deadline scheduler.

Security updates have been issued by Debian (libexif and tomcat8), Fedora (python38), openSUSE (libxslt), Oracle (git), Red Hat (bind, freerdp, and git), Scientific Linux (git), SUSE (qemu and tomcat), and Ubuntu (apt, json-c, kernel, linux, linux-raspi2, linux-raspi2-5.3, and openssl).

In traditional build tools like Make, targets and dependencies are alwaysfiles. Imagine if you could specify an entire tree (directory) as adependency: You could exhaustively specify a "build root" filesystem containingthe toolchain used for building some target as a dependency of that target.Similarly, a rule that creates that build root would have the tree as itstarget.Using Merkletrees as first-class citizens in a build system gives greatflexibility and many optimization opportunities. In this article, guest author David Röthlisbergerexplores this idea using OSTree,Ninja, and Python.

Security updates have been issued by Fedora (dovecot, dpdk, knot-resolver, and unbound), Mageia (ant, libexif, and php), SUSE (libmspack), and Ubuntu (php5, php7.0, php7.2, php7.3, php7.4 and unbound).

The LWN.net Weekly Edition for May 28, 2020 is available.

The Python Language Summit is an annual gathering for the developers ofvarious Python implementations, though, this year, the gathering actuallyhappened via videoconference—as with so many other conferences due to the pandemic.The invite-only gathering typically has numerous interesting sessions, ascan be seen in the LWN coverage ofthe summit from 2015 to 2018, as well as in the 2019 summit coverageon the Python SoftwareFoundation (PSF) blog. Those writeups were penned by A. Jesse JiryuDavis, who reprised his role for thisyear's summit. In this article, I will summarize some of the sessions that caught my eye.

Kees Cook takesa look some changes improving security in Linux 5.5. Topics includerestrict perf_event_open() from LSM, generic fast fullrefcount_t, linker script cleanup for exception tables, KASLR for32-bit PowerPC, seccomp for RISC-V, and more.

We are living through interesting times that present challenges in a numberof areas, including running a business. While we think of LWN primarily asa community resource, it is also a business that is not unaffected by theongoing pandemic. It is, we figure, a good time for a status update,especially since we have some news to share.

Stable kernels 5.6.15, 5.4.43, 4.19.125, 4.14.182, 4.9.225, and 4.4.225 have been released. They all containimportant fixes and users should upgrade.

The OpenSSH 8.3 release is out. This primarily a bug-fix release with ahandful of minor new features. It does, however, carry a prominent noticethat ssh-rsa signature algorithm will be disabled in "a near-futurerelease". The announcement includes information on how to determinewhether hosts you care about are affected.

Security updates have been issued by Debian (drupal7 and unbound), Fedora (libEMF and transmission), Mageia (dojo, log4net, nginx, nodejs-set-value, sleuthkit, and transmission), Red Hat (rh-maven35-jackson-databind), SUSE (dpdk and mariadb-connector-c), and Ubuntu (thunderbird).

Here's adetailed blog entry from Dan Carpenter on adding improved lock checkingto the smatch static-analysis tool. "When Smatch gained theability to do cross function analysis in 2010, I knew that I had tore-write the locking check to take advantage of the new cross functionanalysis feature. When you combine cross function analysis with top of theline flow analysis available and in depth knowledge of kernel locks thenthe result is the Ultimate Locking Check! Unfortunately, I have a tendencytowards procrastination and it took me a decade to get around to it, but itis done now. This blog will step through how the locking analysisworks."

The Go programming language comes withtools for writing and running tests: the standard library's testing package, andthe gotest command to run test suites. Like the language itself, Go'sphilosophy for writing tests is minimalist: use thelightweight testing package along with helper functionswritten in plain Go. The idea is that tests are just code, and since a Godeveloper already knows how to write Go using its abstractions and types,there's no need to learn a quirky domain-specific language for writingtests.

Security updates have been issued by Debian (sqlite3), Fedora (libarchive and netdata), openSUSE (dom4j, dovecot23, gcc9, and memcached), Red Hat (devtoolset-9-gcc, httpd24-httpd and httpd24-mod_md, ipmitool, kernel, kpatch-patch, openvswitch, openvswitch2.11, openvswitch2.13, rh-haproxy18-haproxy, and ruby), and SUSE (freetds, jasper, libxslt, and sysstat).

Version 017 of thedecidedly non-traditional GoboLinux distribution has been released."This release introduces a simplified model for recipe management and contribution that's fully integrated with the Compile build tool.The recipe tree is now a plain Git repository managed via GitHub clonedinto your /Data/Compile/Recipes directory and used by the GoboLinux Compiletool directly."

Hibernation is normally thought of as a laptop feature — and an old and obsolete laptop feature at that. One does not normally consider itto be relevant in cloud settings. But, at the 2020 Power Management and Schedulingin the Linux Kernel summit (OSPM), Andrea Righi argued that there mayactually be a place for hibernation on cloud-based systems if it can bemade to work reliably.

Security updates have been issued by Arch Linux (chromium, dovecot, openconnect, and powerdns-recursor), Debian (cracklib2, feh, netqmail, ruby-rack, tomcat7, and transmission), Fedora (dovecot, kernel, log4net, openconnect, python-markdown2, and unbound), Mageia (ansible, clamav, dovecot, file-roller, glpi, kernel, kernel-linus, libntlm, microcode, nmap, pdns-recursor, unbound, viewvc, and wireshark), openSUSE (ant, autoyast2, dpdk, file, freetype2, gstreamer-plugins-base, imapfilter, libbsd, libvpx, libxml2, nextcloud, openconnect, openexr, opera, pdns-recursor, python, python-rpyc, and tomcat), and SUSE (salt, tomcat6, and zstd).

The 5.7-rc7 kernel prepatch is out."So it looks like I was worried for nothing last rc. Of course,anything can still change, but everything _looks_ all set for aregular release scheduled for next weekend. Knock wood."

The kernel's CPU scheduler is good at distributing tasks across amultiprocessor system, but does it do so fairly? If some tasks get a lotmore CPU time than others, the result is likely to be unhappy users.Vincent Guittot ran a session at the 2020 Power Management and Schedulingin the Linux Kernel summit (OSPM) looking into this issue, with a focuson detecting load imbalances between CPUs and what to do with a workloadthat cannot be balanced.

As Rafael Wysocki conceded at the beginning of a session at the 2020 Power Management and Schedulingin the Linux Kernel summit (OSPM), the combination of the deadline scheduling class with CPU idle statesmight seem a little strange. Deadline scheduling is used in realtimesettings, where introducing latency by idling the CPU tends to be frownedupon. But there are reasons to think that these two technologies mightjust be made to work together.

Security updates have been issued by CentOS (firefox, ipmitool, kernel, squid, and thunderbird), Debian (pdns-recursor), Fedora (php and ruby), Red Hat (dotnet and dotnet3.1), SUSE (dom4j, dovecot23, memcached, and tomcat), and Ubuntu (clamav, libvirt, and qemu).

Frequency scaling — adjusting a CPU's operating frequency to save power when theworkload demands are low — is common practice across systems supported byLinux. It is, however, viewed with some suspicion in data-center settings, wherepower consumption is less of a concern and there is a strong emphasis ongetting the most performance out of the hardware. At the 2020 Power Management and Schedulingin the Linux Kernel summit (OSPM), Giovanni Gherdovich worried thatfrequency scaling may be about to go extinct in data centers; he made aplea for improving its behavior for such workloads while there is stilltime.

The purpose of a cpuidle governor is to decide which idle state a CPUshould go into when it has no useful work to do; the cpuidle driverthen actually puts the CPU into that state. But, at the 2020 Power Management and Schedulingin the Linux Kernel summit (OSPM), Abhishek Goel presented a newcpuidle driver that doesn't actually change the processor's power state at all.Such a driver will clearly save no power, but it can be quite useful as atool for evaluating and debugging cpuidle policies.

The patent suit filed against the GNOMEFoundation last September hasnow been resolved. "In this walk-away settlement, GNOME receivesa release and covenant not to be sued for any patent held by RothschildPatent Imaging. Further, both Rothschild Patent Imaging and LeighRothschild are granting a release and covenant to any software that isreleased under an existing Open Source Initiative approved license (andsubsequent versions thereof), including for the entire Rothschild portfolioof patents, to the extent such software forms a material part of theinfringement allegation." There is no mention of what thefoundation had to give — if anything — for this settlement,

Here's a preprint paper fromMarc Ohm, Henrik Plate, Arnold Sykosch, and Michael Meier looking atattacks on language-specific repositories. "Recent years saw anumber of supply chain attacks that leverage the increasing use of opensource during software development, which is facilitated by dependencymanagers that automatically resolve, download and install hundreds of opensource packages throughout the software life cycle. This paper presents adataset of 174 malicious software packages that were used in real-worldattacks on open source software supply chains, and which were distributedvia the popular package repositories npm, PyPI, and RubyGems. Thosepackages, dating from November 2015 to November 2019, were manuallycollected and analyzed. The paper also presents two general attack trees toprovide a structured overview about techniques to inject malicious codeinto the dependency tree of downstream users, and to execute such code atdifferent times and under different conditions."

Security updates have been issued by Arch Linux (keycloak, qemu, and thunderbird), Debian (dovecot), Fedora (abcm2ps and oddjob), Red Hat (java-1.7.1-ibm, java-1.8.0-ibm, and kernel-rt), SUSE (ant, bind, and freetype2), and Ubuntu (bind9 and linux, linux-aws, linux-aws-5.3, linux-gcp, linux-gcp-5.3, linux-gke-5.3,linux-hwe, linux-kvm, linux-oracle, linux-oracle-5.3, linux-raspi2 ).

The LWN.net Weekly Edition for May 21, 2020 is available.

With the releaseof Python 3.9.0b1, the first of four planned betas for the developmentcycle, Python 3.9 is now feature-complete. There is still plenty todo in terms of testing and stabilization before the October finalrelease. The release announcement lists a half-dozen Python EnhancementProposals (PEPs) that were accepted for 3.9. We have looked at someof those PEPs along the way; there are some updates on those. It seemslike a good time to fill in some of the gaps on what will be coming in Python 3.9

Stable kernels 5.6.14, 5.4.42, 4.19.124, 4.14.181, 4.9.224, and 4.4.224 have been released with importantfixes. Users should upgrade.

Just in case anybody out there is still using qmail: a remote codeexecution vulnerability has just been disclosed. Its CVE number isCVE-2005-1513 because, as it turns out, the problem was reported 15 yearsago but the fix was refused by the maintainer."As a proof of concept, we developed a reliable, local and remote exploitagainst Debian's qmail package in its default configuration. This proofof concept requires 4GB of disk space and 8GB of memory, and allows anattacker to execute arbitrary shell commands as any user, except root(and a few system users who do not own their home directory)."

Developers of safety-critical systems tend to avoid Linux kernels for anumber of fairly obvious reasons; Linux simply was not developed with thatsort of use case in mind. There are increasingly compelling reasons to useLinux in such systems, though, leading to a search for the best way to doso safely. At the 2020 Power Management and Schedulingin the Linux Kernel summit (OSPM), José Martins described Bao, a minimalhypervisor aimed at safety-critical deployments.

Security updates have been issued by Debian (bind9 and clamav), Fedora (kernel, moodle, and transmission), Oracle (kernel), Red Hat (ipmitool, kernel, ksh, and ruby), Slackware (bind and libexif), SUSE (dpdk, openconnect, python, and rpmlint), and Ubuntu (linux, linux-aws, linux-gcp, linux-kvm, linux-oracle, linux-riscv and linux-gke-5.0, linux-oem-osp1).

AWK is a text-processing language with a history spanning more than 40years. It has a POSIXstandard, several conforming implementations, and is still surprisingly relevant in 2020 — both for simple text processing tasks and for wrangling "big data". Therecentreleaseof GNU Awk 5.1 seems like a good reason to survey the AWK landscape, seewhat GNU Awk has been up to, and look at where AWK is being used these days.