LWN.net

The Linux deadline scheduler supports realtime systems whereapplications need tobe sure of getting their work done within a specific period of time. Itallocates CPU time to deadline tasks in such a way as to ensure that eachtask's specific timing constraints are met.However, the currentimplementation does not work well on asymmetric CPU configurations like Arm'sbig.LITTLE. Dietmar Eggemann recently posteda patch set to address this problem by adding the notion of CPUcapacity to the deadline scheduler.

Security updates have been issued by Debian (libexif and tomcat8), Fedora (python38), openSUSE (libxslt), Oracle (git), Red Hat (bind, freerdp, and git), Scientific Linux (git), SUSE (qemu and tomcat), and Ubuntu (apt, json-c, kernel, linux, linux-raspi2, linux-raspi2-5.3, and openssl).

In traditional build tools like Make, targets and dependencies are alwaysfiles. Imagine if you could specify an entire tree (directory) as adependency: You could exhaustively specify a "build root" filesystem containingthe toolchain used for building some target as a dependency of that target.Similarly, a rule that creates that build root would have the tree as itstarget.Using Merkletrees as first-class citizens in a build system gives greatflexibility and many optimization opportunities. In this article, guest author David Röthlisbergerexplores this idea using OSTree,Ninja, and Python.

Security updates have been issued by Fedora (dovecot, dpdk, knot-resolver, and unbound), Mageia (ant, libexif, and php), SUSE (libmspack), and Ubuntu (php5, php7.0, php7.2, php7.3, php7.4 and unbound).

The LWN.net Weekly Edition for May 28, 2020 is available.

The Python Language Summit is an annual gathering for the developers ofvarious Python implementations, though, this year, the gathering actuallyhappened via videoconference—as with so many other conferences due to the pandemic.The invite-only gathering typically has numerous interesting sessions, ascan be seen in the LWN coverage ofthe summit from 2015 to 2018, as well as in the 2019 summit coverageon the Python SoftwareFoundation (PSF) blog. Those writeups were penned by A. Jesse JiryuDavis, who reprised his role for thisyear's summit. In this article, I will summarize some of the sessions that caught my eye.

Kees Cook takesa look some changes improving security in Linux 5.5. Topics includerestrict perf_event_open() from LSM, generic fast fullrefcount_t, linker script cleanup for exception tables, KASLR for32-bit PowerPC, seccomp for RISC-V, and more.

We are living through interesting times that present challenges in a numberof areas, including running a business. While we think of LWN primarily asa community resource, it is also a business that is not unaffected by theongoing pandemic. It is, we figure, a good time for a status update,especially since we have some news to share.

Stable kernels 5.6.15, 5.4.43, 4.19.125, 4.14.182, 4.9.225, and 4.4.225 have been released. They all containimportant fixes and users should upgrade.

The OpenSSH 8.3 release is out. This primarily a bug-fix release with ahandful of minor new features. It does, however, carry a prominent noticethat ssh-rsa signature algorithm will be disabled in "a near-futurerelease". The announcement includes information on how to determinewhether hosts you care about are affected.

Security updates have been issued by Debian (drupal7 and unbound), Fedora (libEMF and transmission), Mageia (dojo, log4net, nginx, nodejs-set-value, sleuthkit, and transmission), Red Hat (rh-maven35-jackson-databind), SUSE (dpdk and mariadb-connector-c), and Ubuntu (thunderbird).

Here's adetailed blog entry from Dan Carpenter on adding improved lock checkingto the smatch static-analysis tool. "When Smatch gained theability to do cross function analysis in 2010, I knew that I had tore-write the locking check to take advantage of the new cross functionanalysis feature. When you combine cross function analysis with top of theline flow analysis available and in depth knowledge of kernel locks thenthe result is the Ultimate Locking Check! Unfortunately, I have a tendencytowards procrastination and it took me a decade to get around to it, but itis done now. This blog will step through how the locking analysisworks."

The Go programming language comes withtools for writing and running tests: the standard library's testing package, andthe gotest command to run test suites. Like the language itself, Go'sphilosophy for writing tests is minimalist: use thelightweight testing package along with helper functionswritten in plain Go. The idea is that tests are just code, and since a Godeveloper already knows how to write Go using its abstractions and types,there's no need to learn a quirky domain-specific language for writingtests.

Security updates have been issued by Debian (sqlite3), Fedora (libarchive and netdata), openSUSE (dom4j, dovecot23, gcc9, and memcached), Red Hat (devtoolset-9-gcc, httpd24-httpd and httpd24-mod_md, ipmitool, kernel, kpatch-patch, openvswitch, openvswitch2.11, openvswitch2.13, rh-haproxy18-haproxy, and ruby), and SUSE (freetds, jasper, libxslt, and sysstat).

Version 017 of thedecidedly non-traditional GoboLinux distribution has been released."This release introduces a simplified model for recipe management and contribution that's fully integrated with the Compile build tool.The recipe tree is now a plain Git repository managed via GitHub clonedinto your /Data/Compile/Recipes directory and used by the GoboLinux Compiletool directly."

Hibernation is normally thought of as a laptop feature — and an old and obsolete laptop feature at that. One does not normally consider itto be relevant in cloud settings. But, at the 2020 Power Management and Schedulingin the Linux Kernel summit (OSPM), Andrea Righi argued that there mayactually be a place for hibernation on cloud-based systems if it can bemade to work reliably.

Security updates have been issued by Arch Linux (chromium, dovecot, openconnect, and powerdns-recursor), Debian (cracklib2, feh, netqmail, ruby-rack, tomcat7, and transmission), Fedora (dovecot, kernel, log4net, openconnect, python-markdown2, and unbound), Mageia (ansible, clamav, dovecot, file-roller, glpi, kernel, kernel-linus, libntlm, microcode, nmap, pdns-recursor, unbound, viewvc, and wireshark), openSUSE (ant, autoyast2, dpdk, file, freetype2, gstreamer-plugins-base, imapfilter, libbsd, libvpx, libxml2, nextcloud, openconnect, openexr, opera, pdns-recursor, python, python-rpyc, and tomcat), and SUSE (salt, tomcat6, and zstd).

The 5.7-rc7 kernel prepatch is out."So it looks like I was worried for nothing last rc. Of course,anything can still change, but everything _looks_ all set for aregular release scheduled for next weekend. Knock wood."

The kernel's CPU scheduler is good at distributing tasks across amultiprocessor system, but does it do so fairly? If some tasks get a lotmore CPU time than others, the result is likely to be unhappy users.Vincent Guittot ran a session at the 2020 Power Management and Schedulingin the Linux Kernel summit (OSPM) looking into this issue, with a focuson detecting load imbalances between CPUs and what to do with a workloadthat cannot be balanced.

As Rafael Wysocki conceded at the beginning of a session at the 2020 Power Management and Schedulingin the Linux Kernel summit (OSPM), the combination of the deadline scheduling class with CPU idle statesmight seem a little strange. Deadline scheduling is used in realtimesettings, where introducing latency by idling the CPU tends to be frownedupon. But there are reasons to think that these two technologies mightjust be made to work together.

Security updates have been issued by CentOS (firefox, ipmitool, kernel, squid, and thunderbird), Debian (pdns-recursor), Fedora (php and ruby), Red Hat (dotnet and dotnet3.1), SUSE (dom4j, dovecot23, memcached, and tomcat), and Ubuntu (clamav, libvirt, and qemu).

Frequency scaling — adjusting a CPU's operating frequency to save power when theworkload demands are low — is common practice across systems supported byLinux. It is, however, viewed with some suspicion in data-center settings, wherepower consumption is less of a concern and there is a strong emphasis ongetting the most performance out of the hardware. At the 2020 Power Management and Schedulingin the Linux Kernel summit (OSPM), Giovanni Gherdovich worried thatfrequency scaling may be about to go extinct in data centers; he made aplea for improving its behavior for such workloads while there is stilltime.

The purpose of a cpuidle governor is to decide which idle state a CPUshould go into when it has no useful work to do; the cpuidle driverthen actually puts the CPU into that state. But, at the 2020 Power Management and Schedulingin the Linux Kernel summit (OSPM), Abhishek Goel presented a newcpuidle driver that doesn't actually change the processor's power state at all.Such a driver will clearly save no power, but it can be quite useful as atool for evaluating and debugging cpuidle policies.

The patent suit filed against the GNOMEFoundation last September hasnow been resolved. "In this walk-away settlement, GNOME receivesa release and covenant not to be sued for any patent held by RothschildPatent Imaging. Further, both Rothschild Patent Imaging and LeighRothschild are granting a release and covenant to any software that isreleased under an existing Open Source Initiative approved license (andsubsequent versions thereof), including for the entire Rothschild portfolioof patents, to the extent such software forms a material part of theinfringement allegation." There is no mention of what thefoundation had to give — if anything — for this settlement,

Here's a preprint paper fromMarc Ohm, Henrik Plate, Arnold Sykosch, and Michael Meier looking atattacks on language-specific repositories. "Recent years saw anumber of supply chain attacks that leverage the increasing use of opensource during software development, which is facilitated by dependencymanagers that automatically resolve, download and install hundreds of opensource packages throughout the software life cycle. This paper presents adataset of 174 malicious software packages that were used in real-worldattacks on open source software supply chains, and which were distributedvia the popular package repositories npm, PyPI, and RubyGems. Thosepackages, dating from November 2015 to November 2019, were manuallycollected and analyzed. The paper also presents two general attack trees toprovide a structured overview about techniques to inject malicious codeinto the dependency tree of downstream users, and to execute such code atdifferent times and under different conditions."

Security updates have been issued by Arch Linux (keycloak, qemu, and thunderbird), Debian (dovecot), Fedora (abcm2ps and oddjob), Red Hat (java-1.7.1-ibm, java-1.8.0-ibm, and kernel-rt), SUSE (ant, bind, and freetype2), and Ubuntu (bind9 and linux, linux-aws, linux-aws-5.3, linux-gcp, linux-gcp-5.3, linux-gke-5.3,linux-hwe, linux-kvm, linux-oracle, linux-oracle-5.3, linux-raspi2 ).

The LWN.net Weekly Edition for May 21, 2020 is available.

With the releaseof Python 3.9.0b1, the first of four planned betas for the developmentcycle, Python 3.9 is now feature-complete. There is still plenty todo in terms of testing and stabilization before the October finalrelease. The release announcement lists a half-dozen Python EnhancementProposals (PEPs) that were accepted for 3.9. We have looked at someof those PEPs along the way; there are some updates on those. It seemslike a good time to fill in some of the gaps on what will be coming in Python 3.9

Stable kernels 5.6.14, 5.4.42, 4.19.124, 4.14.181, 4.9.224, and 4.4.224 have been released with importantfixes. Users should upgrade.

Just in case anybody out there is still using qmail: a remote codeexecution vulnerability has just been disclosed. Its CVE number isCVE-2005-1513 because, as it turns out, the problem was reported 15 yearsago but the fix was refused by the maintainer."As a proof of concept, we developed a reliable, local and remote exploitagainst Debian's qmail package in its default configuration. This proofof concept requires 4GB of disk space and 8GB of memory, and allows anattacker to execute arbitrary shell commands as any user, except root(and a few system users who do not own their home directory)."

Developers of safety-critical systems tend to avoid Linux kernels for anumber of fairly obvious reasons; Linux simply was not developed with thatsort of use case in mind. There are increasingly compelling reasons to useLinux in such systems, though, leading to a search for the best way to doso safely. At the 2020 Power Management and Schedulingin the Linux Kernel summit (OSPM), José Martins described Bao, a minimalhypervisor aimed at safety-critical deployments.

Security updates have been issued by Debian (bind9 and clamav), Fedora (kernel, moodle, and transmission), Oracle (kernel), Red Hat (ipmitool, kernel, ksh, and ruby), Slackware (bind and libexif), SUSE (dpdk, openconnect, python, and rpmlint), and Ubuntu (linux, linux-aws, linux-gcp, linux-kvm, linux-oracle, linux-riscv and linux-gke-5.0, linux-oem-osp1).

AWK is a text-processing language with a history spanning more than 40years. It has a POSIXstandard, several conforming implementations, and is still surprisingly relevant in 2020 — both for simple text processing tasks and for wrangling "big data". Therecentreleaseof GNU Awk 5.1 seems like a good reason to survey the AWK landscape, seewhat GNU Awk has been up to, and look at where AWK is being used these days.

CZ.NIC staff member Petr Špaček has a blog post describing a newly disclosed DNS resolver vulnerability called NXNSAttack. It allows attackers to abuse the delegation mechanism to create a denial-of-service condition via packet amplification. "This is so-called glueless delegation, i.e. a delegation which contains only names of authoritative DNS servers (a.iana-servers.net. and b.iana-servers.net.), but does not contain their IP addresses. Obviously DNS resolver cannot send a query to “name”, so the resolver first needs to obtain IPv4 or IPv6 address of authoritative server 'a.iana-servers.net.' or 'b.iana-servers.net.' and only then it can continue resolving the original query 'example.com. A'.This glueless delegation is the basic principle of the NXNSAttack: Attacker simply sends back delegation with fake (random) server names pointing to victim DNS domain, thus forcing the resolver to generate queries towards victim DNS servers (in a futile attempt to resolve fake authoritative server names)." At this time, Ubuntu has updated its BIND package to mitigate the problem; other distributions will no doubt follow soon. More details can also be found in the paper [PDF].

The kernel's CPU scheduler does its best to make the right decisions forjust about any workload; over the years, it has been extended to betterhandle mobile-device scheduling as well. But handset vendors still end upapplying their own patches to the scheduler for the kernels they ship.Shipping out-of-tree code in this way leads to a certain amount ofcriticism from the kernel community but, asVincent Donnefort pointed out in his session at the 2020 Power Management and Schedulingin the Linux Kernel summit (OSPM), those patches are applied for areason. He looked at a set of vendor scheduler patches to see why they arebeing used.

Security updates have been issued by Debian (dpdk and exim4), Fedora (openconnect, perl-Mojolicious, and php), Red Hat (kernel and kpatch-patch), Slackware (sane), and Ubuntu (bind9, dpdk, exim4, linux, linux-aws, linux-aws-hwe, linux-gcp, linux-gke-4.15, linux-hwe, linux-oem, linux-oracle, linux-snapdragon, and linux, linux-aws, linux-lts-xenial, linux-raspi2, linux-snapdragon).

The MMTests benchmarkingsystem is normally associated with its initial use case: testingmemory-management changes. Increasingly, though, MMTests is not limited tomemory management testing; at the 2020 Power Management and Schedulingin the Linux Kernel summit (OSPM), Dario Faggioli talked about how heis using it to evaluate changes to the CPU scheduler, along with adiscussion of the changes he had to make to get useful results for systemshosting virtualized guests.

A task's "nice" value describes its priority within the completely fairscheduler; its semantics have roots in ancient Unix tradition. LastAugust, a "latencynice" parameter was proposed to provide similar control over a task'sresponse-time requirements. At the 2020 Power Management and Schedulingin the Linux Kernel summit (OSPM), Parth Shah, Chris Hyser, and DietmarEggemann ran a discussion about the latency nice proposal; it seems thateverybody agrees that it would be a useful feature to have, but there is awide variety of opinions about what it should actually do.

Security updates have been issued by Debian (apache-log4j1.2, exim4, libexif, and openconnect), Fedora (chromium, condor, java-1.8.0-openjdk, java-1.8.0-openjdk-aarch32, mingw-ilmbase, mingw-OpenEXR, sleuthkit, and squid), Mageia (jbig2dec, libreswan, netkit-telnet, ntp, and suricata), openSUSE (mailman and nextcloud), SUSE (autoyast2, file, git, gstreamer-plugins-base, libbsd, libvirt, libvpx, libxml2, mailman, and openexr), and Ubuntu (dovecot and json-c).

Linus has released the 5.7-rc6 kernelprepatch, which contains a bit more churn than he would like."That said, there's nothing particularly scary in here, and it's notlike this rc6 is outrageously big or out of control. I was just hoping forless."

Over the years, the kernel's CPU scheduler has become increasingly aware ofhow much load every task is putting on the system; this information is usedto make smarter task placement decisions. Sometimes, though, this logiccan go wrong, leading to a situation that Valentin Schneider describes as"utilization inversion". At the 2020 Power Management and Schedulingin the Linux Kernel summit (OSPM), he described the problem and someapproaches that are being considered to address it.

Linux is not heavily used in safety-critical systems — yet. There is anincreasing level of interest in such deployments, though, and that isdriving a number of initiatives to determine how Linux can be made suitablefor safety-critical environments. At the 2020 Power Management and Schedulingin the Linux Kernel summit (OSPM), Michal Sojka shone a light on onecorner of this work: testing the thermal characteristics of Linux systemswith an eye toward deployment in avionics systems.

Security updates have been issued by Debian (apt, inetutils, and log4net), Fedora (kernel, mailman, and viewvc), Gentoo (chromium, freerdp, libmicrodns, live, openslp, python, vlc, and xen), Oracle (.NET Core, container-tools:1.0, and kernel), Red Hat (kernel-rt), Scientific Linux (kernel), SUSE (kernel, libvirt, python-PyYAML, and syslog-ng), and Ubuntu (json-c).

It seems that the Rust programming language hasonly been around for five years. "With all that's going on inthe world you'd be forgiven for forgetting that as of today, it has beenfive years since we released 1.0 in 2015! Rust has changed a lot these pastfive years, so we wanted reflect back on all of our contributors' worksince the stabilization of the language."

Libre Graphics World is running anextensive interview with several Inkscape developers."I'd say we're at the point of supporting SVG as much as possible,but we've mostly given up trying to add editing features to the SVGspecification. As the W3C is dominated by web browsers who don't need multipage or connectors.I dare not say much more about W3C-specific things. I know that I'mpersonally disappointed that Inkscape's considerable importance in the SVGcreation space does not lend itself to getting the feature we intend tobuild into Inkscape into the actual SVG specification. This does lead tothe problem that going forwards we're likely to have browserincompatibilities."

Life gets complicated for the kernel when there is nothing for the systemto do. The obvious response is to put the CPU into an idle state tosave power, but which one? CPUs offer a wide range of sleep states withdifferent power-usage and latency characteristics. Picking too shallow astate will waste energy, while going too deep hurts latency and can impactthe performance of the system as a whole. The timer-events-oriented (TEO) cpuidle governoris a relatively new attempt to improve the kernel's choice of sleep states;at the 2020 Power Management and Scheduling in the Linux Kernel Summit,Pratik Sampat presented avariant of the TEO governor that tries to improve its choices further.

The 5.6.13, 5.4.41, and 4.19.123 stable kernels have been released.They contain important fixes throughout the kernel tree; users should upgrade.

Security updates have been issued by Debian (apt and libreswan), Fedora (glpi, grafana, java-latest-openjdk, mailman, and oddjob), Oracle (container-tools:2.0, container-tools:ol8, kernel, libreswan, squid:4, and thunderbird), SUSE (apache2, grafana, and python-paramiko), and Ubuntu (apt and libexif).

The LWN.net Weekly Edition for May 14, 2020 is available.

A project that has been floating around in the Python world for a number ofyears is now working its way toward inclusion into the language—or not."Subinterpreters", which are separate Python interpreters that cancurrently be created via the C API for extensions, are seen by some as away to get a more Go-like concurrency model for Python. The first steptoward that goal is to expose that API in the standard library. But thereare questions about whether subinterpreters are actually a desirablefeature for Python at all, as well as whether the hoped-for concurrencyimprovements will materialize.