LWN.net

Stable kernels 5.4.3, 5.3.16, and 4.19.89 have been released. They all containimportant fixes throughout the tree and users should upgrade.

Security updates have been issued by Debian (davical, intel-microcode, libpgf, php-horde, spamassassin, spip, and thunderbird), Mageia (clementine, dnsmasq, git, jasper, kdelibs4, kernel, libcroco, libgit2, libvirt, ncurses, openafs, proftpd, qbittorrent, signing-party, squid, and wireshark), openSUSE (java-1_8_0-openjdk and postgresql), Oracle (kernel), Red Hat (chromium-browser and openslp), and SUSE (kernel, libssh, and xen).

The second 5.5 kernel prepatch is out."Things look normal - rc2 is usually fairly calm, and so it was thisweek too."

ZDNet reportson a police raid at the NGINX office. "Moscow police executed the raid after last week the Rambler Group filed a copyright violation against NGINX Inc., claiming full ownership of the NGINX web server code. The Rambler Group is the parent company of rambler.ru, one of Russia's biggest search engines and internet portals.According to copies of the search warrant posted on Twitter today, Ramblerclaims that Igor Sysoev developed NGINX while he was working as a systemadministrator for the company, hence they are the rightful owner of theproject."

The saga of get_user_pages() — and the problems it causes withinthe kernel — has been extensively chronicled here; see the LWN kernelindex for the full series. In short, get_user_pages() is usedto pin user-space pages in memory for some sort of manipulation outside ofthe owning process(es); that manipulation can sometimes surprise otherparts of the kernel that think they have exclusive rights to the pages inquestion. Thispatch series from John Hubbard does not solve all of the problems, butit does create some infrastructure that may make a solution easier to comeby.

Security updates have been issued by Fedora (knot-resolver and xen), openSUSE (kernel), and SUSE (haproxy, kernel, and openssl).

Linux offers two modes for file I/O: buffered and direct. Buffered I/Opasses through the kernel's page cache; it is relatively easy to use andcan yield significant performance benefits for data that is accessedmultiple times. Direct I/O, instead, goes straight between a user-spacebuffer and the storage device. It can be much faster for situations wherecaching by the operating system isn't necessary, but it is complex to useand contains traps for the unwary. Now, it seems, Jens Axboe has come upwith away to get many of the benefits of direct I/O with a lot less bother.

Security updates have been issued by CentOS (firefox and nss-softokn), Fedora (samba), Oracle (nss, nss-softokn, nss-util, nss-softokn, and thunderbird), Scientific Linux (thunderbird), SUSE (firefox), and Ubuntu (librabbitmq and samba).

The LWN.net Weekly Edition for December 12, 2019 is available.

An effort to protect package downloads from the PythonPackage Index (PyPI) has resulted in a Python Enhancement Proposal(PEP) and, perhaps belatedly, some discussion in the wider community. Thebasic idea is to use TheUpdate Framework (TUF) to protect PyPI users from some maliciousactors who are aiming to interfere with the installation and update ofPython modules. But the name of the PEP and its wording, coupled with some recent typosquatting problems on PyPI, causedsome confusion along the way. There are some competing interests anddifferent cultures coming together over this PEP; the process has not run assmoothly as anyone might want, though that seems to be resolving itself atthis point.

Security updates have been issued by Arch Linux (crypto++ and thunderbird), Debian (cacti, freeimage, git, and jackson-databind), Fedora (nss), openSUSE (clamav, dnsmasq, munge, opencv, permissions, and shadowsocks-libev), Red Hat (nss, nss-softokn, nss-util, rh-maven35-jackson-databind, and thunderbird), Scientific Linux (nss, nss-softokn, nss-util, nss-softokn, and thunderbird), SUSE (caasp-openstack-heat-templates, crowbar-core, crowbar-openstack, crowbar-ui, etcd, flannel, galera-3, mariadb, mariadb-connector-c, openstack-dashboard-theme-SUSE, openstack-heat-templates, openstack-neutron, openstack-nova, openstack-quickstart, patterns-cloud, python-oslo.messaging, python-oslo.utils, python-pysaml2, libssh, and strongswan), and Ubuntu (git, libpcap, libssh, and thunderbird).

The Electronic Frontier Foundation has posted a detailedstudy on third-party corporate surveillance on the Internet (andbeyond). "Both Google and Apple encourage developers to use ad IDsfor behavioral profiling in lieu of other identifiers like IMEI or phonenumber. Ostensibly, this gives users more control over how they aretracked, since users can reset their identifiers by hand if theychoose. However, in practice, even if a user goes to the trouble to resettheir ad ID, it’s very easy for trackers to identify them across resets byusing other identifiers, like IP address or in-app storage. Android’sdeveloper policy instructs trackers not to engage in such behavior, but theplatform has no technical safeguards to stop it. In February 2019, a studyfound that over 18,000 apps on the Play store were violating Google’spolicy."

A new mechanism to help thwart return-orientedprogramming (ROP) and similar attacks has recently been added to theOpenBSD kernel. It will block system calls that are not made via the Clibrary (libc) system-call wrappers. Instead of being able to stringtogether some "gadgets" that make a system call directly, an attacker wouldneed to be able to call the wrapper, which is normally at a randomized location.

The Kubernetes scheduler is being overhauled with a series of improvementsthat will introduce a new framework and enhanced capabilities that couldhelp cluster administrators to optimize performance andutilization. Abdullah Gharaibeh, co-chair of the Kubernetes schedulingspecial interest group (SIGScheduling), detailed what has been happening with thescheduler in recent releases and what's on the roadmapin a session at KubeCon + CloudNativeCon North America 2019.

The Git project has released Git v2.24.1, v2.23.1, v2.22.2, v2.21.1,v2.20.2, v2.19.3, v2.18.2, v2.17.3, v2.16.6, v2.15.4, andv2.14.6. "These releases fix various security flaws, which allowed anattacker to overwrite arbitrary paths, remotely execute code, and/oroverwrite files in the .git/ directory etc." The release notescontained in this announcement have the details.

Google Open Source has announcedGoogle Summer of Code (GSoC) 2020, a program that introduces universitystudents to open-source development. "And the 'special sauce' that haskept this program thriving for 16 years: the mentorship aspect of theprogram. Participants gain invaluable experience working directly withmentors who are dedicated members of these open source communities; mentorshelp bring students into their communities while teaching them, guidingthem and helping them find their place in the world of open source."Applications for interested organizations open on January 14.

Security updates have been issued by Debian (firefox-esr, jruby, and squid3), Fedora (librabbitmq, libuv, and xpdf), openSUSE (calamares and opera), Oracle (kernel and nss), Red Hat (httpd24-httpd, kernel, kernel-alt, kpatch-patch, nss-softokn, sudo, and thunderbird), SUSE (apache2-mod_perl, java-1_8_0-openjdk, and postgresql), and Ubuntu (eglibc, firefox, and samba).

Daniel Vetter has posted asummary of his LPC talk on kernel graphics drivers."Unfortunately the business case for 'upstream first' on the kernelside is completely broken. Not for open source, and not for any fundamentalreasons, but simply because the kernel moves too slowly, is too big,drivers aren’t well contained enough and therefore customer will not oreven can not upgrade. For some hardware upstreaming early enough ispossible, but graphics simply moves too fast: By the time the upstreameddriver is actually in shipping distros, it’s already one hardwaregeneration behind. And missing almost a year of tuning and performanceimprovements. Worse it’s not just new hardware, but also GL and Vulkanversions that won’t work on older kernels due to missing features,fragmenting the ecosystem further."

By the end of the merge window, 12,632 non-merge changesets had beenpulled into the mainline repository for the 5.5 release. This is thus abusy development cycle — just like the cycles that preceded it. Just overhalf of those changesets were pulled after the writing of our first 5.5 merge-window summary. As isoften the case later in the merge window, many of those changes wererelatively boring fixes. There were still a number of interesting changes,though; read on for a summary of what happened in the second half of thismerge window.

Security updates have been issued by CentOS (SDL), Debian (htmldoc, librabbitmq, nss, openjdk-7, openslp-dfsg, and phpmyadmin), Fedora (chromium, community-mysql, kernel, libidn2, oniguruma, proftpd, and rabbitmq-server), Mageia (ansible, clamav, evince, firefox, graphicsmagick, icu, libcryptopp, libtasn1, libtiff, libvncserver, libvpx, lz4, nss, openexr, openjpeg2, openssl, phpmyadmin, python-psutil, python-twisted, QT, sdl2_image, SDL_image, sysstat, thunderbird, and tnef), Oracle (firefox), Red Hat (java-1.8.0-ibm and nss), Scientific Linux (firefox and kernel), SUSE (kernel), and Ubuntu (nss).

Linus has released the 5.5-rc1 kernelprepatch and closed the merge window for this development cycle. "Everything looks fairly regular - it's a tiny bit larger (in commitcounts) than the few last merge windows have been, but not biggerenough to really raise any eyebrows. And there's nothing particularlyodd in there either that I can think of: just a bit over half of thepatch is drivers, with the next big area being arch updates. Which ispretty much the rule for how things have been forever by now.Outside of that, the documentation and tooling (perf and selftests)updates stand out, but that's actually been a common pattern for awhile now too, so it's not really surprising either."

A "split lock" is a low-level memory-bus lock taken by the processor for a memoryrange that crosses a cache line. Most processors disallow split locks, butx86 implements them, Split locking may be convenient for developers, butit comes at a cost: a single split-locked instruction can occupy the memorybus for around 1,000 clock cycles. It is thus understandable that interestin eliminating split-lock operations is high. What is perhaps lessunderstandable is that a patch set intended to detect split locks has beenpending since (at least) May 2018, and it still is not poised to enter themainline.

William Tolley has disclosed a severe VPN-related problem in most currentsystems: "I am reporting a vulnerability that exists on most Linux distros, andother *nix operating systems which allows a network adjacent attackerto determine if another user is connected to a VPN, the virtual IPaddress they have been assigned by the VPN server, and whether or notthere is an active connection to a given website. Additionally, we areable to determine the exact seq and ack numbers by counting encryptedpackets and/or examining their size. This allows us to inject data intothe TCP stream and hijack connections." There are various partialmitigations available, but a full solution to the problem has not yet beenworked out. Most VPNs are vulnerable, but Tor evidently is not.

Security updates have been issued by Debian (libav), Fedora (kernel, libuv, and nodejs), Oracle (firefox), Red Hat (firefox and java-1.7.1-ibm), SUSE (clamav, cloud-init, dnsmasq, dpdk, ffmpeg, munge, opencv, and permissions), and Ubuntu (librabbitmq).

In November, the topic of init systems and, in particular, support forsystems other than systemd reappeared on theDebian mailing lists. After one month of sometimes fraught discussion,this issue has been brought to the project's developers to decide in theform of a general resolution (GR) — the first such since the project voted on the status ofdebian-private discussions in 2016. The issues under discussion arecomplex, so the result is one of the most complex ballots seen for sometime in Debian, with seven options to choose from.

Greg Kroah-Hartman has announced the release of the 5.4.2, 5.3.15,and 4.19.88 stable kernels. They contain arelatively large collection of important fixes throughout the tree; users of thosekernel series should upgrade.[Update: A bit later, the 4.14.158,4.9.206, and 4.4.206 stable kernels were also released.]

Security updates have been issued by Arch Linux (firefox), Fedora (cyrus-imapd, freeipa, haproxy, ImageMagick, python-pillow, rubygem-rmagick, sqlite, squid, and tnef), openSUSE (haproxy), Oracle (microcode_ctl), and Ubuntu (squid, squid3).

The LWN.net Weekly Edition for December 5, 2019 is available.

One of the features of the Clang/LLVM compiler that has been rather lackingfor GCC may finally be getting filled in. In a mid-November postto the gcc-patches mailing list, David Malcolm described a newstatic-analysis framework for GCC that he wrote. It could be the starting point for awhole range of code analysis for the compiler.

Making a comparison between Linux and Kubernetes is often one of apples tooranges. There are, however, some similarities and there is an effort within the Kubernetes community to make Kubernetes more like a Linuxdistribution. The idea was outlined in a session about Kubernetesrelease engineering at KubeCon+ CloudNativeCon North America 2019. "You might have heard thatKubernetes is the Linux of the cloud and that's like super easy to say, but what does it mean? Cloud is prettyfuzzy on its own," Tim Pepper, the Kubernetes release special interest group(SIG Release)co-chair said. He proceeded to provide some clarity on how the twoprojects are similar.

Security updates have been issued by CentOS (389-ds-base, ghostscript, kernel, and tcpdump), Debian (libonig), Fedora (clamav, firefox, and oniguruma), openSUSE (calamares, cloud-init, haproxy, libarchive, libidn2, libxml2, and ucode-intel), Scientific Linux (SDL and tcpdump), Slackware (mozilla), and Ubuntu (haproxy, intel-microcode, and postgresql-common).

ZDNet reportsthat two more malicious modules have been removed from the Python PackageIndex. "The two libraries were created by the same developer and mimicked other more popular libraries -- using a technique called typosquatting to register similarly-looking names.The first is 'python3-dateutil,' which imitated the popular 'dateutil'library. The second is 'jeIlyfish' (the first L is an I), which mimickedthe 'jellyfish' library." The latter of the two had been in PyPIfor nearly a year.

Firefox 71 is available. New features include improvements to the Lockwiseintegrated password manager and native MP3 decoding. The releasenotes have more details.

Security updates have been issued by Arch Linux (intel-ucode and libtiff), Debian (exiv2), Oracle (SDL), Red Hat (kernel, patch, and python-jinja2), and Ubuntu (graphicsmagick, linux, linux-aws, linux-aws-5.0, linux-gcp, linux-gke-5.0, linux-hwe, linux-kvm, linux-oem-osp1, linux-oracle, linux-oracle-5.0, linux-raspi2, linux, linux-aws, linux-aws-hwe, linux-gcp, linux-gke-4.15, linux-hwe, linux-kvm, linux-oem, linux-oracle, linux-raspi2, linux-snapdragon, linux, linux-aws, linux-gcp, linux-gcp-5.3, linux-kvm, linux-oracle, linux, linux-aws, linux-kvm, linux-raspi2, linux-snapdragon, linux-lts-xenial, linux-aws, and sqlite3).

Mark Wielaard has posted asummary of the discussion thus far on the governance of the GNUproject. "The mentoring and apprenticeship discussion focused on theGNU maintainers as being the core of the GNU project. But as was pointedout there are also webmasters, translators, infrastructure maintainers(partially paid FSF staff and volunteers), education and conferenceorganizers, etc. All these people are GNU stakeholders. And how we organizegovernance of the GNU project should also involve them."

The 5.5 merge window got underway immediately after the release of the 5.4 kernel onNovember 24. The first week has been quite busy despite the USThanksgiving holiday landing in the middle of it. Read on for a summary ofwhat the first 6,300 changesets brought for the next major kernel release.

Security updates have been issued by Debian (389-ds-base, asterisk, file, nss, proftpd-dfsg, ssvnc, and tnef), Fedora (chromium, djvulibre, freeradius, ImageMagick, jhead, kernel, phpMyAdmin, python-pillow, and rubygem-rmagick), Mageia (bzip2, chromium-browser-stable, curl, dbus, djvulibre, glib2.0, glibc, gnupg2, httpie, libreoffice, libssh2, mosquitto, nginx, python-sqlalchemy, unbound, and zipios++), openSUSE (bluez, clamav, cpio, freerdp, openafs, phpMyAdmin, strongswan, and webkit2gtk3), Red Hat (samba and SDL), Scientific Linux (389-ds-base), and SUSE (haproxy, python-Django, and tightvnc).

Version 7.4.0 of the PHP language has been released. New features includetypedproperties,arrowfunctions,weakreferences, and more; see the release announcementand migrationguide for more information.

The5.4.1,5.3.14,4.19.87,4.14.157,4.9.204, and4.4.204stable kernels have all been released; they contain a relatively large setof important fixes and updates. For good measure,4.9.205 and4.4.205followed a full 30 seconds later with one problematic patch reverted.

On the Redox site, creator Jeremy Soller gives an update on the Unix-like operating system written in Rust. It is running on a System76 Galaga Pro laptop: "This particular hardware has full support for the keyboard, touchpad, storage, and ethernet, making it easy to use with Redox." Meanwhile, he and the other Redox developers have been focusing on making it self-hosting: "Building Redox OS on Redox OS has always been one of the highest priorities of the project. Rustc seems to be only a few months of work away, after which I can begin to improve the system while running on it permanently, at least on one machine. With Redox OS being a microkernel, it is possible that even the driver level could be recompiled and respawned without downtime, making it incredibly fast to develop for. With this in place, I would work more efficiently on porting more software and tackling more hardware support issues, such as filling in the USB stack and adding graphics drivers.But, more importantly than what I will be able to do, is the contributions by others that will be unlocked by having a fully self-hosted, microkernel Operating System written in Rust, Redox OS."

Security updates have been issued by Debian (libvpx and vino), Fedora (grub2 and nss), and SUSE (cloud-init, libarchive, libtomcrypt, ncurses, and ucode-intel).

Security updates have been issued by Debian (haproxy and libvorbis), Fedora (mod_auth_mellon and xen), Oracle (389-ds-base, kernel, and tcpdump), SUSE (bsdtar, java-11-openjdk, java-1_7_0-openjdk, and libxml2), and Ubuntu (nss and python-psutil).

Security updates have been issued by Debian (bsdiff, libvpx, tiff, and xmlrpc-epi), Fedora (freeimage, imapfilter, kernel, mingw-freeimage, and thunderbird), openSUSE (cups and djvulibre), Oracle (SDL), SUSE (ardana-db, ardana-keystone, ardana-neutron, ardana-nova, crowbar-core, crowbar-openstack, crowbar-ui, openstack-barbican, openstack-heat-templates, openstack-keystone, openstack-neutron, openstack-neutron-gbp, openstack-neutron-lbaas, openstack-nova, openstack-octavia, openstack-sahara, python-psutil, release-notes-suse-openstack-cloud, freerdp, mailman, and slurm), and Ubuntu (ruby2.3, ruby2.5).

The Linux kernel scheduler is a complicated beastand a lot of effort goes into improving it during every kernel releasecycle. The 5.4 kernel release includes a few improvements to the existingSCHED_IDLE scheduling policy that can help users improve thescheduling latency of their high-priority (interactive) tasks if they usethe SCHED_IDLE policy for the lowest-priority (background)tasks. Read on for a description of this work contributed by Viresh Kumar.

Security updates have been issued by Debian (libxdmcp, nss, php-imagick, and ruby2.1), openSUSE (java-11-openjdk), Red Hat (389-ds-base, kernel, kernel-rt, python-jinja2, qemu-kvm-ma, and tcpdump), SUSE (bluez, clamav, cpio, cups, gcc9, libpng16, libssh2_org, mailman, sqlite3, squid, strongswan, tiff, and webkit2gtk3), and Ubuntu (redmine).

Stable kernels 5.3.13, 4.19.86, 4.14.156, 4.9.203, and 4.4.203 have been released. They all containimportant fixes and users should upgrade.

Security updates have been issued by Debian (chromium, enigmail, isc-dhcp, libice, libofx, and pam-python), Fedora (chromium, ghostscript, mingw-cfitsio, mingw-gdal, mingw-libidn2, and rsyslog), Gentoo (adobe-flash, chromium, expat, and firefox), openSUSE (apache2-mod_perl, haproxy, java-11-openjdk, and ncurses), Oracle (ghostscript, kernel, php:7.2, php:7.3, and sudo), Red Hat (chromium-browser, python27-python, and SDL), and Ubuntu (dpdk and libvpx).

Linus has released the 5.4 kernel."Not a lot happened this last week, which is just how I likeit". Significant features in this release includethe haltpollCPU governor,the iocost (formerly io.weight) I/Ocontroller,the EROFS filesystem,an implementation of the exFAT filesystemthat may yet be superseded by a better version,the fs-verity file integrity mechanism,support for the BPFcompile once, run everywhere mechanism,the dm-clonedevice mapper target,the virtiofsfilesystem,kernel lockdown support (at last),kernel symbol namespaces, and a newrandom-number generator meant to solve theearly-boot entropy problem.See the KernelNewbies 5.4page for a lot more details.

When virtiowas merged in Linux v2.6.24, its author, Rusty Russell, described the goal as being for "common drivers to be efficiently usedacross most virtual I/O mechanisms". Today, much progress has been made toward that goal, with virtiosupported by multiple hypervisors and guest drivers shipped by many operatingsystems. But these applications of virtio are implemented in software, whereasMichael Tsirkin's "VirtIOwithout the Virt" talk at KVM Forum 2019 laid out howto implement virtio in hardware.

Security updates have been issued by Fedora (dpdk, mingw-djvulibre, mingw-hunspell, mingw-ilmbase, mingw-OpenEXR, php-symfony, php-symfony3, and rsyslog), openSUSE (chromium and squid), SUSE (aspell, cups, djvulibre, and dpdk), and Ubuntu (djvulibre).