LWN.net

Stable kernels 4.16.4, 4.14.36, 4.9.96, 4.4.129, and 3.18.106 have been released. All of themcontain important fixes and users should update.

Security updates have been issued by Arch Linux (roundcubemail, xfig, and zsh), Debian (linux-tools), Fedora (java-1.8.0-openjdk and mingw-libid3tag), Gentoo (chromium), openSUSE (hdf5, ocaml, PackageKit, phpMyAdmin, salt, and virtualbox), Oracle (patch), Red Hat (java-1.6.0-sun, java-1.7.0-oracle, java-1.8.0-oracle, patch, and python-paramiko), Scientific Linux (patch), SUSE (kernel and PackageKit), and Ubuntu (linux, linux-aws, linux-kvm, linux-raspi2, linux-snapdragon, linux, linux-raspi2, linux-azure, linux-euclid, linux-hwe, linux-gcp, linux-oem, linux-lts-xenial, linux-aws, and mysql-5.5, mysql-5.7).

It is a good thing that strong coffee was served at the 2018 Linux Storage,Filesystem, and Memory-Management Summit; full awareness was required fromthe first session, in which Josef Bacik discussed some issues that havearisen in the interaction between filesystems and the memory-managementsubsystem. Filesystems cache a lot of data from files, but also a lot ofmetadata about those files. It turns out, though, that management of thecached metadata does not work as well as one might like.

At the 2018 Legal andLicensing Workshop (LLW), which is a yearly gathering of lawyers and technical folks organized by the Free Software FoundationEurope (FSFE), attendees got more details on a recent hearing in a German GPLenforcement case. Marcus von Welser is a lawyer who represented thedefendant, Geniatech, in a case that was brought by PatrickMcHardy. In the presentation, von Welser was joined by Armijn Hemel, who helped Geniatech in its compliance efforts. The hearingwas of interest for a number of reasons, not least because McHardywithdrew his request for an injunction once it became clear that the judgewas leaning infavor of the defendants—effectively stopping this case dead in its tracks.

Daniel Vetter looks atsome kernel-development statistics, with a focus on patches written bythe maintainers who commit them. "Naively extrapolating the relative trend predicts that around the year 2025 large numbers of kernel maintainers will do nothing else than be the bottleneck, preventing everyone else from getting their work merged and not contributing anything of their own. The kernel community imploding under its own bureaucratic weight being the likely outcome of that.This is a huge contrast to the 'everything is getting better, bigger, andthe kernel community is very healthy' fanfare touted at keynotes and theyearly kernel report. In my opinion, the kernel community is very much notlooking like it is coping with its growth well and an overall healthycommunity."

Each kernel development cycle includes a vast number of changes that arenot intended to change visible behavior and which, as a result, gounnoticed by most users and developers. One such change in 4.17 is arewiring of how system-call implementations are invoked within the kernel.The change is interesting, though, and provides an opportunity to look atthe macro magic that handles system-call definitions.

Security updates have been issued by Debian (gunicorn, libreoffice, libsdl2-image, ruby1.8, and ruby1.9.1), Fedora (java-1.8.0-openjdk, jgraphx, memcached, nghttp2, perl, perl-Module-CoreList, and roundcubemail), Gentoo (clamav, librelp, mbedtls, quagga, tenshi, and unadf), Mageia (freeplane, libcdio, libtiff, thunderbird, and zsh), openSUSE (cfitsio, chromium, mbedtls, and nextcloud), and Red Hat (chromium-browser, kernel, and rh-perl524-perl).

The 4.17-rc2 kernel prepatch is out."We've still got some known fallout from the merge window, but itshouldn't affect most normal configurations, so go out and test."

The first article in this series describedthe interface to the "rhashtable"resizable hash-table abstraction in Linux 4.15. While a knowledge ofthe interface can result in successful use of rhashtables, it oftenhelps to understand what is going on "under the hood", particularly whenthose details leak out through the interface, as is occasionally thecase with rhashtable. The centerpiece for understanding theimplementation is knowing exactly how the table is resized. So thisfollow-on article will explain that operation; it will also present theconfiguration parameters that were skimmed over last time and discusshow they affect the implementation.

Version 4.0 of the FFmpegmultimedia toolkit is out. There is a long list of new filters, formats,and more; see the announcement for details.

The furor over the Meltdown and Spectre vulnerabilities has calmed a bit —for now, at least — but that does not mean that developers have stoppedworrying about them. Spectre variant 1 (the bounds-check bypassvulnerability) has been of particular concern because, while the kernel isthought to contain numerous vulnerable spots, nobody really knows how tofind them all. As a result, the defenses that have been developed forvariant 1 have only been deployed in a few places. Recently, though,Dan Carpenter has enhanced the smatch tool to enable it to find possiblyvulnerable code in the kernel.

Greg Kroah-Hartman has released stable kernel 4.9.95 with important fixes throughout thetree. Users should update.

Security updates have been issued by Debian (libreoffice and mysql-5.5), Fedora (corosync), Oracle (java-1.8.0-openjdk), Red Hat (java-1.8.0-openjdk), Scientific Linux (java-1.8.0-openjdk), and Ubuntu (openssl).

In the performance-conscious world of high-speed networking, anything thatcan be done to avoid copying packet data is welcome. The MSG_ZEROCOPY feature added in 4.14enables zero-copy transmission of data, but does not address the receiveside of the equation. It now appears that the 4.18 kernel will include a zero-copy receive mechanism by Eric Dumazetto close that gap, at least for some relatively specialized applications.

Stable kernels 4.16.3, 4.15.18, and 4.14.35 have been released. This is the last4.15.y kernel and users should move to 4.16.y.

Security updates have been issued by Debian (opencv and wireshark), Fedora (corosync and pcs), Oracle (firefox, kernel, libvncserver, and libvorbis), Slackware (gd), SUSE (kernel), and Ubuntu (apache2).

The LWN.net Weekly Edition for April 19, 2018 is available.

It is normally the grumpy editor's job to lookat accounting software; he does so with an eye toward getting the business off of theproprietary QuickBooks application and moving to something free. It may bethat Beancount deserves a look ofthat nature before too long but, in the meantime, a slightly less grumpyeditor has been messing with this text-based accounting tool for a varietyof much smaller projects. It is an interesting system, with a lot ofcapabilities, but its reliance on hand-rolling for various piecesmay scare some folks off.

The release of pip 10.0 has been announced. Some highlights of thisrelease include the removal of Python 2.6 support, limited PEP 518 support (withmore to come), a new "pip config" command, and other improvements.

The new PyPI has been launched. Browsertraffic and API calls (including "pip install") have been redirected fromthe old pypi.python.org to the new site. The old PyPI will shut down onApril 30. LWN covered the new PyPI last week.

Developers of database management systems are, by necessity, concernedabout getting data safely to persistent storage. So when the PostgreSQLcommunity found out that the way the kernel handles I/O errors could resultin data being lost without any errors being reported to user space, a fairamount of unhappiness resulted. The problem, which is exacerbated by theway PostgreSQL performs buffered I/O, turns out not to be unique to Linux,and will not be easy to solve even there.

Security updates have been issued by Debian (freeplane and jruby), Fedora (kernel and python-bleach), Gentoo (evince, gdk-pixbuf, and ncurses), openSUSE (kernel), Oracle (gcc, glibc, kernel, krb5, ntp, openssh, openssl, policycoreutils, qemu-kvm, and xdg-user-dirs), Red Hat (corosync, glusterfs, kernel, and kernel-rt), SUSE (openssl), and Ubuntu (openssl and perl).

Security updates have been issued by Debian (corosync, linux-tools, qemu, qemu-kvm, and r-cran-readxl), openSUSE (evince, memcached, nodejs4, ntp, pdns-recursor, python-gunicorn, python3-gunicorn, and python3), and Ubuntu (ruby1.9.1, ruby2.0, ruby2.3).

Microsoft has issued apress release describing the security dangers involved with theInternet of things ("a weaponized stove, baby monitors that spy, thecontents of your refrigerator being held for ransom") and introducing"Microsoft Azure Sphere" as a combination of hardware and software toaddress the problem. "Unlike the RTOSes common to MCUs today, ourdefense-in-depth IoT OS offers multiple layers of security. It combinessecurity innovations pioneered in Windows, a security monitor, and a customLinux kernel to create a highly-secured software environment and atrustworthy platform for new IoT experiences."

Alpine Linux-based postmarketOS is touch-optimized and pre-configured forinstallation on smartphones and other mobile devices. The postmarketOSblog introducespostmarketOS-lowlevel which is a community project aimed at creatingfree bootloaders and cellular modem firmware, currently focused on MediaTekphones. "But before we get started, please keep in mind that theseare moon shots. So while there is some little progress, it's mostly aboutletting fellow hackers know what we've tried and what we're up to, in thehopes of attracting more interested talent to our cause. After all, ourphilosophy is to keep the community informed and engaged during thedevelopment phase!"

Security updates have been issued by Arch Linux (lib32-openssl and zsh), Debian (patch, perl, ruby-loofah, squirrelmail, tiff, and tiff3), Fedora (gnupg2), Gentoo (go), Mageia (firefox, flash-player-plugin, nxagent, puppet, python-paramiko, samba, and thunderbird), Red Hat (flash-plugin), Scientific Linux (python-paramiko), and Ubuntu (patch, perl, and ruby).

Version 1.10 of the Subversion version-control system is out.Improvements include a new interactive resolver for merge conflicts, betterpath-based authorization, LZ4 compression, and more; see therelease notes for details.

By the time the 4.17 merge window was closed and 4.17-rc1 was released, 11,769 non-merge changesets had been pulled into themainline repository. 4.17 thus looks to be a typically busy developmentcycle, with a merge window only slightly more busy than 4.16 had.Some 6,000 of those changes were pulled after last week's summary was written. There was alot of the usual maintenance work in those patches (over 10% of thosechanges were to device-tree files, for example), but also some moresignificant changes.

Linus has released 4.17-rc1 and closed themerge window for this release. "This does not seem to be shaping upto be a particularly big release, and there seems to be nothingparticularly special about it. The most special thing that happened ispurely numerology: we've passed the six million git objects mark, and thatis reason enough to call the next kernel 5.0. Except I probably won't,because I don't want to be too predictable."

A comparison of the feature sets for a handful of terminal emulators wasthe subject of a recent article; here I follow that up byexamining the performance of those terminals. This might seem like alesser concern, but as it turns out, terminals exhibit surprisinglyhigh latency for such fundamental programs. I also examine what istraditionally considered "speed" (but is really scroll bandwidth) andmemory usage, with the understanding that the impact of memory useis less than it was when I looked at this a decade ago (inFrench).Subscribers can read on for part 2 from guest author Antoine Beaupré.

The stable kernel train just keeps on rolling; Greg Kroah-Hartman has announcedthe release of the 4.9.94, 4.4.128, and 3.18.105 stable kernels. All contain a largenumber of fixes throughout the tree and users should upgrade.

The rhashtable data structure is a generic resizable hash-tableimplementation in the Linux kernel, which LWN first introduced as "relativistichash tables" back in 2014. I thought at the time that it might be fun to makeuse of rhashtables, but didn't, until an opportunity arose through my work onthe Lustre filesystem. Lustre is a cluster filesystem that is currently indrivers/staging while the code is revised to meet upstreamrequirements. One of those requirements is to avoid duplicatingsimilar functionality where possible. As Lustre contains a resizablehash table, it really needs to be converted to use rhashtables instead — atlast I have my opportunity.Subscribers can read on for a look at the rhashtable API by guest authorNeil Brown.

Security updates have been issued by Arch Linux (apache), openSUSE (libvirt, openssl, policycoreutils, and zziplib), Oracle (firefox and python-paramiko), and Red Hat (python-paramiko).

Greg Kroah-Hartman has released three new stable kernels: 4.16.2, 4.15.17, and 4.14.34. Users of those kernel series shouldupgrade.

Security updates have been issued by Debian (poppler), Fedora (koji and libofx), Gentoo (adobe-flash), Oracle (kernel), Red Hat (qemu-kvm-rhev and sensu), and Scientific Linux (firefox).

The LWN.net Weekly Edition for April 12, 2018 is available.

A "simple" utility to make a system beep is hardly the first place one wouldcheck for security flaws, but the strange case of the "Holey Beep"should perhaps lead to some rethinking. A Debian advisory for the beep utility, which was followedby another for Debian LTS, led to aseemingly satirical site publicizingthe bug (and giving it the "Holey Beep" name). But that site also exploitsa new flaw in the GNUpatch program—and the increased scrutiny on beep hasled to more problems being found.

The Python Package Index (PyPI) isthe principal repository of libraries for the Python programming language,serving more than 170 million downloads each week. Fifteen years after PyPIlaunched, a new edition is in beta at pypi.org, with features like bettersearch, a refreshed layout, and Markdown README files(and with some old features removed, like viewing GPG package signatures). StartingApril 16, users visiting the site or running pip install willbe seamlessly redirected to the new site. Two weeks after that, the legacy site isexpected to be shut down and the team will turn toward newfeatures; in the meantime, it is worth a look at what the new PyPI bringsto the table.

Security updates have been issued by Debian (pcs), Fedora (drupal7), openSUSE (git and mercurial), Red Hat (firefox and qemu-kvm-rhev), SUSE (libvirt and xen), and Ubuntu (patch).

Car manufacturers, like most companies, navigate a narrow lane between thebenefits of using free and open-source software and the perceived or realimportance of hiding their trade secrets. Many are usingfree software in some of the myriad software components that make up amodern car, and even work in consortia to develop free software. At therecent LibrePlanetconference, free-software advocate Jeremiah Foster covered progress in theautomotive sector and made an impassioned case for more free software in theirembedded systems.Subscribers can read on for a report on the talk by guest author Andy Oram.

Red Hat has announcedthe general availability of Red Hat Enterprise Linux 7.5. This versionfeatures enhanced hybrid cloud security and compliance, improved storageperformance and efficiency, simplified management, and production-readyLinux containers. RHEL 7.5 is available for x86, IBM Power, IBM z Systems, and 64-bit Arm. This release also brings support for single-host KVM virtualization and Open Container Initiative (OCI)-formatted runtime environment and base image to IBM z Systems.

The 3.18.104 kernel has been released witha single bugfix. If you had build errors in 3.18.103 then this update isfor you, otherwise there is no need to upgrade.

Security updates have been issued by CentOS (libvorbis and thunderbird), Debian (pjproject), Fedora (compat-openssl10, java-1.8.0-openjdk-aarch32, libid3tag, python-pip, python3, and python3-docs), Gentoo (ZendFramework), Oracle (thunderbird), Red Hat (ansible, gcc, glibc, golang, kernel, kernel-alt, kernel-rt, krb5, kubernetes, libvncserver, libvorbis, ntp, openssh, openssl, pcs, policycoreutils, qemu-kvm, and xdg-user-dirs), SUSE (openssl and openssl1), and Ubuntu (python-crypto, ubuntu-release-upgrader, and wayland).

Jim Gettys refutesthe claim that the early designers of Internet software were notconcerned about security. "Government export controls crippledInternet security and the design of Internet protocols from the verybeginning: we continue to pay the price to this day".

Several security vulnerabilities were found in Etherpad and version1.6.4 has been released with fixes. The vulnerabilities includearbitrary code execution and information disclosure. Site admins are urgedto update Etherpad to 1.6.4 as soon as possible.

Security updates have been issued by Arch Linux (openssl and zziplib), Debian (ldap-account-manager, ming, python-crypto, sam2p, sdl-image1.2, and squirrelmail), Fedora (bchunk, koji, libidn, librelp, nodejs, and php), Gentoo (curl, dhcp, libvirt, mailx, poppler, qemu, and spice-vdagent), Mageia (389-ds-base, aubio, cfitsio, libvncserver, nmap, and ntp), openSUSE (GraphicsMagick, ImageMagick, spice-gtk, and wireshark), Oracle (kubernetes), Slackware (patch), and SUSE (apache2 and openssl).

The Linux network stack does not lack for features; it also performs wellenough for most uses. At the highest network speeds, though, any overheadat all is too much; that has driven the most demanding users towardspecialized, user-space networking implementations that can outperform thekernel for highly constrained tasks. The express data path (XDP)development effort is an attempt to win those users back, with some apparentsuccess so far. With the posting of the AF_XDP patch set by Björn Töpel,another piece of the XDP puzzle is coming into focus.

The4.16.1,4.15.16,4.14.33,4.9.93,4.4.127, and3.18.103stable kernels have all been released; each contains a fairly long list ofimportant fixes.

As the 4.17 merge window opened, it seemedpossible that the kernel lockdown patch set could be merged at last.That was before the linux-kernel mailing list got its hands on the issue.What resulted was not one of the kernel community's finest moments. But itdid result in a couple of evident conclusions: kernel lockdown will almostcertainly not bemerged for 4.17, but something that looks very much like it is highlylikely to be accepted in a subsequent merge window.

Security updates have been issued by Debian (sharutils), Fedora (firefox, httpd, and mod_http2), openSUSE (docker-distribution, graphite2, libidn, and postgresql94), Oracle (libvorbis and thunderbird), Red Hat (libvorbis, python-paramiko, and thunderbird), Scientific Linux (libvorbis and thunderbird), SUSE (apache2), and Ubuntu (firefox, linux-lts-xenial, linux-aws, and ruby1.9.1, ruby2.0, ruby2.3).