LWN.net

Security updates have been issued by Debian (ruby-sprockets), Red Hat (ansible and rh-git29-git), Scientific Linux (firefox), SUSE (ceph), and Ubuntu (libjpeg-turbo, ntp, and openslp-dfsg).

In the 4.18 kernel, a new feature was merged to allow infrared (IR)decoding to be done using BPF. Infrared remotes use many differentencodings; if a decoder were to be written for each, we would end up withhundreds of decoders in the kernel. So, currently, the kernel only supportsthe most widely used protocols. Alternatively, the lirc daemon canbe run to decode IR. Decoding IR can usually be expressed in a few lines ofcode, so a more lightweight solution without many kernel-to-userspacecontext switches would be preferable. This article will explain how IRmessages are encoded, the structure of a BPF program, and how a BPF programcan maintain state between invocations. It concludes with a look at thesteps that are taken to end up with a button event, such as a volume-up keyevent.

Security updates have been issued by Debian (bouncycastle and ca-certificates), Fedora (cantata, cinnamon, php-symfony3, and transifex-client), openSUSE (ghostscript, openssl, openvpn, php7, rubygem-yard, thunderbird, ucode-intel, and unzip), and SUSE (libqt4, nodejs8, and openslp).

The 4.18-rc4 kernel prepatch has beenreleased. "Things look pretty normal here, and size-wise this looksgood too, so it's another of those 'solid progress to release'weeks. Boring is good."

The 4.17.5 and 4.14.54 stable kernels have been released withyet another set of important fixes.

For those with a significant chunk of spare time and nothing better to do:Swapnil Bhartiya interviewed LWN editor Jonathan Corbet in February has now posted the resulting video onthe Patreon site.

Security updates have been issued by Debian (dokuwiki, libsoup2.4, mercurial, php7.0, and phpmyadmin), Fedora (ant, gnupg, libgit2, and libsoup), openSUSE (cairo, git-annex, postgresql95, and zsh), Scientific Linux (firefox), Slackware (mozilla), SUSE (nodejs6 and rubygem-yard), and Ubuntu (AMD microcode, devscripts, and firefox).

Large data centers routinely use control groups to balance the use of theavailable computing resources among competing users. Block I/O bandwidthcan be one of the most important resources for certain types of workloads,but the kernel's I/O controller is not a complete solution to the problem.The upcoming block I/O latency controllerlooks set to fill that gap in the near future, at least for some classes ofusers.

Security updates have been issued by Oracle (firefox), SUSE (exiv2, ghostscript, libvorbis, openssl, openvpn, php7, tiff, and unzip), and Ubuntu (libarchive-zip-perl and php7.2).

The LWN.net Weekly Edition for July 5, 2018 is available.

LWN reported on June 29 that Gentoo'sGitHub mirror had been compromised. Gentoo now considers the incidentresolved and the full report isavailable. "An unknown entity gained control of an admin account for the Gentoo GitHub Organization and removed all access to the organization (and its repositories) from Gentoo developers. They then proceeded to make various changes to content. Gentoo Developers & Infrastructure escalated to GitHub support and the Gentoo Organization was frozen by GitHub staff. Gentoo has regained control of the Gentoo GitHub Organization and has reverted the bad commits and defaced content."

The venerable Linux boot menu has made its appearance at boot time sincethe days when LILO was thestandard boot loader, through the days of GRUB, and onward totoday's GRUB 2and others.It is sometimes configured out by distributions as something that willpotentially confuse less-technical users, but it has been a mainstay ofFedora for many releases. A recent proposalto hide the menu, starting in Fedora 29, has met a mixed reaction, butthose who are not in favor are also those most able to revert to theexisting behavior.

Security updates have been issued by Arch Linux (git-annex and gitlab), CentOS (kernel, libvirt, pki-core, and qemu-kvm), Debian (cups, exiv2, and gosa), Fedora (ant, drupal7-backup_migrate, glusterfs, libsoup, standard-test-roles, and xen), Oracle (firefox and python), Scientific Linux (python), SUSE (cairo, git, and zsh), and Ubuntu (exiv2, libsoup2.4, and php7.2).

Dan Callahan is a developer advocate at Mozilla and no stranger toPyCon (we covered a talk of his at PyCon2013). He was also the champion at Mozilla for the grant that helped revamp the Python Package Index (PyPI). AtPyCon 2018, he gave a keynote talk [YouTube video] that focused on platforms of varioussorts—and where Python fits into the platforms of the future.

There are many things to love about the Linux Plumbers Conference (LPC),but the event's web site has not often been considered one of them.This year, your editor took on the task of finding a new system to handleproposal submission, review, and scheduling, despite his own poor trackrecord when it comes to creating attractive web sites. The search finallysettled on a system called Indico; readon for some impressions of this interesting free event-management system.

Stable kernels 4.17.4, 4.14.53, 4.9.111, 4.4.139, and 3.18.114 have been released. They all containimportant fixes and users should upgrade.

Security updates have been issued by Fedora (drupal7-backup_migrate, firefox, and podman), Red Hat (python), Scientific Linux (glibc, kernel, libvirt, pcs, samba, samba4, sssd and ding-libs, and zsh), and Ubuntu (kernel, linux, linux-aws, linux-azure, linux-gcp, linux-kvm, linux-oem, linux-raspi2, linux, linux-aws, linux-kvm, linux-raspi2, linux-snapdragon, linux, linux-raspi2, linux-hwe, linux-azure, linux-lts-trusty, linux-lts-xenial, linux-aws, linux-oem, and zziplib).

For many years, Linux system administrators have gone out of their way toavoid swapping. The advent of nonvolatile memory is changing the equation,though, and swapping is starting to look interesting again — if it canperform well enough. That is not the case in current kernels, but alongstanding project to allow the swapping of transparent huge pagespromises to improve that situation considerably. That work is reaching itsfinal stage and might just enter the mainline soon.

Security updates have been issued by Debian (chromium-browser, mosquitto, python-pysaml2, simplesamlphp, tiff, and tomcat7), Fedora (kernel, libgxps, nodejs, and phpMyAdmin), Mageia (ansible, firefox, java-1.8.0-openjdk, libcrypt, libgcrypt, ncurses, phpmyadmin, taglib, and webkit2), openSUSE (GraphicsMagick, ImageMagick, mailman, Opera, and rubygem-sprockets), and SUSE (ImageMagick, kernel, mariadb, and python-paramiko).

SUSE has announcedthat it has been acquired again — this time by an investment company calledEQT. Some more bright-future talk can be found in this blogentry: "In keeping with our 25-year history, SUSE intends toremain committed to an open source development and business model andactively participate in communities and projects to bring open sourceinnovation to the enterprise as high quality, reliable and usablesolutions. Our truly open, open source model, where open refers to thefreedom of choice provided to customers and not just the code used in oursolutions, is embedded in SUSE culture, differentiates us in the marketplace and has been key to our years of success."

The 4.18-rc3 kernel prepatch is out fortesting. "And things look fairly normal - a bit over a third isdrivers (various parts: input, usb, sound, gpu, mtd, networking, ...), withthe rest being arch updates (arm/arm64 - dts files dominating, microblaze,powerpc, x86), filesystems (xfs, some btrfs, some minor core vfs),networking, documentation updates and tooling."

Lubomir Rintel has a look at the NetworkManager 1.12 release on his blog. There are lots of new features, including WiFi improvements (FILS, Wake on WLAN, IWD), colored nmcli output, removal of some unloved code, and checkpoint/restore:"One of the lesser known goodies provided by NetworkManager is the checkpoint/restore functionality. It allows the user to roll back to a working network configuration if any changes render a machine inaccessible over a network.The user needs to define a checkpoint first, then conduct the potentially dangerous changes and finally confirm that the changes didn’t disrupt connectivity. A checkpoint is essentially a snapshot of an active network configuration along with a timer. Should the changes cause a networking outage, the timer expires before the user can confirm success and the changes are reverted, hopefully restoring connectivity."

On his blog, Cornelius Schumacher writes about some lessons he's learned about governance in nearly 20 years of experience with open-source projects. "Governance is important. Your project does have a governance model even if you don't think about it or if you don't write down the rules. It governs how your project will work and how people will be able to collaborate. It will also define a big part of your culture. You don't want to leave these things to chance. So be conscious about governance.That doesn't mean that you have to write rules and policies for everything. Often a healthy culture where people learn by following the example of the leaders and other members of the community works well. It might be tempting to create a formal structure to cover all [kinds] of possible scenarios. But creating and maintaining policies is an expensive process. Don't be formal where you are not sure it's needed."

One of the the most commonly repeated mistakes in system-call design is a failure to check for unknown flags whereverflags are accepted. If there is ever a point where callers can get awaywith setting unknown flags, then adding new flags becomes a hazardous act.In the case of mmap(),though, developers found a clever way around this problem. A recentdiscussion has briefly called that approach into question, though, andraised the issue of what constitutes a kernel regression. No changes areforthcoming as a result, but the discussion does provide an opportunity tolook at both the specific hack and how the kernel community decides whethera change is a regression or not.

Linux Mint 19 "Tara" has been released in Cinnamon and MATE editions. See moreabout new features in Tara for Cinnamonand MATEand the release notes (Cinnamon, MATE) for additionaldetails. "In Linux Mint 19, the star of the show is Timeshift. Although it was introduced in Linux Mint 18.3 and backported to all Linux Mint releases, it is now at the center of Linux Mint's update strategy and communication.Thanks to Timeshift you can go back in time and restore your computer to the last functional system snapshot. If anything breaks, you can go back to the previous snapshot and it's as if the problem never happened."

Brett Cannon is a longtime Python core developer and member of the open-sourcecommunity. He got to check off one of his bucket-list items when he gave akeynote [YouTube video]at PyCon 2018. That keynote was a rather personal look at what he sees as some problem areas in the expectations of the users of open-sourcesoftware with respect to those who produce it. While there is lots to behappy for in the open-source world, there are some sharp edges (and worse)that need filing down.

Security updates have been issued by Arch Linux (firefox), Debian (firefox-esr, lava-server, libgcrypt20, mariadb-10.0, and zendframework), Fedora (firefox, podman, webkitgtk4, and xen), openSUSE (procps and unixODBC), Oracle (pki-core), Red Hat (firefox), SUSE (kernel, procps, and tomcat6), and Ubuntu (file and nasm).

The Gentoo project's GitHub account hasbeen compromised and used to distribute malicious code. "Thisdoes NOT affect any code hosted on the Gentoo infrastructure. Since themaster Gentoo ebuild repository is hosted on our own infrastructure andsince Github is only a mirror for it, you are fine as long as you are usingrsync or webrsync from gentoo.org."

Worth a read: Geoff Huston'swriteup of how the net has evolved over the last ten years and where itmay be going. "Perhaps this increased resistance to change isbecause as the size of the network increases, its inertial mass alsoincreases. We used to quote Metcalf’s Law to each other, reciting themantra that the value of a network increases in proportion to the square ofthe number of users. A related observation appears to be that a network’sinherent resistance to change, or inertial mass, is also directly relatedto the square of the number of users as well."

Lennart Poettering describesthe new systemd "portable service" feature, which appears to be a new takeon containers. "Both resource bundling and isolation/sand-boxing areconcepts systemd has implemented to varying degrees for a longertime. Specifically, RootDirectory= and RootImage= have been around for along time, and so have been the various sand-boxing features systemdprovides. The Portable Services concept builds on that, putting thesefeatures together in a new, integrated way to make them more accessible andusable."

The latest feature release of Python, 3.7.0, hasbeen announced, along with the latest maintenance release forPython 3.6, 3.6.6.As noted on the "What's New InPython 3.7" page, there are many significant changes in therelease. These include postponedevaluation of type annotations (for performance and simpler forwardreferences to types), insertion-order preservation for dict objects is nowpart of the language, data classes havebeen added, async and await are now keywords,there are usability and performance improvements for asyncio, a newC API for thread-local storage has been added, and more. 3.7 will getbug fix updates until shortly after 3.8 is released (in roughly 18 months) and then willget security updates until mid-2023.3.6 will continue to get bug fixes through the end of 2018 and securityfixes into 2021.

Security updates have been issued by Debian (exiv2, firefox-esr, graphicsmagick, php-horde-crypt, ruby-passenger, tomcat7, and xen), Fedora (dcraw, file, kernel-tools, and mupdf), openSUSE (firefox and tiff), Oracle (kernel, libvirt, pki-core, and qemu-kvm), Red Hat (patch), SUSE (jpeg, python-Django, tiff, and unixODBC), and Ubuntu (jasper).

The LWN.net Weekly Edition for June 28, 2018 is available.

While there has been quite a bit of work on various aspects of networkingperformance, including bufferbloat reduction, queue management, and more,much of that work has been oriented toward the needs of high-end users.But there is more to the Internet than data centers and high-speed links.A large number of Internet-connected devices can be found behindconsumer-level routers on relatively slow broadband links. For some time,a group of developers has been working on the "Common Applications KeptEnhanced" (CAKE) queuing discipline, which is aimed directly at the needsof those users.

Collabora introducesdebos, a tool to build customized images for Debian and derivatives. "As opposite to debootstrap and other tools, debos doesn't need to be run as root for making actions that require root privileges in the images. debos uses fakemachine a library that setups qemu-system allowing you to work in the image with root privileges and to create images for all the architectures supported by qemu user."

Python 3 adoption has clearly picked up over the last few years, thoughthere is still a long way to go. Big Python-using companies tend to have awhole lot of Python 2.7 code running on their infrastructure and Facebook isno exception. But Jason Fried came to PyCon 2018 to describe what hashappened at the company over the last four years or so—it has gone fromusing almost no Python 3 to it becoming the dominant version of Pythonin the company. He was instrumental in helping to make that happen and his talk [YouTube video]may provide other organizations with some ideas on how to tackle theirmigration.

Security updates have been issued by Arch Linux (cantata and qutebrowser), Debian (imagemagick, php5, and redis), Fedora (cri-o and libgxps), Oracle (glibc, kernel, libvirt, samba, samba4, sssd and ding-libs, and zsh), Red Hat (ansible, dpdk, kernel, kernel-alt, kernel-rt, libvirt, pki-core, podman, qemu-kvm, and qemu-kvm-rhev), Scientific Linux (kernel, libvirt, pki-core, and qemu-kvm), SUSE (firefox, gcc43, and kernel), and Ubuntu (openssl).

One of the new features merged for the 4.18 kernel is a new polling interface using the asynchronousI/O mechanism. As part of this work, the internal implementation of howthe various polling-related system calls (poll(),select(), and epoll_wait()) work was significantlychanged. The reporting of a significant performance regression has now putall of that work into doubt, though. While it could be reverted, the morelikely outcome would appear to be another set of changes to how pollingworks in the kernel.

Mozilla has announcedthe release of Firefox 61. Key highlights include the ability to easilyadd custom search engines, speedier response times when switching betweentabs, retained display lists, an accessibility Tools Inspector, andWebExtension Tab Management. See the releasenotes for additional information.

The combination of an "unsuspecting library employee" and a bunch of boredchildren has created a popular program using the Raspberry Pi and othertools to teachcoding to kids. Qumisha Goss is a librarian at the Parkman branch of theDetroit Public Library; she started the "Parkman Coders" program and came to PyCon 2018 in Cleveland, Ohio to tell the assembled Pythonistas all about it. She also hadsome thoughts on ways to make the Python community a more diverse place,along with some concerns for her students that are much bigger thanthe diversity topic.

Security updates have been issued by Slackware (firefox), SUSE (gpg2 and zlib), and Ubuntu (openssl, openssl1.0).

The latest set of stable kernel updates consists of4.17.3,4.16.18,4.14.52, and4.9.110. Each contains a fair number ofimportant updates. Note that 4.16.18 is the end of the line for the 4.16series.

As attackers have lost the easy ability to execute code stored in writablememory, they have increasingly turned to return-orientedprogramming (ROP) and related techniques to compromise vulnerablesystems. ROP attacks use the code that is present in the program underattack and are hard to defend against in software. In response, hardwarevendors are developing ways to defeat ROP-like techniques at a lowerlevel. One of the results is Intel's Control-FlowEnforcement Technology (CET) [PDF], which adds two mechanisms (shadowstacks and indirect-branch tracking) that are intended to resist theseattacks. Yu-cheng Yu recently posted a set of patches showing how this technology is to be used to defend Linuxsystems.

SUSE has announcedthe release of SUSE Linux Enterprise 15, SUSE Manager 3.2, and SUSE LinuxEnterprise High Performance Computing 15. "SUSE Linux Enterprise 15 is a modern, modular operating system that helps simplify multimodal IT, makes traditional IT infrastructure more efficient and provides an engaging platform for developers. As a result, customers can easily deploy and transition business-critical workloads across on-premise and public cloud environments."

Security updates have been issued by CentOS (git), Debian (bouncycastle and lava-server), Fedora (ansible, epiphany, kernel, kernel-tools, matrix-synapse, mingw-podofo, pass, podofo, python-prometheus_client, redis, rubygem-sinatra, and thunderbird-enigmail), Gentoo (file and pnp4nagios), Mageia (file, glibc, kernel, librsvg, and libvorbis), openSUSE (go1.9, mariadb, phpMyAdmin, and redis), and SUSE (firefox, kernel modules packages, and python).

Systemd v239 has been released with a long list of changes; click below forthe full set. "A new system.conf setting NoNewPrivileges= is now available which may be used to turn off acquisition of new privileges system-wide (i.e. set Linux' PR_SET_NO_NEW_PRIVS for PID 1 itself, and thus also for all its children). Note that turning this option on means setuid binaries and file system capabilities lose their special powers. While turning on this option is a big step towards a more secure system, doing so is likely to break numerous pre-existing UNIX tools, in particular su and sudo."

Version 5.28.0 of the Perl language has been released."Perl 5.28.0 represents approximately 13 months of development since Perl5.26.0 and contains approximately 730,000 lines of changes across 2,200files from 77 authors". The full list of changes can be found overhere; some highlights include Unicode 10.0 support, string- andnumber-specific bitwise operators, a change to more secure hash functions,and safer in-place editing.

The second 4.18 kernel prepatch is out fortesting. "Anyway, it's early in the rc series yet, but things lookfairly normal."

A new archive of linux-kernel mailing list (LKML) posts going back to 1998 is now available at lore.kernel.org. It is based on public-inbox (which we looked at back in February. Among other things, public-inbox allows retrieving the entire archive via Git: "Git clone URLs are provided at the bottom of each page. Note, that due to its volume, the LKML archive is sharded into multiple repositories, each roughly 1GB in size. In addition to cloning from lore.kernel.org, you may also access these repositories on git.kernel.org." The full announcement, which includes information about a new Patchwork instance as well as ways to link into the new archive, can be found on kernel.org.

Security updates have been issued by Debian (php-horde-image), openSUSE (kernel), Scientific Linux (git), SUSE (bluez, kernel, mariadb, and mariadb, mariadb-connector-c, xtrabackup), and Ubuntu (openjdk-7).