LWN.net

The4.14.9,4.9.72,4.4.108, and3.18.90stable kernel updates have been released with a large set of importantfixes. The 4.14.9 update includes thekernel page-table isolation precursorpatches that also just landed in 4.15-rc5.

The 4.15-rc5 kernel prepatch is out."This (shortened) week ended up being fairly normal for rc5, with theexception of the ongoing merging of the x86 low-level prep for kernelpage table isolation that continues and is noticeable. In fact, abouta third of the rc5 patch is x86 updates due to that."

Jann Horn has reported eight bugs in theeBPF verifier, one for the 4.9 kernel and seven introduced in 4.14, to theoss-security mailing list. Someof these bugs result in eBPF programs being able to read and write arbitrarykernel memory, thus can be used for a variety of ill effects, includingprivilege escalation. As Ben Hutchings notes,one mitigation would be to disable unprivileged access to BPF using thefollowing sysctl:kernel.unprivileged_bpf_disabled=1. More information can also be foundin this ProjectZero bug entry. The fixes are not yet in the mainline tree, but are inthe netdev tree. Hutchings goes on to say: "There is a publicexploit that uses several of these bugs to get root privileges. It doesn'twork as-is on stretch [Debian 9] with the Linux 4.9 kernel, but is easy to adapt. Irecommend applying the above mitigation as soon as possible to all systemsrunning Linux 4.4 or later."

In the previous article of this series, I discussed how to use eBPF to safely run code supplied byuser space inside of the kernel. Yet one of eBPF's biggest challengesfor newcomers is that writing programs requires compiling and linking tothe eBPF library from the kernel source. Kernel developers might alwayshave a copy of the kernel source within reach, but that's not so forengineers working on production or customer machines.

The Register reportsthat the grsecurity defamation suit filed against Bruce Perens has beentossed out of court. "On Thursday, the judge hearing the case, SanFrancisco magistrate judge Laurel Beeler, granted Peren's motion to dismissthe complaint while also denying – for now – his effort to invokeCalifornia's anti-SLAPP law."

The Free Software Foundation (FSF) has announced that it added PureOS to its list of endorsed Linux distributions. "'PureOS is a GNU operating system that embodies privacy, security, and convenience strictly with free software throughout. Working with the Free Software Foundation in this multi-year endorsement effort solidifies our longstanding belief that free software is the nucleus for all things ethical for users. Using PureOS ensures you are using an ethical operating system, committed to providing the best in privacy, security, and freedom,' said Todd Weaver, Founder & CEO of Purism."

Here's thelatest from Eben Moglen on the Software Freedom Law Center's trademarkattack against the Software Freedom Conservancy. "We propose ageneral peace, releasing all claims that the parties have against oneanother, in return for an iron-clad agreement for mutual non-disparagement,binding all the organizations and individuals involved, with strongsafeguards against breach. SFLC will offer, as part of such an overallagreement, a perpetual, royalty-free trademark license for the SoftwareFreedom Conservancy to keep and use its present name, subject to agreedmeasures to prevent confusion, and continued observance of thenon-disparagement agreement."In the spirit of non-disparagement,it also says: "In view of this evidence and the sworn pleadingsubmitted by the Conservancy, we have now moved to amend our petition, tostate as a second ground for the cancellation that the trademark wasobtained by fraud."

Security updates have been issued by Debian (bouncycastle, enigmail, and sensible-utils), Fedora (kernel), Mageia (dhcp, flash-player-plugin, glibc, graphicsmagick, java-1.8.0-openjdk, kernel, kernel-linus, kernel-tmb, mariadb, pcre, rootcerts, rsync, shadow-utils, and xrdp), and SUSE (java-1_8_0-ibm and kernel).

Security updates have been issued by Debian (libreoffice, openafs, and otrs2) and SUSE (ImageMagick).

The LWN.net Weekly Edition for December 21, 2017 is available.

The Docker (now Moby) project hasdone a lot to popularize containers in recent years. Along the way,though, it has generated concerns about its concentration of functionalityinto a single, monolithic system under the control of a single daemonrunning with root privileges: dockerd. Those concerns werereflected in a talkby Dan Walsh, head of the container team at Red Hat, at KubeCon +CloudNativeCon. Walsh spoke about the work the container team is doingto replace Docker with a set of smaller, interoperable components. His rallying cry is "no big fatdaemons" as he finds them to be contrary to the venerated Unix philosophy.

As we briefly mentioned in our overview article aboutKubeCon + CloudNativeCon, there are multiple container "runtimes", which areprograms that can create and execute containers that are typically fetchedfrom online images. That space is slowly reaching maturity both in termsof standards and implementation: Docker's containerd 1.0 was releasedduring KubeCon, CRI-O 1.0 was released a few months ago, and rkt isalso still in the game. With all of those runtimes, it may be a confusingtime for those looking at deploying their own container-based systemor Kubernetes cluster fromscratch. This article will try to explain what container runtimes are, what they do, how they compare with each other, andhow to choose the right one. It also provides a primer on containerspecifications and standards.

The December 21 LWN Weekly Edition will be the final one for 2017; asusual, we will take the last week of the year off and return onJanuary 4. It's that time of year where one is moved to look backover the last twelve months and ruminate on what happened; at LWN, we alsoget the opportunity to mock the predictions wemade back in January. Read on for the scorecard and a year-end notefrom LWN.

Four stable kernels have been released; 4.14.8, 4.9.71, 4.4.107, and 3.18.89. They all contain important fixes andusers should upgrade.

Security updates have been issued by Debian (otrs2), Fedora (glibc, kernel, libextractor, LibRaw, nodejs, optipng, python34, python35, qt5-qtbase, wayland, and xen), and Slackware (ruby).

At the end of October, the KAISER patch setwas unveiled; this work separates the page tables used by the kernel fromthose belonging to user space in an attempt to address x86 processor bugsthat can disclose the layout of the kernel to an attacker. Those patcheshave seen significant work in the weeks since their debut, but they appearto be approaching a final state. It seems like an appropriate time foranother look.

Downloads of Ubuntu 17.10 have been disabled due to anissue that can cause it to corrupt the firmware on some laptops.Lenovo laptops appear to be the most affected, but the problem isapparently not limited to them. The intel-spi driver has been named as thesource of the problem; it's not clear whether other distributions may alsobe affected. If you downloaded 17.10, you might want to hold off oninstalling it.

The Mozilla Thunderbird Blog looksat recent releases of the Thunderbird email client, including a fifthpoint release for version 52 ESR and 58 beta. "Thunderbird 57 betawas also very successful. While Thunderbird 58 is equally stable and offersfurther cutting-edge improvements to Thunderbird users, the user communityis starting to feel the impact of Mozilla platform changes which arephasing out so-called legacy add-ons. The Thunderbird technical leadershipis working closely with add-on authors who face the challenge of updatingtheir add-ons to work with the Mozilla interface changes. With a fewusually simple changes most add-ons can be made to work in Thunderbird 58beta. https://wiki.mozilla.org/Thunderbird/Add-ons_Guide_57explains what needs to be done, and Thunderbird developers are happy tolend a hand to add-on authors." The project has also added four newstaff members.

The Fedora Project has announceda number of changes to its modularity initiative afterfailing to meet its initial set of goals."From an end-user’s perspective, Fedora will ship with two sets ofrepositories. One will be the traditional Fedora repositories (fedora,updates, and updates-testing) and the other will be a new set ofrepositories providing alternative and supplementary modules. We haven’tdecided on a final name for these yet, so we will use the placeholder termsmodular, modular-updates, and modular-updates-testing."

Security updates have been issued by Debian (libxml2), Fedora (kernel, perl-DBD-MySQL, and python26), openSUSE (389-ds and pdns-recursor), Red Hat (heketi and rh-ruby24-ruby), Scientific Linux (postgresql), and SUSE (java-1_6_0-ibm).

Robert Haas gets intothe details of how PostgreSQL concurrency works and why an occasionalVACUUM is necessary. "The second approach to providing transactionswith atomicity and isolation is multi-version concurrency control(MVCC). The basic idea is simple: instead of locking a row that we want toupdate, let’s just create a new version of it which, initially, is visibleonly to the transaction which created it. Once the updating transactioncommits, we’ll make the new row visible to all new transactions that startafter that point, while existing transactions continue to see the oldrow."

By their nature, low-level libraries go mostly unnoticed by users andeven some programmers. Usually, they are only noticed when something goeswrong. However, HarfBuzzdeserves to be an exception. Not only does the adoption of HarfBuzz meanthat free software's ability to convert Unicodecharacters to a font's specific glyphs is as advanced as any proprietaryequivalent, but its increasing use means that professional typography cannow be done from the Linux desktop as easily as at a print shop.

Parrot 3.10, the latest version of the security oriented GNU/Linuxdistribution, has been released. "The first big news is the introduction of a full firejail+apparmor sandboxing system to proactively protect the OS by isolating its components with the combination of different tecniques. The first experiments were already introduced in Parrot 3.9 with the inclusion of firejail, but we took almost a month of hard work to make it even better with the improvement of many profiles, the introduction of the apparmor support and enough time to make all the tests."

Stable kernels 4.14.7, 4.9.70, 4.4.106, and 3.18.88 have been released. They all containimportant fixes and users should upgrade.

Security updates have been issued by Arch Linux (chromium, lib32-openssl-1.0, openssl-1.0, and tor), Debian (kildclient, openafs, openssl1.0, otrs2, reportbug, rsync, and sensible-utils), Fedora (tor), Mageia (deluge, evince, lynx, openssl, and rsync), openSUSE (chromium, GraphicsMagick, kernel, mercurial, and openssl), Red Hat (chromium-browser), SUSE (openssl), and Ubuntu (php5).

Linus has released the 4.15-rc4 kernelprepatch. "I would like to say that I hope things will continue tocalm down, but I already know I have more stuff pending. That, togetherwith the holidays, makes me strongly suspect that this will be one of those'we'll do an rc8' releases, but we'll see."

The dreaded UnicodeDecodeError exception is one of the signature"features" of Python 3. It is raised when the language encounters a byte sequencethat it cannot decode into a string; strictly treating stringsdifferently from arrays of byte values was something that came withPython 3. Two Python Enhancement Proposals (PEPs) bound forPython 3.7 look toward reducing those errors (and the related UnicodeEncodeError) forenvironments where they are prevalent—and often unexpected.

One of the keys to fitting the Linux kernel into a small system is toremove any code that is not needed. The kernel's configuration systemallows that to be done on a large scale, but it still results in thebuilding of a kernel containing many smaller chunks of unused code anddata. With a bit of work, though, the compiler and linker can be made towork together to garbage-collect much of that unused code and recover thewasted space for more important uses.<p>Click below (subscribers only) for a detailed article from Nicolas Pitre onhow to use link-time garbage collection to create a smaller kernel image.

Security updates have been issued by Debian (erlang), Fedora (python-dulwich), Gentoo (curl, opencv, openssl, and webkit-gtk), openSUSE (libapr-util1 and php5), Red Hat (qemu-kvm-rhev), and Ubuntu (linux, linux-aws, linux-kvm, linux-raspi2 and linux-lts-xenial, linux-aws).

In a vote that was not any kind of surprise, the US Federal Communications Commission (FCC) voted to end the "net neutrality" rules that stop internet service providers (ISPs) and others from blocking or throttling certain kinds of traffic to try to force consumers and content providers to pay more for "fast lanes". Ars Technica covers the vote and the reaction to it, including the fact that the fight is not yet over: "Plenty of organizations might appeal, said consumer advocate Gigi Sohn, who was a top counselor to then-FCC Chairman Tom Wheeler when the commission imposed its rules.'I think you'll see public interest groups, trade associations, and small and mid-sized tech companies filing the petitions for review,' Sohn told Ars. One or two 'big companies' could also challenge the repeal, she thinks.Lawsuit filers can challenge the repeal on numerous respects, she said. They can argue that the public record doesn't support the FCC's claim that broadband isn't a telecommunications service, that 'throwing away all protections for consumers and innovators for the first time since this issue has been debated is arbitrary and capricious,' and that the FCC cannot preempt state net neutrality laws, she said."

Linux Foundation Director of IT infrastructure security, Konstantin Ryabitsev, has put together a lengthy guide to using Git and PGP to protect the integrity of source code. In a Google+ post, he called it "beta quality" and asked for help with corrections and fixes. "PGP incorporates a trust delegation mechanism known as the 'Web of Trust.' At its core, this is an attempt to replace the need for centralized Certification Authorities of the HTTPS/TLS world. Instead of various software makers dictating who should be your trusted certifying entity, PGP leaves this responsibility to each user.Unfortunately, very few people understand how the Web of Trust works, and even fewer bother to keep it going. It remains an important aspect of the OpenPGP specification, but recent versions of GnuPG (2.2 and above) have implemented an alternative mechanism called 'Trust on First Use' (TOFU).You can think of TOFU as 'the SSH-like approach to trust.' With SSH, the first time you connect to a remote system, its key fingerprint is recorded and remembered. If the key changes in the future, the SSH client will alert you and refuse to connect, forcing you to make a decision on whether you choose to trust the changed key or not.Similarly, the first time you import someone's PGP key, it is assumed to be trusted. If at any point in the future GnuPG comes across another key with the same identity, both the previously imported key and the new key will be marked as invalid and you will need to manually figure out which one to keep.In this guide, we will be using the TOFU trust model."

Two new stable kernels have been released by Greg Kroah-Hartman: 4.14.6 and 4.9.69. As usual, they contain fixes all overthe kernel tree; users of those series should upgrade.

Security updates have been issued by Arch Linux (qt5-webengine and quagga), Debian (xrdp), Oracle (kernel), Red Hat (eap7-jboss-ec2-eap, go-toolset-7 and go-toolset-7-golang, and java-1.8.0-ibm), and SUSE (intel-SINIT and tomcat).

The LWN.net Weekly Edition for December 14, 2017 is available.

The MAP_FIXED option to the mmap()system call allows a process to specify that a mapping should be placedat a given virtual address if at all possible. It turns out, though, that"if at all possible" can involve a bit more collateral damage than somewould like, and can even lead to exploitable vulnerabilities. A new, saferoption is in the works but, as is often the case, it has run into a bit ofnon-technical difficulty.

The CloudNative Computing Foundation (CNCF) held its conference,KubeCon + CloudNativeCon, in December 2017. There were 4000 attendees at this gathering in Austin, Texas,more than all the previous KubeCons before, which shows the rapid growth of thecommunity building around the tool that was announced by Google in2014. Large corporations are also taking a larger part in the community, with major players in the industry joining the CNCF, which is a project of the Linux Foundation. The CNCF now features three of the largest cloudhosting businesses (Amazon, Google, and Microsoft), but also emergingcompanies from Asia like Baidu and Alibaba.

Linaro has announced the 17.12 release of its "Enterprise ReferencePlatform" distribution. "The goal of the Linaro Enterprise Reference Platform is to provide a fullytested, end to end, documented, open source implementation for ARM basedEnterprise servers. The Reference Platform includes kernel, a communitysupported userspace and additional relevant open source projects, and isvalidated against existing firmware releases."

Security updates have been issued by Debian (tiff), openSUSE (firefox, fossil, GraphicsMagick, and libheimdal), Red Hat (rh-java-common-lucene and rh-java-common-lucene5), and Ubuntu (libxml2).

For various reasons related to accounting and security, there is recurringinterest in having the kernel identify the container that holds any givenprocess. Attempts to implement that functionality tend to run into thesame roadblock, though: the kernel has no concept of what a "container" is,and there is seemingly little desire to change that state of affairs. A solution to this problem may exist in the form of a neglectedpatch called "ptags", which enables the attachment of arbitrary tags toprocesses.

<p>Social networking is often approached by the free-software community with acertain amount of suspicion—rightly so, since commercial social networksalmost always generate revenue by exploiting user data in one way oranother. While attempts at a free-software approach to social networking have so far not metwidespread success, the new ActivityPub federation protocol and itsimplementation in the free-software microblogging system Mastodon are gainingpopularity and already show some of the advantages of a community-drivenapproach.

Fedora 25 has reached its end of life. There will be no more updates.Users are advised to upgrade.

Security updates have been issued by Debian (chromium-browser, evince, pdns-recursor, and simplesamlphp), Fedora (ceph, dhcp, erlang, exim, fedora-arm-installer, firefox, libvirt, openssh, pdns-recursor, rubygem-yard, thunderbird, wordpress, and xen), Red Hat (rh-mysql57-mysql), SUSE (kernel), and Ubuntu (openssl).

Worth a read: thisAPNIC blog entry from Mark Nottingham on the near-term evolution ofvarious Internet protocols. "The newest change on the horizon is DOH — DNS over HTTP. A significant amount of research has shown that networks commonly use DNS as a means of imposing policy (whether on behalf of the network operator or a greater authority).Circumventing this kind of control with encryption has been discussed for a while, but it has a disadvantage (at least from some standpoints) — it is possible to discriminate it from other traffic; for example, by using its port number to block access.DOH addresses that by piggybacking DNS traffic onto an existing HTTP connection, thereby removing any discriminators."

"Load tracking" refers to the kernel's attempts to track how much load eachrunning process will put on the system's CPUs. Good load tracking canyield reasonable predictions about the near-future demands on the system;those, in turn, can be used to optimize the placement of processes and theselection of CPU-frequency parameters. Obviously, poor load tracking willlead to less-than-optimal results. While achieving perfection in load trackingseems unlikely for now, it appears that it is possible to do better thancurrent kernels do. The utilization estimationpatch set from Patrick Bellasi is the latest in a series of efforts tomake the scheduler's load tracking work well with a wider variety ofworkloads.

Artifex Software, Inc. and Hancom, Inc. have announceda confidential agreement to settle their legal dispute. The case filed byArtifex concerned the use of Artifex’s GPL licensed Ghostscript in Hancom'soffice product. "While the parties had their differences in the interpretation of the open source license, the companies were able to reach an amicable resolution based on their mutual respect for and recognition of the copyright protection and the open source philosophy."

A very early alpha version of the Elisa music player has been released."Elisa allows to browse music by album, artist or all tracks. The music is indexed using either a private indexer or an indexer using Baloo. The private one can be configured to scan music on chosen paths. The Baloo one is much faster because Baloo is providing all needed data from its own database. You can build and play your own playlist."

The Debian project has released updates to oldstable "jessie" and stable"stretch". Debian 9.3 "stretch" and Debian 8.10 "jessie" are available with theusual set of corrections for security issues and adjustments for seriousproblems.

Stable kernels 4.14.5, 4.9.68, 4.4.105, and 3.18.87 have been released. They all containimportant fixes and users should upgrade.

Security updates have been issued by CentOS (postgresql), Debian (firefox-esr, kernel, libxcursor, optipng, thunderbird, wireshark, and xrdp), Fedora (borgbackup, ca-certificates, collectd, couchdb, curl, docker, erlang-jiffy, fedora-arm-installer, firefox, git, linux-firmware, mupdf, openssh, thunderbird, transfig, wildmidi, wireshark, xen, and xrdp), Mageia (firefox and optipng), openSUSE (erlang, libXfont, and OBS toolchain), Oracle (kernel), Slackware (openssl), and SUSE (kernel and OBS toolchain).

The 4.15-rc3 kernel prepatch is out."I'm not thrilled about how big the early 4.15 rc's are, but rc3 isoften the biggest rc because it's still fairly early in thecalming-down period, and yet people have had some time to startfinding problems. That said, this rc3 is big even by rc3 standards.Not good." 489 changesets were merged since 4.15-rc2.