LWN.net

GCC 7.3 is out. This is mainly a bug-fix release, but it does also containthe "retpoline" support needed to build the kernel (and perhaps other code)with resistance to the Spectre variant-2 vulnerability.

Here's a34c3 conference report in CSO suggesting that the BSDs are losingdevelopers. "von Sprundel says he easily found around 115 kernel bugs across thethree BSDs, including 30 for FreeBSD, 25 for OpenBSD, and 60 forNetBSD. Many of these bugs he called 'low-hanging fruit.' He promptlyreported all the bugs, but six months later, at the time of his talk, manyremained unpatched.'By and large, most security flaws in the Linux kernel don't have a longlifetime. They get found pretty fast,' von Sprundel says. 'On the BSD side,that isn't always true. I found a bunch of bugs that have been around avery long time.' Many of them have been present in code for a decade ormore."

Security updates have been issued by CentOS (firefox), Debian (firefox-esr, gcab, and poppler), Fedora (clamav and firefox), Mageia (bind, firefox, glibc, graphicsmagick, squid, systemd, and virtualbox), openSUSE (firefox, GraphicsMagick, libexif, and libvpx), Red Hat (389-ds-base, dhcp, kernel, kernel-alt, kernel-rt, and nautilus), Scientific Linux (389-ds-base, dhcp, kernel, and nautilus), Slackware (curl), SUSE (kernel and webkit2gtk3), and Ubuntu (firefox, libtasn1-6, and mysql-5.5).

The LWN.net Weekly Edition for January 25, 2018 is available.

2017 was a big year for the Prometheus project, as it publishedits 2.0 release in November. The new release ships numerousbug fixes, new features, and, notably, a new storage engine that brings majorperformance improvements. This comes at the cost of incompatible changes tothe storage and configuration-file formats. An overview ofPrometheus and its new release was presented to the Kubernetes community in a talkheld during KubeCon+ CloudNativeCon. This article covers what changed in this new releaseand what is brewing next in the Prometheus community; it is a companion tothis article, which provided a generalintroduction to monitoring with Prometheus.

Richard Fontana exploresthe intersection of containers and copyleft licensing onopensource.com."One imperfect way of framing the question is whether GPL-licensedcode, when combined in some sense with proprietary code, forms a singlemodified work such that the proprietary code could be interpreted as beingsubject to the terms of the GPL. While we haven’t yet seen much of thatconcern directed to Linux containers, we expect more questions to be raisedas adoption of containers continues to grow. But it’s fairlystraightforward to show that containers do not raise new or concerning GPLscope issues."

Security updates have been issued by openSUSE (clamav-database and virtualbox), Oracle (firefox and kernel), Red Hat (firefox), Scientific Linux (firefox), and Ubuntu (gcab).

Firefox 58 has been released. "With this release, we’re building on the great foundation provided by our all-new Firefox Quantum browser. We're optimizing the performance gains we released in 57 by improving the way we render graphics and cache JavaScript. We also made functional and privacy improvements to Firefox Screenshots. On Firefox for Android, we’ve added support for Progressive Web Apps (PWAs) so you can add websites to your home screen and use them like native apps."

Sometimes, a data structure proves to be inadequate for its intended task.Other times, though, the problem may be somewhere else — in the API used toaccess it, for example. Matthew Wilcox's presentation during the 2018linux.conf.au Kernel miniconf made the case that, for the kernel'svenerable radix tree data structure, thelatter situation holds. His response is a new approach to an old datastructure that he is calling the "XArray".

The Free Software Foundation blog has a guestpost from GNU MediaGoblin founder Christopher Lemmer Webber announcing that ActivityPub has been made anofficial W3C recommended standard. "ActivityPub is a protocol for building decentralized social networking applications. It provides both a server-to-server protocol (i.e. federation) and a client-to-server protocol (for desktop and mobile applications to connect to your server). You can use the server-to-server protocol or the client-to-server protocol on their own, but one nice feature is that the designs for both are very similar. Chances are, if you've implemented support for one, you can get support for the other with very little extra effort! We've worked hard to make ActivityPub easy to understand."

Stable kernels 4.14.15, 4.9.78, and 4.4.113 have been released. They all containimportant fixes and users should upgrade.

Security updates have been issued by Debian (smarty3), Fedora (bind, bind-dyndb-ldap, dnsperf, glibc, kernel, libtasn1, libvpx, mariadb, python-bottle, ruby, and sox), Red Hat (rh-eclipse46-jackson-databind), SUSE (kernel), and Ubuntu (kernel, linux, linux-aws, linux-euclid, linux-hwe, linux-azure, linux-gcp, linux-oem, linux-lts-trusty, linux-lts-xenial, linux-aws, and rsync).

The Qubes project has described anew, not-yet-implemented design intended to address a number of problemsthat this high-security distribution project has encountered."One possible solution to these problems is actually to 'move Qubesto the cloud.' Readers who are allergic to the notion of having theirprivate computations running in the (untrusted) cloud should not give upreading just yet. Rest assured that we will also discuss other solutionsnot involving the cloud. The beauty of Qubes Air, we believe, lies in thefact that all these solutions are largely isomorphic, from both anarchitecture and code point of view."

By now, almost everybody has probably seen the press coverage of Linus Torvalds's remarks about one of thepatches addressing Spectre variant 2. Less noted, but much moreinformative, is David Woodhouse's responseon why those patches are the way they are. "That's why my initialidea, as implemented in this RFC patchset, was to stick with IBRS onSkylake, and use retpoline everywhere else. I'll give you 'garbagepatches', but they weren't being 'just mindlessly sent around'. If we'regoing to drop IBRS support and accept the caveats, then let's do it as aconscious decision having seen what it would look like, not just drop itquietly because poor Davey is too scared that Linus might shout at himagain."

BPF is an increasingly capable tool for instrumenting and tracing theoperation of the kernel; it has enabled the creation of the growing set ofBCC tools. Unfortunately, BCC has no support for a cross-developmentworkflow where the development machine and the target machine running thedeveloped code are different. Cross-development is favored byembedded-systems kernel developers who tend to develop on an x86 host andthen flash and test their code on SoCs (System on Chips) based on the ARMarchitecture. In this article, I introduce BPFd, a project to enable crossdevelopment using BPF and BCC.

The minor release of openSUSE Leap 42.2 will reachits end-of-life (EOL) on January 26. "The major release of the Leap 42 series has so far provided a support life cycle of 27 months and is expected to last until early 2019; when openSUSE Leap 42.3 will reach its EOL. That gives the major version of Leap 42 more than 36 months of life-cycle support. However, the EOL for the Leap 42 series is dependent on the release of the next major version, which will be openSUSE Leap 15 and it’s expected to be released later this Spring."

The Document Liberation Project has announced five new or improvedlibraries to export EPUB3 and import AbiWord, MS Publisher, PageMaker andQuarkXPress files. "The libraries have been originallydeveloped for the LibreOffice 6.0 major release, but can be used by anyother software thanks to the OSI (Open Source Initiative) compliantlicense."

Security updates have been issued by CentOS (bind), Debian (openocd), Mageia (unbound), Oracle (bind and microcode_ctl), Red Hat (bind, java-1.6.0-sun, libvirt, and qemu-kvm), Scientific Linux (bind), SUSE (kernel and perl-XML-LibXML), and Ubuntu (gimp, intel-microcode, mysql-5.5, mysql-5.7, and openssh).

Back in mid-1997, your editor (Jonathan Corbet) and Liz Coolbaugh wereengaged in a long-running discussion on how to trade our nice, stable,reliably paying jobs for a life of uncertainty, poverty, andaround-the-clock work. Not that we thought of it in those terms,naturally. We eventually settled on joining Red Hat's nascent "supportpartner" program; while we were waiting for it to get started, we decidedto start a weekly newsletter as a side project — not big andprofessional like the real press — to establish ourselves in the community.Thus began an amazing journey that has just completed its 20th year.

As might have been expected from watching the commit stream, the 4.15kernel is not ready for release, so we'll get 4.15-rc9 instead.Linus said: "I really really wanted to just release 4.15 today, but things haven'tcalmed down enough for me to feel comfy about it, and Davem tells mehe still has some networking fixes pending. Laura Abbott found andfixed a very subtle boot bug introduced this development cycle onlyyesterday, and it just didn't feel right to say that we're done."

Security updates have been issued by Debian (bind9, couchdb, lucene-solr, mysql-5.5, openocd, and php5), Mageia (gdk-pixbuf2.0, golang, and mariadb), openSUSE (curl, gd, ImageMagick, lxterminal, ncurses, newsbeuter, perl-XML-LibXML, and xmltooling), Oracle (kernel), and SUSE (xmltooling).

Wired recommendsswitching to F-Droid for Android apps."A polluted ocean of apps is plaguing Android, an operating systembuilt upon Free and Open-Source Software (FOSS) but now barely resemblingthose venerable roots. Today, the average Android device is not onlysusceptible to malware and trackers, it’s also heavily locked down andloaded with proprietary components—characteristics that are hardly thecalling cards of the FOSS movement. Though Android bears the moniker of open-source, the chain of trust between developers, distributors, and end-users is broken."

The OpenSSL project has announceda number of changes to how the project is developed. These includeshutting down the openssl-dev mailing list in favor of discussing allpatches on GitHub and the addition of a new, read-only (for the world)openssl-project list. "We are changing our release schedule so thatunless there are extenuating circumstances, security releases will go outon a Tuesday, with the pre-notification being the previous Tuesday. Wedon’t see a need to have people ready to sacrifice their weekend every timea new CVE comes out."

Here's abrief update from Greg Kroah-Hartman on the kernel's handling of theMeltdown and Spectre vulnerabilities. "This shows that my kernel isproperly mitigating the Meltdown problem by implementing PTI (Page TableIsolation), and that my system is still vulnerable to the Spectre variant1, but is trying really hard to resolve the variant 2, but is not quitethere (because I did not build my kernel with a compiler to properlysupport the retpoline feature)."

Linux’s deadline scheduler is a global early deadline first scheduler forsporadic tasks with constrained deadlines. These terms were defined in the first part of this series. In thisinstallment, the details of the Linux deadline scheduler and how it can beused will be examined.

Security updates have been issued by Arch Linux (bind, irssi, nrpe, perl-xml-libxml, and transmission-cli), CentOS (java-1.8.0-openjdk), Debian (awstats, libgd2, mysql-5.5, rsync, smarty3, and transmission), Fedora (keycloak-httpd-client-install and rootsh), and Red Hat (java-1.7.0-oracle and java-1.8.0-oracle).

Git v2.16.0 is now available. "It is comprised of 509 non-mergecommits since v2.15.0, contributed by 91 people, 26 of which are newfaces." The release notes are included in the link below.

Version 3.0 of theWine Windows emulation layer has been released. "This releaserepresents a year of development effort and over 6,000 individualchanges." Most of the improvements seem to be around Direct3Dgraphics, but it also now possible to package up Wine as an Android app;see the release notes fordetails.

This is the second article of a series discussing various methods ofreducing the size of the Linux kernel to make it suitable for smallenvironments.The first articleprovided a short rationale for this topic, and covered the link-timegarbage collection, also called the ld --gc-sections method. We've seenthat, though it is pretty straightforward, link-time garbage collection hasissues of its own when applied to the kernel, making achieving optimalresults more difficult than it is worth. In this article we'll have a look at what thecompiler itself can do using link-time optimization.

Security updates have been issued by CentOS (linux-firmware and microcode_ctl), Fedora (icecat and transmission), Oracle (java-1.8.0-openjdk and microcode_ctl), Red Hat (java-1.8.0-openjdk), Scientific Linux (java-1.8.0-openjdk), Slackware (bind), SUSE (kernel), and Ubuntu (eglibc).

The LWN.net Weekly Edition for January 18, 2018 is available.

Prometheus is a monitoring toolbuilt from scratch by SoundCloud in 2012. It works by pulling metrics frommonitored services and storing them in a time series database (TSDB). Ithas a powerful query language to inspect that database, create alerts, andplot basic graphs. Those graphs can then be used to detect anomalies ortrends for (possibly automated) resource provisioning. Prometheus also hasextensive service discovery features and supports high availabilityconfigurations.That's what the brochure says, anyway; let's see how it works in the handsof an old grumpy system administrator. I'll be drawing comparisonswith Munin and Nagios frequently because those are the tools I haveused for over a decade in monitoring Unix clusters.

Greg Kroah-Hartman has released stable kernels 4.14.14, 4.9.77, 4.4.112, and 3.18.92. All of them contain important fixesand users should upgrade.

Security updates have been issued by Debian (bind9, wordpress, and xbmc), Fedora (awstats, docker, gifsicle, irssi, microcode_ctl, mupdf, nasm, osc, osc-source_validator, and php), Gentoo (newsbeuter, poppler, and rsync), Mageia (gifsicle), Red Hat (linux-firmware and microcode_ctl), Scientific Linux (linux-firmware and microcode_ctl), SUSE (kernel and openssl), and Ubuntu (bind9, eglibc, glibc, and transmission).

Many techniques in software security are complicated and require a deepunderstanding of the internal workings of the computer and the software undertest. Some techniques, though, are conceptually simple and do not rely onknowledge of the underlying software. Fuzzing is a useful example: running aprogram with a wide variety of junk input and seeing if it does anythingabnormal or interesting, like crashing. Though it might seem unsophisticated,fuzzing is extremely helpful in finding the parsing and input processingproblems that are often the beginning of a security vulnerability.

Alison Chaiken looksin detail at how the kernel boots on opensource.com."Besides starting buggy spyware, what function does early bootfirmware serve? The job of a bootloader is to make available to a newlypowered processor the resources it needs to run a general-purpose operatingsystem like Linux. At power-on, there not only is no virtual memory, but noDRAM until its controller is brought up."

The deadline scheduler enables the user to specify a realtime task'srequirements using well-defined realtime abstractions, allowing the system to makethe best scheduling decisions, guaranteeing the scheduling of realtimetasks even in higher-load systems.This article, the first in a series of two, provides an introduction torealtime scheduling (deadline scheduling in particular) and some of the theory behind it.

Security updates have been issued by Debian (ca-certificates, gdk-pixbuf, and graphicsmagick), Fedora (qtpass), openSUSE (python-openpyxl and syncthing), Slackware (kernel), and Ubuntu (gdk-pixbuf).

The 2018 Linux Storage, Filesystem, and Memory-Management Summit will beheld April 23-25 in Park City, Utah. The call for proposals has just goneout with a tight deadline: they need to be received by January 31."LSF/MM is an invitation-only technicalworkshop to map out improvements to the Linux storage, filesystem andmemory management subsystems that will make their way into themainline kernel within the coming years."

While some aspects of the kernel's defenses against the Meltdown andSpectre vulnerabilities were more-or-less in place when the problems weredisclosed on January 3, others were less fully formed. Additionally,many of the mitigations (especially for the two Spectre variants) had notbeen seen in public prior to the disclosure, meaning that there was a lotof scope for discussion once they came out. Many of those discussions areslowing down, and the kernel's initial response has mostly come intofocus. The 4.15 kernel will include a broad set of mitigations, while someothers will have to wait for later; read onfor details on where things stand.

The Linux kernel's generic power domain (genpd) subsystem has beenextended to support active state management of the power domains in the 4.15 development cycle. Power domains weretraditionally used to enable or disable power to a region of a system onchip (SoC) but, with the recent updates, they can control the clock rate oramount of power supplied to that region as well.These changes improve the kernel's ability to run the system's hardware atthe optimal power level for the current workload.<p>Click below (subscribers only) for the full article contributed by VireshKumar.

Security updates have been issued by Arch Linux (qtpass), Debian (libkohana2-php, libxml2, transmission, and xmltooling), Fedora (kernel and qpid-cpp), Gentoo (PolarSSL and xen), Mageia (flash-player-plugin, irssi, kernel, kernel-linus, kernel-tmb, libvorbis, microcode, nvidia-current, php & libgd, poppler, webkit2, and wireshark), openSUSE (gifsicle, glibc, GraphicsMagick, gwenhywfar, ImageMagick, libetpan, mariadb, pngcrush, postgresql94, rsync, tiff, and wireshark), and Oracle (kernel).

The 4.15-rc8 kernel prepatch is out fortesting. Among other things, it includes the "retpoline" mechanismintended to mitigate variant 2 of the Spectre vulnerability. Testingof this change will be hard, though, since it requires a version of GCCthat almost nobody has — watch LWN for a full article in the near future."I'm still hoping that this will be the last rc, despite all the Meltdown and Spectre hoopla. But we will just have tosee, it obviously requires this upcoming week to not come with any hugesurprises."

GnuBee is the brand namefor a line of open hardware boards designed to provideLinux-based network-attached storage. Given the success of thecrowdfunding campaigns for the first two products, the GB-PC1 andGB-PC2(which support 2.5 and 3.5 inch drives respectively), there appears to be amarket for these devices. Given that Linux is quite good at attachingstorage to a network, it seems likely they will perform their core functionmore than adequately. My initial focus when exploring my GB-PC1 is not theperformance but the openness: just how open is it really? The best analogyI can come up with is that of a door with rusty hinges: it can be opened,but doing so requires determination.

Security updates have been issued by Arch Linux (intel-ucode), Debian (gifsicle), Fedora (awstats and kernel), Gentoo (icoutils, pysaml2, and tigervnc), Mageia (dokuwiki and poppler), Oracle (kernel), SUSE (glibc, kernel, microcode_ctl, tiff, and ucode-intel), and Ubuntu (intel-microcode).

Nextcloud has announcedNextcloud Talk, a fully open source video meeting software that is on-premisehosted and end-to-end encrypted. "Nextcloud Talk makes it easier thanever to host a privacy-respecting audio/video communication service forhome users and enterprises. Business users have optional access to theSpreed High Performance Back-end offering enterprise-class scalability,reliability, and features through a Nextcloud subscription. With theeasy-to-use interface, users can engage colleagues, friends, partners orcustomers, working in real time through High Definition (H265 based) audioand video in web meetings and webinars."

Security updates have been issued by Arch Linux (glibc and lib32-glibc), Debian (ming and poco), Fedora (electron-cash, electrum, firefox, heketi, microcode_ctl, and python-jsonrpclib), openSUSE (clamav-database and ucode-intel), Red Hat (flash-plugin), SUSE (OBS toolchain), and Ubuntu (webkit2gtk).

The LWN.net Weekly Edition for January 11, 2018 is available.

A focus on privacy is a key feature being touted by a number of differentprojects these days—from KDE to Tails to Nextcloud. One of thebiggest privacy leaks for most people is their phone, so it is no surprisethat there are projects looking to address that as well. A new entrant inthat category is eelo, which is a non-profitproject aimed at producing not only a phone, but also a suite of webservices. All of that could potentially replace the Google or Apple mothership,which tend to collect as much personal data as possible.