LWN.net

Git does not handle large files very well. While there iswork underway to handle large repositories through the commitgraph work, Git's internal design has remained surprisingly constantthroughout its history, which means that storing large files into Git comeswith a significant and, ultimately, prohibitive performancecost. Thankfully, other projects are helping Git address thischallenge. This article compares how Git LFS and git-annex address this problemand should help readers pick the right solution for their needs.

Security updates have been issued by Debian (php7.0), Fedora (keepalived, kernel, kernel-headers, kernel-tools, mingw-uriparser, and uriparser), openSUSE (pdns-recursor), Oracle (kernel), SUSE (compat-openssl098, glibc, java-1_8_0-ibm, kernel, opensc, python, python-base, python-cryptography, python-pyOpenSSL, samba, and soundtouch), and Ubuntu (cups).

There are a lot of claims regarding the relative security of containersversus virtual machines (VMs), but there has been little in the way ofactually trying to measure those differences. James Bottomley gave a talkin the refereed track of the 2018 Linux Plumbers Conference (LPC)that described work that targets filling in that gap. He and his colleagueshave come up with a measure that, while not perfect, gives a starting point for furtherefforts.

Version15 of the Nextcloud productivity and communications platform is out.New features include Mastodon integration, two-factor authentication, anumber of user-interface improvements, and more.

For those who would like a deeper understanding of how the human interfacedevice (HID) protocol works, Peter Hutterer has posted adetailed overview. "Originally HID was designed to work overUSB. But just like Shrek the technology world is obsessed with layers sothese days HID works over different transport layers. HID over USB is whatyour mouse uses, HID over i2c may be what your touchpad uses. HID worksover Bluetooth and it's celebrity-diet version BLE. Somewhere, someone outthere is very slowly moving a mouse pointer by sending HID over carrierpigeons just to prove a point. Because there's always that one guy."

Kernel bugs can have all kinds of unfortunate consequences, frominconvenient crashes to nasty security vulnerabilities. Some of the mostfeared bugs, though, are those that corrupt data in filesystems. Thelosses imposed on users can be severe, and the resulting problems may notbe noticed for a long time, making recovery difficult. Filesystemdevelopers, knowing that they will have to face their users in the realworld, go to considerable effort to prevent this kind of bug from findingits way into a released kernel. A recent failure in that regard raises anumber of interesting questions about how kernel development is done.

Security updates have been issued by Debian (chromium-browser and lxml), Fedora (cairo, hadoop, and polkit), Mageia (tomcat), openSUSE (apache2-mod_jk, Chromium, dom4j, ImageMagick, libgit2, messagelib, ncurses, openssl-1_0_0, otrs, pam, php5, php7, postgresql10, rubygem-activejob-5_1, tiff, and tomcat), Red Hat (chromium-browser and rh-git218-git), Slackware (php), SUSE (audiofile, cri-o and kubernetes packages, cups, ImageMagick, libwpd, SMS3.2, and systemd), and Ubuntu (lxml).

The 4.20-rc6 kernel prepatch is out fortesting."Most of it looks pretty small and normal. Would I have preferred forthere to be less churn? Yes. But it's certainly smaller than rc5 was,so we're moving in the right direction, and we have at least one morerc to go."

The stable kernel process continues to churn out releases;4.19.8,4.14.87, and4.9.144are now available with another set of important fixes.

Filesystem developers tend toward a high level of conservatism when itcomes to making changes; given the consequences of mistakes, this seemslike a healthy survival trait. One might rightly be tempted to regard arecent disagreement over the backporting of filesystem-related fixes to thestable kernels as an example of this conservatism, but there is more toit. The kernel development process has matured in many ways over theyears; perhaps this discussion hints at some of the changes that will beneeded to continue that maturation in the future.

Security updates have been issued by Arch Linux (jupyter-notebook), CentOS (ghostscript), Debian (libphp-phpmailer and policykit-1), Fedora (bird), Gentoo (ede), Mageia (flash-player-plugin), openSUSE (dom4j, dpdk, glib2, nextcloud, postgresql94, and qemu), Oracle (kernel), SUSE (firefox, libarchive, libgit2, libreoffice, ncurses, openssl-1_0_0, squid, and tiff), and Ubuntu (ghostscript, openssl, openssl1.0, and wavpack).

Signals have existed in Unix systems for years, despite the generalconsensus that they are an example of a baddesign. Extensions and new ways of using signals pop up from time totime, fixing the issues that have been found. A notable addition was theintroduction of signalfd()nearly 10 years ago. Recently, the kernel developers have discussed how to avoidrace conditions related to process-ID (PID) recycling, which occurs when aprocess terminates and another one is assigned the same PID. A process that failsto notice that its target has exited may try to send a signal to the wrongrecipient, with potentially grave consequences. A patch set from ChristianBrauner is trying to solve the issue by adding signaling via file descriptors.

Microsoft has announcedthat its "Edge" browser is joining the Chromium world. "Today we’re announcing that we intend to adopt the Chromium open source project in the development of Microsoft Edge on the desktop to create better web compatibility for our customers and less fragmentation of the web for all web developers.As part of this, we intend to become a significant contributor to the Chromium project, in a way that can make not just Microsoft Edge — but other browsers as well — better on both PCs and other devices."

Security updates have been issued by Mageia (kio-extras), Red Hat (flash-plugin and openstack-neutron), Slackware (gnutls and nettle), SUSE ( aphp53, apache2, apache2-mod_jk, compat-openssl097g, firefox, llvm4, mozilla-nspr, mozilla-nss, apache2-mod_nss, glib2, kvm, mariadb, ncurses, openssl-1_0_0, openssl1, pam, php5, php7, qemu, rubygem-activejob-5_1, tomcat, and wireshark), and Ubuntu (libraw and spamassassin).

The LWN.net Weekly Edition for December 6, 2018 is available.

Videos from the 2018 Linux Plumbers Conference (November 13-15, Vancouver)have now been posted for all sessions, including the Kernel Summit andNetworking tracks. They can be found by going to thedetailed schedule and clicking on the session of interest.

Daniel Vetter began his talk in the refereed track of the 2018 Linux Plumbers Conference (LPC)by noting that it would be in a somewhat similar vein to other talks he hasgiven, since it is about tooling and workflows that are outside of thekernel norm. But, unlike those other talks that concerned changes that hadalready taken place, this talk was about switching open-source graphics projectsto using a hosted version of GitLab, which has not yet happened.In it, he wanted to share his thoughts about why he thinks migrating toGitLab makes sense for the kernel graphics community—and maybe the kernelas a whole.

Stable kernels 4.19.7, 4.14.86, and 4.9.143 have been released, with the usual setof important fixes throughout the tree.

Security updates have been issued by Debian (suricata), Fedora (cobbler), Oracle (ghostscript), Red Hat (ansible), and Scientific Linux (ghostscript and ruby).

A critical flaw in the Kubernetes container orchestration system has been announced. It will allow any user to compromise a Kubernetes cluster by way of exploiting any aggregated API server that is deployed for it. This affects all Kubernetes versions 1.0 to 1.12, but is only fixed in the supported versions (in 1.10.11, 1.11.5, and 1.12.3). "With a specially crafted request, users that are authorized to establish a connection through the Kubernetes API server to a backend server can then send arbitrary requests over the same connection directly to that backend, authenticated with the Kubernetes API server’s TLS credentials used to establish the backend connection. [...] In default configurations, all users (authenticated and unauthenticated) are allowed to perform discovery API calls that allow this escalation. [...] There is no simple way to detect whether this vulnerability has been used. Because the unauthorized requests are made over an established connection, they do not appear in the Kubernetes API server audit logs or server log. The requests do appear in the kubelet or aggregated API server logs, but are indistinguishable from correctly authorized and proxied requests via the Kubernetes API server." Kubernetes users should obviously update as soon as possible.

Back in 2011, Harald Hoyer and Kay Sievers came up with a proposal forFedorato merge much of the operating system into /usr; former top-leveldirectories, /bin, /lib, and /sbin, would then become symbolic links pointing into thecorresponding subdirectories of /usr.Left out of the merge would be things likeconfiguration files in /etc, data in/var, and user home directories. This change was aimed atfeatures like atomic upgrades and easy snapshots. The switchto a merged /usr was successful for Fedora 17; many otherdistributions (Arch,OpenSUSE, Mageia,just to name a few) have followed suit. More recently, Debian has beenworking toward a merged /usr, but it ran into some surprisingproblems that are unique to the distribution.

Security updates have been issued by Fedora (glibc, qemu, and tmux), Mageia (messagelib), Oracle (ghostscript), Red Hat (ghostscript, OpenShift Container Platform 3.10, OpenShift Container Platform 3.11, OpenShift Container Platform 3.2, OpenShift Container Platform 3.3, OpenShift Container Platform 3.4, OpenShift Container Platform 3.5, OpenShift Container Platform 3.6, and OpenShift Container Platform 3.8), Slackware (mozilla), and Ubuntu (linux, linux-gcp, linux-kvm, linux-raspi2, linux-hwe, linux-gcp, perl, and poppler).

The BPF verifier is charged with ensuring that any given BPF program issafe for the kernel to load and run. Programs that fail to terminate areclearly unsafe, as they present an opportunity for denial-of-serviceattacks. In current kernels, the verifier uses a heavy-handed technique toblock such programs: it disallows any program containing loops. This works, but at thecost of disallowing a wide range of useful programs; if the verifier coulddetermine whether any given loop would terminate within a bounded time,this restriction could be lifted. John Fastabend presented a plan fordoing so during the BPFmicroconference at the 2018 Linux Plumbers Conference.

CentOS has released CentOS Linux 7.6 (1810). "Updates released sincethe upstream release are all posted, across all architectures. We stronglyrecommend every user apply all updates, including the content releasedtoday, on your existing CentOS Linux 7 machine by just running 'yumupdate'." See the releasenotes for more information.

Security updates have been issued by Debian (nsis, openssl, poppler, and tiff), Fedora (dnsdist, drupal7, kernel, kernel-headers, kernel-tools, net-snmp, perl, php-Smarty2, and samba), Gentoo (connman, nagios-core, php, and webkit-gtk), Mageia (apache-mod_perl, kdeconnect-kde, and python-requests), Red Hat (rh-postgresql10-postgresql), and SUSE (kernel).

The 4.20-rc5 kernel prepatch is out; amongother things, it contains the STIBP changes described in this article. Linus is also thinking aboutrelease timing: "So my current suggestion is that we plan on aChristmas release, everybody gets their pull requests for the next mergewindow done *before* the holidays, and then we see what happens. I think weall want to have a calm holiday season without either the stress of a mergewindow _or_ the stress of prepping for one."

The4.19.6,4.14.85,4.9.142,4.4.166, and3.18.128stable kernels have all been released; each contains another large set ofimportant fixes.

As of today, Fedora 27 will not be getting any more updates, including security updates. Users should be planning to upgrade more or less immediately. "Fedora 28 will continue to receive updates until 4 weeks after the release of Fedora 30.The maintenance schedule of Fedora releases is documented on theFedora Project wiki. The Fedora Project wiki also containsinstructions on how to upgrade from a previous release of Fedorato a version receiving updates."

The BPF virtual machine is the same on all architectures where it issupported; architecture-specific code takes care of translating BPF tosomething the local processor can understand. So one might be tempted tothink that BPF programs would be portable across architectures but, in manycases, that turns out not to be true. During the BPF microconference at theLinux PlumbersConference, Alexei Starovoitov (assisted by Yonghong Song, who has donemuch of the work described) explainedthe problem and the work that has been done toward "compile once, run everywhere" BPF.

Security updates have been issued by Debian (libarchive, perl, and qemu), Fedora (glibc, glusterfs, links, and moodle), Gentoo (libsndfile and postgresql), openSUSE (openssh, rubygem-loofah, and tiff), Oracle (ruby), Red Hat (ruby), and Ubuntu (libssh and linux-aws).

The Software Freedom Conservancy reportsthat the first hearing in the appeal of the GPLenforcement lawsuit against VMware has been held in Germany."The hearing yesterday was a tiny step in a long process towardresolving this issue, and, as we understand the situation, nothing is yetdecided."

The Go Blog looksforward to version 2 of the Go language. "A major differencebetween Go 1 and Go 2 is who is going to influence the design and howdecisions are made. Go 1 was a small team effort with modest outsideinfluence; Go 2 will be much more community-driven. After almost 10 yearsof exposure, we have learned a lot about the language and libraries that wedidn’t know in the beginning, and that was only possible through feedbackfrom the Go community."

The Spectre class of hardware vulnerabilities was apparently so-namedbecause it can be expected to haunt us for some time. One aspect of thathaunting can be seen in the fact that, nearly one year after Spectre wasdisclosed, the kernel is still unable to prevent one user-space processfrom attacking another in some situations. An attempt to provide thatprotection using a new x86 microcode feature called STIBP has run intotrouble once its performance impact was understood; now a more nuancedapproach may succeed in providing protection where it is needed withoutslowing down everybody else.

Security updates have been issued by Gentoo (openssl and rpm), Mageia (icecast and yaml-cpp), Oracle (kernel and sos-collector), Red Hat (rh-ruby23-ruby, rh-ruby24-ruby, and rh-ruby25-ruby), Slackware (samba), SUSE (tomcat6), and Ubuntu (ghostscript).

The LWN.net Weekly Edition for November 29, 2018 is available.

Malware inserted into a popular npmpackage has put some users at risk of losing Bitcoin, which is certainlyworrisome. More concerning, though, is the implications of how the malwaregot into the package—and how the package got distributed. This is not thefirst time we have seen package-distribution channels exploited, nor willit be the last, but the underlying problem requires more than a technicalsolution. It is, fundamentally, a social problem: trust.

Security updates have been issued by Arch Linux (powerdns-recursor and samba), Debian (ghostscript), Fedora (community-mysql, flatpak, gettext, git, php-PHPMailer, php-phpmailer6, and wireshark), Oracle (kernel and NetworkManager), Scientific Linux (ghostscript, kernel, NetworkManager, and sos-collector), SUSE (dpdk, java-1_7_1-ibm, kernel, python-oslo.cache, python-oslo.concurrency, python-oslo.db, python-oslo.log, python-oslo.messaging, python-oslo.middleware, python-oslo.serialization, python-oslo.service, python-oslo.utils, python-oslo.versionedobjects, python-oslo.vmware, python-oslotest, qemu, rubygem-loofah, tiff, tomcat, and util-linux), and Ubuntu (git, openjdk-8, openjdk-lts, samba, systemd, and webkit2gtk).

A recurring topic in filesystem-developer circles is on handlingcase-insensitive file names. Filesystems for other operating systems doso but, by and large, Linux filesystems do not. In the Kernel Summit trackof the 2018 Linux Plumbers Conference (LPC),Gabriel Krisman Bertazi described his plans for making Linux filesystemsencoding-aware as part of an effort to make ext4, and possibly otherfilesystems, interoperable with case-insensitivity in Android, Windows, and macOS.

Greg Kroah-Hartman has released stable kernels 4.19.5, 4.14.84, 4.9.141, 4.4.165, and 3.18.127. They all contain important fixes andusers should upgrade.

Security updates have been issued by Debian (gnuplot and samba), Fedora (flatpak, kernel-headers, kernel-tools, mariadb-connector-c, php-PHPMailer, php-phpmailer6, and xml-security-c), Gentoo (binutils, libav, mupdf, spice-gtk, strongswan, and tablib), Mageia (libpng(12), mariadb, and openssl), Oracle (ghostscript), Red Hat (.NET Core, ghostscript, java-1.7.1-ibm, kernel, kernel-alt, kernel-rt, NetworkManager, rh-nginx112-nginx, rh-nginx114-nginx, and sos-collector), Scientific Linux (389-ds-base, binutils, curl and nss-pem, fuse, git, glibc, glusterfs, GNOME, gnutls, jasper, java-1.7.0-openjdk, java-11-openjdk, kernel, krb5, libcdio, libkdcraw, libmspack, libreoffice, libvirt, openssl, ovmf, python, python-paramiko, samba, setup, sssd, thunderbird, wget, wpa_supplicant, X.org X11, xerces-c, xorg-x11-server, zsh, and zziplib), SUSE (dom4j, glib2, java-1_7_0-ibm, java-1_7_1-ibm, openssh, postgresql94, procps, qemu, and tiff), and Ubuntu (samba).

The kernelci.org project develops andoperates a distributed testing infrastructure for the kernel. It continuously builds,boots, and tests multiple kernel trees on various types of boards. Kevin Hilman and Gustavo Padovan led a session in the Testing& Fuzzing microconference at the 2018 Linux Plumbers Conference (LPC)to describe the project, its goals, and its future.

"Who's on Team Xmas Tree?" asked Dan Williams at the beginning of his talkin the Kernel Summit track of the 2018Linux Plumbers Conference. Hewas referring to a rule for the ordering of local variable declarationswithin functions that is enforced by a minority of kernel subsystemmaintainers — one of many examples of "local customs" that can surprisedevelopers when they submit patches to subsystems where they are notaccustomed to working. Documenting these varying practices is a small partof Williams's project to create a kernel maintainer's manual, but it seemsto be where the effort is likely to start.

Security updates have been issued by Debian (gnuplot5, icecast2, liblivemedia, otrs2, phpbb3, roundcube, squid3, and xml-security-c), Fedora (kio-extras, tmux, and xen), Gentoo (asterisk, chromium, exiv2, ghostscript-gpl, and thunderbird), openSUSE (libwpd, openssl, openssl-1_1, postgresql10, and SDL2_image), Red Hat (chromium-browser, rh-mysql57-mysql, rh-nginx110-nginx, and rh-nginx18-nginx), SUSE (exiv2, libgcrypt, rpm, and tiff), and Ubuntu (firefox and qemu).

Linus has released the 4.20-rc4 kernelprepatch. "Nothing looks particularly odd or scary, although we dohave some known stuff still pending."

Greg Kroah-Hartman has released a number of stable kernels over the lastfew days, 3.18.126 on November 22, and,on November 23: 4.19.4, 4.14.83, and 4.9.139. Two problems were reported for4.9.139, which quickly led to the release of 4.9.140. As usual, these kernels containimportant fixes; users of those series should upgrade.

Security updates have been issued by Arch Linux (flashplugin, lib32-libtiff, and webkit2gtk), Debian (libphp-phpmailer and openjdk-7), Mageia (flash-player-plugin, Ghostscript, and poppler), openSUSE (chromium and virtualbox), and SUSE (java-1_8_0-ibm, libwpd, openssl, openssl-1_1, realtime-kernel, salt, and SDL_image).

Security updates have been issued by Debian (ceph, openssl, and pixman), Fedora (kernel-headers, kernel-tools, libconfuse, python-urllib3, and xen), Mageia (gettext and roundcubemail), openSUSE (GraphicsMagick and libwpd), Oracle (thunderbird), Slackware (openssl), and Ubuntu (libapache2-mod-perl2).

Stable kernels 4.19.3, 4.18.20, 4.14.82, 4.9.138, and 4.4.164 have been released with the usual setof important fixes. This is the last 4.18.y kernel release and users shouldupgrade to 4.19.y.

Security updates have been issued by Arch Linux (libtiff), CentOS (java-1.7.0-openjdk, spice-server, and thunderbird), Debian (jasper, liblivemedia, ruby-i18n, and ruby-rack), Fedora (curl, elfutils, firefox, kde-connect, kio-extras, libarchive, poppler, and webkit2gtk3), openSUSE (chromium, GraphicsMagick, kernel, libmatroska, mkvtoolnix, SDL2_image, and squid), Oracle (qemu), and Red Hat (flash-plugin and kernel).

There has been a great deal of discussion around the kernel project'srecently adopted code of conduct (CoC), but little of that has happened in anopen setting. That changed to an extent when a panel discussion was heldduring the Kernel Summit track at the 2018 Linux Plumbers Conference.Panelists Mishi Choudhary, Olof Johansson, Greg Kroah-Hartman, and ChrisMason took on a number of issues surrounding the CoC in a generallycalm and informative session.