LWN.net

The LWN.net Weekly Edition for September 13, 2018 is available.

There are ways to get fixes into the stable kernel trees, but theyrequire humans to identify which patches should go there. Sasha Levin andJulia Lawall have taken a different approach: use machine learning todistinguish patches that fix bugs from others. That way, all bug-fixpatches could potentially make their way into the stable kernels. Levinand Lawall gave a talk describing their work at the 2018Open Source Summit North America in Vancouver, Canada.

The STACKLEAK kernel security feature has been in the works for quite sometime now, but has not, as yet, made its way into the mainline. That is notfor lack of trying, as Alexander Popov has posted 15 separate versions ofthe patch set since May 2017. He described STACKLEAK and its tortuous pathtoward the mainline in a talk[YouTube video] at the 2018Linux Security Summit.

Security updates have been issued by Debian (kamailio, libextractor, and mgetty), Fedora (community-mysql, ghostscript, glusterfs, iniparser, okular, and zsh), openSUSE (compat-openssl098, php5, and qemu), Red Hat (firefox), SUSE (libzypp, zypper, python3, spark, and zsh), and Ubuntu (zsh).

From the kernel development community's point of view, hardwarevulnerabilities are not much different from the software variety: eitherway, there is a bug that must be fixed in software. But hardware vendorstend to take a different view of things. This divergence has beenreflected in the response to vulnerabilities like Meltdown and Spectrewhich was seen by many as being severely mismanaged. A recent discussionon the Kernel Summit discussion list has shed some more light on how thingswent wrong, and what the development community would like to see happenwhen the next hardware vulnerability comes around.

PostgreSQL 11 had its third beta releaseon August 9; a fourth beta (or possibly a release candidate) is scheduledfor mid-September. While the final release of the relationaldatabase-management system (currently slated for lateSeptember) will have something new for many users, its development cycle wasnotable for being a period when the community hit its stride in twostrategic areas: partitioning and parallelism.

Security updates have been issued by Debian (libextractor), Fedora (godot and iniparser), Oracle (kernel), Red Hat (chromium-browser and Fuse 7.1), SUSE (compat-openssl098, openssh, php5, php53, qemu, and tiff), and Ubuntu (kernel, linux, linux-aws, linux-azure, linux-gcp, linux-kvm, linux-raspi2, and linux-hwe, linux-azure, linux-gcp).

The Git 2.19.0 release is out. Significant changes include a new "rangediff" capability, directory rename detection, and more; see thisGitHub blog entry for more information. "We can use git diff to show the difference between the two end states, but that doesn’t provide information about the individual commits. And if the base on which the commits were built has changed, the resulting state might be quite different, even if the changes in the commits are largely the same.Git 2.19 introduces git range-diff, a tool for comparing two sequences of commits, including changes to their order, commit messages, and the actual content changes they introduce."

The kernel's CPU scheduler must, as its primary task, determine whichprocess should be executing in each of a system's processors at any giventime. Making an optimal decision involves juggling a number offactors, including the priority (and scheduling classes) of the runnableprocesses, NUMA locality, cache locality, latency minimization,control-group policies, power management, overall fairness, and more. Onemight think that throwing another variable into the mix — and a complex oneat that — would not be something anybody would want to attempt. The recentcoschedulingpatch set from Jan Schönherr does exactly that, though, by introducingthe concept of processes that should be run simultaneously.

Stable kernels 4.14.69, 4.9.126, 4.4.155, and 3.18.122 have been released. They all containthe usual set of important fixes and users should upgrade.

Security updates have been issued by Debian (chromium-browser, curl, discount, firefox-esr, ghostscript, and openssh), Fedora (curl, firefox, ghostscript, glibc, mod_perl, thunderbird, and unixODBC), openSUSE (chromium, firefox, GraphicsMagick, nodejs4, and thunderbird), Oracle (kernel), and SUSE (java-1_7_1-ibm and kvm).

The third 4.19 prepatch is out fortesting. Linus says: "Things look fairly normal".

The 4.18.7 stable kernel update isavailable; it contains 145 fixes. Note that there are updates for theother active stable kernels in the review process as well; they can beexpected almost any time.

The Maintainer's Summit, which is an invite-only gathering of 30 or so kernel developers to discuss process issues with Linus Torvalds, has moved from November 12 in Vancouver, Canada to October 22 in Edinburgh, Scotland in conjunction with Open Source Summit Europe. The technical side of the discussions will still be held as the Kernel Summit track at the Linux Plumbers Conference November 13-15 in Vancouver. There was, it seems, some confusion about the Maintainer's Summit, as Theodore Y. Ts'o said in the announcement of the move: "Last Friday (just before Labor Day) I learned that Linus had gottenconfused about when and where the Maintainer's Summit was going to beheld this year. And most unfortunately, he has already scheduled afamily vacation overlapping with the week of the Maintainer's Summit. [...] The Kernel Summit track will still be held in Vancouver alongsidePlumber's. Technical discussions will take place there; we simplywon't have the time, or necessarily, the right people, to havetechnical discussions at the Maintainer's Summit."

Security updates have been issued by Debian (qemu and xen), Mageia (libxkbcommon, sleuthkit, and wireshark), openSUSE (apache-pdfbox, dovecot22, and php7), SUSE (enigmail, kernel, nodejs4, and php7), and Ubuntu (firefox and transfig).

The Harvard Business School's "Working Knowledge" site has anarticle arguing that it can pay for companies to allow their developersto contribute back to the projects whose software they use."And that presents an interesting dilemma for firms that rely heavilyon open source. Should they allow employees on company time to make updatesand edits to the software for community use that could be used bycompetitors? New research by Assistant Professor Frank Nagle, a member ofthe Strategy Unit at Harvard Business School, shows that paying employeesto contribute to such software boosts the company’s productivity from usingthe software by as much as 100 percent, when compared with free-ridingcompetitors."

Network packet headers contain a great deal of information, but thekernel often only needs a subset of that information to be able to performfiltering or associate any given packet with a flow. The piece of code thatfollows the different layers of packet encapsulation to find the importantdata is called a flow dissector. In current Linux kernels, the flowdissectoris written in C. A patch set has beenproposed recently to implement it in BPF with the clear goal of improvingsecurity, flexibility, and maybe even performance.

Security updates have been issued by Debian (curl, gdm3, git-annex, lcms2, and sympa), Fedora (discount, dolphin-emu, gd, obs-build, osc, tcpflow, and yara), openSUSE (wireshark), Slackware (curl, firefox, ghostscript, and thunderbird), SUSE (apache-pdfbox, curl, dovecot22, and libvirt), and Ubuntu (libtirpc).

The LWN.net Weekly Edition for September 6, 2018 is available.

Security and convenience rarely go hand-in-hand, but if your job (or life)requires extraordinary care against potentially targeted attacks, thesecurity side of that tradeoff may win out. If so, running a system likeQubes OS on your desktop or CopperheadOS on your phone might make sense,which is just what Konstantin Ryabitsev, Linux Foundation (LF) director of ITsecurity, has done. He reported on the experience in a talk[YouTube video] entitled "Life Behind the Tinfoil Curtain" at the 2018LinuxSecurity Summit North America.

The Spectre variant 2 vulnerability allows the speculative execution ofincorrect (in an attacker-controllable way) indirect branch predictions,resulting in the ability to exfiltrate information via side channels. The kernel hasbeen reasonably well protected against this variant since shortly after itsdisclosure in January. It is, however, possible for user-space processesto use Spectre v2 to attack each other; thus far, the mainline kernel hasoffered relatively little protection against such attacks. A recent proposalfrom Jiri Kosina may change that situation, but there are still somedisagreements around the details.

The GNOME Project has announced the release of GNOME 3.30"Almería". "This release brings automatic updates in Software, moregames, and a new Podcasts application. Improvements to core GNOMEapplications include a refined location and search bar in Files, a[Thunderbolt] panel in Settings, support for remoting using RDP in Boxes, andmany more." The release notescontain more information.

GopherCon is the majorconference for the Go language, attendedby 1600 dedicated "gophers", as the members of its community like to callthemselves. Held for the last five years in Denver, it attracts programmers,open-source contributors, and technical managers from all over NorthAmerica and the world. GopherCon's highly-technical program is an intensemix of Go internals and programming tutorials, a few of which we willexplore in this article.Subscribers can read on for a report from GopherCon by guest author JoshBerkus.

Mozilla has released Firefox 62.0, with several new features. The FirefoxHome (default New Tab) allows users to display up to 4 rows of top sites,Pocket stories, and highlights; for those using containers there is menuoption to reopen a tab in a different container; Firefox 63 will remove alltrust for Symantec-issued certificates, and it is optional in Firefox62; FreeBSD support for WebAuthn was added; and more. See the releasenotes for details.

Greg Kroah-Hartman has released stable kernels 4.18.6, 4.14.68, 4.9.125, 4.4.154, and 3.18.121. They all contain important fixes andusers should upgrade.

Security updates have been issued by Debian (lcms2), openSUSE (yubico-piv-tool), Oracle (kernel), and SUSE (cobbler and kvm).

Part 1 of this series introduced the Julia project's goals anddevelopment process, along withthe language syntax, including the basics of control flow, datatypes, and, in more detail, how to work with arrays. In this part, user-defined functions and the central concept of multiple dispatch are described. It will also survey Julia'smodule and package system, cover some syntax features, show how to makeplots, and briefly dip into macros and distributed computing.

Security updates have been issued by openSUSE (ImageMagick, libressl, postgresql10, spice, and spice-gtk), Red Hat (collectd, kernel, Red Hat Gluster Storage, Red Hat Virtualization, RHGS WA, rhvm-appliance, and samba), and SUSE (crowbar, crowbar-core, crowbar-ha, crowbar-openstack, crowbar-ui, kernel, spice, and spice-gtk).

It is common for kernel code to generate unique integers for identifiers.When one plugs in a flash drive, it will show up as/dev/sdN; that N (a letter derived from anumber) must be generated in thekernel, and it should not already be in use for another drive or unpleasantthings will happen. One might think that generating such numbers would notbe a difficult task, but that turns out not to be the case, especially insituations where many numbers must be tracked. The IDA (for "IDallocator", perhaps) API exists to handle this specialized task. In pastkernels, it has managed to make the process of getting an unused numbersurprisingly complex; the 4.19 kernel has a new IDA API that simplifies thingsconsiderably.

The annual Maintainer and Kernel Summits will be held in Vancouver, BC onNovember 12 to 15, in conjunction with the Linux Plumbers Conference.The program committee is looking for topics for both summits; read on fordetails on how to submit ideas and, perhaps, get an invitation to theMaintainer Summit.

Security updates have been issued by Debian (dojo, libtirpc, mariadb-10.0, php5, ruby-json-jwt, spice, spice-gtk, tomcat8, and trafficserver), Fedora (ghc-hakyll, ghc-hs-bibutils, ghostscript, mariadb, pandoc-citeproc, phpMyAdmin, and xen), Mageia (java-1.8.0-openjdk, libarchive, libgd, libraw, libxcursor, mariadb, mercurial, openssh, openssl, poppler, quazip, squirrelmail, and virtualbox), openSUSE (cobbler, libressl, wireshark, and zutils), and SUSE (couchdb, java-1_7_0-ibm, java-1_7_1-ibm, OpenStack, and spice).

The 4.19-rc2 kernel prepatch is out fortesting."As usual, the rc2 release is pretty small. People are taking abreather after the merge window, and it takes a bit of time for bugreports to start coming in and get identified."

Linux Mint Debian Edition v3 "Cindy" has been released, featuring theCinnamon desktop. LMDE 3 is based on Debian 9 "stretch". "There are no point releases in LMDE. Other than bug fixes and security fixes Debian base packages stay the same, but Mint and desktop components are updated continuously. When ready, newly developed features get directly into LMDE, whereas they are staged for inclusion on the next upcoming Linux Mint point release."The release notes provideadditional information.

Security updates have been issued by Debian (389-ds-base, bind9, and squirrelmail), Fedora (dolphin-emu), openSUSE (libX11), SUSE (cobbler, GraphicsMagick, ImageMagick, liblouis, postgresql10, qemu, and spice), and Ubuntu (libx11).

Google has announcedthe existence of a new cryptographic library called "Tink"."Tink aims to provide cryptographic APIs that are secure, easy to usecorrectly, and hard(er) to misuse. Tink is built on top of existinglibraries such as BoringSSL and Java Cryptography Architecture, butincludes countermeasures to many weaknesses in these libraries, which werediscovered by Project Wycheproof, another project from our team."

The developers of the Android system have, among their many goals, the wishto better protect Android devices against persistent compromise. It is badif a device is taken over by an attacker; it's worse if it remainscompromised even after a reboot. Numerous mechanisms for ensuring theintegrity of installed system files have been proposed and implementedover the years. But it seems there is always room for one more; to fillthat space, the fs-veritymechanism is being proposed as a way to protect individual files frommalicious modification.

Security updates have been issued by Debian (libx11), Fedora (bouncycastle, libxkbcommon, libzypp, nodejs, ntp, openssh, tomcat, xen, and zypper), Red Hat (ansible, kernel, and opendaylight), and SUSE (apache2, cobbler, ImageMagick, libtirpc, libzypp, zypper, and qemu).

The LWN.net Weekly Edition for August 30, 2018 is available.

Many services, from web hosting and video streaming to cloud storage,need to move data to and from storage. They also often require that each per-clientI/O flow be guaranteed a non-zero amount of bandwidth and a bounded latency. Anexpensive way to provide these guarantees is to over-provisionstorage resources, keeping each resource underutilized, and thushave plenty of bandwidth available for the few I/O flows dispatched toeach medium. Alternatively one can use an I/O controller. Linux providestwo mechanisms designed to throttle some I/O streams to allow others tomeet their bandwidth and latency requirements. These mechanisms work, butthey come at a cost: a loss of as much as 80% of total available I/Obandwidth. I have run some tests to demonstrate this problem; someupcoming improvements to the bfq I/Oscheduler promise to improve the situation considerably.

At the North America edition of the 2018Linux Security Summit (LSS NA), which was held in late August in Vancouver,Canada, Kees Cook gave a presentation on some of the dangers that come withprograms written in C. In particular, of course, the Linux kernel ismostly written in C, which means that the security of our systems rests ona somewhat dangerous foundation. But there are things that can be done tohelp firm things up by "Making C Less Dangerous" as the titleof his talk suggested.

Version 3.1 of the Bison parser generator has been released."It introducesnew features such as typed midrule actions, brings improvements in thediagnostics, fixes several bugs and portability issues, improves theexamples, and more".

Security updates have been issued by CentOS (bind and postgresql), Debian (linux-4.9 and tomcat8), Red Hat (java-1.7.1-ibm and java-1.8.0-ibm), Slackware (kernel), SUSE (kernel and openssl1), and Ubuntu (linux-azure, linux-oem, linux-gcp and poppler).

Julia is a young computer languageaimed at serving the needs of scientists, engineers, and otherpractitioners of numerically intensive programming. It was first publiclyreleased in 2012. After an intense period of language development, version1.0 was released onAugust 8. The 1.0 release promises years of languagestability; users can be confident that developments in the 1.x series willnot break their code. This is the first part of a two-part article introducing the world of Julia. This part will introduce enough of the language syntax and constructs to allow you to begin to write simple programs. The following installment will acquaint you with the additional pieces needed to create real projects, and to make use of Julia's ecosystem.

The Netdev 0x12 networkingconference was held in mid-July. The conference team has provided a brief introduction. Participants at the eventhave put together a set of reports of the talks that were held during theconference; tutorials and workshops were held on Day 1, Day 2 includes eleven talks, includingthe keynote by Van Jacobson, while Day 3 coversanother ten topics.

Stable kernels 4.4.153 and 3.18.120 have been released. They both containimportant fixes and users should upgrade.

Security updates have been issued by Debian (ruby2.1 and twitter-bootstrap3), Fedora (freeipa), openSUSE (libreoffice), Oracle (bind), Red Hat (bind), Scientific Linux (bind), SUSE (graffana, kafka, logstash, monasca-installer and libreoffice), and Ubuntu (intel-microcode and libgd2).

Dat is a new peer-to-peer protocolthat uses some of the concepts ofBitTorrent and Git. Dat primarilytargets researchers and open-data activists as it is a great tool for sharing, archiving, andcataloging large data sets. But it can also be used to implementdecentralized web applications in a novel way.Subscribers can read on for more on Dat by guest author Antoine Beaupré.

Security updates have been issued by Debian (dropbear, libextractor, and libgit2), Fedora (chromium, obs-build, and osc), openSUSE (GraphicsMagick, ImageMagick, kbuild, virtualbox, libgit2, nextcloud, and phpMyAdmin), Red Hat (java-1.7.1-ibm, java-1.8.0-ibm, rh-postgresql10-postgresql, and rh-postgresql96-postgresql), and SUSE (gdm, openssh, openssl, python, and xen).

By the time Linus Torvalds released4.19-rc1 and closed the merge window for this development cycle, 12,317 non-mergechangesets had found their way into the mainline; about 4,800 of thoselanded after last week's summary waswritten. As tends to be the case late in the merge window, many of those changes were fixes for the biggerpatches that went in early, but there were also a number of new featuresadded.

Linus has released 4.19-rc1 and closed themerge window for this development cycle. "This was a fairlyfrustrating merge window, partly because 4.19 looks to be a pretty bigrelease (no single reason), and partly just due to random noise. We had theL1TF hw vulnerability disclosure early in the merge window, which justadded the usual frustration due to having patches that weren't public. Thatjust shows just how good all our infrastructure for linux-next and variousautomated testing systems have become, in how painful it is when it'slacking."