LWN.net

It has been understood for years that kernel performance can be improved bydoing things in batches. Whether the task is freeing memory pages,initializing data structures, or performing I/O, things go faster if thework is done on many objects at once; many kernel subsystems have beenreworked to take advantage of the efficiency of batching. It turns out,though, that there was a piece of relatively low-hanging fruit at the core of the kernel's networkstack. The 4.19 kernel will feature some work increasing the batching ofpacket processing, resulting in some impressive performance improvements.

During this year's Akademy conference, Lays Rodrigues introduced Atelier, a cross-platform, open-source system that allows users to control their 3D printers. Asshe stated in her talkabstract, it is "a project with a goal to make the 3Dprinting world a better place". Read on for an overview of what theAtelier team is up to and what it has accomplished so far.

Security updates have been issued by Debian (confuse, jetty9, kamailio, kernel, libxcursor, and mutt), Fedora (blktrace, docker-latest, libgit2, and yubico-piv-tool), Mageia (chromium-browser-stable, flash-player-plugin, kernel, kernel-linus, kernel-tmb, microcode, openslp, and wpa_supplicant), openSUSE (apache2, curl, GraphicsMagick, perl-Archive-Zip, and xen), Oracle (kernel and mariadb), Red Hat (rh-postgresql95-postgresql), Slackware (ntp and samba), SUSE (apache2, curl, kernel, kernel-livepatch-tools, libgcrypt, mysql, openssl, perl, procps, rsyslog, shadow, wireshark, and xen), and Ubuntu (kernel).

The 1.0release of the Flatpak applicationdistribution system is out. There are a number of performanceimprovements, the ability to mark applications as being at end-of-life,up-front confirmation of requested permissions, and more. "Apps cannow request access the host SSH agent to securely access remote servers orGit repositories."

Greg Kroah-Hartman has released two batches of stable kernels. The firstset has fixes in various parts of the tree, while the second batch has asingle fix for a problemwith the page-table-entry inversion that is done as a mitigation for the L1TF speculative-executionvulnerability. The first batch includes: 4.18.2, 4.17.16, 4.14.64, 4.9.121, 4.4.149, and 3.18.119. The second batch is: 4.18.3, 4.17.17, 4.14.65, 4.9.122, and 4.4.150. Users should upgrade, presumably tosomething in the second batch unless they are running the 3.18 series.

Security updates have been issued by Debian (intel-microcode, keystone, php-horde-image, and xen), Fedora (rsyslog), openSUSE (apache2, clamav, kernel, php7, qemu, samba, and Security), Oracle (mariadb and qemu-kvm), Red Hat (docker, mariadb, and qemu-kvm), Scientific Linux (mariadb and qemu-kvm), SUSE (GraphicsMagick, kernel, kgraft, mutt, perl-Archive-Zip, python, and xen), and Ubuntu (postgresql-10, postgresql-9.3, postgresql-9.5, procps, and webkit2gtk).

As of this writing, Linus Torvalds has pulled just over 7,600 non-mergechangesets into the mainline repository for the 4.19 development cycle.4.19 thus seems to be off to a faster-than-usual start, perhaps because theone-week delay in the opening of the merge window gave subsystemmaintainers a bit more time to get ready. There is, as usual, a lot ofinteresting new code finding its way into the kernel, along with the usualstream of fixes and cleanups.

Over at Google's Project Zero blog, Natalie Silvanovich looks at some of the bugs the project has found in WebAssembly, which is a binary format to run code in the browser for web applications. She also looks to the future: "There are two emerging features of WebAssembly that are likely to have a security impact. One is threading. Currently, WebAssembly only supports concurrency via JavaScript workers, but this is likely to change. Since JavaScript is designed assuming that this is the only concurrency model, WebAssembly threading has the potential to require a lot of code to be thread safe that did not previously need to be, and this could lead to security problems.WebAssembly GC [garbage collection] is another potential feature of WebAssembly that could lead to security problems. Currently, some uses of WebAssembly have performance problems due to the lack of higher-level memory management in WebAssembly. For example, it is difficult to implement a performant Java Virtual Machine in WebAssembly. If WebAssembly GC is implemented, it will increase the number of applications that WebAssembly can be used for, but it will also make it more likely that vulnerabilities related to memory management will occur in both WebAssembly engines and applications written in WebAssembly."

The Debian project is celebrating the 25th anniversary of its founding by Ian Murdock on August 16, 1993. The "Bits from Debian" blog had this to say: "Today, the Debian project is a large and thriving organization with countless self-organized teams comprised of volunteers. While it often looks chaotic from the outside, the project is sustained by its two main organizational documents: the Debian Social Contract, which provides a vision of improving society, and the Debian Free Software Guidelines, which provide an indication of what software is considered usable. They are supplemented by the project's Constitution which lays down the project structure, and the Code of Conduct, which sets the tone for interactions within the project.Every day over the last 25 years, people have sent bug reports and patches, uploaded packages, updated translations, created artwork, organized events about Debian, updated the website, taught others how to use Debian, and created hundreds of derivatives." Happy birthday to the project from all of us here at LWN.

Greg Kroah-Hartman has released a new batch of stable kernels: 4.18.1, 4.17.15, 4.14.63, 4.9.120, and 4.4.148. These include the fixes for the L1 terminal fault vulnerability and a fewother fixes here and there. Users should upgrade.

Security updates have been issued by Debian (fuse), Fedora (cri-o, gdm, kernel-headers, postgresql, units, and wpa_supplicant), Mageia (iceaepe, kernel-linus, kernel-tmb, and libtomcrypt), openSUSE (aubio, libheimdal, nemo-extensions, and python-Django1), Red Hat (flash-plugin), SUSE (apache2, kernel, php7, qemu, samba, and ucode-intel), and Ubuntu (gnupg).

The LWN.net Weekly Edition for August 16, 2018 is available.

Social networks are typically walled gardens; users of a service caninteract with other users and their content, but cannot see or interactwith data stored in competing services. Beyond that, though, these walledgardens have generally made it difficult or impossible to decide to switchto a competitor—all of the user's data is locked into a particular site. Overtime, that has been changing to some extent, but a new project has thepotential to make it straightforward to switch to a new service withoutlosing everything. The DataTransfer Project (DTP) is a collaborative project between several internetheavyweights that wants to "create an open-source, service-to-servicedata portability platform".

Security updates have been issued by CentOS (kernel), Debian (kernel, linux-4.9, postgresql-9.4, and ruby-zip), Fedora (cgit, firefox, knot-resolver, mingw-LibRaw, php-symfony, php-symfony3, php-symfony4, php-zendframework-zend-diactoros, php-zendframework-zend-feed, php-zendframework-zend-http, python2-django1.11, quazip, sox, and thunderbird-enigmail), openSUSE (python-Django and seamonkey), Oracle (kernel), Red Hat (kernel, kernel-rt, and redhat-virtualization-host), Scientific Linux (kernel), Slackware (openssl), SUSE (clamav, firefox, kernel, and samba), and Ubuntu (kernel, libxml2, linux, linux-aws, linux-kvm, linux-raspi2, linux-snapdragon, linux-hwe, linux-azure, linux-gcp, linux-lts-trusty, linux-lts-xenial, linux-aws, linux-raspi2, and samba).

A kernel bug that allows a remote denial of service via crafted packets wasfixed recently and the resulting patchwas merged on July 23. But an announcement of the flaw(which is CVE-2018-5390) was not released until August 6—a two-week window where userswere left in the dark. It was not just the patch that might have alertedattackers; the flaw was publicized in other ways, as well,before the announcement, which has led to some discussion of embargopolicies on the oss-security mailing list. Within free-software circles,embargoes are generally seen as a necessary evil, but delaying thedisclosure of an already-public bug does not sit well.

The Meltdown CPU vulnerability, first disclosed in early January, was frighteningbecause it allowed unprivileged attackers to easily read arbitrary memoryin the system. Spectre, disclosed at the same time, was harder to exploitbut made it possible for guests running in virtual machines to attack thehost system and other guests. Both vulnerabilities have been mitigated tosome extent (though it will take a long time to even findall of the Spectre vulnerabilities, much less protect against them). But now the newly disclosed "L1 terminal fault" (L1TF) vulnerability(also going by the name Foreshadow) brings back boththreats: relativelyeasy attacks against host memory from inside a guest. Mitigations areavailable (and have been mergedinto the mainline kernel), but they will be expensive for some users.

Security updates have been issued by Arch Linux (thunderbird), Debian (gdm3 and samba), openSUSE (cgit and lxc), SUSE (grafana, kafka, logstash, openstack-monasca-installer and samba), and Ubuntu (gdm3 and libarchive).

Hundreds (at least) of kernel bugs are fixed every month. Given thekernel's privileged position within the system, a relatively large portionof those bugs have security implications. Many bugs are relatively easilynoticed once they are triggered; that leads to them being fixed. Somebugs, though, can be hard to detect, a result that can be worsened by thedesign of in-kernel APIs. A proposed change to how user-space accessorswork will, hopefully, help to shine a light on one class of stealthy bugs.

Security updates have been issued by Debian (blender, openjdk-8, postgresql-9.6, and sam2p), Fedora (libmspack, mingw-glib2, mingw-glibmm24, and rsyslog), Mageia (blender, glpi, godot, kernel, lftp, libjpeg, libsndfile, libsoup, mariadb, mp3gain, openvpn, and soundtouch), openSUSE (cgit, libvirt, mailman, NetworkManager-vpnc, and sddm), Slackware (bind), and SUSE (ffmpeg, glibc, and libvirt).

Linus has released the 4.18 kernel."It was a very calm week, and arguably I could just have released onschedule last week, but we did have some minor updates."Some of the significant features in this release includeunprivileged filesystem mounts,restartable sequences,a new zero-copy TCP receive API,support for active state management forpower domains,the AF_XDP mechanism forhigh-performance networking,the core bpfilter packet filterimplementation,and more. See the KernelNewbies 4.18 page formore details.

"Mounting" a filesystem is the act of making it available somewhere in thesystem's directory hierarchy. But a mount operation doesn't just glue adevice full of files into a specific spot in the tree; there is a whole setof parameters controlling how that filesystem is accessed that can bespecified at mount time. The handling of these mount parameters is thelatest obstacle to getting the proposed newmounting API into the mainline; should the new API reproduce what isarguably one of the biggest misfeatures of the current mount()system call?

Security updates have been issued by CentOS (java-1.7.0-openjdk, openslp, and yum-utils), Fedora (exiv2, kernel-headers, kernel-tools, libgit2, and thunderbird-enigmail), openSUSE (blueman, cups, gdk-pixbuf, libcdio, libraw, libsoup, libtirpc, mysql-community-server, polkit, python-mitmproxy, sssd, virtualbox, and webkit2gtk3), Oracle (kernel), Red Hat (cobbler), SUSE (ceph, firefox, NetworkManager-vpnc, openssh, and wireshark), and Ubuntu (openjdk-7 and openjdk-8).

The bzip2 compression algorithm has been slowly falling out offavor, but is still used heavily across the net. A searchfor "bzip2 source" returns bzip.org as the first three results. But itwould seem that the owner of this domain has let it go, and it is now parkedand running ads. So we no longer have an official home forbzip2. If a new repository or tarball does turn up at thatdomain, it should be looked at closely before being trusted. (Thanks toJason Kushmaul).

Greg Kroah-Hartman has released the 4.17.14, 4.14.62, 4.9.119, 4.4.147, and 3.18.118 stable kernels. There are importantfixes in each and users should upgrade.

Security updates have been issued by Arch Linux (kernel, linux-hardened, linux-lts, and linux-zen), Debian (kamailio and wpa), Fedora (kernel-headers, kernel-tools, moodle, and vim-syntastic), and openSUSE (clamav, enigmail, and java-11-openjdk).

The LWN.net Weekly Edition for August 9, 2018 is available.

The Speck cipheris geared toward good performance in software, which makes it attractivefor smaller, often embedded, systems with underpowered CPUs that lackhardware crypto acceleration. But it alsocomes from the US National Security Agency (NSA), which worries lots ofpeople outside the US—and, in truth, a fair number of US citizens as well.The NSA has earned a reputation for promulgating various types ofcryptographic algorithms with dubious properties. While the technicalarguments against Speck, which is a fairly simple and straightforwardalgorithm with little room for backdoors, have not been all thatcompelling, the political arguments are potent—to the point where it isbeing dropped by the mainproponent for including it in the kernel.

Once upon a time, the only way to control how the kernel's CPU schedulertreated any given process was to adjust that process's priority. Priorities are nolonger enough to fully control CPU scheduling, though, especially whenpower-management concerns are taken into account. The utilizationclamping patch set from Patrick Bellasi is the latest in a series ofattempts to allow user space to tell the scheduler more about any specificprocess's needs.

Security updates have been issued by Debian (slurm-llnl), Fedora (libmspack), openSUSE (cups, kernel, kernel-firmware, libcgroup, and ovmf), Oracle (kernel), and SUSE (cups, enigmail, libcdio, and pidgin).

The O'Reilly Open Source Conference (OSCON) returned to Portland, Oregonin July for its20th meeting. Previously, we covered some retrospectives and community-management talksthat were a big part of the conference. Of course, OSCON is also atechnology conference, and there were lots of talks on various open-sourcesoftware platforms and tools.Subscribers can read on for the second part of an OSCON report by guest authorJosh Berkus.

Security updates have been issued by Debian (kernel), Fedora (ceph, exiv2, myrepos, and seamonkey), openSUSE (libofx and znc), Oracle (kernel), Red Hat (qemu-kvm-rhev), SUSE (clamav, kernel, and rubygem-sprockets-2_12), and Ubuntu (gnupg, lftp, libxcursor, linux-hwe, linux-azure, linux-gcp, linux-raspi2, and lxc).

Software patents account for morethan half of all utility patents granted in the US over the past fewyears. Clearly, many companies see these patents as a way to fortune and growth, even whilesoftware patents are hated by many people working in the free andopen-source movements. The field of patenting has now joined the onwardmarch of artificial intelligence. This was the topic of a talk at OSCON2018 by Van Lindberg, an intellectual-property lawyer, board member andgeneral counsel for the Python Software Foundation, and author of the bookIntellectualProperty and Open Source. The disruption presented by deeplearning ranges from modest enhancements that have already beenexploited—making searches for prior art easier—to harbingers ofautomatic patent generation in the future.

The WireGuard VPN tunnel has beenunder development — and attracting attention — for a few years now; LWN ran a review of it in March. While WireGuardcan be found in a number of distribution repositories, it is not yetshipped with the mainline kernel because its author, Jason Donenfeld, hasn'tgotten around to proposing it for upstreaming. That changed on onJuly 31, when Donenfeld postedWireGuard for review. Getting WireGuard itself into the mainline would probablynot be all that hard; merging some of the support code it depends on could beanother story, though.

Ars technica coversthe release of Android 9 "Pie". "Android Pie is a major update for Android. Large chunks of the OS get a UI makeover in line with Google's updated Material Design guidelines. There is an all-new notification panel, a reworked recent-apps screen, new settings, and tons of system UI changes. There's support for devices with notched displays (like the iPhone X) and a gesture navigation system (also like the iPhone X). So far, battery life on the preview builds has been great, with improvements like the AI-powered adaptive battery system, a new auto-brightness algorithm, and changes to CPU background processing."

Version60 of the Thunderbird email client has been released. "Thisversion of Thunderbird is packed full of great new features, fixes, andchanges that improve the user experience and make for a worthwhileupgrade." There are improvements in calendar management and thehandling of attachments, among other things; see therelease notes for details.

Greg Kroah-Hartman has released stable kernels 4.17.13, 4.14.61, 4.9.118, and 4.4.146. They all contain important fixes andusers of those series should upgrade.

Security updates have been issued by Arch Linux (cgit, python-django, and python2-django), Debian (ant, cgit, libmspack, python-django, symfony, vim-syntastic, and xml-security-c), Fedora (kernel-headers, libao, libvorbis, mingw-gdal, mingw-xerces-c, and python-XStatic-jquery-ui), openSUSE (bouncycastle, java-10-openjdk, libgcrypt, libsndfile, mutt, nautilus, ovmf, python-dulwich, rpm, util-linux, wireshark, and xen), Oracle (kernel), Red Hat (kernel, openslp, rhvm-setup-plugins, and xmlrpc), and SUSE (glibc, kernel-firmware, libsoup, openssl, and yast2-ftp-server).

Richard Hughes announcesthat the Linux Vendor Firmware Servicewill start distributing firmware updates for Lenovo systems."Obviously, this is a big deal. Tens of thousands of people arelikely to be offered a firmware update in the next few weeks, and hundredsof thousands over the next few months."

As expected, Linus has released 4.18-rc8rather than the final 4.18 release. "So as already mentioned acouple of times in some of the relevant threads, this last week wasn'tentirely painless, and 4.18 ended up being one of those releases that getsan extra week of rc testing before release".

Greg Kroah-Hartman has announced the release of three new stable kernels:4.17.12, 4.14.60, and 4.9.117. As usual, there are fixes throughoutthe kernel tree, so users of those kernels should upgrade.

Security updates have been issued by Debian (busybox, graphicsmagick, and libmspack), Fedora (pam_yubico), Scientific Linux (openslp), Slackware (lftp), SUSE (cups, libtirpc, and thunderbird), and Ubuntu (clamav).

Whenever one is engaged in large-scale changes to a software project, it isnice to have some assurance that regressions are not being introduced inthe process. Test suites can be helpful in that regard. But while thetesting of low-level components can be relatively straightforward, testingat the user-interface level can be harder. Web applications, which mustalso interact with web browsers, can be especially challenging in thisregard. While working on just this sort ofproject, your editor finally got around to looking at SeleniumWebDriver as a potential source of help for the testing problem.

Security updates have been issued by Debian (busybox and mutt), Fedora (bibutils and wireshark), openSUSE (glibc and rsyslog), Slackware (blueman), SUSE (cups, ovmf, and polkit), and Ubuntu (bouncycastle, libmspack, and python-django).

The LWN.net Weekly Edition for August 2, 2018 is available.

<p>Memory allocation for applications is a bit of a balancing act betweenvarious factors including CPU performance, memory efficiency, and how thememory is actually being allocated and deallocated by the application. Different programs may have diverse needs,but it is often the kind of workload that the application is expected to handle thatdetermines which memory allocator performs best. That argues for adiversity of memory allocators (and allocation strategies) but, on theother hand, thatcomplicates things for Linux distributions. As a result, Fedora isdiscussing ways to rein in the spread of allocators used by its packages.

The OpenWrt community has announcedthe first release of the OpenWrt 18.06 stable version series. "Itincorporates over 4000 commits since branching the previous LEDE 17.01release and has been under development for well over a year. With thisrelease, the re-merged OpenWrt project attempts to define a baseline for future development based on the technological modernizationand refined release processes done by the former LEDE project."

A PEP that has been around for a while, without being either accepted orrejected, was reintroduced recently on the python-ideas mailing list.PEP 505("None-aware operators") would provide some syntactic sugar, in the form ofnew operators, to handle cases where variables might be the special None value. It is a featurethat other languages support, but has generally raised concerns about being"un-Pythonic" over the years. At this point, though, the Python projectstill needs to figure out how it will begoverned—and how PEPs can be accepted or rejected.

Security updates have been issued by Debian (ruby2.3), Fedora (java-1.8.0-openjdk, java-openjdk, poppler, python-cryptography, and zziplib), Oracle (openslp), Red Hat (Red Hat Virtualization), and SUSE (kernel).

Version 2.28 of the GNU C Library is out. Changes include support forIntel's "Control-flow Enforcement Technology", Unicode 11.0.0 support, awrapper for statx(), ISO Cthreads support, several security fixes, and more.

The O'Reilly Open SourceConference (OSCON) returned to Portland, Oregon this July for the 20th convocation of this venerable gathering. While some of theprogram focused on retrospectives, there were also talks and tutorials onmultiple technical topics and open-source community management. To give youa feel for the whole conference, we will explore it in a two-part article. This installment will cover a retrospective of opensource and some presentations on releasing projects as open source at yourorganization. A second article will include a few of the technicaltopics at the conference.