LWN.net

Security updates have been issued by Debian (389-ds-base, firefox-esr, and symfony), Fedora (poppler), SUSE (audit, ovmf, and webkit2gtk3), and Ubuntu (aria2, FFmpeg, gnome-shell, and sudo).

Control groups are a useful mechanism for managing resource usage in thesystem, but what happens when the control groups themselves become aresource problem? In a plenary session at the 2019 Linux Storage,Filesystem, and Memory-Management Summit, Roman Gushchin described problemshe has been facing with deleted control groups that take their time beforeactually going away. Some of these problems have been fixed, but the issuehas not been truly resolved.

Shakeel Butt started his 2019 Linux Storage, Filesystem, andMemory-Management Summit session by noting that memory makes up a big partof the total cost of equipping a data center. As a result, data-centeroperators try to make the best use of memory they can, generallyovercommitting it significantly. In this session, Butt described a schemein use at Google to try to improve memory utilization; while the need forthe described functionality was generally agreed upon, the developers inthe room were not entirely happy with the solution presented.

The performance of the fget() function in the kernel was the topicof a discussion led by Dave Watson at the 2019 Linux Storage, Filesystem,and Memory-Management Summit (LSFMM).fget() is used to take a reference to afile (i.e. bump a reference count), based on its file descriptor, and toreturn the struct file pointer for it; references are dropped withfput(). Some recent profiling at Watson'semployer, Facebook, found the function to be taking a sizable portion ofthe CPU time for some applications, so he wanted to talk about some of thethings he has tried to make that situation better.

In a combined filesystem and storage session at the 2019 Linux Storage, Filesystem, and Memory-Management Summit (LSFMM), Dennis Zhou wanted to talkabout discard, which is the process of sending commands(e.g. TRIM) to block devices to indicateblocks that are no longer in use. Discard is a "serious black box", hesaid; it is a third way to interact with a drive, but Linux developers haveno real insight into what its actual effects will be. That can lead toperformance and other problems.

While persistent memory is normally valued for its persistence, there isalso a subcurrent of interest in using it in settings where persistence isnot important. In particular, the fact that this memory is relativelyinexpensive makes it appealing to use instead of ordinary RAM inbudget-conscious settings. At the 2019 Linux Storage, Filesystem, andMemory-Management Summit, two sessions in the memory-management tracklooked at how the kernel's NUMA mechanism could be pressed into service tomanage non-persistent uses of persistent memory.

Sometimes, the kernel's no-regression rule may not have the desiredresult. Andrea Arcangeli led a session at the 2019 Linux Storage,Filesystem, and Memory-Management Summit to make the point that the recentreversion of a fix after a performance regression was reported has led toworse performance overall — with, as is his wont, a lot of technicalinformation to back up that point. With a wider understanding of what is atstake here, he hopes, the reversion can itself be reverted.

Security updates have been issued by Debian (jquery, librecad, and phpbb3), Fedora (bubblewrap, java-11-openjdk, libvirt, openssh, and pacemaker), Mageia (virtualbox), openSUSE (chromium, ImageMagick, and java-11-openjdk), and SUSE (openssl-1_1).

There is a new Firefox browser release available; its main claim to fame isthat it has a fix for the certificate issue that disabled all extensions.

Linus has released the 5.1 kernel, right onschedule. Some of the significant changes in the release include BPF spinlocks, more year-2038 preparation, the TEO CPU-idle governor, The io_uring fast asynchronous I/O mechanism,initial support for pidfds (file descriptors that refer to a process), theSafeSetID security module, and much more.See the KernelNewbies 5.1page for lots of details.

The5.0.13 and4.19.40stable kernel updates have been released; they were originally due onMay 6, but went out ahead of schedule.

Compound pages are created by the kernel asa way of combining a number of small pages into a single, larger unit.Such pages are implemented as a single "head page" at the beginning,followed by a number of "tail pages". Matthew Wilcox has concluded thatit would be beneficial to minimize the use of tail pages in the kernel; heran a session during the memory-management track at the 2019 Linux Storage,Filesystem, and Memory-Management Summit to explore how that could be done.The discussion ranged widely, veering into the representation of DMA I/Ooperations, but few hard conclusions were reached.

Among the many other things crammed into the page structurethat is usedto represent a page of memory in the kernel is a set of flags to track thestate of the page. These flags have been in short supply for some time;LWN looked at the problem nearly ten yearsago. Jérôme Glisse ran a session during the memory-management track of the2019 Linux Storage, Filesystem, and Memory-Management Summit to exploreways of making some flags available for new uses. While there may be someeasily available bits in the field that holds the page flags, obtaining asignificant number of them may be tricky.

The5.0.12,4.19.39,4.14.116, and4.9.173stable kernel updates have been released; each contains another set ofimportant fixes.

The expiration of the extension signing certificate has evidently caused allextensions to be disabled, leading to a fair amount of discomfort onthe net. There is evidently afix being rolled out, but it requires that the "Studies" mechanism beenabled in the privacy preferences. Meanwhile, the best short-termapproach seems to be to avoid restarting Firefox if possible.

The Alliance for Open Media developedthe AV1 patent-free video codec and sponsors the development of dav1d, a referenceoptimized decoder for AV1. The 0.3.0release of dav1d is now available. "This third release continues to increase the ARM and SSSE3 speed, with more optimizations, as announced, and we get between 12 and 25% speed increases on those CPUs, depending on the samples.However, more surprisingly, we got a speedup on AVX-2 CPU, by optimizing the MSAC (entropy decoding), while we did not find a good solution in the past. This brings 4-5% speed improvements, which is quite huge, knowing the maturity of the AVX-2 code."

Security updates have been issued by Debian (linux-4.9 and otrs2), Fedora (gradle, java-1.8.0-openjdk, jetty, kernel, ruby, and runc), openSUSE (dovecot23, jasper, libsoup, ntfs-3g_ntfsprogs, and webkit2gtk3), SUSE (openssl), and Ubuntu (python-gnupg).

Version 9.1 of the GCC compiler suite is out. "In this release C++17 support is no longer marked experimental. TheC++ front-end implements the full C++17 language (already previous GCCmajor version implemented that) and the C++ standard library support isalmost complete. The C++ front-end and library also have numerous furtherC++2a draft features. GCC has a new front-end for the D language.GCC 9.1 has newly partial OpenMP 5.0 support and almostcomplete OpenACC 2.5 support." See this page for anextensive list of changes.

The userfaultfd()system call allows one process to handle page faults for another — in userspace. Its original use case was to support transparent containermigration, but other uses have developed over the years. At the 2019 LinuxStorage, Filesystem, and Memory-Management Summit, Andrea Arcangelidescribed a scheme to add write-protectionsupport to userfaultfd(). After a year of lost time fightingspeculative-execution problems, Arcangeli is about ready to move this featureinto the mainline.

Since its inception, the DAX mechanism (which provides for direct access tofiles stored on persistent memory) has been seen as somewhat experimentaland incomplete. At the 2019 Linux Storage, Filesystem, andMemory-Management Summit, Dan Williams ran a session where he said thatperhaps the time has come to end that experiment. Some of theunimplemented DAX features may never actually need to be implemented, andit might just be possible to declare DAX finished. But first there are afew more details to take care of.

Stable kernels 5.0.11, 4.19.38, 4.14.115, and 4.9.172 have been released. They all containimportant fixes and users should upgrade.

Security updates have been issued by Debian (proftpd-dfsg and signing-party), Fedora (php-horde-horde and php-horde-turba), and Ubuntu (php5).

Version 1.0.0 of the GNU Guix package manager has been released. "This 1.0 release is a major milestone for Guix. It represents 7 yearsof hard work with more than 40,000 commits by 260 people, 19 releases,and an equally amazing amount of work on documentation, translation,artwork, web design, mentoring, outreach, and many other activities thattogether have made it a thriving project." See thisblog entry for more information.

The LWN.net Weekly Edition for May 2, 2019 is available.

James Bottomley began his talk at the 2019 Linux Storage, Filesystem, andMemory-Management Summit (LSFMM) by noting that the main opposition to his ideaswas not present at the summit, which was likely to mean the ideas got a mucheasier reception than they would have otherwise. In particular, PeterZijlstra and Ingo Molnar expressed some strong reservations to the workthat Bottomley's colleague Mike Rapoport postedrecently; none of those three were in attendance at LSFMM. The idea is touse address spaces to reduce the attack surface available to virtualmachines (VMs) and containers such that kernel bugs of various sorts haveless reach on multi-tenant systems.

The Android system is designed to provide a responsive user experience onsystems that, in a relative sense at least, have limited amounts ofCPU and memory. Doing so requires a number of techniques, includingregular use of a low-memory process killer, that are not seen elsewhere. In amemory-management-track session at the 2019 Linux Storage, Filesystem, andMemory-Management Summit, Suren Baghdasaryan covered a number of issuesrelated to how Android ensures that interactive processes have enoughmemory to get their jobs done.

"People think that memory encryption sounds really cool; it will make mysystem more secure so I want it". At least, that is how Dave Hansencharacterized the situation at the beginning of a session on the topicduring the memory-management track at the 2019 Linux Storage, Filesystem,and Memory-Management Summit. This session, also led by Kirill Shutemov,covered a number of aspects of the memory-encryption problem on Intelprocessors and beyond. One clear outcome of the discussion was also raisedby Hansen at the beginning: users of memory encryption need to think hardabout where that extra security is actually coming from.

The splice()system call is, at its core, a write operation; it attempts to implementzero-copy I/O by moving pages from a pipe to a file. At the 2019 LinuxStorage, Filesystem, and Memory-Management Summit, Miklos Szeredi describeda nascent idea for rsplice() — a "reverse splice" system call.There were not a lot of definitive outcomes from this discussion, but onething was clear: rsplice() needs a much better description (andsome code posted) before the development community can begin to form anopinion on it.

Security updates have been issued by Fedora (libmediainfo, php-horde-horde, and php-horde-turba), SUSE (hostinfo, supportutils, libjpeg-turbo, and openssl), and Ubuntu (dovecot, libpng1.6, and memcached).

The 2019 version of the Linux Storage, Filesystem, and Memory-ManagementSummit opened with a plenary talk by Brendan Gregg on observing the stateof Linux systems using BPF. It is, he said, an exciting time; theBPF-based "superpowers" being added to the kernel are growing in capability andmaturity. It is now possible to ask many questions about what is happeningin a production Linux system without the need for kernel modifications oreven basic debugging information.

Fedora Magazine has announced therelease of Fedora 30. "Fedora Editions are targeted outputs geared toward specific “showcase” uses. Since we first started using this concept in the Fedora 21 release, the needs of the community have continued to evolve. As part of Fedora 30, we’re combining cloud and server into the Fedora Server edition. We’re bringing in Fedora CoreOS to replace Fedora Atomic Host as our container-focused deliverable in the Fedora 30 timeframe — stay tuned for that. The Fedora Workstation edition continues to focus on delivering the latest in open source desktop tools.Of course, we produce more than just the editions. Fedora Spins and Labs target a variety of audiences and use cases, including the Internet of Things. And, we haven’t forgotten our alternate architectures, ARM AArch64, Power, and S390x."

Security updates have been issued by CentOS (kernel, openwsman, and ovmf), Debian (gst-plugins-base1.0 and libvirt), Fedora (libX11, poppler, python-urllib3, samba, and wpewebkit), openSUSE (GraphicsMagick), SUSE (atftp, glibc, libssh2_org, and wpa_supplicant), and Ubuntu (wavpack).

Determining the license that any given package uses can be difficult, but it isessential in order to properly comply with that license and, thus, thedeveloper's wishes. There is an enormous amount of "open source" software available these days thatis not clearly licensed, which is where the ClearlyDefined project comesin. The project is collecting a curated list of packages,source location, and license information; some of that collection can beautomated, but ClearlyDefined is targeting the community to providecuration in the form of cleanups and additions.

The Apache Software Foundation (ASF) and GitHub have announced [ASF,GitHub]that all ASF projects using Git have moved to GitHub and the ASF Git service hasbeen decommissioned. (Thanks to Paul Wise)Update: ASF has anotherannouncement with more details. An older git service was decommissioned,but ASF projects are still available on https://gitbox.apache.org/. "Asstated above, our GitHub integration is an augmentation of our existingservice. It is available to all committers on git-based projects to makeuse of, should they so wish. All new git repositories will automatically beavailable on both GitHub and Gitbox." (Thanks to Lars Francke)

Security updates have been issued by Arch Linux (chromium, libpng, and openssh), Debian (checkstyle, evolution, gst-plugins-base0.10, gst-plugins-base1.0, imagemagick, libpng1.6, monit, and systemd), Fedora (aria2, php-symfony, php-symfony3, php-symfony4, and python-jinja2), openSUSE (ceph, libssh2_org, libvirt, php7, python3, samba, wget, and xerces-c), Red Hat (rh-python35-python), Slackware (bind), SUSE (libssh2_org), and Ubuntu (evince, gst-plugins-base0.10, gst-plugins-base1.0, and mysql-5.7).

Linus has released the 5.1-rc7 kernelprepatch for testing. "But it's all pretty tiny. Plus about 30% ofthe patches are marked for stable, so on the whole it really does feel like5.1 is on target for a regular release next weekend."

The5.0.10,4.19.37,4.14.114,4.9.171,4.4.179, and3.18.139stable kernel updates have all been released; each contains a moderatelylarge set of important fixes.

Adrian Ratiu continues his series on eBPF with part 3, which looks at various ways to write and build eBPF programs. It starts by looking at using "restricted C" with the LLVM eBPF compiler, moves into looking at the BPF Compiler Collection (BCC), then bpftrace, and finally the IOVisor cloud-based eBPF tools."Not everyone has kernel sources at hand, especially in production, and it's also a bad idea in general to tie eBPF-based tools to a specific kernel source revision. Designing and implementing the interactions between eBPF program's backends, frontends, loaders and data structures can be very complex, error-prone and time consuming, especially in C which is considered a dangerous low-level [language]. In addition to these risks developers are also in a constant danger of re-inventing the wheel for common problems, with endless design variations and implementations. To alleviate all these pains is why the BCC project exists: it provides an easy-to-use framework for writing, loading and running eBPF programs, by writing simple python or lua scripts in addition to the 'restricted C' as exemplified above."

The recently discovered vulnerability inThunderbolt has restarted discussions about protecting the kernelagainst untrusted, hotpluggable hardware. That vulnerability, known as Thunderclap, allows a hostile externaldevice to exploit Input-OutputMemory Management Unit (IOMMU) mapping limitations and access systemmemory it was not intended to. Thunderclap can be exploited byUSB-C-connected devices; while we have seen USB attacks in the past, thisvulnerability is different in that PCI devices, often considered astrusted, can be a source of attacks too. One way of stopping those attackswould be to make sure that the IOMMU is used correctly and restricts the deviceto accessing the memory that was allocated for it. Lu Baolu has postedan implementation of that approach in the form of bounce buffers foruntrusted devices.

Security updates have been issued by Debian (gpac and mercurial), Fedora (kernel-headers and kernel-tools), openSUSE (GraphicsMagick, kauth, lxc, lxcfs, python, qemu, and xmltooling), SUSE (freeradius-server, ImageMagick, libvirt, samba, and wireshark), and Ubuntu (bind9).

Over at Opensource.com, Jason Brock tries out Linux graphics tools, with an eye toward their ability to replace the proprietary tools he uses on a day-to-day basis. Overall, the tools held their own for a variety of tasks (e.g. logo and ad design, publication layout), though the lack of a certain type of tool brought the overall grade down to a B+: "The lack of available wireframing and prototyping applications really brought down the average, but I'd still call it a successful exercise. As I mentioned at the beginning, design is a craft and it relies on collaboration. All of the tools I looked at—Inkscape, LibreDraw, GIMP, and Scribus—can run just as well on Windows or MacOS as they do on any Linux distribution. The ability to create robust artwork and share editable files with stakeholders and colleagues on the platform of their choice means that a serious argument could be made that these tools are even more versatile than their proprietary counterparts."

The release of the 5.1-rc6 kernel prepatchon April 21 indicates that the 5.1 development cycle is getting closeto its conclusion. So naturally the time has come to put together somestatistics describing where the changes merged for 5.1 came from. It is,for the most part, a fairly typical development cycle.

Security updates have been issued by Debian (putty and systemd), Fedora (kernel, kernel-headers, and kernel-tools), Gentoo (ming and qemu), openSUSE (openexr and slurm), SUSE (ImageMagick, jasper, ntfs-3g_ntfsprogs, openssh, and webkit2gtk3), and Ubuntu (php5 and tcpflow).

The LWN.net Weekly Edition for April 25, 2019 is available.

An April Fools joke that went sour seems to be at least the proximate causefor a rather large upheaval in the Devuan community.For much of April 1 (or March 31 depending on time zone), theDevuan web site looked like it had been taken over by attackers, which was worrisome to many, but it was all a prank.The joke wasclever, way over the top, unprofessional, or some combination of those,depending on who is describing it, but the incident and the threads on the devuan-dev mailinglist have led to rancor, resignations, calls for resignations, and more.

The Mozilla Blog introducesMozilla's 2019 InternetHealth Report. "In the Report’s three spotlight articles, weunpack three big issues: One examines the needfor better machine decision making — that is, asking questions likeWho designs the algorithms? and What data do they feed on?and Who is being discriminated against? Another examines ways to rethinkthe ad economy, so surveillance and addiction are no longer designnecessities. The third spotlight article examinesthe rise of smart cities, and how local governments can integrate techin a way that serves the public good, not commercial interests."

In his keynote at the 2019 Legal andLicensing Workshop (LLW), longtime workshop participant Andrew Wilson looked at the past, but he went much further back than, say, the history of freesoftware—or even computers. His talk looked at technological liberty inthe context of classical liberal philosophic thinking. He mapped some ofthat thinking to the world of free and open-source software (FOSS) and tosome other areas where our liberties are under attack.

Security updates have been issued by Arch Linux (dovecot, flashplugin, ghostscript, and jenkins), Fedora (glpi, hostapd, python-urllib3, and znc), openSUSE (apache2, audiofile, libqt5-qtvirtualkeyboard, php5, and SDL2), Scientific Linux (kernel), SUSE (curl and dovecot23), and Ubuntu (advancecomp and freeradius).

The problem of "sustainability" for open-source software is a common topic ofconversation in our community these days. We covered a talk by Bradley Kuhn onsustainability a month ago. Another longtime community member, Luis Villa,gave his take on the problem of making open-source projects sustainable atthe 2019 Legal and Licensing Workshop (LLW) in Barcelona. Villa is one of theco-founders of Tidelift, which is acompany dedicated to helping close the gap so that the maintainers ofopen-source projects get paid in order to continue their work.

Security updates have been issued by CentOS (java-1.7.0-openjdk), Debian (ghostscript and wget), Gentoo (apache, glib, opendkim, and sqlite), Red Hat (kernel, kernel-alt, kernel-rt, ovmf, polkit, and python27-python), Scientific Linux (java-1.7.0-openjdk), and SUSE (php72).