LWN.net

I was sure that somewhere there must bephysically-lightweight sensors with simple power, simple networking, anda lightweight protocol that allowed them to squirt their data down thenetwork with a minimum of overhead. So my interest was piqued when Jan-Piet Mens spoke at FLOSSUK's Spring Conference on "Small Things for Monitoring". Once he started passingworking demonstration systems around the room without interrupting thedemonstration, it was clear that MQTT was what I'd been looking for.

Security updates have been issued by Arch Linux (freetype2, libraw, and powerdns), CentOS (389-ds-base and kernel), Debian (php5, prosody, and wavpack), Fedora (ckeditor, fftw, flac, knot-resolver, patch, perl, and perl-Dancer2), Mageia (cups, flac, graphicsmagick, libcdio, libid3tag, and nextcloud), openSUSE (apache2), Oracle (389-ds-base and kernel), Red Hat (389-ds-base and flash-plugin), Scientific Linux (389-ds-base), Slackware (firefox and wget), SUSE (xen), and Ubuntu (wget).

For those who are curious about the rather complex way in which X serverpointer acceleration works, Peter Hutterer has put together a four-partseries on the topic:part 1,part 2,part 3,andpart 4."The input for the acceleration profile is a speed in mickeys, a threshold (in mickeys) and a max accel factor (unitless). Mickeys are a bit tricky. This means the acceleration is device-specific, the deltas for a mouse at 1000 dpi are 20% larger than the deltas for a mouse at 800 dpi (assuming same physical distance and speed)".

The LWN.net Weekly Edition for May 10, 2018 is available.

The CoreOS blog is carrying anarticle describing the path forward now that CoreOS is owned by RedHat. "Since Red Hat’s acquisition of CoreOS was announced, wereceived questions on the fate of Container Linux. CoreOS’s first project,and initially its namesake, pioneered the lightweight, 'over-the-air'automatically updated container native operating system that fast rose inpopularity running the world’s containers. With the acquisition, ContainerLinux will be reborn as Red Hat CoreOS, a new entry into the Red Hatecosystem. Red Hat CoreOS will be based on Fedora and Red Hat EnterpriseLinux sources and is expected to ultimately supersede Atomic Host as RedHat’s immutable, container-centric operating system." Someinformation can also be found in thisRed Hat press release.

The amount of available data is growing larger these days, to the pointthat some data sets are far larger than any one company or organization can create and maintain. So companies andothers want to share data in ways that are similar to how they share code. Some of thosecompanies are members of the Linux Foundation (LF), which is part of why thatorganization got involved in the process of creating licenses for thisdata. LF VP of Strategic Programs Mike Dolan came to the 2018 Legal andLicensing Workshop (LLW) to describe how the Community Data LicenseAgreement (CDLA) came about.

Mozilla has released Firefox 60. From the releasenotes: "Firefox 60 offers something for everyone and a littlesomething extra for everyone who deploys Firefox in an enterprise environment. This release includes changes that give you more content and more ways to customize your New Tab/Firefox Home. It also introduces support for the Web Authentication API, which means you can log in to websites in Firefox with USB tokens like YubiKey.Firefox 60 also brings a new policy engine and Group Policy support forenterprise deployments. For more info about why and how to use Firefox inthe enterprise, see this blog post."

Stable kernels 4.16.8, 4.14.40, and 4.9.99 have been released. They all containimportant fixes and users should upgrade.

Security updates have been issued by Debian (kernel), Gentoo (rsync), openSUSE (Chromium), Oracle (kernel), Red Hat (kernel and kernel-rt), Scientific Linux (kernel), SUSE (kernel and php7), and Ubuntu (dpdk, libraw, linux, linux-lts-trusty, linux-snapdragon, and webkit2gtk).

Version 1.14 of theBattle for Wesnoth role-playing strategy game — the first release in over threeyears — is available. "Along with the long-awaited debut on Steam,this new release series brings forth a vast number of additions and changesin all areas: a new single-player campaign, a visual and functional refreshof the multiplayer lobby and add-ons manager, a refurbished display engine,new unit graphics and animations, and much more."

<p>In a plenary session on the second day of the Linux Storage, Filesystem,and Memory-Management Summit (LSFMM), Dave Chinner described his ideas fora virtual block address-space layer. It would allow "space accounting to beshared and managed at various layers in the storage stack". One of thetargets for this work is for filesystems on thin-provisioned devices, wherethe filesystem is larger than the storage devices holding it (and administrators areexpected to add storage as needed); in current systems, running out ofspace causes huge problems for filesystems and users because the filesystemcannot communicate that error in a usable fashion.

Security updates have been issued by Debian (wget), SUSE (patch), and Ubuntu (qpdf).

In a short filesystem-only discussion at the 2018 Linux Storage,Filesystem, and Memory-Management Summit (LSFMM), Jérôme Glisse wanted totalk about some (more) changes to support GPUs, FPGAs, and RDMA devices.In other talks at LSFMM, he discussedchanges to struct page in support of these kinds of devices, but here he was looking to discussother changes to support mapping a device's memory into multiple processes. It should benoted that I had a hard time following the discussion in this session, sothere may be significant gaps in the article.

The removal of an old joke from the GNU C Library manual might not seemlike the sort of topic that would inspire a heated debate. At times,though, a small action can serve as an inadvertent proxy for a moresignificant question, one which is relevant to both the developers and theusers of the project. In this case, that question would be: how is the project governed and whomakes decisions about which patches are applied?

At the 2018 Linux Storage, Filesystem, and Memory-Management Summit(LSFMM), Allison Henderson led a session to discuss an XFS feature she has beenworking on: parent pointers. These wouldbe pointers stored in extended attributes (xattrs) that would allow various tools toreconstruct the path for a file from its inode.In XFS repair scenarios, that path will help with reconstruction as well asprovide users with better information about where the problems lie.

Security updates have been issued by Debian (libdatetime-timezone-perl, libmad, lucene-solr, tzdata, and wordpress), Fedora (drupal7, scummvm, scummvm-tools, and zsh), Mageia (boost, ghostscript, gsoap, java-1.8.0-openjdk, links, and php), openSUSE (pam_kwallet), and Slackware (python).

The 4.17-rc4 kernel prepatch is out."Two thirds of the 4.17-rc4 patch is drivers, which sounds about right.Media, networking, rdma, input, nvme, usb. A little bit of everything, inother words." The codename has been changed, for the first timesince 4.10, to "Merciless Moray".

The mount()system call suffers from a number of different shortcomings that has ledsome to consider a different API. At last year's Linux Storage,Filesystem, and Memory-Management Summit (LSFMM), that someone wasMiklos Szeredi, who led a session to discuss hisideas for a new filesystem mounting API. Since then, David Howells has beenworking with Szeredi and VFS maintainer Al Viro on this API; at the 2018LSFMM, he presented that work.

The DMA zone (ZONE_DMA) is a memory-management holdover from thedistant past. Once upon a time, many devices (those on the ISA bus inparticular) could only use 24 bits for DMA addresses, and were thuslimited to the bottom 16MB of memory. Such devices are hard to find oncontemporary computers. Luis Rodriguez scheduled the lastmemory-management-track session of the 2018 Linux Storage, Filesystem, andMemory-Management Summit to discuss whether the time has come to removeZONE_DMA altogether.

A system's page tables are organized into a tree that is as many as fivelevels deep. In many ways those levels are all similar, but the kerneltreats them all as being different, with the result that page-tablemanipulations include a fair amount of repetitive code. During thememory-management track of the 2018 Linux Storage, Filesystem, andMemory-Management Summit, Kirill Shutemov proposed reworking how pagetables are maintained. The idea was popular, but the implementation islikely to be tricky.

Security updates have been issued by Debian (jackson-databind, quassel, and redmine), Fedora (community-mysql and php), Red Hat (chromium-browser), Scientific Linux (java-1.7.0-openjdk), and Slackware (seamonkey).

At a plenary session heldrelatively early during the 2018 Linux Storage, Filesystem, andMemory-Management Summit, the developers discussed a number of problemswith the kernel's get_user_pages() interface. During the waninghours of LSFMM, a tired (but dedicated) set of developers convened again inthe memory-management track tocontinue the discussion and try to push it toward a real solution.

<p>Chris Mason and Josef Bacik led a brief discussion on the block-I/Ocontroller for control groups (cgroups) in the filesystem track at the 2018 LinuxStorage, Filesystem, and Memory-Management Summit. Mostly they were justaiming to get feedback on the approach they have taken. They are trying toaddress the needs of their employer, Facebook, with regard to the latencyof I/O operations.

Memory hotplugging is one of the least-loved areas of the memory-managementsubsystem; there are many use cases for it, but nobody has taken ownershipof it. A similar situation exists for hardware pagepoisoning, a somewhat neglected mechanism for dealing with memory errors.At the 2018 Linux Storage, Filesystem, and Memory-Management summit, MichalHocko and Mike Kravetz dedicated a pair of brief memory-management tracksessions to problems that have been encountered in these subsystems, one ofwhich seems more likely to get the attention it needs than the other.

The memory-management subsystem is a central point that handles all of thesystem's memory, so it is naturally subject to scalability problems assystems grow larger. Two sessions during the memory-management track ofthe 2018 Linux Storage, Filesystem, and Memory-Management Summit looked atspecific contention points: the zone locks and the mmap_semsemaphore.

Security updates have been issued by CentOS (firefox, java-1.7.0-openjdk, java-1.8.0-openjdk, librelp, patch, and python-paramiko), Debian (kernel and quassel), Gentoo (chromium, hesiod, and python), openSUSE (corosync, dovecot22, libraw, patch, and squid), Oracle (java-1.7.0-openjdk), Red Hat (go-toolset-7 and go-toolset-7-golang, java-1.7.0-openjdk, and rh-php70-php), and SUSE (corosync and patch).

The LWN.net Weekly Edition for May 3, 2018 is available.

At the 2018 Linux Storage, Filesystem, and Memory-Management Summit, MimiZohar gave a presentation in thefilesystem track on the Linux integrity subsystem. There is a lotof talk that the integrity subsystem (usually referred to as "IMA", whichis the integritymeasurement architecture, though there is more to the subsystem) iscomplex and not documented well, she said. So she wanted to give an overview of the subsystem and then todiscuss some filesystem-related concerns.

Greg Kroah-Hartman has released a full set of stable kernels: 4.16.7, 4.14.39, 4.9.98, 4.4.131, and 3.18.108. All of them contain important fixesand users should update.

Containers are, of course, all the rage these days; in fact, during his2018 Legal andLicensing Workshop (LLW) talk, Dirk Hohndelsaid with a grin that he hears "containers may take off". But, whilecontainers are easy to set up and use, license compliance for containers is "incrediblyhard". He has been spending "way too much time" thinking about containercompliance recently and, beyond the standard "let's go shopping" solutionto hard problems, has come up with some ideas.Hohndel is a longtime member of the FOSS community who is now the chiefopen source officer at VMware—a company that ships some container images.

Google has announcedthe open-sourcing of gVisor, a sandboxed container runtime."gVisor is more lightweight than a VM while maintaining a similarlevel of isolation. The core of gVisor is a kernel that runs as a normal,unprivileged process that supports most Linux system calls. This kernel iswritten in Go, which was chosen for its memory- and type-safety. Just likewithin a VM, an application running in a gVisor sandbox gets its own kerneland set of virtualized devices, distinct from the host and othersandboxes."

Security updates have been issued by Debian (kernel), Fedora (haproxy), openSUSE (flac, GraphicsMagick, and quassel), Oracle (kernel), Red Hat (python-paramiko and redhat-virtualization-host), and SUSE (corosync).

There is a new initiative in the Fedora community based on what used to becalled "Fedora Atomic Workstation". From thiswhitepaper [PDF]: "The descriptive name for this product is ​image-mode container-based Fedora Workstation based on rpm-ostree, which isclear but terrible for branding. Therefore, we call it Team Silverblue.The long-term goal for this effort is to transform Fedora Workstation intoan image-based system where applications are separate from the OS andupdates are atomic."

Version 8.1 of the GCC compiler suite is out."Are you tired of your existing compilers?Want fresh new language features and better optimizations?Make your day with the new GCC 8.1!" See this page for a completelist of changes in this release.

Martin Pitt describes hisexperience running a fully free-software Android phone."I previously used Opera as a web browser, because it is relativelylightweight (important on my previous phone) and the really good builtin adblocker. But these days Firefox is really fast and good enough, so Ireplaced it with Fennec, which is more or less Firefox with some non-freebits removed. After installing uBlock Origin I’ve never lookedback."

Christoph Lameter works in a different computing environment than most ofus; he supports high-volume trading applications that need every bit ofperformance that the fastest hardware can give them. Even then, it seemsthat isn't fast enough. In a memory-management-track session at the 2018Linux Storage, Filesystem, and Memory-Management Summit, Lameter describedsome of the problems he has encountered and approaches he is considering toaddress them.

Allocating chunks of memory that are both large and physically contiguoushas long been a difficult thing to do in the kernel. But there are times wherethere is no alternative. Two sessions in the memory-management track ofthe 2018 Linux Storage, Filesystem, and Memory-Management Summit exploredways of making those allocations more reliable. It turns out that some usecases have a rather larger value of "large" than others.

Memory control groups allow the system administrator to impose memory-uselimits on the members of control groups. In many ways, these limits behavelike the overall limit on available memory, but there are also somedifferences. The behavior of the memory controller also changed with theadvent of the version-2 control-group API, creating problems for at leastone significant user. Three sessions held in the memory-management track ofthe Linux Storage, Filesystem, and Memory-Management Summit explored someof these problems.

The recent fsync() woesexperienced by PostgreSQL led to a session on the firstday (April 23) of the 2018 Linux Storage, Filesystem, and Memory-Management Summit (LSFMM). Those problemsalso led to a second-day session with PostgreSQL developer Andres Freund who gave anoverview of how PostgreSQL does I/O and where that ran aground on someassumptions that had been made. The session led to a fair amount ofdiscussion with the filesystem-track developers; real solutions seem to bein the offing.

One of the core jobs of the memory-management subsystem is to make memoryavailable to other parts of the kernel when the need arises. Thememory-management track of the 2018 Linux Storage, Filesystem, andMemory-Management Summit hosted a pair of sessions on new or improvedallocation functions for the kernel covering the slab allocators andprotectable memory.

Security updates have been issued by Fedora (cups-filters, ghostscript, glusterfs, PackageKit, qpdf, and xen), Mageia (anki, libofx, ming, sox, webkit2, and xdg-user-dirs), Oracle (corosync, java-1.7.0-openjdk, and pcs), Red Hat (java-1.7.0-openjdk), Scientific Linux (corosync, firefox, gcc, glibc, golang, java-1.7.0-openjdk, java-1.8.0-openjdk, kernel, krb5, librelp, libvncserver, libvorbis, ntp, openssh, openssl, PackageKit, patch, pcs, policycoreutils, qemu-kvm, and xdg-user-dirs), Slackware (libwmf and mozilla), and Ubuntu (apache2, ghostscript, mysql-5.7, wavpack, and webkit2gtk).

The Fedora 28 release has been announced."The headline feature for Fedora 28 Server is the inclusion of thenew Modular repository. This lets you select between different versions ofsoftware like NodeJS or Django, so you can chose the stack you need foryour software." Some users will also appreciate that proprietaryblobs (such as the NVIDIA drivers) are now easier to obtain and install.

Matthew "Willy" Wilcox has been doing a fair amount of work in thememory-management area recently. He showed up at the 2018 Linux Storage,Filesystem, and Memory-Management Summit with a list of discussion topicsrelated to that work; it was enough to fill a plenary session with somespillover into the memory-management track the next day. Some of histopics were fairly straightforward; others look to be somewhat moreinvolved.

The kernel's memory-management subsystem has to manage a great deal ofconcurrency; that leads to an ongoing series of locking challenges thatsometimes seem intractable. Two recurring locking issues — the LRU locksand the mmap_sem lock — were the topic of sessions held during thememory-management track of the 2018 Linux Storage, Filesystem, andMemory-Management Summit. In both cases, it quickly became clear that,while some interesting ideas are being pursued, easysolutions are not on offer.

When kernel code needs to work directly with user-space pages, it oftencalls get_user_pages()(or one of several variants) to fault those pages into RAM and pin themthere. This function is not entirely easy to use, though, and recentchanges have made it harder to use safely. Jan Kara and Dan Williams led aplenary session at the 2018 Linux Storage, Filesystem, andMemory-Management Summit to discuss potential solutions, but it is notentirely clear that any were found.

Stable kernels 4.16.6, 4.14.38, 4.9.97, 4.4.130, and 3.18.107 have been released. They all containimportant fixes throughout the tree and users should upgrade.

Security updates have been issued by Arch Linux (drupal), Debian (chromium-browser, gunicorn, libvorbis, openjdk-8, roundcube, sdl-image1.2, slurm-llnl, and tor), Fedora (boost, cups-filters, ghostscript, gsoap, memcached, mod_http2, and qpdf), openSUSE (Chromium and mysql-community-server), and Red Hat (glusterfs, OpenShift Container Platform 3.1, OpenShift Container Platform 3.2, OpenShift Container Platform 3.3, OpenShift Container Platform 3.4, OpenShift Container Platform 3.5, OpenShift Container Platform 3.6, OpenShift Container Platform 3.7, OpenShift Container Platform 3.8, OpenShift Container Platform 3.9, and openvswitch).

The 4.17-rc3 kernel prepatch is out."And by now, I think we've fixed all the nastiest fall-out from themerge window. In particular, the PTI large-page fallout that hit somepeople with particular configurations should all be good."

The memory-management subsystem is maintained by a small but dedicatedgroup of developers. How healthy is that development community? MichalHocko raised that question during the memory-management track at the 2018Linux Storage, Filesystem, and Memory-Management Summit. Hocko is worried,but it appears that his concerns are not universally felt.

The non-uniform memory architecture (NUMA) was designed around the ideathat there are two types of memory on complex systems: local (faster) andremote (slower). During the memory-management track of the 2018 LinuxStorage, Filesystem, and Memory-Management Summit, Anshuman Khandualasserted that the situation has since become rather more complicated.Perhaps, he said, the time has come to rethink how we view NUMA systems.