LWN.net

The userfaultfd() system callallows user space to intervene in the handling of page faults. As AndreaArcangeli and Mike Rapaport described in a 2017 Linux Storage, Filesystem,and Memory-Management Summit session dedicated to the subject,userfaultfd() was originally created to help with the livemigration of virtual machines between physical hosts. It allows pages tobe copied to the new host on demand, after the machine itself has beenmoved, leading to faster, more predictable migrations. Work onuserfaultfd() is not finished, though; there are a number of otherfeatures that developers would like to add.

A processor's translation lookaside buffer (TLB) caches the mappings fromvirtual to physical addresses. Looking up virtual addresses is expensive,so good performance often depends on making the best use of the TLB. Inthe memory-management track of the 2017 Linux Storage, Filesystem, andMemory-Management Summit, Mike Kravetz described a SPARC processor featurethat can improve TLB performance and explored ways in which that featurecould be supported.

Version1.6 of the Kubernetes orchestration system is available. "Inthis release the community’s focus is on scale and automation, to help youdeploy multiple workloads to multiple users on a cluster. We are announcingthat 5,000 node clusters are supported. We moved dynamic storageprovisioning to stable. Role-based access control (RBAC), kubefed, kubeadm,and several scheduling features are moving to beta. We have also addedintelligent defaults throughout to enable greater automation out of thebox."

Google has announcedthe launch of opensource.google.com. "Today, we’re launching opensource.google.com, a new website for Google Open Source that ties together all of our initiatives with information on how we use, release, and support open source.This new site showcases the breadth and depth of our love for open source. It will contain the expected things: our programs, organizations we support, and a comprehensive list of open source projects we've released. But it also contains something unexpected: a look under the hood at how we "do" open source."

When the transparent huge page feature was added to the kernel, it onlysupported anonymous (non-file-backed) memory. In 2016, support for huge pages in the page cache wasadded, but only the tmpfs filesystem was supported. There is interest inexpanding support to other filesystems, since, for some workloads, theperformance improvement can be significant. Kirill Shutemov led the onlysession that combined just the filesystem and memory-management tracks atthe 2017 Linux Storage, Filesystem, and Memory-Management Summit in adiscussion of adding huge-page support to the ext4 filesystem.

Security updates have been issued by Debian (eject, gst-plugins-bad1.0, gst-plugins-base1.0, gst-plugins-good1.0, gst-plugins-ugly1.0, gstreamer1.0, php5, and tiff), Fedora (kernel), Gentoo (curl, deluge, libtasn1, and xen-tools), Mageia (mbedtls, putty, and roundcubemail), openSUSE (dbus-1, gegl, mxml, open-vm-tools, partclone, qbittorrent, tcpreplay, and xtrabackup), and Ubuntu (eject, gst-plugins-base0.10, gst-plugins-base1.0, and gst-plugins-good0.10, gst-plugins-good1.0).

DAX is the mechanism that enables direct access to files stored inpersistent memory arrays without the need to copy the data through the pagecache. At the 2017 Linux Storage, Filesystem, and Memory-ManagementSummit, Ross Zwisler led a plenary session on the future of DAX. Development inthis area offers a number of interesting trade-offs between data safety andenabling the highest performance.

DragonFly BSD 4.8 has been released. "DragonFlyversion 4.8 brings EFI boot support in the installer, further speedimprovements in the kernel, a new NVMe driver, a new eMMC driver, and Intelvideo driver updates." DragonFly is an independent BSD variant,perhaps best known for the HAMMER filesystem.

The Free Software Foundation has announcedthe winners of the 2016 Free Software Awards. The Award for Projectsof Social Benefit went to SecureDropand the Award for the Advancement of Free Software went to Alexandre Oliva. "SecureDrop is an anonymous whistleblowing platform used by major news organizations and maintained by Freedom of the Press Foundation. Originally written by the late Aaron Swartz with assistance from Kevin Poulsen and James Dolan, the free software platform was designed to facilitate private and anonymous conversations and secure document transfer between journalists and sensitive sources."

Stable kernels 4.10.6, 4.9.18, and 4.4.57 have been released. All of themcontain important fixes and users should upgrade.

Security updates have been issued by Debian (apt-cacher, jbig2dec, libplist, python3.2, tnef, and xrdp), Fedora (firefox, mbedtls, and sane-backends), Mageia (flash-player-plugin, freetype2, glibc, kernel, kernel-linus, kernel-tmb, libquicktime, libwmf, and tnef), and Ubuntu (thunderbird).

The 4.11-rc4 kernel prepatch is out fortesting. "So on the whole things look fine. There's changes allover, and in mostly the usual proportions. Some core kernel code shows upin the diffstat slightly more than it usually does - we had an audit fixand a bpf hashmap fix, but on the whole it all looks very regular."

In the memory-management subsystem, the term "mapping" refers to theconnection between pages in memory and their backing store — the file thatrepresents them on disk. One of the fundamental assumptions in thekernel is that a given page in the page cache belongs to exactly one mapping.But, as Miklos Szeredi explained in a plenary session at the 2017 LinuxStorage, Filesystem, and Memory-Management Summit, there are situationswhere it would be desirable to associate the same page with multiplemappings. Achieving this goal may not be easy, though.<p>Click below (subscribers only) for continuing coverage from LSFMM 2017

The Eudyptula Challenge is aseries of programming exercises for the Linux kernel. It starts from avery basic "Hello world" kernel module, moves up in complexity to gettingpatches accepted into the main kernel. The challengewill be closed to new participants in a few months, when 20,000 people havesigned up. LWN covered the Eudyptula Challenge in May 2014,when it was fairly new. At this time over 19,000 people have signed up andonly 149 have finished.

Security updates have been issued by Arch Linux (libpurple), Debian (audiofile, cgiemail, and imagemagick), Fedora (cloud-init, empathy, and mupdf), Mageia (firefox and thunderbird), Scientific Linux (icoutils and openjpeg), Slackware (mcabber and samba), and Ubuntu (eglibc).

Back in 2015, the OpenSSL project announced itsintent to move away from its rather quirky license. Now it has announcedthat thechange is moving forward. "After careful review, consultationwith other projects, and input from the Core Infrastructure Initiative andlegal counsel from the SFLC, the OpenSSL team decided to relicense the codeunder the widely-used ASLv2." It is worth noting that this changeand the way it is being pursued are not universally popular, in the OpenBSD camp, at least.

Laszlo Agocs takesa look at improvements to the basic OpenGL enablers that form thefoundation of Qt Quick and the optional OpenGL-based rendering path ofQPainter in Qt 5.9. "Asexplained here, such shader programs will attempt to cache the programbinaries on disk using GL_ARB_get_program_binaryor the standard equivalents in OpenGL ES 3.0. When no support is providedby the driver, the behavior is equivalent to the non-cached case. The filesare stored in the global or per-process cache location, whichever is writable. The result is a nice boost in performance when a program is created with the same shader sources next time."

Security updates have been issued by Debian (audiofile, jhead, libxslt, samba, suricata, and wordpress), Fedora (openslp), Mageia (icoutils, kdelibs4, and virtualbox), Oracle (icoutils and openjpeg), Red Hat (icoutils and openjpeg), and Ubuntu (audiofile, git, and samba).

The LWN.net Weekly Edition for March 23, 2017 is available.

GitLab 9.0 has been releasedwith many new features and improvements. "In the last several releases, GitLab has transformed how development teams get from idea to production. In just a few minutes, you can deploy GitLab to a container scheduler, add CI/CD with auto deployed review apps, utilize ChatOps, and analyze your cycle time. With 9.0 you can now watch your deploys with deploy boards and monitor application performance with Prometheus."

The NTPsec Project has announced the 0.9.7 release of NTPsec, withassistance from the Mozilla Foundation's "Secure Open Source" initiative.NTPsec is an implementation of the Network Time Protocol (NTP)."NTPsec 0.9.7 incorporates significant improvements in security, accuracy,precision, visualization, and usability, with assistance, contributions,and audits provided by infosec researchers and other technical contributors.For this release, the NTPsec Project worked particularly closely with theMozilla Foundation's "Secure Open Source" initiative, who funded an infosecaudit, and with Cure53.de, who provided the audit."

The GNOME Project has announced the release of GNOME 3.24, "Portland"."This release is the result of 6 months’ hard work by the GNOME community.It contains major new features such as night light, as well as many smallerimprovements and bug fixes. GNOME's existing applications have beenimproved and there is also a new Recipes app. Improvements to our platforminclude refined notifications and several revamped settings panels."

Greg Kroah-Hartman has released stable kernels 4.10.5, 4.9.17, and 4.4.56. All of them contain important fixesand users should upgrade.

Security updates have been issued by Arch Linux (irssi), Fedora (qemu), openSUSE (mbedtls), and Ubuntu (eglibc, glibc).

In a morning plenary session on the first day of the 2017 Linux Storage,Filesystem, and Memory-Management Summit, Jérôme Glisse led a discussion onmemory that cannot be addressed by the CPU because it lives in devices likeGPUs or FPGAs. There is often a substantial pile of memory on thesedevices and it can be accessed much more quickly by the devices than thesystem RAM can be. Making it easier for user-space programmers to use thatmemory transparently is the goal of the heterogeneous memory management (HMM) patchesthat Glisse has been working on.

Matthew Garrett announces a new,hopefully more efficient process for reviewing bootloaders to be used withShim in UEFI secure bootsystems. "To that end, we're adopting a new model. A mailing listhas been created at shim-review@lists.freedesktop.org, and members of thislist will review submissions and provide a recommendation to Microsoft onwhether these should be signed or not."

The Android Developers Blog introducesthe first developer preview of Android O. This version includesbackground limits, notification channels, autofill APIs, PIP for handsets,font resources in XML, adaptive icons, and much more. "Building on the work we began in Nougat, Android O puts a big priority on improving a user's battery life and the device's interactive performance. To make this possible, we've put additional automatic limits on what apps can do in the background, in three main areas: implicit broadcasts, background services, and location updates. These changes will make it easier to create apps that have minimal impact on a user's device and battery. Background limits represent a significant change in Android, so we want every developer to get familiar with them."

KDevelop is KDE's Integrated Development Environment (IDE). Version 5.1has been releasedwith LLDB support, Analyzer run mode, initial OpenCL language support,improved Python language support, and more.

Red Hat has announcedthe release of Red Hat Enterprise Linux 6.9. "Red Hat Enterprise Linux 6.9 delivers new hardware support developed in collaboration with Red Hat partners which helps to provide a smooth transition of Red Hat Enterprise Linux 6 production deployments to Red Hat Enterprise Linux 7 environments. Additionally, Red Hat Enterprise Linux 6.9 adds updates to TLS 1.2 to further enhance secure communications and provide broader support for the latest PCI-DSS standards, better equipping enterprises to offer more secure online transactions."

Security updates have been issued by Debian (sitesummary), Fedora (jasper, knot-resolver, R, rkward, rpm-ostree, rpy, w3m, and xen), openSUSE (firefox), Red Hat (bash, coreutils, glibc, gnutls, kernel, libguestfs, ocaml, openssh, qemu-kvm, quagga, samba, samba4, subscription-manager, tigervnc, and wireshark), and Ubuntu (eglibc, glibc, firefox, freetype, gnutls26, NVIDIA graphics, and nvidia-graphics-drivers-375).

The opening session of the 2017 Linux Storage, Filesystem, andMemory-Management Summit covered a familiartopic: how to represent (possibly massive) persistent-memory arraysto various subsystems in the kernel. This session, led by Dan Williams,focused in particular on the ZONE_DEVICE abstraction and whetherthe kernel should use page structures to represent persistent memory ornot.

Linux.com takesa look at the Intel Edison."The Intel Edison is a physically tiny computer that draws a small amount of power and breaks out plenty of connections to allow it to interact with other electronics. It begs to be the brain of your next electronics tinkering project, with all the basics in a tiny package and an easy way to connect other things you might need."

Security updates have been issued by Arch Linux (firefox, mbedtls, and wordpress), CentOS (firefox, openjpeg, and tomcat6), Debian (deluge, ioquake3, r-base, and wireshark), Fedora (qemu, rabbitmq-server, and sscg), Gentoo (adobe-flash, openoffice-bin, and putty), openSUSE (Chromium, irssi, putty, and roundcubemail), Oracle (firefox and openjpeg), Red Hat (firefox and openjpeg), Scientific Linux (firefox and openjpeg), and SUSE (firefox).

OpenSSH 7.5 is out. This is primarily a bug-fix release, but it alsomakes the use of privilege separation mandatory and removes support forbuilding against old, unsupported OpenSSL releases.

The 4.11-rc3 kernel prepatch is out."As is our usual pattern after the merge window, rc3 is larger thanrc2, but this is hopefully the point where things start to shrink andcalm down."

The4.10.4,4.9.16, and4.4.55 stable kernels are out with anotherset of important fixes.

Ubuntu has discontinued support for the 32-bit powerpc architecture inZesty Zappus (17.04)."We are well into Feature Freeze at this point, so an update is overdue. Asof Feature Freeze in February, the status is that powerpc packages are nolonger considered for proposed-migration, and we have discontinued all CDimage builds for powerpc in zesty.For the moment, uploads continue to be built for powerpc in Launchpad, andpackages are still published in the archive. You should expect both to bediscontinued before the 17.04 release."

Brendan Gregg showshow to do scheduler profiling with the perf sched command."perf sched timehist was added in Linux 4.10, and shows the schedulerlatency by event, including the time the task was waiting to be woken up(wait time) and the scheduler latency after wakeup to running (schdelay). It's the scheduler latency that we're more interested intuning."

Security updates have been issued by Arch Linux (linux-zen), Debian (calibre, libdatetime-timezone-perl, tzdata, wireshark, and wordpress), Fedora (icoutils and tcpreplay), Mageia (wavpack), openSUSE (dracut and qemu), and SUSE (firefox and xen).

The remaining users of RHEL 5 (and derivatives) will want to know that maintenanceof the EPEL-5 repository is coming to an end. "In the end,EPEL-5 went live sometime in April of 2007 and over the next 10 years grewto a repository of over 5000 source packages and 200,000 unique ipaddresses checking in per day at its peak of 240,000 in early 2013. Whileevery package built for EPEL is done with the RHEL packages, all of thesepackages have been useful for the various community rebuilds (CentOS,Scientific Linux, Amazon Linux) of RHEL. This meant that growth in thoseeco-systems brought more users into using EPEL and helping on packaging aslater RHEL releases came out. However as these newer releases and rebuildsgrew in usage, the number of EPEL-5 users has gradually fallen to around160,000 unique ip addresses per day. Also over that time, the number ofpackages supported by developers has fallen and the repository has shrunkin size to 2000 source packages."

The 2017 Linux Plumbers Conference (LPC) has announced its call for refereed presentations. "Refereed Presentations are 45 minutes in length and should focus on aspecific aspect of the 'plumbing' in the Linux system. Examples ofLinux plumbing include core kernel subsystems, core libraries,windowing systems, management tools, device support, mediacreation/playback, and so on. The best presentations are not aboutfinished work, but rather problems, proposals, or proof-of-conceptsolutions that require face-to-face discussions and debate." Proposals are due by May 6 and LPC will be held in Los Angeles, CA, US on September 13-15 in conjunctionwith The Linux Foundation Open Source Summit North America.

The GNU Guile project hasannounced the release of Guile 2.2.0, which is an implementation of the Scheme Lisp dialect. "More than 6 years in the making, Guile 2.2 includes a new optimizingcompiler and high-performance register virtual machine. Compared tothe old 2.0 series, real-world programs often show a speedup of 30% ormore with Guile 2.2.Besides the compiler upgrade, Guile 2.2 removes limitations on userprograms by lowering memory usage, speeding up the "eval" interpreter,providing better support for multi-core programming, and last but notleast, removing any fixed limit on recursive function calls.Not only does Guile 2.2 run fast, it also supports the creation ofuser-space concurrency facilities that multiplex millions ofconcurrent lightweight "fibers". Seehttps://www.gnu.org/software/guile/news/gnu-guile-220-released.htmlfor pointers to promising experiments."

Kent Overstreet has announced a new majorrelease of his bcachefs filesystem.Changes in this release includewhole-filesystem encryption, backup superblocks, better multiple-devicesupport, a user-space filesystem checker, and more. "We can also nowmigrate filesystems to bcachefs in place! The bcache migrate command takesan existing filesystem, fallocates a big file in it, creates a newfilesystem (in userspace) on the block device but using only the spacereserved by that file it fallocated - and then walks the contents of theoriginal filesystem creating pointers to all your existing data."There is an on-disk format change, but there's a chance it's the last one.

Security updates have been issued by CentOS (thunderbird), Fedora (ettercap, jasper, qbittorrent, and tcpreplay), Oracle (tomcat6), Red Hat (rabbitmq-server), Slackware (pidgin), SUSE (flash-player), and Ubuntu (libxml2, linux, linux-aws, linux-gke, linux-raspi2, linux-snapdragon, and linux-lts-xenial).

The LWN.net Weekly Edition for March 16, 2017 is available.

Sometimes it seems that things have gone relatively quiet on the year-2038front. But time keeps moving forward, and the point in early 2038 when32-bit time_t values can no longer represent times correctly isnow less than 21 years away. That may seem like a long time, but therelatively long life cycle of many embedded systems means that some systems deployed today will still be inservice when that deadline hits. One of the developers leading the effortto address this problem is Arnd Bergmann; at Linaro Connect 2017 he gave anupdate on where that work stands.

Stable kernels 4.10.3, 4.9.15, and 4.4.54 have been released. All of themcontain the usual set of important fixes.

Security updates have been issued by Arch Linux (flashplugin, jasper, kernel, lib32-flashplugin, and roundcubemail), Debian (chromium-browser and mariadb-10.0), Fedora (ettercap), openSUSE (firefox, mozilla-nss and thunderbird), Oracle (thunderbird), Red Hat (flash-plugin, kernel, policycoreutils, rabbitmq-server, and tomcat6), Scientific Linux (tomcat6), and Ubuntu (imagemagick).

Version1.18 of the MATE desktop has been released. "The release isfocused on completing the migration to GTK3+ and adopting new technologiesto replace some of deprecated components MATE Desktop 1.16 still reliedon."

Robert Haas describesthe many parallelism enhancements in the upcoming PostgreSQL 10release. "The Gather node introduced in PostgreSQL 9.6 gathersresults from all workers in an arbitrary order. That's fine if the datathat the workers were producing had no particular ordering anyway, but ifeach worker is producing sorted output, then it would be nice to gatherthose results in a way that preserves the sort order. This is what GatherMerge does. It can speed up queries where it's useful for the results ofthe parallel portion of the plan to have a particular sort order, and wherethe parallel portion of the plan produces enough rows that performing anordinary Gather followed by a Sort would be expensive."