LWN.net

<p>I recently took a deep dive into web site archival for friends whowere worried about losing control over the hosting of their workonline in the face of poor system administration or hostileremoval.This makes web site archival an essential instrument in thetoolbox of any system administrator.As it turns out, some sites are much harder to archive thanothers. This article goes through the process of archiving traditionalweb sites and shows how it falls short when confronted with the latestfashions in the single-page applications that are bloating the modern web.<p>Subscribers can read on for a look at web archiving by guest author Antoine Beaupré.

Security updates have been issued by Arch Linux (bitcoin-daemon and bitcoin-qt), Debian (firefox-esr, hylafax, libarchive-zip-perl, mediawiki, okular, openafs, strongswan, and texlive-bin), Fedora (gitolite3, kernel-headers, and lcms2), Mageia (dropbear, kernel, lcms2, libcgroup, libextratcor, mailman, mpg123, okular, php, soundtouch, unixODBC, webkit2, and xml-security-c), openSUSE (aubio, bouncycastle, chromium, ffmpeg-4, firefox, gdm, GraphicsMagick, hylafax+, ImageMagick, jhead, liblouis, nemo-extensions, nextcloud, nodejs6, obs-service-refresh_patches, okular, openslp, pango, phpMyAdmin, python-Django, python-Django1, and seamonkey), Oracle (spice and spice-gtk), Slackware (firefox and kernel), and SUSE (ant, apache2, gnutls, libzypp, zypper, nodejs6, nodejs8, and xorg-x11-libs).

The 4.19-rc5 kernel prepatch has beenreleased by Greg Kroah-Hartman. "As almost everyone knows, it's beenan 'interesting' week from a social point-of-view. But from the technicalside, -rc5 looks totally normal."

The kernel's namespace abstraction allowsdifferent groups of processes to have different views of the system. Thisfeature is most often used with containers; it allows each container tohave its own view of the set of running processes, the network environment,the filesystem hierarchy, and more. One aspect of the system that remainsuniversal, though, is the concept of the system time. The recently postedtimenamespace patch set (from Dmitry Safonov with a lot of work by AndreiVagin) seeks to change that.

The Ubuntu blog has announced the release of version 1.0.0 of the Mir display server. "Whether for building a device or for writing a shell for the desktop, Mir can give you a graphics stack that is fast, light, and secure. The Mir graphical stack works across different graphics platforms and driver models and is easy to integrate into your kiosk, digital signage, or purpose built graphical solution. It was first conceived over 6 years ago as part of an initiative by Canonical to unify the graphical environment across all devices, including desktop, TV, and mobile devices and continues to be developed with new features and modern standards."

Security updates have been issued by Debian (hylafax, sympa, and texlive-bin), Fedora (curl and gitolite3), Mageia (bouncycastle, ghostscript, and libx11), openSUSE (webkit2gtk3), Oracle (spice and spice-gtk and spice-gtk and spice-server), Red Hat (rubygem-smart_proxy_dynflow, spice and spice-gtk, and spice-gtk and spice-server), Scientific Linux (spice and spice-gtk and spice-gtk and spice-server), and SUSE (ImageMagick, kernel, liblouis, openslp, and python-paramiko).

Security updates have been issued by Debian (glusterfs, php5, reportbug, and suricata), openSUSE (chromium and exempi), Red Hat (openstack-rabbitmq-container), SUSE (couchdb, crowbar, crowbar-core, crowbar-ha, crowbar-init, crowbar-openstack, crowbar-ui, gdm, OpenStack, pango, and webkit2gtk3), and Ubuntu (bind9, lcms, lcms2, and lcms2).

A story in The New Yorker magazine may help explain some of the timing of the recent upheavals in kernel-land. Longtime followers of kernel development will find the article to be a mixed bag—over the top in spots, fairly accurate elsewhere. "Torvalds’s decision to step aside came after The New Yorker asked him a series of questions about his conduct for a story on complaints about his abusive behavior discouraging women from working as Linux-kernel programmers. In a response to The New Yorker, Torvalds said, 'I am very proud of the Linux code that I invented and the impact it has had on the world. I am not, however, always proud of my inability to communicate well with others—this is a lifelong struggle for me. To anyone whose feelings I have hurt, I am deeply sorry.'"

The LWN.net Weekly Edition for September 20, 2018 is available.

Stable kernels 4.18.9, 4.14.71, 4.9.128, and 4.4.157 have been released. They all containthe usual set of important fixes and users should upgrade.

Android's ProjectTreble is meant as a way to reduce the fragmentation in the Androidecosystem. It also makes porting Android 8 ("Oreo"—the first versionto mandate Treble) more difficult, according to Fedor Tcymbal. Hedescribed the project and what it means for silicon and device vendors in atalk atOpenSource Summit North America 2018 in Vancouver, Canada.

Facebook runs a lot of programs and it tries to pack as many as it can ontoeach machine. That means running close to—and sometimes beyond—theresource limits on any given machine. How the system reacts when, for example,memory is exhausted, makes a big difference in Facebook getting its workdone. Tejun Heo came to 2018 Open Source Summit North America to describe the resource controlwork that has been done by the team he works on at Facebook.

Security updates have been issued by Debian (chromium-browser and libapache2-mod-perl2), Oracle (kernel), and Ubuntu (ghostscript, glib2.0, and php5).

Version 7.0.0 of the LLVM compiler suite is out."It is the result of the community's workover the past six months, including: function multiversioning in Clangwith the 'target' attribute for ELF-based x86/x86_64 targets, improvedPCH support in clang-cl, preliminary DWARF v5 support, basic supportfor OpenMP 4.5 offloading to NVPTX, OpenCL C++ support, MSan, X-Rayand libFuzzer support for FreeBSD, early UBSan, X-Ray and libFuzzersupport for OpenBSD, UBSan checks for implicit conversions, manylong-tail compatibility issues fixed in lld which is now productionready for ELF, COFF and MinGW, new tools llvm-exegesis, llvm-mca anddiagtool".The list of new featuresis long; see theoverall release notes,theClang release notes,theClang tools release notes, and theLLD linker release notes for more information.

A couple of surprising things happened in the kernel community onSeptember 16: Linus Torvalds announcedthat he was taking a break from kernel development to focus on improvinghis own behavior, and the longstanding "code of conflict" was replacedwith a code of conduct based on the ContributorCovenant. Those two things did not quite come packaged as a set, butthey are clearly not unrelated. It is atime of change for the kernel project; there will be challenges to overcomebut, in the end, less may change than many expect or fear.

Security updates have been issued by Fedora (ghostscript, icu, nspr, nss, nss-softokn, nss-util, and okular), Red Hat (java-1.7.1-ibm, java-1.8.0-ibm, OpenStack Platform, openstack-neutron, and openstack-nova), and Ubuntu (clamav and php5, php7.0, php7.2).

The PostgreSQL community has, after an extended discussion, announced theadoption of a codeof conduct "which is intended toensure that PostgreSQL remains an open and enjoyable project for anyone tojoin and participate in".

Versity Software has announced that it has released ScoutFS under GPLv2. "ScoutFS is the first GPL archiving file system ever released, creating aninherently safer and more user friendly option for storing archival datawhere accessibility over very large time scales, and the removal of vendorspecific risk is a key consideration."

Security updates have been issued by Debian (discount, ghostscript, intel-microcode, mbedtls, thunderbird, and zutils), Fedora (ghostscript, java-1.8.0-openjdk-aarch32, kernel-headers, kernel-tools, libzypp, matrix-synapse, nspr, nss, nss-softokn, nss-util, zsh, and zypper), Mageia (kernel, kernel-linus, and kernel-tmb), openSUSE (chromium, curl, ffmpeg-4, GraphicsMagick, kernel, libzypp, zypper, okular, python3, spice-gtk, tomcat, and zsh), Oracle (kernel), Slackware (php), SUSE (curl, libzypp, zypper, and openssh-openssl1), and Ubuntu (curl and firefox).

SpamAssassin 3.4.2 is out, the first release from this spam-filteringproject since 3.4.1 came out in April 2015. It fixes some remotelyexploitable security issues, so SpamAssassin users probably want toupdate in the near future. "The exploit has been seen in the wild but not believe to have beenpurposefully part of a Denial of Service attempt. We are concerned thatthere may be attempts to abuse the vulnerability in the future. Therefore, we strongly recommend all users of these versions upgrade toApache SpamAssassin 3.4.2 as soon as possible."

Behavioral changes can make desktop users grumpy; that is doubly true forchanges that arrive without notice and possibly risk data loss. Such asituation recently arose in the Fedora 29 development branch in theform of a new "suspend-then-hibernate" feature. This feature will almostcertainly be turned off before Fedora 29 reaches an official release,but the discussion and finger-pointing it inspired reveal somesignificant differences of opinion about how this kind of change should bemanaged.

Linus has released 4.19-rc4 and made a setof announcements that should really be read in their entirety."I actually think that 4.19 is looking fairly good,things have gotten to the 'calm' period of the release cycle, and I'vetalked to Greg to ask him if he'd mind finishing up 4.19 for me, sothat I can take a break, and try to at least fix my own behavior."

The4.18.8,4.14.70,4.9.127, and4.4.156 stable kernels have been released.Each contains a relatively large set of important fixes and updates.

Linux Journal covers the new Academy Software Foundation (ASWF), which is a project aimed at open-source collaboration in movie-making software that was started by theAcademy of Motion Picture Arts and Sciences (AMPAS) and the Linux Foundation. "Still at the early stages, the ASWF has yet to develop any of its own projects, but there is interest in having them host a number of very popular projects, such as Industrial Light & Magic’s OpenEXR HDR image file format, color management solution OpenColorIO, and OPenVDB, which is used for working with those hard-to-handle objects like clouds and fluids.Along with promoting cooperation on the development of a more robust set of tools for the industry, one of the goals of the organization moving forward is to put out a shared licensing template that they hope will help smooth the tensions over licensing. It follows that with the growth of projects, navigating the politics over usage rights is bound to be a tricky task."

Security updates have been issued by CentOS (firefox), Fedora (firefox, openssh, pango, and zziplib), Mageia (flash-player-plugin and ntp), Oracle (kernel), Red Hat (flash-plugin), Slackware (ghostscript), SUSE (podman and spice-gtk), and Ubuntu (firefox).

Over at Opensource.com, Red Hat's Michael Tiemann looksat open source fromthe perspective of the economic theories of Ronald Coase, who won the 1991Nobel Prize for Economics. Those theories help explain why companies likeRed Hat (and Cygnus Solutions, which Tiemann founded) have prospered evenin the face of economic arguments about why they shouldnot. "Successful open source software companies 'discover' marketswhere transaction costs far outweigh all other costs, outcompete theproprietary alternatives for all the good reasons that even the economicnay-sayers already concede (e.g., open source is simply a betterdevelopment model to create and maintain higher-quality, more rapidlyinnovative software than the finite limits of proprietary software), andthen—and this is the important bit—help clients achieve strategicobjectives using open source as a platform for their own innovation. Withopen source, better/faster/cheaper by itself is available for the low, lowprice of zero dollars.As an open source company, we don't cry about that. Instead, we look at how open source might create a new inflection point that fundamentally changes the economics of existing markets or how it might create entirely new and more valuable markets."

/e/ is Gaël Duval's project to build a privacy-oriented smartphonedistribution; the first beta isnow available with support for a number of devices. "At ourcurrent point of development, we have an '/e/' ROM in Beta stage: forkedfrom LineageOS 14.1, it can be installed on several devices (read the list). The number of supported devices will grow over time, depending onmore build servers and more contributors who can maintain or port tospecific devices (contributors welcome). The ROM includes microG configuredby default with Mozilla NLP so users can have geolocation functionalityeven when GPS signal is not available."

Linux kernel developers tend to take a dim view of the C++ language; it isseen, rightly or wrongly, as a sort of combination of the worst (from asystem-programming point of view) features of higher-level languages andthe worst aspects of C. So it takes a relatively brave person todare to discuss that language on the kernel mailing lists. David Howellsmust certainly be one of those; he not only brought up the subject, but isworking to make the kernel's user-space API (UAPI) header files compatiblewith C++.

Security updates have been issued by Debian (ghostscript and openssh), Oracle (firefox), Scientific Linux (firefox and OpenAFS), SUSE (tomcat), and Ubuntu (openjdk-lts).

The HHVM project has announcedthat the Hack language and PHP will truly be going separate ways. The HHVMv3.30 release, due by the end of the year, will be the last to support codewritten in PHP. "Ultimately, we recommend that projects eithermigrate entirely to the Hack language, or entirely to PHP7 and the PHPruntime." HHVM was first announcedin 2011 as a compiler for the PHP language.

The LWN.net Weekly Edition for September 13, 2018 is available.

There are ways to get fixes into the stable kernel trees, but theyrequire humans to identify which patches should go there. Sasha Levin andJulia Lawall have taken a different approach: use machine learning todistinguish patches that fix bugs from others. That way, all bug-fixpatches could potentially make their way into the stable kernels. Levinand Lawall gave a talk describing their work at the 2018Open Source Summit North America in Vancouver, Canada.

The STACKLEAK kernel security feature has been in the works for quite sometime now, but has not, as yet, made its way into the mainline. That is notfor lack of trying, as Alexander Popov has posted 15 separate versions ofthe patch set since May 2017. He described STACKLEAK and its tortuous pathtoward the mainline in a talk[YouTube video] at the 2018Linux Security Summit.

Security updates have been issued by Debian (kamailio, libextractor, and mgetty), Fedora (community-mysql, ghostscript, glusterfs, iniparser, okular, and zsh), openSUSE (compat-openssl098, php5, and qemu), Red Hat (firefox), SUSE (libzypp, zypper, python3, spark, and zsh), and Ubuntu (zsh).

From the kernel development community's point of view, hardwarevulnerabilities are not much different from the software variety: eitherway, there is a bug that must be fixed in software. But hardware vendorstend to take a different view of things. This divergence has beenreflected in the response to vulnerabilities like Meltdown and Spectrewhich was seen by many as being severely mismanaged. A recent discussionon the Kernel Summit discussion list has shed some more light on how thingswent wrong, and what the development community would like to see happenwhen the next hardware vulnerability comes around.

PostgreSQL 11 had its third beta releaseon August 9; a fourth beta (or possibly a release candidate) is scheduledfor mid-September. While the final release of the relationaldatabase-management system (currently slated for lateSeptember) will have something new for many users, its development cycle wasnotable for being a period when the community hit its stride in twostrategic areas: partitioning and parallelism.

Security updates have been issued by Debian (libextractor), Fedora (godot and iniparser), Oracle (kernel), Red Hat (chromium-browser and Fuse 7.1), SUSE (compat-openssl098, openssh, php5, php53, qemu, and tiff), and Ubuntu (kernel, linux, linux-aws, linux-azure, linux-gcp, linux-kvm, linux-raspi2, and linux-hwe, linux-azure, linux-gcp).

The Git 2.19.0 release is out. Significant changes include a new "rangediff" capability, directory rename detection, and more; see thisGitHub blog entry for more information. "We can use git diff to show the difference between the two end states, but that doesn’t provide information about the individual commits. And if the base on which the commits were built has changed, the resulting state might be quite different, even if the changes in the commits are largely the same.Git 2.19 introduces git range-diff, a tool for comparing two sequences of commits, including changes to their order, commit messages, and the actual content changes they introduce."

The kernel's CPU scheduler must, as its primary task, determine whichprocess should be executing in each of a system's processors at any giventime. Making an optimal decision involves juggling a number offactors, including the priority (and scheduling classes) of the runnableprocesses, NUMA locality, cache locality, latency minimization,control-group policies, power management, overall fairness, and more. Onemight think that throwing another variable into the mix — and a complex oneat that — would not be something anybody would want to attempt. The recentcoschedulingpatch set from Jan Schönherr does exactly that, though, by introducingthe concept of processes that should be run simultaneously.

Stable kernels 4.14.69, 4.9.126, 4.4.155, and 3.18.122 have been released. They all containthe usual set of important fixes and users should upgrade.

Security updates have been issued by Debian (chromium-browser, curl, discount, firefox-esr, ghostscript, and openssh), Fedora (curl, firefox, ghostscript, glibc, mod_perl, thunderbird, and unixODBC), openSUSE (chromium, firefox, GraphicsMagick, nodejs4, and thunderbird), Oracle (kernel), and SUSE (java-1_7_1-ibm and kvm).

The third 4.19 prepatch is out fortesting. Linus says: "Things look fairly normal".

The 4.18.7 stable kernel update isavailable; it contains 145 fixes. Note that there are updates for theother active stable kernels in the review process as well; they can beexpected almost any time.

The Maintainer's Summit, which is an invite-only gathering of 30 or so kernel developers to discuss process issues with Linus Torvalds, has moved from November 12 in Vancouver, Canada to October 22 in Edinburgh, Scotland in conjunction with Open Source Summit Europe. The technical side of the discussions will still be held as the Kernel Summit track at the Linux Plumbers Conference November 13-15 in Vancouver. There was, it seems, some confusion about the Maintainer's Summit, as Theodore Y. Ts'o said in the announcement of the move: "Last Friday (just before Labor Day) I learned that Linus had gottenconfused about when and where the Maintainer's Summit was going to beheld this year. And most unfortunately, he has already scheduled afamily vacation overlapping with the week of the Maintainer's Summit. [...] The Kernel Summit track will still be held in Vancouver alongsidePlumber's. Technical discussions will take place there; we simplywon't have the time, or necessarily, the right people, to havetechnical discussions at the Maintainer's Summit."

Security updates have been issued by Debian (qemu and xen), Mageia (libxkbcommon, sleuthkit, and wireshark), openSUSE (apache-pdfbox, dovecot22, and php7), SUSE (enigmail, kernel, nodejs4, and php7), and Ubuntu (firefox and transfig).

The Harvard Business School's "Working Knowledge" site has anarticle arguing that it can pay for companies to allow their developersto contribute back to the projects whose software they use."And that presents an interesting dilemma for firms that rely heavilyon open source. Should they allow employees on company time to make updatesand edits to the software for community use that could be used bycompetitors? New research by Assistant Professor Frank Nagle, a member ofthe Strategy Unit at Harvard Business School, shows that paying employeesto contribute to such software boosts the company’s productivity from usingthe software by as much as 100 percent, when compared with free-ridingcompetitors."

Network packet headers contain a great deal of information, but thekernel often only needs a subset of that information to be able to performfiltering or associate any given packet with a flow. The piece of code thatfollows the different layers of packet encapsulation to find the importantdata is called a flow dissector. In current Linux kernels, the flowdissectoris written in C. A patch set has beenproposed recently to implement it in BPF with the clear goal of improvingsecurity, flexibility, and maybe even performance.

Security updates have been issued by Debian (curl, gdm3, git-annex, lcms2, and sympa), Fedora (discount, dolphin-emu, gd, obs-build, osc, tcpflow, and yara), openSUSE (wireshark), Slackware (curl, firefox, ghostscript, and thunderbird), SUSE (apache-pdfbox, curl, dovecot22, and libvirt), and Ubuntu (libtirpc).

The LWN.net Weekly Edition for September 6, 2018 is available.

Security and convenience rarely go hand-in-hand, but if your job (or life)requires extraordinary care against potentially targeted attacks, thesecurity side of that tradeoff may win out. If so, running a system likeQubes OS on your desktop or CopperheadOS on your phone might make sense,which is just what Konstantin Ryabitsev, Linux Foundation (LF) director of ITsecurity, has done. He reported on the experience in a talk[YouTube video] entitled "Life Behind the Tinfoil Curtain" at the 2018LinuxSecurity Summit North America.