LWN.net

Many benchmarks have been used by kernel developers over the years totest the performance of the scheduler. But recent kernel commit messageshave shown a particular pattern of tools being used (some relatively new),all of which were created specifically for developing scheduler patches.While each benchmark is different, having its own unique genesis story andintended testing scenario, there is a unifying attribute; they were allwritten to scratch a developer's itch.

Tails 3.0 has been released.Tails, the amnesic incognito live system, is a Debian-based live systemaimed at preserving privacy and anonymity. Version 3.0 is based on Debian9 (stretch). "It brings a completely new startup and shutdown experience, a lot of polishing to the desktop, security improvements in depth, and major upgrades to a lot of the included software."

Free electrons has released the initialversion of the ElixirCross-Referencer, a Linux source code cross-referencing online tool.Elixir uses a new engine written in Python that replaces LXR, theengine used in free electron's previous online tool. "Another reason that motivated a complete rewrite was that we wanted to provide an up-to-date reference (including the latest revisions) while keeping it immutable, so that external links to the source code wouldn’t get broken in the future. As a direct consequence, we would need to index many different revisions for each project, with potentially a lot of redundant information between them. That’s when we realized we could leverage the data model of Git to deal with this redundancy in an efficient manner, by indexing Git blobs, which are shared between revisions. In order to make sure queries under this strategy would be fast enough, we wrote a proof-of-concept in Python, and thus Elixir was born."

Firefox 54.0 has been released. The releasenotes are somewhat sparse, however thisblog post contains more information about some changes under-the-hood."To make Firefox run even complex sites faster, we’ve been changing it to run using multiple operating system processes. Translation? The old Firefox used a single process to run all the tabs in a browser. Modern browsers split the load into several independent processes. We named our project to split Firefox into multiple processes ‘Electrolysis (E10S)’ after the chemical process that divides water into its core elements. E10S is the largest change to Firefox code in our history. And today we’re launching our next big phase of the E10S initiative."

Fedora Magazine announcedthe release of Fedora 26 Beta. A final release is expected in July.The beta is available for Workstation, Server, Atomic Host, Spins, Labs,and ARM products. Fedora 26 brings many changes which can be seen in thechange set.

Security updates have been issued by Debian (tiff, tiff3, and zziplib), Fedora (libsndfile, log4j12, and postgresql), Oracle (qemu-kvm), and Scientific Linux (qemu-kvm).

The 4.12-rc5 prepatch is out; it is ratherlarger than others in this cycle, Linus Torvalds said. "It's not like rc5 is *huge*, but it definitely isn't the nice andsmall one I was hoping for. There's nothing in [particular] that looksvery worrisome, and it may well just be random timing - the rc sizesdo fluctuate a lot depending on just which subsystem gets synced upthat particular rc, and we may just have hit that "everybody happenedto sync up this week" case."

Security updates have been issued by Arch Linux (irssi, lib32-libtasn1, and wireshark-cli), Debian (libmwaw, otrs2, and tor), Fedora (ansible, freeradius, gnutls, mingw-poppler, mosquitto, oniguruma, perltidy, picocom, systemd, and wget), Mageia (ansible, dropbear, gajim, libsndfile, libxslt, lxc, zoneminder, and zziplib), openSUSE (ffmpeg, libnettle, mysql-connector-cpp, mysql-workbench, and wireshark), and Ubuntu (irssi).

PostgreSQL version 10 had its first beta release on May18, just in time for the annual PGCon developerconference. The latest annual release comes with a host of majorfeatures, including new versions of replication and partitioning, andenhanced parallel query. Version 10 includes 451 commits, nearly half amillion lines of code and documentation, and over 150 new or changedfeatures since version 9.6. The PostgreSQLcommunity will find a lot to get excited about in this release, as the project has delivered a long list of enhancements toexisting functionality. There's also a few features aimed at fulfillingnew use cases, particularly in the "big data" industry sector.

Security updates have been issued by Debian (ettercap), Fedora (mingw-poppler), Mageia (gc, libnl3, libtasn1, nss, puppet, and wireshark), and openSUSE (catdoc, gajim, GraphicsMagick, irssi, java-1_8_0-openjdk, kernel, libxml2, rxvt-unicode, and yodl).

Version 1.18 of the Rust programming language has been released."One of the largest changes is a long time coming: core team membersCarol Nichols and Steve Klabnik have been writing a new edition of “TheRust Programming Language”, the official book about Rust. It’s being written openly on GitHub, andhas over a hundred contributors in total. This release includes the first draft ofthe second edition in our online documentation. 19 out of 20 chaptershave a draft; the draft of chapter 20 will land in Rust 1.19."

G'MIC is a generic, extensible framework for image processing, often usedas a plug-in for GIMP. Version 2.0 has been released. "Oneof the major new features of this version 2.0 is the re-implementation ofthe plug-in code, from scratch. The repository G’MIC-Qt developed by Sébastien (an experienced memberof the team) is a Qt-based version of the plug-in interface, being asindependent as possible of the widget API provided by GIMP." Theannouncement has much more details about G'MIC and how it can be used. LWNlooked at G'MIC in August 2014.

Security updates have been issued by Debian (dropping support for some packages), Fedora (sudo), openSUSE (chromium), Slackware (irssi), and Ubuntu (freeradius and nagios3).

The LWN.net Weekly Edition for June 8, 2017 is available.

Over the course of the day, the 2017 Python Language Summit hosted ahandful of lightning talks, several of which were worked into the dynamicschedule when an opportunity presented itself. They ranged from thetraditional "less than five minutes" format to some that strayed welloutside of that time frame—some generated a fair amount of discussion aswell. Topics were all over the map: board elections, beta releases,Python as a security vulnerability, Jython, and more.

In his 2017 Python Language Summit session, Jukka Lehtosalo updatedattendees on the status of type checking for the language, in general, andfor the mypy static type checker.There are new features in the typing module and in mypy, as wellas work in progress and planned features for both. For a feature, typehints, that is really only around three yearsold, there has been a lot of progress made—but, of course, there isstill more to come.

There is no viable way to prevent data from being collected about us in thecurrent age of computing. But if institutions insist on knowing ourfinancial status, purchasing habits, health information,political preferences, and so on, they have a responsibility to keep thisdata—known as personally identifiable information (PII)—from leaking tounauthorized recipients. At the 2017 Strata dataconference in London, Steve Touw presented a sessionon privacy-enhancing technologies. In a fast-paced 40 minutes hecovered the EU regulations about privacy, the most popular technicalmeasures used to protect PII, and some pointed opinions about what worksand what should be thrown into the dustbin.

The Tor Browser Team has announced the first stable release in the 7.0 series. "This release brings us up to date with Firefox 52 ESR which contains progress in a number of areas:Most notably we hope having Mozilla's multiprocess mode (e10s) and content sandbox enabled will be one of the major new features in the Tor Browser 7.0 series, both security- and performance-wise. While we are still working on the sandboxing part for Windows (the e10s part is ready), both Linux and macOS have e10s and content sandboxing enabled by default in Tor Browser 7.0. In addition to that, Linux and macOS users have the option to further harden their Tor Browser setup by using only Unix Domain sockets for communication with tor."

Greg Kroah-Hartman has released stable kernels 4.11.4, 4.9.31, 4.4.71, and 3.18.56. All of them contain important fixesand users should upgrade.

Security updates have been issued by Arch Linux (chromium), Debian (apng2gif and ming), Gentoo (freetype, libpcre, minicom, pidgin, webkit-gtk, and wireshark), openSUSE (deluge and postgresql93), and Ubuntu (libnl3, lintian, linux, linux-aws, linux-gke, linux-raspi2, linux-snapdragon, linux, linux-raspi2, linux-hwe, and linux-lts-xenial).

Mark Shannon is concerned that the Python core developers may be replayinga mistake: treating two distinct things as being thesame. Treating byte strings and Unicode text-strings interchangeably ispart of what led to Python 3, so he would rather not see that happenagain with types and classes. The Python typingmodule, which is meant to support type hints, currentlyimplements types as classes. That leads to several kinds of problems, asShannon described in his session at the 2017 Python Language Summit.

The GnuPG Project has announced the launch of a funding campaign to furthersupport and improve its mail and data encryption software, GnuPG."The 6 person development team is currently financed from asuccessful campaign in early 2015, regular donations from the LinuxFoundation, Stripe, Facebook, and a few paid development projects. Toensure long-term stability the new campaign focuses on recurring donationsand not one-time donations."

Last month LWN pointed to an article aboutthe Artifex v. Hancom case, in which Hancom used Artifex's Ghostscript inits office product. The Free Software Foundation looks at the caseand the recent ruling. "On the latter, the judge found that thebusiness model of Artifex indicated a loss of revenue, but also noted thatharm could be found even where money isn't involved. The judge, quoting a prior case,noted that there are 'substantial benefits, including economicbenefits, to the creation and distribution of copyrighted works underpublic licenses that range far beyond traditional license royalties.'While not [dispositive], this last note is particularly interesting formany free software developers, who generally share their work at nocost."

Many bytes have been expended over the years discussing the virtues of thekernel's random number generation subsystem. One of the biggest recurringconcerns has to do with systems that are unable to obtain sufficiententropy during the boot process to meet early demands for random data. Thelatest discussion on this topic got off to a bit of a rough start, but itmay lead to an incremental improvement in this area.

The Gentoo security team has announced that the SPARC architecture will nolonger be supported by the security team. "This decision follows thecouncil decision on 2016-12-11, 'The council defers to the security team,but is supportive of dropping security support for sparc if it is unable togenerally meet the security team timelines.'"

Security updates have been issued by Arch Linux (tomcat7 and tomcat8), Debian (freeradius, perl, and yodl), Fedora (libtasn1 and poppler), Gentoo (dbus, filezilla, git, imageworsener, munge, mupdf, qemu, rpcbind, and shadow), and Ubuntu (libtasn1-6 and puppet).

The kernel uses a variety of lock types internally, but they all share onefeature in common: they are a simple either/or proposition. When a lock isobtained for a resource, the entire resource is locked, even ifexclusive access is only needed to a part of that resource. Many resourcesmanaged by the kernel are complex entities for which it may make sense toonly lock a smaller part; files (consisting of a range of bytes) or aprocess's address space are examples of this type of resource. For years,kernel developers have talked about adding "range locks" — locks that wouldonly apply to a portion of a given resource — as a way of increasingconcurrency. Work has progressed in thatarea, and range locks may soon be added to the kernel's locking toolkit.

Rivendell 2.16.0 has been released. Rivendell is a radio automation systemtargeted for use in professional broadcast environments. This versionincludes audio store hashing, kernel GPIO, Modbus TCP support, and more.

Security updates have been issued by Arch Linux (gajim and libusbmuxd), Debian (perl), Fedora (chromium, chromium-native_client, dropbear, squirrelmail, sudo, and wget), Mageia (git, menu-cache, and pcmanfm), and openSUSE (libupnp).

Version 8.0 of the GDB debugger is out. Changes in this release includesome Python scripting enhancements, DWARF version 5 support, some newtargets, and more.

The 4.12-rc4 kernel prepatch has beenreleased. "Things remain fairly calm for 4.12, although not quite as calm as itappeared earlier in the week. I think two thirds of the commits camein on Friday or the weekend.But timing aside, it all looks fairly normal."

The kernel's filesystem and block layers are places where a lot of thingscan go wrong, often with unpleasant consequences. To make things worse, whenthings do go wrong, informing user space about the problem can be difficultas a consequence of how block I/O works. That can result in user-spaceapplications being unaware of trouble at the I/O level, leading to lost data and enragedusers. There are now two separate (and complementary) proposals underdiscussion that aim to improve how error reporting is handled in the blocklayer.

Security updates have been issued by Arch Linux (freeradius and libtasn1), Debian (nss, openldap, picocom, strongswan, wordpress, and zookeeper), Mageia (openvpn), openSUSE (mariadb), Oracle (kernel and sudo), and SUSE (strongswan).

The LWN.net Weekly Edition for June 2, 2017 is available.

In a keynote on the first day of PyCon 2017,Jake VanderPlas looked at the relationship between Python and science. Overthe last ten years or so, there has been a large rise in the amount ofPython code being used—and released—by scientists. There are reasons forthat, which VanderPlas described, but, perhaps more importantly, thegrowing practiceof releasing all of this code can help solve one of the major problems facing science today:reproducibility.

One of the ways to harden the kernel is by tightening permissions on memoryto write-protect as much run-time data as possible. This means thekernel makes some data structures read-only to prevent malicious oraccidental corruption. However, inevitably, most data structures needread/write access at some point. Because of this, a blanket read-onlypolicy for these structures wouldn't work. Therefore, we need a mechanism that keepssensitive data structures read-only when "at rest", but allows writes whenthe need arises.

The Perl 5.26.0 release is out. "Perl 5.26.0 represents approximately 13 months of development since Perl5.24.0 and contains approximately 360,000 lines of changes across 2,600files from 86 authors". See thispage for a list of changes in this release; new features includeindented here-documents, the ability to declare references to variables,Unicode 9.0 support, and the removal of the current directory(".") from @INC by default.

In something of a follow-on to his session(with Cory Benfield) at the 2016 Python Language Summit, Christian Heimesgave an update on the state of the Python ssl module.In it, he covered some changes that have been made in the last year as wellas some changes that are being proposed. Heimes and Benfield are theco-maintainers of the ssl module.

Security updates have been issued by Arch Linux (vlc), CentOS (kernel, nss, and sudo), Debian (nss, tnef, wordpress, and xen), Fedora (kernel and puppet), SUSE (libtirpc, rpcbind), and Ubuntu (libsndfile, nvidia-graphics-drivers-375, and openldap).

At the 2017 Python Language Summit, Nathaniel Smith led a session on Trio—anasynchronous library he has recently been working on that uses theasync and await keywords that have come about in recent Python releases. It is meant to be an alternative to the asynciomodule. The session was targeted at relaying what Smith has learned in the process of writing Trio and to see wherethings might go from here.

Lars Knoll takes a lookat the Qt 5.9 LTS release. "With Qt 5.9, we have had a strong focus on performance and stability. We’ve fixed a large number of bugs all across Qt, and we have done a lot of work to improve our continuous integration system. This will make it a lot easier for us to create new releases (both patch level and minor releases) from 5.9 onward.We’ve also added automated performance regression testing to our testinginfrastructure, something that will allow us to continuously monitor ourwork on improving the performance of Qt." Qt 5.9 will be supportedfor three years.

Last week, Containers as kernel objectslooked at an attempt to add a formal "container" concept to the kernel,partly as a way of ensuring that kernel upcalls (calls to a user-spaceprogram from inside the kernel) would run inside the correct namespaces.This week, David Howells is back with adifferent approach: a way for a daemon process to intercept and handlespecific key-related upcalls.In particular, the keyctl() system call is enhanced with aKEYCTL_SERVICE_CREATE command, which returns a special filedescriptor. Subsequent calls can add "filters" describing the upcalls thatshould be intercepted; they are described by name and a set of flagsindicating a set of relevant namespaces. If the calling program'snamespaces match those of a process creating an upcall, that program willbe allowed to handle the call. See the patch posting for a more detaileddescription of how it works.

Security updates have been issued by Arch Linux (postgresql, postgresql-libs, samba, and sudo), Debian (gajim, libpodofo, openldap, pngquant, qemu-kvm, sudo, and tiff), Fedora (lxterminal, menu-cache, and pcmanfm), Gentoo (sudo), openSUSE (libraw, miniupnpc, and sudo), Oracle (kernel, nss, and sudo), Red Hat (kernel and sudo), Scientific Linux (kernel and sudo), Slackware (sudo), SUSE (java-1_6_0-ibm, java-1_8_0-openjdk, openstack-components, and sudo), and Ubuntu (sudo).

Victor Stinner sees a need to improve Python performance in order to keepit competitive with other languages. He brought up some ideas for doingthat in a 2017 Python Language Summit session. No solid conclusions werereached, but there is a seemingly growing segment of the core developerswho are interested in pushing Python's performance much further, possiblybreaking the existing C API in the process.

The Linux asynchronous I/O (AIO) layer tends to have many critics and fewdefenders, but most people at least expect it to actually be asynchronous. Intruth, an AIO operation can block in the kernel for a number of reasons,making AIO difficult to use in situations where the calling thread trulycannot afford to block. A longstanding patch set aiming to improve thissituation would appear to be nearing completion, but it is more of a stepin the right direction than a true solution to the problem.

The proceedings of the RISC-V workshop, held May 8-11 in Shanghai China,are availablewith links to slides and videos.This workshop was a four day event broken down as follow:

KDE has releasedPlasma 5.10. There are a number of new features in this release, includingmedia controls on lock screen, pause music on suspend, Software CentrePlasma Search (KRunner) suggests to install non-installed apps, filecopying notifications have a context menu on previews, 'desktop edit mode',when opening toolbox reveals applet handles, performance optimizations inPager and Task Manager, 'Often used' docs and apps in app launchers inaddition to 'Recently used', and much more.

Security updates have been issued by Arch Linux (lib32-nss), Debian (bind9, exiv2, fop, imagemagick, libical, libonig, libsndfile, mosquitto, openjdk-7, rzip, strongswan, and tnef), Fedora (git, kernel, lynis, moodle, mupdf, samba, systemd, and webkitgtk4), Mageia (perl-Image-Info and vlc), openSUSE (ffmpeg2, git, java-1_7_0-openjdk, libplist, libsndfile, and samba), Oracle (kernel and samba3x), Red Hat (nss), Scientific Linux (nss), and Ubuntu (imagemagick, juju-core, libtiff, strongswan, and webkit2gtk).

Linus has released the 4.12-rc3 kernelprepatch. "Hey, things continue to look good, and rc3 isn't evenvery big. I'm hoping there's not another shoe about to drop, but so farthis really feels like a nice calm release cycle, despite the size of themerge window."

The 3.1.0 release of the Mailman mailing list manager is out. "Two years after the original release of Mailman 3.0, this version contains ahuge number of improvements across the entire stack. Many bugs have beenfixed and new features added in the Core, Postorius (web u/i), and HyperKitty(archiver). Upgrading from Mailman 2.1 should be better too. We are seeingmore production sites adopt Mailman 3, and we've been getting great feedbackas these have rolled out.Important: mailman-bundler, our previous recommended way of deploying Mailman3, has been deprecated. Abhilash Raj is putting the finishing touches onDocker images to deploy everything, and he'll have a further announcement in aweek or two."New features include support for Python 3.5 and 3.6, MySQL support, new REST resources and methods, user interface and user experience improvements, and more.