LWN.net

In a keynote on the first day of PyCon 2017,Jake VanderPlas looked at the relationship between Python and science. Overthe last ten years or so, there has been a large rise in the amount ofPython code being used—and released—by scientists. There are reasons forthat, which VanderPlas described, but, perhaps more importantly, thegrowing practiceof releasing all of this code can help solve one of the major problems facing science today:reproducibility.

One of the ways to harden the kernel is by tightening permissions on memoryto write-protect as much run-time data as possible. This means thekernel makes some data structures read-only to prevent malicious oraccidental corruption. However, inevitably, most data structures needread/write access at some point. Because of this, a blanket read-onlypolicy for these structures wouldn't work. Therefore, we need a mechanism that keepssensitive data structures read-only when "at rest", but allows writes whenthe need arises.

The Perl 5.26.0 release is out. "Perl 5.26.0 represents approximately 13 months of development since Perl5.24.0 and contains approximately 360,000 lines of changes across 2,600files from 86 authors". See thispage for a list of changes in this release; new features includeindented here-documents, the ability to declare references to variables,Unicode 9.0 support, and the removal of the current directory(".") from @INC by default.

In something of a follow-on to his session(with Cory Benfield) at the 2016 Python Language Summit, Christian Heimesgave an update on the state of the Python ssl module.In it, he covered some changes that have been made in the last year as wellas some changes that are being proposed. Heimes and Benfield are theco-maintainers of the ssl module.

Security updates have been issued by Arch Linux (vlc), CentOS (kernel, nss, and sudo), Debian (nss, tnef, wordpress, and xen), Fedora (kernel and puppet), SUSE (libtirpc, rpcbind), and Ubuntu (libsndfile, nvidia-graphics-drivers-375, and openldap).

At the 2017 Python Language Summit, Nathaniel Smith led a session on Trio—anasynchronous library he has recently been working on that uses theasync and await keywords that have come about in recent Python releases. It is meant to be an alternative to the asynciomodule. The session was targeted at relaying what Smith has learned in the process of writing Trio and to see wherethings might go from here.

Lars Knoll takes a lookat the Qt 5.9 LTS release. "With Qt 5.9, we have had a strong focus on performance and stability. We’ve fixed a large number of bugs all across Qt, and we have done a lot of work to improve our continuous integration system. This will make it a lot easier for us to create new releases (both patch level and minor releases) from 5.9 onward.We’ve also added automated performance regression testing to our testinginfrastructure, something that will allow us to continuously monitor ourwork on improving the performance of Qt." Qt 5.9 will be supportedfor three years.

Last week, Containers as kernel objectslooked at an attempt to add a formal "container" concept to the kernel,partly as a way of ensuring that kernel upcalls (calls to a user-spaceprogram from inside the kernel) would run inside the correct namespaces.This week, David Howells is back with adifferent approach: a way for a daemon process to intercept and handlespecific key-related upcalls.In particular, the keyctl() system call is enhanced with aKEYCTL_SERVICE_CREATE command, which returns a special filedescriptor. Subsequent calls can add "filters" describing the upcalls thatshould be intercepted; they are described by name and a set of flagsindicating a set of relevant namespaces. If the calling program'snamespaces match those of a process creating an upcall, that program willbe allowed to handle the call. See the patch posting for a more detaileddescription of how it works.

Security updates have been issued by Arch Linux (postgresql, postgresql-libs, samba, and sudo), Debian (gajim, libpodofo, openldap, pngquant, qemu-kvm, sudo, and tiff), Fedora (lxterminal, menu-cache, and pcmanfm), Gentoo (sudo), openSUSE (libraw, miniupnpc, and sudo), Oracle (kernel, nss, and sudo), Red Hat (kernel and sudo), Scientific Linux (kernel and sudo), Slackware (sudo), SUSE (java-1_6_0-ibm, java-1_8_0-openjdk, openstack-components, and sudo), and Ubuntu (sudo).

Victor Stinner sees a need to improve Python performance in order to keepit competitive with other languages. He brought up some ideas for doingthat in a 2017 Python Language Summit session. No solid conclusions werereached, but there is a seemingly growing segment of the core developerswho are interested in pushing Python's performance much further, possiblybreaking the existing C API in the process.

The Linux asynchronous I/O (AIO) layer tends to have many critics and fewdefenders, but most people at least expect it to actually be asynchronous. Intruth, an AIO operation can block in the kernel for a number of reasons,making AIO difficult to use in situations where the calling thread trulycannot afford to block. A longstanding patch set aiming to improve thissituation would appear to be nearing completion, but it is more of a stepin the right direction than a true solution to the problem.

The proceedings of the RISC-V workshop, held May 8-11 in Shanghai China,are availablewith links to slides and videos.This workshop was a four day event broken down as follow:

KDE has releasedPlasma 5.10. There are a number of new features in this release, includingmedia controls on lock screen, pause music on suspend, Software CentrePlasma Search (KRunner) suggests to install non-installed apps, filecopying notifications have a context menu on previews, 'desktop edit mode',when opening toolbox reveals applet handles, performance optimizations inPager and Task Manager, 'Often used' docs and apps in app launchers inaddition to 'Recently used', and much more.

Security updates have been issued by Arch Linux (lib32-nss), Debian (bind9, exiv2, fop, imagemagick, libical, libonig, libsndfile, mosquitto, openjdk-7, rzip, strongswan, and tnef), Fedora (git, kernel, lynis, moodle, mupdf, samba, systemd, and webkitgtk4), Mageia (perl-Image-Info and vlc), openSUSE (ffmpeg2, git, java-1_7_0-openjdk, libplist, libsndfile, and samba), Oracle (kernel and samba3x), Red Hat (nss), Scientific Linux (nss), and Ubuntu (imagemagick, juju-core, libtiff, strongswan, and webkit2gtk).

Linus has released the 4.12-rc3 kernelprepatch. "Hey, things continue to look good, and rc3 isn't evenvery big. I'm hoping there's not another shoe about to drop, but so farthis really feels like a nice calm release cycle, despite the size of themerge window."

The 3.1.0 release of the Mailman mailing list manager is out. "Two years after the original release of Mailman 3.0, this version contains ahuge number of improvements across the entire stack. Many bugs have beenfixed and new features added in the Core, Postorius (web u/i), and HyperKitty(archiver). Upgrading from Mailman 2.1 should be better too. We are seeingmore production sites adopt Mailman 3, and we've been getting great feedbackas these have rolled out.Important: mailman-bundler, our previous recommended way of deploying Mailman3, has been deprecated. Abhilash Raj is putting the finishing touches onDocker images to deploy everything, and he'll have a further announcement in aweek or two."New features include support for Python 3.5 and 3.6, MySQL support, new REST resources and methods, user interface and user experience improvements, and more.

On his blog, Siddhesh Poyarekar looks at tunables in the GNU C library (glibc). The idea for centralizing the handling of tunable parameters in the library started back 2013, but was added to glibc in version 2.25 that was released in February."Tunables is an internal implementation detail in glibc. It is a way to manage ways in which we allow behaviour in glibc to be modified. As of now the only way to manage glibc is via environment variables and the way to do that was strewn all over the place in the source code. Tunables provide one place to add the tunable parameter with all of the characteristics it would have and then the framework will handle everything from there. The user of that tunable (e.g. malloc for MALLOC_MMAP_THRESHOLD_ or malloc.mmap.threshold in tunables parlance) would then simply access the tunable from the list and do what it wants to do, without bothering about where it came from."

This article is a tour of some of the newest features in the gnuplot plotting utility.Some of these features are already present inthe 5.0 release, and some are planned for the nextofficial release, which will be gnuplot 5.2. Highlights in theupcoming releaseinclude hypertext labels, more control over axes, a long-awaited ability toadd labels to contours, better lighting effects, and more; read on for thedetails.

Security updates have been issued by CentOS (kernel), Debian (graphicsmagick, imagemagick, kde4libs, and puppet), Fedora (FlightCrew, kernel, libvncserver, and wordpress), Gentoo (adobe-flash, smb4k, teeworlds, and xen), Mageia (kernel, kernel-linus, kernel-tmb, and perl-CGI-Emulate-PSGI), openSUSE (GraphicsMagick and rpcbind), Oracle (kernel), Red Hat (kernel and kernel-rt), and Scientific Linux (kernel).

The Free Software Foundation's blog is carrying an interview with AJ Jordon, who runs the gplenforced.org site to support GPL enforcement efforts and to help other projects indicate their support. "gplenforced.org is a small site I made that has exactly two purposes: host a badge suitable for embedding into a README file on GitLab or something, and provide some text with an easy and friendly explanation of GPL enforcement for that badge to link to.Putting badges in READMEs has been pretty trendy for a while now — people add badges to indicate whether their test suite is passing, their dependencies are up-to-date, and what version is published in language package managers. gplenforced.org capitalizes on that trend to add the maintainer's beliefs about license enforcement, too."

Alpine Linux 3.6.0 has been released.Alpine is an independent, minimalist distribution that is built around musllibc and busybox to keep it small and resource efficient.This version adds support for 64-bit little-endian POWER machines (ppc64le)and 64-bit IBM z Systems (s390x).

The Devuan project set out to create a systemd-less Debian, and now DevuanJessie 1.0.0 Stable has been released."There have been no significant bug reports since Devuan Jessie RC2 was announced only three weeks ago and the list of releasecritical bugs is now empty. So finally Devuan Jessie Stable isready for release! As promised, this will also be aLong-Term-Support (LTS) release. Our team will participate inproviding patches, security updates, and release upgrades beyondthe planned lifespan of Debian Jessie."

Greg Kroah-Hartman has announced the release of the 4.11.3, 4.9.30, 4.4.70, and 3.18.55 stable kernels. They contain a ratherlarge set of patches all over the tree and users should upgrade.

Security updates have been issued by CentOS (samba and samba4), Mageia (samba), openSUSE (bash and samba), Oracle (samba and samba4), Slackware (samba), SUSE (ghostscript and java-1_7_0-openjdk), and Ubuntu (firefox and samba).

The LWN.net Weekly Edition for May 25, 2017 is available.

At the 2016 Python Language Summit, Larry Hastings introduced Gilectomy, his project to removethe global interpreter lock (GIL) from CPython. The GIL serializes accessto the Python interpreter, so it severely limits the performance ofmulti-threaded Python programs. At the 2017 summit, Hastings was back toupdate attendees on the progress he has made and where Gilectomy is headed.

In a brief session at the 2017 Python Language Summit, Maciej Szulik gavean update on the state and plans for bugs.python.org (bpo). It is the Roundup-based bug tracker forPython; moving to GitHub has not changed that. He described the work thattwo Google Summer of Code (GSoC) students have done to improve the bugtracker.

As part of a discussion in 2014 about where to host some ofthe Python repositories, Brett Cannon was delegated the task of determining where they should endup. In early 2016, he decided that Python'scode and other repositories (e.g. PEPs) should land at GitHub;at last year's language summit, he gave an overview of where thingsstood with a few repositories that had made the conversion. Since thattime, the CPython repository has made the switch and he wanted to discuss some of theworkflow issues surrounding that move at this year's summit.

The Samba Team has issued anadvisory regarding CVE-2017-7494: "All versions of Samba from3.5.0 onwards are vulnerable to a remote code execution vulnerability,allowing a malicious client to upload a shared library to a writable share,and then cause the server to load and execute it." Distributors arealready shipping the fix; there's also a workaround in the advisory forthose who cannot update immediately.

Your operating system generates a lot of run-time data and statistics thatare useful for monitoring system security and performance. How you get thisinformation depends on the operating system you're running. It could be afrom report in a fancy GUI, or obtained via a specialized API, or simply textvalues read from the filesystem in the case of Linux and/proc. However, imagine if you could get this data viaan SQL query, and obtain the output as a database table or JSONobject. This is exactly what osquery letsyou do on Linux, macOS, and Windows.

Check Point has issued anadvisory that a number of video-player applications can be compromisedvia specially crafted subtitles. "By crafting malicious subtitlefiles, which are then downloaded by a victim’s media player, attackers cantake complete control over any type of device via vulnerabilities found inmany popular streaming platforms, including VLC, Kodi (XBMC), Popcorn-Timeand strem.io. We estimate there are approximately 200 million video playersand streamers that currently run the vulnerable software, making this oneof the most widespread, easily accessed and zero-resistance vulnerabilityreported in recent years."

Ned Deily, release manager for the Python 3.6 and 3.7 series, openedup the 2017edition of the Python Language Summit with a look at the releaseprocess and where things stand. It was an "abbreviated update" to his talk at last year's summit, he said. He looked to the future for 3.6 and 3.7, but also looked a bit beyond those two.This is the start of LWN's coverage of the language summit; look for more articles over the next week or so.

Security updates have been issued by CentOS (libtirpc and rpcbind), Debian (libtasn1-3, libtasn1-6, and samba), Fedora (FlightGear, openvpn, and python-fedora), openSUSE (libtirpc and libxslt), Oracle (libtirpc and rpcbind), Red Hat (samba, samba3x, and samba4), Scientific Linux (samba and samba4), SUSE (java-1_7_0-ibm, java-1_7_1-ibm, java-1_8_0-ibm, samba, and tomcat), and Ubuntu (jbig2dec, miniupnpc, rtmpdump, and samba).

The kernel has, over the years, gained comprehensive support forcontainers; that, in turn, has helped to drive the rapid growth of a numberof containerization systems. Interestingly, though, the kernel itself hasno concept of what a container is; it just provides a number of facilitiesthat can be used in the creation of containers in user space. DavidHowells is trying to change that state of affairs with a patch set adding containers as a first-classkernel object, but the idea is proving to be a hard sell in the kernelcommunity.

The Document Foundation looks at the progress made in improving the qualityand reliability of LibreOffice's source code by using Google's OSS-Fuzz."Developers have used the continuous andautomated fuzzing process, which often catches issues just hours after theyappear in the upstream code repository, to solve bugs - and potentialsecurity issues - before the next binary release.LibreOffice is the first free office suite in the marketplace to leverageGoogle's OSS-Fuzz. The service, which is associated with other source codescanning tools such as Coverity, has been integrated into LibreOffice'ssecurity processes - under Red Hat's leadership - to significantly improvethe quality of the source code."

Security updates have been issued by Arch Linux (lynis), CentOS (kdelibs, libtirpc, rpcbind, and samba), Debian (miniupnpc), Fedora (chromium, chromium-native_client, and kernel), Oracle (kdelibs and samba), Red Hat (libtirpc and rpcbind), and Scientific Linux (kdelibs, libtirpc, rpcbind, and samba).

Richard Hughes describeshis work to address the MouseJackvulnerability in Logitech (and other) receivers. This vulnerability allows anattacker to pair new devices with the receiver with no user interaction orawareness, and, thus, take over the machine. "This makessitting in a café quite a dangerous thing to do when any affected hardwareis inserted, which for the unifying dongle is quite likely as it’sexplicitly designed to remain in an empty USB socket."Logitech has provided firmware updates, but not for "unsupported" platformslike Linux. Hughes has filled that gap by getting documentation and afixed firmware image from Logitech and adding support for these devices tofwupd. He is now looking for testers to ensure that the whole thing worksacross all devices. This is important work that is well worth supporting.

GNU Guix and GuixSD 0.13.0 have been released. GNU Guix is a transactionalpackage manager for the GNU system and the Guix System Distribution,GuixSD, is an advanced distribution of the GNU system. A couple ofhighlights in this version: Guix can now be used on aarch64 systems, andGuixSD now supports Btrfs and adds the LXDE desktop as an option. See theannouncement for more information.

FreeBSD has releasedits status report for the first quarter of 2017. As usual there arereports from the FreeBSD Core Team, the FreeBSD Foundation, the FreeBSDPorts Collection, and the FreeBSD Release Engineering Team, followed bymore information about ongoing projects, and more.

Security updates have been issued by Arch Linux (fop), Debian (dropbear, icu, and openjdk-7), Fedora (chicken, cinnamon-settings-daemon, jbig2dec, libtirpc, sane-backends, and smb4k), Mageia (flash-player-plugin, vlc, and webmin), Oracle (libtirpc and rpcbind), Red Hat (kdelibs, libtirpc, rpcbind, and samba), and SUSE (kernel).

The Debian-based Parsixdistribution has announcedthat it will be shutting down six months after the Debian "Stretch"release. "Parsix GNU/Linux 8.15 (Nev) will be fully supported duringthis time and users should be able to upgrade their installations to DebianStretch without any significant issues. We will make all necessary changes,and updates to ensure a smooth transition to Debian Stretch."

The 4.12-rc2 kernel prepatch is out."I'm back on the usual Sunday schedule, and everything else looksfairly normal too. This rc2 is maybe a bit bigger than usual, but thewhole merge window was bigger than most, so maybe it's just that. Andit's not like it's huge".

The4.11.2,4.10.17,4.9.29,4.4.69, and3.18.54stable kernel updates have all been released with the usual set ofimportant fixes. Note that this is the final update for the 4.10 kernel.

Back in 2014, the revelation that thekernel'smemory-management subsystem would not allow relatively small allocationrequests to fail created a bit of a stir. The discussion has settled downsince then, but the "too small to fail" rule still clearly creates acertain amount of confusion in the kernel community, as is evidenced by arecent discussion inspired by the 4.12 merge window. It would appear thatthe rule remains in effect, but developers are asked to act as if it did not.

The CoreOS Blog introduces the firstbeta release, v0.0.1, of zetcd. "Distributed systems commonly relyon a distributed consensus to coordinate work. Usually the systemsproviding distributed consensus guarantee information is delivered in orderand never suffer split-brain conflicts. The usefulness, but rich designspace, of such systems is evident by the proliferation of implementations;projects such as chubby, ZooKeeper, etcd, and consul, despite differing in philosophyand protocol, all focus on serving similar basic key-value primitives fordistributed consensus. As part of making etcd the most appealing foundationfor distributed systems, the etcd team developed a new proxy, zetcd, toserve ZooKeeper requests with an unmodified etcd cluster."

Security updates have been issued by Debian (deluge, jbig2dec, mysql-connector-java, and nss), Fedora (jasper), Mageia (mhonarc and radicale), openSUSE (smb4k), SUSE (kdelibs4 and rpcbind), and Ubuntu (jasper and openjdk-7).

Among the many features merged for the 4.11kernel was the "shared memory communications over RDMA" (SMC-R)protocol from IBM. SMC-R is ahigh-speed data-center communications protocol that is claimed to be muchmore efficient than basic TCP sockets. As it turns out, though, the merging of this code was a surprise — and an unpleasantone at that — to a relevant segment of the kernel development community.This issue and the difficulties in resolving it are an indicator of how theincreasingly fast-paced kernel development community can go off track.

Security updates have been issued by Debian (shadow), Fedora (rpcbind), Gentoo (gst-plugins-bad and tomcat), Red Hat (ansible and openshift-ansible, openstack-heat, and Red Hat OpenStack Platform director), and Ubuntu (bash, FreeType, linux-aws, linux-gke, linux-raspi2, linux-snapdragon, and linux-lts-xenial).

The LWN.net Weekly Edition for May 18, 2017 is available.

The Android Developers blog looksat the latest Android O Developer Preview, which is now in publicbeta. The developer preview also contains an early version of a projectcalled Android Go which is built specifically for Android devices that have1GB or less of memory.