LWN.net

The release of pip 10.0 has been announced. Some highlights of thisrelease include the removal of Python 2.6 support, limited PEP 518 support (withmore to come), a new "pip config" command, and other improvements.

The new PyPI has been launched. Browsertraffic and API calls (including "pip install") have been redirected fromthe old pypi.python.org to the new site. The old PyPI will shut down onApril 30. LWN covered the new PyPI last week.

Developers of database management systems are, by necessity, concernedabout getting data safely to persistent storage. So when the PostgreSQLcommunity found out that the way the kernel handles I/O errors could resultin data being lost without any errors being reported to user space, a fairamount of unhappiness resulted. The problem, which is exacerbated by theway PostgreSQL performs buffered I/O, turns out not to be unique to Linux,and will not be easy to solve even there.

Security updates have been issued by Debian (freeplane and jruby), Fedora (kernel and python-bleach), Gentoo (evince, gdk-pixbuf, and ncurses), openSUSE (kernel), Oracle (gcc, glibc, kernel, krb5, ntp, openssh, openssl, policycoreutils, qemu-kvm, and xdg-user-dirs), Red Hat (corosync, glusterfs, kernel, and kernel-rt), SUSE (openssl), and Ubuntu (openssl and perl).

Security updates have been issued by Debian (corosync, linux-tools, qemu, qemu-kvm, and r-cran-readxl), openSUSE (evince, memcached, nodejs4, ntp, pdns-recursor, python-gunicorn, python3-gunicorn, and python3), and Ubuntu (ruby1.9.1, ruby2.0, ruby2.3).

Microsoft has issued apress release describing the security dangers involved with theInternet of things ("a weaponized stove, baby monitors that spy, thecontents of your refrigerator being held for ransom") and introducing"Microsoft Azure Sphere" as a combination of hardware and software toaddress the problem. "Unlike the RTOSes common to MCUs today, ourdefense-in-depth IoT OS offers multiple layers of security. It combinessecurity innovations pioneered in Windows, a security monitor, and a customLinux kernel to create a highly-secured software environment and atrustworthy platform for new IoT experiences."

Alpine Linux-based postmarketOS is touch-optimized and pre-configured forinstallation on smartphones and other mobile devices. The postmarketOSblog introducespostmarketOS-lowlevel which is a community project aimed at creatingfree bootloaders and cellular modem firmware, currently focused on MediaTekphones. "But before we get started, please keep in mind that theseare moon shots. So while there is some little progress, it's mostly aboutletting fellow hackers know what we've tried and what we're up to, in thehopes of attracting more interested talent to our cause. After all, ourphilosophy is to keep the community informed and engaged during thedevelopment phase!"

Security updates have been issued by Arch Linux (lib32-openssl and zsh), Debian (patch, perl, ruby-loofah, squirrelmail, tiff, and tiff3), Fedora (gnupg2), Gentoo (go), Mageia (firefox, flash-player-plugin, nxagent, puppet, python-paramiko, samba, and thunderbird), Red Hat (flash-plugin), Scientific Linux (python-paramiko), and Ubuntu (patch, perl, and ruby).

Version 1.10 of the Subversion version-control system is out.Improvements include a new interactive resolver for merge conflicts, betterpath-based authorization, LZ4 compression, and more; see therelease notes for details.

By the time the 4.17 merge window was closed and 4.17-rc1 was released, 11,769 non-merge changesets had been pulled into themainline repository. 4.17 thus looks to be a typically busy developmentcycle, with a merge window only slightly more busy than 4.16 had.Some 6,000 of those changes were pulled after last week's summary was written. There was alot of the usual maintenance work in those patches (over 10% of thosechanges were to device-tree files, for example), but also some moresignificant changes.

Linus has released 4.17-rc1 and closed themerge window for this release. "This does not seem to be shaping upto be a particularly big release, and there seems to be nothingparticularly special about it. The most special thing that happened ispurely numerology: we've passed the six million git objects mark, and thatis reason enough to call the next kernel 5.0. Except I probably won't,because I don't want to be too predictable."

A comparison of the feature sets for a handful of terminal emulators wasthe subject of a recent article; here I follow that up byexamining the performance of those terminals. This might seem like alesser concern, but as it turns out, terminals exhibit surprisinglyhigh latency for such fundamental programs. I also examine what istraditionally considered "speed" (but is really scroll bandwidth) andmemory usage, with the understanding that the impact of memory useis less than it was when I looked at this a decade ago (inFrench).Subscribers can read on for part 2 from guest author Antoine Beaupré.

The stable kernel train just keeps on rolling; Greg Kroah-Hartman has announcedthe release of the 4.9.94, 4.4.128, and 3.18.105 stable kernels. All contain a largenumber of fixes throughout the tree and users should upgrade.

The rhashtable data structure is a generic resizable hash-tableimplementation in the Linux kernel, which LWN first introduced as "relativistichash tables" back in 2014. I thought at the time that it might be fun to makeuse of rhashtables, but didn't, until an opportunity arose through my work onthe Lustre filesystem. Lustre is a cluster filesystem that is currently indrivers/staging while the code is revised to meet upstreamrequirements. One of those requirements is to avoid duplicatingsimilar functionality where possible. As Lustre contains a resizablehash table, it really needs to be converted to use rhashtables instead — atlast I have my opportunity.Subscribers can read on for a look at the rhashtable API by guest authorNeil Brown.

Security updates have been issued by Arch Linux (apache), openSUSE (libvirt, openssl, policycoreutils, and zziplib), Oracle (firefox and python-paramiko), and Red Hat (python-paramiko).

Greg Kroah-Hartman has released three new stable kernels: 4.16.2, 4.15.17, and 4.14.34. Users of those kernel series shouldupgrade.

Security updates have been issued by Debian (poppler), Fedora (koji and libofx), Gentoo (adobe-flash), Oracle (kernel), Red Hat (qemu-kvm-rhev and sensu), and Scientific Linux (firefox).

The LWN.net Weekly Edition for April 12, 2018 is available.

A "simple" utility to make a system beep is hardly the first place one wouldcheck for security flaws, but the strange case of the "Holey Beep"should perhaps lead to some rethinking. A Debian advisory for the beep utility, which was followedby another for Debian LTS, led to aseemingly satirical site publicizingthe bug (and giving it the "Holey Beep" name). But that site also exploitsa new flaw in the GNUpatch program—and the increased scrutiny on beep hasled to more problems being found.

The Python Package Index (PyPI) isthe principal repository of libraries for the Python programming language,serving more than 170 million downloads each week. Fifteen years after PyPIlaunched, a new edition is in beta at pypi.org, with features like bettersearch, a refreshed layout, and Markdown README files(and with some old features removed, like viewing GPG package signatures). StartingApril 16, users visiting the site or running pip install willbe seamlessly redirected to the new site. Two weeks after that, the legacy site isexpected to be shut down and the team will turn toward newfeatures; in the meantime, it is worth a look at what the new PyPI bringsto the table.

Security updates have been issued by Debian (pcs), Fedora (drupal7), openSUSE (git and mercurial), Red Hat (firefox and qemu-kvm-rhev), SUSE (libvirt and xen), and Ubuntu (patch).

Car manufacturers, like most companies, navigate a narrow lane between thebenefits of using free and open-source software and the perceived or realimportance of hiding their trade secrets. Many are usingfree software in some of the myriad software components that make up amodern car, and even work in consortia to develop free software. At therecent LibrePlanetconference, free-software advocate Jeremiah Foster covered progress in theautomotive sector and made an impassioned case for more free software in theirembedded systems.Subscribers can read on for a report on the talk by guest author Andy Oram.

Red Hat has announcedthe general availability of Red Hat Enterprise Linux 7.5. This versionfeatures enhanced hybrid cloud security and compliance, improved storageperformance and efficiency, simplified management, and production-readyLinux containers. RHEL 7.5 is available for x86, IBM Power, IBM z Systems, and 64-bit Arm. This release also brings support for single-host KVM virtualization and Open Container Initiative (OCI)-formatted runtime environment and base image to IBM z Systems.

The 3.18.104 kernel has been released witha single bugfix. If you had build errors in 3.18.103 then this update isfor you, otherwise there is no need to upgrade.

Security updates have been issued by CentOS (libvorbis and thunderbird), Debian (pjproject), Fedora (compat-openssl10, java-1.8.0-openjdk-aarch32, libid3tag, python-pip, python3, and python3-docs), Gentoo (ZendFramework), Oracle (thunderbird), Red Hat (ansible, gcc, glibc, golang, kernel, kernel-alt, kernel-rt, krb5, kubernetes, libvncserver, libvorbis, ntp, openssh, openssl, pcs, policycoreutils, qemu-kvm, and xdg-user-dirs), SUSE (openssl and openssl1), and Ubuntu (python-crypto, ubuntu-release-upgrader, and wayland).

Jim Gettys refutesthe claim that the early designers of Internet software were notconcerned about security. "Government export controls crippledInternet security and the design of Internet protocols from the verybeginning: we continue to pay the price to this day".

Several security vulnerabilities were found in Etherpad and version1.6.4 has been released with fixes. The vulnerabilities includearbitrary code execution and information disclosure. Site admins are urgedto update Etherpad to 1.6.4 as soon as possible.

Security updates have been issued by Arch Linux (openssl and zziplib), Debian (ldap-account-manager, ming, python-crypto, sam2p, sdl-image1.2, and squirrelmail), Fedora (bchunk, koji, libidn, librelp, nodejs, and php), Gentoo (curl, dhcp, libvirt, mailx, poppler, qemu, and spice-vdagent), Mageia (389-ds-base, aubio, cfitsio, libvncserver, nmap, and ntp), openSUSE (GraphicsMagick, ImageMagick, spice-gtk, and wireshark), Oracle (kubernetes), Slackware (patch), and SUSE (apache2 and openssl).

The Linux network stack does not lack for features; it also performs wellenough for most uses. At the highest network speeds, though, any overheadat all is too much; that has driven the most demanding users towardspecialized, user-space networking implementations that can outperform thekernel for highly constrained tasks. The express data path (XDP)development effort is an attempt to win those users back, with some apparentsuccess so far. With the posting of the AF_XDP patch set by Björn Töpel,another piece of the XDP puzzle is coming into focus.

The4.16.1,4.15.16,4.14.33,4.9.93,4.4.127, and3.18.103stable kernels have all been released; each contains a fairly long list ofimportant fixes.

As the 4.17 merge window opened, it seemedpossible that the kernel lockdown patch set could be merged at last.That was before the linux-kernel mailing list got its hands on the issue.What resulted was not one of the kernel community's finest moments. But itdid result in a couple of evident conclusions: kernel lockdown will almostcertainly not bemerged for 4.17, but something that looks very much like it is highlylikely to be accepted in a subsequent merge window.

Security updates have been issued by Debian (sharutils), Fedora (firefox, httpd, and mod_http2), openSUSE (docker-distribution, graphite2, libidn, and postgresql94), Oracle (libvorbis and thunderbird), Red Hat (libvorbis, python-paramiko, and thunderbird), Scientific Linux (libvorbis and thunderbird), SUSE (apache2), and Ubuntu (firefox, linux-lts-xenial, linux-aws, and ruby1.9.1, ruby2.0, ruby2.3).

As of this writing, 5,392 non-merge changesets have been pulled into themainline repository for the 4.17 release. The 4.17 merge window is thusoff to a good start, but it is far from complete. The changes pulled thusfar cover a wide part of the core kernel as well as the networking, driver,and filesystem subsystems.

Security updates have been issued by Arch Linux (drupal), Debian (openjdk-7), Fedora (exempi, gd, and tomcat), SUSE (python-paramiko), and Ubuntu (kernel, libvncserver, linux, linux-aws, linux-kvm, linux-raspi2, linux-snapdragon, linux-lts-trusty, and linux-raspi2).

The LWN.net Weekly Edition for April 5, 2018 is available.

It has been known for quite some time that Python 2 will reach its endof life in 2020—after being extended by five years from its original 2015expiry. After that, there will be no support, bug fixes, or security patches forPython 2, at least from the Python Software Foundation and the coredevelopers. Some distributions will need to continue to support the finalPython 2 release, however, since their support windows extend pastthat date; the enterprise and long-term support distributions willlikely be supporting it well into the 2020s and possibly beyond. But evenshorter-support-cycle distributions need to consider their plan for asweeping change of this sort—in less than two years.

Security updates have been issued by Debian (apache2, ldap-account-manager, and openjdk-7), Fedora (libuv and nodejs), Gentoo (glibc and libxslt), Mageia (acpica-tools, openssl, and php), SUSE (clamav, coreutils, and libvirt), and Ubuntu (kernel, libraw, linux-hwe, linux-gcp, linux-oem, and python-crypto).

The Linux Foundation and Nitrokey have announceda program whereby anybody who appears in the kernel's MAINTAINERS file orwho has a kernel.org email address can obtain a free Nitrokey Start crypto card. Theintent, of course, is that kernel developers will use these devices tosafeguard their GnuPG keys and, as a result, improve the security of thekernel development process as a whole. "A digital smartcard tokenlike Nitrokey Start contains a cryptographic chip that is capable ofstoring private keys and performing crypto operations directly on the tokenitself. Because the key contents never leave the device, the operatingsystem of the computer into which the token is plugged in is not able toretrieve the private keys themselves, therefore significantly limiting theways in which the keys can be leaked or stolen."See this LWN article for a look at crypto cards.

One of the trickiest aspects to concurrency in the kernel is waiting for aspecific event to take place. There is a wide variety of possible events,including a process exiting, the last reference to a data structure goingaway, a device completing an operation, or a timeout occurring.Waiting is surprisingly hard to get right — race conditions abound to trapthe unwary — so the kernel hasaccumulated a large set of wait_event_*() macros to make the task easier. Anattempt to add a new one, though, has led to the generalization of specifictypes of waits for 4.17.

Many large institutions, especially government agencies, would like todistribute their software—including the software of the vendors with whomthey contract—as free software. They have a variety of reasons, rangingfrom the hope that opening the code will boost its use, all the way toa mature understanding of the importance of community, transparency, andfreedom. There are special steps institutions can take to help ensure success,some stemming from best practices performed by many free-software projectsand others specific to large organizations. At the 2018 LibrePlanet conference,Cecilia Donnelly laid out nine principles for the successful creation and maintenance of a software project under thesecircumstances.

Security updates have been issued by Debian (beep and jruby), Fedora (libvncserver), and Ubuntu (openjdk-7 and openjdk-8).

Version 2.17.0 of the Git source-code management system is out. Itincludes a long list of relatively minor tweaks. "Since Git 1.7.9,'git merge' defaulted to --no-ff (i.e. even when the side branch beingmerged is a descendant of the current commit, create a merge commit insteadof fast-forwarding) when merging a tag object. This was appropriatedefault for integrators who pull signed tags from their downstreamcontributors, but caused an unnecessary merges when used by downstreamcontributors who habitually 'catch up' their topic branches with taggedreleases from the upstream. Update 'git merge' to default to --no-ff onlywhen merging a tag object that does *not* sit at its usual place inrefs/tags/ hierarchy, and allow fast-forwarding otherwise, to mitigate theproblem."

The GnuCash 3.0 release is out. "The headline item for this release is that GnuCash now uses the Gtk+-3.0Toolkit and the WebKit2Gtk API. This change was forced on us by some majorLinux distributions dropping support for the WebKit1 API." Thisrelease also includes some new reports, a rewritten CSV importer, andmore. LWN looked at GnuCash from abusiness-accounting point of view in August 2017.

The OpenBSD 6.3 release is out. "The release was scheduled for April15, but since all the components are ready ahead of schedule it is beingreleased now." This release includes mitigation for the Meltdownvulnerability but not for Spectre on x86.

The UEFI secure boot mechanism is intended to protect the system againstpersistent malware threats — unpleasant bits of software attached to theoperating system or bootloader that will survive a reboot. While Linuxhas supported secure boot for some time, proponents have long said thatthis support is incomplete in that it is still possible for the root userto corrupt the system in a number of ways. Patches that attempt toclose this hole have been circulating for years, but they have beencontroversial at best. This story may finally come to a close, though, ifLinus Torvalds accepts the "kernel lockdown" patch series during the 4.17merge window.

Security updates have been issued by Debian (dovecot, irssi, libevt, libvncserver, mercurial, mosquitto, openssl, python-django, remctl, rubygems, and zsh), Fedora (acpica-tools, dovecot, firefox, ImageMagick, mariadb, mosquitto, openssl, python-paramiko, rubygem-rmagick, and thunderbird), Mageia (flash-player-plugin and squirrelmail), Slackware (php), and Ubuntu (dovecot).

Linus has released the 4.16 kernel, asexpected. "We had a number of fixes and cleanups elsewhere, but noneof it made me go 'uhhuh, better let this soak for another week'".Some of the headline changes in this release include initial support forthe Jailhousehypervisor, the usercopy whitelistinghardening patches, some improvements to the deadline scheduler and, ofcourse, a lot of Meltdown and Spectre mitigation work.

The stable kernel update machine continues to generate releases:4.15.15,4.14.32,4.9.92, and4.4.126 are now available with another setof important fixes.

Terminals have a special place in computing history, surviving alongwith the command line in the face of the rising ubiquity of graphicalinterfaces. Terminal emulators have replacedhardwareterminals, which themselves were upgrades from punched cards and toggle-switch inputs. Modern distributions now ship with asurprising variety of terminal emulators. While some people may behappy with the default terminal provided by their desktop environment,others take great pride at using exotic software for running theirfavorite shell or text editor. But as we'll see in this two-part series,not all terminals are created equal: they vary wildly in terms of functionality, size, andperformance.

Security updates have been issued by Debian (memcached, openssl, openssl1.0, php5, thunderbird, and xerces-c), Fedora (python-notebook, slf4j, and unboundid-ldapsdk), Mageia (kernel, libvirt, mailman, and net-snmp), openSUSE (aubio, cacti, cacti-spine, firefox, krb5, LibVNCServer, links, memcached, and tomcat), Slackware (ruby), SUSE (kernel and python-paramiko), and Ubuntu (intel-microcode).