When a kernel subsystem maintainer has a set of commits to send up thechain toward the mainline, the git request-pullcommand is usuallythe right tool for the job. But various maintainers have noticed over theyears that this command can sometimes generate confusing results whenconfronted with anything but the simplest of histories. A briefconversation on the linux-kernel mailing list delved into why thissituation comes about and what maintainers can do in response.
Security updates have been issued by Debian (aspell, graphite-web, imagemagick, mediawiki, milkytracker, nfs-utils, and openjdk-11), Fedora (kernel, kernel-headers, kernel-tools, mediawiki, and radare2), openSUSE (dhcp, libpcap, lighttpd, and tcpdump), Scientific Linux (java-1.8.0-openjdk), Slackware (python), SUSE (bluez, kernel, and python-xdg), and Ubuntu (aspell).
The 5.4-rc4 kernel prepatch is out fortesting. "This release cycle remains pretty normal. In fact, the rc's have beena bit on the smaller side of the average of the last few releases, andrc4 continues this, if only barely."
kmalloc() is a frequently used primitive for the allocation ofsmall objects in the kernel. During the 2019Linux Storage, Filesystem, and Memory Management Summit, VlastimilBabka led a session about the unexpectedalignment problems developers face when using this function. After a fewmonths he has come back with the secondversion of a patch set implementing a natural alignment guarantee forkmalloc(). From the strong opposition it facedinitially, it seemed that the change would not get accepted. However, itended up in Linus Torvalds's tree. Let's explore what happened.
After more than two years of development, the Linux trace toolkit next generation (LTTng)project has released version 2.11.0 of the kernel and user-space tracingtool. The release covers the LTTng tools, LTTng user-space tracer, andLTTng kernel modules. It includes a number of new features that aredescribed in the announcement including session rotation, dynamic user-space tracing,call-stack capturing for the kernel and user space, improved networkingperformance, NUMA awareness for user-space tracing buffer allocation, andmore. "The biggest feature of this release is the long-awaited sessionrotation support. Session rotations now allow you to rotate anongoing tracing session much in the same way as you would rotatelogs.The 'lttng rotate' command rotates the current trace chunk ofthe current tracing session. Once a rotation is completed, LTTng doesnot manage the trace chunk archive anymore: you can read it, modify it,move it, or remove it.Because a rotation causes the tracing session’s current sub-buffersto be flushed, trace chunk archives are never redundant, that is, theydo not overlap over time, unlike snapshots.Once a rotation is complete, offline analyses can be performed onthe resulting trace, much like in 'normal' mode. However, the bigadvantage is that this can be done without interrupting tracing, andwithout being limited to tools which implement the 'live' protocol."
Greg Kroah-Hartman has announced the release of the 5.3.7, 4.19.80, 4.14.150, 4.9.197, and 4.4.197 stable kernels. All five containimportant fixes throughout the kernel tree, as usual. Users of thoseseries should upgrade.
Security updates have been issued by Debian (poppler, sudo, and wordpress), Oracle (java-1.8.0-openjdk), Red Hat (java-1.8.0-openjdk), Scientific Linux (java-1.8.0-openjdk, java-11-openjdk, and kernel), and SUSE (kernel and postgresql10).
Ubuntu has announced the release of 19.10 "Eoan Ermine" in desktop and server editions as well as all of the different flavors: Ubuntu Budgie, Kubuntu, Lubuntu, Ubuntu Kylin, Ubuntu MATE,Ubuntu Studio, and Xubuntu. "The Ubuntu kernel has been updated to the 5.3 based Linux kernel, andour default toolchain has moved to gcc 9.2 with glibc 2.30. Additionally,the Raspberry Pi images now support the new Pi 4 as well as 2 and 3.Ubuntu Desktop 19.10 introduces GNOME 3.34 the fastest release yet withsignificant performance improvements delivering a more responsiveexperience. App organisation is easier with the ability to drag and dropicons into categorised folders and users can select light or dark Yarutheme variants. The Ubuntu Desktop installer also introduces installingto ZFS as a root filesystem as an experimental feature." More information can also be found in the release notes.
The final days of the 5.3 kernel development cycle included an extensive discussion of thegetrandom() API and the reversion of an ext4 improvement thatwas indirectly causing boot hangs due to a lack of entropy. Blockingfilesystem improvements because they are too effective is clearly not agood long-term development strategy for the kernel, so there was aconsensus that some sort of better solution had to be found. What was lackingwas an idea of what that solution should be. It is thus surprising thatthe problem appears to have been dealt with in 5.4 with little in the wayof dissent or disagreement.
Google has announcedversion 1.0 of its Bazel buildsystem. "A growing list of Bazel users attests to the widespread demand for scalable, reproducible, andmulti-lingual builds. Bazel helps Google be more open too: several largeGoogle open source projects, such as Angular and TensorFlow, useBazel. Users have reported 3x test time reductions and 10x faster buildspeeds after switching to Bazel."
Security updates have been issued by Arch Linux (sudo), Debian (libsdl1.2 and libsdl2), Mageia (e2fsprogs, kernel, libpcap and tcpdump, nmap, and sudo), openSUSE (GraphicsMagick and sudo), Oracle (java-1.8.0-openjdk, java-11-openjdk, jss, and kernel), Red Hat (java-1.8.0-openjdk and java-11-openjdk), Scientific Linux (jss), SUSE (gcc7 and libreoffice), and Ubuntu (leading to a double-free, libsdl1.2, and tiff).
When last we looked in on the progress ofthe WireGuard VPN tunnel toward themainline kernel, it seemed like the main sticking point had been overcome. The Zinc cryptography API used by WireGuard wasgenerally seen as a duplication of effort with the existing kernelcryptographic algorithms, so an effort to rework Zinc to use that existingcode seemed destined to route around that problem and bring WireGuard tothe mainline. In the six months since then, though, things have gonefairly quiet in WireGuard-land; that all changed based on a conversation atthe recent Kernel Recipesconference in Paris.
The problems with field-programmablegate arrays (FPGAs) is not exactly an obvious talk topic for agraphics-related conference like the 2019 X.Org Developers Conference (XDC). BenWidawsky acknowledged that, but said that he sees parallels in thesituation with FPGA support in the free-software world and the situation withgraphics hardware support in the past. It is his hope that the tools fordeveloping with FPGAs can make the same journey that graphics drivers havemade over the last two decades or so.
Security updates have been issued by Debian (apache2 and unbound), Fedora (opendmarc, runc, and sudo), openSUSE (epiphany, GraphicsMagick, and libopenmpt), Oracle (kernel and sudo), Red Hat (java-1.8.0-openjdk, jss, kernel, kernel-rt, and kpatch-patch), SUSE (crowbar-core, crowbar-openstack, grafana, novnc, openstack-keystone, openstack-neutron, openstack-neutron-lbaas, openstack-nova, openstack-tempest, python-pysaml2, python-urllib3, rubygem-chef, rubygem-easy_diff, sleshammer, libpcap, sudo, and tcpdump), and Ubuntu (aspell and libsdl1.2).
The pull request changing the name of Perl 6 to Raku has beenmerged. See thefull text for more information. "This document describes the stepsto be taken to effectuate a rename of 'Perl 6' to 'Raku', as described inissue #81. It does not pretend to be complete in scope or in time. Tochange a name of a project that has been running for 19+ years will taketime, a lot of effort and a lot of cooperation. It will affect people inforeseen and unforeseen ways." (Thanks to Sean Whitton)
Security updates have been issued by Debian (sudo and xtrlock), openSUSE (sudo), Red Hat (Single Sign-On), Slackware (sudo), SUSE (binutils, dhcp, ffmpeg, kernel, kubernetes-salt, sudo, and tcpdump), and Ubuntu (sudo).
The KDE project has announced therelease of version 5.17 of the Plasma desktop environment."Night Color, the color-grading system that relaxes your eyes whenthe sun sets, has landed for X11. Your Plasma desktop also recognizes whenyou are giving a presentation, and stops messages popping up in the middleof your slideshow. If you are using Wayland, Plasma now comes withfractional scaling, which means that you can adjust the size of all yourdesktop elements, windows, fonts and panels perfectly to your HiDPImonitor."
Version3.8.0 of the Python language has been released. New features includethe controversial assignment expressions,positional-only arguments,the Vectorcallmechanism, and more; see the what's new in Python3.8 document for more information.
Version7.2 of PyPy, an implementation of the Python language, is out. Withthis release, Python 3.6 support is deemed ready: "This releaseremoves the 'beta' tag from PyPy3.6. While there may still be some smallcorner-case incompatibilities (around the exact error messages inexceptions and the handling of faulty codec errorhandlers) we are happywith the quality of the 3.6 series and are looking forward to working on aPython 3.7 interpreter."
Race conditions can be some of the trickiest bugs to find. The resultingproblems can be subtle, and reproducing the problem in order to track itdown can be difficult or impossible; often code inserted to narrow down arace condition will cause it to stop manifesting entirely. A tool that canfind race conditions automatically would thus be a valuable thing for thekernel community to have. In late September, Marco Elver announceda tool called KCSAN (the Kernel Concurrency Sanitizer) that doesexactly that — and which has already found a number of real problems.
The 5.4-rc3 kernel prepatch is out fortesting. "Things continue to look fairly normal, with rc3 beinglarger than rc2, as people are starting to find more regressions, but 5.4so far remains on the smaller side of recent releases."
As notedearlier,when compiling Linux-kernel code that does a plain C-language load orstore, as in"a=b", the C standard grants the compiler the rightto assume that the affected variables are neither accessed nor modifiedby any other thread at the time of that load or store.The compiler is therefore permitted to carry out a surprisinglylarge number of optimizations, any number of which might ruin yourconcurrent code's day.Given that current compilers usually do not emit diagnostics warning ofpotential ruined days, it would be good to have other tools take on thistask.
Security updates have been issued by Debian (lucene-solr and ruby-openid), Fedora (krb5 and SDL2), openSUSE (kernel and libopenmpt), and Ubuntu (python2.7, python3.4).
The Google Open Source Blog has an announcement of the release of the SchedViz tool that is used internally at the company "to discover many opportunities for better scheduling choices and to root-cause many latency issues". SchedViz provides a GUI to explore kernel traces: "The SchedViz UI displays collections in several ways. A zoomable and pannable heatmap shows system cores on the y-axis, and the trace duration on the x-axis. Each core in the system has a swim-lane, and each swim-lane shows CPU utilization (when that CPU is being kept busy) and wait-queue depth (how many threads are waiting to run on that CPU.) The UI also includes a thread list that displays which threads were active in the heatmap, along with how long they ran, waited to run, and blocked on some event, and how many times they woke up or migrated between cores. Individual threads can be selected to show their behavior over time, or expanded to see their details."
It is no secret that much of the work on the in-kernel BPF virtual machine and associated user-space support code is being done at Facebook. But lessis known about how Facebook is actually using BPF. At Kernel Recipes 2019,BPF developer Alexei Starovoitov describeda bit of that work, though even he admitted that he didn't know what mostof the BPF programs running there were doing. He also summarized recentdevelopments with BPF and some near-future work.
Security updates have been issued by Debian (clamav, libtomcrypt, and rsyslog), Fedora (suricata), SUSE (libopenmpt and python-requests), and Ubuntu (libsoup2.4 and octavia).
The input stack for Linux is an essential part of interacting with oursystems, but it is also an area that is lacking in terms of developers.There has been progress over the last few years, however; Peter Huttererfrom Red Hat came to the 2019 X.Org Developers Conference to talk about some of the work that has been done. He gave a status report on the inputstack that covered development work that is going on now as well as thingsthat have been completed in the last two years or so. Overall, things arelooking pretty good for input on Linux, though the "bus factor" for thestack is alarmingly low.
Richard Stallman has issued a brief statement saying that there will not beany radical changes in the GNU Project's goals, principles andpolicies. "I would like to make incremental changes in how somedecisions are made, because I won't be here forever and we need to readyothers to make GNU Project decisions when I can no longer do so. But thesewon't lead to unbounded or radical changes."
Security updates have been issued by Fedora (chromium), openSUSE (rust and sqlite3), SUSE (dnsmasq, firefox, and kubernetes, patchinfo), and Ubuntu (python2.7, python3.5, python3.6, python3.7).
OpenSSH 8.1 is out. It includes some security fixes, including theencryption of keys at rest to defend them against speculative-executionattacks. There is also an experimental new signature and verificationmechanism for public keys.
Stable kernels 5.3.5, 5.2.20, 4.19.78, 4.14.148, 4.9.196, and 4.4.196 have been released. They all containthe usual set of important fixes. This is the last 5.2 kernel and usersshould move to the 5.3.y kernel series now.
Security updates have been issued by Debian (openjpeg2, openssh, and xen), openSUSE (dovecot23, jasper, libseccomp, lxc, putty, and singularity), Red Hat (bind, kernel, polkit, python, and wget), and Ubuntu (unbound).
One of the many changes in the 5.4 kernel is the completion (insofar asanything in the kernel is truly complete) of the pidfd API. Getting that work done has been "awild ride so far", according to its author Christian Brauner during asession at the 2019 Kernel Recipes conference. He went on to describethe history of this work and some lessons for others interested in addingmajor new APIs to the Linux kernel.
While Richard Stallman has resigned from the Free Software Foundation andMIT, he continues to hold onto his position as the head of the GNU project. Now, the FSF has announced that it is"working with GNU leadership on a shared understanding of the relationship for the future" and is seeking comments from thecommunity on what that should be.Meanwhile, a group of maintainers for specific GNU projects has posteda joint statement calling for new leadership at GNU. "We believethat Richard Stallman cannot represent all of GNU. We think it is now timefor GNU maintainers to collectively decide about the organization of theproject. The GNU Project we want to build is one that everyone can trust todefend their freedom."
The second 5.4 kernel prepatch is out fortesting. "So nothing looks particularly worrisome, but usually rc2is fairly calm and it takes a while for any regressions to benoticed." This release also changes the code name to "NestingOpossum".
The5.3.4,5.2.19,4.19.77,4.14.147,4.9.195, and4.4.195stable kernel updates have all been released; each contains a relativelylarge set of important fixes and updates.
Common Vulnerability and Exposure (CVE) numbers have been used for manyyears as a way of uniquely identifying software vulnerabilities. It hasbecome increasingly clear in recent years that there are problems with CVEnumbers, though, and increasing numbers ofvulnerabilities are not being assigned CVE numbers at all. At the 2019 Kernel Recipes event, GregKroah-Hartman delivered a "40-minute rant with an unsatisfactoryconclusion" on CVE numbers and how the situation might be improved. The conclusion may be"unsatisfactory", but it seems destined to stir up some discussionregardless.
Security updates have been issued by Arch Linux (exim, ruby, ruby-rdoc, ruby2.5, and systemd), Debian (openconnect), Mageia (thunderbird), openSUSE (lxc and mosquitto), Oracle (kernel and patch), Scientific Linux (patch), SUSE (firefox, java-1_7_0-ibm, and sqlite3), and Ubuntu (clamav).
Version 4.0 of theCalibre ebook management application is out. "It has been two years since calibre 3.0. This time has been spent mostly in making the calibre Content server ever more capable as well as migrating calibre itself from Qt WebKit to Qt WebEngine, because the former is no longer maintained.The Content server has gained the ability to Edit metadata, Add/removebooks and even Convert books to and from all the formats calibre itselfsupports. It is now a full fledged interface to your calibrelibraries."
The kernel's printk()function seems like it should be relatively simple; all it does is format astring and output it to the kernel logs. That simplicity hides a lot ofunderlying complexity, though, and that complexity is why kernel developersare still unhappy with printk() after 28 years. At the 2019 LinuxPlumbers Conference, John Ogness explainedwhere the complexity in printk() comes from and what is being doneto improve the situation.
Security updates have been issued by CentOS (kernel), Debian (jackson-databind, libapreq2, and subversion), Fedora (glpi, memcached, and zeromq), openSUSE (rust), Oracle (kernel), Red Hat (patch), and SUSE (dovecot23, git, jasper, libseccomp, and thunderbird).
Version 12 of the PostgreSQL database management system is out. "PostgreSQL 12 enhancements include notable improvements to queryperformance, particularly over larger data sets, and overall spaceutilization. This release provides application developers with newcapabilities such as SQL/JSON path expression support, optimizations forhow common table expression ('WITH') queries are executed, and generatedcolumns. The PostgreSQL community continues to support the extensibilityand robustness of PostgreSQL, with further additions tointernationalization, authentication, and providing easier ways toadministrate PostgreSQL. This release also introduces the pluggabletable storage interface, which allows developers to create their ownmethods for storing data."
The Document Foundation (TDF) isthe home of the LibreOfficefree-software office suite; it provides financial, governance, andother administrative services to LibreOffice. The foundation wasestablished in part to ensure that commercial entities did not have undueinfluence on the project, which limited the types of activities in which itcan engage. In particular, selling branded versions of LibreOffice in themacOS and Windows app stores has not been something that TDF could tackle.The TDFboard of directors is looking to change that with the creation of a new entity, The Document Collective (TDC), to engage in commercial activity thatis complementary to that of TDF members—hopefully as an income source tohelp support TDF.
LWN.net