LWN.net comments

Next steps for kernel workflow improvement

tdz — 2019-11-03T15:20:32+00:00

When requiring signed patches, what would you do if you loose your keys? Occasional patch submitters may have a problem proving their identity if they return with new keys.

I think that patchwork already is the answer. It just needs a lot more features and a better UI.

Fedora 31 looks awesome at first, but then

tome — 2019-11-03T14:48:26+00:00

> See: https://pagure.io/minimization/issue/13 and friends.

That link is worthwhile reading. Thanks for posting it.

Next steps for kernel workflow improvement

pabs — 2019-11-03T12:58:40+00:00

The ARC spec is supposed to fix the issues with forwarders:

http://arc-spec.org/
https://en.wikipedia.org/wiki/Authenticated_Received_Chain

Next steps for kernel workflow improvement

daniels — 2019-11-03T10:23:27+00:00

GitLab at least has a comprehensive API which can be used to pull the feed of recent events, create/modify/etc merge requests and comments on them, and so on, from the client of your choice. There are standalone CLI clients and rich bindings for whichever language you care to use it from. That's true of most web-based services created in the last 5-10 years.

Next steps for kernel workflow improvement

daniels — 2019-11-03T10:21:17+00:00

DKIM is a good idea, but mailing lists sure do break it pretty hard.

Python adopts a 12-month release cycle

Otus — 2019-11-03T09:08:47+00:00

> PEP 602 now says that deprecations will last two releases which is two years instead of the current 18 months

This is an improvement, but I'm a bit confused about how it's stated. PEP 602 says the policy is documented in PEP 387, but that one only requires one release with the deprecation warning.

The return of Python dictionary "addition"

mathstuf — 2019-11-02T21:02:59+00:00

How would it be a SyntaxError with this code? That being such a different exception type depending on whether literals were used or not…icky.

a = 'a'
{a: 1, a: 2, 'a': 3}

Python adopts a 12-month release cycle

burki99 — 2019-11-02T14:08:46+00:00

Thanks - this sounds very reasonable

Python adopts a 12-month release cycle

ermo — 2019-11-02T13:06:20+00:00

From PEP602: One year of full support, four more years of security fixes

Python adopts a 12-month release cycle

burki99 — 2019-11-02T12:23:37+00:00

Yearly releases with two years of support is basically what PHP does (https://www.php.net/supported-versions.php), but they add a third year of security fixes. For Python, the distributions probably will have to develop a long-term support scheme with security fixes since it seems a bit hard to imagine RHEL or Ubuntu LTS doing major version Python upgrades during the lifetime of their distros.

The return of Python dictionary "addition"

rav — 2019-11-02T09:50:17+00:00

> it's the same as writing out the KV pairs of a and b in order, or at least I hope it is

For the curious, dict literals indeed keep the last value if the same key is specified multiple times, at least according to my test on Python 3.7.4:

>>> {"a": 1, "a": 2}
{'a': 2}

I was actually surprised by this - I had guessed it would be a SyntaxError.

Next steps for kernel workflow improvement

jgg — 2019-11-01T23:35:43+00:00

It would be a neat lwn article to mine the mailing list for 'git penetration' data ie what % of patches use git 1.8, 2.23, etc

I wonder how practical an email impersonation attack is? Maybe we should start by strengthening DKIM checking in patchworks and related?

Rethinking the governance of the GNU Project

fuhchee — 2019-11-01T23:29:17+00:00

Surely an accusation as serious as "exclusion" is not something one should throw around as a mere rhetorical device.

Fedora 31 looks awesome at first, but then

pizza — 2019-11-01T23:28:19+00:00

My current laptop (Dell Latitude e7440) had Fedora 23 installed about four years ago, and my frankenstein workstation had Fedora 12 installed just under ten years ago. Both have been incrementally upgraded since then, and are now running Fedora 31.

(I might add that the only remaining "original" part of the workstation is a Model M keyboard..)

The server that still runs my online presence was first set up with Fedora 11 (over a decade ago), now running Fedora 29. It replaced two systems whose OS images dated back to Red Hat Linux 7.1 (mid 2000) and RHL4 (early 1997) respectively, incrementally upgraded through the end of the RHL series and the first five years of Fedora. Needless to say, there were lots of incremental hardware changes along the way. Yay for hand-me-down parts!

Next steps for kernel workflow improvement

estansvik — 2019-11-01T21:58:43+00:00

It could be kept for the giggles, but there's a minor typo: s/pubic-inbox/public-inbox/

Fedora 31 looks awesome at first, but then

JFlorian — 2019-11-01T20:20:04+00:00

This is one of the reasons I've fallen in love with the Intel NUCs for desktop use. I'd love top-of-the-line video, but since I don't play games, does it really matter? My biggest wow in video comes from "it just works" without any special tricks, even if the trick is as trivial as enabling rpm-fusion. Glad those resources exist, just prefer when I don't need them. The two NUCs I've bought so far have been stellar in simplicity of installing/maintaining/using Fedora.

Fedora 31 looks awesome at first, but then

JFlorian — 2019-11-01T20:13:14+00:00

> it's better lo leave alone and do a fresh install.

Gotta disagree there. I trust my working Fedora system to upgrade itself successfully far more than I do getting anywhere near the BIOS and other initial setup woes. I have a notebook that can't boot from USB so boot-media is a PITA there. I have had Dell premium workstations fight me to the bitter ends in getting a new install to succeed because multiple NVMe led to all sorts of quirkiness. The problem varies by era, but I want as little of the BIOS and other vendor-imposed gunk as possible. (My favorite hardware is virtual because it has been sanitized.) But ... once Fedora is installed, I'm good.

Obviously, this is all anecdotal. If I do run into problems, I'd much rather reach out to Fedora folks for help than most vendors. I get answers (and sometimes questions first) rather than excuses.

Next steps for kernel workflow improvement

logang — 2019-11-01T20:11:02+00:00

Yes, I really think any tooling improvements should just start and end with git. Yes, it will take a while for people to upgrade but changes to workflow are going to take a long time anyway and if there's a compelling reason to do so, then people will put the effort in.

My vague ideas for features in git would be:

* Support the entire flow for sending git patches inside git itself. This means branches need first class ways of storing cover letters, versions, recipient lists, etc. Instead of needing to do: format-patch, write cover letter, figure out send lists, notice a mistake, format patch, copy over cover-letter, send. It would nice if git just stored all this with the branch and all you needed to do was 'git send' when it's all ready.

* Support for easily importing patchsets from a mailbox into branches, with the cover letter and recipient lists. (Obviously this will need to solve the base-tree information problem first, possibly by including public repos that already have the base commits with the patches).

* Support for reviewing a patchset inside git itself and having the review comments sent via email to everyone on the recipient list and author, etc.

* Support for branch queues: if people are now importing tons of branches into their repos from their mailboxes, then they need some way of organizing these branches and determining which need attention next

* If the above features start being used by a majoriy, maybe then git could start to allow different transports other than email. So imagine a .git_maintainers file that contains a mapping of email addresses to desired transport. If the recipient's address isn't in this file, it simply falls back to email. A new transport might simply be that instead of emailing the patches they get pushed to a specified branch queue in a world-writable git repo. Sadly, this likely means that git will need to support some spam mitigations too.

* After that, interested parties could probably write a github-like web service that just provides a new front end for git's existing features. Then maintainers that want this could set it up for themselves, or kernel.org could offer this for maintainers that want it.

* And once there's a majority using this flow, adding structured data or tags from CI bots should be a bit easier because it's just a matter of changing the tooling everyone already uses.

Next steps for kernel workflow improvement

mathstuf — 2019-11-01T19:20:50+00:00

My main issue with the email workflow (which has its benefits) is that patches get lost in the firehose. There's no public triage of patches to know anything about them, updates to patchsets is usually done through completely new threads (making it hard to see what was already discussed), joining a mailing list leaves a chasm of "before I was subscribed" and "afterwards" for older information (I read my mail in mutt). If there was better metadata tracking and linking, I'd find it a lot easier.

Fedora 31 looks awesome at first, but then

rahulsundaram — 2019-11-01T18:10:04+00:00

FWIW, I think Thinkpad's are standard issue for Red Hat developers, so it wouldn't be a surprise if it very well supported in Fedora. I happen to use one as well and it has been pretty solid. There are some thermal management issues that Lenova has acknowledged (not distro specific) and working on with some firmware fixes but I have been getting a steady stream of firmware updates via LVFS/fwupd so it may already be fixed

Next steps for kernel workflow improvement

q_q_p_p — 2019-11-01T17:44:40+00:00

As someone who sends patches to kernel from time to time, I don't have much to complain about current workflow, yet I wish that any enhancement to development process would avoid proprietary solutions like GitHub, still work through SSH (it would be great if it was integrated with git, like git send-email) and preferably wasn't web based (dedicated client that does only what it is supposed to do, would be great).

KernelCI joins the Linux Foundation

khilman — 2019-11-01T17:39:11+00:00

The pine64 you donated has been active since you donated it: https://kernelci.org/boot/sun50i-a64-pine64-plus/

Thanks for the donation.

KernelCI joins the Linux Foundation

khilman — 2019-11-01T17:36:45+00:00

Raspberry Pi 4 is has been ordered, but not added yet.
For Qualcomm boards, we have 13 listed: https://kernelci.org/soc/qcom/
The 96boards are there as well. The one based on the 410c is called apq8016-sbc

Next steps for kernel workflow improvement

jake — 2019-11-01T17:16:54+00:00

> Did you mean to post the entire article to the front page?

No, I'm pretty sure that was not what we meant to do :)

Fixed now, thanks!

jake

Next steps for kernel workflow improvement

tshow — 2019-11-01T17:06:09+00:00

Did you mean to post the entire article to the front page?

Fedora 31 looks awesome at first, but then

kpfleming — 2019-11-01T16:17:23+00:00

Lenovo Thinkpad T480s, with the Intel GPU (in the CPU), not a dedicated GPU. Equivalent experience with the X1 Carbon (6th generation), also chosen to have nearly all-Intel components.

They're also well supported by the Linux Firmware Service, so we get BIOS and other firmware updates in Fedora as well!

Fedora 31 looks awesome at first, but then

sdalley — 2019-11-01T16:13:21+00:00

Ooh, interesting. What laptop brand/model was that?

Fedora 31 is here

rahulsundaram — 2019-11-01T15:13:19+00:00

> And then, I'd guess, use Fedora's build infrastructure to build their Spins, use Fedora's storage to host them, use Fedora's bandwidth to serve them, etc.

Some of it is Red Hat willing to fund it but a lot of storage, bandwidth etc is sponsored by other vendors or volunteer mirrors

> (And all this ignores hard to quantify stuff like brand-dilution.)

This is part of why there is editions, spins and labs that denote different things with varying levels of support. If some group wants to do a LXDE variant and another group wants to do a LXQT variant etc the project leadership can either choose to tell one group no, you can't do that or let the variants exist and let people who want to use them, form a community around it. Is it somewhat wasteful of resources? Sure but it is the price the project is willing to pay to help foster a community of volunteers. If it a project that was run entirely by paid staff, the management would be quite different

Fedora 31 is here

pebolle — 2019-11-01T14:12:20+00:00

> there are a few folks who care enough to volunteer their time and energy to make them happen.

And then, I'd guess, use Fedora's build infrastructure to build their Spins, use Fedora's storage to host them, use Fedora's bandwidth to serve them, etc.

And some community manager evaluates their status, writes reports for the various boards that govern Fedora. Those boards discuss these Spins and make decisions. Someone has to mention the Spins while explaining to Red Hat where all the money was spent on, etc.

And I wouldn't be surprised if having Spins complicates the release of the non-Spin products in various, small ways.

It might not be a large amount of resources but I can't see how it could be zero.

(And all this ignores hard to quantify stuff like brand-dilution.)

Fedora 31 looks awesome at first, but then

kpfleming — 2019-11-01T13:25:35+00:00

I chose my laptop specifically because it wouldn't require any proprietary drivers, put Fedora 28 on it, and it's been rock solid. I've now upgraded 28->29, 29->30, and 30->31, with no issues at all. Very very happy Fedora user!

The return of Python dictionary "addition"

sytoka — 2019-11-01T12:51:50+00:00

A few more years and Python will finally be of the same quality as Perl and Raku ;-)

Rethinking the governance of the GNU Project

Kluge — 2019-11-01T12:44:07+00:00

I read that article, and I think their behavior was justifiable in that case. I understand why some people feel differently because they don't like copyright assignment, but I think its an area where reasonable people can disagree.

In the case of the glibc joke, RMS is being a jerk.

Fedora 31 is here

pizza — 2019-11-01T12:35:44+00:00

> I think I understand some of the forces at play, but still: why should Fedora put resources into both LXDE and LXQT and in two different ways to keep Gnome 2 alive? Or in KDE, for that matter. A case of well meaning people, making reasonable decisions ending up with below-par results?

Because it's not "Fedora" putting in resources -- Instead, for each of those independent spins, there are a few folks who care enough to volunteer their time and energy to make them happen.

Fedora 31 is here

pebolle — 2019-11-01T10:05:06+00:00

A few things prompted me to post my sarcastic comment, as far as I can tell.

Firstly, it's a dig at a comment that's mainly a list of obscure names and obscure jargon. Obscure for anyone that's not an lwn regular, at the least.

Then it amazed me I sort of knew what most of the names and jargon actually refers to. And, on the other hand, that almost two decades into Fedora and its predecessors, there are still things I'm only vaguely familiar with at best: LXDE, LXQT, Spins, Labs, rpm-ostree. Yes, these two feelings are a bit at odds.

And lastly it showed me once again the uncanny ability of the free software community to fracture itself in ever smaller niches. (What would be a nice alternative for that overstretched word "community"?) The LXDE and LXQT split seems a nice example. It makes us look fringe. I guess we actually are fringe...

I think I understand some of the forces at play, but still: why should Fedora put resources into both LXDE and LXQT and in two different ways to keep Gnome 2 alive? Or in KDE, for that matter. A case of well meaning people, making reasonable decisions ending up with below-par results?

BPF and the realtime patch set

Cyberax — 2019-11-01T05:08:30+00:00

I seriously doubt that any RT system would need BPF-based packet filtering. Tracing is probably more realistic use-case, but probably not really that urgent.

Please remember that the kernel is for running userspace!

ringerc — 2019-11-01T04:37:06+00:00

Thanks for all the help! I like how sytemtap works - and the excellent, comprehensive documentation.

So I'm hoping it'll be possible to transition it over to BPF in a useful way rather than the community doing the "out with the old, in with the new" thing and jumping on bpftrace/bcc/whatever.

Please remember that the kernel is for running userspace!

ringerc — 2019-11-01T04:35:25+00:00

Not much yet - I studied systemtap first, because I wanted to start with something fairly mature and that will be available on customer systems in the forseeable future.

I'm hoping to find time to look at eBPF-tools, bcc and bpftrace soon though.

Quotes of the week

flussence — 2019-11-01T04:24:02+00:00

There's probably a few billion devices running MontaVista linux and the like (and still online, unfortunately)...

Fedora 31 is here

flussence — 2019-11-01T04:17:58+00:00

Jokes and criticism are allowed in the line. Slashdot-level trolling and snark - the door's over yonder.

Rethinking the governance of the GNU Project

flussence — 2019-11-01T00:57:53+00:00

Will you remember who commanded the biggest cult of personality, or remember who actually showed up, did the work, wrote the software? What really matters to you here? The Free Software, or the spectacle?

Rethinking the governance of the GNU Project

anselm — 2019-11-01T00:28:38+00:00

So the FSF is officially in charge of the GNU project, but RMS claims he is still in charge of the GNU project, even though he is no longer the head of the FSF. Let me get the popcorn.

Rethinking the governance of the GNU Project

atai — 2019-10-31T23:25:11+00:00

If the FSF is now flexible enough, GNUtls can return to the GNU Project or the FSF as RMS is not now involved with the FSF

Rethinking the governance of the GNU Project

atai — 2019-10-31T23:21:30+00:00

This is important because RMS no longer heads the FSF. People in disagreement with RMS can continue to work with the FSF, as RMS is not involved.

BPF and the realtime patch set

naptastic — 2019-10-31T23:19:55+00:00

I've been watching the -rt tree try to get merged since it started, and with how much of it has been merged, I'm frankly baffled that something that can disable interrupts and preemption for an unpredictable period of time snuck its way into the kernel.

With -rt merged, there will be four selections for preemption model. What would make sense, IMO, is for BPF's behavior to change depending on that setting. For PREEMPT_NONE || PREEMPT_VOLUNTARY, the current behavior seems obviously correct to me. For the other settings, though, the user is saying "I'm willing to sacrifice bandwidth to get bounded latencies" and the whole kernel, including BPF, should respect that.

Could BPF do the same thing interrupts did, with a top half and a bottom half, and only the top half runs in atomic context? Could it be done in such a way that performance for BPF isn't adversely affected if preemption is voluntary or off?

BPF and the realtime patch set

togga — 2019-10-31T22:09:38+00:00

Well, how much you dislike BPF, you are confirming that today, there are no better alternative for both packet filtering, LSM and tracing.

The intersection between LSM or tracing with RT patch is likely not an empty set. If RT is driven by latency requirements, i'd also say that fast high level packet routing is interesting in this area whenever the network is involved in the latency (remote interaction).

Fedora 31 looks awesome at first, but then

rgmoore — 2019-10-31T21:10:14+00:00

My experience is that the new approach to updates works very well. It gives the benefits of updating- especially that you don't need to re-do all the customization you've made to your system- and the benefits of booting into a minimal setup image so you don't have to worry about the problems of trying to update running programs. For quite a while now, I have only done a reinstall when I've upgraded my system disk. It isn't necessary or desirable otherwise.

Fedora 31 is here

rahulsundaram — 2019-10-31T18:57:06+00:00

Thank you. Appreciate the explanation

Fedora 31 looks awesome at first, but then

rahulsundaram — 2019-10-31T18:56:26+00:00

> Word of advice: you're using Podman and DockerHub, you'll need to switch the type from OCI images (the default) to Docker ones when building them, otherwise DockerHub will silently ignore them and there's no warning or log message.

You could switch to using quay.io (which has public repositories for free although that isn't immediately obvious in the homepage) instead of DockerHub as well

Unifying kernel tracing

fuhchee — 2019-10-31T18:53:34+00:00

Thanks for that blast from the past, Karim.

stapbpf

smakarov — 2019-10-31T18:17:31+00:00

I've been involved with the work to push stapbpf closer to a full-featured tracer. Complete feature parity with SystemTap's LKM backend will not be achievable, because BPF (even with recent enhancements) has inherent limitations. That said, there are important stapbpf features still in the pipeline:

https://sourceware.org/bugzilla/buglist.cgi?component=bpf...

At the moment, if you want to understand what BPF can and can't do, bcc/bpftrace versus SystemTap LKM is probably a more fair comparison.

user_string_n() would have been implementable rather quickly, if all we cared about is x86_64:

function user_string_n (addr:long, n:long) {
// In BPF the same bpf_probe_read() functions that read data from a kernel
// address are also used by bcc tools to read from user space addresses.
// It Happened To Work On x86_64, So Why Not^tm?
return kernel_string_n (addr, n) // <- calls bpf_probe_read_str()
}

I missed the availability of this misfeature my first time going through the BPF helper documentation, which says "this function reads data from an address" without specifying whether it's a kernel or user address. Looking at the implementation I noted it calls a function that takes kernel addresses. "Seems like BPF doesn't support reading from userspace yet," thought I, and left it at that.

The issue caught the attention of Torvalds in February:

https://lore.kernel.org/lkml/CAHk-=wjJqVXydQf_dprBmr=zXsy...

Now in October it's being fixed:

https://lore.kernel.org/bpf/637eba87807516061f1fee9353605...

Then existing BPF tools relying on the misfeature will be fixed, e.g.:

https://github.com/iovisor/bpftrace/issues/614

When the bpf_probe_read_{user,kernel} patch makes it into a release, stapbpf can add userspace tapset functions that won't return the wrong data when there's a discrepancy between user and kernel addressing.