//TODO(rose)

Final GSoC status report

2023-08-25T00:00:00+00:00

It's 5am and I have a headache. The perfect time for some reflection!

Not only that, but I've just had to play the part of Static Site Ungenerator, because I found out that I deleted the source of the last post and I didn't want to lose it in the upcoming publish. If your Atom feed went funky, sorry.

This document is my Final Work Submission, but is fun for all the family, including the ones who don't work at Google. Hi everyone!

#</a> What we wanted to happen</h2>
Going into the summer, the plan was to add functionality to wlroots</a> so that its users (generally Wayland compositors) could more easily switch to a smarter frame schedule. I've had many goes at explaining the problem and they all sucked, so here we go again: if a compositor puts some thought into when it starts its render, desktop latency as perceived by the user can decrease. The computer will feel snappier.
wlroots started the summer with no accommodations for compositors that wanted to put thought into when they start to render. It assumed exactly no thought was to be put in, and left you on your own if you were to decide otherwise. But that has all changed!
The aim of my work could have comprised three things, but I added a fourth and then didn't have time for the third:

measurement - a way to determine how long a render job took, from start (on the CPU) to finish (on the GPU).</li>
scheduling - the system that chooses when to tell a compositor that it should render. this wants a way for wlroots users to dictate when they want to start rendering a new frame, relative to when this frame is due to be displayed</li>
prediction - some clever maths that learns from the time took by previous renders and guesses how long the next one will take. this allows for moving the render start time closer to the frame deadline (good), carefully enough to avoid missing the deadline (very bad)</li>
bonus! tracing - measurement, but for humans. we're getting 60 new numbers every second, and it's going to be hard to make sense of them if they're just being printed out to the console</li> </ol> #</a> What happened</h2>
After some flailing around</a> trying to add a delay to the existing scheduling, I started writing patches worth landing.
First came the render timer API</a>. Now we can measure the duration of our render passes. This MR brought an abstraction for timers, and an implementation for wlroots' gles2 renderer.
Next, the scene timer API</a>. `wlr_scene</code> does some of its own work before setting off the render pass itself, so it needed to become aware of timers and expose a way to use them.`
`Meanwhile, I was having another stab atconfiguring a frame delay</a>. It wasn't very good, and the design of wlroots' scheduling and the complexity of the logic underneath it turned out to take a long time to get through. With this MR, though, I had a better idea of where I was trying to go. A long thought process followed, much of which lives in this issue</a>, and further down we'll see what came of that.`
Before working on a prediction algorithm, I wanted to be able to see live feedback on how render timings behaved and which frames were missed so that I could do a good (informed) job of predicting them. I took a detour into the world of tracing. libuserevents</code></a> was spawned and so was the work to make use of it in wlroots. Linux's user_events tracing interface was appealing because it meant that GPUVis, an existing tool that can display a timeline of CPU and GPU events, would be able to show wlroots' events. Unfortunately Linux and I have so far struggled to get along and this work is still in progress - no submission yet because it's broken. Even more unfortunately, this meant that I wasn't able to get around to prediction.
`Then I got tired of fighting that, and despite the words of discouragement...`
“it's kind of impossible and I don't think we can do much better than !4214”</h3> - a fool, a moron, a silly goose (me) </div> a refactor of wlroots' frame scheduling that allows us to do much better than !4214: !4307</a>! This hasn't quite made it past the finish line, but it's close; I can feel it in my frames. It (in my opinion) neatly extracts the hairy logic that lived in wlr_output</code> into a helper interface, allowing users to swap out which frame scheduler they use, or to forgo the helpers and roll their own without there being bits and pieces left over in the parts of wlroots that they do care about. This is the most exciting piece of the puzzle IMO; wlr_output</code> has grown to have its fingers in many pies, and this MR reduces that and leaves wlr_output</code> a little bit more friendly in a way that took a lot of brain cycles but turned out clean. This new interface doesn't come with a frame delay option for free, but an implementation of the interface that has this feature is underway: !4334</a>. It fits nicely! We hashed it out a little on IRC because the frame delay option is a surprisingly tricky constraint on the interface, but I think the conclusion is good. It was definitely a lot easier to write this with confidence after the scheduling redesign :) To make this scheduling design possible and clean, a couple</a> of little changes</a> were needed</a> in other areas, and thankfully the case for these changes was easy to make. They're helpful to me, but also make those parts of wlroots less surprising and/or broken. There was also a discussion</a> about the fate of wlr_output.events.needs_frame</code>, which is an extra complexity in wlroots' frame scheduling. It turned out that while removing it is possible, it wasn't necessary for the new scheduling system, so it continues in the background. #</a> Loose ends</h2> While libuserevents</code> is usable, the wlroots integration is not ready. There is sadly no "stock" plug-and-play prediction algorithm in wlroots. The new scheduling infrastructure has not landed but I'm sure it will Soon™. The implementation with the frame delay option will hopefully follow shortly after. When (touch wood) it does, compositors will have to bring their own prediction algorithm, but a "good enough" algorithm can be very simple and given the current interface design can easily be swapped out for a stock one if one materialises. And finally, the funniest one. I wrote an implementation of the timer API for wlroots' Vulkan renderer, and then put off submitting it for two months because everything else was more important. gles2 is the default renderer and supports roughly every GPU in existence. Writing the Vulkan timer was fun but landing it was less of a priority than every other task I had and nothing really depended on it, so it remains stuck on my laptop to this day. Perhaps I should get round to that. #</a> Closure</h2> The project didn't go how I expected it to - not even close. I even wrote up a schedule as part of my application that almost immediately turned out completely wrong. I'm not bothered, though, because it was fun, I made myself useful, and I met some cool people. If you're considering doing something like I did, I can happily recommend Simon</a> as a mentor, X.Org, and GSoC, in that order. Much love to Simon for making me feel comfortable when I really didn't know what I was doing, and for participating in my wildly off-topic free software rambles. I've only interacted with a small part of the X.Org community so far but it struck me from the start how welcoming everyone is; I have no doubts that the other X.Org project mentors are as lovely in their own ways. And of course, as a strong proponent of software that doesn't suck</del> that's free, I have to appreciate that GSoC gave me a welcoming place to do my part in that and relieve my worldly pressures (did you know you have to pay for internet??). Thanks everyone for putting up with me. If you would like to put up with me some more, click the links on the left - I'm not going anywhere, there's still work to do!

GSoC status report 4 2023-07-09T00:00:00+00:00 I'm suffering from having a mortal form again, but things are moving in the general direction of progress. Or "Rose, it's 2 in the morning!" Yeah yeah, whatever, you're not my mum. #</a> Imperfections</h2> Some would call this whining - skip this section if you're here for technology :) You're not supposed to make yourself work when you don't have energy to because you'll feel bad. People have tried telling me this and I've tried listening but to really take it on board I had to figure out what low energy actually feels like, so here we are, skipping a week of status reporting and holding a suspiciously high Factorio play time. I spent some of that play time making a cool blue circuit factory! Downtime is a good idea, hopefully - we'll find out next week whether it worked. It's surprising that one of the hardest problems given to me by the Fates has been fighting against myself, which sounds overly dramatic but in a literal sense is true. I would be moving faster if I felt up to it, but I don't feel up to it because I moved too fast recently. It's my fault because I wore myself out, but it's not my fault to rest when I need to, so instinctively I remain undecided on whether it's my fault. Sadly this isn't a balance that I've learned to strike, at least not for large scale work that I care about. Add this to a general guilt for doing less than others seem to be doing (a velocity- rather than the famous competence-based impostor syndrome) and the work that was once appealing becomes more distant. LoC metrics are a favourite of crap managers, quick glancers, and the part of my subconscious that judges my self worth. It's not ideal and it's even not-idealer when your work is mostly thinking and not actually that much coding - see the previous report</a> for a bunch of musings about what code should be written and not much written code. It's valid work! But the goblin in my skull disagrees. The mortal form disappoints me. I was hoping to discover my inner cold programming machine but I just found some boring human imperfections. Yawn! This isn't what I was expecting to write about but I think it's helping. I'm sure these aren't unique experiences but they worry me nonetheless, which is partially because I'm hardwired to be worrying about something most of the time. In a couple of days it will all be OK because I'll be able to play Counter-Strike again and that will for sure make my productivity go up, or down. The paradox of relaxing! #</a> The Happenings</h2> As predicted</a>, I have to face prediction. Before I do that, I want to get a feel for the behaviour of compositors' performance so I'm not mathsing in the dark, and my weapon of choice is Linux's tracing system which either is called ftrace or has a component called ftrace. I can't tell which. We've met Linux's tracing before. The screenshots</a> from GPUVis were made of data extracted from it, which makes it an attractive answer to the question "where do I put all my data". In theory, if wlroots gains the ability to output events to this system, GPUVis will automatically be able to display these events as it does all the others. The mechanism for userspace to emit events in this way landed in Linux 6.4 which was unleashed about 12 hours before I realised that my laptop's 6.3 series kernel didn't have support for it and nearly gave up. Until 6.4, the feature was gated behind CONFIG_BROKEN and looked truly like a lost cause. Thankfully Simon noticed that 6.4 held the answer to my problems and I found things to do while I waited for it to hit my distribution. Thrilling! We're back on track. To hide the horrors of a bare UAPI from wlroots, I wrote and published libuserevents</a>, which is my first C library and will make interacting with user_events amazing and great and you should definitely use it. There are whispers of integration into wlroots so far. I hope eventually I'll have a nice tool that can monitor a running compositor and show a graph of the frame times because that will at least be something pretty to look at to get away from thinking. In the background there's a scene timer</a> wriggling its way through review and the dreaded How To Schedule Frame Signals is looming over us all. I forgot to submit the Vulkan timer in all the ruckus. Oh well, apparently no one's supposed to be using the Vulkan backend yet anyway so I doubt there's anyone holding their breath. I've also just noticed that the second status report has links to git branches instead of commits, so they're likely very stale by now. Remind past me to not do that, that moron. Who knows what the future holds? Join us next week</del> time to find out. GSoC status report 3 2023-06-24T00:00:00+00:00 API design is kinda tricky. #</a> Those Who Came Before Me</h2> wlroots is designed to be very very modular. It's clear from the readme: “Pluggable, composable, unopinionated modules for building a Wayland compositor”</h3> - literally the first thing in the readme </div> “or any subset of these features you like, because all of them work independently of one another”</h3> - further down in the summary </div> This design goal seems to have come about from lessons learned in other projects where a more all-in-one approach has become a burden. Note that "pluggable" doesn't just mean you can plug it, it also means that you have to. In my opinion, that's why people shy away from this approach: if you want library users to be able to swap out parts of your code for theirs, you have to force them to be responsible for all the wiring even if they are using the default everything; otherwise they won't have access to the wiring for when they change their mind. This is all very commendable - engineering has happened, people learned, we are making better tradeoffs now than we were before. But what it means for me is that I have to also learn and do engineering. Imagine! More seriously, I want to respect the consensus that making library users do wiring is good, and they should be able to opt out of my code being called by not calling it. What I'm finding, though, is that the Thing I'm Trying To Do (scheduling) sounds like a feature but is more of an optimisation. Really, I'm making the frame signal firing be more interesting than it was before. Since the beginning it has worked in its way with a constant delay of zero, and now I am improving it to recognise that the delay might not be zero. I don't think it's possible to do this work in a neat and separate box, because it lies right in the middle of the frame firing path. There's a fair bit of complexity</a> in the way the signal is fired, so refactoring the whole firing path to separate it from what it touches (wlr_output</code> and the backends) is much harder than it might seem. #</a> The Efforts</h2> Top scientists (me) are working around the clock (or parts of it) to figure out how the scheduling can be decoupled such that it doesn't have to touch the backends at all. And by "working to figure out" I mostly mean "stewing on how it's kind of impossible and I don't think we can do much better than !4214</a>". We'll see, but at the moment it seems like I have to disregard all the pondering from the top of the page and stick my grubby fingers into all kinds of once-sacred functions. There are measurement woes, too: wlr_scene</code> is a whole thing, and it calls the renderer for you. It spends some time doing that and we need to know how long that time is, so we do need to add measurement code inside the function that does the thing (wlr_scene_output_build_state</code>). I've been doing a lot less pondering and a lot more giving up on this one, and my grand plan at the moment is to just throw a timer right in there</a> and let the user query it. This does make scene bigger, but I think it's alright to expect the code that does the rendering to also do the timing thereof. It will need to be aware of the timer to some extent, so we might as well put the whole thing in there and not bother trying to pretend they're separate. Aside from all that, I've implemented</a> a timer for the Vulkan backend, but I haven't submitted it yet. Not much more to say there. Soon, I will have to face prediction and think about statistics. Nooooooo! GSoC status report 2 2023-06-17T00:00:00+00:00 The world tried to stop me, but I pushed through and got some work done on wlroots. #</a> Hurdles</h2> Unfortunately aside from programming I have a physical form that I have to take care of. Recently this form's hands have been hurting because of strain from typing, which is pretty rude of them, so I took some time out to build a keyboard</a>. Woe is me, having to spend my time soldering and cutting. I had a great time. After finishing the keyboard, all of a sudden it was time to move out, so we packed and shuffled and drove and unpacked and complained about the trains in this country. Then I slept, and yesterday I finally found the time to get back to work. This is why there was no update last week. #</a> Progress</h2> I've decided that I need consistent terms for the subtasks I have. So: measurement is reading how long a frame takes to render, prediction is coming up with a number of milliseconds, and scheduling is delaying by that number and all the API funk therein. It turns out that Simon was right and there is a lot of API funk for me to deal with. We've had some discussions about what exactly wlr_output.events.frame</code> is for</a> and whether we really need it, and I've implemented a scheduling mechanism</a> based on that signal to show how we can make good use of it. Bringing together</a> this scheduling, the timer from the first status report for measurement, and some naive prediction (next frame takes at most 1ms more than last frame) christens tinywl as the first user of all this mess. Here are some screenshots from GPUVis</a> showing the old, boring timeline where we rendered immediately after the last present, and the new, sexy timeline where we render immediately before the next one! The purple lines are vblanks, and the boxes in each row represent durations when those processes were running on the CPU. I think blue lines are GPU submissions, but the naming is unclear. Wow, isn't that beautiful. GSoC status report 1 2023-06-03T00:00:00+00:00 The first week of GSoC is over! I'm working on presentation scheduling in wlroots. Here's how it's going. #</a> The Task</h2> Currently in wlroots, the easy and obvious frame schedule for compositors to implement is one where Wayland clients and the compositor are both told to render a new frame as soon as the last one makes it to the screen. This means that clients are almost guaranteed to miss the deadline for this cycle, and their frame submissions will only make it into the compositor's renderer in the next cycle. With this schedule, there are two frames of latency between a client receiving some user input and the new content making it to the screen. If the compositor starts rendering later then clients are more likely to have submitted the frame that was most recently triggered. The tradeoff is that the compositor is less likely to hit its own deadline. It's like the computer is playing The Price Is Right for every frame, and I'm in control of how it guesses. I'm trying to make a smarter frame schedule (where the computer is really good at the game) be another easy and obvious schedule for wlroots compositors to use, so the whole ecosystem can win without having to faff about implementing it themselves. Thanks to Simon</a> for the idea and for mentoring me. #</a> This week</h2> I quickly implemented a way for compositors to delay rendering</a> by a specified number of milliseconds, and Kenny quickly pointed out that the way I did it was silly. Since then, I have a much better understanding of what actually happens for each frame in wlroots. Thanks Kenny! After that review and a bit more discussion I gave it another go, but didn't submit it yet because I haven't finished polishing it. While I was doing that, I was brutally reminded that the C compiler does not care about me and will not attempt to help at all. Don't forget to enable your sanitizers, reader. Arguably I deserved it, because I had recently commented on how I was enjoying writing C for a change. Don't fall into the same trap as me; C does not want you to enjoy it. I also wrote a new API</a> that allows you to find out how long it took to render a frame. For now, this is only implemented on the GLES2 backend. Storing this time for the last few frames is a good way to make an educated guess on the time for the next one, and by also knowing your monitor's (maximum) refresh rate you can come up with a reasonable duration to delay rendering by. In testing the render timer I discovered that my test machine takes a whopping 4 milliseconds to paint the screen a solid colour. A valuable lesson lies here: don't ask questions if you're not strong enough to hear the answer. Or maybe "damage tracking is good". See you next week! GPUs, Part 3: An Example Program 2022-01-31T00:00:00+00:00 Now that we've seen some of the theory behind GPUs, it's time to see how it works in practice. How can a lowly mortal like me speak to a Titan (heh)? What do we need to know before going in? Comes with free wgpu tutorial. #</a> Shaders and the OpenGL model</h2> There are some differing opinions on what to call a program that runs on a GPU. It seems that people from graphics land call them shaders, because they are generally used to calculate the colour or light level of an object. There are a few types of shader and each has its own constraints, but the most generic one and the one that we are interested in is the compute shader. In compute land, they are called kernels, but there's already a thing called kernel and "shader" sounds cooler to me so we're going with that. Since there are so many types of GPU, programs tend to be stored in some intermediate format and compiled to target a certain architecture at runtime. This compilation is performed by the driver, which allows it to be device-specific. We can see this reflected in OpenGL, where the API ingests shaders written in GLSL. This means there is no ahead-of-time compilation step needed if you're happy to write your shaders in GLSL. Vulkan takes a slightly different approach by ingesting SPIR-V which is a binary format, one step below the human-friendly GLSL, but still above the hardware in terms of abstraction. #</a> gl_GlobalInvocationID</code></h3> Things get a little funky in this section, so hold on tight. Work done by compute shaders tends to be intuitively divisible into chunks which exist in some n-dimensional space. For example, a shader which modifies every pixel of an 800x600 image in some way can have its work divided into 800 * 600 = 480000 chunks, one for each pixel. These chunks can be named using the coordinates of the pixel they work on, and so the chunks exist in 2-dimensional space. In this case, the programmer can write a shader that operates on one pixel, and in OpenGL terms, use glDispatchCompute(800, 600, 1)</code> to run the compute shader. The arguments to this function specify how many instances of the shader will be run in each dimension of x, y, and z. Here, z is 1 because the image is 2-dimensional. This size is called gl_NumWorkGroups</code>. But wait, I hear you say, why is one unit of work called a work group? Enter the funk, stage left. Recall that gl_NumWorkGroups</code> tells us how many instances of the shader will be run. Well, the shader itself has a gl_WorkGroupSize</code>, which is another 3-dimensional size that represents how many times the code will be run, per instance of the shader. This means that your code runs inside a... 6-orthotope? 🤔 I can't think of a great example for this, but suppose the image processing shader actually operated on 4x4 groups of pixels. This is when you could use a 4x4x1 gl_WorkGroupSize</code> and a 200x150x1 gl_NumWorkGroups</code>. The general idea is that communication will be faster between threads within a work group than it will be across work groups. All of these gl_*</code> variables are accessible from within the shader, along with a couple that I haven't mentioned. gl_LocalInvocationID</code> is a 3D representation of which unit within a work group is currently being run, and gl_WorkGroupID</code> represents which work group within the entire dispatch is being run. The most important one is probably gl_GlobalInvocationID</code>, which is a 3D identifier that is unique across all invocations of your code. It is defined as gl_WorkGroupID * gl_WorkGroupSize + gl_LocalInvocationID</code>, which has the nice effect of meaning that you can use it as an index into some data. In the image example, the shader code could use gl_GlobalInvocationID</code> directly to access its respective pixel from the image. Nice! #</a> wgpu</h2> APIs for working with GPUs are scary, they have a lot of long words, and worst of all are usually designed for C programmers. wgpu is an exception to at least one of these rules, because it provides a relatively nice Rust interface while retaining design choices of things like Vulkan that allow you relatively close access to GPU (or driver) behaviour. It also supports multiple backends, so you can write code in terms of wgpu and run it on systems supporting Vulkan, Metal, and a couple of versions of DirectX, among others. Before anything can be done with wgpu, you must start with an Instance</code>. While creating this object, a backend is chosen. Then, this instance can be used to create an Adapter</code>, which is like a physical device. At this point, you can specify whether to use a low power adapter or a high performance one, and provide information about window systems if you intend to render to a window. The next link in the chain is called Device</code>, which is more like a session on a device than the device itself. Poor naming, in my opinion. Device</code> is part of a package deal with Queue</code> - the former allows control over resources and the latter allows control over actions. When shaders are running, they need to have access to some objects to work on. There are a few types of these, but the simplest is the buffer, which is a contiguous region in memory. Objects like these are the main way for GPUs to communicate with the rest of a computer, and they can be constructed using the methods on Device</code>. To start with, these methods can be used to create a BindGroupLayout</code> which is a template for a group of objects and their properties. For example, a BindGroupLayout</code> might describe two read-only buffers which are visible to compute shaders and are at least 4096 bytes in size. Then, a BindGroup</code> can be created which has this layout, by naming two buffers with those properties. To use a compute shader, you need a "compute pipeline", which is an object that holds the shader and some bind group layouts. Telling a GPU to run a shader (after you've done all the setup) is done using command buffers, which are built using command encoders. These commands have semantics like "run this compute pipeline using these bind groups", and such a command also contains a size in x, y, and z, which are equivalent to the arguments we saw being passed to glDispatchCompute</code> earlier. The command buffer can be submitted to the GPU via the Queue</code>. #</a> Like "Hello, World!" but worse</h2> The code for this section is available here</a>. If you clone the repository, note that this link points to a specific commit, and not the main branch. I won't write any of the CPU-side code in here, because it's basically a translation of the previous section into Rust. All you need to know is it runs the shader with a size of (64, 1, 1), passing it a buffer of 512 * 4 bytes, and finally prints the contents of the buffer. Now that we know how to run a compute shader, all that's left is knowing how to write one. For this, I use rust-gpu. Diving right in, here's an example shader: #![cfg_attr( target_arch = "spirv", feature(register_attr), register_attr(spirv), no_std )] extern crate spirv_std; use spirv_std::glam::UVec3; #[spirv(compute(threads(8, 1, 1)))] pub fn main_cs( #[spirv(global_invocation_id)] gid: UVec3, #[spirv(storage_buffer, descriptor_set = 0, binding = 0)] buffer: &mut [u32], ) { buffer[gid.x as usize] = 1; } </code></pre> At the beginning, we have some magical incantations probably copied from the rust-gpu repository. I think they allow the #[spirv]</code> attribute to work in the main_cs</code> function's arguments. Then, we import UVec3</code> from glam, a maths library which comes with spirv_std</code>. This type is effectively (u32, u32, u32)</code> and it is the type of the gid</code> argument, which is the same as gl_GlobalInvocationID</code>. On the main function there is an attribute telling the compiler that this is the entry point of a compute shader whose work group size is (8, 1, 1), and in the arguments we see the global ID and a buffer of u32</code>s, which is the first binding in descriptor set 0. I'm pretty sure "descriptor set" is the Vulkan term for bind group, which would make sense because it is possible to pass multiple bind groups to a shader. Overall, the shader sets 8 elements of the buffer to 1</code>, and the runner runs it 64 times. Only the x dimension is used because the buffer is 1-dimensional. By running the program, we can see that it does as expected and prints out a long list of ones (512 of them), all of which came from the GPU! #</a> Wow</h2> Phew! That was a lot of strange words. Arguably, we didn't achieve very much - all we did was set some bits to new values, but isn't that all that computers do anyway? I'd say that we did achieve a lot. We started out knowing a couple of concepts about predication or whatever and ended up running a whole entire program on a processor widely regarded as "for wizards" (not a real quote). Who's the wizard now? You are! If you got this far, I'm proud of you, and you should be proud of yourself. Go and make yourself a cup of tea or something as a reward. When you get back, there might even be some more wizardry to look at 😉 Previous</a> | Next </div> GPUs, Part 2: Architecture 2021-11-14T00:00:00+00:00 What kind of maths can we do on a GPU? How does it get done so quickly? What the hell is a "warp"? Find out all of this and more in today's episode. #</a> Parallel processing</h2> In CPU land, you can sometimes speed up data processing by splitting the data into parts, and processing multiple parts at the same time. There are two ways of doing this. #</a> Multiprocessing</h3> The most common way is multiprocessing, where there are multiple CPUs within a computer, and code can be run on all of them at the same time. In this case, it's as if there are multiple computers all working on a portion of the task, except the CPUs in a multiprocessing system share resources such as main memory to make communication between them much faster than if they were in separate computers. Writing programs to take advantage of such an architecture is fairly easy because once the task can be split into parts, the development process feels like writing separate programs to handle each part. Sometimes it's the same program that handles every part, in which case you only have to write one program, and life is good. #</a> SIMD</h3> On the other end of the spectrum, many modern CPUs support a method of processing called SIMD - Single Instruction (stream) Multiple Data (streams). In a CPU with SIMD, there is still only one control flow path. That is, when the CPU reaches some control flow in the program such as an if statement or loop, the CPU either is executing the body or it isn't. Hence, "single instruction stream". Although there can only be one instruction executing at a time, these instructions can perform the same operation on multiple sets of operands simultaneously. A processor that can operate on 8 sets of operands in this way is said to have 8 SIMD lanes. For example, the vmulps</code> instruction from x86 can be used to multiply 8 pairs of numbers, producing 8 results. Each pair is processed independently of the others, and they are processed in parallel. This style of computation is very well suited to a select few applications. For example, to change the volume of a buffer of audio samples, each sample must be multiplied by some scale factor. In this case, SIMD can be used without much thought, and if you're lucky your compiler will transform the naïve for loop implementation into machine code that makes use of SIMD instructions. This transformation is called auto-vectorisation, and it can be demonstrated with the following Rust function: pub fn scale(buffer: &mut [f32; 8]) { for value in buffer.iter_mut() { *value *= 4.0; } } </code></pre> Compiling with -Ctarget-cpu=haswell</code> to enable AVX2 gives the following assembly: <simd::scale>: c4 e2 7d 18 05 00 00 vbroadcastss 0x0(%rip),%ymm0 00 00 c5 fc 59 07 vmulps (%rdi),%ymm0,%ymm0 c5 fc 11 07 vmovups %ymm0,(%rdi) c5 f8 77 vzeroupper c3 ret </code></pre> Here we can see only one instruction actually doing any multiplication (vmulps</code>) because this instruction can operate on registers containing 8 values each. Since the function only operates on 8 values, there aren't even any branches in the body. However, auto-vectorisation is really hard, and you kind of have to cross your fingers when compiling if you want to take advantage of it. In GPU land, the processors themselves are structured a little differently, and this makes the parallelism easier for the compiler to take advantage of. It can also make programming these processors easier, as we'll see further down. #</a> How GPUs do it</h2> Now, we meet something called predication. I don't think many people use this term, and I had to reverse engineer it from some poorly written Wikipedia articles, but the concept was introduced to me fairly clearly in Alyssa Rosenzweig's talk The Occult and the Apple GPU. Thanks to Alyssa not only for the talk but also for responding to my email with a very detailed explanation of how GPU architects are constantly battling each other to redefine words in different ways. We'll meet some of these later. The idea is that SIMD is pretty cool, but since it doesn't allow for diverging control flow across lanes, it's not cool enough. Predication allows each lane in a SIMD-like processor to independently be enabled or disabled, and this can be done using an execution mask, which is a set of bits with one bit per lane; if a bit is zero, its corresponding lane does not write to any registers or memory, so it is effectively doing no work at all. Since we're in GPU land now, as promised, we have some new words to learn. The lanes in a GPU are more powerful than traditional SIMD lanes, so we call them "threads", to indicate that they're cooler and more flexible. The threads come in groups called "wavefronts" or "warps". All threads in a warp share a program counter, meaning threads in a warp cannot execute different instructions at the same time. The execution mask for a 32-thread warp can either be seen as 1 bit per thread, or a single 32-bit mask per warp, depending on which way you look at it. Here's a visual example of divergence in a program running on a hypothetical 4-thread warp. The lines down the side represent whether that thread is enabled at that point in the program. 0 1 2 3 | | | | let mut array = [0, 1, 2, 3]; | | | | | | | | if array[thread_id] % 2 == 0 { | | array[thread_id] += 1; | | } | | | | | | | | // at this point the array is [1, 1, 3, 3] </code></pre> Because there is only one program counter per warp, if the condition of an if statement evaluates to false in a given thread, that thread has to wait it out by staying disabled until the whole warp gets to the end of the statement. This means that if you have to use an if statement, it's best to keep the body short and/or to have a condition that is usually true, to maximise how many threads are enabled at once. #</a> But why?</h3> By adding predication to SIMD, we give ourselves the ability to write programs in a way that looks much more like the single-threaded scalar processing that we are all used to. The program from above, written using SIMD, might look like this: let mut array = u32x4::new(0, 1, 2, 3); let one = u32x4::splat(1); let mask = (array & one).eq(one); array = mask.select(array, array + 1); </code></pre> This differs from the GPU-style program above in one striking way: this program describes calculations in terms of vectors (u32x4</code>, i.e. a group of four integers) whereas the GPU-style code describes the calculations individually, achieving parallelism implicitly by using the thread_id</code> to index into the data, which splits the data into independent chunks. This means that the GPU-style code can be used on any size of array without modifying the code itself. Another important difference is that the GPU-style code looks to me to be far easier to write (and it was), partially due to this scalar/vector code style distinction. Furthermore, the resulting machine code of the SIMD program is probably quite inefficient due to having to save all of the results of array + 1</code> into registers even though it doesn't end up using them. So, SIMD is good for programs that need to do the same thing a bunch of times, but predicated SIMD is good for programs that need to do nearly the same thing a bunch of times, which is way more programs. #</a> Instruction sets</h2> At the beginning of this post, I asked "what kind of maths can we do on a GPU?". The answer will vary depending on the GPU, so I looked at the manual for AMD's GCN 3 architecture1</a>, because I have a GCN 4 card, and apparently they're similar enough that AMD didn't release a new manual. GCN 3 provides all the arithmetic stuff you'd expect; add, sub, mul, div on integers and single-precision floats are there along with some logical operators, but it also has loads of others. There are a few convenient instructions for graphics programming like trig functions, square root, reciprocal square root, and even cubemap helpers. As if we weren't spoiled already, there are some logs and exps as well. Skimming some documents for Apple's G132</a> and Arm's Bifrost3</a> instruction sets indicates that most of these instructions are fairly standard. #</a> Conclusion</h2> CPUs have some ways to achieve parallelism, but they are restricted by the way people expect to be able to write programs for them. GPUs had no such restriction, so their designs can lean into parallelism as much as they want. As we've seen, this leads to an unfamiliar but powerful architecture. In the next few posts, I'll explore writing and running programs on a GPU, and some real-world examples of the architectural concepts we've met so far. ^1https://developer.amd.com/wordpress/media/2013/12/AMD_GCN3_Instruction_Set_Architecture_rev1.1.pdf</a> </div> ^2https://dougallj.github.io/applegpu/docs.html</a> </div> ^3https://cgit.freedesktop.org/mesa/mesa/tree/src/panfrost/bifrost/ISA.xml?id=07a5ec83fb09de861d940fea69b49cefb08fda75</a> </div> Previous</a> | Next</a> </div> GPUs, Part 1: The Feynman Technique 2021-11-12T00:00:00+00:00 There's a lot of writing about the Feynman Technique, but essentially it boils down to the idea that if you want to learn something, you should try to explain it to a 12-year-old. I'm not a 12-year-old, but I am a first year compsci, which is basically the same thing. I have two problems with GPUs. Firstly, I don't understand them. Secondly, I don't know of any 12-year-old-level resources to learn with. Here, I'll try to kill two birds with one stone. In this series I'll explore GPUs as general-purpose compute engines, their characteristics, and how one can learn to harness them for good. During this journey I'll try to document what I find in a way that allows any curious computer scientist to follow along at home. We can worry about the 12-year-olds later. My favourite language is Rust and I don't really know how to write anything else so I'll be aiming to use only Rust, at least on the CPU side. If possible, I'll use rust-gpu</a> for GPU-side code as well. #</a> Why bother</h2> GPUs are becoming more relevant as new people come up with new work to do, and Moore's law fails to deliver on its promise of increasing single thread performance to do it with1</a>. GPUs sidestep this problem by adding more threads. Their main strength is in tasks where some operation is applied over many inputs independently (like with SIMD - more on that later), but with some clever tricks they can beat CPUs in tasks that at a glance don't seem parallel at all. It's hard to turn a traditional sequential program into one that plays nicely on a GPU, but depending on the workload, it can really pay off. This transformation is what I aim to get familiar with. “They do, like, images and shit, right?”</h3> - My friend Drew </div> Although they are big and scary and foreign, I think that with some time and some honest work even a first year like me could tame the beast. I want to, because they're really fast, and it would make me feel powerful and smart. Your motives may be different, and that's OK. To be clear, I'm not too bothered about the traditional 3D graphics pipeline with its ROPs and TMUs and geometry. Although it would be nice to understand that stuff, I'll be focusing more on general computation to start with. #</a> Goals</h2> My personal goal for this series is to be able to understand piet-gpu</a> from start to finish. piet-gpu is a 2D vector graphics renderer designed to leverage GPU compute to do what has historically been done on CPUs. Whether I'll be able to do this will probably become clear in a few months, but really it's not about the treasure, it's about the friends we made along the way. To get to piet-gpu, by my estimates, we'll need to understand a few things about GPUs: GPU architecture - how do they run programs?</li> CPU-side APIs - how to tell a GPU to run a program</li> I/O - moving data between GPU memory and main memory</li> Algorithmic techniques - taking tasks that look sequential and reframing them until they can be done efficiently in parallel</li> </ul> #</a> Conclusion</h2> This will be a fun exercise in self-motivation I guess. I've learned some cool stuff so far, so I'm optimistic about how far we'll get. The next post, a first look into GPU architecture, should come quite soon. I hope to see you there! ^1This isn't really what Moore's law is, but it's true that single thread performance isn't growing in the same way it used to. Here</a> is an article from 2012 showing the effect already being visible. </div> Previous | Next</a> </div> about me 2021-10-31T00:00:00+00:00 Hi! I'm Rose. I'm a third year CS student at the University of Bristol, and I am secretly building a weapon that convinces its targets to work on my projects for me. I spend a lot of time complaining, but don't let that fool you. A lot of stuff is actually quite cool, in my opinion. I have a few hobby projects: kerux</a>, a matrix server</li> patchy</a>, a bot for mailing lists that aims to make them more comfortable</li> waylix</a>, an experimental frontend for helix</a></li> kernel</a>, a kernel targeting aarch64 - pending name change :D</li> </ul>