airlied

Migrating to blogsport

2017-07-10T06:36:20Z

Due to lots of people telling me LJ is bad, mm'kay, I've migrated to blogspot.

New blog is/will be here: https://airlied.blogspot.com

how much of the conformance test suite does radv pass now?

2017-05-04T04:22:48Z

Test run totals:
Passed: 109293/150992 (72.4%)
Failed: 0/150992 (0.0%)
Not supported: 41697/150992 (27.6%)
Warnings: 2/150992 (0.0%)

This is effectively a pass. The Not Supported stuff isn't missing features as uneducated people are quick to spout, it's more stuff the hardware doesn't support or is pointless to expose on the hardware. (lots of image formats).

This is the results from the Vulkan CTS 1.0.2 branch, against mesa master with one patch (a workaround for some InternalErrors that CTS throws up).

Do not call the driver conformant as that is against the Khronos rules as we haven't paid or filed for approval, but the driver does now effectively pass the latest conformance test suite. I'll update on things if that changes.

Thanks again to everyone involved.

how close to conformant is radv?

2017-03-20T07:26:02Z

I spent some time staring into the results of the VK-GL-CTS test suite on radv, which contains the Vulkan 1.0 conformance tests.

In order to be conformant you have to pass all the tests on the mustpass list for the Vulkan version you want to conform to, from the branch of the test suite for that version.

The latest CTS tests for 1.0 is the vulkan-cts-1.0.2 branch, and the mustpass list is in external/vulkancts/mustpass/1.0.2/vk-default.txt

Using some WIP radv patches in my github radv-wip-conform branch and the 1.0.2 test suite, today's results are on my Tonga GPU:

Test run totals:
Passed: 82551/150950 (54.7%)
Failed: 0/150950 (0.0%)
Not supported: 68397/150950 (45.3%)
Warnings: 2/150950 (0.0%)

That is pretty conformant (in fact it would pass as-is). However I need to clean up the patches in the branch and maybe figure out how to do some bits properly without hacks (particularly some semaphore wait tweaks), but that is most of the work done.

Thanks again to Bas and all other radv contributors.

radv + steamvr

2017-02-27T19:42:01Z

If anyone wants to run SteamVR on top of radv, the code is all public now.

https://github.com/airlied/mesa/tree/radv-wip-steamvr

The external memory code will be going upstream to master once I clean it up a bit, the semaphore hack is waiting on kernel
changes, and the NIR shader hack is waiting on a new SteamVR build that removes the bad use of SPIR-V.

I've run Serious SAM TFE in VR mode on this branch.

radv and doom - kinda

2016-12-23T07:26:51Z

Yesterday Valve gave me a copy of DOOM for Christmas (not really for Christmas), and I got the wine bits in place from Fedora, then I spent today trying to get DOOM to render on radv.

Thanks to ParkerR on #radeon for taking the picture from his machine, I'm too lazy.

So it runs kinda, it hangs the GPU a fair bit, it misrenders some colors in some scenes, but you can see most of it. I'm not sure if I'll get back to this before next year (I'll try), but I'm pretty happy to have gotten it this far in a day, though I'm sure the next few things will me much more difficult to debug.

The branch is here:
https://github.com/airlied/mesa/commits/radv-wip-doom-wine

radv: status update or is Talos Principle rendering yet?

2016-09-27T04:33:56Z

The answer is YES!!

I fixed the last bug with instance rendering and Talos renders great on radv now.

Also with the semi-interesting branch vkQuake also renders, there are some upstream bugs that needs fixing in spirv/nir that I'm awaiting and upstream resolution on, but I've included some prelim fixes in semi-interesting for now, that'll go away when upstream fixes are decided on.

Here's a screenshot:

radv: status update or is dota2 working yet?

2016-08-26T03:05:34Z

Clickbait titles for the win!

First up, massive thanks to my major co-conspirator on radv, Bas Nieuwenhuizen, for putting in so much effort on getting radv going.

So where are we at?

Well this morning I finally found the last bug that was causing missing rendering on Dota 2. We were missing support for a compressed texture format that dota2 used. So currently dota 2 renders, I've no great performance comparison to post yet because my CPU is 5 years old, and can barely get close to 30fps with GL or Vulkan. I think we know of a couple of places that could be bottlenecking us on the CPU side. The radv driver is currently missing hyper-z (90% done), fast color clears and DCC, which are all GPU side speedups in theory. Also running the phoronix-test-suite dota2 tests works sometimes, hangs in a thread lock sometimes, or crashes sometimes. I think we have some memory corruption somewhere that it collides with.

Other status bits: the Vulkan CTS test suite contains 114598 tests, a piglit run a few hours before I fixed dota2 was at:
[114598/114598] skip: 50388, pass: 62932, fail: 1193, timeout: 2, crash: 83 - |/-\

So that isn't too bad a showing, we know some missing features are accounting for some of fails. A lot of the crashes are an assert in CTS hitting, that I don't think is a real problem.

We render most of the Sascha Willems demos fine.

I've tested the Talos Principle as well, the texture fix renders a lot more stuff on the screen, but we are still seeing large chunks of blackness where I think there should be trees in-game, the menus etc all seem to load fine.

All this work is on the semi-interesting branch of
https://github.com/airlied/mesa

It only has been tested on VI AMD GPUs, Polaris worked previously but something derailed it, but we should fix it once we get the finished bisect. CIK GPUs kinda work with the amdgpu kernel driver loaded. SI GPUs are nowhere yet.

Here's a screenshot:

radv: initial hacking on a vulkan driver for AMD VI GPUs

2016-07-19T19:59:41Z

(email sent to mesa-devel list).

I was waiting for an open source driver to appear when I realised I should really just write one myself, some talking with Bas later, and we decided to see where we could get.

This is the point at which we were willing to show it to others, it's not really a vulkan driver yet, so far it's a vulkan triangle demos driver.

It renders the tri and cube demos from the vulkan loader,
and the triangle demo from Sascha Willems demos
and the Vulkan CTS smoke tests (all 4 of them one of which draws a triangle).

There is a lot of work to do, and it's at the stage where we are seeing if anyone else wants to join in at the start, before we make too many serious design decisions or take a path we really don't want to.

So far it's only been run on Tonga and Fiji chips I think, we are hoping to support radeon kernel driver for SI/CIK at some point, but I think we need to get things a bit further on VI chips first.

The code is currently here:
https://github.com/airlied/mesa/tree/semi-interesting

There is a not-interesting branch which contains all the pre-history which might be useful for someone else bringing up a vulkan driver on other hardware.

The code is pretty much based on the Intel anv driver, with the winsys ported from gallium driver,
and most of the state setup from there. Bas wrote the code to connect NIR<->LLVM IR so we could reuse it in the future for SPIR-V in GL if required. It also copies AMD addrlib over, (this should be shared).

Also we don't do SPIR-V->LLVM direct. We use NIR as it has the best chance for inter shader stage optimisations (vertex/fragment combined) which neither SPIR-V or LLVM handles for us, (nir doesn't do it yet but it can).

If you want to submit bug reports, they will only be taken seriously if accompanied by working patches at this stage, and we've no plans to merge to master yet, but open to discussion on when we could do that and what would be required.

virglrenderer project gets mailing-list and hosting

2016-02-15T02:51:26Z

Just FYI:

There is now a mailing list for virglrenderer library hosted at freedesktop. It is to be used for development discussion of the virgl->GL renderer library and for patches to it.

https://lists.freedesktop.org/mailman/listinfo/virglrenderer-devel

The git tree is also now hosted at:
git://anongit.freedesktop.org/git/virglrenderer
https://cgit.freedesktop.org/virglrenderer

My personal repo will only be used for my own development stuff.

I'll also get a freedesktop patchwork instance set up for this asap.

I'm also contemplating bugzilla vs phabricator.

Dave.

virgil3d local rendering test harness

2015-03-16T06:24:12Z

So I've still been working on the virgil3d project along with part time help from Marc-Andre and Gerd at Red Hat, and we've been making steady progress. This post is about a test harness I just finished developing for adding and debugging GL features.

So one of the more annoying issuess with working on virgil has been that while working on adding 3D renderer features or trying to track down a piglit failure, you generally have to run a full VM to do so. This adds a long round trip in your test/development cycle.

I'd always had the idea to do some sort of local system renderer, but there are some issues with calling GL from inside a GL driver. So my plan was to have a renderer process which loads the renderer library that qemu loads, and a mesa driver that hooks into the software rasterizer interfaces. So instead of running llvmpipe or softpipe I have a virpipe gallium wrapper, that wraps my virgl driver and the sw state tracker via a new vtest winsys layer for virgl.

So the virgl pipe driver sits on top of the new winsys layer, and the new winsys instead of using the Linux kernel DRM apis just passes the commands over a UNIX socket to a remote server process.

The remote server process then uses EGL and the renderer library, forks a new copy for each incoming connection and dies off when the rendering is done.

The final rendered result has to be read back over the socket, and then the sw winsys is used to putimage the rendering onto the screen.

So this system is probably going to be slower in raw speed terms, but for developing features or debugging fails it should provide an easier route without the overheads of the qemu process. I was pleasantly surprised it only took two days to pull most of this test harness together which was neat, I'd planned much longer for it!

The code lives in two halves.
http://cgit.freedesktop.org/~airlied/virglrenderer
http://cgit.freedesktop.org/~airlied/mesa virgl-mesa-driver

[updated: pushed into the main branches]

Also the virglrenderer repo is standalone now, it also has a bunch of unit tests in it that are run using valgrind also, in an attempt to lock down some more corners of the API and test for possible ways to escape the host.

is this a protocol? displaylink3

2014-11-29T05:42:13Z

I'm not sure

but if hd0;u]; means anything to anyone from displaylink, or is the first unencrypted bytes they send, then oops.

Looks like I have some work to do next week.

more on Displaylink3 and HDCP encryption

2014-11-07T04:20:27Z

okay another braindump (still nothing working).

The git repo mentioned in previous post has all the code I've hacked up so far.

I finished writing the HDCP protocol stages, and sending all the msgs and getting replies from the device.

So I've successfully reached a point where I've negotiated a HDCP session key with the device, and we are both happy about it. Unfortunately I've no idea what I'm meant to be encrypting to send to the device. The next packet the USB traces contain is 384-bytes of encrypted data.

Now HDCP v2 had a vulnerabilty in its key neg, and I've written code to try and use this fact. So I've taken a trace I made from Windows, and extracted the necessary bits, and using that I've managed to derive the master key used in that trace, and subsequently managed to derived the session key for it. So I've replayed the first encrypted packet from the trace to the device and got an encrypted response the same as in the trace.

I've tried changing a bit in the session key, riv value and data I'm sending, and doing that causes the device not to reply with the answer. This to me implies that the device is using the HDCP cipher to encode the control channel. Now HDCP does say you should only do this for video streams, but maybe DisplayLink forgot to read that bit.

Now where does this leave me, in theory I should be able to replay the full trace (haven't had time yet) and I should see the same picture on screen as I did (though I can't remember what monitor/device I used, so I might have to retrace and restage my tests before then).

However I really need to decrypt the encrypted data in the trace, and from reading the HDCP spec the only values I need to feed the AES engine are ks ^ lc128, riv, streamctr, inputctr. I'm assuming streamctr and inputctr are 0 for the first packet (I could be wrong, maybe they use some wacky streamctr to avoid messing with hdcp), riv and ks I've captured. So lc128 is possibly the crux.

Now what is lc128? Its a secret 128-bit value in the HDCP world given only to HDCP adopters. Its normally something you'd store in hw on the GPU etc as an input to the hw cipher. But in displaylink there is no GPU encrypting the data. Now its possible that displaylink don't use the same lc128 as the HDCP people, unlikely but possible. Maybe they cipher their streams with their own lc128, and only use the offical hdcp lc128 for actual HDCP streams.

I don't think lc128 has leaked, I'm not sure what the consequences of it leaking would be, but hey its just a magic number, and if displaylink are using as an input to their AES code, it must be in RAM at some point, now I need to figure out ways to work that out. I'm not sure how long it would take to brute force as 128-bit key space, probably impossible.

At any point if someone from DisplayLink wants to talk, you know where to find me :-)

a day with DisplayLink USB3 and HDCP

2014-10-31T06:25:25Z

So for some reason I decided to look at the displaylink usb3 adaptors today. (no good news).

This blog post is so I don't forget all of this when I page it out. Notes, HDCP1.0 being broken doesn't matter to this, maybe HDCPv2.0 being a bit broken could be used, but I'm not sure how!

The displaylink USB3 protocol is based on HDCP protocol. I've traced the first few packets and it clearly
looks like the host sends two packets

AKE_Init,
AKE_Transmitter_Info

and the device sends back
AKE_Send_Cert

at least.

AKE_Send_Cert contains a 522 byte certificate, containing a receiver id, public key, some misc bytes and a signature generated with the DCP LLC private key, that you have to verify.

so the HDCP v2.2 spec contains the DP LLC public key, and I've written some code to verify the spec using openssl, but it totally fails to work. This is probably due to me doing something stupid, or not understanding what I'm doing, if you are openssl knowledgeable and want to look, the hack fest is
http://cgit.freedesktop.org/~airlied/dl3dev/

It might be the DisplayLink devices use a different signing key than the DP LLC one.

That repo contains some code to talk to the device (currently disabled) and do the initial sequence, along with an attempt to verify the cert.

Now once I get past this hurdle, the larger one seems to remain, the HDCP 2.0 spec has a global secret 128-bit value called LC128, that everyone who implements HDCP gets and hides somewhere. Its probably sitting in the displaylink driver in hex, but I'd hope they at least hide it better than that. It may also be possibly supplied by the OS, Windows or OSX. (I've no clue yet). That value is used in the key negotiation.

Now it might be possible that Displaylink allow non-HDCP encrypted data to be sent to the device, in which case win if I can find out where/how to do that, or it might be the device requires HDCP and decrypts non-HDCP content before sending it over VGA/DVI. I've no ideas yet on that front either.

Ah well probably enough learning for today, I knew nothing about HDCP this morning, so I can't say it made my life any better learning about it :-P

you have a long road to walk, but first you have to leave the house

2014-07-30T01:47:12Z

or why publishing code is STEP ZERO.

If you've been developing code internally for a kernel contribution, you've probably got a lot of reasons not to default to working in the open from the start, you probably don't work for Red Hat or other companies with default to open policies, or perhaps you are scared of the scary kernel community, and want to present a polished gem.

If your company is a pain with legal reviews etc, you have probably spent/wasted months of engineering time on internal reviews and stuff, so think all of this matters later, because why wouldn't it, you just spent (wasted) a lot of time on it, so it must matter.

So you have your polished codebase, why wouldn't those kernel maintainers love to merge it.

Then you publish the source code.

Oh, look you just left your house. The merging of your code is many many miles distant and you just started walking that road, just now, not when you started writing it, not when you started legal review, not when you rewrote it internally the 4th time. You just did it this moment.

You might have to rewrite it externally 6 times, you might never get it merged, it might be something your competitors are also working on, and the kernel maintainers would rather you cooperated with people your management would lose their minds over, that is the kernel development process.

step zero: publish the code. leave the house.

(lately I've been seeing this problem more and more, so I decided to write it up, and it really isn't directed at anyone in particular, I think a lot of vendors are guilty of this).

DisplayPort MST dock support for Fedora 20 copr repo.

2014-05-23T02:52:29Z

So I've created a copr

http://copr.fedoraproject.org/coprs/airlied/mst/

With a kernel + intel driver that should provide support for DisplayPort MST on Intel Haswell hardware. It doesn't do any of the fancy Dell monitor stuff yet, it primarily for people who have Lenovo or Dell docks and laptops that can't currently multihead.

The kernel source is from this branch which backports a chunk of stuff to v3.14 to support this.

http://cgit.freedesktop.org/~airlied/linux/log/?h=drm-i915-mst-v3.14

It might still have some bugs and crashes, but the basics should in theory work.

why I suck at finishing stuff , or how I learned to stop working and love DisplayPort MST

2014-04-02T12:00:55Z

DisplayPort 1.2 Multi-stream Transport is a feature that allows daisy chaining of DP devices that support MST into all kinds of wonderful networks of devices. Its been on the TODO list for many developers for a while, but the hw never quite materialised near a developer.

At the start of the week my local Red Hat IT guy asked me if I knew anything about DP MST, it turns out the Lenovo T440s and T540s docks have started to use DP MST, so they have one DP port to the dock, and then dock has a DP->VGA, DP->DVI/DP, DP->HDMI/DP ports on it all using MST. So when they bought some of these laptops and plugged in two monitors to the dock, it fellback to using SST mode and only showed one image. This is not optimal, I'd call it a bug :)

Now I have a damaged in transit T440s (the display panel is in pieces) with a dock, and have spent a couple of days with DP 1.2 spec in one hand (monitor), and a lot of my hair in the other. DP MST has a network topology discovery process that is build on sideband msgs send over the auxch which is used in normal DP to read/write a bunch of registers on the plugged in device. You then can send auxch msgs over the sideband msgs over auxch to read/write registers on other devices in the hierarchy!

Today I achieved my first goal of correctly encoding the topology discovery message and getting a response from the dock:
[ 2909.990743] link address reply: 4
[ 2909.990745] port 0: input 1, pdt: 1, pn: 0
[ 2909.990746] port 1: input 0, pdt: 4, pn: 1
[ 2909.990747] port 2: input 0, pdt: 0, pn: 2
[ 2909.990748] port 3: input 0, pdt: 4, pn: 3

There are a lot more steps to take before I can produce anything, along with dealing with the fact that KMS doesn't handle dynamic connectors so well, should make for a fun tangent away from the job I should be doing which is finishing virgil.

I've ordered another DP MST hub that I can plug into AMD and nvidia gpus that should prove useful later, also for doing deeper topologies, and producing loops.

Also some 4k monitors using DP MST as they are really two panels, but I don't have one of them, so unless one appears I'm mostly going to concentrate on the Lenovo docks for now.

Fedora rawhide should have GL 3.3 on radeonsi supported hardware

2014-03-19T21:16:49Z

So to enable OpenGL 3.3 on radeonsi required some patches backported to llvm 3.4, I managed to get some time to do this, and rebuilt mesa against the new llvm, so if you have an AMD GPU that is supported by radeonsi you should now see GL 3.3.

For F20 this isn't an option as backporting llvm is a bit tricky there, though I'm considering doing a copr that has a private llvm build in it, it might screw up some apps but for most use cases it might be fine.

talk from LCA2014 - Virtio GPU

2014-01-10T01:24:40Z

Talked yesterday about virgil project, virtio based GPU and where its going.

watch it here,
http://mirror.linux.org.au/linux.conf.au/2014/Thursday/93-Virgil_A_virtual_3D_GPU_for_qemu_-_Dave_Airlie.mp4

adventures in EGL image buffer passing

2013-12-17T07:06:32Z

So I've been looking into how I can do some buffer passing with EGL and OpenGL with a view to solving my split renderer/viewer problem for qemu.

http://cgit.freedesktop.org/~airlied/eglbufpass/
contains the hacks I've been playing with so far.

The idea is to have a rendernode + gbm using server side renderer, that creates textures and FBOs attached to them, renders into them, then sends them to a client side, which renders the contents to the screen using GL rendering.

This code reuses keithp's fd passing demo code and some of dvdhrm's simple dma-buf code.

Firstly the server uses GBM and rendernodes to create a texture, that it binds to a FBO. It generates an EGLImage from the texture using EGL_GL_TEXTURE_2D_KHR, then uses EGL_MESA_drm_image to get a handle for it, then uses libdrm drmPrimeHandleToFD to create an fd to pass to the server. It passes the fd using the fdpassing code. It then clears the texture, sends the texture info to the client, along with a dirty rect, clears it again, and sends another dirty rect.

The client side, uses EGL + GLES2 with EXT_image_dma_buf_import to create an EGLImage from the dma-buf, then uses GL_OES_EGL_image to create a 2D texture from the EGLImage then just renders the texture to a window.

Shortcomings I've noticed in the whole stack so far:
a) asymmetric interfaces abound:

1) we have an EGLImage importer for dma-buf EXT_image_dma_buf_import, but we have no EGLImage dma-buf exporter yet - hence the MESA_drm_image + libdrm hack.

2) we have an EGLImage exported for Desktop OpenGL, EGL_KHR_gl_image works fine. But we only have EGLImage importers for GLES, GL_OES_EGL_image - hence why the client is using GLES2 to render not GL like I'd like.

b) gallium is missing dma-buf importing via EXT_image_dma_buf_import, I have a quick patch, since we have the ability to import from fds just not from dma-bufs, I should send out my first hack on this.

The demo also has color reversing issues I need to sort out, due to gallium code needing a few more changes I think, but I've gotten this to at least run on my machine with nouveau and the hacked up dma-buf importer patch.

disconnected VM operation and 3D

2013-12-10T04:26:09Z

So one of the stumbling blocks on my road to getting 3D emulation in a VM is how most people use qemu in deployed situations either via libvirt or GNOME boxes frontends.

If you use are using libvirt and have VMs running they have no connection to the running user session or user X server, they run as the qemu user and are locked down on what they can access. You can restart your user session and the VM will keep trucking. All viewing off the VM is done using SPICE or VNC. GNOME Boxes is similar except it runs things as the user, but still not tied to the user session AFAIK (though I haven't confirmed).

So why does 3D make this difficult?

Well in order to have 3D we need to do two things.

a) talk to the graphics card to render stuff
b) for local users, show the user the rendered stuff without reading it back into system RAM, and sticking it in a pipe like spice or vnc, remote users get readback and all the slowness it entails.

No in order to do a), we face a couple of would like to have scenarios:

1. user using open source GPU drivers via mesa stack
2. user using closed source binary drivers like NVIDIA or worse fglrx.

How to access the graphics card normally is via OpenGL and its window APIs like GLX. However this requires a connection to your X server, if your X server dies your VM dies, if your session restarts your VM dies.

For scenario 1, where we have open source kms based drivers, the upcoming render nodes support in the kernel will allow process outside the X server control to use the capabilities of the graphics card via the EGL API. This means we can render in a process offscreen. This mostly solves problem (a) how to talk to the graphics card at all.

Now for scenario 2, so far NVIDIA has mostly got no EGL support for its desktop GPUs, so in this case we are kinda out in the cold, until they have at least EGL support, in terms of completely disconnecting the rendering process from the running user X server lifecycle.

This leaves problem (b), how do we get the stuff rendered using EGL back to the user session to display it. My first initial hand-wave in this area involved EGL images and dma-buf, but I get the feeling on subsequent reads that this might not be sufficient enough for my requirements. It looks like something like the EGLStream extension might be more suitable, however EGLstream suffers from only being implemented in the nvidia tegra closed source drivers from what I can see. Another option floated was to somehow use an embedded wayland client/server somewhere in the mix, I really haven't figured out the architecture for this yet (i.e. which end has the compositor and which end is the client, perhaps we have both a wayland client and compositor in the qemu process, and then a remote client to display the compositor output, otherwise I wonder about lifetime and disconnect issues). So to properly solve the problem for open source drivers I need to either get EGLstream implemented in mesa, or figure out what the wayland hack looks like.

Now I suppose I can assume at some stage nvidia will ship EGL support with the necessary bits for wayland on desktop x86 and I might not have to do anything special and it will all work, however I'm not really sure how to release anything in the stopgap zone.

So I suspect initially I'll have to live with typing the VM lifecycle to the logged in user lifecycle, maybe putting the VM into suspend if the GPU goes away, but again figuring out to integrate that with the libvirt/boxes style interfaces is quite tricky. I've done most of my development using qemu SDL and GTK+ support for direct running VMs without virt-manager etc. This just looks ugly, though I suppose you could have an SDL window outside the virt-manager screen and virt-manager could still use spice to show you the VM contents slower, but again it seems sucky. Another crazy idea I had was to have the remote viewer open a socket to the X server and pass it through another socket to the qemu process, which would build an X connection on top of the pre opened socket,
therefore avoiding it having to have direct access to the local X server. Again this seems like it could be a largely ugly hack, though it might also work on the nvidia binary drivers as well.

Also as a side-note I discovered SDL2 has OpenGL support and EGL support, however it won't use EGL to give you OpenGL only GLES2, it expects you to use GLX for OPENGL, this is kinda fail since EGL with desktop OpenGL should work fine, so that might be another thing to fix!

virgil3d: an update

2013-10-03T05:17:46Z

Okay its been a while, so where is virgil3d up to now I hear you ask?

Initially I wrote a qemu device and a set of guest kernel drivers in order to construct a research platform on which to investigate and develop the virgil protocol, renderer and guest mesa drivers based on Gallium3D and TGSI. Once I got the 3D renderer and guest driver talking I mostly left the pile of hacks in qemu and kernel alone. So with this in mind I've split development into two streams moving forward:

1) the virgil3d renderer and 3D development:
This is about keeping development of the renderer and guest driver continuing, getting piglit tests passing and apps running. I've been mostly focused on this so far, and there has been some big issues to solve that have taken a lot of the time, but as of today I got xonotic to play inside the VM, and I've gotten the weston compositor to render the right way up. Along with passing ~5100/5400 piglit gpu.tests.

The biggest issues in the renderer development have been
a) viewport setup - gallium and OpenGL have different viewport directions, and you can see lots of info on Y=0=TOP and Y=0=BOTTOM in the mesa state tracker, essentially this was more than my feeble brain could process so I spent 2 days with a whiteboard, and I think I solved it. This also has interactions with GL extensions like GL_ARB_fragment_coord_conventions, and FBOs vs standard GL backbuffer rendering.

b) Conditional rendering - due to the way the GL interface for this extension works I had to revisit my assumption that the renderer could be done with a single GL context, I had to rewrite things to use a GL context per guest context in order to give conditional rendering any chance of working. The main problem was using multiple GL queries for one guest query didn't work at all with the cond rendering interface provided by GL.

c) point sprites - these involved doing shader rewrites to stick gl_PointCoord in the right places, messy, but the renderer now has shader variants, however it needs some better reference counting and probably leaks like a sieve for long running contexts.

2) a new virtio-gpu device

The plan is to create a simple virtio based GPU, that can layer onto a PCI device like the other virtio devices, along with another layer for a virtio-vga device. This virtio based gpu would provide a simple indirect multi-headed modesetting interface for use by any qemu guests, and allow the guest to upload/download data from the host side scanouts. The idea would be then to give this device capabilities that the host can enable when it detects the 3d renderer is available and qemu is started correctly. So then the guest can use the virtio gpu as a simple GPU with no 3D, then when things are ready the capability is signalled and it can enable 3D. This seems like the best upstreaming plan for this work, and I've written the guts of it.

In order to test the virtio-gpu stuff I've had to start looking at porting qemu to SDL 2.0 as SDL 1.2 can't do multi-window and can't do argb cursors, but SDL 2.0 can. So I'm hoping with SDL 2.0 and virtio-gpu you can have multiple outputs per the vgpu show up in multiple SDL windows.

I'll be speaking about virgil3d at the KVM Forum in Edinburgh in a couple of weeks and also be attending Kernel Summit.

glamor Xv support (gsoc in a day)

2013-09-23T07:13:02Z

So X.org had a GSOC project to implement Xv support in glamor, but the candidate got a better offer to do something more interesting, so I was bit sleep deprived (sick kid) and didn't want to face my current virgl task and I'm interested in using glamor potentially for virgil so I took a learning day :-)

So I spent the day writing Xv support for glamor for no good reason,

http://cgit.freedesktop.org/~airlied/glamor
git://people.freedesktop.org/~airlied/glamor xv-support

git://people.freedesktop.org/~airlied/xf86-video-ati glamor-xv-support

contains the result of my day, the glamor repo may not be public yet, its waiting on fd.o cgit crawler.

Xv works for YV12 planar videos, I suspect to do packed video support I'd need a GL extension to expose the hw formats for doing packed video, this probably wouldn't be a major extension and maybe someone might do it sometime.

The code supports, brightness, contrast, hue and saturation controls using the code ported from the radeon driver.

I've tested it with mplayer on evergreen card of some variant, and it seems to work fine with the one video I used :-)

virgil3d: design document and sorta build instructions

2013-07-26T04:06:50Z

I've published on Google docs a bit more of a technical document on how virgil3d is designed.

https://docs.google.com/document/d/1cL2sEi5DAd-qY_OzyNiUyIbcU2ZvcI8lhv3PWzFoH_w/pub

I'm hoping to flesh it out a bit more, and of course I'll probably never keep it up to date, but it should be close enough :-)

I've also put up some build instructions here:
https://docs.google.com/document/d/1CNiN0rHdfh7cp9tQ3coebNEJtHJzm4OCWvF3qL4nucc/pub

They are messy and incomplete, any don't go packaging anything.

Introducing Virgil - 3D virtual GPU for qemu

2013-07-18T03:51:13Z

Virgil is a research project I've been working on at Red Hat for a few months now and I think is ready for at least announcing upstream and seeing if there is any developer interest in the community in trying to help out.

The project is to create a 3D capable virtual GPU for qemu that can be used by Linux and eventually Windows guests to provide OpenGL/Direct3D support inside the guest. It uses an interface based on Gallium/TGSI along with virtio to communicate between guest and host, and it goal is to provided an OpenGL renderer along with a complete Linux driver stack for the guest.

The website is here with links to some videos:
http://virgil3d.github.io/

some badly formatted Questions/Answers (I fail at github):
http://virgil3d.github.io/questions.html

Just a note and I can't stress this strongly enough, this isn't end user ready, not even close, it isn't even bleeding edge user ready, or advanced tester usage ready, its not ready for distro packaging, there is no roadmap or commitment to finishing it. I don't need you to install it and run it on your machine and report bugs.

I'm announcing it because there maybe other developers interested or other companies interested and I'd like to allow them to get on board at the design/investigation stage, before I have to solidify the APIs etc. I also don't like single company projects and if I can announcing early can help avoid that then so be it!

If you are a developer interested in working on an open source virtual 3D GPU, or you work for a company who is interested in developing something in this area, then get in touch with me, but if you just want to kick the tyres, I don't have time for this yet.

on open source development and disagreements

2013-03-30T08:08:58Z

So I've been involved in a recent dispute on the wayland project, with a person I'd classify as a poisonous person. Basically a contributor who was doing more damage than good, and was causing unneeded disturbances. I won't comment any further on that here, but just setting the scene for writing this.

So everytime something like this happens in a project, there emerges from the woodwork, people who claim that having public discussions about these sort of things is bad for open source, or makes us look like a crowd of juvenile developers, also how you never see this thing on closed sourced projects, or with open-source projects developer in-house and thrown over the wall. I've also recently seen this crop up when Linus flamed people, and everyone wondered why he didn't do it on some sort of private list or something.

Now I can only think these people are one of:

a) never worked in a company on a major closed source project.

b) if they have, its been top down development, where managers are telling them what to do, and maybe some architect dude has drawn a load of pretty pictures and docs. Of course the architect is never wrong, but its above your pay grade to talk to someone of such authority, so when you find problems with the architecture you hack around them instead of growing a pair and standing your ground, or else you aren't good enough to notice anything wrong.

I've seen plenty of companies where developers leave due to in-fighting or transfer to a different department, this stuff never comes out and you all are none the wiser.

So open source doesn't have top-down development, its all bottom up, most contributors to major projects do so with some ideas of what they want, but they aren't been driven by a management chain. However it means that there is generally nobody to force someone into their views, and when two people collide (or in this case, one person and everyone else), something has to give, and its best to give in public, so nobody can say it was some sort of cabal or closed decision.

Now open-source is about seeing the sausage making process, you get to see all the bits of stuff you don't want to think go into the sausages, you have to face a lot more truth, and you have to be willing to stand up against things without mummy manager to back you up. You can't have all the nice benefits of open-source development without having the bad side, the public blowups and discussion, it just can't work like that. If we take all those discussions to private lists or emails, where do you draw the line, are the people on that private list some sort of shadowy cabal overlords? Do you want an open-source development model that isn't public?

I'm sure people will say why can't we all just get along? and why can't everyone act mature? well a) we are human, b) there is no HR department frontend blocking the people at the gate, there's no interview process to weed out undesirable traits before they join the project. So when someone submits patches that work you generally accept them as a contributor, and it can take a while before you realise they are doing more harm than good, at which point its going to be public.