My journey with Kotlin JS and React

Honestly I have no idea what the purpose of this post could be, but I am really happy with Kotlin JS and react and I want to write down my thoughts after implementing a real world administration application in my spare time. Maybe some of the sources I link can help somebody, maybe my experience can help someone in a similar position.

This is the app I created:

It uses Kotlin JS, Kotlin react wrapper, react, react-dom, react-router-dom, uuid, styled components, Kotlin coroutines.... the react hooks api, the react context api and Bootstrap 4.

TLDR: Even though Kotlin JS is still experimental, i created a complete administration application using it and encountered only a few minor bugs. There's so much gain in being able to use gradle tooling and Kotlin as a language for me, that I would happily accept some minor bugs here.

Motivation

I have to admit: I love Kotlin but I was very sceptical about its non-JVM targets, especially after reading a lot about Kotlin Native and its caveats. The thing is: I feel such a strong demand of frontend stuff, that it would be super super handy for a Kotlin team to be able to use the same language - and toolchain - for backend and frontend projects and even share libraries between the two targets. Maintaining builds and code on a high quality level is that much easier this way, polyglot really shows its costs there.

Chicken and egg

I would love to implement such a thing in the team at my company, but this is not an easy task: Kotlin JS is still experimental. The next Kotlin version (1.4) will break binary compatibility for example because the compiler backend is switched completely. Given there is already Typescript, it's very very hard to convince people that another experimental technology might do a good job for the frontend. This results in the chicken-egg-problem: No one tries it, no one knows whether it works out well, no one gains experience, no one helps moving the platform forward, no one makes any progress. Arguing for a new technology reminds me of the time my team switched to Kotlin from Java for the backend. And after we did it, everyone was much happier than before. Getting the time to proof that a techology is worth it and can be used in production is key.

Elephant in the room: JSX

The biggest downside of React with JavaScript is most probably JSX. It reminds me of the wild days when JSP was in. Not only does it require the build system to do very very complex stuff, but also I don't think it's a good idea to extend code with something that makes it code no more, mixing markup languages and programming languages, introducing many strange constructs that are mostly workarounds for naming clashes and identifiers that can't be mapped. Tooling has to be adjusted, knowledge has to be adjusted, code style has to be adjusted... This is a proper comment on that. Do you know what is a nice way to write the UI? kotlinx html. This is basically what everyone would be happy with. I am. There was only one missing piece in the workflow: When you get html based designs of the page you should program, you have to convert html to kotlin dsl. With JSX you can just paste the html into your component and modify it slightly for variable usage. For kotlin dsl, there is this, that lets you do the same, but additionally, you can just start refactor names, extracting methods and so on in the best IDE / one of the best IDEs out there.

Hooks 

The last time I used react was when hooks didn't exist. The usage of setState was so annoying for me, that I just didn't warm up with the framework at all, because state is the single most important aspect of the application code. Hooks are such a nice addition and make functional component usage so pleasant. Take a look at a simple example with kotlin. It's hard not to like that. Now the downside: Hooks have some constraints that are not too intuitive. And now a proper downside: Even though I never used any return statements in kotlin, placed every hook usage at the top of the component, I got the infamous rendered too few hooks error... I wasn't able to figure it out exactly but I suspect it came from nested component usage where I had a fairly complex list based component that nested a lot of stuff. I removed the complex component completely, but if I weren't able to do that because of design requirements, I would have had a hard time with it.

Build

Everyone who knows me knows: Builds are really my métier. I am doing this excessively for many many years with different build systems and I always ensure projects have clean, stable, maintainable builds that enable proper development and testing workflows. Builds are one of those areas where having only one kind of them in the team is very beneficial, as all of the existing tooling can be reused. Being able to use gradle is a big plus for me (note, gradle with kotlin, not gradle with groovy brrrr). Convince yourself of how easy and simple the gradle build of a kotlin js project can be here and here. I can confirm that it works like that for a complete application development cycle. The good thing is, that the whole webpack stuff is hidden from you so that you don't have to bother with the whole mess. However, if necessary, you can configure things. And this is one of the two issues I faced during development: Hot reloading with the webpack development server. I had to apply this workaround as everyone seems to have to. Annoying to find out, not a problem anymore after the small fix.

Final thoughts

What can I say? I am very happy about what can already be done with Kotlin JS. Finally, I can get back to frontend development with pleasure again, keeping all my gradle and kotlin love :)

BVH accelerated point light shading in deferred rendering

My engine uses a lot of modern techniques like programmable vertex pulling, persistent mapped buffer based multi threaded rendering with a ring buffer and at the core of these techniques, there is this concept of a simple structured buffer. Experiments with compute based ray tracing on kd- and octrees led me to stackless tree traversal on the gpu, which is very very interesting and can be easily found on the internet. And occasionally, I found this article about an alternative to all those clustered, forward plus deferred tile based or whatever approaches for a massive amount of lights. I highly recommend reading it and all the other nice posts over there. He got my interest. I heard about light bvhs only for offline renderers. And structured buffers? I have them. Compute shaders, I have them. My point lights? Yea, maybe I have many of them, but they mostly don't move. And than again, I need rendering and light evaluation not only for my deferred rendering pass, but also for my transparency pass, a regular grid of environment probes or my voxel cone tracing grid...

Long story short, implementing a basic version was very easy, because the concept is so simple.

Assuming a static tree, my implementation needs ~10ms for 100 point lights instead of ~34ms in the most trivial compute shader in the quite dense configuration above on my crappy notebook with integrated intel card. In a less dense configuration, the time goes down to ~4ms and less. It really depends on the amount of overlapping volumes and how efficient the tree is. 500 pointlights scattered over the Sponza atrium takes below 30ms.

BVH update: The most tricky and also the most costly part of the whole thing is probably the creation and update of the BVH which I haven't implemented efficiently yet. My creation happens on any light movement and clusters lights or inner nodes recursively into buckets of 4. 4 gave me better performance than 8 as in the blog post, probably because my light struct layout is not very efficient.

Sphere union: The implementation to find an enclosing sphere for n spheres is from here. I'm not too sure that a really optimal sphere is found, but since I'm feeding every sphere's aabb corner points into the library, some efficiency is already wasted on my side or the program.

Smart property delegation with Kotlin

Every now and then I have a use case where I have a data holder object that is wrapped by some other object. With Kotlin's interface delegation, it's very easy to implement an interface by delegating to the implementing property. However, sometimes you don't have interfaces, so I realized that there's one thing missing from the Kotlin standard lib: Have a property delegating to another property. Or at least I wasn't able to find it and it's kind of difficult to google that. So here's the solution:



I'm using a property reference (you need the reflection dependency for that) for generic property (get and set) usage. This is kind of a lense, known from functional programming I guess.

Nice things can done with that, for example typesafe ui components can be generated like that:

Programmable vertex pulling

So i finally managed to invest some time to implement programmable vertex pulling in my engine. I can really recommend to implement an abstraction over persistent mapped buffers that lets you implement structured buffers of generic structs and then use it on the cpu and the gpu side as a simple array of things.

Nothing comes for free: I find it quite difficult to handle any other layout than std430 because that matches what your c, c++ code is doing, as long as you restrict yourself to always use 16 byte alignment members, I think. My struct framework doesn't do any alignment, so I just added dummy members where appropriate in order to match the layout requirements. Afterwards, struct definitions in glsl have to match your struct on the cpu side and the only things left for your vertices is


struct VertexPacked {
    vec4 position;
    vec4 texCoord;
    vec4 normal;
};
layout(std430, binding=7) buffer _vertices {
    VertexPacked vertices[];
};

...


int vertexIndex = gl_VertexID;
VertexPacked vertex = vertices[vertexIndex];

Combined with persistent mapping, you can get rid of any layout fuddling, synchronization, buffering, mapping...and it just works.

Regarding performance: I am using an array of structs approach because it is the simplest to use. The performance in my test scenes (for example sponza) is completely identical to the traditional approach. No performance differences on a Intel UHD Graphics 620.

Having free indexed access to vertices in your shaders can be beneficial in other situations as well. For example you can implement a kd-tree accelerated ray tracer with compute that uses indices into your regular vertex array.

Pixel perfect picking with deferred rendering

There are several ways to accomplish pixel perfect picking in one's engine. Some tutorials mention an object hierarchy as a scene representation in order to trace rays for picking. This is often done on the CPU, where information about the object is already available when the ray hit callback is invoked when tracing. On the GPU, this could be done with a compute shader or a vertex shader that writes to an output buffer... this output buffer can be read back on the CPU.

With deferred rendering however, I have a simpler approach that doesn't involve any tracing at all. Since I need object IDs for later passes to fetch instance data out of a big buffer, I write them to one output texture in my deferred rendering buffer. The output texture can be of type int or uint, depending on the amount of objects you have to handle. One can pack the index into bits of a regular 8bit rgba texture, into a floating point texture, or whatever texture has some bits space left in your gbuffer. After the gbuffer pass of deferred rendering was done, one can use glReadPixels (with read buffer set) or glGetTexImage and the current mouse position to get the index of the clicked object back to the cpu side of things. Besides the format handling and the bit mangling, this is rather trivial, so I won't post code here, but here's a nice video of the usage in my new ribbon based editor :)

Sparse voxelization on the CPU

The various adventures with Voxel Cone Tracing showed me, that asynchronous and partly done voxelization on the gpu can become really really tricky, because the data structures involved are very hard to implement.

So after ditching clipmaps because they won't allow for enough caching of static sthings, I gave sparse voxelization another try. Like - I think - CryEngine, I wanted to implement voxelization on the cpu, in order to be able to perform it async and partly on demand. The voxelization itself didn't took too much time - I decided to go for the "brick" approach, which allocates either none or all eight children of a given node, whether some of them are empty or not. The advantage here would be that working with offsets is much easier, as first childs always are located at the brick pointer start and the next 7 children are contiguous in memory and one can just increment the pointer to get the next child. Using a pointer based apporach enables required asynchronicity and streaming, because the memory layout doesn't have to fulfill everything we need on the GPU later on for tracing. Super nice Kotlin coroutines enable usage of 100% of the cpu very easily with a fork join approach and just launching thousands of coroutines. Depth of the tree is 11, size is 1000³. I think in order to have this resolution, one would need a 3d texture with resolution of 512³ or 1024³ which usually is too costly for voxel cone tracing approaches.



I didn't squeeze out the last bit of performance here. So yes, maybe this can be implemented much faster. And no excuses, but the video itself eats up a lot of the fps I had - it had about 30 fps, pretty much limited by my laptop cpu.

And this video could be the last thing I can show about voxelization, because the tracing part didn't just work out as expected. In order to work with empty leave nodes, one has to get creative. Because if you intersect a ray with a box, but the hit would actually be with a box that is an empty leaf, you would have to backtrack the tree (hence you need parent node pointers in all nodes, but that's okay) until you get somewhere where some sibling nodes are left to be traced...

My implementation is too bad to be anywhere near realtime and debugging or reason about what's going on is very very tedious... after having my computer completely frozen half a dozen times because I made another stupid mistake on the GPU, I decided that this approach is just not doable in the amount of time I can afford :)

Companion vals

I totally forgot to make a braindump about another feature I stumbled upon lately: companion vals.

There's this really interesting feature proposal for the Kotlin language.

In essence, the proposed feature allows to write the following code:


and the following code:

From my point of view the feature has two aspects.

The first one is making members of companion members part of the surrounding instance. That means we can have properties and their members are automatically exposed, hence you don't need to access them with dot notation.

The second aspect is that other scopes are treated as well: If something is marked as companion, it is automatically available as a receiver in the corresponding scope. The proposal only talks about class properties, which are available in the class body automatically. This enables having Kotlin's scoped extension functions available without the need to use with(AdressPrinter) }{} everywhere.

I extended the Kotlin compiler with these two features and widened the application of the second aspect to all (?) possible scopes. This means if the companion val is top level, it's automatically available in the whole file. If a function parameter is marked as companion, the argument is going to be available as a receiver in the function body and so on. The implementation can be found here and examples can be found in the working tests.

Since the compiler has no simple and no stable API, I also implemented an annotation processor, that fulfils the first aspect. Repository can be found here. This would make the above code compile (with the right imports). Works by generating extension functions for all members, just as you would do in Kotlin by hand anyway if you would want to have this functionality.