Kotlin dependency injection and modularization without a framework

Recently I found myself in yet another discussion about dependency injection frameworks. The internet showed me that there is a weird tension and a lot of discussions about runtime vs compile time di, reflection usage, compile time safety, service locator vs di pattern and many more.

Here's what I think: The only acceptable ... actually good implementation of a di framework is Scala implicits. The reason why the JVM world is so obsessed with di frameworks is because Java is such a limited language, that implementing things with the language only is simply not feasible.

Pure di in Kotlin

It doesn't need many features, but those we need are key to make frameworkless di (I will call it pure di from now on) practical: Primary constructors, default arguments and smart constructors. Implicit parameter passing like in Scala would be an optional bonus on top - this feature is too controversial to just require it for pure di.

About the "testability" requirement

So first, the elephant in the room: You don't need interfaces to create testable implementations for something. Mocking frameworks like mockk can just mock any class you have and replace the implementations. Conclusion: Hiding things behind an interface is a good idea for a lot of reasons, but di doesn't care about them. You decide what you accept as a dependency in your class and that's it. No drawbacks for testability when you aren't able to use the default implementation for testing.

No annotations 

I know there's a standard on the JVM, but as I said in the introduction, we should question that. When a class is declared, just from a domain driven perspective, why on earth should we annotate our constructor with @Inject for example? It's a technical detail of a framework my caller may or not may not use. And even if he uses it, why is the declaration of the constructor not sufficient for anyone else to use it, be it automatically or not by hand? From my pov, using annotations on the dependency is a code smell that we got used to because of CDI. Even worse when configuration file keys are added into the annotation...

Module definitions

A module itself doesn't have to be interfaced. The components of a module can be. The module itself is just a plain old class that defines related components.

Note how Kotlin's primary constructor with default arguments completely replaced the need for any complex override framework sometimes needed for testing or bean definition overrides. Smart constructors (operator fun invoke on an interface companion here) don't exactly relate to dependency injection but can serve as a factory for default implementations.

Multiple module definitions

Using multiple modules with frameworks is often not too easy because of a single container or service locator, that flattens all definitions into a single pool which is used for service retrieval. Service locator will be talked about in the next paragraph, now lets take a brief look how simple multiple modules can and should look in your applications:

Note that it's not necessary to bundle all modules into a single super module - you can group whatever is meaningful for your domain, not what the framework requires you to do. When you really want to squash all definitions, all components of all of your modules into a flat facade, you can either use Kotlin's built-in delegation and interfaces, like so

Or you can use... Scala 3 that has a feature called exports - just kidding, we're doing Kotlin right - or something like what I implemented with this one https://github.com/hannespernpeintner/kotlin-companionvals .

Factories, lazy, optional

All those features di frameworks offer are already built-in in the Kotlin language. Singletons are given by just using val properties. Take a look at this example how factories, lazy things and optional things can be implemented

  

Those features automatically work with IDE features such as auto completion and refactoring, which is one of the most important things in projects and the reason Kotlin is so successful. Also you don't get runtime errors for example for optional dependencies, as Kotlin's built-in nullability gives you compile time errors. An additional bonus is that you can have nullable dependencies on the interface and override with non-nullable implementations in a module. Using those modules non virtual when it's okay to rely on the implementation (for example in testing) safes you from using the double bang operator all over the place.

Service locator

Finally, the probably most important aspect of di frameworks, the piece of code that is the surface your application and your components are allowed to rely on (are they? :) ). The implementation of the service locator is the source of problems in most frameworks, as it always generifies your module graph into something unnecessary generic that works more or less like a big map of types/names to instances/factories. This is also where compile time safety is lost.

Without any frameworks, you can just pass around the module (interface when given) instance you want to use somewhere. I found the best strategy to just use the smallest possible dependencies in your components, even though that may make your primary constructors big - it's just cleaner and more appropriate than passing context objects aka modules directly. For the caller's convenience - which is not an unimportant aspect! - you can provide a smart constructor that takes a complete module.

This is the point where manual declarations are more verbose than the magic wiring frameworks do for you. But hey, that's code. Plain old code. Everyone can go to declarations, refactor them, add more smart constructors, know how they work without having to know any framework. This approach has proven to be appropriate for even big module graphs in my applications.

Inner-module dependencies in components

What if a component that is part of a module needs a component from that very module? Most frameworks solve this problem by making everything lazy. In code, we would reorder statements - with constructors, we have to either pull out default arguments and wire and pass arguments explicitly or change the properties order like so

Not too worse, I think.

Bonus Round 1: Constructor vs field injection

You may have noticed that I did only write about constructor injection. The short reason is, that everything else should never be used as it introduces mutable and invalid state in your application. Whenever you have to deal with an environment that requires you to use such a lifecycle, Kotlin offers the lateinit keyword that can be used perfectly with pure di - but more important it depends on the foreign framework whether it's simple, robust and important to implement. When your environment requires you to use CDI, you should probably stick to it. Or not use those frameworks any more :)

Bonus Round 2: Quasi mixins

Kotlin doesn't allow for multiple inheritance of state, but interfaces and default implementations can become quite powerful and useful for a mix of data driven design and modularization. The idea is to place implementations into interfaces, writing dependencies as abstract state. Interfaces can leverage multiple inheritance and what's left is the implementation of state that can be done declaratively.

Let's consider you have a typical webapp Controller class that fetches some Products and needs some dependencies for that because it's not trivial.

Using interface inheritance could be seen as an abuse of the language feature here, but let's try to stay pragmatic. Using it automatically brings local extensions into scope, enabling implicit parameter passing of contexts, hence dependency injection. This approach gives you the freedom of just not caring about modules at all and just think about fine grained dependencies. Pretty much what di frameworks give you, but without any runtime errors because the source code is your module graph that is already validated on the fly by the compiler :) This approach can also be combined with pure di - you can define generic implementations in interfaces and deliver some default implementations as final classes, just as you wish, I can't see any borders here.

Rendering massive amounts of animated grass

 I recently played Ghost of Tsushima and I was impressed by the amount of foilage that covers the world. Just as I was impressed when I played Horizon: Zero Dawn a few years back.

So my engine can already render a lot of instanced geometry, a lot of per-instance animations and so on, but for that much foilage that is needed for believable vegetation, this is too costly. The answer to the problem is pivot based animation and therefore some simple, stateless animation in the vertex shader.

In addition to that, the instances of for example grass, need to be clustered and culled efficiently. My two-phase occlusion and frustum culling pipeline is exhausted pretty fast, when we use it for 10000000 small instances without any hierarchical component. A cluster is best and easy a small local volume that covers enough instances to not mitigate the benefit of batching. For example it's not worth batching only 10 instances, only to be able to cull them better. 1000 instances seem to work well for me. I generate a cluster's instances randomly, so that I can just render the first n instances and scale n by distance between camera and cluster. This way, the cluster gets thinner and thinner, until completely culled. Hard culling results in pop-ins. For a smooth fadeout without alpha blending enabled - which would again kill the performance of foliage - screen door transparency can be used. This is again a simple few lines, now in the pixel shader, and culling is mostly hidden.

Three things that are for themselves very efficient team up for a nice solution for foliage: Pivot based animation, cluster culling and screen door transparency fading.

As stated under the first video, I don't have nicely authored pivot textures, so I created a veeeeery simple one that just goes from 0-1 from root to leaves of the grass.

Private routes with Kotlin JS and React Router

In order to implement login functionality in a Kotlin React app, I used React Context API to save an optional instance of Login. This context can be used to create a wrap-around component that checks whether a given route can be accessed or not. If not allowed, the request is redirected to the public /login route.

The context class has to provide a way to get the current login data and functionality to logout or login. The state resides in the main component and is accessed through the onChange callback. The component1 function can be used to destructure the useContext result when only read access is needed.

class LoginContextData(login: Login?, private val onChange: (Login?) -> Unit) {
    var login: Login? = login
        private set
    fun login(potentialLogin: Login) {
        login = potentialLogin
        onChange(login)
    }
    fun logout() {
        login = null
        onChange(login)
    }
    operator fun component1() = login
}

// can be global
val LoginContext = createContext(LoginContextData(null) { })

The context is used like

val loginState = useState(null as Login?)
val (login, setLogin) = loginState

LoginContext.Provider(LoginContextData(login) { newLogin ->
    setLogin(newLogin)
}) {
// render your tree here
}

Just like React Router provides the route function, we can write a function that does an if-else on the context and either calls the known route function or gives a redirect:

fun  RBuilder.privateRoute(
    path: String,
    exact: Boolean = false,
    strict: Boolean = false,
    children: RBuilder.(RouteResultProps<*>) -> ReactElement?
): ReactElement {
    val (login) = useContext(LoginContext)

    return route(path, exact, strict) { routerProps: RouteResultProps<*> ->
        if(login != null) {
            children(routerProps)
        } else {
            redirect(from = path, to = "/login")
        }
    }
}

The callsite can just use privateRoute instead or route and done. The login route remains public.

The context can be used to decide whether a navigation item should be rendered or not as well.

My journey with Kotlin JS and React

Honestly I have no idea what the purpose of this post could be, but I am really happy with Kotlin JS and react and I want to write down my thoughts after implementing a real world administration application in my spare time. Maybe some of the sources I link can help somebody, maybe my experience can help someone in a similar position.

This is the app I created:

It uses Kotlin JS, Kotlin react wrapper, react, react-dom, react-router-dom, uuid, styled components, Kotlin coroutines.... the react hooks api, the react context api and Bootstrap 4.

TLDR: Even though Kotlin JS is still experimental, i created a complete administration application using it and encountered only a few minor bugs. There's so much gain in being able to use gradle tooling and Kotlin as a language for me, that I would happily accept some minor bugs here.

Motivation

I have to admit: I love Kotlin but I was very sceptical about its non-JVM targets, especially after reading a lot about Kotlin Native and its caveats. The thing is: I feel such a strong demand of frontend stuff, that it would be super super handy for a Kotlin team to be able to use the same language - and toolchain - for backend and frontend projects and even share libraries between the two targets. Maintaining builds and code on a high quality level is that much easier this way, polyglot really shows its costs there.

Chicken and egg

I would love to implement such a thing in the team at my company, but this is not an easy task: Kotlin JS is still experimental. The next Kotlin version (1.4) will break binary compatibility for example because the compiler backend is switched completely. Given there is already Typescript, it's very very hard to convince people that another experimental technology might do a good job for the frontend. This results in the chicken-egg-problem: No one tries it, no one knows whether it works out well, no one gains experience, no one helps moving the platform forward, no one makes any progress. Arguing for a new technology reminds me of the time my team switched to Kotlin from Java for the backend. And after we did it, everyone was much happier than before. Getting the time to proof that a techology is worth it and can be used in production is key.

Elephant in the room: JSX

The biggest downside of React with JavaScript is most probably JSX. It reminds me of the wild days when JSP was in. Not only does it require the build system to do very very complex stuff, but also I don't think it's a good idea to extend code with something that makes it code no more, mixing markup languages and programming languages, introducing many strange constructs that are mostly workarounds for naming clashes and identifiers that can't be mapped. Tooling has to be adjusted, knowledge has to be adjusted, code style has to be adjusted... This is a proper comment on that. Do you know what is a nice way to write the UI? kotlinx html. This is basically what everyone would be happy with. I am. There was only one missing piece in the workflow: When you get html based designs of the page you should program, you have to convert html to kotlin dsl. With JSX you can just paste the html into your component and modify it slightly for variable usage. For kotlin dsl, there is this, that lets you do the same, but additionally, you can just start refactor names, extracting methods and so on in the best IDE / one of the best IDEs out there.

Hooks 

The last time I used react was when hooks didn't exist. The usage of setState was so annoying for me, that I just didn't warm up with the framework at all, because state is the single most important aspect of the application code. Hooks are such a nice addition and make functional component usage so pleasant. Take a look at a simple example with kotlin. It's hard not to like that. Now the downside: Hooks have some constraints that are not too intuitive. And now a proper downside: Even though I never used any return statements in kotlin, placed every hook usage at the top of the component, I got the infamous rendered too few hooks error... I wasn't able to figure it out exactly but I suspect it came from nested component usage where I had a fairly complex list based component that nested a lot of stuff. I removed the complex component completely, but if I weren't able to do that because of design requirements, I would have had a hard time with it.

Build

Everyone who knows me knows: Builds are really my métier. I am doing this excessively for many many years with different build systems and I always ensure projects have clean, stable, maintainable builds that enable proper development and testing workflows. Builds are one of those areas where having only one kind of them in the team is very beneficial, as all of the existing tooling can be reused. Being able to use gradle is a big plus for me (note, gradle with kotlin, not gradle with groovy brrrr). Convince yourself of how easy and simple the gradle build of a kotlin js project can be here and here. I can confirm that it works like that for a complete application development cycle. The good thing is, that the whole webpack stuff is hidden from you so that you don't have to bother with the whole mess. However, if necessary, you can configure things. And this is one of the two issues I faced during development: Hot reloading with the webpack development server. I had to apply this workaround as everyone seems to have to. Annoying to find out, not a problem anymore after the small fix.

Final thoughts

What can I say? I am very happy about what can already be done with Kotlin JS. Finally, I can get back to frontend development with pleasure again, keeping all my gradle and kotlin love :)

BVH accelerated point light shading in deferred rendering

My engine uses a lot of modern techniques like programmable vertex pulling, persistent mapped buffer based multi threaded rendering with a ring buffer and at the core of these techniques, there is this concept of a simple structured buffer. Experiments with compute based ray tracing on kd- and octrees led me to stackless tree traversal on the gpu, which is very very interesting and can be easily found on the internet. And occasionally, I found this article about an alternative to all those clustered, forward plus deferred tile based or whatever approaches for a massive amount of lights. I highly recommend reading it and all the other nice posts over there. He got my interest. I heard about light bvhs only for offline renderers. And structured buffers? I have them. Compute shaders, I have them. My point lights? Yea, maybe I have many of them, but they mostly don't move. And than again, I need rendering and light evaluation not only for my deferred rendering pass, but also for my transparency pass, a regular grid of environment probes or my voxel cone tracing grid...

Long story short, implementing a basic version was very easy, because the concept is so simple.

Assuming a static tree, my implementation needs ~10ms for 100 point lights instead of ~34ms in the most trivial compute shader in the quite dense configuration above on my crappy notebook with integrated intel card. In a less dense configuration, the time goes down to ~4ms and less. It really depends on the amount of overlapping volumes and how efficient the tree is. 500 pointlights scattered over the Sponza atrium takes below 30ms.

BVH update: The most tricky and also the most costly part of the whole thing is probably the creation and update of the BVH which I haven't implemented efficiently yet. My creation happens on any light movement and clusters lights or inner nodes recursively into buckets of 4. 4 gave me better performance than 8 as in the blog post, probably because my light struct layout is not very efficient.

Sphere union: The implementation to find an enclosing sphere for n spheres is from here. I'm not too sure that a really optimal sphere is found, but since I'm feeding every sphere's aabb corner points into the library, some efficiency is already wasted on my side or the program.