Here we go again: I'm searching for a performant, convenient solution for dependency injection in Kotlin. Koin got my attention, because it seems to be simple and straightforward. Since it uses a lot of small inline functions and seems to have only a few hotspots where performance suckers could lurk, it seemed very promising. But I wouldn't want to use it in my game engine project until I can be sure that the performance impact would be negligible. So I did a ... probably totally flawed microbenchmark, that doesn't show anything, but I want to dump it here nonetheless.
So we have a simple service class and another service class that depends on the first service. The main module depends on the seconds service, so the chain has to be fulfilled.
Using Koin, one can simply write a few lines and everything is wired together automatically. Note that the Koin context has to be started and stopped, which has to be excluded from the benchmark later.
Even though I'm aware that this is an artificial benchmark that may be flawed, it's pretty much clear that using Koin will have a huge impact on performance, that could make program infrastrucutre slower by a factor of 100. Of course, we're talking about dependency injection at object creation time, which should be a rare case in a game engine. Nonetheless, not too good from my sight.
I described the basics about the two-phase occlusion culling
technique in the first part post, but there is one thing, that doesn't
work well with this simple setup: Instancing or sometimes viewed as batching.
The problem is very simple, the solution however is not.
Only a few engines managed to implement the technique, but it is
unavoidable if one needs to render large batches or large amounts of
instances. Quick recap, under the assumption that we compromise
flexibility in favor of speed and use OpenGL 4.5 level graphics APIs:
Avoid buffer changes one, two different vertex layouts (for example for static and animated meshed)
Avoid program changes well defined firstpass program (deferred rendering) or subroutines in a global attribute buffer
Minimize drawcalls Usage of instanced and indirect rendering
The
usage of draw commands and indirect rendering is nice enough. The
amount of drawcommands is reduced by using instanced rendering for
entities that have different uniform attributes but the same vertices
and indices. This results in the command and entitiy buffers of the
following form:
Entities
uint entityId
678
mat4 transform
000...
uint materialIndex
18
uint entityId
1566
mat4 transform
000...
uint materialIndex
2
Commands
vertexOffset
0
indexOffset
0
indexCount
99
instanceCount
2
vertexOffset
1200
indexOffset
99
indexCount
33
instanceCount
3
Firing indirect rendering call is now done with a
count of two, because we have two commands. 5 objects will be drawn. The
shaders are invoked several times and the entity data can be accessed
with an index and fetched from the entities buffer. The resulting
indices in the shader will look like
Shader Invocations
DrawId
InstanceId
Entity buffer index
0
0
0
0
1
1
1
0
???
1
1
???
1
2
???
As mentioned in the previous blog post, there's no
chance for us to know the entity index when the second command is
executed, because the current offset is based on the amount of instances
the previous commands draw. The solution is to use an additional offset
buffer with size of the draw command buffer. For every command, we set
the offset to the appropriate value when creating the commands on the
cpu side. With instance based culling, this problem intensifies, because
the cpu doesn't know the offset anymore. The calculation has to be done
on the GPU now. My solution is still based on vertex shader kernels,
but I will tell you later why this is probably problematic. First how it
is done, because conceptionally, it will be the same for compute
shaders. The layout now looks like this:
Shader Invocations
DrawId
InstanceId
Entity buffer index (command offset + InstanceId)
Offset of the command
0
0
0
0
0
1
1
0
1
0
2
2
1
1
3
2
1
2
4
2
The first step is to determine the
visibility for every single instance. Vertex shader kernels have an
advantage here, because they can be arbitrarily large (compute shader
groups are limited to 1024 or so). A single draw call can determine
visibility for dozens of thousands of instances or entities. Combined
with instancing, we can use DrawId and InstanceId to index into the
command buffer and offset buffer (DrawId) and into the entity buffer
(offset + InstanceId). Since the same kernel sizes are applied to every
command, invocations are wasted if few commands have many more
instances as others. So you might want to launch draw calls without
instancing, one per command, which could be faster here.
The
visibility is again stored into the visibility buffer, which has to be
as large as the entity buffer now. With non-instanced culling, size of
the command buffer was sufficient. One important thing has to be done
now: Every visible entity thread has to increase a counter that is
associated with the corresponding command - since this is done per
invocation, atomic operations are needed. So we need another int buffer
(just like the offset buffer), that holds the visible instances count
per command.
The reason why this is so important is,
that this is the bit of information that is used in a second step to
append entity data and draw commands in parallel. This is done by an algorithm similar/equal to parallel prefix sum - you can google it. Tldr: Each command has to know how many visible instances all previous commands produce in total, so that it can append its own instances there.
I call the seconds step appending step, while the first one is the visibility computation step,
that actually calculates the visibility of an instance. For a massive
amount of instanced commands, one would probably want to launch a
two-dimensional kernel of the size n*m where n is the amount of commands
and m is max(maxInstanceCount, someArbitraryValue) or something. Again,
with vastly different instanceCounts per command, multiple shader calls
could give a benefit.
So now we need a target buffer
that holds the entity data, a target buffer for the commands and a
target buffer for the entity data offsets. Additionally, we need an
atomic counter for the target command index and then an atomic counter
per command, that is the current index of the entities per command. Each
shader invocation has its own command index in the DrawID built-in and
the instance index in the InstanceID built-in. So we can calculate all
information that is needed on the fly:
struct oldCommand = read from the bound command buffer with DrawID
uint oldCommandEntityOffset = read from the oldOffsetBuffer with DrawID
uint oldEntityIndex = oldCommandEntityOffset + InstanceID
uint visibilityBufferIndex = oldEntityIndex
uint targetCommandIndex = atomicAdd(commandConuter, 1)
uint targetInstanceIndex = atomicAdd(commandCounters[targetCommandIndex], 1)
uint targetEntityIndex = sumOfVisibleInstancesOfAllCommandsBeforeCurrentCommand
The
current entity data can then be written to the targetEntityDataBuffer,
as well as the offset for the instance/command. The command can be written by the first active
thread (InstanceID == 0). The resulting buffers contain only the visible
instances, offsets of the visible instances and drawcommands that can
be used to only draw exactly those instances.
Here's a video demonstrating occlusion and frustum culling with this technique:
Bonus: For the visibility computation step, it's nice to launch a vertex shader kernel, since the kernel can be arbitrarily large in two dimensions at max. Since there's no shared memory involved, this will probably perform better than the compute shader equivalent, maybe at the same speed, but it shouldn't be slower. The appending step needs an atomic operation per invocation, because every invocation increments the counter if the corresponding instance is visible (which the other invocations couldn't know). Instead of "global" shared memory, one could use shared memory of a compute shader group. Shared memory in a group is much faster than atomic operations between a gazillion vertex shader invocations, so I will implement this at some time and may be able to show a performance comparison.
There is this one question since beginning of multicore processors: How to multithread your game engine? I can't answer this question in general, but I can answer the question How to multithread the rendering part of my game engine?
Problem
The most simple case is a single-threaded game engine that does this:
while(true) {
simulateWorld();
renderWorld();
}
The update step produces a consistent state of the world. After each update, the rendering is done. After the rendering is done, the next cycle is processed. No synchonization needed, everything works. The bad thing: simulateWorld is a cpu-heavy task, renderWorld is a gpu-heavy task. So while the world is updated, the gpu idles, while the gpu renders, the cpu idles (more or less). And the worst thing: Your framerate is limited by both the cpu and the gpu. Your frametime is timeOf(simulateWorld) + timeOf(renderWorld). Our target frametime is timeOf(renderWorld) and to keep this time as low as possible. More precisely the target is timeOf(renderWorld).
Foundations
Okay I lied: The above mentioned problem is only the overall-globally-problem we solve. There's a shitload of other problems, that mostly depend on the language platform, graphics api, hardware, your skillset and so on. Since our lifetime is limited, and I want to give practical, real-world-relevant help, I make some assumptions. One of these is, that you use OpenGL and a pretty new version of it. Let's say 4.3. Multithreaded rendering could probably be done better and faster with DX 12 or Vulkan, but let's stay cross-platform and simpler with OpenGL.
I assume that you already have at least a little bit knowledge about how game engines work, timesteps, scene representations and so on.
Multiple gpu contexts
In order to be able to issue commands to OpenGL, you have to use a thread that has an OpenGL context bound to it. Architecture-wise, there is the CPU, the OpenGL driver and the GPU. The driver holds a queue, where the commands from the SPU are buffered somehow (implementation dependend). Then again, there is a queue for the GPU - that again buffers commands that came from the driver (implementation dependend). First, all commands the CPU issue are asynchronous, which means there's no actual work done on the GPU yet, but the command is queued by the driver. The order is preserved, that's what you can count on. The implementation can decide how many commands are queued up, until the cpu thread will block on command issuing. And the implementation can decide how many commands are buffered on the GPU side of things.
Now, having that said, let's think about multithreading. What we can control is the CPU side (unless you write your own driver, which I doubt you want to do). Multiple cpu threads would require multiple gpu contexts - one for each thread. While this is theoretically possible with OpenGL, it is a very dumb idea in 19 out of 20 cases. As with traditional multithreading, context switching comes with a very high overhead - so you have to ensure that context switching overhead doesn't eat up the performance you gained from using multiple threads. And here's what's wrong: It seems that all OpenGL drivers (where you issue your comamnds at) except the one for iOS, are implemented singlethreaded. That means you can issue commands from multiple cpu threads, but on the driver thread, those commands are processed sequentially, with a context switch in between. You can find a more detailled explanation here, so that I don't have to lose many further words, but: Don't use multiple contexts for anything else than asynchronous resource streaming.
I assume we only use one OpenGL context. That means, all OpenGL calls must happen on a single thread. The construct used for this is the command pattern. A few hints: You should issue as few commands to OpenGL as possible. You should issue as lightweight commands as possible. Obviously, using only one thread for command execution limits the amount of work that can be done by this (cpu) thread. Nonetheless, you want a single, inifintely running worker thread that owns your OpenGL context and takes commands out of a non-blocking command queue. For convenience, you can seperate commands that should return a result (hence block) or should just be fired and forgotten. It's not nessecary that all commands are properly implemented by yout context wrapper/worker thread class, even though it makes your application more predictable in terms of performance. I implemented a non-blocking, generic command queue in Java here that uses Runnables and Callables as Commands. The principle is also explained more detailed here.
Extractor pattern
Extractor pattern is a term one never finds on the internet when searching for information about rendering. The first time I heard this, was when a (professional graphics programming) collegue of mine explained the basics of renderer architectures to me many many years ago. Fancy name for a simple thing: If you share data between two consumers (not exactly threads in this case), you either have to synchronize the access somehow, or you extract a copy of the data to pass it to the renderer in this case. That implies, that you need a synchronized/immutable command-like data structure, that represents all the information you need to push a render command. That plays nicely with the chapter above, you know, command queue etc. In languages other than Java where you have true value types, the implementation could be easier - although having very large renderstate objects copied over and over might be a drawback here. The chapter below shows another way of handling this problem.
Multibuffering
Let's assume that the GPU only consumes renderstate, but doesn't alter it. That means if the GPU calculates any fancy physic, or results that should be passed to the next render cycle, than it isn't seen as part of our traditional renderstate for now.
That means the only producer of renderable state is the CPU. Since we don't want to render a scene where the first half objects are in timestep n and the second half is still in n-1, we have to be able to access a state of our gamestate, that is concise somehow. This is easily achievable with a task-based architecture, that basically has only one update thread, that sequentially crunches down what you want to do in parallel. For example if you have 10 game objects, the updatethread pushes 10 update-commands into a pool of worker threads and after all of them returned a result, we have a concise world state we can use and used all threads our system can push. I don't think, there's an alternative approach to multithreading that can be used in a general purpouse game engine, but I may be wrong. Instead of creating a new renderstate object and pushing a command to the renderqueue, we use something that's known in multithreading environments for ages: multibuffering.
This should be confused with double or triple buffering that is used to display images on your monitor, although the principle behind it is the same.
I skip explaining why double buffering isn't enough to satisfy our needs. Just use triple buffering and be happy, if you can afford the additional memory consumption. This works the following way.
You have three instances of your renderstate. You have a wrapper, that encapsulates these three instances. Instance A is the currently readable state. The renderer uses this state for rendering, so while rendering is in progress, this instance mustn't be touched by anyone else than the renderer. Instance B is the current staging copy of the renderstate. It is the next state, that the renderer will use for the next frame, after the current frame is finished. Copy C is the isntance the update is currently applied on.
Now while this sounds simple, the important thing is, how to swap those instances correctly. Our main purpouse is to keep the GPU busy and to be able to render maximum fps, although one could theoretically (and practically) limit the framerate somehow (vsync, fps cap etc.).
I realized this with a ever-running thread that does push and wait for a render-scene-drawcall constantly. After the frame is finished, the read copy is swapped with the staging copy. It's important, that the renderer get's a fresh copy on every swap. If the update thread isn't fast enough to provide a new updated renderstate copy, the renderer would render a state that is older than the one just drawn. This will cause flickering objects. I realized this with an ongoing counter, and I prevent swapping to a state with a lower number then the current state has. Let's assume your engine and the machine is capable of pushing 60cps update and 60fps framerate.
Let's face the other swap: The update thread updates the write copy C constantly as fast as possible. When finished, the write state becomes the new staging state B, so we have to swap them. We can't do this if the renderer is currently swapping his read and the staging state. So we need a lock for swap(A,B) and a lock for swap(B,C). If rendering can be done with okayish framerates (which we assume), the updatethread can wait (block) until swap(A, B) is finished. If you are using only coherent memory access, than you're donw here. But you wouldn't search for state-of-the-art rendering, if you don't use incoherent access. So let's get to the final ingredient we need for our super fast renderer.
Lowering draw calls and persistent mapped buffers synchronization
At the very beginning, there is the need to reduce the amount of work the gpu has to do, in order to render your scene. There are paths thorugh the OpenGL API that are more expensive than others, and there are paths with very very small overhead, if you can afford the loss of flexibility that comes with using it. This can give you a rough overview about how costly API calls are. I suggest we skip a long explenation and you read this. Summed up: You want unsynchronized buffers for all your data and handle synchronization by yourself. This results in no driver or gpu work for the synchronization, which is crucial for maxing out your performance. Remember that Vulkan was designed to exactly kill the driver overhead of OpenGL. Furthermore, we reduce draw calls and state changes with bindless resources and one of the most important things: indirect and instanced rendering.
Having all these things implemented, there's one last thing to add to our triple buffering: synchronization. Since the driver works asynchronous and the gpu, too, there's no guarantee that the gpu doesn't use the buffer of our current write copy, that the update threads currently writes to. To adress this, one has to insert a fence sync object into the pipeline and associate it with the state, whenever a state becomes the current write state. When the render thread uses this thread and pushing all render commands of the current frame are pushed, the fence signal is pushed. Whenever this signal is processed by the gpu, the update thread can write to the associated state without any problems. The update thread can prepare the renderstate update and wait until the prepared data can be applied. This effectively blocks the update thread, if the renderer is more than 2 frames behind the update thread.
Special care has to be taken if you have any timestep-dependent calculations in your rendeirng step: Since rendering is decoupled form the update-step, you have to calculate the timestep by yourself somehow in the renderthread, based on the last tick and the time passed since then.
That's it
Using the fastest possible paths through the OpenGL API. You need shader
storage buffers and indirect (and inntanced) rendering, probably
combined with bindless textures
Model a renderstate class that can be your complete draw command
Run a thread permanently, that permanently pushes rendercommands
Translucency seems to be one of the new illumination features that every graphics engine has to provide these days. The effect is most noticeable on organics, like skin (ears for example) and plant leafs. For the latter, there's a nice and easy way to implement it - this solution only works for very thin objects. This means double sided triangles, like paper, curtains, or leafs. Other cases would be more complicated.
You should have a mechanism to support multiple material types in your deferred renderer already. Then you need to disable backface culling, or the back side of the object won't be rendered at all...Let's talk about direct illumination first. Light traveled through the object and direct light is treated at the same time. Since we assume that our translucent objects are infinitely thin, we don't need to determine a thickness, like you would have to, regularly. An artificial thickness can be provided via per-object parameter or through a texture. One could assume, that texture coordinates are the same for the point on the front and the back side of the currently rendered fragment. For normals, another assumption is possible: The normal on the back side should be the current fragment's normal, but just negated. Using different normals or diffuse texture for the backside is not easy to add here, since you can't know if you are currently rendering the front or the back side ob the object.
After calculating the regular lighting, calculate the lighting with the artificial backface normal und multiply it with the object's thickness at this point. Add the two values and you're done. Even though the usage is limited to very thin and simple-colored objects, this can look very nice, for example with a curtain:
Look at the subtle shadow of the sphere above the curtain that you can now see from below the curtain.
Last year, I wrote my masther's thesis about global illumination effects in realtime rendering engines. Since I haven't yet published any infos, because I didn't know how, I finally do it now. The result of the thesis was an implementation of a custom deffered rendering engine with LWJGL.
Addiotionally, I used something that is called boxprojection with cubemap datastructures. Furthermore, I modeled proxy objects (boxes) that I call probes - they are axis aligned bounding boxes with a corresponding cubemap, very much like a dynamic environment map. Instead of just one texture for a snapshot of the environment, I attached multiple textures for the exact same purpose that multiple textures are used for in G-Buffers for deferred rendering. Pre-Rendering all positions and material attributes of a whole level leaves us with the need to evaluate the lighting, when lighting conditions change. That is much cheaper than actually render the complete (environment) map again, like with traditional environment mapping. Additionally, the first and second pass shaders from the deferred rendering can be reused, instead of having additional forward shaders, as with regular environment mapping.
At this point, one would be able to use those environment maps for perfect reflection mapping. Can be used for all dynamic and static objects, reflects static objects with dynamic lighting. But that's not enough; there's something called realtime importance sampling, that could use prefiltered cubemaps, to have a complete lighting model covered - that means specular and diffuse reflections with arbitrary lighting models. One just has to calculate the mipmaps for the cubemaps and everything will work. Another possible tweak is to precalculate the radiance for each cubemap and save it different roughness values in the mipmaps. Have a look at Unreal Engine 4's implementation. Thats how I implemented it to avoid the costly importance sampling in trade of quality.
Since probes have to be placed within the level, on needs to blend between multiple probes. I implemented an algorithm from Sebastien Lagarde, similar to his approach in the game Remember Me.
The result is pretty nice global illumination in high framerates with the possibility to alternate probe update over time, stream probes etc. It can run on my GTX 770 with framerates above 150 fps in the sponza scene - depending on additional quality settings, like multiple bounces etc. Here's a screenshot of what could be done, again, completey dynamic lighting, static geometry. Dynamic geometry would take more ressources.
I will provide more info, if anybody is interested.
The classic bound texture paradigma is so 1990. But getting bindless textures to work is not that easy, especially when you're using something other than C or C++. For all the LWJGL or Java users out there, here are some notes that may help you removing the classic texture pipeline from your engine. I think you can read about the technique in the internet, so I'll keep it mostly short.
First of all, you need a handle (some kind of pointer again, but don't think of it as a pointer, yucks) for a given texture.
long handle = ARBBindlessTexture.glGetTextureHandleARB(textureID);
Afterwards, the handle has to be made resident. I think that's something you need for the combination with partially resident textures.
Inside your shader, you have to use the handle. But what datatype should one use? Maybe I missed something elementary, but there's only one proper way, namely use a datatype made available through an nvidia extension. Since bindless textures are available through an extension as well, here are both calls that you (probably) need:
And now, you can use tha datatype uint64_t. So your uniform would be a uint64_t.
That would work. But most probable, you want to have your data in a uniform or storage buffer, probably together with some other data and datatypes. So here's what I did.
Use a DoubleBuffer (Java native buffer, available via BufferUtils.createDoubleBuffer(int size)) for your data. Since doubles are twice the size of a float, which is 4 byte, we have 8 bytes per texture handle, which is 64 bits, which is the same as a uint64's size, so that's enough. Now one has to take the generated handle's bits and put them into the buffer (for example a ssbo) as they are. This can be done like so:
-
Where primitiveByteSize is 8 in this case. Since we use the underlying buffer as a DoubleBuffer, we have to provide a double value for the handle (or use it as a byte buffer, but nevertheless we need the correct bits or bytes). You can convert a Java long to and from a double like this:
Double.longBitsToDouble(longValue)
Double.doubleToLongBits(doubleValue)
Afterwars, the shader can take the value as a said uint64_t and cast it to a sampler. Sounds ugly, maybe it is.
color = texture(sampler2D(uint64_t(material.handleDiffuse)), UV);
That is the whole story, took me a while to figure it out.
Since I came across this problem and wasn't able to find an easy solution on the internet, I decided to write s small recipe to calculate normals when you want to do manual mipmapping/radiance convolution with cubemaps in OpenGL. I use compute shaders, so for geometry/vertex/pixel-pipeline, you could use layered rendering and other stuff.
First of all, the shader needs the current cubemap face index as a uniform variable. I recommend using the standard OpenGL indices (see link below).
Most likely, you are using the standard cubemap layout. If this is not the case, you have to change the vectors in my code. So with a given face index and a given texel position, the problem can be solved:
What happens here is that I calculate the pixel position in texture space with the help of the invoation position. The compute shader is invoked with cubemapfaceResolution.x/16, cubemapfaceResolution.y/16, 1. Knowing which (OpenGL) world axis the view direction of the virtual camera, facing the current cubemap side from the inside (cubemaps origin) is, the other two axis are the two orthogonal axes. These two axes' values grow with the texelcoordinates we already have. But therefore, they have to be remapped from 0 - 1 to -1 - 1. The resulting vector can be used to sample a cubemap as it is. Normalization could be unnecessary.
Recently, I had a lot of pleasure with OpenGL's compute shaders. With this lot of pleasure came a lot of pain because I made some (rookie) mistakes. So I wanted to share my experience and some advices I have, just in case you have troubles too:
The first thing you should check are your texture formats! No, really, double check it, don't repeat my mistakes. In your compute shaders, you could use your images (not textures) with
of OpenGL version 4.2 as an output texture. Of course you could use GL_READ_ONLY or GL_READ_WRITE if you use the texture differently. Also keep in mind that this call binds an image, not a texture. And that's why you have to provide a mipmap level you want to attach. I used the wrong format once, namely rgba32f, which my rendertarget attachments didn't have, and it resulted in non existent output from my compute shader. Very frustrating but correct behaviour.
Keep in mind that you could use your regular textures via samplers in your compute shaders, too. Simply bind the texture and have a similar line to this in your shader
layout(binding = 1) uniform sampler2D normalMap;
That's helpful if you want to access mip levels easily.
Since even in the OpenGL super bible is a typo that doesn't help to understand the compute shaders built-ins, I recapture them.
With dispatchCompute you have to provide three variables that are your group counts. A compute shader pass is done by a large number of threads and defining clever group counts/sizes will help you to process your data. In graphics related cases, mostly you will need compute shaders to render to a texture target. So it would be clever to have a certain, two-dimensional amount of threads, wouldn't it? Define your group sizes corresponding to your image size: a 320*320 image could be devided into 10*10 groups, or tiles - and each will have 32*32 pixels in it. So you should define your group size as 32, 32, 1. Now you can dispatch 320/group size threads, which will be 10 groups, for x and y dimension. In your shader, you will be able to use the built-in gl_WorkGroupSize to have this information in every invocation of your shaders main method. To uniquely identify your invocation, you can use the gl_GlobalInvocationID. If you use your shader like I said in this example, this would contain your texel's position the invocation would have to write. And that's how you can use compute shaders to manipulate your textures. Additionally, there is a gl_WorkGroupID, that identifies your tile/group of the invoation, and gl_LocalInvocationID, that is your pixels position in its tile. Sometimes, it could be useful to use a flattened identifier - for example if you have a task that requires performing an action just 12 times, but has to be done in the compute shader - and therefore you can use gl_LocalInvocationIndex. You can use it as a conditional to limit some code paths like
For a better understanding, have a look at this post, which has a nice picture and another explanation of the group layout.
What else? Compute shaders are awesome! I like how easy it is to invoke them, independent of something like the graphics pipeline. Use compute shaders!
I just want to share a small screenshot of my OpenGL rendering project. Includes physical based rendering, a global illumination concept daveloped by myself and realtime (glossy) reflections as you can see on the screenshot. Realtime of course - seen on my GTX 770 in full hd at max 300 fps.
Although transformation and class hierarchies in game engines are a topic for itself for sure, I finally arrived at a point where I just want every single of my regular game objects to be a transformable entity. Those objects that don't act as something that can be transformed are outside of my interest, they need some default behaviour I don't care about. I think that is the way the Unity engine took, too. A shared super class might look like a good idea, but we are often warned about such kind of class hierarchies.
While C++ offers multiple inheritance to implement things like this very easily, in languages like Java, you probably have to use composition - which should be favored over inheritance nevertheless. The problem is that sometimes, you get lost in interfaces and class fields... and see yourself write interface implementations again and again while delegating interface calls to field objects.
The last sentence is the catch-word: I tried to learn about Java 8's new stuff and stumbled across default methods. While mainly created to guarantee binary compatibility when changing interfaces, they offer a nice way to implement transformations within one file, without the need of (nearly) no other implementations or stuff. Here's how I did it:
I have a class called Transform. This holds a position, an orientation, a scale and is mostly a data holder. Additionally, I created an interface called Transformable. If this would be a class, I should have implemented the state (which now is in Transform class) in this class. But it's an interface. My gameobjects implement this interface, so I would have to implement all those move(vec3 amount)-etc-methods in this implementation. With default methods, I can now provide implementations on the interface tiself - combined with a pattern I don't know the name for anymore, this could be powerful: methods implemented by the interface can be called by the interface. This means I can use a non-default-implemented method getTransform() on the interface in my default methods.
For all classes that implement Transformable, it's sufficient to provide a transformation, because it needs getTransformation() to be implemented. That's because interfaces are not allowed to have state and one has to add the field to the implementing class.
Where this really shines is in situations where you would use (method)-pointers in C++: When you have an object besides your regular game objects that has to be a transformable, but is attached to another object that controls their transformation, you can implement the getTransform()-method with returning a field object's transform. Best example is for gameobjects that have a physics componentn attached, that should win every transformation war.
Additionally, I made the gameobject interface a subclass of my transformable interface, so that I can have different entity types for game objects, lights, environment probes etc. Some of them are not movable, like a directional light - therefore, I can override the directional light's move-methods to not do anything. And then, the subclass interface can call it's superclass interface's methods, too: For example the default implementation of the game object interface's isDirty()-method uses the superclass interface's hasMoved()- and its own animationHasPlayed()-method, if it's an animated entity.
So for the maximum price of a method call, that the interface has to do on itself when a transformation changes, you can have your transformations interfaced. My experience is, that I have very few problems with undesired class hierarchies in the means of "oh no, now I have to implement x or subclass y, but it's not clean code". As always, I'm still a bit uncertain about if this is good use of default methods. But at least I gave them a try and I don't regret my design descision - let me know why this stuff is bullshit, I'm curious :)
Bound ist ein erzählendes 2D-Jump'n'Run, das ich zusammen mit einigen Kollegen als Studienprojekt begonnen habe und nun als Hobbyprojekt weiterführe. Da es in den kommenden Posts für das ein oder andere Beispiel herhalten wird, ist es vielleicht schön mal ein Bild davon zu sehen.
Gemacht haben wir es mit XNA 4.0, verwenden Farseer Physics, Mercury Particles, Irrklang Audio und ... nein, das wars glaub ich. Die Engine haben wir von Grund auf selbst gebaut. Wir haben Animationsklassen, Physik-Objekte, ein Eventsystem, ein generisches Layer-System, haufenweise Effekte, Gamestate-Funktionalitäten... uuund einen überkrassen Editor, mit dem wir Level bauen und Spielabläufe skripten können (sc ist der King!). Unserer Kreativabteilung belieferte uns mit schicken Grafiken, Storyboards und schönen Rendersequenzen, die die Geschichte des Spiels erzählen. Unsere letzten Errungenschaften sind einige Effekte, wie die Godrays, die im Bild zu sehen sind und deren Implementierung ich wahrscheinlich im nächsten Post vorstellen werde, ein Lua-Hook zum Einbinden von Skripten.. und überhaupt ist immer alles im Umbruch und noch mehr im Kommen.
Spielen tut man übrigens die niedliche kleine Spinne, die ihr im Bild seht. Die Animationen sind der Hammer, vielleicht gibts irgendwann mal ein Video.
Normalerweise gibt es in Spielen nur lokale Beleuchtung, das heißt Licht wird für einzelne Punkte von Flächen ausgewertet. Dabei ist nur wichtig, ob ein Strahl einen Punkt trifft, so etwas wie eine Verdeckung, also eine Wechselwirkung von Flächen und Punkten untereinander spielt für dieses Modell keine Rolle. Wenn man aber mal in die echte Welt schaut, ist die Beleuchtung um einen Gegenstand, der auf einer ebenen Fläche steht und Beleuchtet wird nicht sauber, sondern in der nähe des Gegenstandes stellenweise dunkler. Mal ganz abgesehn davon, dass Flächen selten wirklich glatt sind - die meisten Materialien sind rau und uneben. Screen Space Ambient Occlusion ist deswegen aus vielen Gründen eine richtig feine Sache: Erstens ist es so einfach, dass sogar ich es ansatzweise verstanden habe (hoffentlich) und zweitens wertet es das Gesamtbild eines Videospiels als Pixelshader merklich auf, ohne dass die restliche Architektur des Spiels groß oder überhaupt angefasst werden muss - es funktioniert also mit fast jedem Spiel. Der dritte Grund ist, dass der Shader eventuell sogar noch um indirekte Beleuchtung (SSIL) erweitert werden kann, dazu aber in einem späteren Post mehr.
Wer verstehen möchte, wie man SSAO implementieren kann, der sollte Schritt für Schritt den Post lesen und versuchen den Code zu verstehen. Die Kommentare sollten dies einfach machen. Wie bei allem heutzutage gilt: Angucken und Können ist nicht - es braucht ein bisschen Zeit und Konzentration um zu verstehen und Übung um es zu implementieren. Wer von Computergrafik absolut keine Ahnung hat, der braucht nicht weiterlesen, weil Begriffe fallen werden, mit denen er nichts anfangen kann. Wer schon mal einen Shader geschrieben hat und ansatzweise versteht worum es geht, der kann mit den folgenden Absätzen sicherlich etwas anfangen.
Wie funktionierts?
Ich steh nicht so auf pure Theorie, daher ist meine Umgebung abgesteckt: XNA 4.0 und HLSL, weil ich damit grade n Projekt machen "musste". Wir brauchen sonst nicht viel, um SSAO mit HLSL zu implementieren: Den Depthbuffer, die Colormap und sowas wie ne Noisemap. Außerdem integrieren wir unsere Normalen. Das wars schon. Meine Implementierung basiert übrigens auf mehreren Internetblogs, zum Beispiel diesem hier. Grob gesprochen nehmen wir jeden Pixel des Bildes und betrachten den Tiefenwert der umliegenden, nahen Pixel, die wir in einer Halbkugel um die Normale des Pixels zufällig auswählen. Sind diese weiter vorne als der Pixel, ist davon auszugehen, dass der ursprüngliche Pixel räumlich hinter ihnen liegt, also von ihnen verdeckt wird. Je nach dem, wie groß die Strecke zwischen dem aktuellen Pixel und einem umliegenden ist, kann man hier eine Verdeckung sichtbar machen.
Schritt 1: Samples holen, Zufall schaffen
Es kommt natürlich ein bisschen darauf an, wie ihr euer Spiel baut - ich habe eine Deferred Rendering Engine aus irgendeinem XNA-Tuto nachgebaut und liefere meine Maps als Texturen an meine Posteffekt-Shader.
Als nächstes generieren wir Normalen, die wir später zur Reflektion verwenden. Hier kommt die Noisemap zum Einsatz. Für jeden Bildpunkt kriegen wir also eine zufällige Normale. Das Noise Sample kann noch ein zusätzliches Offset kriegen.
Meine Quelle speichert sich Fragmentkoordinaten etwas komfortabler ab, daher mach ich das mal nach. Was wir dann in einer Variable haben sind die UV-Screenkoordinaten, also Pixel nach x und y und den Tiefenwert im Bildschirmplatz...den Depthbuffer.
float3 ep = float3(input.TexCoord,currentPixelDepth);
Außerdem macht er den Radius wo wir samplen von der tatsächlichen Tiefe abhängig. Je größer die Tiefe, desto größer der Radius (glaube ich :) ). Wahrscheinlich, damit man einen stärkeren Effekt im Hintergrund hat, der ist ansonsten schlecht wahrnehmbar. Für rad wählt meine Quelle den Wert 0.006. Kann man machen, rumspielen hilft.
float radD = rad/currentPixelDepth;
Schritt 2: Die Schleife
Nun müssen wir für jeden Pixel des Bildes einige Schritte durchführen. Bevor wir die Schleife schreiben, definieren wir mal unsere benötigten Variablen und die Variable, wo wir das Ergebnis speichern.
float bl = 0.0f;
float occluderDepth, depthDifference, normDiff;
Wie oft die Schleife durchlaufen wird, muss man selbst entscheiden. Je mehr desto besser ist die Qualität des Effekts. Bei mehr als 16 Samples braucht man logisccherweise mehr Zufallsvektoren in pSphere.
for(int i=0; i<9;++i)
{
// Einen Vektor in der Kugel holen und reflectieren (mit dem Noise-Vektor)
float3 ray = radD*reflect(pSphere[i],fres);
// Wenn der Strahl aus der Halbkugel rausgeht, wird Richtung geändert
Der letzte Schritt dient, um den Verdeckungswert weich abfallen zu lassen. Wer genauer wissen möchte, was passiert, kann hier schauen. Für falloff wählt meine Quelle den Wert 0.000002. Hab ich genauso.
Schritt 3: Fertigmachen
Nach der Schleife wird der Verdeckungswert nochmal mit einem Steuerwert für die Stärke multipliziert und durch die Samples geteilt. strength ist bei "uns" 0.07, totStrength 1.38. Prinzipiell würde es reichen, wenn wir einen float zurückgeben, aber ich hab den Effekt später noch für was anderes nutzen wollen, daher is mein Rückgabewert etwas verschwenderisch.
float ao = 1.0-totStrength*bl*invSamples;
return float4(ao,ao,ao,ao);
Das war ja einfach
Das wars auch schon. Wieviele Zeilen Code waren das? 15? Diese Technik ist nicht völlig frei von Fehlern. Wenn man ein bisschen zu viel am Radius spielt, kriegt man seltsame Ergebnisse und die Nachteile des Viewspace (nur was auf dem Bildschirm sichtbar ist, wird beachtet...) lassen sich auch nicht verneinen. Ich hab bei zu großen Unterschieden bei den Tiefenwerten (?) ein hässliches Kantenbluten gehabt, das ich auch mit einem zusätzlichen Blur-Pass nicht wegbekommen habe (in den Bildern unten erkennbar). In XNA 4.0 hat man außerdem nur Shadermodel 3.0, das bedeutet limitierte Instruktionszahlen. Gerade wenn man den Effekt noch um indirekte Beleuchtung erweitern will, macht sich das sehr schnell bemerkbar. Ein zusätzlicher Blurpass sollte also fast schon eingeplant werden, wenn man SSAO irgendwie ins XNA bringen will. Ich habe hier jetzt keinen, das Ergebnis kann sich trotzdem sehen lassen:
mit SSAO
ohne SSAO
SSAO
Von meiner Seite aus ein allgemeines Danke an alle Leute bei Nvidia, in den Blogs, meine Quelle und auch sonst jeden, der Informationen zum Thema ins Internet gestellt hat, denn er hat diesen Post mit Sicherheit irgendwie beeinflusst.
Viel Spaß beim Nachmachen, für einen der kommenden Posts habe ich noch eine kleine, aber effektive Erweiterung des Shaders. Wenn irgendwas hier falsch oder zu ungenau ist, kommentiert einfach, die Chance ist hoch, dass ich mich Kommentaren annehme :)
