Real-time content creation with UE4 ray tracing【UE4 Ray Tracing Night Week 2020】

Real-time content creation with UE4 ray tracing Epic Games Japan / Technical Artist Developer Relations Hiroyuki Kobayashi

Ray Tracing in UE4.25 #UE4 | @UNREALENGINE

Ray Tracing in UE4.25 Production Ready ! #UE4 | @UNREALENGINE

Are you sure it’s okay? #UE4 | @UNREALENGINE

I tried to create real-time content using evry ray tracing function of UE4! #UE4 | @UNREALENGINE

Example: Train #UE4 | @UNREALENGINE

#UE4 | @UNREALENGINE

Ray tracing verification ● Ambient Occlusion ● Global Illumination ● Sky Light ● Reflection ● Translucency Verification environment RTX 2080ti Full HD #UE4 | @UNREALENGINE

Ray tracing verification Dynamic lighting is important because the train itself keeps moving. Is it okay to move it in the first place? Does the noise increase? #UE4 | @UNREALENGINE

Ambient Occlusion #UE4 | @UNREALENGINE

Ray Tracing Ambient Occlusion SSAO #UE4 | @UNREALENGINE Ray Tracing

Ray Tracing Ambient Occlusion SSAO Ray Tracing High quality and good performance. l though it depends on the resolution, the difference from SSAO is about 1 to 2 ms. #UE4 | @UNREALENGINE

What happens when move the train? #UE4 | @UNREALENGINE

Ray Tracing Ambient Occlusion What happens when the train moves? When stoped The noise increased a little. #UE4 | @UNREALENGINE When moved

Ray Tracing Ambient Occlusion What happens when move the train? Lit There is no problem because it is almost inconspicuous in the Lit mode. #UE4 | @UNREALENGINE Only AO

Global Illumination #UE4 | @UNREALENGINE

Ray Tracing Global Illumination Summer #UE4 | @UNREALENGINE Winter

But... #UE4 | @UNREALENGINE

Ray Tracing Global Illumination Looking at the processing load breakdown... #UE4 | @UNREALENGINE

Ray Tracing Global Illumination Looking at the processing load breakdown... Sky Light #UE4 | @UNREALENGINE Global Illumination Reflection

Ray Tracing Global Illumination Looking at the processing load breakdown... Sky Light #UE4 | @UNREALENGINE Global Illumination Reflection

Ray Tracing Global Illumination Looking at the processing load breakdown... Sky Light Global Illumination Reflection About 31ms!! #UE4 | @UNREALENGINE

It was heavy... #UE4 | @UNREALENGINE

Ray Tracing Global Illumination Brute Force Sample Per Pixel 4 Sky Light #UE4 | @UNREALENGINE Global Illumination Reflection

Ray Tracing Global Illumination Type of RTGI ● Brute Force ● Final Gather We can change from post process volume. #UE4 | @UNREALENGINE

How’s final gather? #UE4 | @UNREALENGINE

Ray Tracing Global Illumination Final Gather Sample Per Pixel 4 #UE4 | @UNREALENGINE

Ray Tracing Global Illumination Final Gather Sample Per Pixel 4 About 9ms!! #UE4 | @UNREALENGINE

But... #UE4 | @UNREALENGINE

Ray Tracing Global Illumination What happens when move the train? Brute Force #UE4 | @UNREALENGINE Final Gather

Ray Tracing Global Illumination What happens when move the train? Brute Force Final gather has problems with movemet #UE4 | @UNREALENGINE Final Gather

Ray Tracing Global Illumination What happens when move the train? Sample Per Pixel 4 It didn't change much even if I increased the number of samples. #UE4 | @UNREALENGINE Sample Per Pixel 64

Try lowering the brute force sample per pixel #UE4 | @UNREALENGINE

Ray Tracing Global Illumination Brute Force Sample Per Pixel 1 About 7ms！！ #UE4 | @UNREALENGINE

Ray Tracing Global Illumination Brute Force Sample Per Pixel 4 It's cleaner with a higher sample size. But not by much. #UE4 | @UNREALENGINE Sample Per Pixel 1

Ray Tracing Global Illumination Brute Force When stoped The noise stands out when you move. But not as much as Final Gather. #UE4 | @UNREALENGINE When moved

Ray Tracing Global Illumination Final Gather ● Fast ● Not good at movement Brute Force ● Slow ● not good at movement, but better than final gather #UE4 | @UNREALENGINE

Ray Tracing Global Illumination Final Gather ● Fast ● Not good at movement Brute Force Neither of them are good at movement! ● Slow ● not good at movement, but better than final gather #UE4 | @UNREALENGINE

What do we do then? #UE4 | @UNREALENGINE

Let's adjust the denoiser. #UE4 | @UNREALENGINE

Global Illumination Denoiser Denoiser related console commands are also available GI and Sky Light denoisers are common #UE4 | @UNREALENGINE

Global Illumination Denoiser r.GlobalIllumination.Denoiser.PreConvolution=2 ● Increasing the value suppresses noise to some extent, but gives an overall vague impression. r.GlobalIllumination.Denoiser.ReconstructionSamples=1 ● Noise jaggies can be suppressed by lowering the value #UE4 | @UNREALENGINE

Global Illumination Denoiser Adjust the denoiser PreConvolution=1 PreConvolution=2 ReconstructionSamples＝16 ReconstructionSamples＝1 Even when you move it, it doesn't stand out much! #UE4 | @UNREALENGINE

Sky Light #UE4 | @UNREALENGINE

Ray Tracing Sky Light Shadow related issues ● Masked material shadow ignores Opacity Mask ● Cast shadow for each material element does not work ● Sky Light looks different in reflection #UE4 | @UNREALENGINE

Ray Tracing Sky Light Masked material shadow ignores Opacity Mask Avoid with r.RayTracing.SkyLight.EnableTwoSidedGeometry = 0 Whether to enable shadows for double-sided materials Turn off because the shadows of plants etc. may become too dark #UE4 | @UNREALENGINE

Ray Tracing Sky Light Cast shadow for each material element does not work Avoid windowpanes that you do not want to cast shadows by making them separate objects #UE4 | @UNREALENGINE

Ray Tracing Sky Light Sky Light looks different in reflection r.RayTracing.Reflections.RayTraceSkyLightContribution=1 Draw skylight shadows in ray tracing reflection It doesn't look exactly the same without denoising, and it gets quite dark. No shadow #UE4 | @UNREALENGINE With shadow

Ray Tracing Sky Light What happens when move the train? When stoped Not as much as GI, but still stands out when moving #UE4 | @UNREALENGINE When moved

Ray Tracing Sky Light Adjust Denoiser Before adjustment Just like GI, adjust the denoiser to solve the problem! #UE4 | @UNREALENGINE After adjustment

Reflection #UE4 | @UNREALENGINE

Ray Tracing Sky Light What happens when move the train? When stoped There is a slight increase in noise, but there is almost no problem. #UE4 | @UNREALENGINE When moved

Ray Tracing Reflection Many RT Reflections do not look right ● AO ● Global Illumination ● Denoised Sky Light and Soft Shadow ● SubSurface Scattering etc #UE4 | @UNREALENGINE

Ray Tracing Reflection Many RT Reflections do not look right #UE4 | @UNREALENGINE

Ray Tracing Reflection The shadow of the Two Sided material turns pink #UE4 | @UNREALENGINE

Ray Tracing Reflection The shadow of the Two Sided material turns pink Solved by rewriting the shader code! #if MATERIAL_SHADINGMODEL_SUBSURFACE || MATERIAL_SHADINGMODEL_TWOSIDED_FOLIAGE uint ShadingModel = GetMaterialShadingModel(PixelMaterialInputs); if ( ShadingModel == SHADINGMODELID_SUBSURFACE || ShadingModel == SHADINGMODELID_TWOSIDED_FOLIAGE ) { Payload.CustomData = GetMaterialSubsurfaceData(PixelMaterialInputs); } #endif Engine\Shaders\Private\RayTracing\RayTracingMaterialHitShaders.usf Around line 385 #UE4 | @UNREALENGINE

Ray Tracing Reflection Before Scheduled to be fixed in 4.26 Thank you Monsyo and Nori! #UE4 | @UNREALENGINE After

Ray Tracing Reflection Sky Light shadow is not drawn. Too bright #UE4 | @UNREALENGINE

Ray Tracing Reflection Sky Light shadow is not drawn and it is too bright RayTraceSkyLightContribution=1 Before #UE4 | @UNREALENGINE or After

Ray Tracing Reflection Sky Light shadow is not drawn and it is too bright RayTraceSkyLightContribution=1 It gets pretty dark, but ... #UE4 | @UNREALENGINE or

Ray Tracing Reflection Sky Light shadow is not drawn and it is too bright RayTraceSkyLightContribution=1 or It doesn't bother me so much in lit mode #UE4 | @UNREALENGINE

Ray Tracing Reflection Soft shadows can also be selected, but they are rough without denoising #UE4 | @UNREALENGINE

Ray Tracing Reflection Soft shadows can also be selected, but they are rough without denoising Distorts normal or is inconspicuous if the material is rough to some extent Reflection #UE4 | @UNREALENGINE Lit

Ray Tracing Reflection Soft shadows can also be selected, but they are rough without denoising Distorts Normal or is inconspicuous if the material is rough to some extent Low Roughness #UE4 | @UNREALENGINE High Roughness

Ray Tracing Reflection I didn't use it this time, but there is also a reflection-related denoiser option #UE4 | @UNREALENGINE

Translucency #UE4 | @UNREALENGINE

Ray Tracing Translucency Many RT Reflections and Refraction do not look right ● AO ● Global Illumination ● Denoised Sky Light and Soft Shadow ● SubSurface Scattering etc #UE4 | @UNREALENGINE

Ray Tracing Translucency Refraction of RT Translucency There is a lot of information that does not appear accurately, and there is a lot of noise when moving it. #UE4 | @UNREALENGINE

Ray Tracing Translucency I didn't use Refraction Turn off Refraction and operate with Max Refraction Rays 1 #UE4 | @UNREALENGINE

Ray Tracing Translucency I didn't use Refraction It may be usable if it is about Prop with a small drawing area #UE4 | @UNREALENGINE

Ray Tracing Translucency Reflection of RT Translucency Can be used even when Refraction is off #UE4 | @UNREALENGINE

Ray Tracing Translucency What happens when move the train? When stoped Like opaque reflection, it doesn't bother you much when you move #UE4 | @UNREALENGINE When moved

Ray Tracing Translucency Nvidia RTX Branch https://github.com/NvPhysX/UnrealEngine/tree/RTX-4.25 Translucent drawing is rasterized, and there is also Hybrid Translucency where only the reflective part is ray-traced #UE4 | @UNREALENGINE

Ray tracing verification summary ● Ambient Occlusion ● Global Illumination ● Sky Light ● Reflection ● Translucency #UE4 | @UNREALENGINE

Ray tracing verification summary Ambient Occlusion ● Excellent quality and performance #UE4 | @UNREALENGINE

Ray tracing verification summary Global Illumination ● Slow performance ● Weak to fast movements ● Real-time is also possible by adjusting the denoiser #UE4 | @UNREALENGINE

Ray tracing verification summary Sky Light ● Beware of shadow-related issues ● Weak to fast movements, but improved by adjusting the denoiser like GI #UE4 | @UNREALENGINE

Ray tracing verification summary Reflection ● Know what isn't exactly reflected in the reflection ● Adjust Normal and Roughness for areas where roughness is noticeable #UE4 | @UNREALENGINE

Ray tracing verification summary Translucency ● Know what isn't exactly reflected in Reflection and Refraction ● It may be difficult to use Refraction in a large area ● We can also turn off Refraction and use only Reflection #UE4 | @UNREALENGINE

A completely dynamic scene has been created! RTX2080ti 1920 x 1080 About 30fps #UE4 | @UNREALENGINE

Let's move the light #UE4 | @UNREALENGINE

Lighting with Sun and Sky #UE4 | @UNREALENGINE

Sun and Sky Simulate lighting from coordinates and time Sun and Sky documentation https://docs.unrealengine.com/ja/Engine/Rendering/LightingAndShadows/SunSky/index.html #UE4 | @UNREALENGINE

Sun and Sky Freely change the time and seasons! * Winter assets are required to create a snowy landscape #UE4 | @UNREALENGINE

Example: Jelly Fish #UE4 | @UNREALENGINE

#UE4 | @UNREALENGINE

Verification content ● Ambient Occlusion ● Global Illumination ● Sky Light ● Translucency ● Reflection #UE4 | @UNREALENGINE

Verification content ● Ambient Occlusion ● Global Illumination ● Sky Light ● Translucency ● Reflection #UE4 | @UNREALENGINE Not used this time

Why did se reflection for jellyfish? #UE4 | @UNREALENGINE

Because I make jellyfish ... ● I want to express a soft sheer feeling like frosted glass ● I want to use DoF #UE4 | @UNREALENGINE

Because I make jellyfish ... The internal organs can be seenthrough vaguely I want to reproduce this #UE4 | @UNREALENGINE

Translucent verification I want an effect like the Roughness of RT Reflection Roughness 0.0 #UE4 | @UNREALENGINE Roughness 0.25

Translucent verification I want an effect like the Roughness of RT Reflection Unfortunately it is not possible with ray tracing translucency #UE4 | @UNREALENGINE

Translucent verification I want an effect like the Roughness of RT Reflection If you let the reflection ray penetrate, it will look translucent ...? #UE4 | @UNREALENGINE

How to get through Ray #UE4 | @UNREALENGINE

Tweak normals in materials #UE4 | @UNREALENGINE

Translucent verification Make it translucent using RT Reflection By using the inverted camera vector as the normal ... #UE4 | @UNREALENGINE

Translucent verification Make it translucent using RT Reflection Tangent Space Normal needs to be turned off #UE4 | @UNREALENGINE

Translucent verification Make it translucent using RT Reflection Ray flies almost in the opposite direction #UE4 | @UNREALENGINE

Translucent verification Make it translucent using RT Reflection Roughness and Normal are devised to make it look like frosted glass #UE4 | @UNREALENGINE

Translucent verification It looks transparent, but it is actually treated as an opaque reflection It is drawn firmly in Depth #UE4 | @UNREALENGINE

Translucent verification It looks transparent, but it is actually treated as an opaque reflection It is drawn firmly in Depth DoF can be used! #UE4 | @UNREALENGINE

I was able to express the transparency and DoF I was aiming for! #UE4 | @UNREALENGINE

Executable file & project is open to the public Executable file https://drive.google.com/file/d/1N7K9O1vj6ypwsAB_v6m5k4E2RzPmR5A9/view?usp=sharing Project https://drive.google.com/file/d/1nlYv0YoLzNXgeU1sN6BSt5CrB9FKO9M1/view?usp=sharing #UE4 | @UNREALENGINE

Is it possible to create real-time content using evry ray tracing function of UE4? #UE4 | @UNREALENGINE

Did it! #UE4 | @UNREALENGINE

Understand the advantages and disadvantages of ray tracing and use it appropriately for each project #UE4 | @UNREALENGINE

Advantages and disadvantages of creating ray tracing content with UE4 Advantages ● Dynamic change in lighting You can easily simulate date and time and seasonal changes by combining with Sun and Sky Can handle lighting on moving objects such as trains and airplanes ● Physical accuracy Reflection and GI are now accurate and dynamic, enabling expressions that were previously difficult ● Iteration speed No need to bake, so of course no Light Map From modeling to checking lighting is much faster #UE4 | @UNREALENGINE

Advantages and disadvantages of creating ray tracing content with UE4 Disadvantages ● Processing load It is considerably heavier than the conventional functions, including GI Depending on the scene, even conventional expressions may be able to produce sufficient quality ● Unsupported part There are parts that I am not good at, such as reflection and translucent reflection ● Noise basically increases when movement is added I was moving it late this time, but the GI noise was still severe If it moves faster, another response may be necessary #UE4 | @UNREALENGINE

Thank you! #UE4 | @UNREALENGINE