Stylized Rendering Insights from Japan (Unreal Fest Gold Coast 2023)

39.4K Views

December 01, 23

スライド概要

Stylized Rendering Insights from Japan
Nori Shinoyama (篠山範明)

There are so many great games in Japan using stylized rendering made with Unreal Engine. In this presentation, we will briefly introduce and outline how different kinds of stylization can be achieved. We will also share tips on combining stylized rendering with new UE5 features such as Nanite and Lumen. You will learn what you need to know when creating unique stylized graphics in UE5.

profile-image

Unreal Engineを開発・提供しているエピック ゲームズ ジャパンによる公式アカウントです。 勉強会や配信などで行った講演資料を公開しています。 公式サイトはこちら https://www.unrealengine.com/ja/

シェア

埋め込む »CMSなどでJSが使えない場合

関連スライド

各ページのテキスト
1.

Stylized Rendering Insights from Japan Nori Shinoyama Epic Games Japan, Developer Relations, Lead

2.

There are many examples of Stylized rendering using Unreal Engine in Japan.

3.

Game

6.

Anime

7.

No Guns Life: Cyclone Graphics turns comics into animation with real-time rendering

9.

Comic / Manga

11.

Through a lot of support of stylized rendering on UE4… Epic Games Japan has supported a lot of games using stylized rendering ● I want to make this kind of unique graphics, but how do I make it with UE? ● Do I need to modify the engine? ● Performance! Performance! Performance!

12.

What to talk about today ● Basic techs of stylized Rendering ● What are the main concerns of many Japanese professionals when using stylized rendering ● What to expect when using stylized rendering with the new features of UE5

13.

What not to talk about today ● Things related to platforms such as PS, Switch, XBox, etc. ● Design and other art-related topics ● Non-realtime rendering use-cases such as comics or movies ● Mainly focus on games

15.

A japanese word I want you to remember in this session ダウンロード

16.

A japanese word I want you to remember in this session ダウンロード = download

18.

Who I am? ● Nori Shinoyama ● ● Epic Games Japan, Developer Relations, Lead ● Twitter: @tempkinder ● International Sake Sommelier

19.

Batten Girls - OiSa -

20.

Basic Elements of Stylized Rendering

21.

Basic Elements of Stylized Rendering Post Processing Outline Shading ©2019 Bandai Namco Online Inc. ©2019 Bandai Namco Studios Inc.

22.

Outline

23.

Outline: two basic methods Inverted Hull Post Processing Image courtesy of Tango Gameworks and Bethesda Softworks aaaaaa

24.

Outline method: Inverted Hull ● Preparing a mesh for outlines, then superimposing it on the original mesh ● ● Extruding each vertex of the outline mesh to each normal direction By erasing front surfaces of the outline meshes, the outlines can appear. outline

25.

Outline method: Inverted Hull ● What titles use it? ● Many titles reproducing comics and anime characters ● Advantage ● Can be adjusted for each characters ● Outline width / color ● Unique expressions ● Tips ● Some outlines are written directly on the textures, and used in combination with other outline methods Image courtesy of Tango Gameworks and Bethesda Softworks

26.

Outline method: Inverted Hull Make front faces transparent Extrusion in the normal direction for the thickness of the outlines

27.

Outline method: Inverted Hull (UE4) ● In UE4, you had to have an outline mesh and synchronize animations Two SkeletalMesh Components in a BP AnimBP for Outline Mesh Need to copy the original character animation to the outline mesh

28.

Outline method: Inverted Hull (UE5) ● Translucent Overlay Material feature was added since UE5.1 ● ● You don’t need to make a custom BP and an animation BP anymore for the inverted hull method You just need to assign the material for Outline to “Overlay Material” in MeshComponent ! ● … Translucent ???

29.

Tips: Translucent Overlay Material ● Can also be used for Opaque / Masked Materials ● ● ● The function is just to render the same mesh with the overlay material It is not limited to translucency (But you might need to WPO / PDO to avoid depth fight)

30.

Outline method: Inverted Hull (Caveat) ● GPU cost ● ● Vertex Shader Cost ● Computation of double the number of vertices Pixel Shader Cost ● Opacity (Mask) need to be calculated on all surfaces of the outline mesh

31.

Outline method: Inverted Hull (Caveat) ● If you don’t want to use a Twosinged node in order to reduce the Pixel Shader cost ● Option 1: Using a model with inverted normals for the outline mesh ● Can be rendered using the same method as UE4’s inverted hull ● Unfortunately, Reversing skeletal mesh normals isn’t possible on UE and must be done on another DDC tool such as Maya ● Option 2: Setting up front face culling for outline meshes ● It only allows back faces of the outline meshes to be drawn and the front faces are discarded in rasterization ● Static mesh component has Reverse Culling option ● Unfortunately, skeletal meshes does not have Reverse Culling setting ● You may need to create your own Mesh Component (and SkeletalMeshSceneProxy, etc.) or modify the Engine code for Reverse Culling of skeletal mesh

32.

Outline method: Post Process

33.

Marketplace ● There are many post processes for outlines in the Marketplace! Try them out!

34.

Edge Detection: Basic Algorithm ● Detect edges from the difference with neighboring pixels in the source images ● ● ● G-Buffers can be used as a source image There are a variety of edge detection filters ● Sobel Filter / Prewitt filter / Laplacian filter / RobertsCross filter… Complicated or special edge detection filters are hardly used for games (as far as I can see) ● Seems to be mainly due to processing load, not due to implementation difficulties -1 0 +1 -1 -1 -1 +1 0 -2 0 +2 -1 +8 -1 0 -1 -1 0 +1 -1 -1 -1 Sobel Source Images Color? Depth? Normal? Laplacian Filtering 0 +1 -1 0 Roberts Cross Edges

35.

Case Study: Valkyrie Elysium [UNREAL FEST WEST '22] Graphics and Optimizations Tips of Valkyrie Elysium

36.

Case Study: Valkyrie Elysium ● Source Images for Character Outlines ● ● ● ● Normal Boundary with Background Depth Material ID? ● By modifying their Engine [UNREAL FEST WEST '22] Graphics and Optimizations Tips of Valkyrie Elysium

37.

Case Study: Valkyrie Elysium ● Source images for BG Outlines ● ● Depth Shading Model [UNREAL FEST WEST '22] Graphics and Optimizations Tips of Valkyrie Elysium

38.

Outline method: Post Process (Caveat) ● Post-process for outlines tends to be heavy on the GPU … Many source textures -1 0 +1 -2 0 +2 -1 0 +1 A lot of texture reads Can be a very complex and heavy loaded postprocess!! Lots of code to accommodate various edge cases

39.

Outline: two basic methods Inverted Hull Post Processing Image courtesy of Tango Gameworks and Bethesda Softworks

40.

Stylized Shading

41.

Stylized Shading: Basic Idea ● In order to achieve unique stylized shading, you need to modify or override the existing UE’s shading system ● LUTs (Look Up Tables) are widely used for stylized rendering

42.

Stylized Shading: Basic Idea Q. How should the stylized shading be implemented? Material Lighting (Shading Model) Post Process

43.

Stylized Shading: Basic Idea Using Materials? ● Method ● By passing lights’ information into materials via Material Parameter Collection(MPC) ● Pros ● Artists can precisely customize materials for each object ● Cons ● Material Might not get any benefit from the various lighting and rendering systems of UE ● Shading, Shadowing, etc. Lighting (Shading Model) Post Process

44.

Stylized Shading: Basic Idea Using Post Process? ● Method ● Using various data in G-Buffer ● Pros ● Can easily change the whole scene ● Cons ● Material Difficult to adjust shading of each object or character Lighting (Shading Model) Post Process

45.

Stylized Shading: Basic Idea Q. How should the unique shading be implemented? A. Many games are using modified engines a. Custom G-buffer layout and/or additional buffers b. Custom shading model c. Custom post process (using the result from the above customizations) Material Lighting (Shading Model) Post Process

46.

Case Study: THE IDOLM@STER STARLIT SEASON [UNREAL FEST EXTREME '22 SUMMER] Graphics Implementation and Optimizations in "THE IDOLM@STER STARLIT SEASON"

47.

Case Study: THE IDOLM@STER STARLIT SEASON [UNREAL FEST EXTREME '22 SUMMER] Graphics Implementation and Optimizations in "THE IDOLM@STER STARLIT SEASON"

48.

Case Study: THE IDOLM@STER STARLIT SEASON ● G-Buffer Customization ● Shading Model Customization [UNREAL FEST EXTREME '22 SUMMER] Graphics Implementation and Optimizations in "THE IDOLM@STER STARLIT SEASON"

49.

Case Study: THE IDOLM@STER STARLIT SEASON [UNREAL FEST EXTREME '22 SUMMER] Graphics Implementation and Optimizations in "THE IDOLM@STER STARLIT SEASON"

50.

Case Study: Hi-Fi RUSH Image courtesy of Tango Gameworks and Bethesda Softworks Hi-Fi RUSH" Development Case Study Graphics and Optimization Introduction | UNREAL FEST 2023 TOKYO

51.

Case Study: Hi-Fi RUSH Hi-Fi RUSH" Development Case Study Graphics and Optimization Introduction | UNREAL FEST 2023 TOKYO

52.

Other tips for stylized shading ● Each game has various implementations for its own shading. ● Case Study ● Shin-megami tensei ● Blue Protocol ● No More Heroes 3

53.

Case Study: Shin Megami Tensei V CGWORLD: 原画イラストの再現を目指した個性的なキャラクターと悪魔たち『真・女神転生V』

54.

Case Study: Shin Megami Tensei V CGWORLD: 原画イラストの再現を目指した個性的なキャラクターと悪魔たち『真・女神転生V』

55.

Case Study: Shin Megami Tensei V ● Shadow Mask CGWORLD: 原画イラストの再現を目指した個性的なキャラクターと悪魔たち『真・女神転生V』

56.

Case Study: Shin Megami Tensei V ● Unique hair anisotropic reflex (a.k.a. angel's ring)

57.

Case Study: Shin Megami Tensei V ● Normal Transfer ● ● ● Another way to achieve unique shading Left: original normal Right: applied a normal map

58.

Case Study: Shin Megami Tensei V ● CGWORLD translated the article into English for this presentation!!!!!! ● Unique characters and demons aimed at reproducing the original illustrations "Shin Megami Tensei V"

59.

Case Study: Blue Protocol

60.

Case Study: Blue Protocol ©2019 Bandai Namco Online Inc. ©2019 Bandai Namco Studios Inc.

61.

Case Study: NO MORE HEROES 3 ©Marvelous Inc. Developed by Grasshopper Manufacture Inc.

62.

Case Study: NO MORE HEROES 3 ● Fresnel node for pseudo-rim lighting and edge enhancement

63.

Case Study: NO MORE HEROES 3 ©Marvelous Inc. Developed by Grasshopper Manufacture Inc.

64.

Stylized Post Processing

65.

Custom Post Process ● We’ve already seen some post processes in the Outline and Shading sections ● In addition to those, there are some interesting filters that will greatly change the image ● ● ● Postarization ● Bilateral Filter ● Kuwahara Filter Hatching etc

66.

Kuwahara Filter

67.

Hatching

68.

Custom Post Process ● But, honestly, most of the game devs I’d asked told us… ● ● They’d implemented and tried many filters in the early stages of their development But they didn’t end up with most of the filters in their final products ● Reason: no one-fits-all solution! ● ● ● ● Difficult or very time-consuming to adjust for different situations Can reduce visibility Can affect the design of icons, characters, fonts, etc. Constant GPU cost ● Nevertheless, here are some examples of titles that partially utilize stylized post processes

69.

Case Study: NARUTO TO BORUTO Shinobi Striker

70.

Case Study: NARUTO TO BORUTO Shinobi Striker You can see the hatching effect in the shadows at the feet from “NARUTO TO BORUTO Shinobi Striker” Official Site

71.

Case Study: Hi-Fi RUSH Image courtesy of Tango Gameworks and Bethesda Softworks Hi-Fi RUSH" Development Case Study Graphics and Optimization Introduction | UNREAL FEST 2023 TOKYO

72.

Hi-Fi RUSH Added hatching effect into Ambient Occlusion pass Hi-Fi RUSH" Development Case Study Graphics and Optimization Introduction | UNREAL FEST 2023 TOKYO

73.

Case Study: Valkyrie Elysium [UNREAL FEST WEST '22] Graphics and Optimizations Tips of Valkyrie Elysium

74.

Case Study: Valkyrie Elysium Custom post-processing to add various colors to shadow areas [UNREAL FEST WEST '22] Graphics and Optimizations Tips of Valkyrie Elysium

75.

Case Study: Valkyrie Elysium Custom post-processing to add various colors to shadow areas [UNREAL FEST WEST '22] Graphics and Optimizations Tips of Valkyrie Elysium

76.

Tips: Custom Post Process ● Default post-processes of UE can be used with Stylized Rendering

77.

Case Study: Demon Slayer Kimetsu no Yaiba The Hinokami Chronicles

78.

Case Study: Demon Slayer Kimetsu no Yaiba The Hinokami Chronicles CyberConnect2 Effects Development Case Study (Unreal Fest Japan 2022) They talked about Demon Slayer Kimetsu no Yaiba The Hinokami Chronicles

79.

Case Study: Demon Slayer Kimetsu no Yaiba The Hinokami Chronicles Capture the sense of movement and speed from japanese animation with high-quality graphics UE’s default depth of field and bloom are used without any modifications CyberConnect2 Effects Development Case Study (Unreal Fest Japan 2022)

80.

Is it difficult to use stylized post processes that significantly affects the entire screen? As I said, such kinds of filters are rarely used in the final products However, there are great demands for them and many people are trying them! Today, I would like to share with you an great example from the Japanese community.

82.

3Dnchu’s Stylized Post Processing ● This project is available! ● ● https://yamato3d.gumroad.com/l/StylizedPP-UE5 https://www.youtube.com/watch?v=NJjwrQialp8&t =1s ● Who 3Dnchu is? ● ● ● ● ● 3D人 - 3Dnchu twitter: @ymt3d Professional game developer runs one of the most popular news site about computer graphics in Japan ● https://3dnchu.com/ great guy who also shares his techniques with the public, such as this stylized PP Youtube Project

83.

Insights from Japan

84.

To make the graphics more unique… ● Prefer the combination of simple methods to complicated and highly technical algorithms ● Artist-driven, one by one and steadily built. ● The 2% visual error is more noticeable than the rest of 98% stable graphics ● A lot of work you need to mitigate the 2%

85.

Engine Customization ● a lot of devs customized rendering code in their engine to fit their graphics requirements ● Where do they customize? ● ● ● ● Mesh Component G-buffer Shading Model Complicated Multi Pass Post Processing

86.

Performance and Optimization ● Stylized rendering may make the picture look simpler than Realistic rendering, But it can require a lot of GPU resources ● Many materials and post-processes created in an artist-driven manner ● Be sure to profile regularly from the early stages of development! ● Ideally, have a graphics engineer regularly check your scenes

87.

Golden Rule: profiling first, optimization second ● There is a lot of information on optimization in the world ● Polycount! Texture Resolution! Shader Complexity!! ● But they may not be effective for your projects ● Basically, always profile before doing any work that reduces the quality of the object or takes up the artist's time! ● MUST regularly profile your game on your target platforms ● Profiling and optimization in the end of dev cycle is not kawaii

88.

Stylized Rendering with UE5 Rendering Features

89.

Stylized Rendering with UE5 Rendering Features ● UE5 Rendering Features ● ● ● Nanite Lumen Temporal Super Resolution

90.

Nanite with stylized rendering? ● Of course, you can use it! ● Caveats ● ● Translucency Overlay Material not yet supported by Nanite Nanite doesn’t support vertex colors. You cannot put any special data into each vertex of Nanite meshes. So you also cannot store per instance data into vertex color Nanite:ON Nanite:OFF (with Overlay Material)

91.

Nanite with stylized rendering? ● Q. Our graphics is highly stylized so we don’t need to have high poly meshes. Still Nanite is needed? ● A. If you want to use VSM or Lumen, as many objects as possible should be Nanite from a perspective of GPU performance

92.

Lumen with Stylized Rendering? Lumen: Off Lumen: On Need to make sure if Lumen fits the expression you want to achieve But you also need to understand the general caveats of Lumen

93.

Lumen with Stylized Rendering? You need to make sure whether your stylized materials works in the Lumen Scene or not

94.

Lumen with Stylized Rendering? It is very difficult to do lighting with Emissive only. Here is example of strong delay during camera switching.

95.

TSR with Stylized Rendering? ● Temporal Super Resolution is UE5’s powerful anti aliasing and upscaling technology ● There may be a few problems on the combination of TSR and Stylized Rendering

96.

TSR with Stylized Rendering? Strong Blur when the camera moved

97.

TSR with Stylized Rendering? ● If such a blur occurs in UE5.1 or later ● ● Try to set the Console variable r.TSR.ShadingRejection.Flickering to 0 It may not be a complete improvement, but it will get better to some extent

98.

TSR with Stylized Rendering? Noise texture scrolling Ghosting… Texture Animation Ghosting…

99.

TSR with Stylized Rendering? I posted an article about this TSR issue on the Epic Developer Community An English version will be available...

100.

Stylized Rendering Insights from Japan Nori Shinoyama Epic Games Japan, Developer Relations, Lead

101.

Summary ● The well-known old techs are still used for stylized rendering, However, they can be extended and combined to meet the requirements of the game. ● There is a lot of information on Stylized Rendering in Japan Please take a look if you are interested! ● Understand the pros and cons of UE5's new features and use them to express yourself uniquely!

103.

Thank you for listening! OiSa! https://www.youtube.com/watch?v=m3w2Lw--G0s