Press Button, Drink Coffee : An Overview of UE4 build pipeline and maintenance【CEDEC 2019】

13.4K Views

October 25, 19

スライド概要

How to build build pipeline for your Unreal Engine 4 game, along with iteration advice and best practices.

Originally presented in Poznan, Poland for GIC 19.

profile-image

Unreal Engineを開発・提供しているエピック ゲームズ ジャパンによる公式アカウントです。 勉強会や配信などで行った講演資料を公開しています。 公式サイトはこちら https://www.unrealengine.com/ja/

シェア

埋め込む »CMSなどでJSが使えない場合

関連スライド

各ページのテキスト
1.

「Press Button, Drink Coffee」 An overview of UE4 build pipeline and maintenance

2.

Introduction Axel Riffard Epic Games Japan Support Engineer ● French 🇫🇷 ● Twitter : @AxRiff ● Hobbies : Whisky, Cats and Disneyland ● Currently Playing: Dragon Quest XI, COD Mobile, The Witcher III (Switch) #UE4 | @UNREALENGINE

3.

FIRST OF ALL … #UE4 | @UNREALENGINE

4.

HELL YEAH POLAND ! #UE4 | @UNREALENGINE

5.

Ruiner ! Frostpunk ! Dead Island ! Dying Light ! The Vanishing of Ethan Carter ! And … #UE4 | @UNREALENGINE

6.

Geralt of Rivia ! <3 #UE4 | @UNREALENGINE

7.

Ok, back on topic #UE4 | @UNREALENGINE

8.

#UE4 | @UNREALENGINE

9.

Disclaimer There is no such thing as THE perfect pipeline. Rely on experience ! In this session, I will only talk to you about mine and give you a few best practices. But donʼt trust me, trust yourself ! #UE4 | @UNREALENGINE

10.

Agenda ● CI : What is it ? ● UnrealAutomationTool ● Building the Engine and your Project ● The other stuff ● The Unreal Way #UE4 | @UNREALENGINE

11.

Agenda ● CI : What is it ? ● UnrealAutomationTool ● Building the Engine and your Project ● The other stuff ● The Unreal Way #UE4 | @UNREALENGINE

12.

#UE4 | @UNREALENGINE

13.

Why ? Dev 1 Dev 2 Dev 3 #UE4 | @UNREALENGINE Versioning System

14.

Consequences ● Local build breaks after updating ● Finding the bad commit wastes time ● Communication issues, fights in the team ● Non technical members are scared to commit things ● Building locally every time is a waste #UE4 | @UNREALENGINE

15.

CI Software CI = Continuous Integration A combination of software and hardware allowing to merge the team source code and assets several times every day. #UE4 | @UNREALENGINE

16.

Why ? Dev 1 Dev 2 Dev 3 #UE4 | @UNREALENGINE Versioning System

17.

Software #UE4 | @UNREALENGINE

18.

CI Software In 2019, there is no bad choice. I have worked with both TeamCity and Jenkins, and was satisfied with both of them #UE4 | @UNREALENGINE

19.

CI Software Jenkins #UE4 | @UNREALENGINE TeamCity

20.

Jenkins #UE4 | @UNREALENGINE

21.

Jenkins Merits︓ ● Free Open Source ● Plenty of documentation and plugins ● Huge Community Drawbacks︓ ● Commercial Support exists but is complicated ● Some plugins are unreliable #UE4 | @UNREALENGINE

22.

TeamCity #UE4 | @UNREALENGINE

23.

TeamCity Merits : ● Building pipelines is crazy fast ● Technical support is great Drawbacks : ● Limitations on the Free version #UE4 | @UNREALENGINE

24.

CI Software Every advice can be applied to TeamCity, or any CI solution. Also, totally unrelated, but this cat looks like Jenkins mascot, so Iʼm going to use him instead. #UE4 | @UNREALENGINE

25.

Hardware #UE4 | @UNREALENGINE

26.

Build Machine Master → Slave → #UE4 | @UNREALENGINE Manages the tasks. Does the tasks.

27.

Build Machine DEV PC Commit Versioning Versioning Poll CI Server Master Slave 1 #UE4 | @UNREALENGINE Slave 2 Slack Distribute Slave 3 Company Server

28.

Build Machine – How many Prototyping Stage : ● Just the Master Production Stage : ● One Master ● One slave for every release platform #UE4 | @UNREALENGINE

29.

Build Machine - OS Master︓Win64 or Linux Slave︓Win64(for platforms SDK), MacOS for Apple Platforms #UE4 | @UNREALENGINE

30.

Build Machine - Windows Storage︓SSD Memory ︓32GB or more CPU︓i7 / i9 #UE4 | @UNREALENGINE

31.

Build Machine - MacOS No C++ and Shader compiling distribution on Mac, So get as many cores as you can for your CPU. If itʼs just for a build machine, no need for GPU. High spec Mac Mini is good enough. #UE4 | @UNREALENGINE

32.

Agenda ● Continuous Integration : What is it ? ● UnrealAutomationTool ● Building the Engine and your Project ● The other stuff ● The Unreal Way #UE4 | @UNREALENGINE

33.

Building in UE4 ● ● ● ● Engine and Editor Game sources Game Assets (lights, shaders, etc.) Tools #UE4 | @UNREALENGINE

34.

Build Commands Find all the command lines here https://github.com/AxRiff/UnrealEngineJenkins #UE4 | @UNREALENGINE

35.

UnrealAutomationTool UnrealAutomationTool is a complete system to build and test your games inside of Unreal Engine UE4 is mainly written in C++, but UnrealAutomationTool is made in C# #UE4 | @UNREALENGINE

36.

UnrealAutomationTool Depending on your company size, type of project, workflow, etc., you can use different parts of UnrealAutomationTool ● BuildGraph Custom Script ● BuildGraph InstalledBuild Script ● BuildCookRun Command Line Tool #UE4 | @UNREALENGINE

37.

BuildGraph #UE4 | @UNREALENGINE

38.

BuildGraph BuildGraph is a script based build automation system inside of UnrealAutomationTool ● XML ● By Epic for Epic ● Original approach #UE4 | @UNREALENGINE

39.

BuildGraph Youʼve got plenty of sample codes in Engine/Build/Graph/Examples/ #UE4 | @UNREALENGINE

40.

BuildGraph Merits : ● Reduce dependency on CI framework. ● Work on it locally -> No more Jenkins master ● XML Based : easy to read and write ● Platform Agnostic ● Can be shared between projects Drawbacks : ● Steep learning curve at first #UE4 | @UNREALENGINE

41.

BuildGraph ● Agents: Group of nodes being executed on the same build machine ● Nodes: Sequence of Tasks ● Tasks: Build Process Step (Updating, Compiling, etc) #UE4 | @UNREALENGINE

42.

BuildGraph <Agent Name="Default Agent" Type="CompileWin64"> <!-- Update the engine version files --> <Node Name="Update Version Files"> <SetVersion Change="$(Change)" Branch="$(EscapedBranch)" Licensee="$(Licensee)" If="$(Versioned)"/> </Node> <!-- Compile the tool executables --> <Node Name="Compile Tools Win64" Requires="Update Version Files" Produces="#ToolBinaries"> <Compile Target="UnrealHeaderTool" Platform="Win64" Configuration="Development" Tag="#ToolBinaries"/> </Node> <!-- Compile the editor executable --> <Node Name="Compile $(EditorTarget) Win64" Requires="Compile Tools Win64" Produces="#EditorBinaries"> <Compile Target="$(EditorTarget)" Platform="Win64" Configuration="Development" Tag="#EditorBinaries"/> </Node> <!-- Compile the game targets --> <Property Name="GameBinaries" Value=""/> <ForEach Name="GameTarget" Values="$(GameTargets)"> <ForEach Name="TargetPlatform" Values="$(TargetPlatforms)"> <Node Name="Compile $(GameTarget) $(TargetPlatform)" Requires="Compile Tools Win64" Produces="#GameBinaries_$(GameTarget)_$(TargetPlatform)"> <Compile Target="$(GameTarget)" Platform="$(TargetPlatform)" Configuration="Development" Tag="#GameBinaries_$(GameTarget)_$(TargetPlatform)"/> </Node> <Property Name="GameBinaries" Value="$(GameBinaries)#GameBinaries_$(GameTarget)_$(TargetPlatform);"/> </ForEach> </ForEach> </Agent> #UE4 | @UNREALENGINE

43.

BuildGraph <Agent Name="Default Agent" Type="CompileWin64"> <!-- Update the engine version files --> <Node Name="Update Version Files"> <SetVersion Change="$(Change)" Branch="$(EscapedBranch)" Licensee="$(Licensee)" If="$(Versioned)"/> </Node> <!-- Compile the tool executables --> <Node Name="Compile Tools Win64" Requires="Update Version Files" Produces="#ToolBinaries"> <Compile Target="UnrealHeaderTool" Platform="Win64" Configuration="Development" Tag="#ToolBinaries"/> </Node> <Agent Name="Default Agent" Type="CompileWin64"> <!-- Compile the editor executable --> <Node Name="Compile $(EditorTarget) Win64" Requires="Compile Tools Win64" Produces="#EditorBinaries"> <Compile Target="$(EditorTarget)" Platform="Win64" Configuration="Development" Tag="#EditorBinaries"/> </Node> <!-- Compile the game targets --> <Property Name="GameBinaries" Value=""/> <ForEach Name="GameTarget" Values="$(GameTargets)"> <ForEach Name="TargetPlatform" Values="$(TargetPlatforms)"> <Node Name="Compile $(GameTarget) $(TargetPlatform)" Requires="Compile Tools Win64" Produces="#GameBinaries_$(GameTarget)_$(TargetPlatform)"> <Compile Target="$(GameTarget)" Platform="$(TargetPlatform)" Configuration="Development" Tag="#GameBinaries_$(GameTarget)_$(TargetPlatform)"/> </Node> <Property Name="GameBinaries" Value="$(GameBinaries)#GameBinaries_$(GameTarget)_$(TargetPlatform);"/> </ForEach> </ForEach> </Agent> #UE4 | @UNREALENGINE

44.

BuildGraph <Agent Name="Default Agent" Type="CompileWin64"> <!-- Update the engine version files --> <Node Name="Update Version Files"> <SetVersion Change="$(Change)" Branch="$(EscapedBranch)" Licensee="$(Licensee)" If="$(Versioned)"/> </Node> <!-- Compile the tool executables --> <Node Name="Compile Tools Win64"version Requires="Update <!-Update the engine filesVersion --> Files" Produces="#ToolBinaries"> <Compile Target="UnrealHeaderTool" Platform="Win64" Configuration="Development" Tag="#ToolBinaries"/> </Node> <Node Name="Update Version Files"> <SetVersion Change="$(Change)" Branch="$(EscapedBranch)" <!-- Compile the editor executable --> <Node Name="Compile $(EditorTarget) Win64" Requires="Compile Tools Win64" Produces="#EditorBinaries"> Licensee="$(Licensee)" If="$(Versioned)"/> <Compile Target="$(EditorTarget)" Platform="Win64" Configuration="Development" Tag="#EditorBinaries"/> </Node> </Node> <!-- Compile the game targets --> <Property Name="GameBinaries" Value=""/> <ForEach Name="GameTarget" Values="$(GameTargets)"> <ForEach Name="TargetPlatform" Values="$(TargetPlatforms)"> <Node Name="Compile $(GameTarget) $(TargetPlatform)" Requires="Compile Tools Win64" Produces="#GameBinaries_$(GameTarget)_$(TargetPlatform)"> <Compile Target="$(GameTarget)" Platform="$(TargetPlatform)" Configuration="Development" Tag="#GameBinaries_$(GameTarget)_$(TargetPlatform)"/> </Node> <Property Name="GameBinaries" Value="$(GameBinaries)#GameBinaries_$(GameTarget)_$(TargetPlatform);"/> </ForEach> </ForEach> </Agent> #UE4 | @UNREALENGINE

45.

BuildGraph <Agent Name="Default Agent" Type="CompileWin64"> <!-- Update the engine version files --> <Node Name="Update Version Files"> <SetVersion Change="$(Change)" Branch="$(EscapedBranch)" Licensee="$(Licensee)" If="$(Versioned)"/> </Node> <!-- Compile the tool executables --> <Node Name="Compile Tools Win64" Requires="Update Version Files" Produces="#ToolBinaries"> <Compile Target="UnrealHeaderTool" Platform="Win64" Configuration="Development" Tag="#ToolBinaries"/> </Node> <!-Compile the tool executables --> <!-- Compile the editor executable --> <Node Name="Compile $(EditorTarget) Win64" Requires="Compile Win64" Produces="#EditorBinaries"> <Node Name="Compile Tools Win64"Tools Requires="Update Version Files" <Compile Target="$(EditorTarget)" Platform="Win64" Configuration="Development" Tag="#EditorBinaries"/> </Node> Produces="#ToolBinaries"> <!-- Compile<Compile the game targets --> Target="UnrealHeaderTool" Platform="Win64" <Property Name="GameBinaries" Value=""/> <ForEach Name="GameTarget" Values="$(GameTargets)"> Configuration="Development" Tag="#ToolBinaries"/> <ForEach Name="TargetPlatform" Values="$(TargetPlatforms)"> <Node Name="Compile $(GameTarget) $(TargetPlatform)" Requires="Compile Tools Win64" </Node> Produces="#GameBinaries_$(GameTarget)_$(TargetPlatform)"> </ForEach> </Agent> #UE4 | @UNREALENGINE </ForEach> <Compile Target="$(GameTarget)" Platform="$(TargetPlatform)" Configuration="Development" Tag="#GameBinaries_$(GameTarget)_$(TargetPlatform)"/> </Node> <Property Name="GameBinaries" Value="$(GameBinaries)#GameBinaries_$(GameTarget)_$(TargetPlatform);"/>

46.

BuildGraph <Agent Name="Default Agent" Type="CompileWin64"> <!-- Update the engine version files --> <Node Name="Update Version Files"> <SetVersion Change="$(Change)" Branch="$(EscapedBranch)" Licensee="$(Licensee)" If="$(Versioned)"/> </Node> <!-- Compile the tool executables --> <Node Name="Compile Tools Win64" Requires="Update Version Files" Produces="#ToolBinaries"> <Compile Target="UnrealHeaderTool" Platform="Win64" Configuration="Development" Tag="#ToolBinaries"/> </Node> <!-- Compile the editor executable --> <Node Name="Compile $(EditorTarget) Win64" Requires="Compile Tools Win64" Produces="#EditorBinaries"> <Compile Target="$(EditorTarget)" Platform="Win64" Configuration="Development" Tag="#EditorBinaries"/> </Node> <!--Compile Compile the game --> executable --> <!-thetargets editor <Property Name="GameBinaries" Value=""/> <Node Name="Compile $(EditorTarget) Win64" Requires="Compile Tools Win64" <ForEach Name="GameTarget" Values="$(GameTargets)"> <ForEach Name="TargetPlatform" Values="$(TargetPlatforms)"> Produces="#EditorBinaries"> <Node Name="Compile $(GameTarget) $(TargetPlatform)" Requires="Compile Tools Win64" Produces="#GameBinaries_$(GameTarget)_$(TargetPlatform)"> <Compile Target="$(EditorTarget)" Platform="Win64" <Compile Target="$(GameTarget)" Platform="$(TargetPlatform)" Configuration="Development" Configuration="Development" Tag="#EditorBinaries"/> Tag="#GameBinaries_$(GameTarget)_$(TargetPlatform)"/> </Node> </Node> <Property Name="GameBinaries" Value="$(GameBinaries)#GameBinaries_$(GameTarget)_$(TargetPlatform);"/> </ForEach> </Agent> #UE4 | @UNREALENGINE </ForEach>

47.

BuildGraph <!-- Compile the game targets --> <Property Name="GameBinaries" Value=""/> <ForEach Name="GameTarget" Values="$(GameTargets)"> <ForEach Name="TargetPlatform" Values="$(TargetPlatforms)"> <Node <!-- Compile the tool executables --> Name="Compile $(GameTarget) $(TargetPlatform)" <Node Name="Compile Tools Win64" Requires="Update Version Files" Produces="#ToolBinaries"> Requires="Compile Tools Win64" <Compile Target="UnrealHeaderTool" Platform="Win64" Configuration="Development" Tag="#ToolBinaries"/> </Node>Produces="#GameBinaries_$(GameTarget)_$(TargetPlatform)"> <Compile Target="$(GameTarget)" Platform="$(TargetPlatform)" <!-- Compile the editor executable --> <Node Name="Compile $(EditorTarget) Win64" Requires="Compile Tools Win64" Produces="#EditorBinaries"> Configuration="Development" <Compile Target="$(EditorTarget)" Platform="Win64" Configuration="Development" Tag="#EditorBinaries"/> </Node> Tag="#GameBinaries_$(GameTarget)_$(TargetPlatform)"/> <!-- Compile the game targets --> </Node> <Property Name="GameBinaries" Value=""/> <Property Name="GameBinaries" <ForEach Name="GameTarget" Values="$(GameTargets)"> <ForEach Name="TargetPlatform" Values="$(TargetPlatforms)"> Value="$(GameBinaries)#GameBinaries_$(GameTarget)_$(TargetPlatform);"/> <Node Name="Compile $(GameTarget) $(TargetPlatform)" Requires="Compile Tools Win64" Produces="#GameBinaries_$(GameTarget)_$(TargetPlatform)"> </ForEach> <Compile Target="$(GameTarget)" Platform="$(TargetPlatform)" Configuration="Development" Tag="#GameBinaries_$(GameTarget)_$(TargetPlatform)"/> </ForEach> <Agent Name="Default Agent" Type="CompileWin64"> <!-- Update the engine version files --> <Node Name="Update Version Files"> <SetVersion Change="$(Change)" Branch="$(EscapedBranch)" Licensee="$(Licensee)" If="$(Versioned)"/> </Node> </ForEach> </Agent> #UE4 | @UNREALENGINE </ForEach> </Node> <Property Name="GameBinaries" Value="$(GameBinaries)#GameBinaries_$(GameTarget)_$(TargetPlatform);"/>

48.

InstalledBuild #UE4 | @UNREALENGINE

49.

InstalledBuild ● Think of it as your own Unreal Launcher ● Created by Epic to deploy a non editable engine Engine/Build/InstalledEngineBuild.xml ● Easy to use for Designers and Artists ● Engine build only. Canʼt use it to build a project. #UE4 | @UNREALENGINE

50.

InstalledBuild Installed Build mains steps 1. 2. 3. 4. 5. 6. 7. 8. 9. Update Version File Compile UnrealHeaderTool Win64 Compile UE4Editor Win64 Compile UE4Game Win64 Compile UE4Game (Platform) Build Tools Win64 Build Tools CS Make Feature Packs Build DDC Win64 #UE4 | @UNREALENGINE

51.

InstalledBuild Engine/Build/BatchFiles/RunUAT.bat BuildGraph -target”Make Installed Build Win64” -script=/Engine/Build/InstalledEngineBuild.xml -clean #UE4 | @UNREALENGINE

52.

InstalledBuild Engine/Build/BatchFiles/RunUAT.bat BuildGraph -target”Make Installed Build Win64” -script=/Engine/Build/InstalledEngineBuild.xml -clean #UE4 | @UNREALENGINE

53.

InstalledBuild Engine/Build/BatchFiles/RunUAT.bat BuildGraph -target”Make Installed Build Win64” -script=/Engine/Build/InstalledEngineBuild.xml -clean #UE4 | @UNREALENGINE

54.

InstalledBuild Engine/Build/BatchFiles/RunUAT.bat BuildGraph -target”Make Installed Build Win64” -script=/Engine/Build/InstalledEngineBuild.xml -clean #UE4 | @UNREALENGINE

55.

BuildCookRun #UE4 | @UNREALENGINE

56.

BuildCookRun #UE4 | @UNREALENGINE

57.

BuildCookRun ● Script located at /Engine/Build/BatchFiles/ ● From build to package, do anything ● Highly customizable ● Command line based : nostalgic feeling ! <3 #UE4 | @UNREALENGINE

58.

BuildCookRun, BuildGraph, InstalledBuild ● Depending on the situation, the best choice will differ ● I think adaptability is the most important thing in game dev, so I personally use BuildCookRun most of the time. #UE4 | @UNREALENGINE

59.

Agenda ● CI : What is it ? ● UnrealAutomationTool ● Building the Engine and your Project ● The other stuff ● The Unreal Way #UE4 | @UNREALENGINE

60.

Command Lines #UE4 | @UNREALENGINE

61.

Engine DLL Build If you donʼt edit the Engine a lot, trigger it manually If you extend the Engine a lot, do it once a day. Morning or Lunch Time is preferred #UE4 | @UNREALENGINE

62.

Build Machine Commit to Source Control Jenkins Update Build Notify #UE4 | @UNREALENGINE

63.

Build Command %ENGINE_ROOT%/Engine/Build/BatchFiles/RunUAT.bat BuildCookRun project=Samples/StarterContent/StarterContent.uproject platform=%ENGINE_PLATFORM_TARGET% clientconfig=%ENGINE_COMPILATION_TARGET% -build ENGINE_ROOT=EngineRoot ENGINE_COMPILATION_TARGET=Development ENGINE_PLATFORM_TARGET=Win64 Yup, thatʼs a little hard to read #UE4 | @UNREALENGINE

64.

Build Command Engine/Build/BatchFiles/RunUAT.bat BuildCookRun -project=Samples/StarterContent/StarterContent.uproject -platform=Win64 -clientconfig=Development -build #UE4 | @UNREALENGINE

65.

Build Command Engine/Build/BatchFiles/RunUAT.bat BuildCookRun -project=Samples/StarterContent/StarterContent.uproject -platform=Win64 -clientconfig=Development -build You need a uproject to use RunUAT. You can use a sample project or blank one ! #UE4 | @UNREALENGINE

66.

Build Command Engine/Build/BatchFiles/RunUAT.bat BuildCookRun -project=Samples/StarterContent/StarterContent.uproject -platform=Win64 -clientconfig=Development -build #UE4 | @UNREALENGINE

67.

Build Command Engine/Build/BatchFiles/RunUAT.bat BuildCookRun -project=Samples/StarterContent/StarterContent.uproject -platform=Win64 -clientconfig=Development -build #UE4 | @UNREALENGINE

68.

Build Command Engine/Build/BatchFiles/RunUAT.bat BuildCookRun -project=Samples/StarterContent/StarterContent.uproject -platform=Win64 -clientconfig=Development -build #UE4 | @UNREALENGINE

69.

Project DLL build Poll every 10 minutes to build Commit to Source Control Jenkins Update Build Notify #UE4 | @UNREALENGINE

70.

Build Command %ENGINE_ROOT%/Engine/Build/BatchFiles/RunUAT.bat BuildCookRun -project= project=%GAME_SUB_PATH%/%GAME_NAME%.uproject platform=%ENGINE_PLATFORM_TARGET% clientconfig=%ENGINE_COMPILATION_TARGET% -build ENGINE_ROOT=EngineRoot ENGINE_COMPILATION_TARGET=Development ENGINE_PLATFORM_TARGET=Win64 GAME_SUB_PATH=C:/GamePath GAME_NAME=GameName #UE4 | @UNREALENGINE

71.

Build Command Engine/Build/BatchFiles/RunUAT.bat BuildCookRun -project=C:/PathToTheGame/GameName.uproject -platform=Win64 -clientconfig=Development -build #UE4 | @UNREALENGINE

72.

#UE4 | @UNREALENGINE

73.

Packaging #UE4 | @UNREALENGINE

74.

Nightly Build ● Well, itʼs build every night. ● Development, Test(for profiling), Shipping for every release platform. Time doesnʼt matter, so clean build #UE4 | @UNREALENGINE

75.

Nightly Build Flow Copy to Shared Folder Jenkins Update Build Zip Notify #UE4 | @UNREALENGINE

76.

Build Command %ENGINE_ROOT%/Engine/Build/BatchFiles/RunUAT.bat BuildCookRun project=%GAME_SUB_PATH%/%GAME_NAME%.uproject -noP4 platform=%ENGINE_PLATFORM_TARGET% clientconfig=%ENGINE_COMPILATION_TARGET% -cook -allmaps -build -stage -pak -archive -archivedirectory=%TEMP_DIR% ENGINE_ROOT=PathToYourEngine GAME_SUB_PATH=PathToYourProject GAME_NAME=UprojectFileWithoutTheExtension ENGINE_COMPILATION_TARGET=Development|Shipping|etc ENGINE_PLATFORM_TARGET=Win64|etc #UE4 | @UNREALENGINE

77.

Build Command Engine/Build/BatchFiles/RunUAT.bat BuildCookRun -project=GamePath/GameName.uproject -noP4 -platform=Win64 -clientconfig=Development -cook -allmaps -build -stage -pak -archive -archivedirectory=C:/TempDirectory/ #UE4 | @UNREALENGINE

78.

Build Command Engine/Build/BatchFiles/RunUAT.bat BuildCookRun -project=GamePath/GameName.uproject -noP4 -platform=Win64 -clientconfig=Development -cook -allmaps -build -stage -pak -archive -archivedirectory=C:/TempDirectory/ #UE4 | @UNREALENGINE

79.

Build Command Engine/Build/BatchFiles/RunUAT.bat BuildCookRun -project=GamePath/GameName.uproject -noP4 -platform=Win64 -clientconfig=Development -cook -allmaps -build -stage -pak -archive -archivedirectory=C:/TempDirectory/ #UE4 | @UNREALENGINE

80.

Build Command Engine/Build/BatchFiles/RunUAT.bat BuildCookRun -project=GamePath/GameName.uproject -noP4 -platform=Win64 -clientconfig=Development -cook -allmaps -build -stage -pak -archive -archivedirectory=C:/TempDirectory/ #UE4 | @UNREALENGINE

81.

Build Command Engine/Build/BatchFiles/RunUAT.bat BuildCookRun -project=GamePath/GameName.uproject -noP4 -platform=Win64 -clientconfig=Development -cook -allmaps -build -stage -pak -archive -archivedirectory=C:/TempDirectory/ #UE4 | @UNREALENGINE

82.

Build Command Engine/Build/BatchFiles/RunUAT.bat BuildCookRun -project=GamePath/GameName.uproject -noP4 -platform=Win64 -clientconfig=Development -cook -allmaps -build -stage -pak -archive -archivedirectory=C:/TempDirectory/ #UE4 | @UNREALENGINE

83.

Build Command Engine/Build/BatchFiles/RunUAT.bat BuildCookRun -project=GamePath/GameName.uproject -noP4 -platform=Win64 -clientconfig=Development -cook -allmaps -build -stage -pak -archive -archivedirectory=C:/TempDirectory/ #UE4 | @UNREALENGINE

84.

Build Command Engine/Build/BatchFiles/RunUAT.bat BuildCookRun -project=GamePath/GameName.uproject -noP4 -platform=Win64 -clientconfig=Development -cook -allmaps -build -stage -pak -archive -archivedirectory=C:/TempDirectory/ #UE4 | @UNREALENGINE

85.

Build Command Engine/Build/BatchFiles/RunUAT.bat BuildCookRun -project=GamePath/GameName.uproject -noP4 -platform=Win64 -clientconfig=Development -cook -allmaps -build -stage -pak -archive -archivedirectory=C:/TempDirectory/ #UE4 | @UNREALENGINE

86.

Build Command Engine/Build/BatchFiles/RunUAT.bat BuildCookRun -project=GamePath/GameName.uproject -noP4 -platform=Win64 -clientconfig=Development -cook -allmaps -build Argument to zip all the assets in one “pak” file -stage -pak -archive -archivedirectory=C:/TempDirectory/ #UE4 | @UNREALENGINE

87.

Build Command Engine/Build/BatchFiles/RunUAT.bat BuildCookRun -project=GamePath/GameName.uproject -noP4 -platform=Win64 -clientconfig=Development -cook -allmaps -build -stage -pak -archive -archivedirectory=C:/TempDirectory/ #UE4 | @UNREALENGINE

88.

Build Command Engine/Build/BatchFiles/RunUAT.bat BuildCookRun -project=GamePath/GameName.uproject -noP4 -platform=Win64 -clientconfig=Development -cook -allmaps -build -stage -pak -archive -archivedirectory=C:/TempDirectory/ #UE4 | @UNREALENGINE

89.

Build Command robocopy %ARCHIVE_DIR%/latest %ARCHIVE_DIR%/%ENGINE_PLATF ORM_TARGET%/archive /E /IS /MOV set PATH=%PATH%;C:/Program Files/7-Zip/ 7z.exe a -tzip -mmt mx1 %ARCHIVE_DIR%/latest/%BUILD_NUMBER%.zip %TEMP_DIR%/ exit 0 #UE4 | @UNREALENGINE

90.

Build Command Wait a sec !! #UE4 | @UNREALENGINE

91.

Build Command EA DICE Studios has open sourced EACopy, a faster alternative to robocopy ! Give it a try !!! https://github.com/electronicarts/EACopy Twitter : @honk_dice #UE4 | @UNREALENGINE

92.

#UE4 | @UNREALENGINE

93.

Incremental Build ● Manual Trigger ● For those “Hey mate, I need to show the build in 40 minutes to our investor / boss / publisher, do smth plz lolwut” ● Contrary to the nightly build, faster is better so … ● Use -partialgc ● Donʼt make it clean #UE4 | @UNREALENGINE

94.

Incremental Build Flow Send to DevKit Jenkins Update Build Notify For DevKit deployment, check your platform But you might get an idea /Engine/Source/Programs/AutomationTool/PlatformName/PlatformNamePlatfo rm.Automation.cs #UE4 | @UNREALENGINE

95.

Build Command Engine/Build/BatchFiles/RunUAT.bat BuildCookRun -project=GamePath/GameName.uproject -noP4 -platform=Win64 -clientconfig=Development -cook -allmaps Instead of cleaning the memory when you run out of it, it will free it -build up at the end of the packaging process -stage -pak -archive -partialgc -archivedirectory=C:/TempDirectory/ #UE4 | @UNREALENGINE

96.

#UE4 | @UNREALENGINE

97.

Jenkins Plugins #UE4 | @UNREALENGINE

98.

Plugins Build Pipeline Plugin https://wiki.jenkins.io/display/JENKINS/Build+Pipeline+Plugin #UE4 | @UNREALENGINE

99.

Plugins Jenkins Slack Plugin https://github.com/jenkinsci/slack-plugin #UE4 | @UNREALENGINE

100.

Plugins Retaliation https://github.com/codedance/Retaliation #UE4 | @UNREALENGINE

101.

Agenda ● CI : What is it ? ● UnrealAutomationTool ● Building the Engine and your Project ● The other stuff ● The Unreal Way #UE4 | @UNREALENGINE

102.

Other CI Jobs #UE4 | @UNREALENGINE

103.

Commandlet Command Line Applet is a command program to be run inside of Unreal Engine environment. #UE4 | @UNREALENGINE

104.

Shared DDC Derived Data Cache (DDC) is a feature to keep assets that were prepared for every platform. To make it simple, itʼs an asset caching feature. #UE4 | @UNREALENGINE

105.

Shared DDC Dev PC 1 Build Assets on 1st launch #UE4 | @UNREALENGINE Copy Built Asset Shared Folder

106.

Shared DDC Dev PC 2 Dev PC 3 Shared Folder Dev PC 4 #UE4 | @UNREALENGINE Copy Built Asset Dev PC 5

107.

Shared DDC Dev PC 4 Modify / Add Asset #UE4 | @UNREALENGINE Copy Built Asset Shared Folder

108.

Shared DDC https://docs.unrealengine.com/enUS/Engine/Basics/DerivedDataCache/index.html BaseEngine.ini [InstalledDerivedDataBackendGraph] MinimumDaysToKeepFile=7 Root=(Type=KeyLength, Length=120, Inner=AsyncPut) AsyncPut=(Type=AsyncPut, Inner=Hierarchy) Hierarchy=(Type=Hierarchical, Inner=Boot, Inner=Pak, Inner=EnginePak, Inner=Local, Inner=Shared) Boot=(Type=Boot, Filename=%GAMEDIR%DerivedDataCache/Boot.ddc, MaxCacheSize=256) Local=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, PurgeTransient=true, DeleteUnused=true, UnusedFileAge=17, FoldersToClean=-1, Path=../../../Engine/DerivedDataCache) Shared=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, DeleteUnused=true, UnusedFileAge=19, FoldersToClean=-1, Path=//mystudio.net/DDC, EnvPathOverride=UE-SharedDataCachePath) AltShared=(Type=FileSystem, ReadOnly=true, Clean=false, Flush=false, DeleteUnused=true, UnusedFileAge=19, FoldersToClean=-1, Path=//mystudio.net/DDC2, EnvPathOverride=UE-SharedDataCachePath2) Pak=(Type=ReadPak, Filename=%GAMEDIR%DerivedDataCache/DDC.ddp) EnginePak=(Type=ReadPak, Filename=../../../Engine/DerivedDataCache/DDC.ddp) #UE4 | @UNREALENGINE

109.

Shared DDC Argument Type Comment ReadOnly Bool Make it false when Shared Clean Bool Delete files that went over UnusedFileAge limit Flush Bool Delete all DDC DeleteUnused Bool Il false, never deletes old assets (overrides UnusedFileAge and Clean) UnusedFileAge Int Limit of days after which an asset will be deleted, Fortnite sets it to 10. FoldersToClean Int Max limit of folders to be flushed each time. 0 or -1 means no limit. #UE4 | @UNREALENGINE

110.
[beta]
Shared DDC
// Flush and Clean argument DerivedDataBackends.cpp:553
if( bFlush )
{
IFileManager::Get().DeleteDirectory( *(Path / TEXT(“”)), false, true );
}
else if( bClean )
{
DeleteOldFiles( *Path );
}

/** Delete the old files in a directory **/

void DeleteOldFiles(const TCHAR* Directory)
{
float MinimumDaysToKeepFile = 7;
GConfig->GetFloat( *GraphName, TEXT("MinimumDaysToKeepFile"), MinimumDaysToKeepFile, GEngineIni );
check(MinimumDaysToKeepFile > 0.0f); // sanity

//@todo
}

#UE4 | @UNREALENGINE

111.

Shared DDC //@todo #UE4 | @UNREALENGINE

112.

Shared DDC del /s /q c:/ServerShareDDCPath/* UE4Editor-cmd.exe ProjectName –run=DerivedDataCache -fill #UE4 | @UNREALENGINE

113.

Lighting Build ● All lights except the ones calculated in real time are to be built. ● Of course, Epic is providing the tools for it. But … #UE4 | @UNREALENGINE

114.

Lighting Build #UE4 | @UNREALENGINE

115.

Lighting Build Itʼs SUPER heavy and slow Let your CI pipe do it ! #UE4 | @UNREALENGINE

116.

Lighting Build UE4Editor-cmd.exe ProjectName -run=resavepackages -buildlighting -allowcommandletrendering #UE4 | @UNREALENGINE

117.

Lighting Build There are times you want to check the lights in the editor / game In this case, build locally, and Swarm is your new BFF #UE4 | @UNREALENGINE

118.

Lighting Build - Swarm With Swarm, you can distribute the task of building lights between the computers of your team. You will need a Coordinator. Your Jenkins master should do the trick. https://wiki.unrealengine.com/Swarm_Agent_Trou bleshooting #UE4 | @UNREALENGINE

119.

Lighting Build - Swarm Engine/Binaries/DotNET/SwarmCoordinator.exe Engine/Binaries/DotNET/SwarmAgent.exe To add it at startup, Create a shortcut the the Agent and copy it here. %APPDATA%/Microsoft/Windows/Start Menu/Programs/Startup #UE4 | @UNREALENGINE

120.

Lighting Build - Swarm #UE4 | @UNREALENGINE

121.

Redirectors ● When you move or rename an asset in the editor, a Redirector is created to know the ”new address” ● Super convenient feature but it has a tendency to bug, so you will need to clean them up frequently ● Once a month is fine, but do it once a week in the critical phases of the project (alpha, beta, gold, patch release) https://docs.unrealengine.com/en-US/Engine/Basics/Redirectors/index.html #UE4 | @UNREALENGINE

122.

Redirectors UE4Editor-cmd.exe ProjectName -run=fixupredirects #UE4 | @UNREALENGINE

123.

Agenda ● CI : What is it ? ● UnrealAutomationTool ● Building the Engine and your Project ● The other stuff ● The Unreal Way #UE4 | @UNREALENGINE

124.

The Unreal Way : Fortnite #UE4 | @UNREALENGINE

125.

Fortnite ● Only engineers have Visual Studio ● We are building with BuildGraph managed by ElectricCommander ● We are heavily using UnrealGameSync #UE4 | @UNREALENGINE

126.

Iteration Iteration can seem complicated without Visual Studio. But actually, itʼs pretty manageable. ● Heavily rely on PIE for testing ● Leverage your CI software properly #UE4 | @UNREALENGINE

127.

Testing a Level on Device So letʼs say your artist has just changed a characterʼs hair color and wants to see how it looks on the release platform screen before committing… First, create an incremental build task on your CI software, and commit the binaries to Project/Binaries/PlatformName #UE4 | @UNREALENGINE

128.

Custom Launch Profile Then in the editor, click on the triangle on the right side of of “Launch”, scroll down and create a new Custom Launch Profile Stored in /Engine/Programs/UnrealFrontend/Profiles In the profile, untick the Build step, and then you can deploy the currently edited level on the machine connected to the computer, even without Visual Studio. Magic !! #UE4 | @UNREALENGINE

129.

Build Machine Jobs What I think a UE4 build pipeline should have : ● Engine Build ● Project Build ● Nightly Package ● Incremental Package ● Incremental Package for the Device Launcher ● Lightmaps Build ● Fixup Redirectors Command ● SharedDDC Cleanup Command ● SharedDDC Fillup Command Any suggestion on what to add ? Tell me in Q&A !!! #UE4 | @UNREALENGINE

130.

Versioning ● Epic uses Perforce. If you want to be as close as possible to our workstyle, you know what to do ● SVN might be a little slow, but it is very reliable and still heavily used ● GIT is free, modern design choices and is very used for mobile and social games (in Japan !) SVN and Perforce have Editor support ! GIT too, but itʼs still experimental ! #UE4 | @UNREALENGINE

131.

UnrealGameSync Made by Epic for Epic. Heavily used on Fortnite Itʼs Perforce only, and it allows non technical staff not to have to deal with P4V #UE4 | @UNREALENGINE

132.

UnrealGameSync #UE4 | @UNREALENGINE

133.

UnrealGameSync Sync Get Engine P4 Server Local Builds (Binary Engine) Build Machine #UE4 | @UNREALENGINE

134.

UnrealGameSync Get the latest version anytime on GitHub and Perforce. UGS dev is unrelated to UE4, so always get the latest version even if youʼve locked your engine version ! https://docs.unrealengine.com/latest/en/Programming/Deployment/Unre alGameSync/Reference/ #UE4 | @UNREALENGINE

135.

Links https://github.com/gpageot/JenkinsUE4 Twitter : @GregoryPAGEOT https://github.com/AxRiff/UnrealEngineJenkins Twitter : @AxRiff https://github.com/electronicarts/EACopy Twitter : @honk_dice Manga by @shiba_zushi, Technical Artist at Epic Games Japan. #UE4 | @UNREALENGINE

136.

Merci ! #UE4 | @UNREALENGINE