In this newsletter:
Unreal Source Control with Git Revisited
Gamers vs. Epic Games Store
The section on Git proved to be so long, that I just had to drop the 2nd part of Unreal Engine multiplayer series to next issue - sorry about that. But this way I could cover Git more thoroughly and after this is now done, I can concentrate better on other subjects in this newsletter. I have also prepared a handy Git Reference which you can download in PDF format. Link is below.
I have followed the controversial relationship between gamers (or at least a very noisy group of gamers) and Epic Games Store. This issue has poisoned online discussion to such epic proportions that I just had to offer my view on this subjects which borderlines also game development, and eternal wars between fanboys.
Unreal Source Control Revisited
Previously I explained my personal workflow with Git for Unreal Projects. My preference is to use Git Bash quite a lot, because I use Git with other programming languages and environments too. I work faster that way, but it really is not the only way. Unreal Engine Source Control Plugin has tools that handle checking in tracked files, adding untracked files, moving and deleting files. However, on my experience, sometimes it is safer and clearer to use command line.
There are situations where you have to use Git Bash (or some other Git tool). Pushing your commits to origin master is one such case, and another one happens when you encounter merge conflict with Blueprints. Although the Blueprint merge is handled in Editor, you have mark it as resolved in Git Bash. Third case is C++ code. If your project contains C++ code, then you have to handle their source control with Git Bash or some other tool. UE Editor Source Control plugin does not process them.
Git GUI Tools
There is a plethora of Git GUI Tools available for those users who are not comfortable with command line like Git Bash. Just make your pick from these
https://git-scm.com/download/gui/windows
Git itself has built-in tools for committing (git-gui) and browsing (gitk). I can recommend at least TortoiseGit and Git Extensions. TortoiseGit requires Git already installed, but Git Extensions installs both Git and Extensions GUI. TortoiseGit also integrates with Windows Explorer, and you can see file status as overlay icons. Also, you can initiate Git commands by right clicking a file or folder, and choose it from context menu. Git Extensions has its own GUI window, which may suit some users better. While TortoiseGit is available only for Windows, Git Extensions is available for Mac and Linux, too.
An important factor when you choose your tools and plugins, is to consider how reliable it is. I would not choose a tool that is maintained by one single developer. There is a high probability that there may be no support in the future, because the developer finds another job and has less time, he may drop the development entirely. All those Git GUI tools are community developed which means that there is very low probability they could suddenly disappear.
Source Control in Unreal Engine Project
Before I describe more details of useful Git operations, let's get a bit bigger picture of source control in UE related projects. As you may have noticed there were other options for source control provider in the settings - namely SVN and Perforce -, and you should have noticed that Git support is beta version. It also seems, that Perforce (HelixCore) is the primary choice, and Unreal Engine Source Control Plugin is built with Perforce and its workflow in mind.
Why use Git then, you may ask. This again is a matter of opinion. Git is used widely in thousands of projects, and Linux kernel development uses Git (actually it was built for it). Git is free and open source, also. Perforce is good, but it is not free for larger studios. So, the question is mostly how much money you are willing to put into your source control. And learning Git properly will help you in the future, because it is very probable that you will land on some project that is handled with Git.
If you have used different version control systems before, you have probably noticed that there are differences in the general workflow. Git is a bit different system than SVN or Perforce. Since Git support is still in beta, it means that certain features do not work, and you have use Git Bash or some other Git tool. Also, the terminology is different, but I try to clarify it in this newsletter.
One crucial difference is file locking, which divides people, but is a matter of opinion. I myself was for file locking long time ago, but then I turned against it, due to various reasons. But this is a matter of opinion, and you should make your own choice. I find it cumbersome and restricting, and it does not really solve the problems of merge conflicts. Actually, file locking has many problems of its own. Git does not support file locking as such, although there are some git services that have built file locking (sort of) on top of it. Git is different since it is basically a distributed system and every repository is independent in a way - you just synchronize your repository with other repositories.
So, all this means, that you have to work a bit differently. I have also found out, that it is really the best and safest way to do many things in Git Bash (or in some Git GUI tool). I will now explain how some things can be done in Git Bash, or using Source Control tools in UE Editor, and in TortoiseGit.
Fetch and pull - what is the difference?
As I explained previously, Git is not centralized, which often confuses people. Of course you have origin master in remote repository, which is the master branch, but there can be other branches, too. And basically every Git client is its own repository too. Just think it in a way that you have your own working tree (current state of your files), and then your local repository (where you can make your own branches and commits), which also contains all the history of master branch and any other branch you want. Then, somewhere is the remote (master) repository (actually the origin/master branch), where every developer in your team pushes their commits. Calling it master helps people to understand the idea, but in reality every local repository can act as remote repository, too.
You synchronize the history of master branch from remote repository to your local repository with
git fetch
This will download and update the data, basically anything that has happened after your last synchronization. It also contains all new modifications by other developers. However, they are not (yet) incorporated into your working version branch, they are just in the history data of your local repository. If other developers have made commits to origin/master branch in remote repository, then your local repository branch is behind the remote repository (the origin/master branch)
Then, with subsequent git commands, like
git merge
You can merge those new changes into your branch. Which you really should do when you are behind in commits. However, the difference with
git pull
is that you will do both those operations. git pull will fetch first, and then attempt merge. It depends on your workflow which way you should handle this.
Editor can do this too, but I recommend NOT to use it, since it will make confusing things, and does not really tell you what happened. In Content Browser, you can right-click any directory to see a drop-down list of operations. There is an operation Sync, which will pull new versions of files, and seemingly replace your versions with them. If you had some modifications of your own, they are not lost, but - without telling you anything - it has rebased your version. Sorting it out can be confusing if you do not notice what happened. So, DO NOT press Sync. Just use "git fetch" and "git merge" or just "git pull".
In TortoiseGit you can right click your project folder to bring up the context menu. Choose TortoiseGit -> Fetch or TortoiseGit -> Pull.
If your merge or pull was successful (meaning there were no conflicts) then your local repository branch is in sync with origin/master branch in remote repository. Your local working tree may still contain modified uncommitted files, meaning that your working tree is not in sync with your local branch, so to speak. When you commit them, and then push them to remote repository, then everything is in sync - your working tree, branch of your local repository and branch of origin/master in remote repository.
Getting Details
When you have done git fetch (but not done merge, yet), you should check what has happened. Are you behind of origin master, what new files or modifications there are. This is important even if you decide that there is no need to merge. If you do not do this regularly, you will just generate problems later. Git does not automatically sync anything. It does it only when you ask it to sync.
Source Control plugin does not provide any filtering tools to list files according to their status. You can always use Check In to see which files have been modified in your working tree, and then cancel it. You can get the same listing in Git Bash with
git status -s
In TortoiseGit you can see the status of files in Windows Explorer as icon overlays. Tables below will show what those symbols and status characters in file listing means. Remember, these reflect the status of your working tree in relation to your current local repository index. You may be behind the remote repository, but that is not shown in the status.
Command line gives you a very accurate view, since it separates the status of index (staging area) and working tree (unstaged area). Left side is index, and right side is working tree. If file has a status in index, then it usually means that you can commit it. If it is on right side, it has not yet staged for commit, and you should add it first with git add.
File may have status character on both left and right column. If you see a file with MM or AM, then it means that you have added it, but made new changes without committing it. There is nothing wrong with that, but you have to do a git add before you commit it.
TortoiseGit gives you a more condensed view, there is just one status icon for any file. TortoiseGit stages file automatically when you commit it, just like UE Source Control plugin does. When you use Commit command, then it collects all unstaged (modified) files, and gives you a choice to pick which ones you want to commit.
However, it is important to understand that after you commit (check in) any files, it is then committed, and unmodified from source control point of view - until you make new modifications in it. Each commit is a point, where files are in base state for that commit. When you modify files they differ from that base state, which is the last commit. But you can go back and revert to a certain commit, if you want.
If you want to know what changes other developers have done before you do a merge (after you synced your local repository with remote repository), you just run
git diff --name-status origin/master
This will show all differences in your working tree and preceding commits (if you are behind of origin/master in commits)
You can also compare different commits with
git diff commitA commitB
and look the log
git log
Or see what others have done to a file
git blame Content/Path/To/File
Most of this information is available through TortoiseGit context menu which is integrated into Windows Explorer like the image below shows.
Revert files
There are situations where you may want to undo all your changes in your working tree, and revert the file back, like it was in the last commit. In Git this is kind of "undo", but revert steps are commits. That is why it pays to commit fairly often, instead of those so called mega commits. You can revert single file with
git checkout -- Content/Path/To/File.uasset
to how it was in last commit. You can do this kind of revert with TortoiseGit by selecting a file in Windows Explorer, and choosing TortoiseGit -> Revert. Similarly, UE Source Control plugin has Revert available in Content Browser.
Instead of just single file, you may want to revert everything back to the last committed state. This is done with.
git reset --hard
Be careful, this cannot be undone. If you revert, then all those changes are gone, really. Those two cases do not change commit history any way - they just change files in your working tree back to how they were in last commit.
If you have already committed files, but want to cancel it, then you can do it with
git revert HEAD
which reverts everything back to way they were before the commit. Then you can make a new commit afterwards.
Deleting, Moving, Renaming and Copying Files
When you delete, move, rename or copy files, you should do it in such way that git will know what has happened, so that the files will remain tracked correctly. If you do these operations in UE Editor, then you can do them in the normal way in Content Browser.
Never do normal delete in command line or Windows Explorer. Instead, in Git Bash you should delete single file with
git rm Content/Path/To/File.uasset
or whole directory
git rm -r Content/Path/To/Directory
This deletes the files AND keeps git history up to date. If you use TortoiseGit, then use TortoiseGit -> Delete in the context menu.
Similarly, moving (same as renaming) is done with
git mv file newdirectory
where you will give the new location directory for the file. In TortoiseGit this is done with TortoiseGit -> Rename in context menu. And copying a file is done with
git cp file newfilename
If you use TortoiseGit, then just copy the file in normal way and the add it for tracking with TortoiseGit -> Add
UE Source Control Operations
Checking out does not really do anything useful in Git, since Git is graph based, and does not support file locking. Unless your Git server has some additional modification that supports file locking, you should not worry about this.
Checking in is actually the same as making a commit. And remember, that it only does a commit, it does not push anything to anywhere. This commit stays on your local repository until you manually push your commits to origin master. Source control plugin is rather useful in this matter, because it provides easy to use tool, where you can commit just one file, or all your modified and pending files.
Visual Diff is the most useful tool in UE Source Control plugin if you use Git. There is no other way to compare Blueprints in different commits and solve merge conflicts. You can always start it for a Blueprint through File->Diff menu.
It will open a window where you can compare two versions of a blueprint side by side, and go through all changes. It will even show what has changed if you choose a graph listed in the list on the left.
When you have done git merge, there may appear conflicts in files. If they are text files, you can sort it out in Visual Studio. But Blueprint conflicts have to be solved in UE Editor. You can start Visual Merge by selecting Merge in File menu. That will open a window quite similar to Diff window, but you have other options like buttons to accept your changes or changes from others (thus overwriting other one). But usually you should combine the differences, which you can do easily by copying parts of graph to a working copy, then you should tell git that the conflict is solved
git add Content/Path/To/ConflictingFile
and then after all conflicts have been solved you have to
git rebase --continue
Then Git will know that everything is in order, and branches are now synchronized.
Install Your Own Git Service
Although you can use pure basic Git on Linux machine, I suspect most users try to avoid that. Unless you are an expert. But there are easier alternatives, which anyone (with some experience on computers) should be able to install and get running.
GitLab
This is my personal favorite, which does not mean that it is best. There is both free and commercial version of GitLab (https://about.gitlab.com). Free version is sufficient for most purposes. Notice, that GitLab runs on Linux only. Gitlab is quite easy to install and configure, and has lots of nice features, like todo lists and project wiki. Commercial version has more features (even file locking) and you can integrate it with Kubernetes and DevOps pipeline.
GitLab is a bit resource hungry, but you can install it on Raspberry Pi 3, if necessary. However, in my experience, you should then disable the graphical desktop on Pi. When desktop was running, the Pi was clearly choking quite often, and had to be rebooted. After I disabled the desktop, both Pi and GitLab ran smoothly for long periods.
Gitea
Gitea (https://gitea.io) is easy to install and very lightweight Git server. Also, as a cross-platform software, Gitea runs on Windows, macOS, Linux, ARM etc. Installation is as simple as running the binary and some configuration. It is worth testing.
All this should give a fair understanding on how to use Git on UE game development projects. I have prepared a handy reference sheet on this subject you can download it here.
I have covered most of the Git basics in here. There are lots of special cases, but I cannot cover all of them. I have intentionally left branching out, since it is quite advanced subject. But branching is a powerful feature, and eventually it will be needed in complex multiplatform projects. I may cover it later, if there is enough interest. And there is also the administration of Git server.
I have prepared a handy Git Reference for Unreal Engine developers. It covers both Git Bash, TortoiseGit and Editor Source Control plugin. You can download it here.
Gamers vs. Epic Games Store
I have followed gamer discussions about Epic Game Store with great disbelief. It just seem so unbelievably absurd, to read how people whine and have totally irrational grudges against Epic Games Store. Their favorite rant seems to be that there is some kind of conspiracy between Tencent and Epic against Steam and whatever. People claim that they will not buy anything from Epic Games Store etc.
All this started immediately when Epic Games Store was launched. It seemed that Steam fanboys started to tremble in their gaming chairs when someone dared to compete with their Holy Cow. But it really went into high gear, when the first Epic Games Store exclusives were announced. Basically everything the Epic Games has announced (about games store) has encountered lots of negativity. But there is no real point in all this negativity, none at all. Let's go through it all.
Epic Games Store is still in its infancy, and there is much to improve, this is something that everyone admits. I just hope that they do not make the same mistakes that Valve has done with Steam. I really hope they develop it as a store, and not as a shouting arena, where users and curators spread negativity on just a whim. Exclusive games should not be that big problem, since the price is still the same it would be anywhere. There is no extra free, anyone can play. If playing it outside Steam is such an overbearing problem for someone, then he should check his priorities in life (assuming he has one).
Epic Games as a whole is quite fair company. First of all they are giving their game engine for free - and it means whole package including the source code. I am a long time Unity developer too, and I see a great difference here. Unity does not give you source code for engine, and they are aggressively marketing their Plus and Pro licences. Although Epic will take a small percentage of your revenue, that will happen only when you start to generate revenue with Unreal Engine. And if you are selling your game in Epic Games store, then that percentage is included in their normal commission.
Unity puts its hand into your pocket when you are still developing your game. You may use Personal edition only if your revenue or funding does not exceed $100K per year. Yes, hopefully you noticed the word "funding", and it means both raised or self-funded). Plus licence is limited to $200K. So if you get $210K funding through Kickstarter, you have to start to pay $125 each month per seat. For a 6 person studio it is almost $10K per year. You will pay that even if your game does not generate any revenue afterwards.
All this comes from one clear difference between Unity and Epic. Epic makes both game engine and games. Unity makes only an engine and provides some services to developers, so it has to generate its income from that. Basically Unity has monetized game development, just like in mobile games. You are offered a F2P product, which is not that free after all, and you have to start pumping money into it on regular basis.
Epic makes lots of tutorials, sample projects and other learning material. Epic Games Store takes smaller percentage, meaning that developer gets more income with same price, and that means a lot for small studios. It even means that some prices could drop a little.
This also reflects in what kind of game development these entities endorse. Focus in Unreal Engine development scene is clearly on quality content and real, traditional and complex games. Unity world on the other hand, is clearly focused on game monetization, often with simple mobile games, simple retro-style platformers etc. Unity offers all kinds of services which are built to provide monetization tools, gaming analytics to find better analyzation, social gaming tools to provide tools to use peer pressure in monetization etc.
Epic has offered their large game service infrastructure (Epic Online Services) for other developers and games. They have acquired several other companies, and then integrated their software into Unreal. Epic also offered Developer Grants which evolved into Mega Grants, where they give promising developers quite a sum of money "with no strings attached". They also offer mod kits of several games, where you can download the full codebase of a game and modify it as much as you like.
As a developer, I see Epic and its CEO Tim Sweeney as a really decent and fair, developer supporting entity - almost like some aliens landing on Earth offering their superior technology just because they happen to like us. They really give lots of support for developers, and I mean concrete support like engine, tools and learning material, software, hardware and money. All that support enables struggling developers to make better games, which probably makes gamers happy, which means more games purchased and therefore more money. I just cannot understand how someone may see all that as evil or dubious. Yes, they are earning lots of money right now, but that is the thing called business, and that is something every company tries to do. Even Steam makes a lot of money, but they are not giving a cent away, everything goes into Gabe's large coffin.
Right now there is a Big Sale on EGS, and even that got totally ridiculous response from gamers. Getting an extra discount (courtesy of Epic) on top of regular discount seems to be offensive for some people. "No, I do not want it at discount, I rather pay more at Steam". I start to suspect, that even if EGS would offer every game for free (paying it from their own coffin to developers), there would be these whiners commenting "No, I do not want it free, I want to pay for it on Steam". Even when some publisher withdrew their games from EGS during the sale, it was seen (again) as a mark of evil Epic-Tencent conspiracy.
All this is just totally irrational behavior, since this is a win-win-win situation where gamers win by getting more and better games at fair price, game developers win since Epic takes smaller cut and devs get more good tools for free, and Epic gets more customers that generate more revenue.
Either all this is just random trolling or there really are so hardheaded Steam fanboys, that you just cannot reason with them, because their whole life is so rooted on Steam. But I just cannot believe that someone is that stupid. There are also Unity fanboy developers, who seem to have these childish theories that Epic leads some kind of conspiracy to make their precious Unity look bad. Well, if I were prone to develop similar tinfoil hat conspiracy theories, I could suggest that perhaps all this is a Steam and Unity controlled trolling operation, where they spread fear, uncertainty and doubt, and agitate gamers to join in the smudge ops. A secret cabal of Valve and Unity executives (wearing hoods and cloaks, of course) have decided in their meetings, that the flood of Unity based asset flip games has to be ensured. All that is missing from this picture is Tom Hanks running around GDC2019 looking for mysterious symbols that point to the secret headquarters of this cabal. But that would be an absurd theory, wouldn't it?
I am not a fanboy of anything, but I respect what Epic Games has offered to us. I respect it as a developer and as a gamer (original Unreal Tournament and Unreal are my favorite games, next to original Quake and Return to Castle Wolfenstein). But I do use Steam, and I do use Unity, Godot Engine and various others. I am not taking sides. I am used to make my choices based on rational reasoning, not on irrational emotional temper tantrums, or biased prejudice.