As far as I can remember my development forays with Unity, there was always one problem. When new version of Unity was released, you had to think very carefully whether you should upgrade to it, or not. Although many developers are totally against upgrading the engine during the development, I do not have any objection, as long as everything works. I consider it a simple benefits vs. problems evaluation. Upgrading should give you more benefits. If upgrading would clearly break everything, leading to huge amount of extra work, then there really is no reason to upgrade. But otherwise I see no reason not to upgrade.
Usually new engine versions are more optimized, and should give better performance. They may contain new features that either make your game looking better, or make development easier etc. It would be stupid to stay on old version, just because "that is how things are done in the industry". Since game development projects may take long time, you also have to think the future. If you always upgrade to the latest engine, then you are able to utilize latest technology in your final product, which could provide new platforms, new ways to play and more customers.
In my Unreal Engine projects I have seen, that upgrading the engine works well. So far, I've neither had nor seen any big problems after upgrade. But benefits have been huge. Better performance, better graphics and more. But Unity… well, things have not been so simple. That is why I had this worry and doubt while reading the future plans for Unity engine, especially the network, multiplayer, Data-Oriented Technology Stack (DOTS) and Entity Component System (ECS).
This is a long term plan, and from technological point of view it is really wonderful - multi-threaded code, high performance, probably easier development, and better networking. But when does this really happen, and how, not to mention that plans do always change, and there are always some delays? What are the repercussions to common developer? What parts of the engine will work in the future, and what does not? These are big questions that deal with the future of any game development project using Unity engine.
I have experience on using Unity components that were experimental (and not generally available) and I just have to admit, that in Unity's case experimental means, that some parts may work, some do not, some are not even implemented yet, and documentation is cryptic. They have lots of enthusiasm to make their engine better, but one problem is that Unity is perhaps too much interested in their services than the engine itself, because their services produce large part of their revenue.
So, all this makes me worried. As I have said, I am not a fanboy of Unreal or any other engine (generally I am not interested in any discussions about what engine is best etc.). Unity has its place in all this, and they have many interesting features, although they are lacking in others. So, I really hope that all those new plans work, but I am worried about what kind of mess this will generate for Unity ecosystem, because the whole core of engine is about to change. All that makes it really hard to make a long term plan for a software project based on Unity. Some of those parts are still so experimental, that you cannot really use them yet, but on the other hand, some of the present components will be deprecated quite soon. Of course everyone wants to use the latest set, because that could be crucial for the performance of your software.
Networking and multiplayer is "under heavy construction". Old UNet will be gone in the future, and replaced by new networking layer. This new networking is tied into ECS, where components should be networked by default. Nice, but when it will be working for real. Also, we have heard lots of talks about ECS, but it still seems to be more talk than hard fact. It will be really interesting and perhaps entertaining to see how teams will adopt ECS, because it changes so much. I have no problem with ECS, but if I were a team leader with several programmers and developers with varied skill levels and background, it is not always easy to adopt new way of thinking for whole team. It is challenging to make long term plans for whole studio, when your core tool is in the midst of upheaval.
As an open-minded individual I have the luxury to choose freely any tools I want to use. I can experiment with any engine that may look interesting. If something works and suits my projects, then I probably adopt it. If it does not work for me, it still does not prevent me to understand that it may work for others. But many studios are more stuck into their ways, many have prejudice or adversity towards other engines and they just stubbornly sit in the circle they have drawn in the sand. They could be in trouble, because that mind set already tells that they are resisting a change. And there will be a huge change when DOTS, ECS and everything else is fully implemented in Unity. So, the head line should actually read "Is there future for Unity developers", because Unity will surely go on, but there could be lots of Unity developers stranded on the beach of "Old Unity".
More about these plans on Unity website: