Who has ever stood in the data center (or clicked around in the Cloud Console) and asked themselves: "Who the hell has the changed again now?"
Don't worry, you're not alone. We all know the feeling when the lovingly built system landscape suddenly starts behaving like a stubborn mule just because someone has "just" adjusted something.
And this is exactly where the problem lies: In IT, especially in the field of infrastructure, conditions often prevail that would be met with a shake of the head in other engineering disciplines. Imagine a civil engineer simply working on the statics of a bridge, without a plan, without control, without anything. Horror scenario, right? In IT, however, this is often a sad reality and this is exactly where we need to rethink!
Our credo must be: Everything that is not data is to be treated as a versioned artifact. Whether these are the configuration files for the web server, the templates for the databases or the orchestration scripts for the containers - everything must be in version control! (Preferably in Git, because Git is practically the gold standard when it comes to version management).
Data is sacred - and is treated separately
But one thing is very important: Data is data and artifacts are artifacts. Data, i.e. the valuable information that makes up the application, should be backed up separately. Preferably in a managed service, because then you don't have to worry about backup, replication and all the other stuff. And, of course, this managed service should ideally also be Infrastructure-as-Code (IaC) based. This closes the circle.
Staging is not rocket science - but it's not an option either
Next item on the agenda: Staging! If you still live in a world where there is only "production", please wake up for a moment. Every serious workload needs at least two stages: Dev and Prod. Dev for playing, testing and trying out, Prod for the real thing when the money is earned. Depending on the complexity of the application, there can of course be more stages. The main thing is that there is a clear separation between development and production. It's like in gastronomy: first you experiment in the test kitchen before the new dish is put on the menu.
Configuration as code - as a template!
And how do you get the configuration under control? Quite simply: Configuration files become templates! Whether containers, instances or classic servers are configured - templates are defined in IaC, ScM (Software Configuration Management) or orchestration. These templates are of course versioned (Git, we've already seen that). And the highlight: the actual data in these templates (passwords, database URLs, etc.) are externalized per stage. This means that there are separate variable files (or a separate vault, or whatever we like) for each stage (Dev, Prod, etc.). The possibilities are really great.
One IaC codebase for all stages - less is more!
Of course, structure is important, but we don't want to overdo it with the love of order. That's why: For each workload there is one IaC/ScM/Orchestration Codebase! This codebase is the holy grail that serves all stages. The differences between Dev, Prod and Co. are elegantly explained via variable overrides solved, e.g. with Terraform and its tfvars files.
In this one Codebase is then fed the variable configuration components for each stage via extra file(s). This keeps things lean, clear and maximally maintainable. Maintaining a separate repository for each stage - that would be madness to the power of three! But this way everything is centralized and still flexible. Less is definitely more here!
Changes start at the bottom - always!
And now comes the most important point of all: Changes start at the bottom - always, even if it is "urgent". Yes, "urgent" is the favorite word of managers worldwide. But it becomes still It's even more urgent when changes are made directly in production and everything goes wrong. Therefore: Changes are tested and "promoted" from bottom to top through the stages.
Breaking changes? No problem - with a plan!
And what about Breaking Changes to IaC/ScM/orchestration? These are also no problem if you approach them correctly. Breaking changes can of course result in adjustments to the var files. But this is all manageable with release notes and a checklist/howto. Communication is the be-all and end-all here.
Conclusion: If you don't want to hear, you have to feel - and feel properly!
If you don't follow these golden rules, you risk disaster. Both on-prem classic and cloud-native. Because believe me, chaos knows no cloud boundaries. Therefore: free your IT infrastructure from chaos and introduce version control as standard. Your nerves (and those of your colleagues) will thank you!
English 
Deutsch