{"id":2822,"date":"2025-07-29T14:25:33","date_gmt":"2025-07-29T12:25:33","guid":{"rendered":"https:\/\/35x.de\/?p=2822"},"modified":"2025-07-29T14:28:18","modified_gmt":"2025-07-29T12:28:18","slug":"vm-vs-infrastructure","status":"publish","type":"post","link":"https:\/\/35x.de\/en\/lessons-learned\/vm-vs-infrastructure\/","title":{"rendered":"Black start: How a single VM brought a highly available infrastructure to its knees"},"content":{"rendered":"<div data-elementor-type=\"wp-post\" data-elementor-id=\"2822\" class=\"elementor elementor-2822\">\n\t\t\t\t<div class=\"elementor-element elementor-element-3d881fc e-flex e-con-boxed e-con e-parent\" data-id=\"3d881fc\" data-element_type=\"container\" data-e-type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-536c8bc0 elementor-widget elementor-widget-heading\" data-id=\"536c8bc0\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h1 class=\"elementor-heading-title elementor-size-default\">Black start: How a single VM brought a highly available infrastructure to its knees<\/h1>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-2db5d01b e-flex e-con-boxed e-con e-parent\" data-id=\"2db5d01b\" data-element_type=\"container\" data-e-type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t<div class=\"elementor-element elementor-element-370a1519 e-con-full e-flex e-con e-child\" data-id=\"370a1519\" data-element_type=\"container\" data-e-type=\"container\">\n\t\t\t\t<div class=\"elementor-element elementor-element-7a4e788c elementor-widget elementor-widget-text-editor\" data-id=\"7a4e788c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Do you have any of these memories? One of those stories from everyday IT life that you tell over a coffee (or a beer after work) and that elicits a knowing nod from your colleagues? A story that stays with you years later because the lessons you learned were so fundamental?<\/p><p>I have one of these. It's about a highly redundant system distributed across two data centers. A system that was designed to withstand the failure of an entire site. And it's about a 24-hour total failure, which should actually be impossible.<\/p><p>This is the anatomy of this failure. It is a reminder that Murphy's Law is omnipresent in IT: anything that can go wrong, will go wrong. Especially when it comes to a \"temporary solution\".<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-c37e1db e-con-full e-flex e-con e-child\" data-id=\"c37e1db\" data-element_type=\"container\" data-e-type=\"container\">\n\t\t\t\t<div class=\"elementor-element elementor-element-1a4264c8 elementor-widget elementor-widget-image\" data-id=\"1a4264c8\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img fetchpriority=\"high\" decoding=\"async\" width=\"480\" height=\"720\" src=\"https:\/\/35x.de\/wp-content\/uploads\/2025\/07\/vm-vs-hochverfugbare-infrastruktur.jpg\" class=\"attachment-large size-large wp-image-2823\" alt=\"VM vs. highly available infrastructure\" srcset=\"https:\/\/35x.de\/wp-content\/uploads\/2025\/07\/vm-vs-hochverfugbare-infrastruktur.jpg 480w, https:\/\/35x.de\/wp-content\/uploads\/2025\/07\/vm-vs-hochverfugbare-infrastruktur-200x300.jpg 200w, https:\/\/35x.de\/wp-content\/uploads\/2025\/07\/vm-vs-hochverfugbare-infrastruktur-8x12.jpg 8w\" sizes=\"(max-width: 480px) 100vw, 480px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-42ebe9fc e-flex e-con-boxed e-con e-parent\" data-id=\"42ebe9fc\" data-element_type=\"container\" data-e-type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-4bdfd76f elementor-widget elementor-widget-heading\" data-id=\"4bdfd76f\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">The setup: A bastion of redundancy (theoretically)<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-3b21a21c e-flex e-con-boxed e-con e-parent\" data-id=\"3b21a21c\" data-element_type=\"container\" data-e-type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-67d2511f elementor-widget elementor-widget-text-editor\" data-id=\"67d2511f\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>About a decade ago, I was part of a team supporting a then state-of-the-art VMWare infrastructure for one of our customers. The setup was, on paper, a fortress of availability:<\/p><ul><li><strong>Two data centers (RZ):<\/strong> Physically separate locations, connected by dark fiber.<\/li><li><strong>Computing power:<\/strong> Several fully equipped IBM BladeCenters in both data centers.<\/li><li><strong>virtualization:<\/strong> A large VMware cluster stretched across both sites, running hundreds of critical Linux VMs.<\/li><li><strong>Storage backend:<\/strong> The centerpiece. A high-performance Fibre Channel SAN system, whose controllers were also distributed across both data centers and the data was mirrored synchronously.<\/li><\/ul><p>The declared goal was crystal clear: absolute redundancy. Does a blade chassis fail? No problem, vMotion moves the VMs elsewhere. If an entire data center fails due to a power outage or natural disaster? No problem, the second data center should take over seamlessly. We felt safe. Too safe.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-17f265da e-flex e-con-boxed e-con e-parent\" data-id=\"17f265da\" data-element_type=\"container\" data-e-type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-609132fb elementor-widget elementor-widget-heading\" data-id=\"609132fb\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">The trigger: the inconspicuous single point of failure<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-464fddcc e-con-full e-flex e-con e-child\" data-id=\"464fddcc\" data-element_type=\"container\" data-e-type=\"container\">\n\t\t\t\t<div class=\"elementor-element elementor-element-54d71ff3 elementor-widget elementor-widget-text-editor\" data-id=\"54d71ff3\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>To understand what went wrong, we need to talk about a core concept of high-availability storage clusters: the <strong>Quorum Witness (QW)<\/strong>.<\/p><p>Imagine a storage cluster with two heads (controllers), one in each data center. If the connection between the two fails, they both face an existential question: \"Am I the one who is still alive, or am I isolated from the rest of the network?\" If both controllers were to conclude that they were the \"master\", they would both accept write access. This would lead to a catastrophic state known as <strong>\"Split-Brain\"<\/strong> is known - two inconsistent versions of the truth that make data recovery almost impossible.<\/p><p>This is where the Quorum Witness comes into play. It is an independent referee. If a connection is lost, each controller tries to reach the QW. Only the controller who can reach the QW is allowed to continue. The other controller assumes that it is isolated and shuts down access to the data (the LUNs) to prevent data corruption. A simple but effective logic.<\/p><p>Our critical vulnerability? The Quorum Witness was not hosted on dedicated, independent hardware (e.g. a small physical server in a third location) as would have been best practice. It was a tiny virtual machine. Provisioned on the very storage cluster it was supposed to monitor.<\/p><p>It was a <strong>Temporary solution<\/strong>. Set up during the initial commissioning with the firm intention: \"We'll get it right later.\" This sentence should still be ringing in our ears.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-49c81b2f e-flex e-con-boxed e-con e-parent\" data-id=\"49c81b2f\" data-element_type=\"container\" data-e-type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-62b6e77d elementor-widget elementor-widget-heading\" data-id=\"62b6e77d\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">The chain reaction: from crash to total failure<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-5dafae03 e-con-full e-flex e-con e-child\" data-id=\"5dafae03\" data-element_type=\"container\" data-e-type=\"container\">\n\t\t\t\t<div class=\"elementor-element elementor-element-6eba1bab elementor-widget elementor-widget-text-editor\" data-id=\"6eba1bab\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>On a perfectly normal Tuesday, the inevitable happened.<\/p><ul><li><strong>Step 1:<\/strong> The SAN cluster in the first data center (let's call it Cluster A) suffered a serious, unrecoverable hardware failure. Several controller components failed simultaneously. Exactly the scenario for which our redundant setup was built. Actually no reason to panic.<\/li><li><strong>Step 2:<\/strong> As the VM with the Quorum Witness had its virtual hard disk on the storage of Cluster A, the QW also crashed with the cluster. It was gone from one second to the next.<\/li><li><strong>Step 3:<\/strong> Now the second, actually completely intact SAN cluster (Cluster B) in the second data center was looking at the situation. Its status checks revealed two things at the same time:<ol><li>The connection to the partner cluster (Cluster A) is dead.<\/li><li>The connection to the referee (Quorum Witness) is also dead.<\/li><\/ol><\/li><li><strong>Step 4:<\/strong> According to the iron rules of cluster logic, Cluster B now had to assume the worst. From his point of view, it was possible that Cluster A and the Quorum Witness were still running and only he, Cluster B, was isolated. To prevent a split brain at all costs, he did the only safe thing: He declared himself as the \"split-brain minority\" and disabled access to all storage volumes.<\/li><\/ul><p>The result was a digital cardiac arrest. From one second to the next, around 200 virtual machines lost their hard disks. The entire system came to a standstill. Total failure.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-5a342b4f e-flex e-con-boxed e-con e-parent\" data-id=\"5a342b4f\" data-element_type=\"container\" data-e-type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-35e018f4 elementor-widget elementor-widget-heading\" data-id=\"35e018f4\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">The \"War Room\": 24 hours of technology and humanity<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-2b7f2f81 e-con-full e-flex e-con e-child\" data-id=\"2b7f2f81\" data-element_type=\"container\" data-e-type=\"container\">\n\t\t\t\t<div class=\"elementor-element elementor-element-3e5dfdf2 elementor-widget elementor-widget-text-editor\" data-id=\"3e5dfdf2\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>What followed was one of the longest crisis calls of my career. The \"war room\" was virtual, a WebEx conference that was supposed to run uninterrupted for 24 hours. A little anecdote in passing: back then, WebEx meetings had a hard time limit of 10 hours. In the middle of the night, at the height of the crisis, we had to frantically set up a new conference and migrate all participants to it.<\/p><p>But what I remember most from these 24 hours was not just the feverish technical troubleshooting. It was the human cohesion. The core team of storage and VMware specialists worked through the night, trying to revive the cluster manually and without a quorum.<\/p><p>The special thing was that many other team members, whose specialist areas (e.g. applications, databases) were not directly in demand at that moment, voluntarily stayed on the line. For hours on end. They were simply there to cover the active technicians' backs, shield questions from outside and provide moral support. It was a tacit promise: \"We are here. You are not alone.\"<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-7eb4b08a e-flex e-con-boxed e-con e-parent\" data-id=\"7eb4b08a\" data-element_type=\"container\" data-e-type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-560e9ba1 elementor-widget elementor-widget-heading\" data-id=\"560e9ba1\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">The lessons learned: What we took away from the abyss<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-2018694d e-con-full e-flex e-con e-child\" data-id=\"2018694d\" data-element_type=\"container\" data-e-type=\"container\">\n\t\t\t\t<div class=\"elementor-element elementor-element-6ec48e70 elementor-widget elementor-widget-text-editor\" data-id=\"6ec48e70\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>After more than 24 hours, the system was up and running again. The \"black start\" had succeeded. But the relief was mixed with the bitter realization of how fragile our \"bulwark\" had actually been. The lessons we learned from this have shaped my work as an architect to this day.<\/p><ul><li><strong>Lesson 1: The pitfall of the temporary solution.<\/strong> This is the most important lesson. The phrase \"We'll get it right later\" is one of the most dangerous in IT. A makeshift solution that goes into production is no longer a makeshift solution. It is a ticking time bomb. Murphy's Law guarantees that it will go off exactly when the conditions are most unfavorable.<\/li><li><strong>Lesson 2: Understanding true redundancy.<\/strong> Redundancy is not a feature that you buy. It's a concept that you have to understand and implement end-to-end. We had redundant servers, redundant storage controllers and redundant sites. But we had overlooked a fatal logical dependency that undermined the whole concept. You have to analyze the entire chain of dependencies, not just the individual links.<\/li><li><strong>Lesson 3: Accessing the documentation.<\/strong> The irony of it all? Our detailed contingency plan, the documentation for restoring the SAN cluster, was... on a wiki. Its VM was on the failed cluster. We didn't have access to our own recovery instructions when we needed them most. Since that day, I've been preaching: Critical emergency documentation belongs in an independent, off-site location (cloud storage, a printed manual in the vault - anything not affected by the outage itself).<\/li><li><strong>Lesson 4: The human factor.<\/strong> In the deepest crisis, when technology fails, it's the team that counts. Technical ability is one half of the equation. Team spirit, trust and mutual support are the other. A team that sticks together can solve almost any technical problem. This incident was impressive proof of that.<\/li><\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-1a9dccc5 e-flex e-con-boxed e-con e-parent\" data-id=\"1a9dccc5\" data-element_type=\"container\" data-e-type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-7bd16083 elementor-widget elementor-widget-heading\" data-id=\"7bd16083\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Conclusion: More than just a memory<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-16a51957 e-con-full e-flex e-con e-child\" data-id=\"16a51957\" data-element_type=\"container\" data-e-type=\"container\">\n\t\t\t\t<div class=\"elementor-element elementor-element-654a1a4e elementor-widget elementor-widget-text-editor\" data-id=\"654a1a4e\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>This story is more than just an anecdote. It is a plea for care, for a deep understanding of the systems we build, and for a healthy dose of paranoia. It is a reminder that the most complex systems can fail in the simplest of places.<\/p><p>I invite you: Take a moment and think about your own systems. Where are your \"provisional solutions\" hiding? What dependencies have you perhaps overlooked? Is your emergency plan really accessible when the emergency occurs?<\/p><p>Such incidents are painful, expensive and nerve-wracking. But the most valuable lessons about engineering, design and teamwork are often only learned in the face of a real crisis. They make us better engineers, better architects and better colleagues.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-c8591bb e-flex e-con-boxed e-con e-parent\" data-id=\"c8591bb\" data-element_type=\"container\" data-e-type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-ce87b49 elementor-widget elementor-widget-text-editor\" data-id=\"ce87b49\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>This is exactly the kind of experience we share in our blog. If you want to find out more about how we use technology, overcome challenges and develop solutions together, take a look at our other articles, e.g. on our <a href=\"https:\/\/35x.de\/en\/cloud-basics\/version-it\/\">Approach to versioning<\/a>. Perhaps you will find exactly the inspiration you need there.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>","protected":false},"excerpt":{"rendered":"<p>Schwarzstart: Wie eine einzige VM eine hochverf\u00fcgbare Infrastruktur in die Knie zwang Haben Sie auch eine dieser Erinnerungen? Eine dieser Geschichten aus dem IT-Alltag, die man bei einem Kaffee (oder einem Bier nach Feierabend) erz\u00e4hlt und die bei Kollegen ein wissendes Nicken ausl\u00f6st? Eine Geschichte, die einen auch Jahre sp\u00e4ter nicht losl\u00e4sst, weil die Lektionen, [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":2823,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_uag_custom_page_level_css":"","site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[27],"tags":[8,17,7,16],"class_list":["post-2822","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-lessons-learned","tag-blog-post","tag-cloud-journey","tag-cloud-migration","tag-cloud-reise"],"uagb_featured_image_src":{"full":["https:\/\/35x.de\/wp-content\/uploads\/2025\/07\/vm-vs-hochverfugbare-infrastruktur.jpg",480,720,false],"thumbnail":["https:\/\/35x.de\/wp-content\/uploads\/2025\/07\/vm-vs-hochverfugbare-infrastruktur-150x150.jpg",150,150,true],"medium":["https:\/\/35x.de\/wp-content\/uploads\/2025\/07\/vm-vs-hochverfugbare-infrastruktur-200x300.jpg",200,300,true],"medium_large":["https:\/\/35x.de\/wp-content\/uploads\/2025\/07\/vm-vs-hochverfugbare-infrastruktur.jpg",480,720,false],"large":["https:\/\/35x.de\/wp-content\/uploads\/2025\/07\/vm-vs-hochverfugbare-infrastruktur.jpg",480,720,false],"1536x1536":["https:\/\/35x.de\/wp-content\/uploads\/2025\/07\/vm-vs-hochverfugbare-infrastruktur.jpg",480,720,false],"2048x2048":["https:\/\/35x.de\/wp-content\/uploads\/2025\/07\/vm-vs-hochverfugbare-infrastruktur.jpg",480,720,false],"trp-custom-language-flag":["https:\/\/35x.de\/wp-content\/uploads\/2025\/07\/vm-vs-hochverfugbare-infrastruktur-8x12.jpg",8,12,true]},"uagb_author_info":{"display_name":"Thomas Ristic","author_link":"https:\/\/35x.de\/en\/author\/admin\/"},"uagb_comment_info":0,"uagb_excerpt":"Schwarzstart: Wie eine einzige VM eine hochverf\u00fcgbare Infrastruktur in die Knie zwang Haben Sie auch eine dieser Erinnerungen? Eine dieser Geschichten aus dem IT-Alltag, die man bei einem Kaffee (oder einem Bier nach Feierabend) erz\u00e4hlt und die bei Kollegen ein wissendes Nicken ausl\u00f6st? Eine Geschichte, die einen auch Jahre sp\u00e4ter nicht losl\u00e4sst, weil die Lektionen,&hellip;","_links":{"self":[{"href":"https:\/\/35x.de\/en\/wp-json\/wp\/v2\/posts\/2822","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/35x.de\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/35x.de\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/35x.de\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/35x.de\/en\/wp-json\/wp\/v2\/comments?post=2822"}],"version-history":[{"count":10,"href":"https:\/\/35x.de\/en\/wp-json\/wp\/v2\/posts\/2822\/revisions"}],"predecessor-version":[{"id":2837,"href":"https:\/\/35x.de\/en\/wp-json\/wp\/v2\/posts\/2822\/revisions\/2837"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/35x.de\/en\/wp-json\/wp\/v2\/media\/2823"}],"wp:attachment":[{"href":"https:\/\/35x.de\/en\/wp-json\/wp\/v2\/media?parent=2822"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/35x.de\/en\/wp-json\/wp\/v2\/categories?post=2822"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/35x.de\/en\/wp-json\/wp\/v2\/tags?post=2822"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}