Debug war stories

This post is a continuation of https://maarten.gent/posts/debugging/debug-complex-containerized-application/ A quick recap: we moved a complex 3-pod containerized application from OKD (Openshift) 3 to 4 and hit a network issue. Some lengthy debugging was required but we felt confident we could move our biggest customers to the new platform once resolved. Set the stage The weekend is family time (or blog time) but not this time: a switch died in the datacenter which brought the storage down of the old OKD cluster....

The story of how we resolved a hard to debug issue migrating a complex containerized application to a Kubernetes cluster. I learned some lessons that might be useful for you too. Problem statement Application Sorry for the vague description but the application involves three pods. One pod contains a container running an OpenVPN (client) process. Users access the application using a web interface and traffic flows back and forth through an VPN tunnel accessing a service running in a restricted network on the other end of the tunnel....