The Problem with Software Methodologies
Boris Demko was an incredible programmer. With 35 years of experience under his belt, Boris had done it all. He'd written professional-grade compilers, renderers, databases, and neural networks. He'd lead the engineering effort at several large tech companies while they were working on their flagship products. Even his side-project programming language, BorisC, would go on to accumulate 50k GitHub stars and thousands of users. Having immense experience in the successful delivery of software, Boris decided he would distill his software development and project management methodology into a simple 25-part process known as "Borisgile". Once published, the Borisgile manifesto (a book detailing the Borisgile methodology) instantly climbed to number one on HackerNews. The printed version became the best seller in Amazon's Computer and Technology book department. Developers across the globe began practicing Borisgile and preaching its benefits. Just about every major city began hosting weekly Borisgile meetups, where evangelists and newcomers alike would sit around and discuss their new religion. All the major tech companies began adopting the Borisgile methodology. From now on, all code was to be written in groups of 3, a strategy defined in Borisgile as "triplet programming". 1 and 2 week development cycles were overhauled and replaced with the flashy new "marathon", a unit of time defined in Borisgile as 36 hours straight of fingers-to-keyboard. Task estimation exercises like Planning Poker were replaced with their Borisgile equivalent: Planning Shots. In planning shots, team members, each of which is equipped with a bottle of Vodka, go through their backlog items 1-by-1 and estimate how many marathons each task will take. The catch is that you have to take 1 shot for every marathon point you estimate. So to estimate that a task will take 2 marathons to complete, you'd have to take 2 shots of vodka.
To the surprise of the major tech companies, the introduction of Borisgile didn't improve the iteration speed and quality of their software. In fact, it decreased it. Immensely. How could this be? The great Boris Demko used Borisgile to ship quality software in record time. Evangelists across the internet swore by its efficacy. Things just didn't add up.
We naively expect methodologies to provide cookie-cutter solutions to hard problems in complex domains. We deceive ourselves with lies like "I'll finish this in time If I work in 2-week sprints", or "My software will be reliable if I write my tests first". Just because methodologies can lead to improvements doesn't mean they're guaranteed to. Software is complex. Every problem and every project is unique. And no methodology, no matter how thorough, can encapsulate and provide instructions for all of the possible issues we'll face. We have to think critically to write good software in a timely manner. The moment we begin suspending our critical thinking in favor of memorized processes from the hottest new methodology is the moment our quality and iteration speed begin to slip. The problem with methodologies is that they undermine the importance of individual competence. And we're desperate enough to believe them.
A talented team of developers following no particular methodology will run circles around an inexperienced team practicing strict Agile. Similar to how a random mix of NBA players with no set offense will absolutely pummel a team of high-schoolers running the world's best offense. But we're so desperate for a sure thing, a solution to a hard problem without putting in the hard work, that we deceive ourselves into believing that an intricate system with fancy terminology will make up for our lack of experience. There's no shortcut to mastery, but we're tricked into thinking there is when a master shares their own methodology.
The majority of people following a methodology have 1% of the experience that the methodology's author does. And this discrepancy in experience leads to wildly different results despite following the same process. To Boris, a man who dreams in C++ and has spent 1/3rd of his life in a text editor, Borisgile is the road he uses to drive his software shipping Ferrari on. But to your typical programmer, Borisgile is a gloomy path to alcoholism and burnout. There are surely aspects of various methodologies we can benefit from, but before we blindly adopt them it's important to ask "why". Why am I doing this? What problem does this solve? Does the time spent following this process outweigh the opportunity cost of not doing other things?
There's no rule that says you must follow a single methodology. You won't be punished as a heretic for picking the best practices from a handful of methodologies. You will not be charged with blasphemy for no longer following a certain methodology when it no longer makes sense. Just as you (hopefully) don't unilaterally align with the opinions of a single ideological group, you shouldn't feel obligated to adopt all the practices of a single methodology. You should think critically about the problems you have and what processes might help. You should experiment with different processes, keep what works, and shamelessly disregard what doesn't. Ignore those who tell you to "trust the process", especially if this "trust" requires suspending your own critical thinking and experiences.
Many evangelists haven't been successful with the methodology they preach, but they preach it nonetheless, giving their audience a false sense of confidence in its efficacy. I suspect their willingness to preach something they haven't proven to work for themselves is a consequence of their neomania: an obsession with the new and the false belief that the newer something is, the better it is. From my experience the opposite is true, the older the solution the better. Older solutions have withstood the test of time. If something's been used to solve a problem for 30 years without being made obsolete by a superior solution, chances are it'll continue to be in use for another 30 years. Take Unix or Democracy for example, chances are they won't be replaced anytime soon. These techno-evangelical types preaching methodologies tend to be the same ones constantly advocating the use of new frameworks, dev tools, and programming languages. They're more concerned with the process than the product. Never forget that you're in the business of making good products, not the business of following good processes.
Software is a complex domain that necessitates critical thought. Although methodologies and the processes they advocate can help to tame this complexity, they are not solutions to it. Be extremely wary of any methodology that takes more time than solving the problem itself. If you spend more time creating tickets, planning sprints, and writing design documents than you do building software, you've got a big problem. Accept the hard truth that to solve a problem well takes time and effort, and is almost never achieved on the first attempt. Never lose sight of the ultimate goal: making good products.