I recently read an article from a consulting firm about the benefits from moving from the, "waterfall process," to the "agile process." The article then proceeded to dutifully outline all of the major elements of agile: sprints, trains, daily scrums, stories, story management, cross-functional teams, etc. The article then talked about why changing process was good and organizations would see a strong uplift in the application development.
And the article wasn't wrong because it espoused falsehoods. It was wrong because it missed the heart of agile, and confused the output of agile with its input.
At its core, agile product development is a set of beliefs around building software to: 1) reduce process, 2) push responsibility to the person doing the work, 3) empower individuals, and 4) increase communications over heavier process.
The output of this methodology is very much what we generally thing of as agile. And, you need to guard against the re-appearance of process, as this is a natural quick-jerk reaction for most people, and takes high-level support to avoid.
I'll give an example. At one company I was at, we had a deadline and ended up pushing out something through Q/A which ended up being very buggy. The was viewed as a process breakdown -- Q/A had signed off on something not ready and hadn't had sufficient time. The result was to add more process, in this case: 1) a 'rule' that Q/A had to have at least a week, and 2) a layer 2 of Q/A to sign of on production releases.
This seemed reasonable. Think it worked? Yeah, right.
Instead, agile methodology would do away with the Q/A sign-off and have developers sign off on their work. Effectively, you blow-out of the process. The individual would be responsible for signing off on his/her own work and when it was ready. If the work turned out to be buggy, or there were integration problems, you would roll back (more about the environment later) and ensure that people communicated for easier integration. There would be lots of social pressure not to release buggy code. And that's it. And it works a lot better.
Also, the article missed another point about only having good engineers. Agile very quickly exposes people who are getting things done, and those who are not. Sit in on a handful of scrums, and you'll very quickly know the people who are getting things done, and who are not. Also, with agile, the bad engineers do not hold up the good ones or create roadblocks. I won't go into the decisions to exit bad engineers, but with agile, at a minimum, they don't cause problems like they can when they push themselves into the critical path.
And, your products get better. Users are horrible at articulating what they want, what they need, non-standard/edge cases, and knowing what they don't want. Users are much better at reacting. With agile, you get users reacting what they want, and your products improve tremendously.
My opinions come not from theory -- in theory, I love waterfall. It seems to make lots of sense to write requirements and figure out what you want. My opinions come from practice; I've never seen waterfall work well at all. The shift out of a process mindset is counter-intuitive -- I've heard dozens of times someone say something like, "But if I were going to build a building, I would want to know what it would be used for, make blueprints, know my materials....before starting to build." Needless to say, that's probably true for a building. It just isn't for software. Have the confidence that agile works. It works fast too.
Well, that's enough for my agile soap box.