Lesson: Agile Development with Ian McFarland

Step #1 Agile 101: Agile development is critical for software flexibility, rigor, and adaptability to change

It was really just a collection of taking best practices and amping them up, specifically Extreme Programming, Kent Beck's work around Extreme Programming. A lot of that idea was just, "We know that testing is good, so we'll test all the time. And, in fact, we'll write tests first. We'll write a test, failing test, that demonstrates what we want the behavior of the code to be. Show that it fails, and then implement just enough code to make the test pass."

This is also where the name "Extreme Programming" comes from. It's taking all these best practices, like testing, talking about your code, code reviews, etcetera, and making them extreme, like taking them to the logical extreme, doing more of them because we know they're good. So talking through your design is something that developers will do naturally, but people don't think to do it all the time, people would only do it when there's a really hard problem.

The fact is, as software tools get better and better, the percentage of time that you're spending working on boring boilerplate code goes way, way down, almost to zero. And you really are engaging in more and more interesting stuff all the time, and so it's more and more important to have that ongoing conversation around what you're building and to think through your problem before you start committing to code. And so that turned into pair programming, the idea of, "Oh, I'm going to sit down with you. I'm stuck on something, I'm going to come sit down with you and ask you something." That just turned into, "Let's do that all the time."

That's across the board, all the practices that you see with deployment, right? We know that getting things out the door really quickly is important, so let's focus on making deployment easy. And, in fact, let's get to where we can do continuous deployment. And so the high art of good Agile process is to be able to have enough testing in place so that every time you commit a tiny little change it's tested enough that it can go live and you can see right away if it does what you need and if it's improving the quality of your product. You get real user feedback as quickly as possible.

A bunch of Agilists from back in the day distilled down their thinking into a few key ideas that really are the ethos out of which all of this stuff comes. Valuing functioning code over documentation, things like that, like simple principles. They're not saying the things on the left that we don't value as much aren't as good, I mean, that they're not valuable, but that they're not as important as the thing on the right. Like individual delivery isn't the valuable thing, it's how the team delivers that matters.

This flavor of concept is the kind of thing that is captured in that document. And it's a short read, its one page, it all fits on one page. It still continues to be treated as something that's sort of a litmus test to go back to, to ask to reground yourself in why it is we're doing all this stuff.

What ends up happening with Agile very often is it ends up being very dogmatic. Any time you have dogmatic Agile, you've completely failed, you've completely missed the point. It isn't a matter of which point system you use or do you do planning poker. It's, "Do you product a result that is meaningful?" These tools are all really useful and can be applied appropriately. But if you apply them prescriptively, you lose a lot of their strength.

The fact is, when somebody is new to a technique, giving them some really concrete tools is really useful and sometimes it's easier to start from a very structured top-down, "You will do these 12 things." But really quickly you need to really step back and say, "Is this solving what we need to do?" The challenge is it's very easy to then also kid yourself that, "Oh, it's not working, we're going to go back to our old ways." So it's a fine line, really, between doing it really well and doing it really poorly.

When you think about Agile and you want to go implement Agile, there are a bunch of pieces. Test-driven development is a huge one. There's an idea in an Agile world, that you really want to be conservative about building stuff until there's a real customer need that drives it, that's also a Lean principle.

So the idea is that in order to do that, you express in a test, in a piece of code that validates that some other piece of software does this feature that you want. Like "I want to be able to log in. I want to be able to, where I take these two accounts and sum them, it should produce the right result." So the idea with test-driven is that you write that test before you write any code that implements the behavior and you do this for a few reasons.

One, by expressing what it is you're trying to do, there's a rigor that you're only going to write enough code to actually make that thing be true. You're not going to go off and do two months of design work to figure out the best way to sum these two account balances. You're just going to write the test. It says that when you do this thing, this thing should happen. And it doesn't. So you're rigorous about writing just enough code so that that does happen. What ends up happening in your code base is you end up with a much simpler code base that only does the things that you want so you don't have a lot of extra complexity in your code. And you also have a very rigorously tested code base so that you know later down the road that when you change how account balances work, that you did or didn't break something.

People talk about wanting to make sure that you didn't break anything. Usually what that means is that you need to know when you did so that you can go fix it. So it's really easy to go fix things at the time you're making the change, because you also have all the mental model in place of what you're trying to accomplish. The test tells you what other thing you didn't think about in your constraints. And it helps you to solve more discreet problems. So that's testing, right?

A big other part of Agile development is pair programming. And the idea again is that you want to sit down with another person and most of what you're doing when you’re writing software is actually you're thinking through problems. "What is the best way to implement such and such?" People who don't program sort of think programming is this very structured, top down, "there is one answer to the question of how you do anything" in software. And that's just not true. There are as many or more answers to how to do something as there are programmers. And so there's a lot of art in how you do software development well.

So the idea of taking two smart people and having them work on the problem together, you're going to come up with a better answer than you would on your own. It's going to be more rigorous. It's going to be more thought through. Things that you didn't think to ask yourself, the other person will help to fill in the blanks. So pair programming is sort of a continuous code review that's a core thing that we do in a good Agile process.

The other thing that good test programs lets you do is it lets you make deep changes and know when you're breaking something again so that you can fix it. That lets you make really fun, really routes changes to something, know that you have the same behavior, know that you're not missing something, and lets you keep the code as concise, we call it well-factored code is code where responsibility is isolated. You're only doing this thing that does this complex operation in one piece of code. So you keep your code well-factored, which means that then it's much simpler to come back and make changes to it later. Or to add behavior. Or to remove behavior. Or to modify behavior.

So by having this test coverage, you have this red/green refactor cycle that you get into. You start with a test that expresses the behavior you want. It fails because you haven't implemented the behavior. You write enough code to make the test green to make the test pass. And that's sort of Phase 1. And that's honestly in traditional programming where you kind of stop. And then a good Agile process would then be rigorous about refactoring that code. We've learned a lot from the process of writing the code. You then want to go back and do a cleanup step. And see "Is there duplication? Can I simplify the implementation of my code?"

One other thing about code simplicity, I talk a lot about how it sort of reduces your cost of maintenance and management of the code. But it also in general tends to perform better. Simpler code, there's fewer instructions that have to be operated. Again, there's less places for things to go wrong, because you have one place where the summing of accounts happens, or one place where log-in credentials are managed. And then so when there's a security issue, you know you've solved it everywhere because there is only one place that this issue occurs.

Agile development underlies everything that we do here now, it's critical to have that kind of flexibility and rigor in your software to be able to respond to change. Building flexible code base is about having adequate testing, ideally test first. If you approach it the right way you get a code base that is flexible enough that when you learn something new you can respond to that change, make a change rapidly, and know that you haven't broken anything. That's really the core flexibility piece; this is why Agile is so critical for high functioning teams, and for teams working in areas of uncertainty.