Book Recommendations

OK, so you're a good developer. Your manager seems to be doing a decent job. Your team gets the job done.

But things could be better. You could use a few tips and tricks here and there. Heck, everyone on the team could. And your manager? You don't even have to say it: managers need all the help they can get, right?

So here's some good news: there's help out there. In fact, there's expert help. People who know more than any of us do about this stuff. People who have seen it all, and done it all. And they wrote it down.

I highly recommend the following books, written by people who really seem to know what they're talking about.

• Code Complete by Steve McConnell: This is, simply put, the most important book on my bookshelf. This is the developer's bible. It's the end-all, be-all for how to write the best code you can. The clear, concise, decisive suggestions in this book should become the standard by which your code is measured.
• The Mythical Man-Month by Frederick P. Brooks, Jr.: This classic series of essays on software engineering puts forth what may be the most important and overlooked lesson to learn in our field, now known as Brooks' Law: "Adding manpower to a late software project makes it later." The book will help you understand why our work is often difficult to estimate.
• Peopleware by Tom DeMarco and Timothy Lister: This is the book you'll want to have in hand when you go to your manager and ask for an office instead of a cubicle. Or a conference room for your project. Or at least a pizza for the team.
• Agile Project Management with Scrum by Ken Schwaber: Understanding the value of agile development is important for any modern developer, and this introduction to Scrum will help you understand how to use Scrum on an agile project. It's helpful for managers and developers to understand the Scrum methodology.
• eXtreme Programming Explained by Kent Beck: This short introduction will explain the much-hyped development methodology known as eXtreme Programming. It may get your interest, or it may send you screaming into the distance.
• Death March by Edward Yourdon: Though you never want to find yourself on one of these projects, this book is still a good guide to making sure that your projects are not destined to become death marches. It gives an interesting perspective on what "success" means on a project.
• Software Estimation: Demystifying the Black Art by Steve McConnell: Hey, look: another Steve McConnell book. This one is extremely helpful for understanding that thing we all seem to do poorly: estimating how long it will take us to complete a project. It turns out that our ability to estimate accurately is the key to our own perceived success within our organization. It's our ability to hit our own targets that impresses product owners, not necessarily our ability to hit their targets.

That's my short list. For further (and even better) recommendations, read these books, and look for the books these authors reference and recommend. Again: these guys are the ones who really know what they're talking about.

Update: I totally forgot one of the more important books I've read on the topic of application development: Alan Cooper's The Inmates are Running the Asylum. It is a great introduction to the idea of designing an experience, not just an application. I'm currently reading Cooper's About Face, which is essentially a practical guide to applying the lessons learned in Inmates.

Ss.

Why Agile? (And What is Agile, Anyway?)

[originally posted on 7/18 on "Dev Blog"]

I'm hoping that by now everyone on our team has heard mention of agile software development, whether from someone else on the team or from a web site or a conference or another developer. Agile is a strategy for software development that favors frequent iterations, quick response to ever-changing requirements, and highly empowered cross-functional teams. There are a handful of popular agile methodologies, including Scrum, Extreme Programming and the Microsoft Solutions Framework for Agile Software Development.

Agile vs. Waterfall
Agile contrasts heavily with the more traditional waterfall strategy for software development. In a waterfall approach, the team follows a strict serial process through requirements gathering, analysis, design, construction, testing, and deployment. Each of these steps has a set of outputs, and no step can begin until the deliverables are received from the previous step. In a waterfall approach, the entire software product must be well documented before work on it begins. For this reason, waterfall is often referred to as a predictive approach.

Agile, on the other hand, is an adaptive approach. Agile methodologies acknowledge right up front that no amount of effort in requirements gathering will result in an accurate prediction of what the eventual released software product will be like. Because business needs change over the course of time (even during development!) the development process needs to be able to keep pace with these changes. In fact, agile methodologies treat requirements change as the normal state of affairs. Agile methodologies adapt to that change constantly, and in fact embrace constant change as the best approach to creating a healthy software product.

Elements of Agile
The most important strategic element of agile methodologies is iterative development. Agile developers create frequent releases of their product. The different specific methodologies prescribe anywhere from two to six weeks as the preferred iteration length. That's right: some agile developers are releasing their product every two weeks. It turns out that some of those releases will only be released internally, but each release must be fully prepared to be put into production: tested, debugged, and compiled for release.

During and between releases, product owners can work with team management to prioritize and document the functionality that will go into upcoming releases. At any time in a product's lifecycle, product owners can add requirements to the project. The way to keep that under control: new requirements are prioritized when they come in, and are not added to a release until they are deemed the most important functionality candidate before an upcoming release is defined and scheduled. In other words, any requirement can get in line for development, but only the highest priority requirements go into any given release.

During the development of a release, the team works together to design, construct, and test all of the functionality for that release. Teams manage themselves, defining the tasks needed to fulfill the requirements for a release, and working closely with each other to make sure everything gets done. The team does its own estimates, and decides what can and can't be done within the allotted time for a release. The team commits to "chewing what it has bitten off," and the team is responsible for its own management and delivery on its commitment.

The Big Difference
The greatest advantage to an agile approach to software development is one simple thing: agile gives product owners feedback early and often. Think about it this way: If you were starting out on a waterfall project, and you estimated that the project might take a year to develop, you would break the project down into sections: maybe two months for requirements gathering, another two for design and architecture, six months for construction, two months for testing and debugging. In that plan, the first time the product owner sees a release-worthy candidate is after ten months (after construction is finished). At that point, the business is ten months further down the road, the requirements are eight months old, and there's approximately a 100% chance that the product owner (or some other stakeholder) will look at what you've produced and say "Oh, wait. That's not what I need. Can we change it?"

In contrast, an agile project would get underway immediately, building the highest priority requirements. The foundation of the system would be laid early on, so there's a good chance that after the first release -- one month into the project -- you'd have a release, but it wouldn't have much to show. In the second iteration, though, you'd be showing important functionality. At the end of months two, three, and four, you'd be demonstrating the basic functionality of the product, in a releasable form -- looking polished, tested and debugged, ready for prime time (sans the lower priority requirements). That's when the product owner says the same thing: "Oh, wait. That's not what I needed." So now you have the chance to change what goes into the next iteration, without any sort of cataclysmic failure implied. The product owner updates the list of prioritized requirements -- just like they would before any release -- and you proceed with the new priorities.

Agile provides the opportunity to roll with the punches -- the punches we know are coming -- and to be successful in a realistic business environment. And in this example, you find out eight months earlier that something needs to change. Nice!

Successful business strategies require agility: the ability to change when necessary, and to adapt to a changing business environment. Agile development allows a dev team to keep up with the ever-changing needs of the business we support.

Ss.

The Case for Code Review

[originally posted on 7/10 to "Dev Blog"]

Code Review is a fun topic, and much like unit testing, it garners a lot of interest from dev teams -- and a lot of promises from dev managers -- but in the end it rarely sees the light of day. Code reviews just seem to take up too much time, and they can be difficult to organize and standardize.

But let us not forget: Code reviews are important. There are all sorts of studies and success stories that tell us just how important code reviews are. The stats suggest that code reviews generally catch between 20 and 70 percent of the bugs in the code being reviewed. That's a pretty good success rate!

Of course there are different methods for code review, and the specific development methodology that applies to your project will often dictate what types of code review are possible. For example, if you are working on a waterfall project (such as one using the CMMI methodologies), you may have time set aside in the schedule for design, then construction, then testing. The testing portion of the project would then be the best time to perform code reviews. The reviews should be built right into the schedule. The time you have allocated for code review may dictate which types of review you choose (I'll introduce a few types below).

If you're using an iterative development methodology (Agile, Scrum, eXtreme Programming, etc.), you'll want to build code reviews into each of your iterations. It is important in these iterative methodologies to find a code review technique that fits well into the tight iteration schedule. In any methodology it's important to make sure your code review technique is a good fit for your team's development style.

Whatever code review technique your team uses, it's important to have specific goals for the code reviews. In general, I've found that it's important to look for two things in a code review: Does the code do what it is supposed to do, and does the code meet the team's coding standards. Of course, it's important that the team actually have a set of coding standards to which the code is compared before that standard can be applied. I'll try to cover that in another article.

Here are some of the approaches to code review that I've been introduced to:

• Formal Inspections: Formal inspections are a highly structured form of review that uses a team of reviewers in specific roles, following focused checklists in their search for defects in your code. They require a certain commitment to a review team, and that can mean a lot of overhead. But the return is also significant: Generally formal inspections find 45%-70% of the defects in the code reviewed.
• Walk-Throughs: This is a very informal style of review, usually structured as a team meeting in which code is discussed in a matter-of-fact, highly technical manner. Experts on the team offer guidance, while other team members point out possible issues and ask pertinent questions about the code under review. While they can be quite educational, the return is low: only 20%-40% of defects are found using this technique.
• Code Reading: Code reading is a more independant style of review, in which the code being reviewed is delivered to the reviewers, who then return their findings (defects, suggestions, etc.) to the code's author, who then makes the proper changes. A meeting to go over the results is optional, and often of little use for finding new defects. The return for code reading is a bit higher than walk-throughs: 25%-65% of defects.
• Pair Programming: While not generally thought of as a code review technique, pair programming warrants mention here because it has review built right into the process of development, and because it is a great way to reveal defects in code. The typical return is 40%-60% of defects found.

My personal preference is strongly rooted in my preference for an iterative development methodology. Given a typical iteration length of one month, code reviews have to fit nicely within that month, and their time on the schedule has to compete with time for design, construction, testing, and deployment. Meetings can prove to be a burden for developers who are trying to maintain momentum on their development tasks, so I prefer a review process that is asynchronous. Of course some code needs more experts reviewing it than other code might, especially if it's going to provide any sort of interface to other systems.

I think a healthy combination of formal inspection and code reading fits the iterative development style well. For code that interfaces with other systems, formal inspection is important before the code is released, and should involve team members from the projects with which the code will be interfacing. For all internal code, the code should be delivered upon completion to all of the other developers on the team, who will then review it, returning their findings to the code author. All code should be reviewed in this way, and code should not be marked "complete" until each developer on the project has had a chance to review it. In any given iteration the amount of code reviewed should not prove to be overwhelming. It's important not to ignore the reviews in an iteration, simply because it will cause defects to pile up, and future reviews will take much longer.

That is, of course, just my preference. We'll need to find the perfect fit for each of our projects and methodologies.

It's important that we find a way to work code review, in one or more of its many forms, into the work we do every day. Whether it's highly formal and performed at regular intervals, or informal and part of our iterative process, our code needs to be reviewed before it goes live. Why leave our bug-finding to the testers? Why wait for our users to find defects, when our teammates can find them more quickly, more efficiently, and before they go live?

Besides: it's fun to find your friends' bugs.

Ss.

Unit Testing: How Do YOU Do It?

[originally posted 6/2 on "Dev Blog"]

As developers, we all understand the importance of unit testing. We know that our code needs testing, and we know that before it goes to anyone else, we have to be that first line of defense against defects. At its simplest, unit testing is just taking a moment to try out your code. None of us is silly enough to think that we don’t need to try out what we’ve built.

But there are different schools of thought on unit testing, and different approaches to it. While simple unit testing isn’t very thorough, it’s easy to do and it’s a fundamental part of how we write code. Especially with web applications, it’s typical for a developer to write code, compile, and view the results in a browser – all with just a few keystrokes. Think back to the days when compilation could take hours (or even days), and you’ll realize that we’re fortunate to live in an era of quick compilation and quick results. And that’s just the point: we rely on those quick results to show us that our code is indeed working correctly.

A more thorough approach, however, is to use test-driven development, like that supported by the xUnit frameworks (NUnit, JUnit, etc.). This approach is an integral part of eXtreme Programming, adopted because it provides immediate quality feedback, which is important for any type of agile development. It turns out that it’s a very smart approach within any methodology (Agile, Waterfall, etc.). While test-driven development adds work to the development process, it pays off in a big way by decreasing the work in debugging. All reports indicate that the gains far outweigh the costs.

I’m not familiar with many other approaches, but I’m sure they are out there. What I would like to know is How do YOU do unit testing? As a new member to the team, I haven’t yet seen my teammates in action. So I thought I would ask. I’d like to hear not just from Max and Luis, who lead our development efforts, but also from the rest of the developers.

To be clear: I’m not asking how your apps get tested after they’ve been developed. I want to know how you test your apps during development. How do you know, on a day-to-day or moment-to-moment basis, that your code is doing what it’s supposed to do? I’m hoping you don’t just compile and pray. What process do you follow, probably between the time you compile and the time you check your code in, to make sure that it’s working properly?

Here’s my painfully honest answer: I have been getting along using the simple, less thorough method I mention above. It’s unit testing, but it’s not the kind I’d be proud of. I want to move to test-driven development, and use NUnit. It strikes me as a smarter and more efficient way to get things done. So I wrote this email to see if others on the team are using it, and if maybe they have some tips.

Anyway, let me know how you unit test. I’m curious to see where we all stand now, and where we’d like to take the team (if anywhere). Maybe I just need to catch up with the rest of you!

Ss.

Little Bobby Tables

[originally posted 5/23 on "Dev Blog"]

Instead of a long-winded essay on anything at all, this week I want to show the team something funny:

(Also available at http://xkcd.com/327/)

I have only this to say about it: If anyone on the team does not get the joke, they should immediately talk to Max, Luis, or myself (or anyone on the team that gets the joke). We all need to know why it is funny to name your child Robert’); DROP TABLE Students; --

Ss.

P.S. Around the house, my wife and I like to refer to our daughter Vivienne as “Little Vivi Tables.”

Stored Procedures and Database Logic in Source Control

[originally posted 5/16 on "Dev Blog"]

Stored procedures are good. Those of us working with SQL Server know it’s true. They allow us to centralize our data logic. They speed up our data access. And they increase the security of our applications.

Source control is also good. No matter what technology we work in, we all know this is true. It gives us access to the historical versions of our code. It gives us features like code branching, labeling, and of course checking out and locking files. In general, it saves our butts.

So it follows that source control for our stored procedures is very good. While this makes sense in theory, it’s not always obvious when you’re working on a project that the database logic should go into source control. And in working with SQL Server and .NET technologies, source control isn’t offered automatically, as it is when you create projects for web sites, class libraries, and other project types. In fact, when you work in SQL Server Management Studio, source control is often completely forgotten about (even though it integrates with SQL Server Management Studio in the same way it does with Visual Studio).

But source control for our database code gives us all of the same benefits that we know and love for our non-database code. And, in general, it saves our butts.

The easiest way to get your database logic into source control is to add a Database Project to your solution in Visual Studio. It provides a structure for your database code, and if you add a few standard folders that correspond to the ones you see in SQL Server Management Studio (Tables, Views, Stored Procedures, Functions, Triggers, etc.), it can help others find your code easily. In Visual Studio 2008, the database project also includes an example of a test script, which pushes us nicely toward database unit testing. You can even debug the script from within Visual Studio!

And it’s easy to add a database project to source control: it works the same as any other project.

There is, of course, a catch: Database projects aren’t available in all versions of Visual Studio. And I think Professional (the version most of us are using), doesn’t have them.

But that’s no reason not to put our database logic into source control. The strategy for using source control without a database project is simple. When you are working in SQL Server Management Studio, create files for each of your stored procedures, table definitions, views, etc. Organize them in the same way you would within a database project. And then put the files into source control, either directly within SQL Server Management Studio, or from Visual Source Safe itself. Again, the project types and Source Safe integration available to you from within SQL Server Management Studio may vary based on the version you have installed, so use the features that you have available to you.

As you work with your database logic – and usually that will mean editing or creating stored procedures – always work from the files that are under source control. Always check them out before you use them, and don’t forget to check them back in after you’ve run them on your target database.

Here’s a tip: In your stored procedure files, add a conditional drop statement to the top of the file, so that you can run the entire file each time you update it. At the bottom of the file, add the permissions you’d like to associate with the stored procedure (if any), so that you never have to edit those permissions by hand.

Stored procedures are some of our most valuable code. When it comes down to it, our database logic needs and deserves the protections provided by source control at least as much as the rest of our code.

Ss.

Remove Methods That are Not Called

[originally posted 5/9 on "Dev Blog"]

As we write lots and lots and lots of code, and our codebase expands beyond mere kilobytes and megabytes to the gigabyte range, it’s important to keep code bloat under control. One of the easiest ways to keep our codebase expansion in check is to remove code that is no longer in use. This usually means removing methods that are not called – though sometimes you can find entire classes to remove.

There are a handful of important reasons to remove old code. There are the general, feel-good reasons:

• Less code is easier to maintain
• Less code means faster searches
• Clean code is much easier to read, and that saves us all a lot of time

Then there are the practical reasons:

• Unused code can have bugs in it, and if someone were to somehow come across those bugs, and try to fix them, they are unwittingly wasting a great deal of time.
• Often we make global changes to implementation details (for example, we change how we access our database, or we change how we raise and log errors). If a developer is making changes to an entire application, then any time spent on unused code is wasted time.
• Upgrades to an application often require that all of the methods in the application be reviewed for possible updates. Any unused methods will cause another big fat waste of time.

The first step in removing unused code is finding the code to remove. The easiest way to find it is to pay attention to the methods that you are rendering obsolete as you are writing code. For example, imagine you have a method with the signature ValidateUser(string username, string password), and you decide to replace it with a new method with the signature ValidateUser(string username, string password, int applicationID). Once you’ve written the new method, and then replaced all calls to the old method with calls to the new method, you are now in a perfect position to remove the old method completely. You know it’s not being called from anywhere, because you just removed all calls to it. In fact, if it’s called from somewhere else, that’s a bug that you need to fix right away. This situation is the best of all possible opportunities to solve the unused method problem: if you remove code just as soon as it is rendered obsolete, you won’t introduce additional unused methods.

Unfortunately, sometimes unused code slips through the cracks, and gets left behind. The next best time for finding that code is when you are working in the code, whether you’re implementing new features or updating old ones. As you become familiar with an application, you get to know what code is used and what it is used for. The developer most familiar with a section of code should know what each method is used for, and should have a sense of what is not being used at all. If you come across code in one of your projects that you know has been left in when it should have been removed, remove it right away. It doesn’t take long, and it will save you and your teammates time in the future.

Of course there’s always that last, dreaded occasion for finding unused methods: when you are handed a project that was written by someone else, and asked to adopt the project as your own. You get someone else’s code, with no idea which methods are used, and which are not. Many developers would cringe at the thought of combing through an entire project, looking for unused methods. After all, it’s tedious and thankless: no one ever gets a big bonus for the code they deleted. But remember that every unused method represents countless wasted hours in the future. You can’t really afford not to find them and remove them.

In this situation, I haven’t found a better method than a simple search and destroy. I walk through every single method in a new project, search for its name in the rest of the application’s code, and delete it if I don’t find any. It takes a long time. But again, it’s nowhere near as much time as I would otherwise spend maintaining, updating, and otherwise babysitting completely unused code.

There are only two reasons I can think of for not removing unused methods:

• We might need the methods in the future
• The methods might be called from outside the application

The first of these methods may or may not be true. It’s hard to say what we’ll need in the future. But in a source controlled environment, the point is moot. We can always go back to source control and pull a copy of the deleted method. Deleted code doesn’t die: It just retires to VSS.

As for methods being called from outside an application, this may be a legitimate concern, depending on how our applications interact with each other. Methods that are called from outside an application should be clearly marked as such, with a continually updated listing, if possible, of the applications that call them. We need to be able to delete unused code, so any method which is not marked as being called from outside the application must be treated as a candidate for deletion.

Unused methods are a natural product of code improvement. As our applications are developed over time, new methods are introduced, existing methods are updated, and eventually some methods fall by the wayside. But leaving unused methods behind creates a mighty clutter, and wastes all of our time.

Ss.