Google I/O 2008 Keynote

Spend your next hour watching Marissa Mayer's keynote speech from Google I/O 2008. She introduces a whole lot of different ideas that Google has explored with respect to design as a science, user experience, and building highly scalable systems with a simple interface. She puts forward the concept of "imagination as a muscle," and how Google embraces the concept via brainstorming sessions, 20% time projects, and other ways to "flex" the imagination. I was there for the speech at Google I/O, and watching it again I'm still impressed with the insight she provides.

Ss.

Scrum for One

I have, over the past few years (and few companies), found myself on a number of "teams" made up of just one member. For those having trouble with the math: yeah, it's just me. There are plenty of books out there that can help software folks manage their teams and their software development process. But I see very little written about the topic of managing the single-person team. So I'll get the party started by providing a handful of tips for developers working solo. I'm an Agile devotee with a preference for Scrum, so I'll illustrate my points using Scrum-specific terminology. But the lessons are the same no matter what your methodology. Adjust accordingly. 

Process Matters

Intra-team communication is an important facet of any software development methodology. On a solo team that communication with other team members is all in your head. But that's no reason to drop the methodologies altogether. In fact, following a strict (albeit modified) process can provide numerous benefits. 

Scrum, for example, creates a framework for setting expectations with your clients. They will know, based on your adoption of Scrum, that they can expect to meet with you before and after each sprint to plan and review your progress. They will know when and where their input is expected. And they will know what sort of reporting to expect.

Scrum also helps you organize the requirements for your project into a manageable task list. It helps you and your clients prioritize your tasks. And it helps you keep track of what you need to do, and when you need to have it done by. These things are all very easy to lose track of when you are working alone.

The daily scrum meeting, though intended to be a discussion amongst team members, remains important on the solo project. Go over the three points you would on a larger team. Put them on a white board or in a notebook. What have you done since yesterday? What will you do today? What is getting in the way of your progress? Now you've got a framework for your day's work.

The Testing Challenge

The biggest challenge for a solo developer is testing your software adequately. The fact is, you can't do it alone. You need outside help. If you're in a solo situation, the decision has already been made: you're the tester. Your software will suffer because of it. So you have to come up with a mitigation strategy. My advice: strengthen your unit testing, and recruit your client's resources for user acceptance testing.

Your unit testing should follow a test-driven-development plan. Write your tests before you write any code. Do not write any code that is not specifically designed to pass your tests. Automate your tests to run with every compilation. Your code should emerge fully tested by the time your development is completed.

User acceptance testing should, in an ideal situation, be the last step in your development process. But without actual testers on the team, you have to get your users involved earlier. Show them your progress during development, so they can provide feedback early. Then recruit anyone who is providing input to the project (graphic designers, writers, etc.) to test your implementation of their input. Recruit random non-experts from your client's organization to try out your software from a a non-technical perspective. Help them help you: provide a test plan that explains what they should cover in their testing, and how they should go about it.

Reporting & Visibility

Scrum proscribes a specific set of reports that you should be providing to your clients: the sprint burndown and product burndown reports. Use these to their fullest extent: publish the reports somewhere where your clients can view them on a daily basis to get updates on your progress. Teach your clients the meaning of the line on the graph: as you make progress, the line slopes down toward the bottom of the graph. I've found that using Microsoft's TFS with the Scrum module installed automates the updates on that report, and provides a trend line that gives my clients an idea of whether or not I'm on target for delivery.

Keep your product and sprint backlogs up-to-date, so they are always useful to your clients. Again, publish them where your clients can get to the latest version. Your clients will appreciate the visibility into your project, and can use the information in those documents for reporting to others within their organization.

Presentation & Planning

Be diligent about your sprint review and sprint planning meetings. Set expectations with your clients about what will happen in each of these meetings. Make sure that your clients understand their roles, and who should be attending these meetings.

Put together a clear presentation for your sprint review. Highlight what you have accomplished. Provide a visual representation of all of your work, even work that has no user interface component. Use Visio to make a diagram if you have to. Give your clients something to see, rather than just something to read about.

Srum On!

Use Scrum (or whatever development methodology you have chosen) to its fullest. Don't be discouraged by the parts of your methodology that don't translate to the single-person team. Most of the components of any good methodology will be just as valid for you working alone as they are for a larger team.

Ss.

The One That Worries Me the Most

As a follow-up to my Top 25 Most Dangerous Programming Errors post, I want to expand on the error that has me the most worried. It turns out it's at the top of the list: Improper Input Validation. The folks who wrote the list call it "the number one killer of healthy software." It's not only highly prevalent, but it's downright easy to defend against. Here are some tips for fighting the good fight.

Understand where your inputs come from

When you're building a web application, your inputs come from four obvious sources:

• URLs
• Query Strings
• Forms
• Cookies

... and a couple of not-so-obvious ones:

• Other sites or resources on either your network or the web
• Your own data

URLs may not be all that obvious to some people. Traditionally URLs are paths to files on a file system, and those paths don't leave much room for malicious activity: if you specify anything other than a known path, you get a 404 error. Fair enough. But with the rise of RESTful development, more sites are using parameterized URLs. You'll see addresses like http://www.flickr.com/sackville (sorry, I haven't posted any photos yet). That username in the URL is input, and it gets processed. There's a SQL query done that uses 'sackville' as a parameter. That's a prime attack point.

Query strings and form inputs should be pretty obvious: they're prime targets for attack.

Cookies, which you may think of as innocuous because they're usually written by your app (rather than by a user), are just as vulnerable as any other input. The reason is simple: anything anywhere in your client's HTTP request (and that includes all of the above) should be suspect. Hackers use custom-built web browsers that emulate your favorite commercial browser, but submit customized requests. They bypass all of your client-side security measures, and submit their own URLs, query strings, form fields, and cookies. We'll get to that "bypass all of your client-side security measures" issue in a moment, and you'll understand why you still need them.

As for your own data and the other resources your app integrates with, the best strategy is to assume that someone has already gotten to those systems, and that their data has been compromised. If it's going into your system, validate it as if it were from an untrusted source. If you're displaying it, encode or escape it (that's another topic altogether).

Define what is OK

Your app doesn't have to accept all forms of input, and in fact it should not. Your responsibility as a developer is to strictly define exactly what sorts of input are acceptable for every single input your app receives. Take a whitelist approach, not a blacklist approach. That means you explicitly define what input is allowed, rather than defining what is not allowed. If a character doesn't make your whitelist, it doesn't make it into the app.

You'll want to define more than one whitelist. Keep one for each type of data your app receives (phone numbers, email addresses, etc.). Err on the side of a more narrow definition of what is OK. If your users complain about a character not being accepted, add that character rather than an entire class of characters. Design your app to limit the variety of input that is accepted: if you are storing US-only phone numbers, there's no reason for them to be letters, and you don't need to store more than 10 digits (save the dashes and parentheses for display).

If your users are choosing from a list your app provides, then there's no reason why you should accept anything that's not on that list. If you can display the list, then you can (and should) validate against it.

Check, check, and then check

Validation should happen in three places: on the client (before submitting input), on the server (as input is received), and at application boundaries (as input is received by one application from another).

As I mentioned above, client-side data validation is not enough. Your app needs to validate -- on the server side -- any data coming in from the client. But that's no reason not to validate on the client side. Client-side validation is your first defense, and can greatly enhance the user experience (by eliminating the round trip to the server just to validate input). There's another huge gain that client-side validation can give you: if you receive invalid input from your client, and catch it on the server using a validation algorithm that is supposed to match the client-side validation exactly, then you have detected an attack on your system. You can respond as you see fit. I recommend something tactical and nuclear.

Server-side validation should start, as stated above, with a validation algorithm that matches the client-side validation. Your business logic may then provide even further validation opportunities, based on the results of some server-side processing.

Any system your server communicates with should perform its own validation of input crossing application boundaries. This includes the app-server-to-DB-server boundary. Your SQL should be validating input, though this is generally limited to data-type checking, data-length checking, and referential integrity checks.

Use Your Tools

Validation is, of course, a common practice, and we have many validation tools at our disposal. I use SQL Server in most of my development, and just using stored procedures for all of my DB interaction gives me a head start -- invalid data types and lengths are automatically rejected. Using foreign key relationships between tables ensures referential integrity. As a .NET developer, I get a vast array of input validation tools built into the framework I'm building with, on the client and server sides. In any modern language you can use regular expressions to build your validation whitelists. Google regular expressions and you'll find some well written, proven regular expressions available for common data types. 

Proper input validation requires some work. It takes some time. But really all we need to do is establish some good habits, build up a set of tools we can use on every project, and make validation into just another thing we always do.

Ss.

Zen and the Art of CSS

This is an article about CSS and HTML, and the on-going debate between CSS positioning and table-based layouts in HTML. As ASP.Net developers, we get handed a graphic design for an application, and are expected to translate that into HTML output. That means placing all of the elements -- graphics as well as text and data -- on the pages of that application.

The History Lesson

When I started in the business, there was no expectation that designs would be translated into pixel-perfect HTML output. Web pages weren't expected to look like their print counterparts. Everything looked pretty dull.

It wasn't long before Netscape threw us all a bone with the TABLE tag. In short order web developers had worked out how to use tables to place anything anywhere on a page. Pixel-perfect layout was possible, and designers started creating the oh-so-beautiful web designs we have all come to expect from the web.

CSS came along soon afterwards, though it was more bark than bite at first. The first browsers that supported CSS didn't really provide the support needed to move away from tables. But that soon changed, and CSS has finally become a viable replacement for table-based layout.

Or so we've been told...

The Debate

Over the past few years I've worked on a handful of very different projects, and I've had to choose on each of them whether I'd use CSS or tables for layout. I've gone back and forth. I realize that CSS promises the separation of content from layout, which is a very nice ideal. But I've also worked with a lot of different designs, and different requirements for how page elements expand, fill the space around them, and align with each other. Each of these issues challenges the web developer in different ways. And when it comes down to it, tables always seem to work. They work really well.

So here's the question: Which is better: CSS or tables? Is it worth it to sacrifice the ability to do pixel-perfect layout in exchange for an idyllic ability to separate content from layout? Is that sacrifice even necessary, or can CSS provide pixel-perfect layout?

The Zen

As I said, I've gone back and forth. I've seen the beautiful code that's possible with CSS. I've seen the perfect layouts that tables provide. But now I've started to see the light: pixel-perfect layout is possible with CSS.

It's not easy. It's not obvious. At times it's not intuitive. And there are still some layouts that are difficult to achieve. Cross-browser compatibility is still an issue (and always will be, so I quit whining about that a long time ago).

But there are some excellent guides available. There are a whole lot of really bad ideas out there, too. In fact, there is more bad advice available than good. Here are some pointers to good advice:

First, you have to understand the HTML behind a good CSS layout. It's not as stripped down as a purist might like. Instead it provides a set of "hooks" into the content that the CSS can grab onto to provide proper layout. The best example I've seen is CSS Zen Garden, which is where my path to enlightenment started. Take a look at their HTML (view the source). The way their content is marked up creates a very flexible foundation for easy CSS positioning.

Second, you have to understand some of the fundamentals of CSS. The "position" declaration is perhaps the most important of all CSS declarations. A full understanding of the difference between relative and absolute positioning of an element makes all the difference in your use of CSS. Until I really understood this concept, CSS was a huge frustration for me. I learned all I needed to know from the explanation provided by the Dutch guy at Quirks Mode.

The next fundamental you need to understand: the "float" declaration. I used it way too much, and never really grasped the issues it introduces. Every time I thought it was the solution, it just caused more problems than I could handle. I had to spend a lot of time with the Floatutorial to realize how and when a float should be used.

The rest of the fundamentals include the box model, quirks mode and the doctype element, and multi-column layouts. I've included links to articles about each of those topics for your enjoyment.

Finally, you need a set of best practices for CSS. You need to establish a pattern that works for you. It should include methods for addressing the most prevelent issues with CSS-based layout (each of the links above help with one or more of these issues). It should answer the persistent questions like:

• How do I structure my HTML?
  
• How do I structure and organize my CSS?
• When do I use a class instead of an id attribute in my HTML element?
• How do I get my content to flow properly within the page?
• How do I get my containers to expand properly?

I'm still working out the kinks in the CSS practices I'm following. I'm still getting the hang of some of the quirks that come up when I'm using the float declaration. But I'm seeing real success, and no tables as far as the eye can see. It's exciting. It's enlightening. Yes, yes: very zen.

Ss.