Estimating software development schedules is a skill, not an art. Here are some tips to get better estimates.
Any system where management writes a schedule and hands it off to programmers is doomed to fail. Only the programmer who is going to do the work can figure out what steps they will need to take to implement that feature. And only the programmer can estimate how long each one will take.
Attempting to estimate a feature before you've figured out how the feature is going to work, in detail, can't possibly work, even if you "multiply your best guess by three," because your "best guess" is based on very little in the way of facts. The nature of software development is that things which seem simple are often surprisingly complicated when you think about all the details. For example, when you think about creating a registration and logon system, you might forget that you're going to need a way to prevent passwords that are the same as the user name, and a way to deal with forgotten passwords, and a way for people to unregister, and so on. Until you try to figure out the design of all these things, you have no basis for an estimate at all.
This is the most important part to making your schedule work. Your tasks should be measured in hours, not days. (When you see a schedule measured in days, or even weeks, it's probably not real). You might think that a schedule with fine grained tasks is merely more precise. Wrong! Very wrong! When you start with a schedule with rough tasks and then break it down into smaller tasks, you will find that you get a different result, not just a more precise one. It is a completely different number. Why does this happen?
When you have to pick fine grained tasks, you are forcing yourself to actually figure out what steps you are going to have to take. Write subroutine foo. Create dialog such and such. Read the wawa file. These steps are easy to estimate, because you've written subroutines, created dialogs, and read wawa files before.
If you are sloppy, and pick big "chunky" tasks ("implement grammar correction"), then you haven't really thought about what you are going to do. And when you haven't thought about what you're going to do, you just can't know how long it will take.
As a rule of thumb, each task should be from 1 to 16 hours. If you have a 40 hour (one week) task on your schedule, you're not breaking it down enough.
Here's another reason to pick fine grained tasks: it forces you to design the feature. If you have a hand-wavy feature called "Internet Integration" and you schedule three weeks for it, you are doomed, buddy. If you have to figure out what subroutines you're going to write, you are forced to pin down the feature. By being forced to plan ahead at this level, you eliminate a lot of the instability in a software project.
FogBugz's Subcases feature is excellent for breaking down large tasks into small ones.
Most programmers have no idea how to guess how long things will take. That's okay. As long as they are continuously learning and continuously updating their estimates as they learn, the schedule will work. (You may have to cut features or slip, but the schedule will still be working correctly, in the sense that it will constantly be telling you when you have to cut features or slip). Most programmers become very good estimators with about one year of experience.
If your schedule is going to take about a year, each programmer will probably take quite a few days of vacation. In FogBugz, you can set up company-wide holidays, and each developer can enter their own planned vacations as well.
You need to enter schedule items for:
In FogBugz, a schedule item is a special kind of case. You should enter schedule items for each developer. If you have five developers on your team, you should have five "debugging" schedule items, one assigned to each developer, so that the calculated dates produced by EBS are correct.
Many rookie software managers think that they can "motivate" their programmers to work faster by giving them nice, "tight" (unrealistically short) schedules. This kind of motivation is brain-dead. Most developers, when behind schedule, will feel doomed and depressed and unmotivated. When they're working ahead of schedule, they're cheerful and productive. As such, the schedule is not the place to play games of chicken.
Why do inept managers try to get programmers to reduce estimates?
When the project begins, the technical managers go off, meet with the business people, and come up with a list of features which they think would take about three months, but would really take nine. When you think of writing code without thinking about all the steps you have to take, it always seems like it will take n time, when in reality it will probably take more like 3n time. When you do a real schedule, you add up all the tasks and realize that the project is going to take much longer than originally thought. Reality sinks in.
Inept managers try to address this by figuring out how to get people to work faster. This never works. You might be able to get 10% more raw code out of people temporarily at the cost of having them burn out 100% in a year. Not a big gain, and it's a bit like eating your seed corn.
You might be able to get 20% more raw code out of people by begging everybody to work super hard, no matter how tired they get. This causes debugging time to double. An idiotic move that backfires in a splendidly karmic way.
The business is best served if estimates are as realistic as possible. While estimating, you must completely ignore the wishful thinking of the client and focus on what is really going to happen.
The idea that software estimation is an "unsolved problem" is simply not true. True, a lot of software developers organizations have managed to remain ignorant of everything that has been learned over the last 40 years about how to estimate software and deliver it on time, but there are also plenty of development teams that reliably produce accurate software estimates and ship within 5% or 10% of estimate.
You can learn more about estimating software schedules from Steve McConnell's book Software Estimation: Demystifying the Black Art, published by Microsoft Press.
If you've never read it, The Mythical Man-Month by Frederick P. Brooks (Addison-Wesley) is a requirement.