By their very nature, estimations are problematic. Never more so than inside the world of software development, where estimations are usually described as predictions, in terms of time or money, of the effort required to complete a set of related tasks.
And while the number of formal methods and tools for reaching a more realistic estimation continue (think COCOMO, function points, etc…), so has software complexity. In other words, accurate estimation is only becoming more and more difficult as software advances.
At the time of writing this article, the mean effort overrun is around 30%, something that has not budged since the 80s. What then happens if we reduce the scope to include only medium to large scale projects? The answer: 9 out of 10 would be considered failures. Not because they never reach production, but because their budget overruns.
The rise of agilism as a new philosophy to be adopted by software development teams means a path to more accurate predictions has finally been discovered. The introduction of new agile methodologies and tools has only served to seed this new philosophy. But, agilism has also simultaneously shifted the established paradigm, the very one that relates estimations with fixed values of time or money.
On the inside of teams that work within an agile framework like Scrum or Kanban, estimations play a crucial role and help identify the effort required to reach objectives. But their use requires an understanding that we’re dealing with flexible values that can change depending on the circumstances of the project.
In Scrum, the team must reach an agreement before estimating the effort required to complete a user story. This is done with the intention of adding it to the set of stories to be completed during a given sprint (a work cycle). In other frameworks, like Kanban, which does not work on sprints, the amount of estimated effort indicates the team’s throughput while it is active, much like horsepower on motor engines.
This way of understanding estimation has allowed for better results while delivering value to consumers. Or, in the case of internal products, the organizations themselves, since these frameworks are iterative.
The preferred way to work is with open scopes and short development cycles. This in turn leaves room for any adjustments needed in function or effort so that there is far less impact on the project as a whole. In the end, this means that estimations can be given in ranges, and not in fixed values.
In companies like Tekton, that offer software development services, estimations appear initially during negotiations with a potential client. These circumstances are usually cloaked in uncertainty, since there likely are many details yet to be discovered or researched. This is why an expert estimation, in which a judgmental process of the requirements is conducted by a professional with certified experience, is usually the best approach.
In cases where the client has completed exhaustive work on gathering requirements or modeling processes, more formal approaches like function point analysis or use case analysis can be used, but these situations are less frequent.
It is worth noting that additional formal methods exist for these types of estimations, that may include mathematical modeling or statistical regression analysis, but the use is limited to having enough data to feed the model. They, in many cases, do not offer better predictability than expert estimation.
The need to estimate is also present inside our agile development teams, where the practice of choice is planning poker. In these cases, it is important to reiterate that the team needs to give an estimation based on its experience, without external influence, to get useful results that are closer to reality. And, since we are dealing here with rough estimates and not fixed values, they can change to allow for the team to face any changes to the project.
Estimating effort needed for a software development project is difficult, but there are methods and practices that can help smooth the process and maintain predictability. Although there are some tendencies that want to reduce the importance or usefulness of estimations, to the point of removing them completely, there are ways to help teams adapt to change, and reduce possible impact on deliverables or costs that may pop up at the end of the project.