Early Cost Estimates for Unfunded Software Projects

Introduction

One of the challenges facing software teams when considering new solutions: “How much will this new project cost?”

There have been a number of costing methodologies that have evolved over the years. I remember my first interaction with an agile coach. I asked him, “How can you determine how long an agile project will take?" The response, “It is done, when it is done.” Actually, this answer would be OK in a situation where other departments in the organization were devoted to agile methodology. Having said that, this response usually doesn’t play well with investors, and CEO – CFO principals.

When in the early project evaluation stage – we won’t have all the requirements or time to create a detailed estimate. Many projects are competing for executive attention, and funding. It cost money and time to do a deep dive to build out of a full proposal for a new product.

This is key: An early costing exercise can help gauge the stakeholder’s appetite for investment. This exercise also considers risks - such that we can get a feel for the stakeholder's tolerance for risk.

A stakeholder may have a high tolerance for risk on one project, while having a lower tolerance for risk on a different project.

Bigger Picture

Over time I have built a mental model of the challenges involved in getting funding approved for software projects:

• Understanding the true value proposition to customers (what are we building, …)
• How are we going to monetize the opportunity? (how are we going to make money, …)
• Market dynamics (competitors, target market is niche, vertical or horizontal, …)
• Risk tolerance for investor on this project
• Technical effort and risks

And then there are the Organization risks and opportunities:

• History (legacy product support, …)
• Adaptability
• Accelerants and Champions
• Team capabilities

All of the above are important, and each warrants a discussion of their own. It is also interesting to consider how the above factors significantly impact each other. For example, if our investor has a very low tolerance for risk for the project, then we must have a higher level of understanding of what we are building, how we are going to make money on the solution, and our organizational readiness.

One Piece of the Puzzle

I believe it is important that we, as technologists, have the ability and tools to provide a response to the business when we are asked, “Is this the size of a breadbox, SUV or semi?”

In this discussion we will cover a methodology that I have evolved (along with numerous-much appreciated-colleagues) over the years. This exercise by no means provides a completed plan with all the details required for a project proposal. It does provide a costing estimate with high profile risks identified. It is somewhere between a "back of the napkin estimate" and a full project proposal.

I've used this approach in pre-seed startups, startups and large enterprises.

I usually refer to this exercise as a modified tee-shirt sizing exercise. It draws on experience from multiple engineering practices, including agile.

Like any methodology, we can start with a list of “what” we think we need to do. Even if we don’t know all the answers, we can create a “thought exercise” list of the things we will likely have to build. I believe the process of building a list, with an engaged team, leads to a set of follow on thoughts, ideas and questions that are invaluable for downstream activities.

The Model

There are many factors that can impact the total effort for a particular task. Here are the ones that I’ve found the most impactful for an early estimation exercise:

• Level of Effort as a range
• Risk (Have we done this before?)
• Complexity (Number of moving parts)
• Group Risk (New 3rd parties? Hiring ramps, Contractors, …)

Note these factors can be cumulative. For example, when something is complex, and you haven’t done it before, it increases overall risk. You should have a reasonable understanding of the project risk profile when you have completed the early estimation process.

It should be noted:

• Originally, the calculations-tables in this spreadsheet were based on thought exercises. They have evolved over several years based on high level analysis of project results as compared to original spreadsheet estimates.
• The actual project task list can change by as much as one third (outlier) during project implementation. Often the removed tasks were replaced by comparable new tasks. At times, I found this turned out to be restating the originally estimated work in a different manner.

Below is a snapshot of the spreadsheet (columns will be explained in the following sections). This example is included in the downloadable spreadsheet to help you get started. Each of the selection cells is driven by a "Validation" worksheet. That means as you select each cell, you will be presented with a dropdown. Note that you can tune percentages to fit your environment.

Base Estimation Equation

Initially, the base estimation equation started by using a sliding scale based on the magnitude of the estimate and the effort range as a ratio to the maximum number of days. It had a number of weighting factors that came into play.

As we reworked the equation through real-life examples, we actually found the final equation (presented below) much simpler and it provided the best results!

The base estimation equation is illustrated below:

Group Risk

The general form of the equation "settled" for a few years, though on occasion we would notice a large miss on some of our projects. When we analyzed these misses, we realized that many of the impactful factors where outside of the teams direct control.

Some examples of these outside factors include new 3rd party relationships which often required legal help, or a new release process for the company.

These factors often caused rework, and other project inefficiencies.

We introduced a "Group Risk" item that would take an individual task line item and apply a separate multiplier. As an aside, this multiplier can be as low as 1x (no impact), and as high as 3x!

Adjustment

I have found that on average, the spreadsheet is approximately 20% below the projected effort once a more detailed plan is put together. There are definitely high-side outliers from this average, though the more often you do this exercise, the better the results seem to be.

I have never seen a project come in under the early estimate.

Therefore, as a baseline, I always add at least 20% to the total estimated time. This number should be adjusted further if the overall program has a high level of risk.

Recently, in a pre-seed startup scenario, there was considerable risk to the project. In this case, I provided an investment range based on a low and high adjustment percentage.

Note that "Pre Group Days" in the following diagram comes from the previous diagram (the result of the base estimation equation).

I usually hide the Pre Group Days column just to minimize the data noise.

A set of questions I ask the team when the total project estimate comes out higher than expected:

• What items are introducing the significant risks into the estimates?
• What are the “BIG” questions or unknowns that we could quickly research?

There is a comment column used to record assumptions, as well as outstanding questions. This can be used as input to the next stage of your planning process if the project is funded.

Costing

Note this spreadsheet does not try to differentiate engineering specialties (architecture, development, testing, quality assurance, dev-ops, …). To suggest that the level of detail in this spreadsheet is precise enough to account for engineering specialties, would exaggerate its accuracy (as shipped). The goal is to answer the breadbox, SUV or semi question. At times, the spreadsheet has been enhanced to represent specific engineering roles, though it does take more time to come up with the estimate.

The estimation spreadsheet focuses on effort, not cost. Based on the project, we usually came up with a blended rate that was a fair representation of the team profile that would likely be involved in the project.

Product Management

Building this spreadsheet with Product Management participation is an incredibly rewarding exercise. It helps the team focus on many important product questions very early in the process, and it helps improve accuracy of the early estimates.

Product Managers will rightfully ask, “Why does this cost so much?” It is an early opportunity for engineering to respond with the various risks the teams may encounter. In my experience, Product Managers can be extremely creative when looking for ways to reduce risk—though the teams should respect the various specialties in the room.

I have found the following scenarios when engaging with Product Management while building this spreadsheet:

• Product Managers can champion with other stake holders to gain support for the project during this exercise
• The team comes away with a better understanding of scope based on a healthy cost versus benefit discussion
• Priority – which order do we think we need things for MVP
• Some items which are costly may be dropped from the project all together – or fulfilled via a partnership or third party.
• Most importantly: The process offers a chance to build trust and respect between Product Management and Engineering

For Product Managers – I know at first this may seem like a “dry” exercise. If at all possible, I would suggest you participate! Depending on how close you are to the technology of your product – this exercise can provide invaluable insights. I’ve also seen these discussions lead to new capability ideas (“The real problem the customer is trying to solve is this – and you're saying this widget is easy to build?”). I’ve also seen new product lines evolve from these discussions. And, as stated above, it promotes trust within the teams.

Summary

These are the benefits from the early estimation exercise:

• You can provide an educated guess to cost, though you should set realistic expectations on the risks associated with the early estimate
• You can have a reasonable coverage of high-risk items identified depending on the level of detail you’ve put into the early estimate
• The early estimate is a reasonable input when you start building a detailed funding proposal and/or plan, if you get a green light to proceed.

If Product Management was involved, everyone walks away with a better understanding of the project. The team has also started building a skill set that can be leveraged the next time someone proposes a new product idea.

In closing, there are a number of variables in building these estimates, many of which are specific to your ecosystem. If you decide to use this spreadsheet, please look at it as a starting point to be tuned to your particular environment!

Questions and Answers

In this section, we will discuss follow on questions or comments that I have received, typically after publishing an article.

If you're curious, below is a fun example of the spreadsheet that a colleague of mine created. Fun Spreadsheet Example Below you will see an example of an estimation spreadsheet to build a chicken coop!

How does this spreadsheet and process relate to a methodology like COCOMO? COCOMO Response COCOMO is an estimation model that has evolved over time with the input of a number of companies. It considers a many factors in creating an estimate. One example of these factors: "Personnel attributes" (capabilities of team members). Carrying the above example a bit further, the model presented in this article presumes you are in the early feasibility stages of a project. You may not know which team members will participate, and you're usually very short on time. As an aside, time-boxing a feasibility estimate can be a good thing. It helps to ensure you spend as little resource as possible on projects that turn out not to be feasible. COCOMO