How to Account for Hidden Costs in Software Development Plans

Though multiple project estimation methodologies exist to prevent overrun scenarios — discovery stage, requirements gathering, evaluation, etc., — arriving at a precise figure isn’t always easy as some managers fail to account for the hidden costs of software development.

Hidden costs represent expenses that are overlooked during the initial project estimation. They can include direct costs (labor, software/hardware purchases), as well as indirect costs such as administrative overheads, extra spending on cybersecurity, and so on.

Why Do Hidden Costs Emerge in Software Development Projects?

Software product development is a complex process. Not only are you building a new technologically complex solution, but you are also facing higher market risks (and rewards).

Project management fundamentals suggest that every project has to be documented as a list of business, technical, functional, and non-functional requirements.

Project requirements include precise, often-quantifiable, descriptions of various product elements and quality standards, which should be delivered. These include:

• Product features and functionality (often captured as user stories)
• Supporting IT infrastructure size and configuration
• Product presentation layer (front-end development and design)
• Integrations with other business systems and products
• Cybersecurity and compliance requirements
• Product maintenance post-deployment

While most of the requirements in the group above can (and should be!) properly estimated, there always remains a percentage of “unknowable” expenses, which were either overlooked or unforeseen during the initial software development planning.

As the adage goes: “We can predict everything, except the future”. Even the best software development plans cannot always factor in the risks (and costs) that could emerge due to:

• The impacts of new market conditions
• Changes in the regulatory environment
• Increased competitive pressure
• Legacy infrastructure and interoperability issues

More often than not, these impacts become evident during the later stages of the software development life cycle (SDLC), namely product architecture design, development, and deployment.

That said, while you cannot completely avoid risks and hidden costs, you can minimize their occurrence as well as their effects on your project timelines and budgets.

Types of Hidden Costs at Different Stages of the Software Development Life Cycle

Understanding when and why the risks of time and budget overruns occur is the key to establishing leaner product delivery practices.

This section highlights the main drivers of hidden costs at every stage of the SDLC with proactive recommendations for addressing those.

Project Planning Stage: Prolonged Decision-Making

Time is money — and in software development, the costs of drawn-out planning can accumulate fast. An IEEE study found that over 25 different organizational factors (including organizational alignment, executive support, and organizational stability among others), affect the on-time delivery of large-scale Agile projects.

Likewise, GitHub found that frequent context switching (e.g., due to changing requirements) and subsequent time required to provision new resources for another build (code deployment) also affect the development speed and costs.

Given that organizations now do around 1,460 code deploys per year on average, slow decision-making and largely manual workflows can substantially reduce time-to-market. This, in turn, increases the risks of a nimbler competitor launching similar functionality in the meantime and/or deploying the new release during worse market conditions.

Additionally, if you are working with a software development partner, establishing a strong project management office (PMO) is a must-do. Without competent product managers on your side, your project will likely suffer from information asymmetry — a lack of alignment between the in-house leadership and the delivery personnel.

Inexperienced PM staff can struggle with settling realistic project goals and requirement clarification/prioritization. This can subsequently result in labor-intensive code reworks, multiple alterations to the same features, and extra development costs.

Recommendation: Set Up an Effective SDLC

To ensure effective delivery, the executive vision has to be translated into effective SDLC plans.

Plan time-boxed Sprints, where the team will work on a particular set of prioritized product backlog items. To scale the effort, implement an interactive development process, where new product elements get shipped in increments, rather than using a “big bang” approach. 

Ensure that all team members follow the predefined steps and procedures of your product SDLC. The team should work towards a pre-established shared objective, rather than constantly attempting to change the position of the finishing line.

If your company does not have a mature PMO, consider appointing a proxy product owner on the software development partner’s side, who would lead the requirements elicitation and product planning activities.

Learn more about Agile software development in action.

Development and Deployment Stages: Scope Creep and Overstaffing

When it comes to the active development phase, the biggest cost drivers will be out-of-scope work.

Remember: Team velocity — the average amount of work a development team can complete within a delivery cycle — has an inverse relationship with the number of changes.

Whenever you come up with ideas that go beyond the pre-estimated and pre-planned scope of work, the development team has to take a U-turn from the already delivered work. Though Agile SDLC welcomes a certain degree of changes and flexibility, there are trade-offs. Context switching decreases the value outputs and increases the project costs. 

Fixed-price projects can help to better control the project scope and costs. However, such an engagement model is better suited for short-term projects with a well-formalized list of requirements. For example, developing an MVP product version as our team did for Bosch or creating a proof of concept of a computer vision solution for retail.

More complex projects such as progressive modernization of legacy applications, however, would require a more synergetic alignment and different approaches to project planning, estimation, and budgeting. 

Case Study

Over 60 OSS and BSS apps for BICS

See more!

The second major issue is project overstaffing — hiring too many technical specialists to accelerate product development. While having more “hands on deck” is often helpful, more people do not always mean better outcomes. Subpar operational workflows, ineffective management, and multiple layers of approvement can protract the timelines. At the same time, your company will then have to bear the costs of keeping all the hires post-delivery.

Recommendation: Incremental Planning and Delivery

Once again, initial clarity in product requirements will largely save you from the extra costs. During the planning stage, you should determine the optimal tech stack for the product, as well as the overall product SDLC, the timeline of releases, and other post-implementation activities.

Then establish the key roles and the optimal team composition for each project stage. Define your capacity needs for each stage and then work out the cooperation terms with the vendor.

When Infopulse was hired to develop an innovative logistics marketplace, we managed to meet a demanding timeline of under 6 months by establishing an Agile SDLC and implementing work parallelization among cross-functional teams.

Post-implementation stage: Infrastructure and Maintenance Costs

Though new product development is often seen as the largest cost factor, ongoing maintenance, in fact, often commands higher budgets. In fact, about 90% of software life cost is attributed to its maintenance phase.

To better understand and budget ahead for the total cost of ownership (TCO) of the delivered product, you should also factor in the following costs.

Hosting and Server Costs

If you plan to host data on-premises, server cost would be the most important part of the TCO, 68% and 55% on average, depending on the configuration. Similarly, you should also account for extra cloud storage and computing resources, both of which increase proportionally with your product user base.

Compare different cloud computing providers to find the most cost-effective option for your business. When estimating cloud costs, factor in product usage levels, future user base growth dynamics, compliance, and security requirements (if applicable to your industry).

Learn more about effective cloud cost optimization techniques.

Third-Party Software Components

Most modern IT portfolios are a panoply of “built” and “borrowed” software components. Many enterprises increasingly rely on a mix of propriety, enterprise open-source, and community open-source software solutions.

Acceleration of open source 

Source

Though open-source software (OSS) comes with no costly licensing fees and fewer development costs, it can be more expensive to support in the long run. Among the recently analyzed codebase, 91% contained outdated versions of open-source components (i.e., such where an available patch was not installed).

Moreover, niche third-party solutions may not be well supported and no longer meet the needs of your product. Open-source projects are abandoned frequently by the original creators and 88% of commercial applications now contain open-source components that had no new development in the past two years.

To avoid getting “locked in” within an abandoned OSS project, opt for the more popular, well-supported alternatives. Docker, Kubernetes, and Apache among others are examples of thriving OSS projects with a stable product, an active development community, and an ever-growing knowledge database.

Tech Support and Maintenance Work

Software and hardware costs make up the bulk of software TCO but so does ongoing support and maintenance work, done by software engineers. When the initial release did not check all the quality boxes, you will have to continuously pay off technical debt instead of investing in new product development. According to McKinsey estimates, companies allocated 10% to 20% of total IT budgets towards repaying tech debt.

Apart from resolving the “old” issues, you also need to account for the new ones. Similar to physical estates, software systems are prone to degradation without regular upkeep. For example, after five years in production, 70% of applications tend to contain at least one security flaw.

To reduce maintenance costs, invest in comprehensive quality assurance (QA) during the development stage. Implement continuous testing practices and increase the coverage of automated test cases. Likewise, it is often best to prioritize newer software components (e.g., frameworks, libraries, etc.) since these require less maintenance.

Post-deployment, set up an Agile vulnerability management process to ensure timely vulnerability detection and patching. Separately, establish an effective escalation process for L2/L3 issues to ensure the faster resolution of more critical issues.

Cybersecurity Issues

Reactive response to cybersecurity risks (aka vulnerability remediation post-incident) is more expensive than a proactive approach — progressive establishment of a modern security perimeter, aimed at threat prevention and early detection.

It is always better to invest in better cybersecurity protection for specific apps and your IT estate as a whole, rather than deal with the aftermath of a breach.

When it comes to application development, we always recommend security-by-design principles. That is establishing security requirements at the planning stage and then embedding the said controls and characteristics into the product and doing security testing.

Conclusion

Hidden costs in software development are inevitable. However, smart leaders know how to identify cost creep at the onset. Effective communication between all project stakeholders, strong project vision, detailed requirements as well as strong alignment with an experienced software development partner, can help you avoid the scenarios of spiraling budget overruns.