Reliable Software Predictions
Waiting until code is written to find out whether software will be reliable is too late to do anything about it. Below are the core elements of predicting software reliability early enough to matter.
Why predictions have to start before there's code to measure
Reliability evaluation can only measure what already exists. Prediction is what gives the program an early, defensible estimate of software reliability, based on historical data, complexity, and process maturity, so problems can be addressed while there is still time and budget.
Before code exists
Initial predictions start at the requirements or architecture phase, not after coding is well underway.
Built on real data
Predictions draw on historical failure data, complexity metrics, and process maturity rather than optimism.
Refreshed as design matures
Predictions are revisited at each milestone so the estimate keeps pace with the actual design.
The core elements of reliable software predictions
Each element below is a step in producing a defensible, data-driven software reliability prediction across the program lifecycle.
Prediction Model Selection
The developer shall select and justify a defensible software reliability prediction method appropriate to the program's lifecycle stage.
- Method selection documented with rationale (e.g., IEEE 1633-based approach). Mission Ready Software invented several of the software reliability models in the IEEE 1633.
- Model appropriate to available data at each lifecycle stage. The Requs AI Predict Machine Learning model was invented for this purpose. Other models require you to answer a fixed set of questions which you often don't know in the early stages. Requs AI Predict provides a prediction based on limited information.
- Consistent method applied across builds for comparability
Input Data
Predictions shall be based on documented inputs including size/complexity estimates and development factors and level of rigor. Requs AI Predict uses the modern story points approach for size/complexity. It measures the development factors and level of rigor from the world's largest benchmarking study of 689 factors correlated to defect density.
- Measure the story points or people effort
- Answer some questions about how the software team is planning to develop, manage and test the software
- Use a model (Requs AI Predict) to predict the escaped defects, arrival rate, MTBSF, availability.
Early Lifecycle Prediction
An initial software reliability prediction shall be produced at the requirements or architecture phase, before detailed design is complete.
- First prediction available in time to inform architecture trade-offs. Requs AI Predict supports early predictions so that you can identify alternative scenarios. Requs AI Optima helps you identify trade-offs instantly between development practices/level of rigor, program increment cadence, story points, test effort and corrective action effort.
- Assumptions documented since data is necessarily limited this early. Requs AI Predict documents your assumptions so you can export them into the RSPP.
- Prediction compared against allocated targets from the outset.
Prediction Updates
The prediction shall be refreshed at each major milestone as the design, code, and defect history mature.
- Predictions updated at PDR, CDR, and prior to major test events. As the program evolves the estimates for story points/program increments and the development practices and level of rigor evolve. You can track the changes in Requs AI Predict.
- Growing use of actual project data as it becomes available. Requs AI Predict Enterprise supports tracking of predicted defects in testing versus actual.
- Revision history maintained to show how the estimate evolved. Requs AI Predict provides a save as function so you can keep track of previous characterizations.
Comparison to Allocation
Each prediction shall be compared against the allocated reliability target, and gaps shall be flagged for resolution.
- Predicted-versus-allocated comparison reported at each milestone. Requs AI Predict Enterprise covers this.
- Shortfalls escalated through the risk assessment task. Requs AI Optima supports rapid identification of the development risks that block the growth towards the reliability allocation.
- Comparison used to justify design or schedule changes when needed. Requs AI Optima supports "what" if scenarios.
What early, grounded predictions buy the program
Time to actually fix problems
Early predictions surface reliability shortfalls while there's still budget and schedule to address them.
An estimate, not a guess
Data-driven predictions replace optimistic assumptions with a defensible, historically grounded number.
Better trade-off discussions
Predicted-versus-allocated gaps give engineering and program management real evidence for design or schedule trade-offs.
A running track record
Refreshed predictions build a program history that improves the accuracy of future estimates.
Find out whether the software will be reliable before it's too late to fix.
Start with the online demo of Requs AI Predict or a discussion of your prediction approach.