Schedule a Software Walkthrough Free Trial
Home / Sensitivity Analysis / Improvement
Step 8 — Sensitivity Analysis → Improvement

Improvement analysis

One of the most popular analyses. Use your survey assessment to determine the quantitative impact of changing specific development practices, toward one of three goals.

Goals

Three reasons to run this analysis

#
Goal
1
Determine which changes to development plans or practices could produce a specific reduction in defects (e.g. 10%).
2
Determine the minimum practices needed to maintain the current defect density prediction.
3
Determine the overall gaps and strengths in the project.
Goal 1

Changes needed for a specific defect reduction

#
Step
1
Complete the full-scale surveys first.
2
Go to the improvement worksheet and identify all negative responses.
3
Analyze which negative responses could realistically be implemented on this project — prerequisites, people, budget, culture.
4
Rank the results by how likely each practice is to actually get implemented.
5
One at a time, change the “What if” response to affirmative and review the what-if statistics against the original prediction.
6
Stop once the objective is met.

Worked example: changing three negative responses to affirmative yields a 33.64% improvement in fielded defect density.

Statistic
What-if scenario
Original prediction
Predicted fielded defect density
12.24
18.45
Predicted percentile group
97
97
Confidence bounds (field)
.0132
.0132
Predicted testing defect density
.211
.211
Confidence bounds (testing)
.234
.234
Improvement in defect density (field)
33.64%
0.00%
Ratio of predicted to what-if fielded defect density
1.51
1.00
Goal 2

Minimum practices to maintain the current prediction

#
Step
1
Complete the full-scale surveys first.
2
Go to the improvement worksheet and identify all positive responses.
3
Identify the practices that are most time-consuming or expensive.
4
Rank the results by expense.
5
One at a time, change the “What if” response to negative and review the statistics. If the prediction doesn’t change, move to the next item; otherwise, revert to affirmative.
6
Stop once you can no longer eliminate a practice without increasing defect density.
Goal 3

Gaps & strengths in a well-rounded project

The projects in the Softrel database with the lowest defect density were also the most “well rounded.” Every survey question maps to one of the categories below, each weighted by a coefficient — e.g. each “Avoiding Big Blobs” question is worth 13 points, each “Domain expertise” question is worth 6.

Worked example: the prediction sits in the P75 group (below-average defect density). Checking every practice affirmative in the “what if” column shows how many total practices fall in each category. In the table below, Execution, Coding, Change Management, and Defect Reduction have the lowest percentage of affirmative responses — and of those, Defect Reduction and Execution carry the biggest coefficients, making them the best candidates to investigate for feasibility on this project.

Category
Coefficient
Prediction — positive responses
What-if — positive responses
Avoiding Big Blobs
13
5
13
Change Management
7
1
5
Coding
15
1
5
Defect Reduction
60
0
1
Defect Tracking
50
1
4
Design
1
1
2
Domain expertise
6
1
3
Execution
20
1
5
Inherent risks
90
1
2
Personnel
5
2
7
Planning
1
1
4
Process
1
2
5
Requirements
1
3
11
System testing
2
4
18
Unit testing
1
2
8
Visualization
20
1
2
Total
300
978

Explore the other sensitivity analyses.

Staffing, test staffing, and defect pileup analyses are also available.