CodeMRI® ROI Tool User Guide

The CodeMRI® ROI tool is a scenario analysis that can help determine the most cost-efficient way to refactor a codebase given a scenario or set of scenarios. For the ROI tool, the different scenarios available are:

  1. Reduce code complexity by reducing the McCabe Complexity score.

  2. Reduce technical health complexity by reducing the size of any Cores.

  3. Both reduce code and technical health complexity.

Initial Set-up

In order to create the scenario analysis, two pieces of information are needed to calculate the Return on Investment information: the Preferences and the Scenarios.

Preferences

First, a set of Preferences should be filled in. The different Preference options are:

  • Cost per developer - The average annual cost of developers working in the scanned codebase: sum of developer cost / total developers.

  • Estimated total cost - The estimated total cost of a refactoring initiative.

  • Yearly code turnover - The feature lines of code modified or added in a year divided by total lines of code.

  • Downstream cost per bug LOC - The Average or expected monetary impact per defect (typical and bad) shipped in the product.

  • Interest rate for capitalization - The interest rate used in technical debt calculations. Using yearly interest payments and interest rate, Silverthread computes technical debt in the codebase.

  • Years to capitalize - The length of time (years) over which to calculate the return on investment of the proposed refactoring initiative.

The default Silverthread values are filled in, however, this may differ for each organization. These values should be updated to properly tune results to your organization.

In Excel, this is done on the Preferences tab.

Figure 1. The Preferences tab of the Excel Return on Investment (ROI) report.

 

On the Web version, it is the top half of the form submittal when first entering the ROI.

Figure 2. The form for submitting information to create a Return on Investment report through the Web.

 

Note that on the Web version, all Preference values must be a positive integer greater than 1. The Yearly code turnover and Interest rate for capitalization fields are considered percentages, and that will be taken into account on submittal. So, please type in 25 for a value of 25%.

Scenario Analysis

The next set of information to fill out are the possible scenarios to evaluate. For the Web, this is the second half of the form as seen in Figure 2 above.

In Excel, this is done at the top of the CodeMRI® Refactoring ROI tab.

Figure 3. The CodeMRI® Refactoring ROI tab from the Return on Investment report showing the Refactoring Initiative steps.

 

The Refactoring Initiative section provides the ability to do one or more scenarios, and each scenario is a row of four sections that need to be filled out: Area, Name, Target Code Quality, Target Design Quality. The target code and design quality values are what the goals are for the end state of the scenario analysis.

Please note that if more than one scenario is filled in, then the scenarios are evaluated together.

Excel allows for five scenarios, the Web does not have a limit. All five scenarios do not need to be filled out in Excel, the options are just available. On the Web, please press the plus button at the end of the scenario row in order to add another scenario, if desired.

Area

The area field is the definition of what part of the codebase should be included for this specific scenario. The available options are:

  • Overall System

    • This option will apply the target code and design quality to every file in the code base.

  • Not in Core

    • This option will apply the target code and design quality only to files that do not belong in a Core.

  • Component

    • This option will apply the target code and design quality to only the files in the component specified in the Name section.

  • Core

    • This option will apply the target code and design quality to only the files in the Core number specified in the Name section.

    • Note that this option is only for files inside a Critical or Emerging Core.

  • Directory (Excel only, coming soon to the Web)

    • This option will apply the target code and design quality to all files under the directory chosen.

    • Note that this option requires a directory path to be filled in, and it must match the file structure as listed in the File List tab.

Name

The name field further defines of the Area field.

Target Code Quality

The target code quality field defines what level of McCabe complexity is desired for that Area/Name combination. Silverthread defines the following for McCabe complexity:

  • Very High Complexity - McCabe greater than 50

  • High Complexity - McCabe between 21 and 50

  • Medium Complexity - McCabe between 11-20

  • Low Complexity - McCabe at 10 or less

Due to this, the target code quality field has the following available options:

  • n/a

    • This option chooses not to change the code quality for the scenario.

  • Low (McCabe 50+)

    • This option specifies that all of the files will have a McCabe Cyclomatic Complexity of at most 50 after refactoring is complete. This requires a minimal effort of code quality improvement, and will only yield decent improvement in overall defect density for codebases with extreme quality challenges.

  • Moderate (McCabe 20+)

    • This option specifies that all files will have a McCabe Cyclomatic Complexity of at most 20 after refactoring is complete. This requires a moderate effort of code quality improvement, and should yield significant improvements in overall defect density for codebases with severe code quality challenges.

  • High (McCabe 5+)

    • This option specifies that all files will have a McCabe Cyclomatic complexity of 5 or below when refactoring is complete. This is the highest level of effort for refactoring code quality, and should yield vast improvements in overall defect density for codebases with code quality challenges.

Target Design Quality

The target design quality field defines what level of Core reduction is desired for the Area/Name combination. The following options are available:

  • n/a

    • This option chooses not to change the Core sizes for the scenario.

  • Low (20% Core Removed)

    • This option specifies that the number of files in this Area/Name combination that are part of a Core should be reduced by 20%. This requires a minimal effort of design quality improvement, and will only yield decent improvement in overall defect density for codebases with extreme design quality challenges.

  • Moderate (60% Core Removed)

    • This option specifies that the number of files in this Area/Name combination that are part of a Core should be reduced by 60%. This requires a moderate effort of design quality improvement, and should yield significant improvements in overall defect density for codebases with severe design quality challenges.

  • High (100% Core Removed)

    • This option specifies that the number of files in this Area/Name combination that are part of a Core should be reduced by 100%, meaning all Emerging and Critical Cores will be completely removed. This is the highest level of effort for refactoring design quality, and should yield vast improvements in overall defect density for codebases with design quality challenges.

Note that the ROI tool does not say how to reduce the size of a core. For one possibility on doing so, please see the Technical Health Improvement Plan, which is also described in the CodeMRI® Care User’s Guide, https://silverthread.atlassian.net/wiki/spaces/CKB/pages/961773820/CodeMRI+Care+User+s+Guide#CodeMRI%C2%AECareUser%27sGuide-ImprovingTechnicalHealth .

 

Results

Once the preferences and scenarios are set, calculations are performed and results are displayed. In Excel, as seen in Figure 3, the results are at the bottom of the CodeMRI® Refactoring ROI tab. On the Web, the results are displayed on a new page.

Figure 4. The ROI results from the Web.

 

Both results show three sets of results:

  • Pessimistic, or Low

    • A conservative estimation of the results, showing more of the worst case scenario.

  • Expected

    • The average estimation of the results.

  • Optimistic, or High

    • The most optimistic estimation of the results, providing everything goes better than expected.

Estimated Outcomes

There are seven outcomes given in the results:

  • Total initiative cost

    • The estimated total cost of the refactoring initiative. It is based on the Estimated total cost value from the preferences.

  • Change in annual cost and risk

    • The change in amount of investment gained or lost per year as a result of the refactoring initiative.

  • Return on Investment (ROI)

    • The percentage of investment returned over the initial cost of the refactoring initiative.

  • Payback period in years

    • The amount of time in years it will take to re-coup the initial investment.

  • Developer years regained

    • The time in years regained by developers not fixing bugs.

  • Bug work reduction per year

    • The number of bugs reduced from the product after the refactoring initiative.

  • Downstream risk reduction

    • The reduction in the risk of bugs reaching the shipped product.