The CodeMRI® ROI tool is a scenario analysis that can help determine the most cost-efficient way to refactor a codebase given a scenario or set of scenarios. For the ROI tool, the different scenarios available are:
Reduce code complexity by reducing the McCabe Complexity score.
Reduce technical health complexity by reducing the size of any Cores.
Both reduce code and technical health complexity.
In order to create the scenario analysis, two pieces of information are needed to calculate the Return on Investment information: the Preferences and the Scenarios.
First, a set of Preferences should be filled in. The different Preference options are:
Cost per developer - The average annual cost of developers working in the scanned codebase: sum of developer cost / total developers.
Estimated total cost - The estimated total cost of a refactoring initiative.
Yearly code turnover - The feature lines of code modified or added in a year divided by total lines of code.
Downstream cost per bug LOC - The Average or expected monetary impact per defect (typical and bad) shipped in the product.
Interest rate for capitalization - The interest rate used in technical debt calculations. Using yearly interest payments and interest rate, Silverthread computes technical debt in the codebase.
Years to capitalize - The length of time (years) over which to calculate the return on investment of the proposed refactoring initiative.
The default Silverthread values are filled in, however, this may differ for each organization. These values should be updated to properly tune results to your organization.
In Excel, this is done on the Preferences tab.
On the Web version, it is the top half of the form submittal when first entering the ROI.
Note that on the Web version, all Preference values must be a positive integer greater than 1. The Yearly code turnover and Interest rate for capitalization fields are considered percentages, and that will be taken into account on submittal. So, please type in 25 for a value of 25%.
The next set of information to fill out are the possible scenarios to evaluate. For the Web, this is the second half of the form as seen in Figure 2 above.
In Excel, this is done at the top of the CodeMRI® Refactoring ROI tab.
The Refactoring Initiative section provides the ability to do one or more scenarios, and each scenario is a row of four sections that need to be filled out: Area, Name, Target Code Quality, Target Design Quality. The target code and design quality values are what the goals are for the end state of the scenario analysis.
Please note that if more than one scenario is filled in, then the scenarios are evaluated together.
Excel allows for five scenarios, the Web does not have a limit. All five scenarios do not need to be filled out in Excel, the options are just available. On the Web, please press the plus button at the end of the scenario row in order to add another scenario, if desired.
The area field is the definition of what part of the codebase should be included for this specific scenario. The available options are:
Overall System
This option will apply the target code and design quality to every file in the code base.
Not in Core
This option will apply the target code and design quality only to files that do not belong in a Core.
Component
This option will apply the target code and design quality to only the files in the component specified in the Name section.
Core
This option will apply the target code and design quality to only the files in the Core number specified in the Name section.
Note that this option is only for files inside a Critical or Emerging Core.
Directory (Excel only, coming soon to the Web)
This option will apply the target code and design quality to all files under the directory chosen.
Note that this option requires a directory path to be filled in, and it must match the file structure as listed in the File List tab.
The name field further defines of the Area field.
For the Overall System and Not in Core areas, the only choice is ‘--’, as no further specification is needed.
For the Directory option, the directory path needs to be filled in, so no further specification is needed.
For the Component area, the choices will be the components that were defined. Please see the CodeMRI® Care User’s Guide, https://silverthread.atlassian.net/wiki/spaces/CKB/pages/961773820/CodeMRI+Care+User+s+Guide#CodeMRI%C2%AECareUser%27sGuide-DefiningtheArchitecture-Components , for more information on component definition.
For the Core area, the choices will be the Core ID of the critical or emerging cores found in the code base.
The target code quality field defines what level of McCabe complexity is desired for that Area/Name combination. Silverthread defines the following for McCabe complexity:
Very High Complexity - McCabe greater than 50
High Complexity - McCabe between 21 and 50
Medium Complexity - McCabe between 11-20
Low Complexity - McCabe at 10 or less
Due to this, the target code quality field has the following available options:
n/a
This option chooses not to change the code quality for the scenario.
Low (McCabe 50+)
This option specifies that all of the files will have a McCabe Cyclomatic Complexity of at most 50 after refactoring is complete. This requires a minimal effort of code quality improvement, and will only yield decent improvement in overall defect density for codebases with extreme quality challenges.
Moderate (McCabe 20+)
This option specifies that all files will have a McCabe Cyclomatic Complexity of at most 20 after refactoring is complete. This requires a moderate effort of code quality improvement, and should yield significant improvements in overall defect density for codebases with severe code quality challenges.
High (McCabe 5+)
This option specifies that all files will have a McCabe Cyclomatic complexity of 5 or below when refactoring is complete. This is the highest level of effort for refactoring code quality, and should yield vast improvements in overall defect density for codebases with code quality challenges.
The target design quality field defines what level of Core reduction is desired for the Area/Name combination. The following options are available:
n/a
This option chooses not to change the Core sizes for the scenario.
Low (20% Core Removed)
This option specifies that the number of files in this Area/Name combination that are part of a Core should be reduced by 20%. This requires a minimal effort of design quality improvement, and will only yield decent improvement in overall defect density for codebases with extreme design quality challenges.
Moderate (60% Core Removed)
This option specifies that the number of files in this Area/Name combination that are part of a Core should be reduced by 60%. This requires a moderate effort of design quality improvement, and should yield significant improvements in overall defect density for codebases with severe design quality challenges.
High (100% Core Removed)
This option specifies that the number of files in this Area/Name combination that are part of a Core should be reduced by 100%, meaning all Emerging and Critical Cores will be completely removed. This is the highest level of effort for refactoring design quality, and should yield vast improvements in overall defect density for codebases with design quality challenges.
Note that the ROI tool does not say how to reduce the size of a core. For one possibility on doing so, please see the Technical Health Improvement Plan, which is also described in the CodeMRI® Care User’s Guide, https://silverthread.atlassian.net/wiki/spaces/CKB/pages/961773820/CodeMRI+Care+User+s+Guide#CodeMRI%C2%AECareUser%27sGuide-ImprovingTechnicalHealth .
Once the preferences and scenarios are set, calculations are performed and results are displayed. In Excel, as seen in Figure 3, the results are at the bottom of the CodeMRI® Refactoring ROI tab. On the Web, the results are displayed on a new page.
Both results show three sets of results:
Pessimistic, or Low
A conservative estimation of the results, showing more of the worst case scenario.
Expected
The average estimation of the results.
Optimistic, or High
The most optimistic estimation of the results, providing everything goes better than expected.
There are seven outcomes given in the results:
Total initiative cost
The estimated total cost of the refactoring initiative. It is based on the Estimated total cost value from the preferences.
Change in annual cost and risk
The change in amount of investment gained or lost per year as a result of the refactoring initiative.
Return on Investment (ROI)
The percentage of investment returned over the initial cost of the refactoring initiative.
Payback period in years
The amount of time in years it will take to re-coup the initial investment.
Developer years regained
The time in years regained by developers not fixing bugs.
Bug work reduction per year
The number of bugs reduced from the product after the refactoring initiative.
Downstream risk reduction
The reduction in the risk of bugs reaching the shipped product.