Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 45 Current »

Basic Setup and Administration

CodeMRI maintains separate directories for installation and data.

The installation directory is defined during setup and contains files related to the execution of CodeMRI.

The data directory is called a “vault”. This is the main workspace where CodeMRI stores data on the codebases being analyzed and configuration information related to those codebases.

Understanding CodeMRI Command Execution

CodeMRI executes through a series of statements issued in a Command Line Interface (CLI), such as DOS or Bash. Commands are in the following format:

cmri <command> --vault </path/to/vault>
-or-
cd <path/to/vault/myvault>
cmri <command>

Alternatively, CodeMRI commands can be issued via the CodeMRI Shell which can be activated as follows:

cmri shell --vault <path/to/vault>
-or-
cd <path/to/vault>
cmri shell

Once the shell is active, commands are in the following format:

<command>

The shell recognizes that all commands are CodeMRI commands, and recognizes the users vault location. The “cmri” keyword is no longer necessary, and the “-vault </path/to/vault>” parameter is no longer necessary. Commands execute quicker in the shell as the CLI overhead is reduced.

For more information on the CLI and the Shell, see CodeMRI CLI Reference

Understanding CodeMRI data structures (vaults, projects, systems)

CodeMRI uses multiple layers of virtual containers to store data (configuration, metadata, analysis) about codebases being scanned. These layers can best be visualized as boxes that contain other boxes.

The outer-most box is a vault.

  • A vault is the main CodeMRI workspace.

  • A user can create multiple vaults.

Inside of a vault are “project” boxes.

  • There can be 1-n projects in a vault.

Inside of a project box are “system” boxes.

  • There can be 1-n systems in a project.

  • A system points to a directory location that contains a codebase.

  • A system maintains the current snapshot of data.

  • Each system may represent a snapshot in time of scan data. To do this, you may set a “revision_date” property for each system in the project.

  • Note: revision_date is currently only used in the Web Platform Trending displays and in KPI output. All other output references the current snapshot that is maintained at the “system box” level.

Create a vault

mkdir myvault
cd myvault
cmri vault create

-or-

#Can run CodeMRI command from any directory
cmri vault create --vault <path/to/vault/myvault>

Use a vault - being in the vault directory

cd <path/to/vault/myvault>
cmri shell

Use a vault - with a command line flag

#Run CodeMRI shell
cmri --vault <path/to/myvault> shell

#Run CodeMRI command from commandline
cmri --vault <path/to/myvault> <other flags and commands>

Use a vault - by setting environment variables

Set the CMRI_VAULT environment variable to point to your vault.

We also recommend you automate this to increase the efficiency of your workflow

  • Linux: Add this to your BASH/ZSH/*.SH profile

# On Linux:
export CMRI_VAULT="/path/to/your/vault"

  • Windows: We recommend setting the user environment variables

# On Windows:
SET CMRI_VAULT=C:\Path\To\My\Vault
# Note that C:\ is merely used as an example, there is no requirement
# that the vault reside on your C drive.

Help Commands

See command listing and invocation help

This will show all of the commands available to you, as well as provide information about scripting.

--help

See a summary of all jobs available at the CLI.

Note that jobs are still subject to licensing.

job list

Call up more detailed help for a particular job.

job describe <job_command_name>

Licensing

Acquire and Install License (Online mode - Typical)

Silverthread will create an account for http://codemri.com. Your login will be your email address. Set your password.

If you are on a LINUX command line prompt:

cmri account login --email dan@silverthreadinc.com --password MYPASSWORD

# HOWEVER
# If you have special characters in your email or password, use the backslash
# character to escape them

# THIS IS WRONG:
cmri account login --email dan@silverthreadinc.com --password MYPASSWORD$%&^

THIS IS RIGHT:
cmri account login --email dan@silverthreadinc.com --password MYPASSWORD\$\%\&\^

If you are on a WINDOWS command line prompt:

cmri account login --email dan@silverthreadinc.com --password MYPASSWORD

# HOWEVER
# If you have special characters in your email or password, use the caret
# character to escape them

# THIS IS WRONG:
cmri account login --email dan@silverthreadinc.com --password MYPASSWORD$%&

THIS IS RIGHT:
cmri account login --email dan@silverthreadinc.com --password MYPASSWORD^$^%^&

If you are in the CMRI INTERACTIVE SHELL

account login --email dan@silverthreadinc.com --password MYPASSWORD

# HOWEVER
# If you have special characters in your password, use single quotes
# around them

# THIS IS WRONG:
cmri account login --email dan@silverthreadinc.com --password MYPASSWORD$%&^

THIS IS RIGHT:
cmri account login --email 'dan@silverthreadinc.com' --password 'MYPASSWORD$%&^'

Result:

NOTE: must be logged in to account in order to generate CodeMRI reports.

Acquire and Install License (Offline mode - Classified Systems)

machine-id generate

  • Forward the ID file that gets generated to Silverthread

  • receive a license file back

  • Store file in C:\Program Files\Silverthread

license add -f "C:\Program Files\Silverthread\license_file_name.lic"

Define Projects and Systems

Create a project

project add --name <project name>
-or-
project add -n <project name>

Delete a project

#Remove a single selected project
project remove 

#Remove multiple selected projects
project remove --force

#Remove a specific project
project remove --selection <project name>

Create a system

system add --selection <project name> --origin <system/file/path> --name <system name> --version <version id>

Delete a system

#Remove a single selected system
system remove 

#Remove multiple selected systems
system remove --force

#Remove a specific system
system remove --selection <project name/system name>

View vault content

vault list

Select Projects and Systems

Select projects and systems for operations and jobs.

See select --help for more information about selection.

select --add <project name>/<system name>
select --add <project name>/*

Deselect projects and systems for operations and jobs.

select --remove <project name>/<system name>
select --remove <project name>/*

View vault selection state

vault list

Produce Reports

NOTE: must be logged in to account in order to generate CodeMRI reports.

Generate diagnostic reports

Create CodeMRI® Health Diagnostic reports for selected systems

job run produce_reports

Generate portfolio reports

Create a CodeMRI® Portfolio report for selected systems

job run produce_portfolio

Generate Technical Health Improvement Plan (THIP) reports

Generate THIP report

Create a CodeMRI® Technical Health Improvement Plan (THIP) report for selected systems

job run produce_thip

NOTE: THIP only considers file cores above the 150-file threshold by default.

To set the threshold higher or lower, add the optional parameter:

job run produce_thip --size <min_num_of_files_in_core>

List all the THIP cores in a specific language

system thip core list [--language <language>] [--output-format <format>] [--out <output_file>]
job run produce_thip

List all the THIP steps for a specific core

system thip step list [--language <language>] [--core <core_number>] [--output-format <format>] [--out <output_file>]
job run produce_thip

  • THIP NOTE: This lists steps for the provided language/core combination along with the following base metrics about the provided steps

  • STEP: The step number used ito reference the step in operations

  • FROM FILE: The file originiating the relationships to cut

  • TO FILE: The file containing referenced entities in the relationships to cut

  • ENTITIES MODIFIED: The number of entity relationships to cut

  • CORE SIZE: The core size after the provided step and all previous steps are completed

  • PRIORITY: True if the step is prioritized, otherwise false

  • EXCLUDED: True if the step will be removed next time produce_thip is run, otherwise false

Prioritize THIP Steps

system thip step prioritize [--steps <steps>] [--language <language>] [--core <core_number]

Exclude THIP Steps

system thip step de-prioritize [--steps <steps>] [--language <language>] [--core <core_number] [--all]

Define File Attributes

NOTE: When setting the following customizable file attributes:

  • Select the system that owns the file whose attribute is being set

  • Using the “system file-property add” command, the “-e” parameter is a relative path (the file path is listed in relation to the system path)

    • For example:

      • if a system path = /home/my_user/my_project/my_system

      • and a file path = /home/my_user/my_project/my_system/src/my_file.java

      • then the relative file path (relative to the system path) = src/my_file.java

Flag a system directory as test code

select <project>/<system>
system file-property add -n fp_Test -v True -e "relative/path/to/dir"
select --remove *

Flag a system directory as third party

select <project>/<system>
system file-property add -n fp_ThirdParty -v True -e "relative/path/to/dir"
select --remove *

Flag a system directory as generated code

select <project>/<system>
system file-property add -n fp_Generated -v True -e "relative/path/to/dir"
select --remove *

Flag a system directory as excluded

select <project>/<system>
system file-exclusion add -e "relative/path/to/dir"
select --remove *

Define Codebase Architecture (Components and Blueprints)

IMPORTANT: Ensure that the relative paths used for the --expressions flag match the relative paths in the CodeMRI reports. If unsure, run job run produce_reports and open any one of the reports to find the File List containing relative paths.

Create a component

#Example
system component add --name MyComponentName --expressions 'relative/path/to/my/files/*'

Remove a component

#Example
system component remove--name MyComponentName --expressions 'relative/path/to/my/files/*'

NOTE: Expressions are relative paths (relative to the system path) that must include standard * wildcard expressions. See Defining & Controlling Codebase Architecture for more information defining and controlling the architecture.

Component Usage

The Create Component command can be repeated multiple times for the same component with different paths to build the component.

#Example
system component add --name MyComponentName --expressions 'relative/path/to/my/files/a/*'
system component add --name MyComponentName --expressions 'relative/path/to/my/files/b/*'
system component add --name MyComponentName --expressions 'relative/path/to/my/files/c/*'

#Alternate example
system component add --name MyComponentName --expressions 'relative/path/to/my/files/a/*', 'relative/path/to/my/files/b/*', 'relative/path/to/my/files/c/*'

The Create Component and Remove Component commands can be used together to extract components from within other components.

#Example
system component add --name MyComponentName --expressions 'relative/path/to/my/files/*'
system component remove--name MyComponentName --expressions 'relative/path/to/my/files/a/*'
system component add --name MyNewComponentName --expressions 'relative/path/to/my/files/a/*'

Declare component relationships

system component-relationship add --from <from_component> --to component1[,component2,...]

  • creates A-->B, A-->C, A-->... relationships

  • IMPORTANT: no space after commas

Verify component definition

system architecture verify

View a listing of component names, expressions, and file counts for each component.

system component list

List all of the declared component relationships in a tabular format

system component-relationship list

Regenerate the Silverthread Database containing the updated component information.

job run produce_silverthread_database

View information about files within a given component.

system file list --component <component>

Produce reports with component data

job run produce_reports

Produce diagrams with component data

job run produce_api_diagrams

NOTE: CodeMRI detects relationships between file, and hence detects relationships between components. API diagrams will show defined and undefined relationships between components. This is a useful feature to help discover the design in an undocumented (or partially documented) architecture.

Produce diagrams with selected component data

CodeMRI is capable of generating API diagrams comprised of selected components.

CodeMRI will accept a list of components in the command line:

job run produce_api_diagrams --component-subset compA compB compC --name MyCompSubset

CodeMRI will also accept a file that contains the selected components:

job run produce_api_diagrams --component-subset-file /path/to/file --name MyCompSubset

See API Diagrams - Component Subsets for more details!

Generate Key Performance Indicator (KPI) Data

KPI Ability

CodeMRI can generate a KPI data report based on KPI input data and a codebase(s) to derive KPI’s from.

KPI Input

CodeMRI accepts a file in YAML format that specifies (1-n) KPI definitions and/or metrics.

To see all available values for “MetricName”:

metrics list-kpi-metrics

KPI CLI Commands

Once a YAML file has been created with the desired KPIs and/or Metrics, KPI reports can be generated with the following steps:

#Select desired system
select --add <project name>/<system name>

#Alternatively, select all systems in a project
select --add <project name>/*

#If revision dates have not be previously applied, do so now
system config set revision_date "MM/DD/YYYY hh:mm:ss"

#Set KPI input file path
metrics config set -o /path/to/file.yaml

#Run KPI job
job run create_kpi_worksheet

KPI Output Report

As a result of running the create_kpi_worksheet job, CodeMRI will generate a KPI report in Excel (.xlsx) format. The KPI Report spreadsheet will be exported to the root directory of your vault. The report contains multiple tabbed sheets displaying complete overview data and detailed percentage/detailed quantity data presentations for both KPIs and Metrics (as some data is collected as a percentage and some is collected as a “count”, percentage and quantity data has been separated for the purposes of displaying data as a chart)

Detailed sheets contain pivot tables of grouped data, along with charts and slicers to better visualize and customize the presentation.

Export data for use on another computer

Generate Metadata

/* Create metadata */
job run produce_silverthread_database

# Create a human-readable representation of the system database, 
# useful for situations where data must be reviewed by an auditor prior 
# to release from a site.
job run create_readable_database

Generate Anonymized Metadata

Anonymize data to mask sensitive information (if desired). You must run produce_silverthread_database first. If you want to generate un-anonymized reports, run produce_reports prior to anonymization. Anonymization is reversible via job clean anonymize_database.

job run anonymize_database
  • No labels