Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 8 Current »

Exploring the User-Defined Architecture of a System

The purpose of this example is to show you how to supplement a CodeMRI system with information about the architecture of the system and see what is gained by having this information.

Example system: Axis2-1.7.9

For Reference: Read about ‘user defined architecture' (UDA)

Defining & Controlling Codebase Architecture

system architecture --help

Example system: Axis-1.7.9

select Axis2/Axis2-1.7.9

Use a UDA file to specify the component structure

This downloadable UDA file has been build to define the architecture of the Axis2 v1.7.9 codebase. These components and dependency relationships have been extracted from the Maven files in that codebase:

Component descriptions defined in UDA files let you express your ideas about the structure of a system. By creating a UDA, you can:

  • Define a named set of components (or modules or subsystems depending on what you call things).

  • Assign source code files in your system to each component, which ‘contains’ or ‘owns’ them.

  • Define some files inside a component to be ‘APIs’ or ‘public interfaces' for the component. Code outside the component should only directly invoke or connect to files in this API

  • Define dependencies between components. Code in other components should only depend on code in this component if a dependency (direct or indirect) is declared.

  • Note that dependencies should not be circular. If component A is defined to depend on component B, then B should not also depend on A directly or indirectly.

Download this UDA file:

Copy this file into your CodeMRI Data Vault in the default location here:

/home/dan/Documents/test_datavault/projects/Axis2/systems/Axis2-1.7.9/uda.json

This uda.json file can have any name and be stored anywhere, but if stored in the vault’s ‘system’ directory, no additional configuration is required.

However, if you do wish to use a name other than ‘uda.json’ or put the UDA file in a directory other than the default, run:

# This is not neccesary if you use the default
> system config set uda_file <YOUR DESIRED FILE LOCATION>.uda

Run additional jobs to tell CodeMRI to find the UDA and analyze it

> job run produce_silverthread_database

Now that UDA information has been added to the DataVault, run (or rerun) other jobs:

# reproduce CodeMRI reports with component columns added
> job run produce_reports 

# create new pictures of the component structure
> job run produce_api_diagrams

This will produce new copies of reports you already have and also produce new kinds of reports that are possible with source code and UDA information combined. Note that new copies of existing reports have not overwritten the old ones:

Architecture information in Excel reports

…

Architecture diagrams

Inside the ApiDiagrams directory, you will find a block diagram of your system.

What you see

What it means

  • Dependency declared in UDA

  • Dependency exists in source code

  • Dependency declared in UDA

  • Dependency does NOT exist in source code

  • Dependency exists in source code

  • Dependency NOT declared in UDA

  • Dependency exists in source code

  • Dependency NOT declared in UDA

  • Introduction of dependency in UDA would introduce circular dependencies

  • Dependency exists in source code

  • Code depends on files inside the private internals of a component, not files in the component’s public interface

The image is laid in the following way:

  • Each box represents a component

  • The size of each box is proportional to the volume of code inside files it owns

  • Arrows represent dependencies

    • Black solid arrows show places where the UDA defines a dependency and there are dependencies found in the source code

    • Black dotted arrows show places where the UDA defines a dependency but there are no dependencies found in the source code

    • Red arrows show places show places where the UDA does not define a dependency but there are dependencies found in the source code. These are illegal and should be fixed

Images are useful, but they only get you so far. Now we will use the command line to explore the system

Writing a Script to Generate the UDA - A Maven example for Axis2

While it is a more advanced maneuver, it is possible to write scripts that analyze your build configuration to generate CodeMRI UDA files. This can be used as a vaulable starting point.

For example, the scripts below (for Linux) examine the Maven POM files in Axis2 v1.7.9 and generate the UDA file downloadable on this page.

Here is the content of the README file below

The Maven POM adapter provides a mechanim to convert a multi-POM Maven project using a parent POM and several child POM files into a Silverthread-specific UDA file. CodeMRI® can use these UDA files to understand the component layout of a codebase.

Requirements:

  • A Linux machine with a working CodeMRI® Platform installation.

  • A multi-POM Maven project using a root parent POM and several child POM files.

    • Child POMs must have the same groupId as the parent POM.

Usage instructions:

Run: ./pom-adapter /path/to/your/source/code

The script will automatically search the source root for the POM file. If a POM file is found, the script will process the POM file and write a uda.json file containing the resulting user-defined architecture to the current directory. After running the script, create and configure the system as follows:

$ cmri system add -n '<system name>' -v '<system version>' -s '<project>' -o '/path/to/your/source/code'
$ cmri system config set -s '<project>/<system name>-<system version>' uda_file uda.json

You may then scan the codebase, and produce architectural diagrams (a.k.a API Diagrams):

$ cmri job run -s '<project>/<system name>-<system version>' produce_api_diagrams

Inspecting Architecture on the Command Line

# Make component info available to the query engine
> job run generate_query_data

Get a listing of components in Axis2 v1.7.9

> system component list
> system component-relationship list

Explore a System and Build Insight (via Query Interface)

Imagine that you are a developer responsible for the axis2-kernel and axis2-integration components. You want to use this system to ask questions and learn more about your system. The following commands utilize a combination of cmri shell commands and possibly also UNIX shell commands as well.

Question

Commands and Output

What information do we have about files?

system file list
system query files --name modules/adb/src/org/apache/axis2/databinding/types/xsd/Date.java

What entities are in that file?

system query entities --file modules/adb/src/org/apache/axis2/databinding/types/xsd/Date.java

What if I only want to get the list of classes?

What files are owned by my component?

system file list
system query files --name modules/adb/src/org/apache/axis2/databinding/types/xsd/Date.java

Can I see the output as a table instead?

system query files --component axis2-integration --output-format csv
system query files --component axis2-integration --output-format tsv

Can I get this without the column header for scripting purposes?

system query files --component axis2-integration --output-format tsv:noheader

Can I get this as JSON?

system query files --component axis2-integration --output-format json

Can I save it off as a file?

system query files --component axis2-integration --out /tmp/compfile.tsv --output-format tsv:noheader

Can I call this from the UNIX command line?

[dan@fedora ~]$ cmri system query files --component axis2-integration --output-format tsv:noheader --selection Axis2/Axis2-1.7.9

That looks great. Can I use it in a script?

Yes.

You might want to do 2 things:

  • Get rid of colors in the output using --no-color

  • Redirect STDERR to /dev/null to get rid of the ‘thank you' message at the bottom. Because they are STDERR, not STDOUT, they should not interfere. They might be annoying however.

Let’s say you want to only see column 1, the file names in the component. You could do this from the command line

cmri system query files --component axis2-integration --output-format tsv:noheader --selection Axis2/Axis2-1.7.9 --no-color 2>/dev/null | sed -s 's/\t.*//'

Let’s say you want to see the files in axis2-integration sorted by their LOC

cmri system query files --component axis2-integration --output-format tsv:noheader --selection Axis2/Axis2-1.7.9 --no-color 2>/dev/null | awk -F '\t' '{print $11, $1}' | sort -n -r -k1

Let’s say you wanted to get the total number of LOC in the axis2-integration

cmri system query files --component axis2-integration --output-format tsv:noheader --selection Axis2/Axis2-1.7.9 --no-color 2>/dev/null | awk -F '\t' '{total = total + $11}END{print total}'

62922

These are simply examples that should make sense to those familiar with UNIX command line scripting. cmri commands can similarly be integrated into other languages by being called as shell commands from Python, Perl, Bash, Windows PowerShell, etc.

What component is a file in?

system query files --name modules/integration/test/org/apache/axis2/AbstractTestCase.java

What components does my component depend on?

system query component-relationships --from-component axis2-integration

NOTE: Some of the relationship_type fields say actual_dependency (one found in the code) while others say declared_dependency (one declared in the UDA)

What components does my component say that it depends on? (Declared)

system query component-relationships --from-component axis2-integration --relationship-type declared_dependency

What components does my component actually depend on?

system query component-relationships --from-component axis2-integration --relationship-type actual_dependency

What files are involved in relationships between two components?

system query file-relationships --from-component axis2-integration --to-component axis2-kernel

What entities within those files have relationships that cross the component boundaries?

system query entity-relationships --from-component axis2-integration --to-component axis2-kernel

What components depend on my component?

axis2-kernel is a utility used by many others so this should yield a lot:

system query component-relationships --to-component axis2-kernel

axis2-integration is at the top of the architecture, so should not yield much:

system query component-relationships --to-component axis2-integration

In fact… note that this is an error in either your code or your UDA. This connection should not exist or should be defined. Note this for the future

What architectural integrity problems exist? (differences between UDA and coded reality)

In this example, we are going to introduce ‘query chaining’ - using output from one command as inputs into the next. See the cmri API documentation for details.

system query component-relationships --severity error

As you can see, code inside axis2-kernel depends on axis2-integration, even though it should not.

Now let’s look at the files that are implicated:

system query component-relationships --severity error 
system query file-relationships --from-component @from_component --to-component @to_component

Let’s go one step deeper to look at entity relationships and find the exact line of code where problems exist:

system query component-relationships --severity error 
system query entity-relationships --from-component @from_component --to-component @to_component

What components depend on my component indirectly?

come back to this

What files depend on my component?

system query file-relationships --to-component axis2-kernel

What entities depend on my component?

system query entity-relationships --to-component axis2-kernel

Note above that the listing includes entity-relationships between things inside axis2-kernel as well as those coming in from outside. In the current expression language, there isn’t a great way to ask for incoming entity relationships that only span the boundary. (Perhaps this will be fixed in the future). For now, let’s try to do it using a combination of shell programming and cmri commands.

# save off the header fields into a file so you can see them
# to a temporary file
[dan@fedora ~]$ cmri system query entity-relationships --to-component axis2-kernel --output-format tsv --no-color --selection Axis2/Axis2-1.7.9 2> /dev/null | head -n 1 > /tmp/er_headers

The headers contained will be:

from_entity, to_entity, from_file, to_file, ref_line, ref_col, from_component, to_component, relationship_type, relationship_state, severity

Now we want to get the listing of entities that depend on axis2-kernel, but exclude the ones where ‘from-component’ is also axis2-kernel. Note that ‘from_component’ is field 7

# get the list entity dependencies coming into axis2-kernel
# get rid of headers
# save to a temporary file
[dan@fedora ~]$ cmri system query entity-relationships --to-component axis2-kernel --output-format tsv:noheader --no-color --selection Axis2/Axis2-1.7.9 > /tmp/er_for_axis2-kernel

# There are many ways to skin this cat.
# The UNIX awk command is one of them.
# The following line only prints a line if field 7 is not axis2-kernel
# This might also be done in a more readable way if done in Perl, Python, etc.
awk '$7 != "axis2-kernel" { print $0 }' /tmp/er_for_axis2-kernel

What files depend on my component directly or indirectly?

come back to this

There are many more things you can do with the cmri command line and the query interface. See the ‘Silverthread CodeMRI CLI Reference’ document to explore further. You should now have enough to get started.

  • No labels