Measuring your CM process


	Unified Configuration Management
Home \| CM Survival \| The CM Portal \| CM About Us \| Email

Measuring your CM Process

Recently a couple of emails from readers have touched on the subject of how to determine the maturation of their CM process. In our opinion there is no single answer, however we believe the following combined activities are a solid start.

Compare your software process to the Capability Maturity Model.
Do a formal CM Gap Analysis (see our GAP questionnaire).
Introduce post project reviews i.e. where project discusses CM effectiveness.
Use metrics to measure your progress and alert you to problems (Trend Analysis).

Below we list a few simple CM metrics that we have used to support the CM audit process. However, be careful, the real power of metrics comes from the ability to compare them with historical data i.e. trend analysis. Hence the longer you collect the CM data the more useful, valuable and accurate your evaluation becomes.

Formula	Metrics Purpose
TE-SYS= SI/(SI+AI+PI)	Testing Efficiency - Ratio of defects caught in System Test
TE-UAT=AI/(AI+PI)	Testing Efficiency - Ratio of defects caught in User Acceptance Test
PR-SYS=UD/SD	Determines Ratio of kits promoted from System Test
PR-UAT=PD/UD	Deternines Ratio of kits promoted from User Acceptance Test
BR=SB/(SB+FB)	Determines Successful Build Ratio
BD=SB/TBD	Determines Build Duration Mean
RR=SR/(SR+FR)	Determines Successful Release Ratio
RF=(SR+FR)/X	Determines Release Frequency
RD=SR/TRD	Determines Release Duration Mean
Key: SI = Number of Incident raised in System Testing AI = Number of Incidents raised in Acceptance Testing PI = Number of Incident raised in a period (e.g. 1 month) in production SD = Number of System Test releases UD = Number of UAT releases (promotions from System Test) PD = Number of Prod releases in period (e.g. 1 month) after original releases SB = Total Successful Builds FB = Total Failed Builds SR = Total Successful Releases FR = Total Failed Releases TBD = Total Build Duration TRD = Total Release Duration X = Period (eg 14 days)

Page Last Modified: 08 Feb 2009