Using Metrics To Manage Software Projects

From Maisqual Wiki

Jump to: navigation, search

Edward F. Weller, Bull HN Information Systems.



  1. Project Management Levels
    1. First Level
    2. Second Level
    3. Third Level
  2. Project Planning
    1. Using defect data to plan test activities
    2. Multiple data views
    3. Using test cost
    4. Sanity check
  3. Project Tracking
    1. Interpreting effort variance
    2. Project tracking and analysis


 author = {Weller, Edward F.},
 title = {Using metrics to manage software projects},
 journal = {Computer},
 volume = {27},
 issue = {9},
 month = {September},
 year = {1994},
 issn = {0018-9162},
 pages = {27--33},
 numpages = {7},
 url = {},
 doi = {10.1109/2.312035},
 acmid = {620050},
 publisher = {IEEE Computer Society Press},
 address = {Los Alamitos, CA, USA},



Five years ago, Bull's Enterprise Servers Operation in Phoenix, Arizona, used a software process that, although understandable, was unpredictable in terms of product quality and delivery schedule. The process generated products with unsatisfactory quality levels and required significant extra effort to avoid major schedule slips. All but the smallest software projects require metrics for effective project management. Hence, as part of a program designed to improve the quality, productivity, and predictability of software development projects, the Phoenix operation launched a series of improvements in 1989. One improvement based software project management on additional software measures. Another introduced an inspection program, since inspection data was essential to project management improvements. Project sizes varied from several thousand lines of code (KLOC) to more than 300 KLOC. The improvement projects enhanced quality and productivity. In essence, Bull now has a process that is repeatable and manageable, and that delivers higher quality products at lower cost. We describe the metrics we selected and implemented, illustrating with examples drawn from several development projects.


Former process was unpredictable in terms of product quality and delivery schedule. A program has been setup to improve quality, productivity, and predictability.

Projects size for this program was from a few hundreds to more than 300 KLOC.

Project Management Levels

There are 3 levels of project management capability based on software metrics.

  • First Level: process has resources as input, product as output, and a black box in between.
  • Second Level: The defect discovery rate is analyzed and used to predict the delivery.. In that case, the end of the test-cycle can be predicted as soon as the fall appears, but not before. Metrics used in this stage are:
    • Effort in person-month
    • Computer resources used
    • Product size
    • Number of defects found in integration and system tests.

Those didn't help much: little or no correlation, availability of data..

  • Third Level: Metrics gathered were:
    • Effort in person-month
    • Computer resources used
    • Estimated product size at each development stage.
    • Product size after coding
    • Product size after each test stage
    • Number of defects found in all development stages from inspections, unit tests, integration tests, and system tests
    • Estimated completion date for each phase.

Project Planning

Once the first size estimate done, project manager uses data for effort and schedule planning.

Then estimate and actual data are compared. Sometimes if initial size estimate shows to be wrong, estimates are revised.

Other view points are needed, and great differences should be checked. On a project a serious requirement error was discovered at the system testing stage. But afterwards this could have been discovered as early as in the unit testing stage.

Stages identified on the process are the following:

  • Requirements Analysis
  • High-Level Design
  • Low-Level Design
  • Coding
  • Unit Tests
  • Integration Tests
  • System Tests
  • Beta Ship
  • General Ship

Results were compared to the Cocomo estimating model output and revised.

Project Tracking

Design documents were inspected and measures applied to them. expected default rate was 0.5 to 1.0 default per page. For measures below 0.5, check that the document were well inspected and complete enough.

As for the working product, if defect rate is low, either size estimate is wrong or detail level in the work product is insufficient.

Process instrumentation and inspections were the keys to successful design completion and overall visibility.

Project tracking and analysis: 7-to-1 differences could be observed between different projects in the default rate. Between similar projects, variance is much lower.

Personal tools