Week 5 Ada—Ramasubbu and Kemerer (2015)

Technical Debt and the Reliability of Enterprise Software Systems:

A Competing Risks Analysis

Key Concept:

Technical Debt: Taking design shortcuts and other maintenance activities software organizations incur what has been referred to as technical debt, that is, accumulated maintenance obligations that must be addressed in the future

Motivation:

Technical debt reduction in an enterprise software systems environment is difficult, and maintenance of such systems is especially challenging because of the interdependencies and potential for conflict between the underlying, vendor-supplied platform and the customization done by individual clients. These interdependencies make it difficult to measure and assess the impact of technical debt on system reliability and therefore to plan the software maintenance activities necessary to reduce the debt.

Research Question:

• Model and empirically analyze the impact of technical debt on system reliability.
• Examine the relative effects of modular and architectural maintenance activities undertaken by clients in order to analyze the dynamics of technical debt reduction.

Main Findings:

• Technical debt decreases the reliability of enterprise systems.
• Modular maintenance targeted to reduce technical debt was about 53% more effective than architectural maintenance in reducing the probability of a system failure due to client errors.
• Modular maintenance had the side-effect of increasing the chance of a system failure due to vendor errors by about 83% more than did architectural maintenance activities.

Contributions:

• This study empirically measure the technical debt accumulated in real world enterprise software and to assess its dynamic impact on system reliability.
• They address this challenge by utilizing a competing risks analysis approach that account for event-specific hazards that impact the failure of enterprise software systems.
• They utilize the empirical results to illustrate how firms could evaluate their business risks exposure due to technical debt accumulation in their enterprise systems and assess the likely effects, both positive and negative, of a range of software maintenance practices.

Competing risks:

Competing-risks survival regression provides a useful alternative to Cox regression in the presence of one or more competing risks. For example, say that you are studying the time from initial treatment for cancer to recurrence of cancer in relation to the type of treatment administered and demographic factors. Death is a competing event: the person under treatment may die, impeding the occurrence of the event of interest, recurrence of cancer. Unlike censoring, which merely obstructs you from viewing the event, a competing event prevents the event of interest from occurring altogether, and your analysis should adjust accordingly. As a technical consequence, an individual observed to fail from a competing risk is assumed to still be at risk between its real-life failure time and its potential future censoring time.

This fact has two important implications. First, the naïve Kaplan–Meier that takes the competing events as censored observations, is biased. Secondly, the way in which covariates are associated with the cause-specific hazards may not coincide with the way these covariates are associated with the cumulative incidence.

A complete understanding of the event dynamics requires that both cause-specific and sub-distribution hazards to be analyzed. The difference between cause-specific and sub-distribution hazards is the risk set. For the cause-specific hazard the risk set decreases each time there is a death from another cause censoring. With the sub-distribution hazard subjects that die from another cause remain in the risk set and are given a censoring time that is larger than all event times.