The inconsistent measurement of Message Chains

Fowler and Beck defined 22 Code Bad Smells. These smells are useful indicators of code that may need to be refactored. A range of tools have been developed that measure smells in Java code. We aim to compare the results of using two smell measurement tools (DECOR which is embedded in the Ptidej tool and Stench Blossom) on the same Java code (ArgoUML). This comparison identifies the code each tool identifies as containing Message Chains. We evaluate the results from these two tools using human judgment on the smells that the code contains. We look in detail at how and why the results differ. Our results show that each tool identified very different code as containing Message Chains. Stench Blossom identified very many more code instances of Message Chains than DECOR. We found three reasons why these discrepancies occurred. First there are significant differences in the definitions of Message Chains used by each tool. Second, the tools use very different measurement strategies. Third, the thresholds embedded in the tools vary. This measurement inconsistency is a problem to practitioners as they may be applying refactoring ineffectively. This inconsistency is also a problem for researchers as it undermines the reliability of making cross study comparisons and prevents mature knowledge the impact of smells being developed.