Using the Bill James Similarity Index to Compare SWMM5 and InfoWorks ICM Hydraulic Networks

This paper presents a novel paradigm for evaluating hydraulic modeling platforms using the Bill James Similarity Index, a statistical technique initially created for baseball statistics. The methodology allows for a systematic comparison of the Storm Water Management Model 5 (SWMM5 utilizing ICM SWMM) and InfoWorks ICM by assessing their computational outputs and performance in standardized situations.

The methodology takes advantage of the unique ability to run both SWMM5 and InfoWorks ICM simultaneously under the same user interface, resulting in equivalent InfoWorks Result (IWR) files. The research uses 100 historical SWMM test files generated from SWMM3, SWMM4, and SWMM5 releases, which constitute the USEPA's standard engine testing suite. ICM SWMM is the background network, while ICM InfoWorks is the current network.

A modified similarity index was created by assigning weighted scores (0-100) to key hydraulic characteristics at shared network nodes. These metrics include flow rates, water depths, and velocities, with weights based on hydraulic relevance. Metrics such as Current Network Mean and Background Network Mean. Root Mean Square Error, Mean Average Error, Mean Simple Least Square Error, Simulated Standard Deviation, Skewness of Background Network, Kurtosis of Background Network, Kurtosis of Current Network, Log Nash-Sutcliffe efficiency. Index of agreement, integral square error. Kling-Gupta, The correlation coefficient and Nash-Sutcliffe efficiency were used to guide the similarity index.

The evaluation framework looks at three major dimensions: computational correctness vs observed data, hydraulic robustness in problem settings (backwater effects, surcharges, flooding etc), and computational efficiency for large networks. Ruby scripting streamlines the analytical procedure, resulting in consistent evaluation across all test situations. This methodology combines sports analytics and hydraulic engineering to provide a consistent, replicable protocol for model comparison. The obtained similarity ratings provide objective criteria for selecting modeling platforms based on project specifications.