Bill James SWMM5 network similarity scores for quasi unitless testing of H&H models

Background:  We want to have one similarity metric for comparing SWMM5 results across different engine versions, different compilation options, 32 vs 64 bit engines, different compilers, different programming languages and different process options in our models.  We have 30,000+ test models ranging from a PCSWMM 3.3 One Subcatchment Model to a 500,000 Element H&H Model. We also want to answer questions such as what is the best Priessman Slot width?  What is the best ponding area to use for the best results?

What are we comparing:  Normal SWMM5 OUT binary file variables for Nodes, Links, Subcatchments and the System Wide parameters.   500 Internal variables for the nodes, links and Subcatchments.  Examples, of internal variables would be upstream, middle and downstream depth, width, hydraulic radius and cross-sectional area for each link.  In addition, we are using quasi units test instead of coding the unit tests, we record the input and output data of our functions automatically and save them into the OUT binary file.  Examples of a function would be buildup, washoff, infiltration and evaporation for a Subcatchment. The input and output would be saved so differences in engines can be better quantized.

There is a well-known Sabermetric tool called Similarity Score which has the intent of the intent of discovering who the most similar historical players are to a certain player in baseball and basketball (this was created by Bill James the baseball writer).  The scale goes from 0 to 1000 and is used to evaluate current players compared to Hall of Fame players.  We expand our past Bill James SWMM similarity score for ICWMM 2019 to construct a similar scale for SWMM5 models based on all internal, OUT parameters and newer unitless function testing. The Bill James score is cross elements and uses 15 statistical methods of comparison gleaned from the renowned missive Rules for Responsible Modeling from CHI.

As William James the Philosopher says, “An idea, to be suggestive, must come to the individual with the force of revelation.”  We want to use the SWMM Similarity Score to help reveal problems with the coding in SWMM5.

 Click here to download a static PDF version of the presentation.