I found a really cool paper that basically benchmarks a lot of MARL algorithms using MGS, a stochastic game simulator written in Java.  This simulator is like my MALSIM except I only implemented features for repeated games not stochastic games.