What if we have a stochastic game S and a set of agents A and in each game s we might have a different subset of A playing?  What type of game is that?  This of course is only relevant when dealing with |A| > 2 agents.  Also, the agents that play in each s could be stochastic or fixed. Of course we can always make it harder by making the number actions varied for the agents.

This seems like it could model tournaments if the games are competitive or stochastic coalition games.

This seems sort of like a Bayesian game…  Not sure though.