Google Research recently wrote a piece The reusable holdout: Preserving validity in adaptive data analysis  (so I’m not go to write much).  It details the problem with statistics generated when the machine learning methods are adapted to the data through data analysis and repeated trials on the same hold out data.  This is a problem that is easily recognizable in machine learning competition leader boards like those on Kaggle (good article on that).  The solution they gave was to use their method detailed here and here (going to be published in Science) that allows us to reuse the hold out data set many times without loosing its validity.  So, that is awesome!  Hopefully the Kaggle competitions and data science research will be greatly improved by having more meaningful feedback.