Then you see that it was for an algorithm competition. The core idea in these competitions is that different algorithms are tested on the same data set. From this, we can gain an objective sense of their relative performance.
The issue/difficulty they were trying to overcome is that deep learning/deep neural networks take a long time to train. ML algorithms are generally compared based on their performance on tasks that have specified training and testing data.
Within the confines of a competition, overfitting is a problem. Competitors frequently tune their engines unknowingly or knowingly (let’s assume they’re working in good faith) to the test data sets few thousands instances. The result is overfitting. It’s very hard to determine if have the authors just fitted their algorithm well or is the speedup is inherent to the base pattern.