I am afraid that a useful conjecture has not been stated in learning theory in spite of the fact that say the new model can answer old unanswered questions that theory folks used to care about. I’m very curious for example about non proof enabling assumption style theory that has predictive power eg one that says “given this real data set this real algorithm this real model and this realistic HP choices we can fit the training labels” Knowing nothing about what these 100 problems are, but expecting their style to pattern match “problem theory people could not solve for N years” I’m willing to buy you dinner at French laundry or whatever if any of them are even close to the the style of the one from above. Not because Bel or whatever the model is called can’t solve it, but because the above question never bothered theoreticians Here’s a question I tried but failed to answer with astra “Given lenet5 and mnist prove that any practically relevant variant of SGD will fit the labels and do so in Flops less than a small fraction of what it would be required to actually train the model till it fits the labels” Now trad theory doesn’t care about this problem, as it sounds too pedestrian, yet I think it’s an instance of what useful math will look like in a few years Sorry about the ramble.