Hypothesis: any general function approximator that is powerful enough (eg has enough parameters) and can be efficiently trained will eventually perform similarly. The underlying substrate (LSTM, CNNs, Transformers etc) won’t matter as much as the data and objective functions.