SSM vs other Approaches


Wasn’t sure what to title this or if it’s best to just make 1 big resource/article dump thread.

This is what I posted in the chat: I just randomly came across this article comparing LSTM to a more traditional SS approach in forecasting while trying to gather articles for a project I’m working on. It’s open access. Granted this is in the context of time series. I’ll do some googling and see if I find any other papers. Also, I really don’t know much about deep learning, so it is hard for me judge that aspect of the comparison.

I work in healthcare typically with physicians/drug companies/gov agencies, helping them best translate their research questions into a model. A mix of analytics and traditional academic paper writing. Many of them come from a very basic frequentist health service / medical training, so just getting them to see the value in using bayesian / more flexible models to understand their problems is a great step forward for me. Being able to extract components and the ease of presenting uncertainty around those components and graph those is very interpretable for people that often view the world through basic cross tabulations and graphs.

Medicine can be pretty gee-whiz about deep learning these days. I think comparisons are really important to say okay, cool, but what is the practical value add of this new technology outside of your study.

So my knowledge is really limited to time series applications and if someone can speak to their other domains where a comparison between deep learning and state space models has been made, please do.

Edit: another article: ARIMA vs ETS vs NN


1 Like

Thanks for sharing these papers!

Something I learned in the Advanced Topics book (page 358, lines 8–9) is that SSMs can be interpreted as instances of neural networks:

Note that an LG-SSM is just a special case of a Gaussian Bayes net (Section 4.2.3), so the entire joint distribution p(y_{1:T} , z_{1:T} |u_{1:T} ) is a large multivariate Gaussian with N_y N_z T dimensions