What we'll cover in Notation and Theory

All discussion and resources for our first topic of the book club which is Notation and theory.
This will be out most “free form” section as we’re not covering self contained notebook. The idea of this week is to cover the prerequisites so we can focus on State Space Models together

For this I need your help,. which of the three topics below would you like an overview of?

  1. An overview of distributions, how they’re specified, and what they represent
  2. Be comfortable with the idea of Bayesian updates
  3. Be able to read notation such as the one below.

For readings I ask that you start by reading

  • the Dynamax README In particular what are state space models
  • Section 8.1 and 8.1.1 from the ProbML advanced topics

In the coming week I will also add additional readings based on the answers above

1 Like

Personally my order of priority as you have them listed is 3, 1, 2.

Specifically though:

  1. Be able to read notation such as the one below.

I’d like to hear your mental process working through notation like that, especially without the academia background. I know personally I can get into the tendency when reading to stare at an equation and think “then blah” and move to the next section of the text, so would be nice to hear your mental representation.

  1. An overview of distributions, how they’re specified, and what they represen

More in the context of state space models. Have background in the standard distributions, but are there specific ones used in this domain that we need to get comfortable with.

4 Likes

I think I generally agree with @tburch with his ordering of importance. I wasn’t sure on 2. though:

  1. Be comfortable with the idea of Bayesian updates

I’d like to know if this is different in SSMs to my more general understanding of Bayesian stats i.e. updating your prior with data from the likelihood gives a posterior with these priors being decisions abut the initial distributions of the parameters in your model. Like does Bayesian updating differ between “offline” and “online” SSMs? Offline appears to be my current understanding of updating with “static priors” but online seems to be constantly updating priors as data comes in… I’m not sure on this tbh.

1 Like

Exciting stuff! Echoing what others have said, notation is the highest priority for me. I’m relatively familiar with this kind of notation but I would love to know how you think through it. @tburch said it better than me:

I know personally I can get into the tendency when reading to stare at an equation and think “then blah” and move to the next section of the text, so would be nice to hear your mental representation

I would slightly favour distributions over updates as I think they can get a little bit thorny to think through here.

My preferred order is 2 (I am a frequentist) and 1/3 (familiar with most distributions and have worked with SSMs before).

My preferred order would align with the order you presented the topics.

  1. I am familiar with the distributions, but it would be nice to have an overview of representation limitations, advantages, and specific usage in SSMs.
  2. Bayesian updating is at the core of most next discussions, and maybe it is good to jump in here.
1 Like