- Probability basics review
- Strong Law of large numbers.
- Introduction to stochastic process; sample paths and finite dimensional distributions
- Kolmogorov”s consistency conditions.
- Renewal processes, reward and cost models, cumulative processes, Poisson process and stopping times.
- Regenerative processes, relation between a time average and mean of limiting distribution, Wald”s equation, renewal equation, renewal theorms.
- Discrete time Markov chains
- Class properties, irreducible, and positive recurrent chains, frequencies and relative frequencies for ergodic chains, costs and rewards for ergodic chains, transient behaviour.