IE611: Introduction to Stochastic processes


  • Probability basics review
  • Strong Law of large numbers.
  • Introduction to stochastic process; sample paths and finite dimensional distributions
  • Kolmogorov”s consistency conditions.
  • Renewal processes, reward and cost models, cumulative processes, Poisson process and stopping times.
  • Regenerative processes, relation between a time average and mean of limiting distribution, Wald”s equation, renewal equation, renewal theorms.
  • Discrete time Markov chains
  • Class properties, irreducible, and positive recurrent chains, frequencies and relative frequencies for ergodic chains, costs and rewards for ergodic chains, transient behaviour.

Useful resources:



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s