SI404: Applied Stochastic Processes

Prof. Radhendushka Shrivastava

Course Description:

In recent years there has been an ever increasing interest in the study of systems
which vary in time in a random manner. Mathematical models of such systems are
known as stochastic processes. In this course we present an elementary account of
some of the important topics in the theory of such processes. We have tried to
select topics that are conceptually interesting and that have found fruitful
application in various branches of science and technology.

Course Contents:

  1. Markov Chains, Markov chains having two states, Transition function and initial distribution, Computations with transition functions, Hitting times, Transition matrix, Transient and re:current states, Decomposition of the state space, Absorption probabilities
  2. Stationary Distributions of a Markov Chain, Birth and death chain, Particles in a box, Average number of visits to a recurrent state, Null recurrent and positive recurrent states, Existence and uniqueness of stationary distributions, Reducible chains, Queuing chain, Convergence to the stationary distribution
  3. Markov Pure Jump Processes, Construction of jump processes, Birth and death processes, Two-state birth and death process, Poisson process, Pure birth process, Properties of a Markov pure jump process, Applications to birth and death processes
  4. Second Order Processes, Mean and Covariance functions, Gaussian processes, The Wiener process

Attendance, Course Load and Grading:

No compulsory attendance but professor is very sentimental about it and as soon as attendance is less than a threshold, he conducts a surprise quiz.

Course load is moderate. 3 hours a week of lectures. But one must be prepared for surprise quizzes and hence, needs to be up to date with all concepts in class (Surprise quizzes are easy, one must have basic understanding of all concepts).

Course was low scoring overall due to many surprise quizzes. But papers were very easy if one studies properly and in fact, it is easy to score a good grade in this course if one just pays attention in class.

Policy: Quiz (15%), Midsem (30%), Endsem (35%), 3 Surprise Quizzes (20%)


Introduction to Stochastic Process, Hoel, Port and Stone

One must follow this text book thoroughly and solve every sum in the exercises. If one does so, he will score in exams as many questions are similar in general.

Added Information:

Attendance is very important due to issues stated above. Professor is one who really students to learn and if people pay attention and study, he gets pretty motivated. In general he keeps high scoring papers, he just wants students to learn everything taught in class well. Follow the text book well, course content and problems are based on it.