马尔可夫链

State is simply the category.

Markov Chains are a combination of probabilities and matrix calculus.

Markove Chains model a process that proceeds in steps(time, sequence, trials, etc.); like a series of probability trees.

Transition diagram 等价于 Transition Matrix，same information represented in different way。

Transition Matrix的行代表to which state，列代表from which state。每一行的概率相加起来等于1。

对于一个特定的initial state vector，在经过许多次state的转移后，转移结果会和initial state vector一样

$$vP=v$$

如果已知Transition Matrix，可以计算出steady state vectors，使用单位矩阵计算。

$$v(P-I)=0$$