Python code explanation for stationary distribution of a Markov chain

2024/10/1 23:28:31

I have got this code:

import numpy as np
from scipy.linalg import eig 
transition_mat = np.matrix([[.95, .05, 0., 0.],\[0., 0.9, 0.09, 0.01],\[0., 0.05, 0.9, 0.05],\[0.8, 0., 0.05, 0.15]])S, U = eig(transition_mat.T)
stationary = np.array(U[:, np.where(np.abs(S - 1.) < 1e-8)[0][0]].flat)
stationary = stationary / np.sum(stationary)>>> print stationary
[ 0.34782609  0.32608696  0.30434783  0.02173913]

But I can't understand the line:

stationary = np.array(U[:, np.where(np.abs(S - 1.) < 1e-8)[0][0]].flat)

Can anyone explain the part: U[:, np.where(np.abs(S - 1.) < 1e-8)[0][0]].flat ?

I know that the routine returns S: eigenvalue, U : eigenvector. I need to find the eigenvector corresponding to the eigenvalue 1. I have wrote the code below:

for i in range(len(S)):if S[i] == 1.0:j = imatrix = np.array(U[:, j].flat)

I am getting output:

:  [ 0.6144763   0.57607153  0.53766676  0.03840477]

but it does not give the same output. why?!

Answer

How to find a stationary distribution.

Ok, I came to this post looking to see if there was a built-in method to find the stationary distribution. It looks like there's not. So, for anyone coming in from Google, this is how I would find the stationary distribution in this circumstance:

import numpy as np#note: the matrix is row stochastic.
#A markov chain transition will correspond to left multiplying by a row vector.
Q = np.array([[.95, .05, 0., 0.],[0., 0.9, 0.09, 0.01],[0., 0.05, 0.9, 0.05],[0.8, 0., 0.05, 0.15]])#We have to transpose so that Markov transitions correspond to right multiplying by a column vector.  np.linalg.eig finds right eigenvectors.
evals, evecs = np.linalg.eig(Q.T)
evec1 = evecs[:,np.isclose(evals, 1)]#Since np.isclose will return an array, we've indexed with an array
#so we still have our 2nd axis.  Get rid of it, since it's only size 1.
evec1 = evec1[:,0]stationary = evec1 / evec1.sum()#eigs finds complex eigenvalues and eigenvectors, so you'll want the real part.
stationary = stationary.real

What that one weird line is doing.

Let's break that line into parts:

#Find the eigenvalues that are really close to 1.
eval_close_to_1 = np.abs(S-1.) < 1e-8#Find the indices of the eigenvalues that are close to 1.
indices = np.where(eval_close_to_1)#np.where acts weirdly.  In this case it returns a 1-tuple with an array of size 1 in it.
the_array = indices[0]
index = the_array[0]#Now we have the index of the eigenvector with eigenvalue 1.
stationary = U[:, index]#For some really weird reason, the person that wrote the code
#also does this step, which is completely redundant.
#It just flattens the array, but the array is already 1-d.
stationary = np.array(stationary.flat)

If you compress all these lines of code into one line you get stationary = np.array(U[:, np.where(np.abs(S-1.)<1e-8)[0][0]].flat)

If you remove the redundant stuff you get stationary = U[:, np.where(np.abs(S - 1.) < 1e-8)[0][0]]

Why your code gives a different stationary vector.

As @Forzaa pointed out, your vector cannot represent a vector of probabilities because it does not sum to 1. If you divide it by its sum, you'll get the vector the original code snippet has.

Just add this line:

stationary = matrix/matrix.sum()

Your stationary distribution will then match.

https://en.xdnf.cn/q/70917.html

Related Q&A

Detecting insertion/removal of USB input devices on Windows 10

I already have some working Python code to detect the insertion of some USB device types (from here).import wmiraw_wql = "SELECT * FROM __InstanceCreationEvent WITHIN 2 WHERE TargetInstance ISA \W…

FastAPI as a Windows service

I am trying to run FastAPI as a windows service.Couldnt find any documentation or any article to run Uvicorn as a Windows service. I tried using NSSM as well but my windows service stops.

Why not use runserver for production at Django?

Everywhere i see that uWSGI and Gunicorn are recommended for production mode from everyone. However, there is a lot more suffering to operate with it, the python manage.py runserver is more faster, sim…

How to create decorator for lazy initialization of a property

I want to create a decorator that works like a property, only it calls the decorated function only once, and on subsequent calls always return the result of the first call. An example:def SomeClass(obj…

How to convert a 24-bit wav file to 16 or 32 bit files in python3

I am trying to make spectrograms of a bunch of .wav files so I can further analyze them(in python 3.6), however, I keep getting this nasty errorValueError: Unsupported bit depth: the wav file has 24-bi…

Get consistent Key error: \n [duplicate]

This question already has answers here:How do I escape curly-brace ({}) characters in a string while using .format (or an f-string)?(23 answers)Closed 8 years ago.When trying to run a script containin…

Cannot take the length of Shape with unknown rank

I have a neural network, from a tf.data data generator and a tf.keras model, as follows (a simplified version-because it would be too long):dataset = ...A tf.data.Dataset object that with the next_x me…

Pre-fill new functions in Eclipse and Pydev with docstring and Not Implemented exception

I am editing my Python source code with Eclipse and Pydev.I want to document all of my functions and raise a "Not Implemented" exception whenever a function have not yet been implemented. For…

How to serialize hierarchical relationship in Django REST

I have a Django model that is hierarchical using django-mptt, which looks like:class UOMCategory(MPTTModel, BaseModel):"""This represents categories of different unit of measurements.&qu…

Django: Loading another template on click of a button

Ive been working on a django project for a few weeks now, just playing around so that I can get the hang of it. I am a little bit confused. I have a template now called "home.html". I was wo…