Message Boards Message Boards

Analyze Markov chains when cardinality of state space is unknown?

Posted 10 years ago
I am trying to find the steady state distribution of a discrete time, finite state Markov chain. My problem is that I do not know the cardinality of the state space and want to perform the analysis using some variable N that represents the total number of states. The state vector would then be something like {1,2,3...N}. I know the structure of the transition probability matrix.  Is there a way for mathematica to solve a markov chain when the cardinality of the sate space is not known?. Any pointers will be appreciated.
POSTED BY: Az q
Suppose you is able to solve the problem for a fixed size of state-space, i.e. construct the transition matrix and compute the steady state vector. Then you may use FindSequenceFunction to guess how elements of the stationary probability vector depend on N.

Here is a simplistic example:
tm[n_Integer /; n > 1] :=
SparseArray[{{i_, j_} /; j - i == 1 && j <= n :> 1/2, {i_, i_} /; i < n :> 1/2, {n, n} -> 1}, {n, n}];

In[150]:= Table[PDF[StationaryDistribution[DiscreteMarkovProcess[1, tm[n]]], n], {n, 2, 12}]
Out[150]= {1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}

--Sasha
POSTED BY: Oleksandr Pavlyk
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract