This greatly simplified system of equations with the syntax changed

In[1]:= RSolve[{a[m + 1] == (a[m]*0.3) - (a[m]*0.2) - (a[m]*0.5) + (0.4*c[m + 1]),

c[m + 1] == (c[m]*0.6) - (c[m]*0.8) - (c[m] 0.9) + (0.1*a[m + 1])}, {a[m], c[m]}, m]

Out[1]= {{a[m] -> 0.0324598 2.^(-2. m)8.44425*10^14^(-1. m) (1. (-3.95578*10^15)^m +

29.8074 (-1.32188*10^15)^m) C[1] - 0.587765 (-1.)^m 2.^(-2. m)8.44425*10^14^(-1. m)

(1.32188*10^15^m - 1. 3.95578*10^15^m) C[2], c[m] -> -0.0534332 (-1.)^m 2.^(-2. m)

8.44425*10^14^(-1. m) (1.32188*10^15^m - 1. 3.95578*10^15^m) C[1] + 0.96754 2.^(-2. m)

8.44425*10^14^(-1. m) (1. (-3.95578*10^15)^m + 0.0335488 (-1.32188*10^15)^m) C[2]}}

finds a solution.

This full system of equations with the same syntax changes

RSolve[{a[m + 1] == (a[m]*0.3*(x[m]/800)) - (a[m]*0.2) - (a[m]*0.5) + (0.4*c[m + 1]),

c[m + 1] == (c[m]*0.6*(z[m]/300)) - (c[m]*0.8) - (c[m] 0.9) + (0.1*a[m + 1]),

x[m + 1] == ((x[m]*0.3*(a[m]/800)) + (x[m]*0.2)) + (0.7*z[m + 1]),

z[m + 1] == ((z[m]*0.6*(c[m]/300)) + (z[m]*0.8)) + (0.9*x[m + 1])}, {a[m], x[m], c[m], z[m]}, m]

doesn't have a solution the system can find.

Sometimes using exact fractions instead of decimals can help you get answers, or better answers,

but that doesn't seem to help in this particular problem.