This greatly simplified system of equations with the syntax changed
In[1]:= RSolve[{a[m + 1] == (a[m]*0.3) - (a[m]*0.2) - (a[m]*0.5) + (0.4*c[m + 1]),
c[m + 1] == (c[m]*0.6) - (c[m]*0.8) - (c[m] 0.9) + (0.1*a[m + 1])}, {a[m], c[m]}, m]
Out[1]= {{a[m] -> 0.0324598 2.^(-2. m)8.44425*10^14^(-1. m) (1. (-3.95578*10^15)^m +
29.8074 (-1.32188*10^15)^m) C[1] - 0.587765 (-1.)^m 2.^(-2. m)8.44425*10^14^(-1. m)
(1.32188*10^15^m - 1. 3.95578*10^15^m) C[2], c[m] -> -0.0534332 (-1.)^m 2.^(-2. m)
8.44425*10^14^(-1. m) (1.32188*10^15^m - 1. 3.95578*10^15^m) C[1] + 0.96754 2.^(-2. m)
8.44425*10^14^(-1. m) (1. (-3.95578*10^15)^m + 0.0335488 (-1.32188*10^15)^m) C[2]}}
finds a solution.
This full system of equations with the same syntax changes
RSolve[{a[m + 1] == (a[m]*0.3*(x[m]/800)) - (a[m]*0.2) - (a[m]*0.5) + (0.4*c[m + 1]),
c[m + 1] == (c[m]*0.6*(z[m]/300)) - (c[m]*0.8) - (c[m] 0.9) + (0.1*a[m + 1]),
x[m + 1] == ((x[m]*0.3*(a[m]/800)) + (x[m]*0.2)) + (0.7*z[m + 1]),
z[m + 1] == ((z[m]*0.6*(c[m]/300)) + (z[m]*0.8)) + (0.9*x[m + 1])}, {a[m], x[m], c[m], z[m]}, m]
doesn't have a solution the system can find.
Sometimes using exact fractions instead of decimals can help you get answers, or better answers,
but that doesn't seem to help in this particular problem.