Inverse
might be doing a better job of simplifying the result. Possibilities to get around the issue using LinearSolve
include post simplification to remove any common factors.
sol2 = Together[x1, Trig -> True];
sol2 /. vars0
(* Out[23]= {(384 (-2304 \[Mu] - 720 \[Mu]^2 - 56 \[Mu]^3))/(
m R^2 \[Delta]^2 \[Mu] (48 + 7 \[Mu]) (-1536 \[Mu] -
224 \[Mu]^2)), -((576 (4 + \[Mu]) (-384 \[Mu] - 56 \[Mu]^2))/(
m R^2 \[Delta]^2 \[Mu] (48 + 7 \[Mu]) (-1536 \[Mu] -
224 \[Mu]^2))), 0, 0} *)
This next also works but gives less simplification.
x2 = LinearSolve[massmat2, rhs, Method -> "CofactorExpansion"];
x2 /. vars0
(* Out[25]= {(-((m^3 R^6 \[Delta]^6 (4 + \[Mu])^2 (6 + \[Mu]))/6144) + (
m^3 R^6 \[Delta]^6 (6 + \[Mu])^2 (3 + 2 \[Mu]))/
6912)/(-(1/256)
m^2 R^4 \[Delta]^4 (4 + \[Mu])^2 (-(1/256)
m^2 R^4 \[Delta]^4 (4 + \[Mu])^2 +
1/288 m^2 R^4 \[Delta]^4 (6 + \[Mu]) (3 + 2 \[Mu])) +
1/288 m^2 R^4 \[Delta]^4 (6 + \[Mu]) (3 +
2 \[Mu]) (-(1/256) m^2 R^4 \[Delta]^4 (4 + \[Mu])^2 +
1/288 m^2 R^4 \[Delta]^4 (6 + \[Mu]) (3 + 2 \[Mu]))), ((
m^3 R^6 \[Delta]^6 (4 + \[Mu])^3)/4096 - (
m^3 R^6 \[Delta]^6 (4 + \[Mu]) (6 + \[Mu]) (3 + 2 \[Mu]))/
4608)/(-(1/256)
m^2 R^4 \[Delta]^4 (4 + \[Mu])^2 (-(1/256)
m^2 R^4 \[Delta]^4 (4 + \[Mu])^2 +
1/288 m^2 R^4 \[Delta]^4 (6 + \[Mu]) (3 + 2 \[Mu])) +
1/288 m^2 R^4 \[Delta]^4 (6 + \[Mu]) (3 +
2 \[Mu]) (-(1/256) m^2 R^4 \[Delta]^4 (4 + \[Mu])^2 +
1/288 m^2 R^4 \[Delta]^4 (6 + \[Mu]) (3 + 2 \[Mu]))), 0, 0} *)