For me works fine.
Thanks, but these are huge expressions. I need short simple answers in degrees. Can you convert these answers in degrees?
Specifying the image of one vector only removes two degrees of freedom from the three present in the general 3D rotation, so your system of equations is underconstrained and does not have a unique solution.
I understand you are looking for a rotation-matrix which transforms one vector into another one. It seems that there are several possiblities.
I would like to suggest a more or less straightforward approach.
Let xx be your original, xxp the target vector (both have equal length of course). Now I can by successive rotations transform xx that it has only a z component, xxp only a y and z component. Then I can find the rotation-matrix that transforms the transformed xx into the transformed xxp, and then transform everything back int the original frame.
Look at this code
xx = {x, y, z} = {3, 5, 7}
xxp = {xp, yp, zp} = {7, 3, 5}
xt1 = FullSimplify[
RotationMatrix[ArcTan[xx[[1]]/xx[[2]]], {0, 0, 1}].xx]
xp1 = FullSimplify[
RotationMatrix[ArcTan[xx[[1]]/xx[[2]]], {0, 0, 1}].xxp]
xt2 = RotationMatrix[ArcTan[xt1[[2]]/xt1[[3]]], {1, 0, 0}].xt1
xp2 = RotationMatrix[ArcTan[xt1[[2]]/xt1[[3]]], {1, 0, 0}].xp1 //
FullSimplify
xt3 = RotationMatrix[ArcTan[xp2[[1]]/xp2[[2]]], {0, 0, 1}].xt2
xp3 = RotationMatrix[ArcTan[xp2[[1]]/xp2[[2]]], {0, 0, 1}].xp2 //
FullSimplify
xp4 = RotationMatrix[ArcTan[xp3[[2]]/xp3[[3]]], {1, 0, 0}].xp3
tRM = RotationMatrix[
ArcTan[xp2[[1]]/xp2[[2]]], {0, 0, 1}].RotationMatrix[
ArcTan[xt1[[2]]/xt1[[3]]], {1, 0, 0}].RotationMatrix[
ArcTan[xx[[1]]/xx[[2]]], {0, 0, 1}] // Simplify
tRM.xx // FullSimplify
tTM = RotationMatrix[-ArcTan[xp3[[2]]/xp3[[3]]], {1, 0, 0}]
tTM.{0, 0, Sqrt[83]}
rotM = Inverse[tRM].tTM.tRM // Simplify
rotM.xx // Simplify
rotM is the Matrix you are looking for, but I agree, you don’t have the angles you wanted.
Here is another way to get a RotationMatrix which transforms x to xp ( and vice versa)
rotM // MatrixForm
RotationMatrix[Pi, (xx + xxp)/2] // MatrixForm
RotationMatrix[Pi, (xx + xxp)/2].xx
RotationMatrix[Pi, (xx + xxp)/2].xxp