Yes you are right , as workaround:
InverseMellinTransform[
Integrate[
MellinTransform[Exp[-x^2]/(c - x), c, s] //
PowerExpand, {x, -Infinity, Infinity}, PrincipalValue -> True] //
Expand, s, c]
(*(-MeijerG[{{0, 1/2}, {}}, {{0, 0, 1/2}, {}}, -c, 1/2] +
MeijerG[{{0, 1/2}, {}}, {{0, 0, 1/2}, {}}, c, 1/2])/(2 \[Pi])*)
Only by solution with MeijerG function.