Why don't you just type it in to get an idea?
In[1]:= DSolve[D[f[x], x] == Sin[x^2], f, x]
Out[1]= {{f -> Function[{x}, C[1] + Sqrt[\[Pi]/2] FresnelS[Sqrt[2/\[Pi]] x]]}}
In[5]:= Clear[h, f]
f[x_] := Sqrt[\[Pi]/2] FresnelS[Sqrt[2/\[Pi]] x]
h[x_] := f[x^2]
In[8]:= D[h[x], x, x]
Out[8]= 8 x^4 Cos[x^4] + 2 Sin[x^4]
So Mathematica agrees with the answer your book presents. That's good, but not good enough: For sure, if the solution is given, you are proposed to find the solution path: Try to find the solution without solving the differential equation.
What happens if you don't know f
itself, but only it's first derivative?
In[9]:= Clear[h, f]
h[x] := f[x^2]
In[11]:= D[h[x], x, x]
Out[11]= 2 Derivative[1][f][x^2] + 4 x^2 (f^\[Prime]\[Prime])[x^2]
What do you do?
- use the derivative in the first summand, take care about the dependency from x - easy, you get one summand of the final solution here
- take the derivative of the derivative (to get the second one), again take care about the dependency from x
That's the way your teacher intends you to go; all steps can by done inside Mathematica. Go ahead!