That is because Plot first calculate PlotPoints number of points, and then refines the plot in between at most MaxRecursion times. This makes it jump 'back and forth'. It can be solved like this:
Plot[20/Pi*Integrate[((x+Sin[20 x])^2+Cos[5 x])*Sin[20 x],{x,z-Pi/20,z+Pi/20}],{z,0,20},MaxRecursion->0,PlotPoints->20]
But you lose the adaptive refining used by plot.
Another possibility is something like:
\[Beta] = {};
Dynamic[ListPlot[{#, 1} & /@ \[Beta], PlotRange -> {{0, 20}, {0, 2}}]]
Plot[(\[Beta] = DeleteDuplicates[Append[\[Beta], z]];
20/Pi*Integrate[((x + Sin[20 x])^2 + Cos[5 x])*Sin[20 x], {x,
z - Pi/20, z + Pi/20}]), {z, 0, 20}, MaxRecursion -> 1,
PlotPoints -> 30]
So you can see what points it is sampling...
