I cannot reproduce the problem. Executing the following code gives the expected output:
Graphics[{
Rectangle[], Rectangle[{1, 1}],
Purple, Rectangle[{0, 1}], Rectangle[{1, 0}],
Yellow, Scale[Rectangle[], 0.5], Scale[Rectangle[{1, 1}], 0.5],
Green, Scale[Rectangle[{0, 1}], 0.5], Scale[Rectangle[{1, 0}], 0.5]
}, Axes -> True]
EDIT:
The above was tested in Mathematica 12.3. But in 12.2 the problem can be reproduced. See my post below.