Lovely Michael! Thanks a lot!

Everything is consistent with your explanation. And thank you for directing me to the right documentation.

Hello Julian, InfiniteLine supports two types of arguments, InfiniteLine[{p1,p2}] and InfiniteLine[ p1, direction ]; I attached a screenshot from the documentation on InfiniteLine (M12.1)

In your example the arguments of InfiniteLine is the midpoint and the direction since there is no curly bracket wrapped around thearguments. You can extract the two points from the InfiniteLine this way:

pointsInfiniteLine = perBi2 /. InfiniteLine -> List // Accumulate

Finally you can display everything and it looks consistent now:

Graphics[{Red, PointSize[0.03], Point[{p1, p2}], Green, 
  Line[{p1, p2}], Blue, perBi2, Point[pointsInfiniteLine]}, 
 Frame -> True, GridLines -> Automatic]