One way is to transpose the matrix, Insert the new column (as a simple list), and transpose back:
mat = {{a, b}, {1, 2}};
newColumn = {x, y};
positionWhereToInsert = 2;
Insert[Transpose[mat], newColumn, positionWhereToInsert] // Transpose
I would be glad to learn about a more elegant method.
