As an inlustration for what I want:
For a small table, I can do it in the following manner:
Suppose I want 30 points,
{i,j}
i = RandomInteger[{1, N}, 30];
j = RandomInteger[{1, M}, 30];
points = Transpose[{i, j}];
Let M be my table, I can plot what I want brutally:
Show[ListPlot[{#}, PlotMarkers -> M[[#[[1]], #[[2]]]], PlotRange -> All] & /@ points]

But, if the required sample size increases (the limit being, say, the bare readablity of the plotted numbers), I am affraid that this isn't the most rational way.