I've had to resort to a related trick when my contours contain singularities:
Notice that I don't use the Min in the ContourPlot argument, but set it with PlotRange.
p4 = ContourPlot[Exp[2 (-x^2 - y^2)], {x, -1, 1}, {y, -1, 1},
Contours -> Range[.1, 1, .1], PlotLegends -> Automatic,
ColorFunction -> "Temperature", PlotPoints -> 100,
ContourLabels -> All, PlotRange -> {0, 0.51},
ColorFunctionScaling -> False,
ClippingStyle -> ColorData["Temperature"][0.51]]

For example compare:
ContourPlot[1/(1 - Exp[2 (-x^2 - y^2)]), {x, -1, 1}, {y, -1, 1},
PlotLegends -> Automatic, ColorFunction -> "Temperature",
PlotPoints -> 100, ContourLabels -> All, PlotRange -> {0, 10}]
to
ContourPlot[1/(1 - Exp[2 (-x^2 - y^2)]), {x, -1, 1}, {y, -1, 1},
PlotLegends -> Automatic, ColorFunction -> "Temperature",
PlotPoints -> 100, ContourLabels -> All, PlotRange -> {0, 10},
ClippingStyle -> ColorData["Temperature"][1]]
