I think guard digit retention will be a problem.

For an example we will start with a number in the ballpark you describe. All variants below seem to be retain an excess of guard digits. They are in fact (mostly) zero bits, but, alas, those still take up memory.

pi50 = N[10^(-4000)*Pi, 50];
pi50 // InputForm
(* Out[353]= 3.1415926535897932384626433832795028841971693993751*10^-4000
Out[354]//InputForm=
3.14159265358979323846264338327950288419716939937510582097494459231`50.*^-4000 *)

I'll cut back to 20 digits. The 17 is because we work here in hex and 17 hex digits exceeds 20 decimal digits.

{m, e} = MantissaExponent[pi50, 16];
pi20 = SetPrecision[Round[m, 16^(-17)], 20]*16^e
InputForm[pi20]
(* Out[358]= 3.1415926535897932384*10^-4000
Out[359]//InputForm=
3.1415926535897932384427420953578946468715017823`20.*^-4000 *)

So we got the desired 20 digits but a slew of guard digits came along for the ride. Here is method 2 and we see the same happening.

pi20b = 2^SetPrecision[Round[Log[2, pi50], 16^(-20)], 24]
InputForm[pi20b]
(* Out[360]= 3.1415926535897932385*10^-4000
Out[361]//InputForm=
3.1415926535897932384626434215219292373261840846`20.0357783006888*^-4000 *)

Method 3 uses an internal function to extract bits. The resulting mantissa remains about the same length as before due to guard digits.

{s, m, m2, e} = NumericalMath`$NumberBits[pi50];
keep = 68;
pi20c = N[FromDigits[m[[1 ;; keep]], 2], 20]*2^(e - keep)
InputForm[pi20c]
(* Out[364]= 3.1415926535897932385*10^-4000
Out[365]//InputForm=
3.1415926535897932384538450330644203682569164055`20.*^-4000 *)

Method 4.

rd = RealDigits[pi50, 16];
keep = 17;
rd2 = {rd[[1, 1 ;; keep]], rd[[2]]};
pi20d = N[FromDigits[rd2[[1]], 16]*16^(-keep + rd2[[2]]), 20]
InputForm[pi20d]
(* Out[374]= 3.1415926535897932384*10^-4000
Out[375]//InputForm=
3.1415926535897932384427420953578946468715017823`20.*^-4000 *)

And method 5.

pi20str = ResourceFunction["RealToHexString"][pi50,{15,20}]
pi20e = ResourceFunction["HexStringToReal"][pi20str, {15, 20}]
InputForm[pi20e]
(* Out[18]= "0c18ead7b2f602e3da88"
Out[19]= 3.1415926535897932384*10^-4000
Out[20]//InputForm=
3.1415926535897932384427420953578946468715017823`20.*^-4000 *)

Some days you just don't get a break.

Related to Gianluca Gorni's response, you can round to a certain decimal place and then reset precision like so.

pi50 = N[Pi, 50];
InputForm[pi20 = SetPrecision[Round[pi50, 10^(-20)], 20]]
(* Out[72]//InputForm= 3.14159265358979323846`20. *)

This cuts down on the retained guard digits.