That is quite curious. I only used:
encoded = Encrypt["abc", "This is a test."]
for my example and it seems to work fine. As you see I also used Decrypt in my program. I ran your program and can now reproduce that. It looks a bit funny indeed. It even "cracks" your complicated password with four simple three letter combinations.
If you run
chars = CharacterRange["a", "z"];
keys = DeleteDuplicates[
Flatten[Table[a <> b <> c, {a, chars}, {b, chars}, {c, chars}]]];
encoded = ToFileName[$TemporaryDirectory, "encoded"];
Encode["~/Desktop/test.m", encoded, "abc"]
Map[If[Get[encoded, #] === $Failed, , Print[#]] &, keys];
it recovers the password but finds three additional ones that work. I am not sure whether this is the intended behaviour.
Cheers,
Marco