Group Abstract Group Abstract

Message Boards Message Boards

Developing Chinese crypto in the Voxelverse.

Posted 4 years ago

enter image description here

You know you're a developer when your various projects converge into one coherent whole, and that is when the code base begins to exceed the scope of one blog post. Here we will synthesize three dimensional generalization of the Autoglyphs with recent developments in practice of Chinese Trigram design. A complete working example is available on the cloud, so we will not repeat all code here. Yes, we are heading toward a coding competition for New Year Water Tiger 2022. First we need to give some background, prove a concept, and show how basic encrypt / decrypt functions work. Raw voxel assets are available on github.

If you have already studied linear recurrences extensivley on OEIS, it probably is okay to skip Tanja's series on Feedback Shift Registers and get straight to Symmetric Crypto. Eventually we are going to try and decrypt a stream cipher (good luck), but for now we have to define Encrypt and Decrypt functions, which make the Autoglyphs more useful than mere works of art. The Autoglyphs are actually patterns and not entirely random outputs. We ask the question if pattern and symmetry allows the possibility that maybe some Eve can obtain decodes (without taking secret keys and brute forcing)?

Let's start with a simple import of $8$ pads, each a 3D Autoglyph (after sorting by symmetry):

AGTests = Map[Partition[#, 64] &,
     Partition[Flatten[Map[ToExpression, Characters /@ StringSplit[Import[
     "https://raw.githubusercontent.com/bradklee/CryptoAssets/master/ExampleAG3Ds/AG3DT"<>ToString[#]<>".txt"],"\n"]]], 64*64]] & /@ Range[8]; 

These pads are definitely not secure, because the corresponding secret keys are already known and listed online:

CharacteristicSeeds = {
   1068947993658975122157353753616942840246806141024,
   875695379343193424823445348694680652740201185481,
   495672220319428784155078550997997180172599139322,
   92962701827533250346487117218119002306342033068,
   807669444641696051849891548387372924883122769942,
   58854672879382211919039871200182349957445942893,
   207752697509440412973534330689716944749507327427,
   751209551927174784144601640564624173088665128391};
Mod[Floor[{#, #/2, #/4}], 2] & /@ CharacteristicSeeds;

One of the chosen AGs has full Octahedral symmetry:

Show[Trigram@AGTests[[-1]], ImageSize -> 600]

AG8

Nothing hidden inside so far, but it's difficult to tell. The main question to ask is whether or not the pattern is stochastic enough to make an effective mask for potentially sensitive information? Maybe not, in which case some intrepid crypto cracker could actually win the subsequent contest while playing fair. Other than overall symmetry, notice that $32 \times 32 \times 32$ blocks on each corner have transpose symmetry on each facet. This is an important property when understanding the minimal encoding. An example follows.

Now we get the first example data set, and depict one of the encryptions:

data = Map[Partition[#, 32] &, Partition[Flatten[
Map[ToExpression, Characters /@ StringSplit[ Import[
          "https://raw.githubusercontent.com/bradklee/CryptoAssets/master/ExCrypt1/CryptogramT"           
      <>ToString[#] <> ".txt"],
          "\n"]]], 32*32]] & /@ Range[24]; 

Trigram@data[[1]]

trigram data 1

For minimum encodings we only use one of eight corners from an AG to pad a $32 \times 32 \times 32$ signal. If you're following the discussion, you may be thinking why $32$ rather than $16$? We will get there, but first a brute force attack over length $8$ seed space:

TableForm[ Partition[Trigram@MinDecode[#, data[[1]]] & /@ CharacteristicSeeds, 4]]

brute force

So okay, we got lucky with a small key space and hit a decode on index $4$. I think the plots look cool, but some people would rather just read the numbers and save time:

Total[Flatten[MinDecode[#, data[[1]]]]] & /@ CharacteristicSeeds
{6639, 5980, 6815, 2441, 6490, 6101, 6739, 6689}

Due to sparseness of readable outputs, the weight of the decode is noticeably less than other fail cases, again index $4$. We can do similar for subsequent cryptograms and get the decrypt index map. Here we go with a few plots:

encryptions = data[[{1, 3, 6}]];
NotExactlySecretKeys = CharacteristicSeeds[[{4, 3, 7}]];
EncryptedTrigrams = Show[Trigram[#], ImageSize -> 200] & /@ encryptions;
decodes = MapThread[MinDecode, {NotExactlySecretKeys, encryptions}];
PublicEncrypted = CubeEncode /@ decodes;

RGBBWResults /@ PublicEncrypted[[1;;2]]

pe1

pe2

RGBBWResults /@ decodes[[1;;2]]

de1

de2

So what's going on with the intermediary result? In order to preserve transpose symmetry we need a permutation step from $16\times 16\times 16 \rightarrow 32\times 32\times 32 $. Essentially this is an extra, necessary encryption, with a public key:

(* Loc2Locs is invertible, thus a Permutation Public Key *)

Loc2Locs[loc_, 1] := 2 loc - # & /@ {{1, 1, 1}, {0, 0, 0}}

Loc2Locs[loc_, 3] := With[{oddPos = Position[loc,
      Complement[loc, Commonest[loc]][[1]]][[1, 1]]},
  SortBy[2 loc - # & /@ {
     ReplacePart[Table[0, {i, 3}], oddPos -> 1],
     ReplacePart[Table[1, {i, 3}], oddPos -> 0],
     {0, 0, 0}}, Total[#] &]]

Loc2Locs[loc_, 6] :=  SortBy[2 loc - # & /@
   Join[IdentityMatrix[3], 1 - IdentityMatrix[3]],
  {Total[#], Sort[Permutations[#]][[1]]} &]

Once we get to $32\times 32\times 32 $, then to finish encryption, we just add the pad mod 5. We could actually have skipped the permutation layer by using a $16\times 16\times 16 $ pad, but the first example is just a means to the second. Here's another mixed-up dataset to get and try to straighten out:

SymCryptograms = Map[Partition[#, 64] &,
     Partition[Flatten[Map[ToExpression, Characters /@ StringSplit[Import[
     "https://raw.githubusercontent.com/bradklee/CryptoAssets/master/ExCrypt2/SymCryptogram"<>ToString[#]<>".txt"],"\n"]]], 64*64]] & /@ Range[8]; 

decsTest = SymDecode[SymCryptograms[[1]], #] & /@ AGTests;
Length /@ decsTest
Column[RGBBWResults /@ decsTest[[4]]]
Out[]={4, 4, 4, 2, 4, 4, 4, 2}

double decode

Basically, the fourth type of Autoglyph has $2\times D_2$ dihedral symmetry, so we were able to embed two characters rather one. The reason for not breaking symmetry is to keep up appearances the that encrypted file could actually still be an Autoglyph 3D:

Show[Trigram@SymCryptograms[[1]], ImageSize -> 800]

sym crypt 1

The symmetry is obviously preserved, with only one plane of reflection through edge midpoints ( $xy$ if $z$ is up). One important question is whether or not you can find a statistical test that says such a graph is not just an Autoglyph in 3D? If yes, this could potentially lead to a decryption algorithm for actually discovering secret keys, or at least the secret pads. Let's see one more encrypt / decrypt pair, of course, this time with Octahedral symmetry (as in the animation above):

decsTest = SymDecode[SymCryptograms[[3]], #] & /@ AGTests;
Length /@ decsTest
Show[Trigram@SymCryptograms[[3]], ImageSize -> 800]
Column[RGBBWResults /@ decsTest[[8]]]

Out[]={2, 2, 2, 2, 2, 2, 2, 1}

 octahedral encode

octahedral decode

Okay, that's all the hints I can give you for now. If you are practicing for the upcoming new years challenge, please complete index maps for the first and second dataset. You can even post them here if you want. We will return shortly with more encrypted data and rules for the competition New Year Water Tiger 2022.

POSTED BY: Brad Klee
5 Replies
Posted 4 years ago
POSTED BY: Brad Klee
Posted 4 years ago
POSTED BY: Brad Klee
Posted 4 years ago

Cipher Text

Anim. An Autoglyph Cipher Text.

Who was Horst Feistel? A war child, born in Berlin, Germany 1915, refugee in USA 1934, later a Cryptographer in Cambgridge, MA, and a physicist at Harvard and MIT, not an enemy! Since that all checks out, we can move forward with a (hint: German language) ciphertext for the holidays, but first a note about protocol.

Encode outputs are well suited for UDP+TLS ~ DTLS. What does that mean in human readable language, and why does that matter in present circumstances? Well, relative to the previously mentioned zoom fail case, an interesting article came up recently on the signal blog: How to build large-scale end-to-end encrypted group video calls. RTP is usually built on top of UDP, notice these acronyms share the P for Packet. The signal article is mainly about syncing similar data streams. However, Signal is gaining a reputation as best in charitable crypto practice. Yes, they use Edward's curves, see also Dariaa Porechna: Elliptic Curve integrated encryption scheme. In fact, Signal's use of Double Ratchet is stronger than the simple-minded method for key streaming we use here. That's okay, because we aren't really working with secretive data. However, Daniel Bernstein's "Analysis" of AES should really convince anyone the dangers of reusing pads or secret keys.

The following computation shows encrypt / decrypt for a longer message. Adapting for real time would lead to many more difficulties, including channel thinning, stream interruptions, and lost keys. If each packet is chosen as an image frame, instead of convenient $\tt MapThread$, we would need an open loop computing new keys, symmetry types, and encrypts. Additionally, the basic encrypt call takes about $0.25s$. That may not be fast enough in practice. We're not trying to compete with Signal here, were just trying to expand on the idea of "Crypto Christmas Cards". So here's the code:

Choose Key0

SecretKeys = Complement[ ToExpression /@ 
    StringSplit[Import["~/OpenAssets/Seeds/AutoGlyphSeeds.txt"], "\n"],
   ToExpression /@  StringSplit[Import[
      "~/OpenAssets/Seeds/AutglyphSeeds.txt"], "\n"]];

SeedRandom["CipherText"];
KEY0 = RandomSample[SecretKeys, 1][[1]];
Length@DeleteCases[SecretKeys, KEY0]
Out[]=255

Create Ciphertext Signals

VerseData = ToUpperCase[
   Characters[StringReplace[CipherText, " " -> "+"]]];

VerseBitmaps = BinToBitmap[Binarize[Rasterize[
       Style[#, 16, Bold, FontFamily -> "Times"]
       ], .65], 16] & /@ VerseData;
Partition[Image[Mod[# + 1, 2], ImageSize -> 64
      ] & /@ VerseBitmaps, 8] // TableForm;
Union[Dimensions /@ VerseBitmaps];

Sig32s = SquareEncode[Plus[#[[1]], 2 #[[2]]]] & /@ 
   Partition[VerseBitmaps, 2];
Image[# /. RGBRep, ImageSize -> 128] & /@ Sig32s;

Prepare Inputs

AsymKeyStream[key_, len_] := With[
  {max = Mod[Floor[key GoldenRatio^#], 2^160] & /@ Range[0, len - 1]},
  {Part[max, 1 ;; Length[#]], Prepend[#, 0]} &@Select[Accumulate[
     ReplaceAll[Mod[Floor[{#, #/2}], 2] & /@ max,
      {{1, 1} -> 1, {_, _} -> 2}]], # <= len &]]

KeyStream = AsymKeyStream[KEY0, 60];
SymStream = Mod[Floor[{#, #/2}], 2] & /@ KeyStream[[1]];
SigPairs = Partition[KeyStream[[2]], 2, 1] /. {x_, y_} :> Sig32s[[{x + 1, y}]];

Time Test

TimeTest = AbsoluteTiming[Table[MapThread[AbsoluteTiming[
        SymEncode[Sequence @@ #1, #2, #3, 4]][[1]] &,
     {SigPairs, KeyStream[[1]], SymStream}],
    {i, 10}]];

Mean[Mean[TimeTest[[2]]]]
Mean[StandardDeviation[TimeTest[[2]]]]
Out[]=0.25s
Out[]=6ms

(* This looks sufficiently random *)
ListPlot[Mean[TimeTest[[2]]] - Mean[Mean[TimeTest[[2]]]], 
 PlotRange -> { -6 Mean[StandardDeviation[TimeTest[[2]]]],
   6 Mean[StandardDeviation[TimeTest[[2]]]]}]

timing

Encode and Compose Frames

(*see output at top*)
EncodeDat =   MapThread[
   SymEncode[Sequence @@ #1, #2, #3, 4] &, {SigPairs, KeyStream[[1]], 
    SymStream}];
ListAnimate[Show[Im256[#], ImageSize -> 512] & /@ EncodeDat];

Decode in 8s

AbsoluteTiming[ DecodeDat = MapThread[
    ResultsBWLin[SquareDecode /@ SymDecode[#1, #2, #3, 4]] &,
    {EncodeDat, KeyStream[[1]], SymStream}];]
DecodeDat2 = Flatten[MapThread[Part[#1, 1 ;; #2] &,
    {DecodeDat, ReplaceAll[SymStream, {{1, 1} -> 2, {_, _} -> 4}]}], 1];
GraphicsGrid[Partition[DecodeDat2, 8], ImageSize -> 64 8]

stille nacht

Not exactly the same as it was during the Christmas Truce of 1914. The algorithm is slow and weak enough not to need export control, but can anyone recover the secret key? Or at least produce an algorithm which decodes the cipher text without knowing the secret keys?

EDIT Dec. 21. "Happy Holidays" Wrapper for outputs.

Other threads on Christmas Cards (from Silvia and from Stephen) have set a standard requiring a little more effort, so here's another call to data at github:

im = Import[
   "https://raw.githubusercontent.com/bradklee/CryptoAssets/master/ExCrypt3/CCC007.png"];

Show[im, ImageSize -> 4 ImageDimensions[im]]

Happy Holidays!

The set has 5 different messages, in $208$ different autoglyphs. Sorry, no square symmetry for these, but do let me know if you manage to obtain a decode!

POSTED BY: Brad Klee
Posted 4 years ago

The previous "encryption" wouldn't even be acceptable for an undergrad at TU/e, so we have to go back to the video series, and watch Number 5, on Block Ciphers. This lecture gives us a good idea how we can get around the "ECB Penguin problem" by using a Feistel Cipher (not a Steifel~Stifle cipher!). Again, we big up the MWE to the cloud, and refer here only to the output functions.

The basic idea is to make secret key encryption multilayered. We happen to have two actively symmetric channels, so Feistel Ciphering is a natural choice. The flow chart on wikipedia is a little more detailed than in Tanja's lecture, but our approach is slightly different. By splitting function F into two parts, and by allowing both addition and subtraction mod 5, we can write a fully symmetric iterator. First we need a way to stream keys, starting from a chosen key. This is not entirely trivial because any function through key space can land on a different symmetry type, but fortunately symmetry types are evenly distributed in keyspace.

ItKeys[key_, nInd_, sym_] :=  Map[With[
{cand = Select[Mod[Floor[key Pi^# + Range[8]], 2^160],
       SameQ[sym, Mod[{#, Floor[#/2]}, 2]] &]}, 
    If[EvenQ[nInd], Max[cand], Min[cand]]] &, Range[0, nInd - 1]]

So we just jump forward by a factor about $\pi^n$ and look in a window of size $8$, always finding two keys of each symmetry type and just choose one per $n$. Nothing to it, but perhaps not the most elusive way to choose a key stream.

Once the keys are known, we can use them to generate autoglyphs as the Feistel cipher iterates. We ususally can take two pads $P_1, P_2$ from each $n^{th}$ autoglyph, then compute out signals $S_1'=S_1 \oplus S_2 \oplus P_1$ and $S_2' = S_1 \ominus S_2 \oplus P_2$ modulo $5$ from in signals $S_1$ and $S_2$. Both of these equations are invertible, $S_1 = (S_1'\oplus S_2'\ominus P_1\ominus P_2 )/2$ and $S_2 = (S_1'\ominus S_2'\ominus P_1\oplus P_2 )/2$. In code it looks like this:

(*left & right get different pads / / sort of.*)
EncodeRound[sig1_, sig2_, AGmat_, sym_] := If[SameQ[sig1, sig2],
  Mod[{#, #}, 5] &@(sig1 + AGmat[[1 ;; 32, 1 ;; 32]]),
  With[{cc = CopyCodes[sym][[2, 1]]}, Mod[
    {Plus[Mod[sig1 + sig2, 5], AGmat[[1 ;; 32, 1 ;; 32]]],
     Plus[Mod[sig1 - sig2, 5], Switch[cc,
          {0, _}, Reverse /@ #, {1, 0}, Reverse@#, {x, x}, #
          ][[1 ;; 32, 1 ;; 32]] &@AGmat]}, 5]]]

Div2 = MapThread[Rule, {Mod[2 Range[0, 4], 5], Range[0, 4]}];

DecodeRound[sig1_, sig2_, AGmat_, sym_] := If[SameQ[sig1, sig2],
  Mod[{#, #}, 5] &@(sig1 - AGmat[[1 ;; 32, 1 ;; 32]]),
  With[{cc = CopyCodes[sym][[2, 1]]}, ReplaceAll[
      Mod[{#[[1]] + #[[2]], #[[1]] - #[[2]]}, 5], Div2] &@
    Mod[{Subtract[sig1, AGmat[[1 ;; 32, 1 ;; 32]]],
      Subtract[sig2, Switch[cc,
           {0, _}, Reverse /@ #, {1, 0}, Reverse@#, {x, x}, #
           ][[1 ;; 32, 1 ;; 32]] &@AGmat]}, 5]]]

(with some care taken for division by 2 mod 5). In the case with full square dihedral symmetry, we do not have two equivalent channels, so it is not a true Feistel cipher. However, we still get multiple autoglyph layers, which seem to produce an acceptable result as long as recursion depth is chosen greater than $2$:

Sig32s = SquareEncode /@ BigramMatrices;
Grid[Transpose[Outer[Im256@SymEncode[
      Sig32s[[3]], Sig32s[[3]], CharacteristicSeeds[[#1]],
      {1, 1}, #2] &, {7, 8}, {0, 1, 2, 3}]], Frame -> All]

OO encodes

Grid[Transpose[Outer[Show[ResultsBW[{SquareDecode[#]}] & /@ SymDecode[
       SymEncode[Sig32s[[3]], Sig32s[[3]],
        CharacteristicSeeds[[#1]], ref[[#1]], #2],
       CharacteristicSeeds[[#1]], ref[[#1]], 0],
     ImageSize -> 128] &, {7, 8}, Range[0, 4]]], Frame -> All]

OO decodes

The second row with only one layer of padding looks very similar to what we saw previously. As the stream goes on, output encodes become more dense with color, and corresponding decodes look like random binary fields. That is what we want.

It's not that difficult to run up four layers, certainly in less than 1s per encrypt call:

AbsoluteTiming[encodes = MapThread[SymEncode[Sig32s[[1]], Sig32s[[2]], #1, #2, 4] &,
    {CharacteristicSeeds[[1 ;; 6]], ref[[1 ;; 6]]}];]
Out[]=l.t. 0.5 s

Grid[Transpose[Partition[Im256 /@ encodes, 2]], Frame -> All]

LOVE encodes

What do the decodes look like? Unless all keys in the stream are known the decodes look random:

Grid[Function[{recd}, MapThread[
    ResultsBW[SquareDecode /@ SymDecode[#1, #2, #3, recd]] &,
    {encodes, CharacteristicSeeds[[1 ;; 6]], ref[[1 ;; 6]]}]] /@ 
  Range[0, 4], Frame -> All]

love decodes

That's it! At least Naive decryption doesn't work, but obviously our minimal example isn't as complex or secure as AES: Advanced Encryption Standard. What is really missing (?) is an analysis whether or not stacking autoglyphs has a dense, pseudorandom limit. We can explore this possibility just by plotting a stream across $n$ and its first integral:

AGStream = AutoGlyphM2D /@ ItKeys[CharacteristicSeeds[[5]], 5, ref[[5]]];
Grid[Transpose[{Im256 /@ AGStream,
   Im256 /@ FoldList[Mod[Plus[#1, #2], 5] &, AGStream]}
  ], Frame -> All]

ag stream

The patterns seem different enough from one another, and end up producing something dense and intricate, but we don't know if it's approaching a desirable limit without taking Tanja's suggestion to apply diehard tests. In this case we don't really care, because We aren't trying to turn Eve into a Rudolph joke just in time for the holidays. If Eve wants to print off a Crypto Christmas Card and give it to herself, she's more than welcome to. Just please make an artistic choice of your own preferred secret key value and recursion depth.

POSTED BY: Brad Klee
Posted 4 years ago
POSTED BY: Brad Klee
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard