Group Abstract Group Abstract

Message Boards Message Boards

22
|
67K Views
|
169 Replies
|
248 Total Likes
View groups...
Share
Share this post:

[WSG22] Daily Study Group: Cryptography (begins January 31!)

POSTED BY: Dariia Porechna
169 Replies
Posted 3 years ago
Attachments:
POSTED BY: Doug Beveridge

Looking forward to this!

POSTED BY: Jamie Peterson

Here is my more "formal" analysis of this, assuming that the first word should be in Mma's Latin wordlist.

text = "yhql ylgl ylfl";
alpha = Alphabet[];
latinWords = WordList["KnownWords", Language -> "Latin"];
candidates = {StringReplace[text, 
      Thread[Rule[alpha, RotateLeft[alpha, #]]]]} & /@ Range[26];
MemberQ[latinWords, StringSplit[#][[1, 1]]] & /@ candidates;
Pick[candidates, %]

Update: Changed an old "A" to "alpha". Thanks, Luis!

It helped to notice that it was three words of four characters that start with the same character (and a lot of "L"). It cannot be "alea iacta est" or "aut Caesar aut nihil" for that reason. :-)

Though I did a brute force to confirm it.

Posted 3 years ago

YHQL YLGL YLFL = VENI VIDI VICI

"I came, I saw, I conquered"

POSTED BY: Luis Marin

For anyone interested in seeing actual historical equipment, the NSA runs the National Cryptological Museum in Maryland. It has an amazing collection from ancient times through Cold War to modern times including Enigmas and they have the only surviving Bombe (the machine designed at Bletchley Park to decrypt Enigma -- the British destroyed all of theirs and had to rebuild one for their recently opened Bletchley Park museum).

POSTED BY: Neil Singer

Over 200 attendees joined the first session of this study group today. Thanks @Dariia Porechna for your presentation. Thanks to the attendees for remaining engaged throughout the session. Looking forward to the rest of the series.

Update on the course quizzes: We are seeing many people who have passed Quiz 1 and a few who have already passed all the quizzes. Keep up the impressive work! We are investigating a couple problem reports from emails received at wolfram-u@wolfram.com. Thank you for sending detailed problem reports when you encounter them.

To clarify the video logging, this is a feature of the course, and videos watched outside of the course (in the Study Group) are not tracked with your course progress. We do have attendance and recording views logged for the Study Group, and this, along with quiz results, allows us to issue certificates of program completion for the Study Group. The Introduction to Cryptography Study Group certificate of completion is considered equivalent to the course completion certificate, and we're pleased to offer both options for earning a certificate for this material.

POSTED BY: Jamie Peterson
Posted 3 years ago
POSTED BY: Brad Klee
POSTED BY: Colten Jackson
Attachments:

Here is a notebook exploring if some of WL's simpler hash functions generates hash collisions for some of WL's wordlists.

This "CirclePlus" operation, defined as the addition of two elements (x and y) modulo 5, can be interpreted as the additive law that, together with the set of elements G = {0,1,2,3,4}, provide a representation for the cyclic group Z/5Z (see the explanations that follow in the notebook).

Furthermore, there exists a generator element "g", that is such that, when being "summed" (with the CirclePlus law) together with itself repeatedly, can generate all the elements of the group. The diagram that follows is the so-called "cycle graph" for this group Z/5G : it basically gives a pictorial representation of the statement about the existence of the generator element from above. Each edge in that graph, represents one iteration of such sum with the generator.

Corrected after reading Hermès' post and Wolfram documentation x(+)y is by default interpreted as CirclePlus[x,y]. It can be entered as [CirclePlus] or esc c+ esc. In the mentioned context it is the group operation, modular addition, such that z = x (+)y belongs to the same group as x and y.

In[26]:= CirclePlus[#, #] & /@ G

Out[26]= {0, 2, 4, 1, 3}

Modulo 5 in the definition of (+) in our case guarantees that the outcomes do not leave the "group" of 5 elements.

(Note: The book by Stinson is from 1995, not 1999 as is stated in the first paragraph.)

(Notebook attached)

Attachments:

Given the number of times Claude Shannon's name has come up during the study group, I thought that SG participants might want to learn more about this amazing man. I thought he was brilliant before joining the study group and my admiration has only increased over the past two weeks.

A Mind at Play: How Claude Shannon Invented the Information Age Kindle Edition by Jimmy Soni (Author), Rob Goodman (Author) In their second collaboration, biographers Jimmy Soni and Rob Goodman present the story of Claude Shannon—one of the foremost intellects of the twentieth century and the architect of the Information Age, whose insights stand behind every computer built, email sent, video streamed, and webpage loaded. Claude Shannon was a groundbreaking polymath, a brilliant tinkerer, and a digital pioneer. He constructed the first wearable computer, outfoxed Vegas casinos, and built juggling robots. He also wrote the seminal text of the digital revolution, which has been called “the Magna Carta of the Information Age.” In this elegantly written, exhaustively researched biography, Soni and Goodman reveal Claude Shannon’s full story for the first time. With unique access to Shannon’s family and friends, A Mind at Play brings this singular innovator and always playful genius to life.

POSTED BY: Joseph Smith
Posted 3 years ago

Also thanks for the proof around 30min. It is quite nice and easy, and lends itself well to a discussion of "confusion" from your suggested article. Here is another notation using Fold:

FeistelForm[kn_] := {#[[2]], Xor[#[[1]], F[#[[2]], kn]]} &
FoldList[#2@#1 &, {L, R}, {
    FeistelForm[k1], FeistelForm[k2], FeistelForm[k3], Reverse,
    FeistelForm[k3], FeistelForm[k2], FeistelForm[k1], Reverse
    }] ;
SameQ[%[[1]], %[[-1]]]
Out[] = True

Let's try and make this a little easier for Physicists (or former) by assuming that $F$ is just addition, then the naive form reduces to linear algebra:

NaiveFeistelForm[kn_] := {{0, 1}, {1, 1}} . # + {0, F[kn]} &
FoldList[#2@#1 &, {L, R}, {
    NaiveFeistelForm[k1], NaiveFeistelForm[k2], NaiveFeistelForm[k3], 
    Reverse,
    NaiveFeistelForm[k3], NaiveFeistelForm[k2], NaiveFeistelForm[k1], 
    Reverse
    }] /. {_Integer?EvenQ -> 0, _Integer?OddQ -> 1};
SameQ[%[[1]], %[[-1]]]
Out[]=True

What happens here is basically a nesting $g \cdot( \ldots( g\cdot(g\cdot s + p_1) + p_2 ) \ldots ) + p_n $, and it's easy to see if distributive holds we get $g^n \cdot s +P$ where $s$ is signal, $P$ a summed pad, and mixing matrix $g$ satisfies $g^3=Id$. This is the same problem I mentioned previously, where Feistel scheme essentially reduces to a one time pad. However if the pair of operations $(\cdot,+)$ do not allow distributive property, then the nested form above might actually be a decent characterization of what is meant (loosely) by "Feistel Scheme".

There is one alternative, applying the pad before the mixing: $g \cdot( \ldots( g\cdot(g\cdot (s + p_1) + p_2 ) \ldots + p_n) $. In this case distributive does not hold if $g\cdot (s + p_1) \neq g\cdot s +g\cdot p_1 $, which is what we want to fix the previous algorithm. Start by generating test data:

SeedRandom["confusion"];
randPad = RandomInteger[{0, 4}, 32];
randMessage = RandomInteger[{0, 4}, 32];

And weakly prove (in terms of test data) distributive holds in the previous case:

SameQ[ Flatten[ Mod[ForwardMix . Partition[Plus[randPad, randMessage], 8], 5]],
 Flatten[Mod[Plus[ ForwardMix . Partition[randPad, 8],
    ForwardMix . Partition[randMessage, 8]], 5]]] 
Out[]=True

This is what we don't want, so we need to add new Addition and Subtraction functions (the idea is taken from Dariia's suggested reference above):

BoxSubtract[dat1_, dat2_, part_, base_] :=With[{ints = (FromDigits[#, base] & /@ 
        Partition[#, part]) & /@ {dat1, dat2}}, Flatten[
   IntegerDigits[#, base, part] & /@ Mod[Subtract @@ ints, base^part]]]

BoxPlus[dat1_, dat2_, part_, base_] :=  With[{ints = (FromDigits[#, base] & /@ 
        Partition[#, part]) & /@ {dat1, dat2}}, Flatten[
   IntegerDigits[#, base, part] & /@ Mod[Plus @@ ints, base^part]]]

(* test inverse property *)
SameQ[BoxSubtract[BoxPlus[randPad, randMessage, 8, 5], randPad, 8, 5], randMessage] 
Out[]=True

Again here is a proof that distributive doesn't hold:

(* not distributive *)
SameQ[  Flatten[ Mod[ForwardMix . Partition[BoxPlus[randPad, randMessage], 8], 5]],
 Flatten[Mod[BoxPlus[  ForwardMix . Partition[randPad, 8],
    ForwardMix . Partition[randMessage, 8]], 5]]]
Out[]=False

Then our Encode / Decode functions need only minor modification, replacing Plus and Subtract for BoxPlus and BoxSubtract:

EncodeRound[sig_, AGmat_] := Flatten[
    Mod[ForwardMix . Partition[BoxPlus[sig, AGmat, 8, 5], 8], 5]]

DecodeRound[sig_, AGmat_] := Mod[BoxSubtract[ReplaceAll[
        Mod[Flatten[ReverseMix . Partition[sig, 8]], 5], Div4], AGmat,
    8, 5], 5]

Using these updated functions, we can produce another set of encrypts:

encrypts 2

These are possibly better than the previous set since we are doing a little more than just padding, but the run statistics really don't look any better. I am still skeptical--What exactly do we gain by requiring non-distributive property? Is one data set easier to decrypt than another?

I think that the non-distributive property must be an important defining part of what makes a Feistel Cipher, but I don't yet completely understand the theory why. Anyone else, agree / disagree? Thoughts?

POSTED BY: Brad Klee

Specify "HexString" as 3rd argument to Hash, by default it produces an integer.

POSTED BY: Dariia Porechna

I tried to set up a UTF-8/UTF-16 conversion code in WL, see attached notebook (for UTF-16 it is also important to know what the underlying endianness of the byte stream is). Hope that it helps! Suggestions for improvements are welcome :) maybe at some point this could be turned into an entry into the function repository? (I haven't seen any existing UTF16-related function there).

Using these functions, the UTF16-LE example of Mr. Beveridge would look as follows:

In[1]:= Hash[UnicodeToUTF16LE[ToCharacterCode["password"], "ByteArray"], "MD4", "HexString"]
Out[1]= 8846f7eaee8fb117ad06bdd830b7586c

This agrees with the Python result.

That's neat! Thank you for taking on the "assignment" responsibly! You could also try some more complex tests like https://resources.wolframcloud.com/FunctionRepository/resources/ChiSquareRandomnessTest/ and others without having to implement them

POSTED BY: Dariia Porechna
Posted 3 years ago
POSTED BY: Donnacha Gayer
Posted 3 years ago

This group might be interested in a post a made a few months ago Twitter steganography.

The interesting use for steganography may be to send encrypted information attached to Twitter photos. As a communication channel it might be very hard to monitor and notice when encrypted information was actually being transmitted.

POSTED BY: Robert Rimmer

What you see in In[1] is due to the fact that in Wolfram an empty byte array is actually a list:

In[1]:= ByteArray[{}]

Out[1]= {}

In[2]:= ByteArray[{}] == List[]

Out[2]= True

So since in WL you can encrypt any expression not just bytes or strings this gets treated as expression List[] and serialized as a generic expression. If you really want to encrypt an empty thing try empty string "". What you see in the inputs 3, 4, and 5 is related to a standard convention on padding, which will be covered in lesson "Block and stream ciphers next week" so I hope you wouldn't mind waiting for that.

POSTED BY: Dariia Porechna
Posted 3 years ago

Nice, just a corretion in the code (A -> alpha)

Thread[Rule[alpha, RotateLeft[alpha, #]]]]} & /@ Range[26];

Update: Code is already corrected in original post.

POSTED BY: Luis Marin

The "Spy Museum" in Washington DC is also a good place to see an Enigma machine:

https://www.spymuseum.org/exhibition-experiences/about-the-collection/collection-highlights/four-rotor-enigma-machine/

POSTED BY: Gustavo Delfino

You can restrict the value of x via an extra constraint:

Solve[{Mod[9 x, 23] == 1, 0 <= x < 23}, x, Integers]
Reduce[{Mod[9 x, 23] == 1, 0 <= x < 23}, x, Integers]

{{x -> 18}}
x == 18

Using the Integers option give some answers:

Solve[Mod[9 x, 23, x] == 1, x, Integers]
Reduce[Mod[9 x, 23, x] == 1, x, Integers]

{{x -> -5}}
x == -5

And:

Solve[Mod[9 x, 23] == 1, x, Integers]

{{x -> ConditionalExpression[
    18 + 23 ConditionalExpression[1, \[Placeholder]], 
    ConditionalExpression[1, \[Placeholder]] \[Element] Integers]}}

I did not check Doug's suggestion (first goes first :) ) but Stinson's book mentioned by Hakan looks impressive. Its 1995 edition is available at https://ia803405.us.archive.org/19/items/pdfy-eAEdqcELZKUUU733/Cryptography%20Theory%20And%20Practice%20-%20Douglas%20Stinson.pdf. Apparently, it is close if not identical (Hakan, pls comment if you can) to the aforementioned 1999 edition.

Thanks! The daily study group is very interesting and informative.

POSTED BY: Joseph Smith

If you are interested in Feistel-like ciphers but not limited to xor as a main operation, you could look at Lai–Massey scheme (IDEA cipher, for instance) starting page 21 https://www.research-collection.ethz.ch/bitstream/handle/20.500.11850/140723/eth-38650-02.pdf

POSTED BY: Dariia Porechna

Nice!

Did you look at NIST's "A Statistical Test Suite for Random and Pseudorandom Number Generators for Cryptographic Applications" (https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-22r1a.pdf) that Dariia mentioned in the session about Randomness/Pseudorandomness? It contains a Fourier related test (section "2.6 Discrete Fourier Transform (Spectral) Test") and a lot of other statistical tests. Though I don't find anything about your other test, the QuantilePlot Analysis.

The notebook "09_Randomness and pseudorandomness.nb" has some other recommended documents as well (in the Further Reading section)..

Posted 3 years ago

I have been looking at how to create a quality metric for various ciphers and came up with a couple of tests. I'm sure there is some standard "cookbook" of tests that get applied but haven't seen such a thing yet. Have a look at my take on this. It considers frequency and statistics to compare the spiky image from lesson 13.

Attachments:
POSTED BY: Mark Ross

Good catch! Thank you! I will fix it in the course notebook!

POSTED BY: Dariia Porechna
Posted 3 years ago

Note that this is not a very serious crypto analysis of hash collisions since we know that these hash functions are not secure (and some of them are not designed to be crypto secure).

As you also mentioned above, Adler32 is not a serious hash function. It is easy to study from first principles and find how to design a collision algorithm. This was done on a previous thread "Hello World = Goodbye World: chosen-prefix collision attack" (editors note: we have since learned that the word "attack" is better replaced by "analysis").

I did not put very much thought or effort into the above, so it would be interesting to see who can improve these results and how. If anyone has a good MD5 collider, that might make a good contribution to Wolfram Function Repository. Happy searching!

POSTED BY: Brad Klee
Posted 3 years ago
POSTED BY: lara wag
Posted 3 years ago
POSTED BY: Richard Hewens
Posted 3 years ago
POSTED BY: Brad Klee
Table[ResourceFunction["CaesarCipher"]["yhql ylgl ylfl", i], {i, 1, 26}] 

shows the decrypted message in item 23.

POSTED BY: Arturo Silva
POSTED BY: Dariia Porechna

I was also curious about the lesser-known Lorenz machine: https://de.wikipedia.org/wiki/Liste_der_Exponate_des_Lorenz-Schl%C3%BCssel-Zusatzes Not a lot of them still present!

POSTED BY: Andreas Rudolph

Thanks very much. I need to think this over before I am confident in understanding it.

POSTED BY: Joseph Smith
Posted 3 years ago
POSTED BY: Doug Beveridge

It's (the field of) Integers (with 's'), and this will be the same as Z.

Great find, the elliptic curves notebook by Jouko Teeriaho, Doug!

POSTED BY: Dariia Porechna

At the bottom of the course page, the last entry is "Track My Progress." When clicked, it lists two conditions for successful completion of the course:

    "Earn your certificate of course completion by watching 25 course videos and ..."

However, it gives me, and I guess others, ZERO credit for watching the videos during Dariia's lectures. I assume this feature is meant for the future course takers, not for the current cohort. Please clarify.

POSTED BY: Zbigniew Kabala

Thank you! Will check it out. Best. M

Good to know! Thank you, Doug.

Posted 3 years ago

I took the first quiz and there is some problem with the evaluation system. I scored 0/14. Attempted again and the same result. Then for one of the question, I picked one answer , submit the result and observe the behavior, and repeated it three more times. No matter what option is selected as an answer, score is always 0

POSTED BY: Mitesh Pandey
Posted 3 years ago

ok, so really to use zero to 25 and then use the mod 25 is the best policy...I should have thought to try that. Thanks Hermes, I will recalculate and we can continue. Great conversation here!

POSTED BY: Sam Fergis

It appears that when they encoded the message and the key, before doing the "encryption" step, they just offsetted by the character code of "a", i.e. offsetted everything in such a way that the letter 'a' gets coded to 0, 'b' to 1, etc., and then after the encryption step, to go back from the new character codes to letters, they did the reverse offset. They didn't add an extra offset of "1" to the character code of "a", in order to make 'a' coded to 1, 'b' to 2, etc.

Posted 3 years ago

Is it not that you must select the correct output type

eg

Hash["abcdefg", "Adler32"]

182125245

Hash["abcdefg", "Adler32", "HexString"]

0adb02bd

POSTED BY: Doug Beveridge

Nice analysis! Some pretty common ways to assess quality of ciphers are also described he https://www.engr.mun.ca/~howard/PAPERS/ldc_tutorial.pdf

POSTED BY: Dariia Porechna

Question for the community: Should hash values calculated with the same algorithms but on different platforms agree?

I tried comparing the hash digest for a simple text string ("abcdefg") calculated with Mathematica and calculated with an online calculator https://www.fileformat.info/tool/hash.htm . The notebook is attached. I tried a few different hash algorithms but the hash digests did not agree with the Mathematica digest.

Attachments:
POSTED BY: Joseph Smith
Posted 3 years ago
POSTED BY: Doug Beveridge
Posted 3 years ago

Questions for Dariia (or other experts): Is there a list where WFR covers diehard tests, and are we missing any? What does it really mean if an encrypt fails diehard? I'm thinking that a combination of steganography with cryptography could fail diehard and still be decently hard metal. For example, did anyone decode the fake NFTs that I made for the "happy new years" contest?

POSTED BY: Brad Klee
POSTED BY: Dariia Porechna
Posted 3 years ago

Hi

Can you use UTF-16 encoding with the Hash function in Mathematica?

In Python :-

enter image description here

POSTED BY: Doug Beveridge
Posted 3 years ago

A longer list. DictionaryWordQ has more words than WordList.

WordList[] // 
AssociationMap[ResourceFunction["CaesarCipher"][#, 3] &] // 
Select[DictionaryWordQ]
POSTED BY: Rohit Namjoshi

Thanks, Luis. I've changed it in my post.

The code is based on the one in the "04Classicalciphers.nb", but in the last minute changed "A" to "alpha" since it's not recommended that variables start with an uppercase. But forgot to change it everywhere. :-)

Posted 3 years ago

Ahead of the presentation, I am hopeful that the explanation of RSA will include an examination of its usage in SSL, i.e., how is it employed to exchange the symmetric key?

POSTED BY: MURL DONLEY
Posted 3 years ago

Interesting historical perspective of cryptography from day 1 - looking forward to the next sessions!

POSTED BY: Updating Name
Posted 3 years ago

The bitcoin elliptical curve Secp256k1 has an order which is a prime number . What is the advantage of having a prime number order (if any ) and I assume no subgroups ?

POSTED BY: Doug Beveridge
POSTED BY: Joseph Smith
Posted 3 years ago
POSTED BY: Doug Beveridge

The question is being edited. Thanks to Doug and Hakan for their very helpful comments.

Posted 3 years ago

Thanks , updated

POSTED BY: Doug Beveridge
Posted 3 years ago
POSTED BY: Doug Beveridge
Posted 3 years ago

Sorry , error , i have updated , the problem remains the same

POSTED BY: Doug Beveridge

Solve[Mod[9 x,23,x]==1,x]

Please note, that the 3-argument form of Mod means, according to the documentation,

Mod[m, n, d] uses an offset d.

Posted 3 years ago

Is there a way to Solve Mod[9 x, 23, x] == 1 algebraically with Mathematica ?

POSTED BY: Doug Beveridge

I did what you suggested but did not work, I am still awaiting contact from the email I sent to wolfram-u@wolfram.com

Thanks, Doug. Great links, very "hand-on"! Best. M

Hi, Michael.

Ah, I see I misspelled the year. I have the first edition from 1995. Thanks for the pointer: I'll see how I can correct that in my notebook.

I like the book even if it's quite old (over 25 years), especially the first chapters which describes some of the classical ciphers together with cryptanalyses/attacks on them together with small instructive examples. And now I'm preparing for tomorrow's lesson on RSA. :-)

There are later versions of the book. The latest is - according to Amazon - a fourth edition from 2018: https://www.amazon.com/Cryptography-Theory-Practice-Textbooks-Mathematics/dp/1138197017/ref=sr_1_1?keywords=cryptography+stinson&qid=1644788501&sprefix=stinson+crypt%2Caps%2C183&sr=8-1

I confirm this problem: Taking the first quiz today and got 0 scores as well. The quiz frame also errors out with the following errors: enter image description here

(It also shows : v1.20220128 )

It's always fun to hear about these kind of discussions, especially about how a function should be named or which parameters / their orders should be, in order to be the most general and not have to be changed in the very near future of its introduction.

Posted 3 years ago
POSTED BY: Sam Fergis
Posted 3 years ago
POSTED BY: Brad Klee

Thanks, Dariia!

I'll check out those more complex tests. Though I do like to implement stuff. :-)

Wolfram does not support UTF16 encoding unfortunately

POSTED BY: Dariia Porechna

Brad, thanks for the link of your analysis. I'll read it more carefully and also the "SHA-1 is a Shambles" article you cited (https://sha-mbles.github.io/ ).

Posted 3 years ago

Three years of Latin in high school helped confirm my decryption and translation. Good fun!

POSTED BY: Robert Lyons

As a question/suggestion: isn't it common to define the term "information security system" for the set of properties i.e. confidentiality, authentication, data integrity, and non-repudiation.

I came, saw and won.

Posted 3 years ago

You're welcome, I just notice the "A" out of place and tested the code. :-)

POSTED BY: Luis Marin
Posted 3 years ago

As Caesar encrypted with a shift of 3, shifting by 23 will decrypt CaesarCipher["yhqlylglylfl", 23]

Producing a famous quote from Caesars Gallic Wars.

POSTED BY: MURL DONLEY
Posted 3 years ago

please make test bigger on Zoom lecture now

POSTED BY: Peter Aptaker
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard