Message Boards Message Boards

raspiSFV - Creating/Testing file hash codes

Trott files downloaded from http://extras.springer.com

Introduction

This is my attempt of a first tiny app ("Hello World!"). The main motivation behind the coding was to make me apply hands-on some of the contents learned from the superficial Introduction to Control Objects. I wanted to experience if the language is really(?) suitable for building a small software, think of the thousands of FREE primitive yet handy utilities which one can download from the internet like unfreez or file renamer. My 'program' is to compute file hash codes like "CRC-32" in a user-friendly manner, i.e. with a click of a button. The real question is how an experienced WL coder would program such an app with similar functionality and user interface. I am a beginner, let me share how far i got. I am a beginner, let me share how far i got. Note that I adopted the code formatting convention introduced by Boris.

Check files through SFV

Simple File Verification (SFV) is a file format for storing CRC32 checksums of files to verify the integrity of files. The following code imports the *.SFV-file from the HDD as text string, listing the lines which the text is made of. From these text lines, only the relevant lines are selected, i.e. lines which contain the file name and its file checksum (also called file hash code or file hash value). SFV files created by other SFV utilities may contain comments and other irrelevant text. A crc32 code consists of 8 hexadecimal chars, a SHA-512 code consists of 128 hex chars. The hex chars can be in upper or lower case, without difference in meaning.

rellines := Select[
   Import[sfvpath, {"Text", "Lines"}]
   , StringMatchQ[#
     , WordCharacter ~~ RegularExpression[".+\\s[[:xdigit:]]{8,128}"] ~~ EndOfLine] &
  ];

The following code separates the relevant text lines into a list of file names and a list of their hash codes. One could use parallel assignment with Set[ ] instead of SetDelayed[ ] to write it more concisely but we really need the latter here.

pairs := Transpose[Flatten[
    Map[
     StringCases[#
       , fn : (StartOfString ~~ __) ~~ WhitespaceCharacter ~~ 
         hs : HexadecimalCharacter .. ~~ EndOfString :> {fn, ToLowerCase@hs}] &
     , rellines]
    , 1]
  ];
fnames := pairs[[1]];
fhashs := pairs[[2]];

With the help of the file names we are able to create their full paths on the computer, assuming that the SFV file is in the same directory (folder) as the files in question.

flist := Map[
   FileNameJoin[{
      FileNameDrop[sfvpath, -1]
      , #
     }] &
  , fnames];

The core functionality of the app is the computation of the file hash value. In Wolfram L we have the function Hash[ ] and FileHash[ ]. The latter is just what we need. By using MapIndexed[ ], we could track the mapping progress through the variable n.

hashes[files_List] :=
  MapIndexed[
    {FileHash[#, htype, All, "HexString"], n = First@#2} &
    , files
   ][[All, 1]];

Create your own SFV file

If the user wants to create an SFV file, he can choose to have the list of hash codes in lower case chars or capital letters/numbers. A checkbox will provide this setting in the graphical user interface (GUI). By default, the Mathematica hash functions output hash values in lower case chars.

yourhashed := If[
   ! lowcase
   , ToUpperCase[hashes[filespath]]
   , hashes[filespath]
  ];

Q: How do we extract the file names from a list of file paths? A: By mapping the function FileNameTake[ ] on the list, with -1 as the second argument.

yourfils := FileNameTake[#, -1] & /@ filespath;

We have the file names in a list, we have the file hashes in another list. With StringJoin[ ] we can join the two strings and do so for all the files. This recreates the official format of an SFV file, its relevant content, plus some comment/information for the end consumer.

out := Join[
    {";", "; NOTE: The hash codes in this SFV file are of type " <> htype, ";", ""}
    , #1 <> " " <> #2 & @@@ Transpose[{yourfils, yourhashed}]
    , {""}
   ];

We want the utility to handle not only CRC32 codes but also all other supported hash codes in Wolfram L. Currently twelve hash code types are supported. Many many more types do exist and are handled by third-party FREE software tools, but these twelve are among the more common ones.

panel = Panel[
   Grid[{
     {RadioButtonBar[Dynamic[htype]
       , {"CRC32" -> Tooltip[Style["CRC32", Bold], "32-bit cyclic redundancy check", TooltipDelay -> Automatic]
        , "Adler32" -> Tooltip["Adler32", "Adler 32-bit cyclic redundancy check", TooltipDelay -> Automatic]
       }, Appearance -> "Vertical"]
      , RadioButtonBar[Dynamic[htype]
       , {"MD5" -> Tooltip["MD5", "128-bit MD5 code", TooltipDelay -> Automatic]
        , "MD4" -> Tooltip["MD4", "128-bit MD4 code", TooltipDelay -> Automatic]
        , "MD2" -> Tooltip["MD2", "128-bit MD2 code", TooltipDelay -> Automatic]
       }, Appearance -> "Vertical"]
      , RadioButtonBar[Dynamic[htype]
       , {"RIPEMD160" -> Tooltip["RIPEMD-160", "160-bit RIPEMD code", TooltipDelay -> Automatic]
        , "RIPEMD160SHA256" -> Tooltip["RIPEMD-256", "RIPEMD-160 following SHA-256 (as used in Bitcoin)", TooltipDelay -> Automatic]
       }, Appearance -> "Vertical"]
      , RadioButtonBar[Dynamic[htype]
       , {"SHA" -> Tooltip["SHA-1", "160-bit SHA-1 code", TooltipDelay -> Automatic]
        , "SHA256" -> Tooltip["SHA-256", "256-bit SHA code", TooltipDelay -> Automatic]
        , "SHA256SHA256" -> Tooltip["SHA-256 Base64", "double SHA-256 code (as used in Bitcoin)", TooltipDelay -> Automatic]
        , "SHA384" -> Tooltip["SHA-384", "384-bit SHA code", TooltipDelay -> Automatic]
        , "SHA512" -> Tooltip["SHA-512", "512-bit SHA code", TooltipDelay -> Automatic]
       }, Appearance -> "Vertical"]
     }
    }, Alignment -> Top, Background -> Lighter[Yellow, 0.8]
   ]
  ];

With the above definitions we are ready to put the pieces together and bring life into the GUI. The GUI consists of one single TabView[ ] statement. It is basically a "one-liner", if you will. But … "Boy, That Escalated Quickly!"

CreateDialog[
 TabView[{
   "Check SFV" -> Column[{
      panel
      , Row[{
         FileNameSetter[Dynamic[sfvpath], "Open", {"SFV files" -> {"*.sfv"}}, WindowTitle -> "Find and open SFV file"]
         , Style[Dynamic[sfvpath], Small]
        }]
      , Button["Check SFV"
       , Column[{
          Style["The status of the SFV file is: ", 18, Green]
          , And @@ Equal @@@ Transpose[Sort /@ {fhashs, hashes[flist]}]
          }] // MessageDialog
       ; n = 0
       , Tooltip -> "Click to check your files against the loaded SFV file"
       , TooltipDelay -> Automatic
       , Method -> "Queued"]
      , Row[{
         ProgressIndicator[Dynamic[n], {0, Dynamic@Length[flist // Quiet]}]
        , StringForm[" processing\[Ellipsis]``/``", Dynamic[n], Dynamic@Length[flist // Quiet]]
       }]
     }]
   , "Create SFV" -> Column[{
      panel
      , Row[{
         Checkbox[Dynamic[lowcase]]
         , Style[" Use lowercase hash values", Small, FontFamily -> "Arial"]
        }]
      , Row[{
         FileNameSetter[Dynamic[filespath], "OpenList", {"All files" -> {"*"}}, WindowTitle -> "Select files to be included in SFV file"]
         , Style[Dynamic@TableForm[filespath], Tiny]
        }]
      , Row[{
         FileNameSetter[Dynamic[sfvpath], "Save", {"SFV files" -> {"*.sfv"}}, WindowTitle -> "Specify SFV file name and saving location"]
         , Style[Dynamic[sfvpath], Small]
        }]
      , Button["Export to SFV file"
       , Export[sfvpath, out, "Text"]
       ; Column[{
          Style["SFV file successfully created at: ", 18, Green]
          , sfvpath
         }] // MessageDialog
       ; n = 0
       , Tooltip -> "Click to create SFV file in your specified saving location"
       , TooltipDelay -> Automatic
       , Method -> "Queued"]
      , Row[{
         ProgressIndicator[Dynamic[n], {0, Dynamic@Length[filespath]}]
         , StringForm[" processing\[Ellipsis]``/``", Dynamic[n], Dynamic@Length[filespath]]
        }]
      }]
   , "About" -> Grid[{
       {}
       , {Style["raspiSFV v1.0", Italic, FontFamily -> "MV Boli"]}
       , {Panel@FlipView[{
            Import["ExampleData/spikey.tiff"]
            , Import["https://abload.de/img/avatar_moose150x150vys5e.png"]
           }] }
       , {Style["©2018 Wolfram L", Italic, FontFamily -> "MV Boli"]}
      }, ItemSize -> 30]
   }], WindowTitle -> "raspiSFV \[LongDash] \"Hello World!\""
]

This pretty much wraps it up. I made very basic use of Dynamic[ ], and all my attempts to wrap the whole set of code lines with DynamicModule[ ] or Module[ ] would give me new problems to deal with. The progress indicator will not work with the default preemptive method of the button; using method "Queued" was very crucial here.

Conclusion

As both experts and beginners should be able to see, this program is very primitive but already looking complex due to the necessary nested placement of interface elements. So much complexity (already), even though the whole program revolves around one single function, namely FileHash[ ], are you aware? With this in mind, a more experienced programmer could hopefully come up with a better GUI-implementation for an SFV software tool — just for the sake of showing/teaching how it's done better, in a more exemplary manner.

Anyone up for the challenge?

Attachments:
POSTED BY: Raspi Rascal

I tried to build this little thingy only for my own educational purposes, to learn what it's like to compose a simple custom user interface from scratch out of the blue, how much effort it takes to place the elements (their alignments) exactly where i want them to be. It was just an exercise for practicing new vocab items. In fact, i used so many functions which i never used before but just grabbed from search hits in the WLDC, which i find amazing. I started learning Wolfram L in April and i am not talented, ambitious, nor productive .. so it really amazes me what can be done even with little code/effort/practice.

I deliberately chose to program such a tiny thing, so that readers of this thread can grasp right away the principle of my implementation and what's awkward about it. I wished that i could present a more robust program with more bells and whistles, production-quality stuff, working on all OS-platforms, all versions, and also in the Cloud.

The code works without glitches on my Win7 system but produces error messages on Raspbian. I certainly have an idea where/what to look for in case of debugging. Maybe i should not have called it "raspisfv" after all, haha.

Could at least any other Windows user confirm that the program works without glitches on their system (Windows)?

POSTED BY: Raspi Rascal
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract