Implement HMACStream and HMACFile

[danielmarschall]

I want to suggest a combination of the following methods:

1. TDECHashAuthentication.HMAC combined with TDECHashExtended.CalcStream
2. TDECHashAuthentication.HMAC combined with TDECHashExtended.CalcFile

Background information: I want to encrypt a large file which does not fit into the memory, and I would like to do "Encrypt-then-HMAC". So I need to HMAC the whole file or stream. (Pre-hasing is not a solution, because it is insecure)

Here is the code I wrote (actually copied and modified) and tested.

type
  TDECHashExtendedAuthentication = class helper for TDECHashAuthentication
    class function HMACFile(const Key: TBytes; const FileName: string;
      const OnProgress:TDECProgressEvent = nil): TBytes;
    class function HMACStream(const Key: TBytes; const Stream: TStream; Size: Int64;
      const OnProgress:TDECProgressEvent = nil): TBytes;
  end;

{ TDECHashExtendedAuthentication }

class function TDECHashExtendedAuthentication.HMACFile(const Key: TBytes;
  const FileName: string; const OnProgress: TDECProgressEvent): TBytes;
var
  fs: TFileStream;
begin
  fs := TFileStream.Create(FileName, fmOpenRead);
  try
    HMACStream(Key, fs, fs.Size, OnProgress);
  finally
    FreeAndNil(fs);
  end;
end;

class function TDECHashExtendedAuthentication.HMACStream(const Key: TBytes;
  const Stream: TStream; Size: Int64;
  const OnProgress: TDECProgressEvent = nil): TBytes;
const
  CONST_UINT_OF_0x36 = $3636363636363636;
  CONST_UINT_OF_0x5C = $5C5C5C5C5C5C5C5C;
var
  HashInstance: TDECHashAuthentication;
  InnerKeyPad, OuterKeyPad: array of Byte;
  I, KeyLength, BlockSize, DigestLength: Integer;
begin
  // Taken from TDECHashAuthentication.HMAC and changed HashInstance.Calc to HashInstance.CalcStream for the message
  HashInstance := TDECHashAuthenticationClass(self).Create;
  try
    BlockSize    := HashInstance.BlockSize; // 64 for sha1, ...
    DigestLength := HashInstance.DigestSize;
    KeyLength    := Length(Key);

    SetLength(InnerKeyPad, BlockSize);
    SetLength(OuterKeyPad, BlockSize);

    I := 0;

    if KeyLength > BlockSize then
    begin
      Result    := HashInstance.CalcBytes(Key);
      KeyLength := DigestLength;
    end
    else
      Result := Key;

    while I <= KeyLength - SizeOf(NativeUInt) do
    begin
      PNativeUInt(@InnerKeyPad[I])^ := PNativeUInt(@Result[I])^ xor NativeUInt(CONST_UINT_OF_0x36);
      PNativeUInt(@OuterKeyPad[I])^ := PNativeUInt(@Result[I])^ xor NativeUInt(CONST_UINT_OF_0x5C);
      Inc(I, SizeOf(NativeUInt));
    end;

    while I < KeyLength do
    begin
      InnerKeyPad[I] := Result[I] xor $36;
      OuterKeyPad[I] := Result[I] xor $5C;
      Inc(I);
    end;

    while I <= BlockSize - SizeOf(NativeUInt) do
    begin
      PNativeUInt(@InnerKeyPad[I])^ := NativeUInt(CONST_UINT_OF_0x36);
      PNativeUInt(@OuterKeyPad[I])^ := NativeUInt(CONST_UINT_OF_0x5C);
      Inc(I, SizeOf(NativeUInt));
    end;

    while I < BlockSize do
    begin
      InnerKeyPad[I] := $36;
      OuterKeyPad[I] := $5C;
      Inc(I);
    end;

    HashInstance.Init;
    HashInstance.Calc(InnerKeyPad[0], BlockSize);
    if Size > 0 then
      TDECHashExtended(HashInstance).CalcStream(Stream, Size, OnProgress, false);
    HashInstance.Done;
    Result := HashInstance.DigestAsBytes;

    HashInstance.Init;
    HashInstance.Calc(OuterKeyPad[0], BlockSize);
    HashInstance.Calc(Result[0], DigestLength);
    HashInstance.Done;

    Result := HashInstance.DigestAsBytes;
  finally
    HashInstance.Free;
  end;
end;

[MHumm]

Thanks for the suggested methods! I'll try to look at those in a timely manner. But one request upfront: would you be able to provide at least a simplistic unit test for those? In case you don't know how to write a unit test just tell me and I'll teach you.

[danielmarschall]

Actually, I have never written a testcase for Delphi, so it would be great if you give me a few information about it. Thank you!

[MHumm]

Ok, let's start like this: I hope you have installed DUnit along with Delphi. If not add it by going to Tools/Manage features and add the "Unit testing" feature.

As next step open the DEC60 projetct group and there activate the DECDUnitTestSuite project.

[MHumm]

Expand the tests node and open TestDECHMAC.pas or TestDECHash.pas, not sure at the moment where it fits into. Search for the class where the test belongs to means, the test class for the class you added your methods above to (sorry, I'll leaving in a few minutes, thus just a few generic pointers at the moment). In that class add a new published method TestXYZ (select a proper name ;-) ). In this method you put the code performing your test into. Have a look at other such methods and you'll see that such methods always set up data first (if that's not already done in SetUp method, which is run prior to each run of any of the TestXXX methods). Then the test uses your method to do someting with the data and afterwards the result is checked via CheckEquals, CHeckGreater and similar methonds provided by the test framework.

[MHumm]

After you implemented your test method press F9 and you should see a GUI with a tree of all tests. You might want to untick all those nodes not applying to your test and then press F9 to run your test. if the test works as it should it will be green, otherwise it will be magenta telling you that the test failed and providing a failure message. Then you have to check if your method works as it should or if the test still has a failure... ;-)

If these instructions aren't good enough yet tell me and I'll guide you further.

[danielmarschall]

Thank you very much for the time you have taken to explain all this to me :-)

I have installed DUnit and am now testing it a little.

Just for fun I ran all tests, but I am surprised that I got hundreds of Exceptions of all kinds where the debugger stops. Is this normal? I guess the Exceptions are expected for the testcase, but I don't want that the compiler halts on every one, but I also don't want to exclude the Exception type, because my productive code needs to halt on these Exceptions.

Also, I wonder, is it normal that these testcases failed:

[MHumm]

The exceptions are expected (when run from the IDE, you can get rid of them when running this outside the IDE or using TestInsight IDE plugin from Stefan Glienke, which can run the tests in the background). DEC can throw exceptions, e.g. if you pass invalid parameters and the unit tests test whether they are actually raised.
What I normally do is to disable all of the exceptions originating from DEC (EDEC... exceptions) and let the RTL ones occur, as they might not be expected.