HackIM 2019 0bfusc8 much

I failed to solve this during CTF. Got good looking flag but it was wrong. Next morning woke up, fixed one character and it was correct. I am also n00b on RE so there probably is better solution.

In short, we are given a binary that asks flag string, runs a lot of obfuscated code and prints out 0 for wrong flag and 1 for correct one. There is only main function
but it has 1.8MB of assembly code and 152kB of stack variables so it is quite clear that reversing manually would take a long time.

Out of ideas, had to try something. So I wrote Frida script to dump stack memory after execution in hope that I could see some changes when trying different flag strings.
But looking that huge dump of numbers with WinMerge was pain. Last idea that came to mind was to calculate count of zero bytes in that dump. Had an idea that if
program advances right direction, it would generate more non zero bytes.

So I had Frida script to dump stack after execution. Modified it to append count of zeros and flag string to file. Next wrote C# code to enumerate last character
of my flag string and call Frida script with that. After guessing all possible characters at the end of string, C# takes output file, sorts results by count of zeros and
takes character that has least zeros on stack. With that I got result like this: hackim19{Get_Hard____Dynamic_Reeersing_AAAP_braceeee_e_e__y_u_traceeee}. So clearly
this is working, but there are some errors. With wrong characters, code gives probability of characters so it it possible to guess right one. Like here:


So right character would be S but code gets A. Stack zero count differs only by one. So we can guess that AAAP is actually ASAP.

The failure
After all coding and guessing finally got flag: hackim19{Get_Hard_In_Dynamic_Reversing_ASAP_brace_me_then_you_trace_me} but it was WRONG!
Wrote code to iterate each character, try all possible characters is ASCII table to replace it. No flag.
Next morning realized I had accidentally fed wrong string to code. There were now two wrong characters. With correct string finally got:
But it was few hours too late. On no…


Here is Frida script. Download here. Edit: there may be some mistakes in pointer addition operations. Please use ptr(somever).add(123) form. ptr(somevar+123) may cause surprises if somevar is string.

IDA view at beginning of main.

C# code to run things. Download here.