picoCTF - asm1 writeup
Description
What does asm1(0x6fa) return? Submit the flag as a hexadecimal value (starting with ‘0x’). NOTE: Your submission for this question will NOT be in the normal flag format.
Category: Reverse Engineering
Points: 200
Source Code
We are given the x86 assembly instructions for a function called asm1. I added comments to get a better sense of what this function is doing:
# ebp+0x8 stores function argument no. 1
# not sure what value is in ebp+0x8
asm1:
<+0>: push ebp ; set up the stack frame
<+1>: mov ebp,esp
<+3>: cmp DWORD PTR [ebp+0x8],0x3a2 ; cmp with 930
<+10>: jg 0x512 <asm1+37>
<+12>: cmp DWORD PTR [ebp+0x8],0x358 ; cmp with 856
<+19>: jne 0x50a <asm1+29>
<+21>: mov eax,DWORD PTR [ebp+0x8]
<+24>: add eax,0x12 ; eax += 18
<+27>: jmp 0x529 <asm1+60>
<+29>: mov eax,DWORD PTR [ebp+0x8]
<+32>: sub eax,0x12 ; eax -= 18
<+35>: jmp 0x529 <asm1+60>
<+37>: cmp DWORD PTR [ebp+0x8],0x6fa ; cmp with 1786
<+44>: jne 0x523 <asm1+54>
<+46>: mov eax,DWORD PTR [ebp+0x8]
<+49>: sub eax,0x12 ; eax -= 18
<+52>: jmp 0x529 <asm1+60>
<+54>: mov eax,DWORD PTR [ebp+0x8] ; eax = ebp+0x8
<+57>: add eax,0x12 ; eax += 18
<+60>: pop ebp ; clean up stack frame
<+61>: ret
My main hiccup for this problem was not knowing that value was in ebp+0x8. But after re-reading the challenge description it shows that the value 0x6fa (1786) is being passed as an argument.
Solution
After knowing what value to work with, find the output became trivial. I have included the code flow below:
asm1:
<+0>: push ebp ; set up the stack frame
<+1>: mov ebp,esp
<+3>: cmp DWORD PTR [ebp+0x8],0x3a2 ; cmp 1786 with 930
<+10>: jg 0x512 <asm1+37> ; branch taken
...
<+37>: cmp DWORD PTR [ebp+0x8],0x6fa ; cmp 1786 with 1786
<+44>: jne 0x523 <asm1+54> ; branch not taken
<+46>: mov eax,DWORD PTR [ebp+0x8] ; eax = 1786
<+49>: sub eax,0x12 ; eax = 1786 - 18 = 1768
<+52>: jmp 0x529 <asm1+60> ; unconditional jump
...
<+60>: pop ebp ; clean up stack frame
<+61>: ret ; return 1768 (0x6e8)
Flag: 0x6e8