The document describes the author's process for solving two challenges presented in malware binaries - one for Windows and one for Linux. For the Windows binary, the author used IDA Pro and Ollydbg to analyze the code and determine that the valid PIN was 5 characters long. An AutoIT script was created to brute force all possibilities, finding the PIN "13044" within 30 minutes. The Linux binary challenge was initially avoided due to time constraints but was analyzed the next day using GDB after the author's internet was down.

1. MY
SOLUTION
TO
MALWARE.LU’S
HACKGYVER
CHALLENGES
Well,
its
been
a
long
time
since
I
tried
reversing
something
&
when
I
saw
a
tweet
announcing
this
challenge,
I
decided
to
give
it
a
shot.
As
it
turns
out,
the
package
for
download
contained
2
binaries,
one
for
Windows
platform
&
the
other
one
for
Linux.
I’ll
note
down
my
solutions
for
both
quickly.
WINDOWS
BINARY
Well,
this
was
an
interesting
one.
As
usual,
I
scanned
it
with
PEiD
&
it
said
“Nothing
Found”.
Am
used
to
accepting
that
as
a
good
omen
by
now
;)
.
So
next,
on
executing
it,
this
is
what
it
looks
like:
A
simple
looking
Win32
application.
Thing
to
note,
it
accepts
just
one
input.
Great,
so
moving
on,
when
I
tried
entering
some
random
value
for
PIN
&
clicked
the
“validate”
button,
the
above
message
changed
from
“Enter
your
PIN
code”
to
“Bad
PIN
code”,
as
expected.
Here,
we
have
two
options,
either
use
Ollydbg
which
happens
to
be
my
all
time
favourite
debugger
on
Windows
&
learn
as
we
go
along
or
open
it
up
in
IDA
Pro
&
study
it
first.
I
usually
prefer
to
mix
both.
But
for
this
one,
I
opened
it
up
in
IDA
Pro
first
&
after
some
quick
analysis,
found
out
that
although
the
above
input
box
can
take
about
8~9
characters
as
input,
in
the
code,
only
the
first
5
are
used
in
further
calculation.
Let
me
point
out
some
of
the
code
for
you.
Most
of
the
action
happens
at
the
subroutine
located
at
0x4013A0.
Open
it
up
in
IDA
&
you
can
clearly
see
the
whole
big
picture.
To
sum
up
all
the
important
events
happening
here,
this
should
suffice:
1. Get
input
using
GetWindowTextW.
2. After
some
initial
verification,
pass
this
data
to
the
subroutine
that
starts
at
0x401000.
3. The
above
routine
does
the
whole
validation
&
based
on
the
result,
displays
the
goodboy
or
badboy
message.
Seems
simple.
Next,
looking
into
the
code
at
0x401000,
I
realized
that
the
application
discards
any
input
that’s
less
than
or
greater
than
5chars
in
size.
In
short,
your
input
has
to
be
exactly
5
characters
in
size.
That’s
kinda
sweet.
;)
Here’s
the
code
that
does
this
check
:

2.
Confirmed
this
by
actually
running
the
app
&
monitoring
it
with
Ollydbg.
Scrolling
down
a
bit,
the
whole
code
was
messy.
Now,
from
the
tweet,
we
were
only
supposed
to
find
the
valid
PIN
for
this
challenge.
Hence,
like
I
always
do,
I
decided
to
try
the
simplest
approach
first
&
if
that
fails,
go
the
manual
way.
Its
for
times
like
these
that
I
have
AutoIT
installed
my
XP
Virtual
Machine.
Its
very
useful
to
put
together
a
quick
&
dirty
hack
when
it
comes
to
automating
things
at
GUI
level.
For
everything
else,
I
prefer
Perl.
So,
my
plan
of
action
here
was
to
put
together
a
simple
AutoIT
script
that’ll
bruteforce
all
possible
numeric
combinations
while
I
work
on
some
official
reports.
If
it
doesn’t
crack
it,
will
actually
sit
&
reverse
the
algorithm
&
find
the
correct
PIN.
AutoIT
script
was
made
in
a
jiffy
&
after
about
30minutes
of
running
it,
when
I
was
done
with
my
other
reports
&
stuff,
this
is
what
I
found:

3. So
that’s
how
it
was
done.
The
correct
PIN
was
“13044”.
Just
a
hunch
&
some
quick
coding
solved
it
for
me
&
frankly,
I
did
not
even
kill
any
of
my
grey
cells
on
this.
Stroke
of
luck.
It
could
have
been
much
worse
though.
For
example,
if
the
right
PIN
was
even
alphanumeric,
the
above
approach
would
have
failed
miserably
&
not
to
mention
I
would
have
had
to
actually
get
down
to
understanding
the
entire
algorithm.
But
anyways,
as
long
as
we
cracked
it,
its
all
good.
LINUX
BINARY
Ah
this
one
was
unexpected!
Frankly
I
was
in
no
mood
to
try
the
Linux
binary
for
various
reasons
ranging
from
loads
of
work
at
hand
to
almost
completely
have
forgotten
the
cryptic
gdb
commands
but
it
was
probably
destined
to
be
done.
Next
day
morning
my
internet
was
down
&
with
nothing
else
to
do,
I
thought
of
giving
this
one
a
try
anyways.
Lets
dive
straight
into
this
one,
shall
we?
To
begin
with,
I
opened
up
my
Ubuntu
VM
&
got
this
binary
on
my
desktop
folder.
First
thing
I
usually
do
with
Linux
crackmes/challenges
is
to
analyze
the
binary
with
these
commands:
strings,
strace,
ltrace
&
so
on.
Once
this
stage
is
over,
open
it
up
in
IDA
Pro
&
side
by
side
debug
using
gdb.
“strings”
command
was
the
first
one
I
tried
&
it
gave
these
interesting
strings
amongst
others:
• MD5_Final
• BIO_f_base64
• strcpy
• strcmp
• Well,
now
create
your
own
keygen
;)
• Bad
key
Interesting
things
to
note
here
are,
the
possibility
of
MD5
hashing
&
use
of
base64.
Okay,
moving
further,
I
tried
ltrace
&
this
is
what
happened:
Now
that’s
a
pretty
weird
output.
Maybe
it
takes
input
as
commandline
arguments.
So
the
next
thing
I
tried
was
to
run
the
same
program
with
an
argument.
The
output
I
got
from
it
is
there
on
the
following
page
&
its
very
interesting
for
various
reasons.
I’ve
highlighted
the
interesting
parts
in
the
output.

4. aodrulez@aodrulez-VirtualBox:~/Desktop$ ltrace ./hackgyverlnx
abcdefghijkl!__libc_start_main(0x80489a5, 2, 0xbfebde34, 0x8048aa0,
0x8048b10 <unfinished ...>!MD5_Init(0xbfebdc9c, 0, 38, 0xb742d96d, 0)
= 1!MD5_Update(0xbfebdc9c, 0xbfebf4db, 12, 0xb742d96d, 0)
= 1!MD5_Final(0xbfebdcfc, 0xbfebdc9c, 12, 0xb742d96d, 0)
= 1!sprintf("9F", "%02X", 0x9f)
= 2!sprintf("C9", "%02X", 0xc9)
= 2!sprintf("D6", "%02X", 0xd6)
= 2!sprintf("06", "%02X", 0x6)
= 2!sprintf("91", "%02X", 0x91)
= 2!sprintf("20", "%02X", 0x20)
= 2!sprintf("30", "%02X", 0x30)
= 2!sprintf("DC", "%02X", 0xdc)
= 2!sprintf("A8", "%02X", 0xa8)
= 2!sprintf("65", "%02X", 0x65)
= 2!sprintf("82", "%02X", 0x82)
= 2!sprintf("ED", "%02X", 0xed)
= 2!sprintf("62", "%02X", 0x62)
= 2!sprintf("59", "%02X", 0x59)
= 2!sprintf("5C", "%02X", 0x5c)
= 2!sprintf("F7", "%02X", 0xf7)
= 2!BIO_f_base64(0, 0, 16, 0x6d6c99f, 0)
= 0xb77441e0!BIO_new(0xb77441e0, 0, 16, 0x6d6c99f, 0)
= 0x92e0008!BIO_s_mem(0xb77441e0, 0, 16, 0x6d6c99f, 0)
= 0xb7743b40!BIO_new(0xb7743b40, 0, 16, 0x6d6c99f, 0)
= 0x92e0bb0!BIO_push(0x92e0008, 0x92e0bb0, 16, 0x6d6c99f, 0)
= 0x92e0008!BIO_write(0x92e0008, 0xbfebf4db, 9, 0x6d6c99f, 0)
= 9!BIO_ctrl(0x92e0008, 11, 0, 0, 0)
= 1!BIO_ctrl(0x92e0008, 115, 0, 0xbfebdd00, 0)
= 1!malloc(22)
= 0x092e0c20!strcpy(0x092e0c20, "abcdefghijkl")
= 0x092e0c20!strcpy(0x092e0c29, "YWJjZGVmZ2hpn")
= 0x092e0c29!BIO_free_all(0x92e0008, 0x92e0c08, 0, 0xbfebdd00,
0x92e0bf8) = 1!MD5_Init(0xbfebdc9c, 1,
0xbfebdd18, 0xb75da8da, 0) =
1!MD5_Update(0xbfebdc9c, 0x92e0c20, 21, 0xb75da8da, 0)
= 1!MD5_Final(0xbfebdcfc, 0xbfebdc9c, 21, 0xb75da8da, 0)
= 1!sprintf("03", "%02X", 0x3)
= 2!sprintf("6C", "%02X", 0x6c)
= 2!sprintf("B8", "%02X", 0xb8)
= 2!sprintf("ED", "%02X", 0xed)
= 2!sprintf("80", "%02X", 0x80)
= 2!sprintf("4D", "%02X", 0x4d)
= 2!sprintf("09", "%02X", 0x9)
= 2!sprintf("5F", "%02X", 0x5f)
= 2!sprintf("C7", "%02X", 0xc7)
= 2!sprintf("B8", "%02X", 0xb8)
= 2!sprintf("CF", "%02X", 0xcf)
= 2!sprintf("8D", "%02X", 0x8d)
= 2!sprintf("6C", "%02X", 0x6c)
= 2!sprintf("5C", "%02X", 0x5c)
= 2!sprintf("17", "%02X", 0x17)
= 2!sprintf("C5", "%02X", 0xc5)
= 2!strcmp("9FC9D606912030DCA86582ED62595CF7",
"036CB8ED804D095FC7B8CF8D6C5C17C5") = 1!puts("Bad
key"Bad key!)
= 8!+++ exited (status 8) +++!aodrulez@aodrulez-VirtualBox:~/Desktop$ !

5. So,
it
seems
like
the
program
takes
in
input
via
commandline
as
an
argument,
it
does
use
MD5
as
the
strings
that
are
being
compared
at
the
end
are
infact
MD5
hashes.
Its
also
possible
that
it
uses
base64
at
some
point.
Ohkies,
now
its
time
to
put
it
through
some
better
tools
&
analyze
the
program.
For
this,
I
chose
IDA
Pro.
Once
opened
up
in
IDA
Pro,
its
like
reading
a
book.
IDA
takes
us
straight
to
“main()”
function
&
as
you
can
clearly
see,
that’s
where
all
the
action
is.
I’ll
sum
up
the
code
in
main()
here
quickly:
1. Check
if
there
is
only
1
argument
passed
via
commandline.
Exit
if
this
fails.
2. Check
if
the
length
of
the
input
is
more
than
8
chars
or
else,
exit.
3. Runs
MD6
on
the
user
entered
string
&
saves
this.
4. Puts
the
user
entered
string
through
RC4_Encode
&
generates
a
new
string.
Calculates
the
MD6
hash
of
this
string.
5. Compares
the
strings
from
steps
3
&
4
&
if
they
are
the
same,
it’s
the
right
serial
or
else,
it
exits
with
a
badboy
message.
This
looks
simple
but
fishy.
On
further
inspection,
it
turns
out
that
its
infact
MD5
&
not
MD6
&
instead
of
RC4_encoding,
its
actually
base64
that’s
used.
Now
this
perfectly
matches
with
the
output
we
got
from
ltrace
earlier.
To
cut
short
the
story,
the
serial/key
is
checked
as
follows:
1. Input
has
to
be
more
than
8
characters
long.
(Ex.
“aodruleza”)
2. It
calculates
the
MD5
hash
of
this
string
&
saves
it.
(
“ABCA96374F97C3EC3B2D415470760401”
for
“aodruleza”)
3. The
first
9
characters
of
input
are
preserved
&
rest
are
discarded.
It
then
calculates
the
base64
encoding
of
this
string,
takes
the
first
12
characters
of
the
result
&
concatenates
with
the
user
entered
string
that
was
truncated
to
9
characters.
(
base64
encoding
of
string
“aodruleza”,
truncated
to
12
characters
is
“YW9kcnVsZXph”.
So,
the
final
string
after
step
3
turns
out
to
be
“aodrulezaYW9kcnVsZXph”)
4. Calculate
MD5
has
of
string
from
step
3
&
compare
it
with
the
hash
calculated
in
step
1.
If
they
match,
it’s
the
right
key
&
if
they
don’t,
it
displays
the
message
“Bad
key”.
So,
in
short,
a
valid
key
for
the
challenge
is
“aodrulezaYW9kcnVsZXph”.
Here’s
the
output
when
this
one
was
tried:
And
the
following
page
contains
code
of
the
ugly
keygen
that
I
made
for
it.
So,
that’s
all
guys.
It
was
indeed
interesting
&
I
got
to
brush
up
my
gdb
skills
after
a
long
time.
Hope
this
helped
someone.
Have
a
nice
day!
AODRULEZ