Hacktivity 2016: Stealthy, hypervisor based malware analysis

1. Stealthy,
Hypervisor-based
Malware Analysis
Tamas K Lengyel
tamas.lengyel@zentific.com
Sergej Proskurin
proskurin@sec.in.tum.de

2. #who
Tamas:
- Maintainer of Xen, LibVMI and DRAKVUF
- Co-Founder of Zentific
- Chapter lead of Malware Analytics at Scale at the
Honeynet Project
- PhD from UConn
Sergej:
- PhD Student at TUM Chair for IT Security
- Honeynet GSoC 2016

3. Agenda
1. Motivation
2. DRAKVUF behind the scenes
3. Xen behind the scenes
4. What’s next?

4. Stealth
Debuggers were not designed to be
stealthy
Debugged process can detect the
debugger
Observer effect

5. Strings in MultiPlug
$:hash:procexp.exe
$:hash:procmon.exe
$:hash:processmonitor.exe
$:hash:wireshark.exe
$:hash:fiddler.exe
$:hash:vmware.exe
$:hash:vmware-authd.exe
$:hash:windbg.exe
$:hash:ollydbg.exe
$:hash:winhex.exe
$:hash:processhacker.exe
$:hash:hiew32.exe
$:hash:vboxtray.exe
$:hash:vboxservice.exe
$:hash:vmwaretray.exe
$:hash:vmwareuser.exe

6. Some other popular strings
CheckRemoteDebugger
Present
IsDebuggerPresent
VIRTUALBOX
VBoxGuestAdditions
QEMU
Prod_VMware_Virtual_
XenVMM
MALTEST
TEQUILABOOMBOOM
VIRUS
MALWARE
SANDBOX
WinDbgFrameClass
SAMPLE
https://github.com/Yara-Rules/rules/blob/master/Antidebug_AntiVM/antidebug_antivm.yar

7. Improving Stealth #1
Move the monitoring component into the
kernel
Windows doesn’t like it if you just
randomly hook stuff (PatchGuard)
What about rootkits?

8. Rootkit problem 2015
http://www.mcafee.com/us/resources/reports/rp-quarterly-threats-aug-2015.pdf
That’s only about
0.36% of all
malware observed
by McAffee

9. Rootkit problem?
Rootkits are either not that big of a deal
Or are we just bad / getting worse at
detecting them?

10. Improving Stealth #2
Move the monitoring component into a
hypervisor
Harder to detect
Greater visibility
A lot easier said than done

11. 2014: DRAKVUF released
https://youtu.be/EZPXy314q3E

12. Lots of work behind the scenes
https://github.com/tklengyel/drakvuf

13. Complete rework in Xen

14. Are we done?
Nope
Malware can detect if it’s running in a
virtualized environment
Hypervisors were not designed to be
stealthy either

15. Pafish
https://github.com/a0rtega/pafish

16. CPUID hypervisor guest status
static inline int cpuid_hv_bit() {
int ecx;
__asm__ volatile("cpuid"
: "=c"(ecx)
: "a"(0x01));
return (ecx >> 31) & 0x1;
}

17. CPUID hypervisor guest status
cpuid =
['0x1:ecx=0xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx']

18. The fix verified

19. CPUID VM vendor IDs
Leaf 0x40000000
- EBX-EDX: XenVMMXenVMM
No way to override without recompiling
- Introduce CPUID events in Xen 4.8
- On-the-fly filtering of CPUID results
from dom0

21. 60GB free disk space?
LVM copy-on-write allows us to quickly
deploy lightweight duplicates
Analysis clones will only use extra space
if they change files
And only as much space as they actually
changed

22. The fix verified

23. Uptime check
int gensandbox_uptime() {
/* < ~12 minutes */
return GetTickCount() < 0xAFE74 ?
TRUE : FALSE;
}

24. Uptime check
Let your VM sit idle for a while, take
memory snapshot
Start each analysis clone by loading this
memory snapshot
Could also just return fake value

25. The fix verified

26. Memory size check
Who uses a machine with <1Gb RAM?
We can increase sandbox memory size
but that limits how many we can run
Xen memory sharing allows CoW!

27. CoW memory
https://tklengyel.com/nss2013-100.pdf

28. CoW memory over time
https://tklengyel.com/nss2013-100.pdf

29. Fun fact
Memory sharing based honeypots first
tested live at Hacktivity 2012!
Was really looking forward for those 1337
h4ck3rs on the public wifi!
Got nothing. Network is very nicely VLAN
isolated between clients (broadcast traffic
still got through)...

30. Xen memory-sharing status
It works but marked ‘experimental’
Fixes for Xen 4.8 to co-exist with other
‘experimental’ features
Memory sharing is known to open the
gates for cross-VM RowHammer attacks
For more details see:
https://www.usenix.org/system/files/conference/usenixsecurity16/sec16_paper_razavi.pdf

31. CPU count check

32. Multi-vCPU tracing
Particularly challenging due to how
external monitoring is implemented
Easy to end up in a race-condition with
concurrently active CPUs

33. DRAKVUF tracing in the beginning
1. Inject 0xCC into target function entry points
2. Mark pages Execute-only in the EPT
3. If anything tries to read the page
a. Remove 0xCC and mark page R/W/X
b. Singlestep
c. Place 0xCC back and mark page X-only
4. When 0xCC traps to Xen
a. Remove 0xCC
b. Singlestep
c. Place 0xCC back

34. EPT-lookup

35. EPT-lookup
All vCPUs share a single EPT
Standard way hypervisors use EPT

36. Race with multi-vCPU EPT
RACE

37. Using 0xCC is also racy
We have to remove 0xCC to allow
execution to continue
Another vCPU could fetch the instruction
just at that moment
We can potentially miss an event from
being logged

38. Some ways around
We can pause CPUs
We can emulate instructions
...or!

39. Xen alternate p2m (altp2m)

40. Xen altp2m
Introduced by Intel to support #VE and
VMFUNC
- Allow the guest to handle EPT faults without the
associated cost of a VMEXIT
- Allow the guest to switch around EPTs without
trapping into the hypervisor
- Also allows external tools to make use of
multiple tables

41. Xen altp2m
Also includes a pretty exotic feature
- GFN remapping
Similar to memory-sharing, but intra-VM
- Allow a GFN to point to a different MFN
https://blog.xenproject.org/2016/04/13/stealthy-monitoring-with-xen-altp2m

42. Xen altp2m GFN remapping

43. Normal mapping with EPT
Guest physical memory
Machine physical memory

44. DRAKVUF’s altp2m setup
Altp2m view 0
Used only when
singlestepping
Machine physical
memory

45. DRAKVUF’s altp2m setup
Altp2m view 1
Default during
execution
Machine physical
memory

46. DRAKVUF’s altp2m setup
Altp2m view 2
Used only when
singlestepping
Machine physical
memory

47. Xen altp2m exposure
By default the altp2m interface is guest
accessible
- Required for VMFUNC
- NOT required for DRAKVUF
DRAKVUF XSM policy
- Prohibit guest-access to altp2m
- Will be a lot easier on Xen 4.8

48. The fix verified

49. I/O activity? Time?
I/O can be relatively easily emulated
- TODO
RDTSC is trappable but..
- Hiding time-dilation from all possible
time-sources is likely not possible
- TODO

50. Detect virtualization vs DRAKVUF
Virtualization is now everywhere
- Not enough to detect if environment is virtual
- Likely not possible to hide all virtualization
artifacts anyway
Guest should not be able to detect
DRAKVUF!
- Stealth = indistinguishable from a regular VM

51. New: guest debug events
Malware is known to perform
self-debugging
- Prevents other debuggers to attach
- Can be used for stealth
Case in point:
https://blog.avast.com/2013/05/29/analysis-of-a-self-debugging-sirefef-cryptor
https://blog.malwarebytes.com/threat-analysis/2013/07/zeroaccess-anti-debug-
uses-debugger/
http://research.dissect.pe/docs/blackhat2012-paper.pdf

53. Tricky tricky breakpoints
0xCC can also be used by in-guest
debuggers
- These will also trap to DRAKVUF
- Need to be reinjected into the guest
- Not that big of a deal.. If you do it right..

54. What’s the length of 0xCC?
/*
* Injects a hardware/software CPU trap, to take effect the next time the HVM
* resumes.
*/
int xc_hvm_inject_trap(
xc_interface *xch, domid_t dom, int vcpu, uint32_t vector,
uint32_t type, uint32_t error_code, uint32_t insn_len,
uint64_t cr2);
Hint: 0xCC = 0b11001100

55. The obvious answer: 1
#define TRAP_int3 3
rc = xc_hvm_inject_trap(xch, domain_id, req.vcpu_id,
TRAP_int3, /* Vector 3 for INT3 */
HVMOP_TRAP_sw_exc, /* Trap type, here a software intr */
~0u, /* error code. ~0u means 'ignore' */
1, /* Instruction length. Xen INT3 events are
* exclusively specific to 0xCC with no operand,
* providing a guarantee that this is 1 byte only.
*/
0 /* cr2 need not be preserved */);

56. Correct answer: it depends
Intel® 64 and IA-32 Architectures Software Developer’s Manual

57. x86 instruction prefixes
Have absolutely no affect on 0xCC
- No sane debugger adds any for this reason
- You can use the same prefix multiple times
- The CPU just ignores them
- Except it changes the instruction length at VMEXIT…
Recommended read:
https://fgiesen.wordpress.com/2016/08/25/how-many-x86-instructions-are-there

58. What about Linux? And ARM?
ARM has virtualization extensions since
the Cortex A15
Some things are similar, some things are
not
Work in progress

60. The problems on ARM
altp2m only available on Intel systems
The ARM SLAT doesn’t have a concept
of Execute-only memory
- Memory has to be readable AND executable
No stealthy single-stepping
- No Monitor Trap Flag equivalent on ARM

61. Honeynet GSoC 2016
Porting Xen altp2m to ARM!
- 38 patches and counting
- Expected to land in Xen 4.9
- Some aspects of altp2m have been
revamped to better fit ARM
- Especially around TLB handling
https://github.com/sergej-proskurin/xen/tree/arm-altp2m-v4

62. Sneak peak into what’s next
Hiding shadow copies with R/X mapping
- Experiments with splitting the TLB on
ARM
- It works surprisingly well but there are
limitations and gotchas
Even more exotic altp2m setups
- TLB splitting vs TLB partitioning

63. Thanks!
Tamas K Lengyel
tamas.lengyel@zentific.com
@tklengyel
Sergej Proskurin
proskurin@sec.in.tum.de
Zentific https://zentific.com
DRAKVUF https://drakvuf.com

64. References
https://github.com/Yara-Rules/rules/blob/master/Antidebug_AntiVM/antidebug_antivm.yar
http://www.mcafee.com/us/resources/reports/rp-quarterly-threats-aug-2015.pdf
https://github.com/a0rtega/pafish
https://tklengyel.com/nss2013-100.pdf
https://fgiesen.wordpress.com/2016/08/25/how-many-x86-instructions-are-there
https://blog.xenproject.org/2016/04/13/stealthy-monitoring-with-xen-altp2m/
https://www.usenix.org/system/files/conference/usenixsecurity16/sec16_paper_razavi.pdf
https://blog.avast.com/2013/05/29/analysis-of-a-self-debugging-sirefef-cryptor
https://blog.malwarebytes.com/threat-analysis/2013/07/zeroaccess-anti-debug-uses-debu
gger/
http://research.dissect.pe/docs/blackhat2012-paper.pdf
https://github.com/sergej-proskurin/xen/tree/arm-altp2m-v4