This presentation gives a basic detail on kernel security and different approaches to improving it.

1. Analyzing Kernel Security
and
Approaches for Improving It
Milan Rajpara
IT Systems and Network Security
Gujarat Technological University
C DAC
Ahmedabad
Pune

2. Agenda
• Kernel Introduction
• Necessity for Kernel Security
• Kernel breach
• Analyzing Kernel Security
• Improving Approaches
• Future Work
Milan Rajpara
October 8, 2013
2

3. What is Kernel ?
• A computer program that manages
input/output requests from software
and translates them into data
processing instructions for the
central processing unit and other
electronic components of a
computer. [Wikipedia]
• The kernel is a fundamental part of a
modern computer's operating
system.
• OS rests on a outer ring, and
application above that.
Fig: Privilege rings for the x86 available in protected mode
[Source: Wikipedia]
Milan Rajpara
October 8, 2013
3

4. Necessity for Kernel Security
• Kernel, a vary basic (core) part of the Operating Systems
• Single vulnerability will be exposes large number of systems
• Increasing of Cloud Usage with Virtual Systems
• Smartphones now is in every hand
Milan Rajpara
October 8, 2013
4

5. We talk on ..
• Kernels for General Purpose Operating System
• Some Linux flavor gives Server Optimized Kernel
• Ex. Ubuntu older then 12.04, were gave this option. Since 12.04, linux-image-server is merged into linuximage-generic, there is no difference between Generic and Server kernel. [4]
• Windows do not disclose.
• Kernels which Constructed in C language
• Almost kernels are in C
• Improvement for Monolithic kernels
• All work performed in Virtual environment
• The Xen, and VMware used
Milan Rajpara
October 8, 2013
5

6. How Kernel Affected ?
• By Kernel level rootkits
• Manipulating pointers
• Manipulating data
• Direct Kernel Object Manipulation (DKOM)
• By Boot-kits
• Via hooking techniques
• Direct Hardware or Firmware injection
Milan Rajpara
October 8, 2013
6

7. Effect of this Attacks
• Escalate a process’ privileges by overwriting the process’ credentials
• Hide itself by illicitly removing data structures identifying their presence from
loaded drivers
• Eliding task structures for the processes from the kernel’s process accounting list
• Alter the overall behavior of OS without injecting any malicious code into the
kernel address space, by just pointer manipulating.
Milan Rajpara
October 8, 2013
7

8. How to analyze the Kernel Security
• Find the most critical objects of the kernel, without prior knowledge of the OS
kernel data layout in memory
• Identifying OS Kernel Objects for Run-time Security Analysis
• Sort-out objects which are vulnerable to hijack
• Do Kernel Data Disambiguation
• This will make the system easy to analyze
Milan Rajpara
October 8, 2013
8

9. Most critical objects in Kernel
• Windows and Linux, the core kernel part are mostly written in C
• 40% inter-data structure relations are Pointer based
• 35% of these are Generic Pointers
• Pointers which defines at run time, no initial value or data type is associated
• 28% kernel data structure are well known objects
Milan Rajpara
October 8, 2013
9

10. Generic Pointer Problem
• It is the weak link in kernel security
• Use of void pointers *, assists hackers to point somewhere else
• Use of NULL pointers (to implements linklist), helps hackers to hide / change
runtime objects.
• Use of Casting in C
• Enables the hackers to exploit data structure layout in physical memory
Milan Rajpara
October 8, 2013
10

11. To Find Critical Objects
1. Memory Mapping techniques
• Travers address space from global variables via pointer dereferencing until reaching
running object.
• according to a predefined kernel data definition for each kernel version.
2. Value Invariant Approaches
• Use the value invariants of certain fields or of a whole data structure as a signature to
scan the memory for matching running instances. Ex. DeepScanner, DIMSIM
• Drawbacks of this approaches
- Not very accurate
- Require a predefined definition of the kernel data layout
- Not effective when memory mapping and object reachability information is not available.
- High performance overhead
Milan Rajpara
October 8, 2013
11

12. To Find Critical Objects
3. DIGGER
[1]
• Uncover all system runtime objects without any prior knowledge of the OS kernel
data layout in memory.
• First it performs offline and constructs type-graph (which is used to enable
systematic memory traversal of the object details).
• Then it uses the 4-byte pool memory tagging schema (to uncover kernel runtime
objects from the kernel address space.)
• (+)
• Accurate result
• Low performance overhead
• Fast and nearly complete coverage
Milan Rajpara
October 8, 2013
12

13. DIGGER & KDD
• DIGGER uses the KDD (Kernel Data Disambiguator) to precisely models the
direct and indirect relations between data structures.
• KDD is a static analysis tool that operates offline on an OS kernel’s source code
• Generates a type-graph for the kernel data with direct and indirect relations
between structures, models data structures [2]
• KDD disambiguates pointer-based relations (including generic pointers)
• by performing static points-to analysis on the kernel’s source code.
• Points-to analysis is the problem of determining statically a set of locations to
which a given variable may point to at runtime.
Milan Rajpara
October 8, 2013
13

14. KDD Operation
Source: Ref [2]
AST: Abstract Syntax Tree (high-level intermediate representation for the source code )
Milan Rajpara
October 8, 2013
14

15. KDD Operation
• Interprocedural Analysis 1: Takes AST and differentiate it
• Gets: Variables, Procedure definition, Procedure call, etc.. .
• Interprocedural Analysis 2: Do points-to analysis across different files to perform
whole-program analysis.
• Context Sensitive Analysis:
• It uses Procedure Dependency Graph (PDG) consists of nodes representing the statements of the
data dependency in the program.
• context-sensitive analysis solves two problems: the calling context and the indirect (implicit)
relations between nodes.
Milan Rajpara
October 8, 2013
15

16. Soundness and Precision of KDD
• The points-to analysis algorithm is sound if the points-to set for each variable
contains all its actual runtime targets, and is imprecise if the inferred set is larger
than necessary.
• Check on C programs from the SPEC2000 and SPEC2006 benchmark suites.
• Achieved a high level of precision and 100% of soundness.
• And 96% precision on Windows (WRK*, Vista) and Linux kernel (v3.0.22). [2]
*WRK – Windows Research Kernel, the only available code from windows [6]
Milan Rajpara
October 8, 2013
16

17. DIGGER Approach
Source: Ref [1]
Milan Rajpara
October 8, 2013
17

18. DIGGER Approach
• Static Analysis Component: from KDD
• Signature Extraction Component:
• When the object manager allocates a memory pool block, it associates with a pool tag
(pool tag is a unique four-byte tag for each object type.) Uses this tag to uncover the
kernel objects running instances, and they are static and cannot be changed during
object runtime.
• Dynamic Memory Analysis Component: Extract the object details,
• From Pool Tag, it gets the pool block start memory address and the object’s start
address.
Milan Rajpara
October 8, 2013
18

19. Analyzing Kernel through DIGGER Gives …
• Disambiguate the points-to relations between data structures, all without any
prior knowledge of the OS kernel data layout.
• Robust and quite small signature size to uncover runtime objects, enhancing
performance
• Able to keep track of all critical objects of kernel
Milan Rajpara
October 8, 2013
19

20. Protection of Kernel
• Protect the generic pointers.
• Microsoft added a feature PatchGuard, which blocks kernel mode drivers from
altering sensitive parts of the Windows kernel.
• But TDL (rootkit) manages to circumvent this protection as well, by altering a machine's MBR so
that it can intercept Windows startup routines. [7]
• One approach is use of “Object Partitioning” to protect kernel data structure. [3]
• Uses Sentry, that creates access control protections for security-critical kernel data.
Milan Rajpara
October 8, 2013
20

21. Sentry Architecture
• Sentry protects critical data and
enforces data access restrictions
based upon the origin of the access
within the code of the kernel and its
modules or drivers. [3]
• The data integrity model is
straightforward and matches that of
the Biba ring policy [9]
• The malicious code that modifies
privileges by directly writing to
memory is in a loaded module and
not in the core kernel code, so Sentry
will prevent the write
Milan Rajpara
October 8, 2013
21

22. Kernel Memory Access Control
• Protect data structure from DCOM
• Sentry’s design uses a hypervisor to remain isolated from an untrusted kernel
• To keep the overhead low, Sentry uses memory partitioning to lay out sensitive
data on separate memory pages and protects those pages using the hypervisor
• The policy enforcer mediates attempted writes to protected data and uses the
policy to determine when writes should be permitted.
Milan Rajpara
October 8, 2013
22

23. Working of Sentry
• Identifying Security-Critical Members
• Activation of mediated access
• Instruction emulation
• Secure execution history extraction
Milan Rajpara
October 8, 2013
23

24. Evaluation of Sentry
• Performance
• Low performance overhead
• more performance van be achieved by memory layout optimization
• False Positive Analysis
• There were no instances when security-critical kernel data protected by Sentry was
directly modified by a benign driver.
• Sentry provided a 100% detection rate for DKOM rootkits
Milan Rajpara
October 8, 2013
24

25. Future Work
• Detect all kernel data structures automatically, beyond the kernel version
• The DIGGER can only be used to analyze Windows Kernels.
• The current prototype of Sentry only protects two key structures.
• Other kernel data structures may also require similar protection.
• This may gives versatile performance of Sentry, (if more data structure included)
Milan Rajpara
October 8, 2013
25

26. References
[1] Amani S. Ibrahim, James Hamlyn-Harris, John Grundy, Mohamed Almorsy, "Identifying OS Kernel Objects for
Run-Time Security Analysis", DOI: 10.1007/978-3-642-34601-9_6
[2] Amani S. Ibrahim, John Grundy, James Hamlyn-Harris, Mohamed Almorsy, "Operating System Kernel Data
Disambiguation to Support Security Analysis", DOI: 10.1007/978-3-642-34601-9_20
[3] Abhinav Srivastava, Jonathon Giffin, "Efficient Protection of Kernel Data Structures via Object Partitioning", DOI:
10.1145/2420950.2421012
[4] RFC: Linux kernel merging. https://lists.ubuntu.com/archives/kernel-team/2011-October/017471.html
[5] Rootkits detail by Symantec http://www.symantec.com/avcenter/reference/windows.rootkit.overview.pdf
[6] Windows Research Kernel https://www.facultyresourcecenter.com/curriculum/pfv.aspx?ID=7366&c1=enus&c2=0
[7] TDL Rootkit: http://www.theregister.co.uk/2010/11/16/tdl_rootkit_does_64_bit_windows
[8] Windows hooks: http://msdn.microsoft.com/en-us/library/ms644959(v=vs.85).aspx
[9] K. J. Biba. Integrity considerations for secure computer systems. Technical Report MTR-3153, Mitre, Apr. 1977
Milan Rajpara
October 8, 2013
26

27. Thank you 
Questions
__________________________
- Milan Rajpara
27