Objects in memory, Objects at python level, Objects in python virtual machine

4. Object structure

5. Objects with same class

6. Python Memory Management

7. Allocation Strategy For small requests, the allocator sub-allocates <Big> blocks of memory. Requests greater than 256 bytes are routed to the system's allocator.

8. Pymalloc structure Arena 256kb Pool 4kb Fixed size block

9. Fixed size block * Request in bytes Size of allocated block Size class idx * ------------------------------------------------------------------------------ * 1-8 8 0 * 9-16 16 1 * 17-24 24 2 * 25-32 32 3 * 33-40 40 4 * 41-48 48 5 * 49-56 56 6 * 57-64 64 7 * 65-72 72 8 * ... ... ... * 241-248 248 30 * 249-256 256 31

10. Garbage Collection Python performs garbage collection via reference counting and a cyclic garbage collector that is able to detect and break reference cycles.

11. Example >>> import gc >>> gc.disable() # 2.9mb >>> a = range(1000000) # 18mb >>> del a # 14mb >>> gc.enable() # 14mb >>> gc.collect() # 14mb 0 >>> gc.disable() ‏ >>> a = range(1000000) # 18mb >>> a.append(a) # 18mb >>> del a # 18mb >>> gc.enable() # 18mb >>> gc.collect() # 14 mb 1 >>>

13. weakref example >>> from weakref import ref, proxy >>> class A(object):pass ... >>> a = A() ‏ >>> ref_a = ref(a) ‏ >>> a.a = 16 >>> ref_a().a 16 >>> proxy_a = proxy(a) ‏ >>> proxy_a.a 16 >>> proxy_a <weakproxy at 0x94e25f4 to A at 0x94e0fcc> >>> del a >>> ref_a() ‏ >>> ref_a().a Traceback (most recent call last): File &quot;<stdin>&quot;, line 1, in <module> AttributeError: 'NoneType' object has no attribute 'a' >>> proxy_a <weakproxy at 0x94e23c4 to NoneType at 0x81479b8>

14. List object typedef struct { PyObject_VAR_HEAD // ob_size /* Vector of pointers to list elements. list[0] is ob_item[0], etc. */ PyObject **ob_item; // 0 <= ob_size <= allocated // len(list) == ob_size Py_ssize_t allocated; } PyListObject;

15. List creation PyObject * PyList_New(Py_ssize_t size) ‏ { PyListObject *op; size_t nbytes; .... nbytes = size * sizeof(PyObject *); ..... op->ob_item = (PyObject **) PyMem_MALLOC(nbytes); memset(op->ob_item, 0, nbytes); ... op->ob_size = size; op->allocated = size; ... }

16. Integer objects typedef struct { PyObject_HEAD long ob_ival; } PyIntObject;

17. Integer optimization. Problem >>> a = -2 >>> b = -2 >>> id(a) == id(b) ‏ True >>> a = 300 >>> b = 300 >>> id(a) == id(b) ‏ False

18. Integer optimization. Solution #define NSMALLPOSINTS 257 #define NSMALLNEGINTS 5 /* References to small integers are saved in this array so that they can be shared. The integers that are saved are those in the range -NSMALLNEGINTS (inclusive) to NSMALLPOSINTS (not inclusive). */ static PyIntObject * small_ints[NSMALLNEGINTS + NSMALLPOSINTS];

20. Objects creation Constructor: obj = type->tp_new(type, args, kwds); type = Py_TYPE(obj); type->tp_init(obj, args, kwds); Classes created by metaclasses

21. Object attributes Attributes stored in __dict__ or in object when __slots__ used Attribute can be accessed via descriptor protocol The most famous descriptor - property

23. Objects in PVM typedef struct { PyObject_HEAD int co_argcount; /* #arguments, except *args */ int co_nlocals; /* #local variables */ int co_stacksize; /* #entries needed for evaluation stack */ int co_flags;/* CO_..., see below */ PyObject *co_code; /* instruction opcodes */ PyObject *co_consts; /* list (constants used) */ PyObject *co_names;/* list of strings (names used) */ PyObject *co_varnames;/* tuple of strings (local variable names) */ PyObject *co_freevars;/* tuple of strings (free variable names) */ PyObject *co_cellvars; /* tuple of strings (cell variable names) */ /* The rest doesn't count for hash/cmp */ PyObject *co_filename;/* string (where it was loaded from) */ PyObject *co_name;/* string (name, for reference) */ int co_firstlineno;/* first source line number */ PyObject *co_lnotab;/* string (encoding addr<->lineno mapping) */ void *co_zombieframe; /* for optimization only (see frameobject.c) */ } PyCodeObject;

24. CodeObject example File test.py a = 17 b = a + 3 print b

25. CodeObject Example >>> import test 34 >>> pyc = open('test.pyc') ‏ >>> pyc.read(8) ‏ 'b3f2>C8eH' >>> from marshal import load >>> co = load(pyc) ‏ >>> dir(co) ‏ [... 'co_argcount', 'co_cellvars', 'co_code', 'co_consts', 'co_filename', 'co_firstlineno', 'co_flags', 'co_freevars', 'co_lnotab', 'co_name', 'co_names', 'co_nlocals', 'co_stacksize', 'co_varnames'] >>> co.co_consts (17, None) ‏ >>> co.co_names ('a', 'b') ‏

26. CodeObject example >>> from dis import dis >>> dis(co) ‏ 1 0 LOAD_CONST 0 (17) ‏ 3 STORE_NAME 0 (a) ‏ 2 6 LOAD_NAME 0 (a) ‏ 9 LOAD_CONST 0 (17) ‏ 12 BINARY_ADD 13 STORE_NAME 1 (b) ‏ 3 16 LOAD_NAME 1 (b) ‏ 19 PRINT_ITEM 20 PRINT_NEWLINE 21 LOAD_CONST 1 (None) ‏ 24 RETURN_VALUE