In Python every class can have instance attributes. By default Python uses a dict to store an object’s instance attributes. This is really helpful as it allows setting arbitrary new attributes at runtime.
However, for small classes with known attributes it might be a
bottleneck. The dict wastes a lot of RAM. Python can’t just allocate
a static amount of memory at object creation to store all the
attributes. Therefore it sucks a lot of RAM if you create a lot of
objects (I am talking in thousands and millions). Still there is a way
to circumvent this issue. It involves the usage of __slots__ to
tell Python not to use a dict, and only allocate space for a fixed set
of attributes. Here is an example with and without __slots__:
Without __slots__:
class MyClass(object):
def __init__(self, name, identifier):
self.name = name
self.identifier = identifier
self.set_up()
# ...With __slots__:
class MyClass(object):
__slots__ = ['name', 'identifier']
def __init__(self, name, identifier):
self.name = name
self.identifier = identifier
self.set_up()
# ...The second piece of code will reduce the burden on your RAM. Some people have seen almost 40 to 50% reduction in RAM usage by using this technique.
On a sidenote, you might want to give PyPy a try. It does all of these optimizations by default.
Below you can see an example showing exact memory usage with and without __slots__ done in IPython thanks to https://github.com/ianozsvald/ipython_memory_usage
Python 3.4.3 (default, Jun 6 2015, 13:32:34)
Type "copyright", "credits" or "license" for more information.
IPython 4.0.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: import ipython_memory_usage.ipython_memory_usage as imu
In [2]: imu.start_watching_memory()
In [2] used 0.0000 MiB RAM in 5.31s, peaked 0.00 MiB above current, total RAM usage 15.57 MiB
In [3]: %cat slots.py
class MyClass(object):
__slots__ = ['name', 'identifier']
def __init__(self, name, identifier):
self.name = name
self.identifier = identifier
num = 1024*256
x = [MyClass(1,1) for i in range(num)]
In [3] used 0.2305 MiB RAM in 0.12s, peaked 0.00 MiB above current, total RAM usage 15.80 MiB
In [4]: from slots import *
In [4] used 9.3008 MiB RAM in 0.72s, peaked 0.00 MiB above current, total RAM usage 25.10 MiB
In [5]: %cat noslots.py
class MyClass(object):
def __init__(self, name, identifier):
self.name = name
self.identifier = identifier
num = 1024*256
x = [MyClass(1,1) for i in range(num)]
In [5] used 0.1758 MiB RAM in 0.12s, peaked 0.00 MiB above current, total RAM usage 25.28 MiB
In [6]: from noslots import *
In [6] used 22.6680 MiB RAM in 0.80s, peaked 0.00 MiB above current, total RAM usage 47.95 MiB