kmatch98 / circuitpython_memory_saving Goto Github PK

Comments are bad

Only joking... but they kind of are.
Comments in circuitpython code in .py format take up memory on the device when running the code even though they don't do anything. If you need to save some memory, you could delete all your comments. However, a better solution is to leave the comments in the code and compress to a .mpy file (See above/below)

But, code.py can't be compressed.

Simple workaround:

• Rename code.py to mycode.py
• Compress mycode.py to mycode.mpy on your computer and only save/copy the .mpy file to the device. This not only compresses the python scripts but removes the comments.
• Create a new code.py which imports your compressed code:

import mycode

This does make the development steps a little harder than just saving the code.py file to the device but in a memory-constrained device like QT PY it can be worth the effort.

Based on the two recent CircuitPython meeting, it is clear that something bust be said about bytearray.

If you only have to store positive integer from 0 to 255, then that is the optimal structure to use as each entry cost one byte, where a list of int might cost 4 bytes per entry.

So if short table of short value are needed, putting them as bytearray is beneficial, making a list of tuple would be overkill as the list is an object, each tuple is an object and each int take some space.

A typical bad example is this table: https://github.com/adafruit/Adafruit_CircuitPython_IS31FL3731/blob/9993369654a47ca8763b0971e1dc847717b21cc6/adafruit_is31fl3731/keybow2040.py#L67 :

        lookup = [
            (120, 88, 104),  # 0, 0
            (136, 40, 72),  # 1, 0
            (112, 80, 96),  # 2, 0
            (128, 32, 64),  # 3, 0
            (121, 89, 105),  # 0, 1
            (137, 41, 73),  # 1, 1
            (113, 81, 97),  # 2, 1
            (129, 33, 65),  # 3, 1
            (122, 90, 106),  # 0, 2
            (138, 25, 74),  # 1, 2
            (114, 82, 98),  # 2, 2
            (130, 17, 66),  # 3, 2
            (123, 91, 107),  # 0, 3
            (139, 26, 75),  # 1, 3
            (115, 83, 99),  # 2, 3
            (131, 18, 67),  # 3, 3
        ]

All values are between 0 and 255 and with a code a little bit smarter, it can be an array of 48 bytes.

If the needed data are big, then reading them from file into a bytearray rather than to declare that in the code will make the code smaller and the memory usage more efficient. Typically a map for a game (containing reference to background tile) could be into a resource file.

One way to avoid memory activity that has a cost in fragmentation and CPU, it is best to use any function that do "readinto" an existing buffer. And that buffer should be the same everytime we have to read (like when you change level in a game). This mean a local variable in a function allocated on the stack at each call is worst than a global (to the library) variable.

Bad example are like this: https://github.com/adafruit/Adafruit_CircuitPython_MLX90640/blob/2668229b7b72dcfc2317c1e607be8ecb1c505cd9/adafruit_mlx90640.py#L123

Every time this function is called, for every "image" a new temporary array of 834 element is created:

    def getFrame(self, framebuf):
        """Request both 'halves' of a frame from the sensor, merge them
        and calculate the temperature in C for each of 32x24 pixels. Placed
        into the 768-element array passed in!"""
        emissivity = 0.95
        tr = 23.15
        mlx90640Frame = [0] * 834

Since you only make one call at a time to getFrame, it can be a more global and always the same buffer.

Maybe blurring the distinction between memory on the stack and memory in the heap is not great... and I don't exactly know how CP work inside and what is stored where, so this can give general idea on what to do or not to do.

So maybe special advice that are for people writing library would be separated.

A library is typically always a *.mpy file... so that advice is not needed anymore.

The library better be split in module, so that if you support multiple hardware, so that user will only import the piece they need.

Not a Python expert, but let's say a module is a folder with a init.py and various other *.py files where a library would be a single library.py file.

https://github.com/adafruit/Adafruit_CircuitPython_IS31FL3731 is split into piece to permit that kind of import.

There are specific advise for library, such as avoiding to define all kind of constant (like you would do in an Arduino library) but only create those the user will need, not those for internal usage. (I am not expert in how Python treat constant... it seems that except for uppercase name, there is no such thing as a constant).

I am pretty sure there is something "special" about the way "string" are manadged in Python.

Something about "invariant" or I don't know what.

So when formatting string, even in a print there might be more or less optimal way to do it and avoid a lot of memory allocation (if it's in the stack, I guess it is fine, if it is in the heap, maybe this will make a lot of fragmentation over time).

I totally don't know the proper and best way of doing things... let's say for a program that just want to print, or display values.

There is the str() function that can convert a value into a string, and then you can concatenate.
There is the print("Value: ", value, ".") way, without formatting.

There is that notation that I have seen used:
line = "Temperature %0.1f C" % (bme680.temperature + temperature_offset)

I am pretty sure one is better than the other, no big saving, except if you do a lot of string processing...
But I would love to know the recommended way (I think that kattni already asked that kind of question in meeting and got an answer from danh or tannewt ... I wish I had a good enough memory to remember what is the best and why... and maybe the question was about what is more "pythonic" rather than what is more memory efficient).

So there could be things to say about "print" and "string" but this would need to be ask to more expert people. :-)

Normally I create font files that contain the entire alphabet but for applications that don't display much text, you can save space by deleting all the characters that you don't actually use. This would save ram when you do bitmap_font.load_font(), right?

See the "Optimize File Size" section here
https://learn.adafruit.com/custom-fonts-for-pyportal-circuitpython-display/conversion