Fix to MicroPython's Memory Allocation Problem on ESP8266

I've been trying to write code that allows me to display text on my ST7735 display. The font module I found is around 4kB of source code that needs to be loaded by Python, compiled into byte code and executed by the Python Virtual Machine, or VM (for more details refers here). This process works somewhat like Java's VM approach, but on devices with limited memory, i.e. the ESP8266, can become troublesome. Loading, compiling and storing the byte code takes up precious memory, which may not always be allocatable; thus throwing an MemoryError: memory allocation failed, allocating XXX bytes.. In this little post I want to show you how you can pre-compile modules for the ESP8266, so you will avoid wasting space. And if that option is too cumbersome for you, I also present a little hack to increase the ESP's heap memory, but that's more of a bodge-job for lazy people: i.e. me :p

The nice approach

Rather than trying to change the operation of the MicroPython VM as such, I want to show you how you can pre-compile some Python code to be readily executable on the ESP8266. This is achieved by loading the source code into the MicroPython image as frozen byte code. Byte code means that the code is ready to run on the Python VM, and frozen implies that the code is no longer changeable. Therefore, frozen byte code can be seen as a pre-installed and ready-to-use library that can be loaded into your code at any moment. Now you know what you will be doing, let me show you how it's done.

First you need the actual MicroPython repository and a functioning cross compiler (I've covered this here already). In the repository, the esp8266 directory contains the source code which will become the ESP's MicroPython image. In here, a subdirectory named modules contains python code for modules that will be converted into byte code and included into the MicroPython image, which in turn can be flashed to the ESP.

esp8266-modules

 

To demonstrate the entire process from writing your code to a finished image, I wrote a module with a function that print a simple Hello Maxi to the console. The code looks like so:

# hello.py

def say_hi():
    print('Hello Maxi')

The file hello.py is then saved into the modules directory. Now, in the terminal, navigate to the esp8266 directory and compile the project by typing:

make

If you get some errors, make sure you imported the xtensa bin directory to your PATH variable and that you make axtls successfully. The resulting compilation process will show you that it also generated MPY code from the source code in modules:

esp8266-compiled

 

Connect your ESP8266, clear its flash memory using (you may need to use root privileges for this):

esptool.py --port /dev/ttyXXX erase_flash

Then flash the new image to it using:

make PORT=/dev/ttyUSB0 FLASH_MODE=qio FLASH_SIZE=8m deploy

Once you rebooted your ESP and connected with it, you can import this code and run it immediately:

esp8266-sayhi

 

How cool is that! Now I do no longer get any memory allocation errors when I want to load my 4kB font module, or any other "large" module.

The dirty hack

Well... If you are developing rather large libraries that you need to modify frequently, recompiling an entire image just for the sake of a few bug-fixes may be a bit annoying. So I give to you a little hack that has also worked for me: increase the ESP's heap memory. This can be done by changing a line of code in the main.c file from:

STATIC char heap[28 * 1024];

to

STATIC char heap[40 * 1024];

Now instead of allocating 28kB of heap memory, you have 40kB. You still need to compile the MicroPython image and flash it to the ESP like before, but with the increased memory you should be able to run all your modules quite happily. That said: your milage may vary with Python stability...

 

2 Responses to “ Fix to MicroPython's Memory Allocation Problem on ESP8266 ”

  1. Hi Max,

    Thank you for sharing the dirty hack, I tried it and it works, I am just wondering how to calculate the upper limit of heap you can use in main.c, I guess it depends on esp8266 RAM which if remember correctly is 96K, do you know it?

Leave a Reply

*
*