Skip to main content

ESP8266, D1 and the esp-open-sdk

ESP8266 started with terrible documentation. Yet for a cheap-and-tiny WiFi module, people hacked around it, added code to it, and manipulated its extra I/O pins to make it do more than it was originally designed for which is a WiFi module to be controlled by an external processor via a serial interface with AT commands in an embedded system. Why extend it to external microcontrollers or microprocessors when it has its own powerful 32-bit processor and plenty of flash memories?

ESP8266 has many uses and people created many quick-to-used boards for it such as D1. No, we don't have to limit ourselves to a particular board. People also created various SDKs for the module and flooded the Internet with incompatible documentations. This is a bitter-sweet situation. Now, we have to assemble our own tool's stack.

Espressif Systems, the founder of ESP8266, has a catalog of documentation. They are now proactive to the world community.

This is a collection of programming notes around ESP8266 with D1 and the esp-open-sdk. There are many simpler ESP8266 software development frameworks built on top of the esp-open-sdk. However, we have to compete for the module's resources with those frameworks. The fake reasons for D1 are that it has an integrated power supply circuit for 3.3v and 5v, and the Arduino UNO form factor. However, D1 is not power compatible with UNO. I/O pins in D1 are all directly connected to ESP8266 with 3.3v power rating.

NOTE: These are NOT getting started notes.

Double Precision


Yes. The compiler support the IEEE 754 binary64 double-precision floating point. The processor has an iterative 32 bit multiplier. In term of general performance, the ESP8266 is much better than IBM PC and comparable to Intel 80486 which in the last millennium was broadly used in server, desktop and notebook computers.

Gibberish Serial Output


This is due to the bootloader's baud rate of 74880 and our serial monitor does not match. To change the module baud rate to 115200 use the following command in our code:

uart_div_modify(0, UART_CLK_FREQ / 115200);

If we change our serial monitor baud rate then we will miss the boot message.

LED Flashing Rapidly


I got the following note from https://github.com/pfalcon/esp-open-sdk/issues/279#issuecomment-320531134:

The high speed flashing LED means that the ESP failed to boot and is sending an error over serial interface. If you monitor the serial output with the bootloader's baud rate of 76800, you should see an error. If the error message contains the line rf_cal[0] !=0x05,is 0xFF, this means that the rf_cal data has been erased, and not restored. To restore it flash esp-open-sdk/sdk/bin/esp_init_data_default.bin to the appropriate memory address.

The correct address for the init data depends on the capacity of the flash chip.

Address Flash Modules
0x7c000 512 kB most ESP-01, -03, -07 etc.
0xfc000 1 MB ESP8285, PSF-A85, some ESP-01, -03 etc.
0x1fc000 2 MB
0x3fc000 4 MBESP-12E, NodeMCU devkit 1.0, WeMos D1 mini.
0x7fc000 8 MB
0xffc000 16 MB WeMos D1 mini pro.

So, in our case it is at 0x3fc000 which is the last 16 kB of the 4 MB flash memory. In fact, all the addresses in the table above are pointing to the last 16 kB of their respective flash memory. Hence, the esp_init_data_default is located at the last 16 kB of the flash memory.

> cd esp-open-sdk/sdk/bin
> esptool.py write_flash 0x3FC000 esp_init_data_default.bin


We only need to do this flashing once. We need to do this after we erase the flash. We also need to fill two areas near the address with blanks. Then, the system will initialize the two areas with system data.

> esptool.py write_flash 0x3FB000 blank.bin
> esptool.py write_flash 0x3FE000 blank.bin

The Last 20 kB


The last 20 kB of the flash memory are used by the system as RF_CAL parameter area (4 kB), default RF parameter area (4 kB), and system parameter area (12 kB). The area immediately before them is designated as the user data area. Hence, we may keep our application parameters near to the last 20 kB of the flash memory. The area from 0x3FA000 to 0x3FAFFF may be safe for us.

Update in RAM and Then Copy Back to Flash


Writing to the flash is actually a 4 kB chunk at a time because we cannot erase smaller bytes than that. We have to erase before we can write. An erase will set all bits to 1s. A write can only turn a 1 into a 0 and not the other way round. A blank means all bits are 1s.

The flash is not really meant for random RW. The system is designed so that it can quickly move codes in large chunks from flash into the iRAM for execution. Writing into the flash is also in large chunks so that flashing our codes during development will be fast.

This is about keeping our persistent and editable data in the flash.

First, copy the target flash sector of 4 kB into RAM. Make necessary changes in the RAM. Erase the flash sector. Then copy the edited RAM back into the flash. Warning: If power fails during writing then the flash sector will remain blank since it was erased, and all our persistent data in the sector will be gone. We need a read/write (RW) protection.

Please refer to the ESP8266 Flash RW Operation for details which include a technique for RW protection. However, the technique will incur flash wastage. I.e. To use one sector of flash we have to allocate three sectors. One for our data. One for switching. And the other one for a tiny flag.

A better RW protection technique should be to allocate only two sectors. One for our data and the other one for switching. Then reserve four bytes at the end of both sectors as flags to store serial numbers. The flag containing the higher serial number will indicate that the sector stores the latest data. On update, the sector with the lower serial number will be overridden with the latest data and its serial number will be made the higher. The sectors will alternately hold the latest data. We don't have to worry about the serial number to overflow because when it does then the flash's lifespan will already long pass due. On power failure, only the latest data will be gone.

Once in a while we can get away with sector switching when a data that we want to write does not have to turn any bit with 0 to 1.

Keep CODES in FLASH Cache


Otherwise the codes will be kept in an area for the limited iRAM segment and may cause it to overflow. Compiler error message may contain: ...section `.text' will not fit in region `iram1_0_seg' and region `iram1_0_seg' overflowed...

Make sure all functions that should be kept in flash cache are decorated with ICACHE_FLASH_ATTR.
i.e void ICACHE_FLASH_ATTR *func(void){}

And add -DICACHE_FLASH to CFLAGS in makefile. Don't forget to include the - sign before D.

Our codes will still be run from iRAM. The system will copy our codes into iRAM before executing them. There will be a slight delay in execution due to copying. Once copied into iRAM the codes will run as fast as it takes.

We may not need to decorate functions with ICACHE_FLASH_ATTR when there is enough iRAM.

Remove ALL unused codes


So that we can add more codes. The SDK comes with its own codes that we have to linked to. Most of the codes will not be used.

Add  -fdata-sections -ffunction-sections to CFLAGS in makefile.
Add -Wl,--gc-sections to LDLIBS in makefile.





Comments

Popular posts from this blog

Setting Up PyScripter for Quantum GIS

PyScripter is a general purpose Python Integrated Development Environment (IDE). Quantum GIS (QGIS) is a desktop GIS application that can be extended with Python plugins. Both are open source softwares. We intend to use PyScripter as an IDE to build QGIS Python plugin. We are using PyScripter 2.4.1.0 and QGIS 1.6.0 in Windows. PyScripter does not come with Python. On the other hand, QGIS is built in with Python. Thus, we will setup up PyScripter to use the build in Python in QGIS. We assume both PyScripter and QGIS are already installed. Preparing PyScripter batch file We assume that QGIS is installed in C:\OSGeo4W\ folder and PyScripter is installed in C:\Program Files\PyScripter\ . 1. Copy qgis.bat in C:\OSGeo4W\ bin to pyscripter.bat 2. Edit pyscripter.bat to remove the last line that read something like this start "Quantum GIS" /B "%OSGEO4W_ROOT%"\apps\qgis\bin\qgis.exe %* and replace it with this in one line Start "PyScripter" /B "C:\Progr

Git My Way

Update : This blog was written about two years before Github dropping the so-called divisive "master" terminology , and renamed it to "main". But I'm not going to change anything here. I just want to keep the original context as is. Please read master as main . After all this while I still could not remember how to do git right, and I kept on coming back to this blog, each and every time I wanted to update a Github repo. If you are careless then you may encounter "fatal: Couldn't find remote ref master", and Google will lead you into a pit of old and unrelated discussions. I used to bookmark a number of git tutorials. Most of it works but only to a certain extend until I forgot most of the required steps later. The steps were not intuitive and can easily be forgotten. Git uses alien English. Then I realize that I have to list my own way of using git. I believe all of you have too because we depend on it. I use git with Github and mostly to upda

Get Current User of Drupal 9 Externally

You have a stand-alone application that is not a Drupal module but resides in a Drupal sub-folder . And you want Drupal to manage your users. You want to access the currently logged-in Drupal user from your application. The following function will give you the current  user id, name, email and roles: use Drupal\Core\DrupalKernel; use Symfony\Component\HttpFoundation\Request; /** * Get Drupal current session user details. * Passing Drupal folder, or its relative folder such as '..' * when it is called from a Drupal sub-folder. * Return ['id', 'name', 'email', 'roles'] */ function get_drupal_current_user($drupal_dir) { // Change the directory to the Drupal root. chdir($drupal_dir); $drupal_root = getcwd(); if ($drupal_root === false) return []; $autoloader = require_once 'autoload.php'; $kernel = new DrupalKernel('prod', $autoloader); $request = Request::createFromGlobals(); // Emulate Drupal /index.php to get