How to build the firmware.bin file

I only changed 3 characters to 3 different characters, so the size should remain the same. My suspicion is that the source code has been changed since the 1.0.1 firmware.bin release (though the version still comes up 1.0.1).

If so (question for you Robert), where can I get the exact source code that was used to build the 1.0.1 release firmware.bin (without any source code modifications that may have been added since that file was generated on your end)?

Another question for Robert:
Are the other 3 files that are flashed along with firmware.bin (namely, boot_app0.bin, bootloader_dio_80m.bin, and partitions.bin) always the same?

The reason I ask is that if I generate a new firmware.bin and then upload it using esptool.py directly (along with those 3 files), the firmware works properly.

If I use VS Code to do the upload (Terminal/Run task/PlatformIO: Uplaod Firmware), then the bootloader_dio_80m.bin and partitions.bin that it uploads appear to be different (sizes) from the ones used with the esptool.py script; and the resulting firmware SEEMS to work properly until I try to load a game; when the game tries to boot, it hangs forever saying Loadingā€¦ .
Thus, uploading directly from VS Code seems flawed.

The code in question is right here: Release 1.0.1

More specifically the commit hash is: bedb121196fedc818cba38f98dc20698e854089a

I am still skeptical this code will produce the results we are looking for, as it is my understanding that PlatformIO compiles extra (device specific) dependencies as part of the build process. Meaning that we wont be able to reproduce the exact build ourselves. :frowning:

Well, I thought that too, but youā€™ll notice that in that zip file, the lib/Makerphone directory is empty. According to Robert, it can be filled from here: CircuitMess-Ringo but there isnā€™t any tag 1.0.1 there (only 1.0.0); so that part of release 1.0.1 is missing, sigh.

What we really want is the Ringo-firmware tree for 1.0.1 with the lib/Makerphone directory filled in.

1 Like

@frankprindle @cskwrd The error about size is because of the platformio.ini file - this line is extremely important when it comes to that:

board_build.partitions = min_spiffs.csv

It basically defines the partition scheme that the board is going to use when uploading data to the device. The default partition scheme is not big enough to store the data since it has several smaller partitions. You can find out more here.
Make sure that the platformio.ini file is the same as the one I posted up there.

The platformio.ini file is ignored since this is just an idea of how you can use this library - there are a number of other IDEs that can be used and there is just no point in uploading every single config file for every one of them.

MAKERphone is 1.0 because itā€™s the library - only some .h and .cpp files from it are used, while the actual firmware (apps and menu) that you see on the phone is the one that updates. Nothing is missing there.

That MAKERphone directory isnā€™t filled for several reasons, one being that itā€™s extremely large compared to the firmware folder and that can be a nuisance when working with the files quite often.

About the three files, I guess that they should change if you make some changes to the firmware (although I have no idea to which extent).
We have no problems with it though. PlatformIO is a bit tough to configure and can produce some weird stuff, but once it gets done, it works nicely.

So let me summarize my process so that it is crystal clear that Iā€™m trying to duplicate CircuitMessā€™ methology exactly:

  1. Fetch CircuitMess-Ringo-firmware release 1.0.1 from here: CircuitMess-Ringo-firmware Release 1.0.1 and unzip it.

  2. Put those 2 directories and 4 files into the VS Code ProjectIO directory I created (which I named Firmware).

  3. Fetch CircuitMess-Ringo release 1.0.0 from here: CircuitMess-Ringo Release 1.0.0 and unzip it.

  4. Put those 3 directories and 7 files into the (empty) Firmware/lib/MAKERphone directory created as a result of step 2.

  5. Modify the platformio.ini file in the VS Code ProjectIO directory I created to read:
    [env:lolin32]
    platform = espressif32
    board = lolin32
    framework = arduino
    board_build.partitions = min_spiffs.csv
    monitor_speed = 115200
    upload_speed = 921600

  6. Run VS Code, opening the workspace I previously created to contain the ProjectIO directory Firmware into which I just copied all that stuff.

  7. Select from the VS Code menu bar Terminal/Run taskā€¦/PlatformIO: Build Firmware (where Firmware is the name of the ProjectIO directory).

  8. VS Code proceeds to compile (with 80 warnings, yikes), link, and load generating .pio/build/lolin32/firmware.bin.

  9. The resulting firmware.bin file is of size 1,546,736 bytes, which is different from the firmware.bin file downloaded from GitHub for release 1.0.1 of CircuitMess-Ringo-firmware, which is of size 1,540,192 bytes).

  10. Copy the firmware.bin file to a directory I use to flash the phone per previously provided instructions for using esptool.py thusly:
    esptool.py --chip esp32 --port COM6 --baud 921600 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 80m --flash_size detect 0xe000 boot_app0.bin 0x1000 bootloader_dio_80m.bin 0x10000 firmware.bin 0x8000 partitions.bin

  11. Run the script to execute esptool.py per step 10. It proceeds to flash the phone (connected in my case to COM6). The sizes of the 4 files as reported by esptool.py are:
    boot_app0.bin: Compressed 8192 bytes to 47ā€¦
    bootloader_dio_80m.bin: Compressed 15856 bytes to 10276ā€¦
    firmware.bin: Compressed 1546736 bytes to 832722ā€¦
    partitions.bin: Compressed 3072 bytes to 143ā€¦

  12. The firmware APPEARS to run the same way as the released (but 6544 bytes shorter) firmware.bin from the official release, BUTā€¦ there is at least one operational difference. With the release firmware, the string ā€œloadingā€¦ā€ that appears immediately to the right of the home screen triangle showing signal strength disappears after a minute or two. With the firmware that I just built above, that string never disappears. This indicates to me that the released firmware.bin was NOT built from release 1.0.1 of CircuitMess-Ringo-firmware and release 1.0.0 of CircuitMess-Ringo.

Thus I have failed in two ways to reproduce the released 1.0.1 firmware.bin; first, itā€™s size is 6544 bytes larger than the released firmware.bin; second it is missing the functionality that removes that ā€œloadingā€¦ā€ string after several minutes.

I think Iā€™ve followed the instructions that Robert provided exactly, so Iā€™m wondering how CircuitMess managed to generate their 1.0.1 firmware.bin and what Iā€™m missing in the generation process (perhaps a compiler optimization option that I need to set)??? The whole discussion about using other IDEs is worrysome because a given IDE provides its own C++ compiler and will generate different object code from the same source code, so if CircuitMess is using VS Code, then I think I should stick to that if I ever expect to match.

1 Like

I later noticed that the firmware.bin that I built from CircuitMess-Ringo-firmware release 1.0.1 and CircuitMess-Ringo release 1.0.0 comes up and lists its version as 0.9.9!

Upon further investigation, I found that the source code for MAKERphone.h and MAKERphone.cpp was modified AFTER release 1.0.0 of CircuitMess-Ringo. Those changes, which affect the ā€œloadingā€¦ā€ issue and the version number are part of no numbered release of CircuitMess-Ringo, yet are part of the released 1.0.1 firmware.bin. Pretty sloppy configuration management.

Upon replacing CircuitMess-Ringo Release 1.0.0 with the latest (unreleased) code and recompiling, I still get 80 warnings, but the firmware.bin that is generated is now 1,546,880 bytes (still more than 6K larger than the 1.0.1 release firmware.bin generated by CircuitMess). Upon flashing that file, the ā€œloadingā€¦ā€ string does disappear after a while as it should, and the firmware version says 1.0.1.

CircuitMess needs to add a release 1.0.1 to CircuitMess-Ringo that corresponds to the code used in firmware release 1.0.1.

Now it just remains unexplained that the generated firmware.bin is 6688 bytes larger than the released firmware.bin.

Greetings,

I believe this whole confusion was because of one simple oversight on our part.

When we pushed a new GitHub release for our firmware, we forgot to do the same for the Ringo library.
The Ringo library release 1.0.0 is the latest release, but itā€™s not the library version that we used to compile the 1.0.1 firmware binary.

To solve this you can download the current master branch of the Ringo library, or download it zipped from the new 1.0.1 release that got pushed a few moments earlier from writing this.

We will keep in mind all your constructive feedback in our future version management : )

Hey Emil, thanks for the info. It took a while, but I figured it out.

Any idea why the firmware.bin sizes are still different? Is there a way to select different optimization levels for the VS Code C++ compiler (analogous to -O3 etc. for gnu c++)? Perhaps we are compiling with different optimization levels? Or maybe itā€™s because we are using different versions of VS Code (Iā€™m using 1.40.0)?

I tried compiling the binary with the newest releases and Iā€™m getting the same-sized firmware binary.
Could you double check that you are indeed compiling with the latest releases?

Also, is the size difference significant, and could you point me to how could I reproduce compiling this firmware binary thatā€™s not the same size?

Emil, the size difference is only significant in that making the binaries compare is the only true way to be sure Iā€™m replicating your process exactly.

The way I made this is first I followed Robertā€™s instructions in the third and fifth posts of this topic. Then I did all the steps I outlined in the 13th post of this topic. Then I replaced the 1.0.0 release of CircuitMess-Ringo with the new 1.0.1 release and repeated the steps in the 13th post. That gave me the firmware binary that is now 6688 bytes larger than your binary. Like I said above, perhaps Robert left out some details as to how I should set up VS Code; if there are any options I need to select like optimization level. I am also using version 1.40.0 of VS Code - perhaps you are using an earlier version and that could account for the difference, I donā€™t know.

Ok, Iā€™ve succeeded in reproducing the problem.
Iā€™ve followed the instructions provided in the thread on a fresh installation on a 2nd PC and yes, the compiled binary is indeed larger than expected.

Currently, I have no apparent answer as to why this is happening, but as far as I tested, the firmware seems just as functional as the original one.

If I had to make a guess it would probably be something about the compiler version or the environment that was used to compile the original binary.
I will probably look into this sometime in the future but in the end, it seems like just a harmless difference in compilation environments.

@frankprindle I hope this cleared up this misunderstanding concerning the compilation of the firmware.

Thatā€™s great to know. Iā€™m going to stop worrying about the size difference now that I know it even happens to you. Iā€™m pretty confident I know how to get the release sources now. Maybe Iā€™ll take a crack at getting rid of some or all of the 80 warnings; Iā€™ve always found that ignored warnings are a bad idea.

1 Like

Emil, I was able to take the source code for the 1.0.1 release and eliminate all 80 warnings so that it compiles cleanly. Many of those were simple unused variables, but 10 specific warnings turned out to be because of suspected coding errors. Anyway the changes are concentrated in 19 source files. Iā€™ll send the annotated files to contact@circuitmess.com. It seems to run just fine.

4 Likes

Okay, thanks for that.

This seems like a standalone issue so I would prefer to have it in a separate forum thread to avoid irrelevant posts in this thread.

Also, a better course of action might be to fork our repository on github and open a pull request for our library/firmware, whichever you edited.

I will go over the files youā€™ve sent and will message you back via email.

1 Like

I have opened pull requests to CircuitMess-Ringo-firmware and CircuitMess-Ringo that removes all warnings and fixes the bugs hidden among them.

Remember that:

Compiler warnings should always be taken seriously, since bugs will hide among them.

2 Likes

Has anybody tried to build it from the command line? Iā€™m having some success, but am missing setup() and loop(). Those functions are defined in /home/nelson/ringo/CircuitMess-Ringo-firmware/src/main.cpp but that file isnā€™t getting included in the build.

The command line of which build environment? Arduino? VS Code? Something else?

Unix shell. Probably using platformio.

You got me there. I searched high and low through my Windows 10 machine running VS-Code with Platformio extension and couldnā€™t find hide-nor-hair of a Makefile. Yet I know the compiler options must be being set somewhere because in this compilation ā€œcharā€ is unsigned which is NOT the default for g++. Do you have a Makefile, and if so where did you get it? Main.cpp is in the same directory as the rest of the firmware, so something would have to be intentionally NOT compiling it, while compiling everything else.