[sanerdemirel]

Fixes #9342.

UpdaterClass::begin() picks a 4 KB write buffer when total free heap is above 8 KB, and a 256 byte buffer otherwise. It then makes one allocation. If the heap is fragmented, there can be more than 8 KB free but no free block of 4 KB. The allocation fails and begin() returns UPDATE_ERROR_OOM, although the update would work with 256 bytes.

As suggested in the issue, begin() now walks through {FLASH_SECTOR_SIZE, 256} and uses new (std::nothrow) for each size. The first size that works is used. If both fail, it still reports UPDATE_ERROR_OOM.

The low-heap rule is unchanged: with 8 KB or less free, only 256 is tried, so the buffer does not take the last of the heap.

The diff is cores/esp8266/Updater.cpp only, +7/-5. The 256 byte path was already used when the heap was low, so no code needs to learn about a new buffer size.

Tests

This PR adds no new tests. An earlier version had host tests that replaced the nothrow operator new[] for the whole test binary, to refuse chosen sizes. As discussed in the review, that is the wrong level, and under valgrind the hook is never called, so I dropped them. The fallback tests can follow on top of a mocking allocator that works under valgrind:

• 4096 refused: begin() still succeeds with 256.
• 4096 and 256 refused: begin() reports UPDATE_ERROR_OOM, and the updater can be used again afterwards.

Results on the current head, on Ubuntu 24.04 with g++ 13 and clang-format-18:

• make FORCE32=0 D=1 test passes: 89 test cases, 33214 assertions. The existing Updater host tests run against the changed begin().
• make CI (-Werror, valgrind, coverage) passes: 0 errors, no leaks reported.
• tests/restyle.sh with clang-format-18 changes nothing.

Not tested

• The fallback path itself, by a test (see above).
• I did not run this on hardware. The change is only the buffer choice in begin().
• I did not measure upload speed with the 256 byte buffer. That path was already in use for low heap.
• Only g++ 13 on x86-64 (64-bit mode, as in CI) was used. I did not build the core for the ESP8266.

[mcspr]

i believe this has to go to malloc, since we don't particularly care that this specific call is overloaded but only that some sized allocation get rejected.
also note it missed the linkage part, this gets pulled everywhere in the test runtime

thanks otherwise! i don't mind this here, but only in a way that valgrind actually works.
could just drop it too, in that case i'll create a separate pr for similar cases. it would have to create a separate 'mocking' allocator with stubs that we then could override through -Wl,--wrap=foobar

[sanerdemirel]

Thanks for the review. You are right: replacing operator new[] for the whole test runtime is the wrong level, and under valgrind the hook is never called, so those tests only skipped themselves there.

I have dropped the allocation-hook tests from this PR. What is left is the change in Updater.cpp itself (try FLASH_SECTOR_SIZE, then 256 B, under the same low-heap rule as before). The existing Updater host tests still pass against it, also under make CI with valgrind.

When your mocking allocator with -Wl,--wrap is in, I am happy to add the fallback tests on top of it: one that refuses the 4 KB buffer and checks that begin() still succeeds with 256 B, and one that refuses both and checks the clean out-of-memory error.