Capped Allocator about arduinojson HOT 4 CLOSED

Hi @mathieucarbou,

I must be honest with you: I don't understand anything about this use case, and from where I stand, it looks like a very convoluted solution to a rather simple problem.

If you believe that an allocator would solve your problem, why don't you replace JsonDocument's allocator?
ArduinoJson already allows you to replace the allocator; you don't need me to do that.

Best regards,
Benoit

from arduinojson.

ESP-DASH (https://github.com/ayushsharma82/ESP-DASH) is a web framework working on top of websocket.

It communicates through websocket evens to send the updates and initial layout to the browser.

When the application is huge, ESP-DASH has to build a Json document that can be big (i.e. in my app, the measureJson tell a size more than 12k of data).

There is no way to stream that directly because the underlying wesocket layer works by sending packets, so the Json document has to be serialised in a buffer that will be put in a queue for sending (this is a low level library doing that)

So at this time, the heap needs to contain both the JsonDocument (which can use more than 12k), and the websocket buffer (12k). To fix that and not overflow, each time we append a new UI element to the json websocket message, we use measureJson to check the payload size and if it reaches a limit, we send it, then clear the json document for the next UI elements.

The problem is that it causes a LOT of measureJson calls, that could be avoided if we could set a cap in the memory used by a json document.

The data is user-driven, we have no control over the sizes: for example, a first part of the UI could be big in app A and small in app B. it depends on the titles, components, charts, etc.

I have tried to replaced the allocator but bumped into the issue described in #2033. This is not possible to set a limit because some API in the allocator does not allow to keep track of what is really allocated.

from arduinojson.

I'm assuming you are referring to ArduinoJson's Allocator interface not providing the block size when deallocating or reallocating. You can easily work around this limitation by prefixing each block with its size. I use this technique in the SpyingAllocator for example.

Alternatively, since your goal is to keep a safety cushion, you could check the amount of free heap before allocating a buffer.

from arduinojson.

Thanks! I didn't think about these ideas. I will try them.

from arduinojson.