Tiny request for documentation: how to return a large PROGMEM char*? about espasyncwebserver HOT 21 CLOSED

And in answer to my own question: would it make sense to tell the callback the amount of data already written? Then we'd have an index in case we need it...

from espasyncwebserver.

the best way to send such content is to use the chunked response.

const PROGMEM char html[] = "...";
void respond(AsyncRequest *request){
  AsyncWebServerResponse *response = request->beginChunkedResponse("text/plain", [](uint8_t *buffer, size_t maxLen) -> size_t {
    static size_t index = 0;
    size_t htmlLen = strlen(html);
    size_t leftToWrite = htmlLen - index;
    if(! leftToWrite)
     return 0;//end of transfer
    size_t willWrite = (leftToWrite > maxLen)?maxLen:leftToWrite;
    memcpy(buffer, html+index, willWrite);
    index += willWrite;
    return willWrite;
  });
  request->send(response);
}

something like that :)

from espasyncwebserver.

hmmm, it can't be that easy: http://stackoverflow.com/questions/6223355/static-variables-in-class-methods

because if it is, i'd still use the callback, because I know the size of the HTML and the browser will close the connection as soon as content_length bytes are received...

from espasyncwebserver.

either way you need to use one f the two (callback or chunked) to send that data. It will not otherwise let you allocate all of the 5K html. It's the total size that is the issue and the fact that the ESP will let you alloc only up to 4K for some reason.

from espasyncwebserver.

ok, assuming the static variable is separate for each lambda function instanciated (I'm a bloody c++ n00b), with the following it seems to work (with c strings that is):

  AsyncWebServerResponse *response = request->beginResponse("text/html", strlen_P(indexhtml),
    [](uint8_t *buffer, size_t maxLen) -> size_t {
      static PGM_P currentPosition=indexhtml;
      if (strlen_P(currentPosition)>maxLen) {
        // We have more to read than fits in maxLen Buffer
        memcpy_P((char*)buffer, currentPosition, maxLen);
        currentPosition += maxLen;
        return maxLen;
      } else {
        // Last chunk, buffer will not be filled...
        memcpy_P((char*)buffer, currentPosition, strlen_P(currentPosition));
        return strlen_P(currentPosition); // Return from here to end of indexhtml
      }
    }
  );
  response->addHeader("Server", PMS_DEVICE_ID);
  request->send(response);  

Still one or two tiny bugs though, see wireshark:

1. http header is sent separately (by former convention I presume, poison for windows delayed ack :) ) in a small packet
2. the last two packets should be sent in one go as well? Why is there a delay between 1514 and 404?

from espasyncwebserver.

oh, strike that. The static function is not destroyed if the request is done. I'll get the last chunk over and over with each subsequent request :) So, static is not a good idea...

from espasyncwebserver.

maybe a custom response class would do here (the progress will be a class member so will not be an issue).
Check the callback/chunked response implementations (they are short). Will be easy to do that way.
To fill the first packet with what is available, the AbstractResponse class needs to be edited. Have to see where and what exactly.

from espasyncwebserver.

aalll right, I did the changes, tested it, works. Wanted to create a pull request (by pushing my new local branch to this repo), but Git says access denied? Is there a comprehensive tutorial on creating pull requests?

from espasyncwebserver.

Sorry, I still didn't manage to create a pull request. in any case, here's the diff

diff --git a/src/AsyncWebServerResponseImpl.h b/src/AsyncWebServerResponseImpl.h
index ca13ee3..0fe9657 100644
--- a/src/AsyncWebServerResponseImpl.h
+++ b/src/AsyncWebServerResponseImpl.h
@@ -51,6 +51,7 @@ class AsyncStreamResponse: public AsyncAbstractResponse {
 class AsyncCallbackResponse: public AsyncAbstractResponse {
   private:
     AwsResponseFiller _content;
+    size_t _contentLengthSent;
   public:
     AsyncCallbackResponse(String contentType, size_t len, AwsResponseFiller callback);
     bool _sourceValid(){ return !!(_content); }
@@ -60,6 +61,7 @@ class AsyncCallbackResponse: public AsyncAbstractResponse {
 class AsyncChunkedResponse: public AsyncAbstractResponse {
   private:
     AwsResponseFiller _content;
+    size_t _contentLengthSent;
   public:
     AsyncChunkedResponse(String contentType, AwsResponseFiller callback);
     bool _sourceValid(){ return !!(_content); }
diff --git a/src/ESPAsyncWebServer.h b/src/ESPAsyncWebServer.h
index 8b228eb..6fee84a 100644
--- a/src/ESPAsyncWebServer.h
+++ b/src/ESPAsyncWebServer.h
@@ -76,7 +76,7 @@ class AsyncWebHeader {
  * REQUEST :: Each incoming Client is wrapped inside a Request and both live together until disconnect
  * */

-typedef std::function<size_t(uint8_t*, size_t)> AwsResponseFiller;
+typedef std::function<size_t(uint8_t*, size_t, size_t)> AwsResponseFiller;

 class AsyncWebServerRequest {
   private:
diff --git a/src/WebResponses.cpp b/src/WebResponses.cpp
index de673f0..288cf2d 100644
--- a/src/WebResponses.cpp
+++ b/src/WebResponses.cpp
@@ -375,13 +375,16 @@ AsyncCallbackResponse::AsyncCallbackResponse(String contentType, size_t len, Aws
   _code = 200;
   _content = callback;
   _contentLength = len;
+  _contentLengthSent = 0; //Nothing sent yet
   if(!len)
     _sendContentLength = false;
   _contentType = contentType;
 }

 size_t AsyncCallbackResponse::_fillBuffer(uint8_t *data, size_t len){
-  return _content(data, len);
+  size_t sentBytes=_content(data, len, _contentLengthSent);
+  _contentLengthSent+=sentBytes;
+  return sentBytes;
 }

 /*
@@ -392,13 +395,16 @@ AsyncChunkedResponse::AsyncChunkedResponse(String contentType, AwsResponseFiller
   _code = 200;
   _content = callback;
   _contentLength = 0;
+  _contentLengthSent = 0; //Nothing sent yet
   _contentType = contentType;
   _sendContentLength = false;
   _chunked = true;
 }

 size_t AsyncChunkedResponse::_fillBuffer(uint8_t *data, size_t len){
-  return _content(data, len);
+  size_t sentBytes=_content(data, len, _contentLengthSent);
+  _contentLengthSent+=sentBytes;
+  return sentBytes;
}

and example code:

const char indexhtml[] PROGMEM = "..."; // large char array, tested with 5k
AsyncWebServerResponse *response = request->beginResponse(
  String("text/html"),
  strlen_P(indexhtml),
  [](uint8_t *buffer, size_t maxLen, size_t alreadySent) -> size_t {
    if (strlen_P(indexhtml+sentCounter)>maxLen) {
      // We have more to read than fits in maxLen Buffer
      memcpy_P((char*)buffer, indexhtml+sentCounter, maxLen);
      return maxLen;
    }
    // Ok, last chunk
    memcpy_P((char*)buffer, indexhtml+sentCounter, strlen_P(indexhtml+sentCounter));
    return strlen_P(indexhtml+sentCounter); // Return from here to end of indexhtml
  }
);
response->addHeader("Server", "MyServerString");
request->send(response);  

Works like a charm... *edit: Thanks for all your good work, of course ;)

from espasyncwebserver.

Thanks! I'll merge the changes. As for the merge, did you fork the repo or did you clone it? Because you need to fork it in order to be able to create pull requests :)

from espasyncwebserver.

can you please try the latest changes :) the sent length was already there it just needed some massaging for chunked responses.

from espasyncwebserver.

whoops, small typo (sentCounter vs. alreadySent) for README.md :)

const char indexhtml[] PROGMEM = "..."; // large char array, tested with 5k
AsyncWebServerResponse *response = request->beginResponse(
  String("text/html"),
  strlen_P(indexhtml),
  [](uint8_t *buffer, size_t maxLen, size_t alreadySent) -> size_t {
    if (strlen_P(indexhtml+alreadySent)>maxLen) {
      // We have more to read than fits in maxLen Buffer
      memcpy_P((char*)buffer, indexhtml+alreadySent, maxLen);
      return maxLen;
    }
    // Ok, last chunk
    memcpy_P((char*)buffer, indexhtml+alreadySent, strlen_P(indexhtml+alreadySent));
    return strlen_P(indexhtml+alreadySent); // Return from here to end of indexhtml
  }
);
response->addHeader("Server", "MyServerString");
request->send(response);  

testing changes now...

from espasyncwebserver.

Nope, doesn't seem to work. opened new issue #14

from espasyncwebserver.

Oh and second ament to comment above: Is is really a "chunked" response? The contentLength is set, so I'd call it callback in README.md, just to not confuse people...

from espasyncwebserver.

I screwed something up... in something I did not even touch... reverting and hunting :)

from espasyncwebserver.

no worries, but let's reopen this then until stuff works again...

from espasyncwebserver.

alright give it a go again :)

from espasyncwebserver.

yep, compiles, no errors so far. readme looks fine also. I think this is solved. Thanks man, really appreciate this.

from espasyncwebserver.

hey no problem :) i think having the sent index is good thing. no clue how I overlooked to add it :)

from espasyncwebserver.

oh terribly sorry to have been too fast, still not all sentCounters replaced in readme.md.

from espasyncwebserver.

yeah have that on my end :) will commit with next changes

from espasyncwebserver.