Comments (14)
Thanks for reporting this issue.
To fix this issue, please update the library to v4.3.14.
from firebase-esp8266.
Thank you. I'm waiting for the new patche comes online.
Will feedback to you when I finish.
from firebase-esp8266.
Hi Mobitz.
After I update the library to 4.3.14, the problem is still there.
To be more precise, after 1st attempt of backup, it said -> Download failed,
and then from 2nd attempt, it return -> file is still opened.
Is there any specific requirement for the backup file ?
from firebase-esp8266.
it's not possible.
Uninstall library by delete from libraries folder and install the latest version from Library Manager.
from firebase-esp8266.
I tried that delete and re-installed library also, the issue still there.
Even after I manually deleted the backup file created by esp and tried to backup again, it still showed "file is still opened"
from firebase-esp8266.
In ESP8266, you should reserve the BearSSL rx buffer large enough for your data at that node to download and upload.
As this is the test result of node data size up to 10k bytes when download.
The BearSSL rx buffer size should be 10k up too (12k in this case) as following.
fbdo.setBSSLBufferSize(12000/* Rx buffer size in bytes from 512 - 16384 */, 512 /* Tx buffer size in bytes from 512 - 16384 */);
Test result of 10k data backup (download) and restore (upload)..
Connecting to Wi-Fi...........
Connected with IP: 192.168.137.30
Firebase Client v4.3.14
Token info: type = id token (GITKit token), status = on request
Token info: type = id token (GITKit token), status = ready
Downloaded 10%
Downloaded 19%
Downloaded 29%
Downloaded 39%
Downloaded 49%
Downloaded 59%
Downloaded 69%
Downloaded 79%
Downloaded 89%
Downloaded 99%
Downloaded 100%
Backup completed
backup file, /test.bak
file size, 10312
Restore...
Uploading file /test.bak (10312) to /test_backup
Uploaded 4%
Uploaded 9%
Uploaded 14%
Uploaded 19%
Uploaded 24%
Uploaded 29%
Uploaded 34%
Uploaded 39%
Uploaded 44%
Uploaded 49%
Uploaded 54%
Uploaded 59%
Uploaded 64%
Uploaded 69%
Uploaded 74%
Uploaded 79%
Uploaded 84%
Uploaded 89%
Uploaded 94%
Uploaded 99%
Uploaded 100%
Restore completed
from firebase-esp8266.
If the BearSSL rx buffer is too small to handle the data, the invalid or truncated SSL record will be stored in rx buffer and cannot decrypt it to raw (text) data then no or empty data returns from BearSSL engine.
from firebase-esp8266.
I raised the BearSSL to 12000 as you did, and successfully backup, but only with maximum of 2 child nodes, when I raised to 3 child node, Download failed, and the maximum of the backup file I got here is 294B
Kinda surprise with your test succesfully with file size up to 10kb...
Here is content of my child node, nothing too heavy apparently
from firebase-esp8266.
It's normal that library is currently supported large file download/upload (and OTA) in RTDB.
Megabytes of data download/upload tested.
You can test with this complete test code.
#include <Arduino.h>
#include <ESP8266WiFi.h>
#include <FirebaseESP8266.h>
#include <addons/TokenHelper.h>
#include <addons/RTDBHelper.h>
#define WIFI_SSID "WIFI_AP"
#define WIFI_PASSWORD "WIFI_PASSWORD"
#define API_KEY "API_KEY"
#define DATABASE_URL "URL"
#define USER_EMAIL "USER_EMAIL"
#define USER_PASSWORD "USER_PASSWORD"
FirebaseData fbdo;
FirebaseAuth auth;
FirebaseConfig config;
bool taskCompleted;
void setup()
{
Serial.begin(115200);
WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
Serial.print("Connecting to Wi-Fi");
while (WiFi.status() != WL_CONNECTED)
{
Serial.print(".");
delay(300);
}
Serial.println();
Serial.print("Connected with IP: ");
Serial.println(WiFi.localIP());
Serial.println();
Serial.printf("Firebase Client v%s\n\n", FIREBASE_CLIENT_VERSION);
config.api_key = API_KEY;
auth.user.email = USER_EMAIL;
auth.user.password = USER_PASSWORD;
config.database_url = DATABASE_URL;
config.token_status_callback = tokenStatusCallback;
Firebase.begin(&config, &auth);
Firebase.reconnectWiFi(true);
fbdo.setBSSLBufferSize(12000, 512);
}
// The Firebase download callback function
void rtdbDownloadCallback(RTDB_DownloadStatusInfo info)
{
if (info.status == fb_esp_rtdb_download_status_init)
{
Serial.printf("Downloading file %s (%d) to %s\n", info.remotePath.c_str(), info.size, info.localFileName.c_str());
}
else if (info.status == fb_esp_rtdb_download_status_download)
{
Serial.printf("Downloaded %d%s\n", (int)info.progress, "%");
}
else if (info.status == fb_esp_rtdb_download_status_complete)
{
Serial.println("Backup completed\n");
}
else if (info.status == fb_esp_rtdb_download_status_error)
{
Serial.printf("Download failed, %s\n", info.errorMsg.c_str());
}
}
// The Firebase upload callback function
void rtdbUploadCallback(RTDB_UploadStatusInfo info)
{
if (info.status == fb_esp_rtdb_upload_status_init)
{
Serial.printf("Uploading file %s (%d) to %s\n", info.localFileName.c_str(), info.size, info.remotePath.c_str());
}
else if (info.status == fb_esp_rtdb_upload_status_upload)
{
Serial.printf("Uploaded %d%s\n", (int)info.progress, "%");
}
else if (info.status == fb_esp_rtdb_upload_status_complete)
{
Serial.println("Restore completed\n");
}
else if (info.status == fb_esp_rtdb_upload_status_error)
{
Serial.printf("Upload failed, %s\n", info.errorMsg.c_str());
}
}
void loop()
{
if (Firebase.ready() && !taskCompleted)
{
taskCompleted = true;
FirebaseJson data;
// Prepare node data to test
Serial.println("\nPrepare data... ");
// Warning memory allocation error can be occurred
for (int i = 0; i < 8; i++)
{
data.set("/data/group" + String(i) + "/a", "________________________________________________________");
data.set("/data/group" + String(i) + "/b", "________________________________________________________");
data.set("/data/group" + String(i) + "/b", "________________________________________________________");
data.set("/data/group" + String(i) + "/c", "________________________________________________________");
data.set("/data/group" + String(i) + "/d", "________________________________________________________");
data.set("/data/group" + String(i) + "/e", "________________________________________________________");
data.set("/data/group" + String(i) + "/f", "________________________________________________________");
data.set("/data/group" + String(i) + "/g", "________________________________________________________");
data.set("/data/group" + String(i) + "/h", "________________________________________________________");
data.set("/data/group" + String(i) + "/i", "________________________________________________________");
data.set("/data/group" + String(i) + "/j", "________________________________________________________");
data.set("/data/group" + String(i) + "/k", "________________________________________________________");
data.set("/data/group" + String(i) + "/l", "________________________________________________________");
data.set("/data/group" + String(i) + "/m", "________________________________________________________");
}
Serial.println("\nSet data... ");
if (!Firebase.setJSON(fbdo, "/test_backup", data))
Serial.println(fbdo.errorReason());
Serial.println("\nBackup test...\n");
if (Firebase.backup(fbdo, StorageType::FLASH, "/test_backup", "/test.json", rtdbDownloadCallback))
{
Serial.printf("backup file, %s\n", fbdo.getBackupFilename().c_str());
Serial.printf("file size, %d\n", fbdo.getBackupFileSize());
}
else
Serial.println(fbdo.fileTransferError().c_str());
Serial.println("\nRestore test... \n");
if (!Firebase.restore(fbdo, StorageType::FLASH, "/test_backup", "/test.json", rtdbUploadCallback))
Serial.println(fbdo.fileTransferError().c_str());
}
}
from firebase-esp8266.
Result of the test code.
Connecting to Wi-Fi...........
Connected with IP: 192.168.137.54
Firebase Client v4.3.14
Token info: type = id token (GITKit token), status = on request
Token info: type = id token (GITKit token), status = ready
Prepare data...
Set data...
Backup test...
Downloaded 15%
Downloaded 30%
Downloaded 46%
Downloaded 61%
Downloaded 77%
Downloaded 92%
Downloaded 100%
Backup completed
backup file, /test.json
file size, 6650
Restore test...
Uploading file /test.json (6650) to /test_backup
Uploaded 7%
Uploaded 15%
Uploaded 23%
Uploaded 30%
Uploaded 38%
Uploaded 46%
Uploaded 53%
Uploaded 61%
Uploaded 69%
Uploaded 76%
Uploaded 84%
Uploaded 92%
Uploaded 99%
Uploaded 100%
Restore completed
from firebase-esp8266.
Hi Mobitz.
I tried your code and yes, it can download up to 12kB content ( with your sample, up to group 13).
I will try again get it to work with my code.
Another irrelevant issue, the stream pause + stream.clear() you showed me in previous topic to gain memory, I couldn't get it resumed, for fbdo pause and resume, it worked.
from firebase-esp8266.
I case of stream, the pointers to callback functions were removed after using stream.clear();.
To resume the callback, you can set the callback function again with following.
fbdo.pauseFirebase(false);
stream.pauseFirebase(false);
Firebase.setStreamCallback(stream, streamCallback, streamTimeoutCallback);
FYI
When ESP8266 core update is available (newer than current release v3.1.2), stream time out checking will be better (nearly real-time server connection tracking) as it relies on TCP Keepalive by calling stream.keepAlive
as in the new stream examples.
from firebase-esp8266.
Sorry for a late feedback, I successfully set the backup with raising the BearSSL Buffersize.
My NodeMCU get overflowed when I tried to set those buffer size to12k-512b for fbdo and 5k-1024 for stream, then I lower the fbdo to 10k, seems OK now. Can these settings can be change schedulely for backup node ? (raise before and lower after task ?)
Thanks for the suggestion about stream resuming, will test it asap, my daily work get busier and can only touch the esp less than 1 hour/day
from firebase-esp8266.
Yes, you can change BearSSL
buffer size to any value before opening the session (starting new server connection).
The new BearSSL
buffer memory allocation with new size will take effect when current or last session was closed and re-open.
Note that, the session will keep alive (reuse) in this library to reduce the SSL handshake waiting time during connecting to server.
To close current session, use fbdo.clear()
.
from firebase-esp8266.
Related Issues (20)
- ESP8266 is crashing while writing to firebase realtime database HOT 5
- soft wdt reset issue in esp8266 HOT 1
- I'm trying to read this data HOT 2
- WiFiClientSecure issue HOT 6
- Get Token status HOT 1
- BUG Token info: type = id token (GITKit token), status = on request HOT 1
- PLEASE HELP HOT 3
- [HELP] Cannot connect esp8266 and firebase. Compilation error: invalid conversion from 'int' to 'FirebaseConfig*' {aka 'firebase_cfg_t*'} [-fpermissive] HOT 1
- I've been getting a connection refused after a few seconds
- I've been getting a connection refused after a few seconds after connection to firebase HOT 1
- HELP : Executed HTTPS and connection Firebase HOT 2
- HELP :: captivePortal and firebase connection HOT 4
- BUG FirebaseESP8266.h with version @^4.4.12 of Library HOT 4
- FirebaseJson is confilcted with json inside ESP8266WebServer library HOT 6
- HELP
- Content Available for FCM_HTTPv1_JSON_Message HOT 1
- Unable to Fetch Data from Firebase
- Wi-Fi Reset Doesn't Initialize Firebase Variables and LED Status Doesn't Update Post-Reconnection HOT 6
- ERROR CODE: -3, REASON: NOT CONNECTED STREAM TIMED OUT, RESUMING... HOT 8
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from firebase-esp8266.