hamishcunningham / fishy-wifi Goto Github PK

Hello,

I'm very new to Lua so I may be being stupid, but:

I'm running nodemcu on an esp8266, with a DHT22 for temperature/humidity and a TSL2561 for light levels. I grabbed a copy of your TSL2561 library (thanks!), but I found I couldn't run that and the DHT22 code, because the esp8266 ran out of heap when require()-ing the second lib.

I tried setting the variable that I'd loaded your lib into to nil, and package.loaded["lux"] = nil, and then also package.loaded["tsl2561lib"] = nil and package.loaded["i2cutils"] = nil as well, but still a lot of heap space wasn't being returned.

So then I altered your library to use local variables wherever possible, and that seems to have solved the problem.

Is this a valid change, or am I missing something? (Probably I am...)

Thanks,
Rob.

The issue

LoRa is quickly becoming the de-facto communication standard for IoT devices. Because it allows for communication over greater distances and uses less power than WiFi, it would be a very good feature for the Elf to possess. This is especially true if you wanted the Elf running in a remote location with limited power capabilities.

Main Goals

• Establish a test point-to-point LoRa connection
• Build a gateway device
• Send collected data to the gateway
• Have the gateway registered on TTN

The ultrasonic distance sensor used in the WaterElf project produces "dirty" readings - while it can produce usable readings, the readings are also interspersed with zeroes and nonsensical numbers.

I can look at cleaning up the readings that come from the ultrasonic sensor (discarding nonsense and zeroes). I can also produce some functions to interpret the data from other sensors (mean, standard deviation... trend? - "Oh no, water level changed more quickly than usual, send a text")

Implement captive portal in joinme, and use joinme in the main waterelf32 sketch.

A decent portion of the required code appears to already be in place, but large chunks of code are commented out, and joinme is never used by waterelf32, so the file at least needs refactoring/cleanup and needs to be implemented in the main sketch.

Collect and analyze the data from the Ultrasonic Sensor:

• Set up notifications from IFTTT, when something is wrong (e.g. not enough water in the bucket)
• Probably do the same thing with other sensors.
• Possibly collect the daily or hourly data and log them to Adafruit.io?

I'm using my TSL2561 outside in a greenhouse, and with the integration time set to 402ms the levels were hitting the end of the scale (65535), even if I turned off the 16x gain.

I could have just switched to 101ms to get the full range of brightness, but then the maximum output value becomes 37177, so some granularity is lost.

So instead I've changed the code to try four different combinations of gain on/off and integration time, and choose the first that isn't out of range. It's a bit slower, but I'm only sampling once every five minutes.

What do you think?

function M.getchannels()
  local rc0,rc1 = nil,nil
  local dev_addr = M.TSL2561_ADDR_FLOAT 
  i2c.setup(busid, SDA_PIN , SCL_PIN, i2c.SLOW)

  -- 16 = fixed-point fudge factor

  -- nom,denom = 16 * (322/322), 16
  settimegain(dev_addr, M.TSL2561_INTEGRATIONTIME_402MS, M.TSL2561_GAIN_16X)

  enable(dev_addr)
  tmr.delay(500000)
  local ch0,ch1 = getFullLuminosity(dev_addr)
  disable(dev_addr)

  if rc0 == nil and ch0 < 65535 then
    rc0 = ch0
  end
  if rc1 == nil and ch1 < 65535 then
    rc1 = ch1
  end

  if ( rc0 == nil or rc1 == nil) then
    -- nom,denom = 16 * (322/81), 16
    -- nom,denom = 322,81
    settimegain(dev_addr, M.TSL2561_INTEGRATIONTIME_101MS, M.TSL2561_GAIN_16X)

    enable(dev_addr)
    tmr.delay(500000)
    ch0,ch1 = getFullLuminosity(dev_addr)
    disable(dev_addr)

    if rc0 == nil and ch0 < 37177 then
      rc0 = ch0*322/81
    end
    if rc1 == nil and ch1 < 37177 then
      rc1 = ch1*322/81
    end

    if ( rc0 == nil or rc1 == nil) then
      -- nom,denom = 16 * (322/322), 1
      -- nom,denom = 16,1
      settimegain(dev_addr, M.TSL2561_INTEGRATIONTIME_402MS, M.TSL2561_GAIN_1X)

      enable(dev_addr)
      tmr.delay(500000)
      ch0,ch1 = getFullLuminosity(dev_addr)
      disable(dev_addr)

      if rc0 == nil and ch0 < 65535 then
        rc0 = ch0*16
      end
      if rc1 == nil and ch1 < 65535 then
        rc1 = ch1*16
      end

      if ( rc0 == nil or rc1 == nil) then
        -- nom,denom = 16 * (322/81), 1
        -- nom,denom = 5152,81
        settimegain(dev_addr, M.TSL2561_INTEGRATIONTIME_101MS, M.TSL2561_GAIN_1X)

        enable(dev_addr)
        tmr.delay(500000)
        ch0,ch1 = getFullLuminosity(dev_addr)
        disable(dev_addr)

        if rc0 == nil then
          rc0 = ch0*5152/81
        end
        if rc1 == nil then
          rc1 = ch1*5152/81
        end
      end
    end
  end
  return rc0,rc1
end

In theory this outputs values in the range 0 to 2,364,640 although even with a bright torch focused directly on the sensor I couldn't get the numbers anywhere near that high.

ps. Watch out for the TSL2561_GAIN_1X constant that's been renamed from TSL2561_GAIN_0X

pps. I didn't bother with the 13.7ms integration time - see my explanation here: http://ideasandbox.blogspot.co.uk/2015/06/tsl2561-light-sensor-ranges.html

Once SPIFFS is implemented, CSS and Favicon customisation will be much easier to implement.

Wiring up the board the waterelf32 sketch does not seem to be reading the correct pin numbers for the following sensors:

• Light sensor
• Ultrasonic sensor
• Water temperature

The humidity sensor is setup with correct pins as of c7e3cb4

Your script is great and it works which is far more than i managed to achieve with me own poor skills.

Got one request. Could it give you the IP that it receives,.. so that then you can address it?

I've tried lots of things... but nothing seems to work.. the following just gives null! and then fails to call the conn:on("sent", finish).

ip = wifi.sta.getip()
tmr.delay(5000000)
print(ip)

How would you feel about calling some form of wifi watchdog that would start AP if there was no connection to wifi. Otherwise turn of AP mode for security reasons?

This is not really an issue, but might be changed to reflect facts of the mentioned board and not suggest useless wirings on a proto board?!

IMHO the following points are obsolete with the used Olimex board. I will comment why and reference their board schematics...

• need gpio 15 low and cp_en high

• GPIO15 is to be found on pad 19 of the MOD-WIFI-ESP8266-DEV, which by the schematic is already pulled to GND by the default solder bridge (called MTDO on the board). So no special wiring for this is needed.
• CHIP_EN is also pulled up to 3.3V using a 12k resistor already. Noting this is nice, but also no wiring needed for this to be pulled high.

• to flash hold gpio0 low -- sometimes I think it doesn't float high so might try using 4.7k resistor to connect it to high?

GPIO0 on pin 21 of the devboard is also pulled up by a solder bridge, but seems to be able to be pulled down to GND by simply connecting pad 21 to GND (as you did). Reading the schematic shows it is pulled high over an 2K resistor already.

• to get normal mode after flashing perhaps most useful

Should be accomplished by letting pad 21 off from GND again. Maybe add it to your DIP switch row as well!?

Save ESP32 memory by putting configuration parameter in ESP32 on-board flash chip
Parameters included:

• HTML Element
• Pin Number
• Constant Sensor value
• ???CSS/Favicon???

Benefits:

• Create easy to read code (Long HTML element)
• Save precious memory of ESP32
• Configuration can be saved on-board (i.e. Remember Last Connected SSID)

Additionally, modern web browsers accept compressed files as a response, compressing the HTML text file (.gz) could create more space on SPIFFS.

I fully cloned your Github repository onto my local computer and tried building NodeMCU with "make nodemcu-with-dns".

This is the error I received:

make nodemcu-with-dns
cd /root/esp8266-stuff/nodemcu-firmware; PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/root/esp8266-stuff/esp-open-sdk-compiler/xtensa-lx106-elf/bin make UNIVERSAL_TARGET_DEFINES=-DUSE_DNS
make[1]: Betrete Verzeichnis '/root/esp8266-stuff/nodemcu-firmware'
make[2]: Betrete Verzeichnis '/root/esp8266-stuff/nodemcu-firmware/app'
make[3]: Betrete Verzeichnis '/root/esp8266-stuff/nodemcu-firmware/app/user'
DEPEND: xtensa-lx106-elf-gcc -M -Os -Os -ffunction-sections -fno-jump-tables -fdata-sections -g -O2 -Wpointer-arith -Wundef -Werror -Wl,-EL -fno-inline-functions -nostdlib -mlongcalls -mtext-section-literals -DUSE_DNS -D__ets__ -DICACHE_FLASH -DLWIP_OPEN_SRC -DPBUF_RSV_FOR_WLAN -DEBUF_LWIP -I include -I ./ -I ../../include/ets -I ../libc -I ../platform -I ../lua -I ../wofs -I ../include -I ./ -I ../../include -I ../../include/eagle user_main.c
/root/esp8266-stuff/esp-open-sdk-compiler/xtensa-lx106-elf/bin/xtensa-lx106-elf-gcc: 1: �ELF����: not found
/root/esp8266-stuff/esp-open-sdk-compiler/xtensa-lx106-elf/bin/xtensa-lx106-elf-gcc: 2: @���@@@@@0�0��p�p�@p�@����@@,�
,�
: not found
/root/esp8266-stuff/esp-open-sdk-compiler/xtensa-lx106-elf/bin/xtensa-lx106-elf-gcc: 3: Syntax error: word unexpected (expecting ")")
xtensa-lx106-elf-gcc -Os -Os -ffunction-sections -fno-jump-tables -fdata-sections -g -O2 -Wpointer-arith -Wundef -Werror -Wl,-EL -fno-inline-functions -nostdlib -mlongcalls -mtext-section-literals -DUSE_DNS -D__ets__ -DICACHE_FLASH -DLWIP_OPEN_SRC -DPBUF_RSV_FOR_WLAN -DEBUF_LWIP -I include -I ./ -I ../../include/ets -I ../libc -I ../platform -I ../lua -I ../wofs -I ../include -I ./ -I ../../include -I ../../include/eagle -o .output/eagle/debug/obj/user_main.o -c user_main.c
/root/esp8266-stuff/esp-open-sdk-compiler/xtensa-lx106-elf/bin/xtensa-lx106-elf-gcc: 1: �ELF����: not found
/root/esp8266-stuff/esp-open-sdk-compiler/xtensa-lx106-elf/bin/xtensa-lx106-elf-gcc: 2: @���@@@@@0�0��p�p�@p�@����@@,�
,�
: not found
/root/esp8266-stuff/esp-open-sdk-compiler/xtensa-lx106-elf/bin/xtensa-lx106-elf-gcc: 3: Syntax error: word unexpected (expecting ")")
make[3]: *** [.output/eagle/debug/obj/user_main.o] Fehler 2
make[3]: Verlasse Verzeichnis '/root/esp8266-stuff/nodemcu-firmware/app/user'
make[2]: *** [.subdirs] Fehler 2
make[2]: Verlasse Verzeichnis '/root/esp8266-stuff/nodemcu-firmware/app'
make[1]: *** [.subdirs] Fehler 2
make[1]: Verlasse Verzeichnis '/root/esp8266-stuff/nodemcu-firmware'
make: *** [nodemcu-with-dns] Fehler 2

I asumed that maybe my xtensa binarys were somehow corrupt. But exchanging them didn't resolve the error.

I hope you can help me out.

Use SSL in joinme to secure AP credentials.

Looks like this will require switching from ESPWebServer to ESPWebServerAsync as ESPWebServer has no SSL capabilities.

A big question is how to get an SSL cert signed by a root CA for each device to avoid triggering warnings from browsers.

One option may be to use captive portal from #11 to masquerade as some domain in the form of esp<mac address>.<some TLD we control> ( e.g. esp240ac40c1f79.wegrow.social ) which would allow one cert per device, and let us use certs issued for that subdomain from a service such as letsencrypt.
However this would not allow SSL once joined to a network unless we are the primary DNS server ( we could create a DNS record for each subdomain pointing to the devices local IP if it is static, however this might interfere with cert issuing ).
Another issue is the matter of renewing SSL certs, especially if using letsencrypt or similar services as their certs expire in 90 days.

Another, simpler, option is to issue a self signed cert to each device when it is provisioned, and get users to add the root cert to their device(s). However getting users to add our root cert is likely to be a pain point, and without adding our root the user will see a browser warning page.

In general the problem of acquiring and maintaining SSL certs on IoT devices appears to be currently unsolved, and no CAs provide a simple/supported way to do so.

Please comment with any suggestions on cert management you have.

As noted in https://www.sparkfun.com/datasheets/Sensors/Temperature/DHT22.pdf, the DHT22 has an average sensing period of 2s.

The function startPeripherals will occasionally not detect the DHT22 as a reading has not yet been made before initialising the sensor with dht.begin():

Adding a delay(2000) before line 816 seems to work as a fix for this.

Issue for refactoring https://github.com/hamishcunningham/fishy-wifi/blob/c7e3cb415f0ee15786c1d1fe610efdc433e5f889/ardesp/waterelf32/waterelf32.ino

Problem

The pins for SDA and SCL on the esp32 are 23 and 22. Calling Wire.begin with no args seems to use incorrect pins for the esp32. This results in the TSL2591 never being found in startPeripherals:

Furthermore the Adafruit_TSL2591_Library's function tsl.begin() calls Wire.begin with no args, making it incompatible with the esp32.

Example

Running the example tsl2591.ino results in:

[W][esp32-hal-i2c.c:231] i2cWrite(): Ack Error! Addr: 29
[W][esp32-hal-i2c.c:334] i2cRead(): Ack Error! Addr: 29
No sensor found ... check your wiring?

Possible Fix

Replacing all Wire.begin() with Wire.begin(23, 22) in Adafruit_TSL2591_Library and waterelf32.ino.

After applying fix, tsl2591.ino can be run with the ESP32.

Problems with 'Fix'

After applying the fix the waterelf32.ino reports errors with addresses 20 and 29 being busy.

startPeripherals...
[W][esp32-hal-i2c.c:231] i2cWrite(): Ack Error! Addr: 4e
Soft AP started
AP SSID: WaterElf-BC130CC40A24; IP address(es): local=0.0.0.0; AP=192.168.99.1
[D][WiFiGeneric.cpp:258] _eventCallback(): Event: 13 - AP_STOP
[D][WiFiGeneric.cpp:258] _eventCallback(): Event: 13 - AP_STOP
HTTP server started
doing flow controller and mcp init...
[E][esp32-hal-i2c.c:465] i2cInitFix(): Busy at initialization!
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 20
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 20
[W][esp32-hal-i2c.c:263] i2cRead(): Busy Timeout! Addr: 20
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 20
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 20
[W][esp32-hal-i2c.c:263] i2cRead(): Busy Timeout! Addr: 20
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 20
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 20
[W][esp32-hal-i2c.c:263] i2cRead(): Busy Timeout! Addr: 20
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 20
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 20
[W][esp32-hal-i2c.c:263] i2cRead(): Busy Timeout! Addr: 20
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 20
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 20
[W][esp32-hal-i2c.c:263] i2cRead(): Busy Timeout! Addr: 20
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 20
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 20
[W][esp32-hal-i2c.c:263] i2cRead(): Busy Timeout! Addr: 20
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 20
Air Temp: 26.20 C, Humidity: 25.90 %RH, 
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[W][esp32-hal-i2c.c:263] i2cRead(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[W][esp32-hal-i2c.c:263] i2cRead(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[W][esp32-hal-i2c.c:263] i2cRead(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[W][esp32-hal-i2c.c:263] i2cRead(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
Light: 0 Lux
Water Level: 103 cm, 
Water Level: 74 cm, 
Water Level: 75 cm, 
getAnalog
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 20
[W][esp32-hal-i2c.c:263] i2cRead(): Busy Timeout! Addr: 20
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 20
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 20
[W][esp32-hal-i2c.c:263] i2cRead(): Busy Timeout! Addr: 20
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 20
free heap=140748
Air Temp: 26.20 C, Humidity: 25.90 %RH, 
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[W][esp32-hal-i2c.c:263] i2cRead(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[W][esp32-hal-i2c.c:263] i2cRead(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[W][esp32-hal-i2c.c:263] i2cRead(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[W][esp32-hal-i2c.c:263] i2cRead(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
Light: 0 Lux
Water Level: 103 cm, 
Water Level: 105 cm, 
Water Level: 70 cm, 
getAnalog
free heap=140748
Air Temp: 26.20 C, Humidity: 25.90 %RH, 
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[W][esp32-hal-i2c.c:263] i2cRead(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[W][esp32-hal-i2c.c:263] i2cRead(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[W][esp32-hal-i2c.c:263] i2cRead(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[W][esp32-hal-i2c.c:263] i2cRead(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
Light: 0 Lux
Water Level: 103 cm, 
Water Level: 96 cm, 
Water Level: 122 cm, 
getAnalog
free heap=140752
Air Temp: 26.20 C, Humidity: 25.90 %RH, 
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[W][esp32-hal-i2c.c:263] i2cRead(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[W][esp32-hal-i2c.c:263] i2cRead(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[W][esp32-hal-i2c.c:263] i2cRead(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[W][esp32-hal-i2c.c:263] i2cRead(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
Light: 0 Lux
Water Level: 103 cm, 
Water Level: 91 cm, 
Water Level: 100 cm, 
getAnalog
free heap=140752
Air Temp: 26.20 C, Humidity: 25.90 %RH, 
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[W][esp32-hal-i2c.c:263] i2cRead(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[W][esp32-hal-i2c.c:263] i2cRead(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[W][esp32-hal-i2c.c:263] i2cRead(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
[W][esp32-hal-i2c.c:263] i2cRead(): Busy Timeout! Addr: 29
[E][esp32-hal-i2c.c:160] i2cWrite(): Busy Timeout! Addr: 29
Light: 0 Lux
Water Level: 103 cm, 
Water Level: 117 cm, 
Water Level: 78 cm, 
getAnalog
free heap=140396