Giter Club home page Giter Club logo

meshview's Introduction

Meshview
========

This project watches a MQTT topic for meshtastic messages, imports them to a
database and has a web UI to view them.

Example
-------
An example instance, https://meshview.armooo.net, is running with with data
from the MQTT topic msh/US/bayarea/#.


Running
-------
$ python3 -m venv env
$ ./env/bin/pip install -r requirements.txt
$ ./env/bin/python main.py

Now you can hit http://localhost:8080/

meshview's People

Contributors

armooo avatar

Stargazers

Johannes le Roux avatar Brian Leschke avatar Dominik Antal avatar avatar avatar Jon Davis avatar Matthew Davies avatar

Watchers

avatar Jon Davis avatar avatar

meshview's Issues

not receiving any mqtt packets

Hi meshview team,

Here is the process I follow to install meshview ...

johannes@gamesrv:~$ git clone https://github.com/armooo/meshview.git
Cloning into 'meshview'...
remote: Enumerating objects: 564, done.
remote: Counting objects: 100% (109/109), done.
remote: Compressing objects: 100% (79/79), done.
remote: Total 564 (delta 53), reused 60 (delta 30), pack-reused 455 (from 1)
Receiving objects: 100% (564/564), 239.34 KiB | 2.92 MiB/s, done.
Resolving deltas: 100% (361/361), done.
johannes@gamesrv:~$ cd meshview/
johannes@gamesrv:~/meshview$ python3 -m venv env
johannes@gamesrv:~/meshview$ ./env/bin/pip install -r requirements.txt
Collecting protobuf (from -r requirements.txt (line 1))
  Using cached protobuf-5.27.3-cp38-abi3-manylinux2014_x86_64.whl.metadata (592 bytes)
Collecting aiomqtt (from -r requirements.txt (line 2))
  Using cached aiomqtt-2.3.0-py3-none-any.whl.metadata (5.2 kB)
Collecting cryptography (from -r requirements.txt (line 4))
  Using cached cryptography-43.0.0-cp39-abi3-manylinux_2_28_x86_64.whl.metadata (5.4 kB)
Collecting aiosqlite (from -r requirements.txt (line 5))
  Using cached aiosqlite-0.20.0-py3-none-any.whl.metadata (4.3 kB)
Collecting aiohttp (from -r requirements.txt (line 6))
  Using cached aiohttp-3.10.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (7.5 kB)
Collecting aiodns (from -r requirements.txt (line 7))
  Using cached aiodns-3.2.0-py3-none-any.whl.metadata (4.0 kB)
Collecting Jinja2 (from -r requirements.txt (line 8))
  Using cached jinja2-3.1.4-py3-none-any.whl.metadata (2.6 kB)
Collecting aiohttp-sse (from -r requirements.txt (line 9))
  Using cached aiohttp_sse-2.2.0-py3-none-any.whl.metadata (3.7 kB)
Collecting asyncpg (from -r requirements.txt (line 10))
  Using cached asyncpg-0.29.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.4 kB)
Collecting seaborn (from -r requirements.txt (line 11))
  Using cached seaborn-0.13.2-py3-none-any.whl.metadata (5.4 kB)
Collecting pydot (from -r requirements.txt (line 12))
  Using cached pydot-3.0.1-py3-none-any.whl.metadata (9.9 kB)
Collecting plotly (from -r requirements.txt (line 13))
  Using cached plotly-5.23.0-py3-none-any.whl.metadata (7.3 kB)
Collecting sqlalchemy[asyncio] (from -r requirements.txt (line 3))
  Using cached SQLAlchemy-2.0.32-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (9.6 kB)
Collecting paho-mqtt<3.0.0,>=2.1.0 (from aiomqtt->-r requirements.txt (line 2))
  Using cached paho_mqtt-2.1.0-py3-none-any.whl.metadata (23 kB)
Collecting typing-extensions>=4.6.0 (from sqlalchemy[asyncio]->-r requirements.txt (line 3))
  Using cached typing_extensions-4.12.2-py3-none-any.whl.metadata (3.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy[asyncio]->-r requirements.txt (line 3))
  Using cached greenlet-3.0.3-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.8 kB)
Collecting cffi>=1.12 (from cryptography->-r requirements.txt (line 4))
  Using cached cffi-1.17.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.5 kB)
Collecting aiohappyeyeballs>=2.3.0 (from aiohttp->-r requirements.txt (line 6))
  Using cached aiohappyeyeballs-2.4.0-py3-none-any.whl.metadata (5.9 kB)
Collecting aiosignal>=1.1.2 (from aiohttp->-r requirements.txt (line 6))
  Using cached aiosignal-1.3.1-py3-none-any.whl.metadata (4.0 kB)
Collecting attrs>=17.3.0 (from aiohttp->-r requirements.txt (line 6))
  Using cached attrs-24.2.0-py3-none-any.whl.metadata (11 kB)
Collecting frozenlist>=1.1.1 (from aiohttp->-r requirements.txt (line 6))
  Using cached frozenlist-1.4.1-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (12 kB)
Collecting multidict<7.0,>=4.5 (from aiohttp->-r requirements.txt (line 6))
  Using cached multidict-6.0.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.2 kB)
Collecting yarl<2.0,>=1.0 (from aiohttp->-r requirements.txt (line 6))
  Using cached yarl-1.9.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting pycares>=4.0.0 (from aiodns->-r requirements.txt (line 7))
  Using cached pycares-4.4.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.1 kB)
Collecting MarkupSafe>=2.0 (from Jinja2->-r requirements.txt (line 8))
  Using cached MarkupSafe-2.1.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.0 kB)
Collecting numpy!=1.24.0,>=1.20 (from seaborn->-r requirements.txt (line 11))
  Using cached numpy-2.1.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (60 kB)
Collecting pandas>=1.2 (from seaborn->-r requirements.txt (line 11))
  Using cached pandas-2.2.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (19 kB)
Collecting matplotlib!=3.6.1,>=3.4 (from seaborn->-r requirements.txt (line 11))
  Using cached matplotlib-3.9.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (11 kB)
Collecting pyparsing>=3.0.9 (from pydot->-r requirements.txt (line 12))
  Using cached pyparsing-3.1.2-py3-none-any.whl.metadata (5.1 kB)
Collecting tenacity>=6.2.0 (from plotly->-r requirements.txt (line 13))
  Using cached tenacity-9.0.0-py3-none-any.whl.metadata (1.2 kB)
Collecting packaging (from plotly->-r requirements.txt (line 13))
  Using cached packaging-24.1-py3-none-any.whl.metadata (3.2 kB)
Collecting pycparser (from cffi>=1.12->cryptography->-r requirements.txt (line 4))
  Using cached pycparser-2.22-py3-none-any.whl.metadata (943 bytes)
Collecting contourpy>=1.0.1 (from matplotlib!=3.6.1,>=3.4->seaborn->-r requirements.txt (line 11))
  Using cached contourpy-1.2.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.8 kB)
Collecting cycler>=0.10 (from matplotlib!=3.6.1,>=3.4->seaborn->-r requirements.txt (line 11))
  Using cached cycler-0.12.1-py3-none-any.whl.metadata (3.8 kB)
Collecting fonttools>=4.22.0 (from matplotlib!=3.6.1,>=3.4->seaborn->-r requirements.txt (line 11))
  Using cached fonttools-4.53.1-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (162 kB)
Collecting kiwisolver>=1.3.1 (from matplotlib!=3.6.1,>=3.4->seaborn->-r requirements.txt (line 11))
  Using cached kiwisolver-1.4.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.4 kB)
Collecting pillow>=8 (from matplotlib!=3.6.1,>=3.4->seaborn->-r requirements.txt (line 11))
  Using cached pillow-10.4.0-cp312-cp312-manylinux_2_28_x86_64.whl.metadata (9.2 kB)
Collecting python-dateutil>=2.7 (from matplotlib!=3.6.1,>=3.4->seaborn->-r requirements.txt (line 11))
  Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB)
Collecting pytz>=2020.1 (from pandas>=1.2->seaborn->-r requirements.txt (line 11))
  Using cached pytz-2024.1-py2.py3-none-any.whl.metadata (22 kB)
Collecting tzdata>=2022.7 (from pandas>=1.2->seaborn->-r requirements.txt (line 11))
  Using cached tzdata-2024.1-py2.py3-none-any.whl.metadata (1.4 kB)
Collecting idna>=2.0 (from yarl<2.0,>=1.0->aiohttp->-r requirements.txt (line 6))
  Using cached idna-3.7-py3-none-any.whl.metadata (9.9 kB)
Collecting six>=1.5 (from python-dateutil>=2.7->matplotlib!=3.6.1,>=3.4->seaborn->-r requirements.txt (line 11))
  Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB)
Using cached protobuf-5.27.3-cp38-abi3-manylinux2014_x86_64.whl (309 kB)
Using cached aiomqtt-2.3.0-py3-none-any.whl (15 kB)
Using cached cryptography-43.0.0-cp39-abi3-manylinux_2_28_x86_64.whl (4.0 MB)
Using cached aiosqlite-0.20.0-py3-none-any.whl (15 kB)
Using cached aiohttp-3.10.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
Using cached aiodns-3.2.0-py3-none-any.whl (5.7 kB)
Using cached jinja2-3.1.4-py3-none-any.whl (133 kB)
Using cached aiohttp_sse-2.2.0-py3-none-any.whl (6.7 kB)
Using cached asyncpg-0.29.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.3 MB)
Using cached seaborn-0.13.2-py3-none-any.whl (294 kB)
Using cached pydot-3.0.1-py3-none-any.whl (22 kB)
Using cached plotly-5.23.0-py3-none-any.whl (17.3 MB)
Using cached aiohappyeyeballs-2.4.0-py3-none-any.whl (12 kB)
Using cached aiosignal-1.3.1-py3-none-any.whl (7.6 kB)
Using cached attrs-24.2.0-py3-none-any.whl (63 kB)
Using cached cffi-1.17.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (479 kB)
Using cached frozenlist-1.4.1-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (281 kB)
Using cached greenlet-3.0.3-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (625 kB)
Using cached MarkupSafe-2.1.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (28 kB)
Using cached matplotlib-3.9.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (8.3 MB)
Using cached multidict-6.0.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (130 kB)
Using cached numpy-2.1.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.0 MB)
Using cached packaging-24.1-py3-none-any.whl (53 kB)
Using cached paho_mqtt-2.1.0-py3-none-any.whl (67 kB)
Using cached pandas-2.2.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.7 MB)
Using cached pycares-4.4.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (290 kB)
Using cached pyparsing-3.1.2-py3-none-any.whl (103 kB)
Using cached tenacity-9.0.0-py3-none-any.whl (28 kB)
Using cached typing_extensions-4.12.2-py3-none-any.whl (37 kB)
Using cached yarl-1.9.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (322 kB)
Using cached SQLAlchemy-2.0.32-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.2 MB)
Using cached contourpy-1.2.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (309 kB)
Using cached cycler-0.12.1-py3-none-any.whl (8.3 kB)
Using cached fonttools-4.53.1-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.9 MB)
Using cached idna-3.7-py3-none-any.whl (66 kB)
Using cached kiwisolver-1.4.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.5 MB)
Using cached pillow-10.4.0-cp312-cp312-manylinux_2_28_x86_64.whl (4.5 MB)
Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
Using cached pytz-2024.1-py2.py3-none-any.whl (505 kB)
Using cached tzdata-2024.1-py2.py3-none-any.whl (345 kB)
Using cached pycparser-2.22-py3-none-any.whl (117 kB)
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Installing collected packages: pytz, tzdata, typing-extensions, tenacity, six, pyparsing, pycparser, protobuf, pillow, paho-mqtt, packaging, numpy, multidict, MarkupSafe, kiwisolver, idna, greenlet, frozenlist, fonttools, cycler, attrs, asyncpg, aiohappyeyeballs, yarl, sqlalchemy, python-dateutil, pydot, plotly, Jinja2, contourpy, cffi, aiosqlite, aiosignal, aiomqtt, pycares, pandas, matplotlib, cryptography, aiohttp, seaborn, aiohttp-sse, aiodns
Successfully installed Jinja2-3.1.4 MarkupSafe-2.1.5 aiodns-3.2.0 aiohappyeyeballs-2.4.0 aiohttp-3.10.5 aiohttp-sse-2.2.0 aiomqtt-2.3.0 aiosignal-1.3.1 aiosqlite-0.20.0 asyncpg-0.29.0 attrs-24.2.0 cffi-1.17.0 contourpy-1.2.1 cryptography-43.0.0 cycler-0.12.1 fonttools-4.53.1 frozenlist-1.4.1 greenlet-3.0.3 idna-3.7 kiwisolver-1.4.5 matplotlib-3.9.2 multidict-6.0.5 numpy-2.1.0 packaging-24.1 paho-mqtt-2.1.0 pandas-2.2.2 pillow-10.4.0 plotly-5.23.0 protobuf-5.27.3 pycares-4.4.0 pycparser-2.22 pydot-3.0.1 pyparsing-3.1.2 python-dateutil-2.9.0.post0 pytz-2024.1 seaborn-0.13.2 six-1.16.0 sqlalchemy-2.0.32 tenacity-9.0.0 typing-extensions-4.12.2 tzdata-2024.1 yarl-1.9.4
johannes@gamesrv:~/meshview$ ./env/bin/python main.py
Traceback (most recent call last):
  File "/home/johannes/meshview/main.py", line 41, in <module>
    asyncio.run(main(args))
  File "/usr/local/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/johannes/meshview/main.py", line 17, in main
    database.init_database(args.database)
  File "/home/johannes/meshview/meshview/database.py", line 10, in init_database
    engine = create_async_engine(database_connetion_string, echo=False, pool_size=20, max_overflow=50)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/johannes/meshview/env/lib/python3.12/site-packages/sqlalchemy/ext/asyncio/engine.py", line 120, in create_async_engine
    sync_engine = _create_engine(url, **kw)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in create_engine
  File "/home/johannes/meshview/env/lib/python3.12/site-packages/sqlalchemy/util/deprecations.py", line 281, in warned
    return fn(*args, **kwargs)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^
  File "/home/johannes/meshview/env/lib/python3.12/site-packages/sqlalchemy/engine/create.py", line 697, in create_engine
    raise TypeError(
TypeError: Invalid argument(s) 'pool_size','max_overflow' sent to create_engine(), using configuration SQLiteDialect_aiosqlite/NullPool/Engine.  Please check that the keyword arguments are appropriate for this combination of components.

When I follow the steps in the readme I get the above error.

If I then edit meshview/database.py and remove pool_size and max_overflow ...

engine = create_async_engine(database_connetion_string, echo=False)

Then service do start, but with the following warnings ...

johannes@gamesrv:~/meshview$ ./env/bin/python main.py --bind 192.168.3.14
sys:1: SAWarning: relationship 'Packet.to_node' will copy column packet.to_node_id to column node.node_id, which conflicts with relationship(s): 'Packet.from_node' (copies packet.from_node_id to node.node_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to isolate the columns that should be written towards.   To silence this warning, add the parameter 'overlaps="from_node"' to the 'Packet.to_node' relationship. (Background on this warning at: https://sqlalche.me/e/20/qzyx) (This warning originated from the `configure_mappers()` process, which was invoked automatically in response to a user-initiated operation.)
sys:1: SAWarning: relationship 'PacketSeen.node' will copy column packet_seen.node_id to column node.node_id, which conflicts with relationship(s): 'Packet.from_node' (copies packet.from_node_id to node.node_id), 'Packet.to_node' (copies packet.to_node_id to node.node_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to isolate the columns that should be written towards.   To silence this warning, add the parameter 'overlaps="from_node,to_node"' to the 'PacketSeen.node' relationship. (Background on this warning at: https://sqlalche.me/e/20/qzyx) (This warning originated from the `configure_mappers()` process, which was invoked automatically in response to a user-initiated operation.)

I can however open the webpage but then no packets are being received even using your default topic.

Any advice?

Thanks!
Johannes

Feature Request: Graphs for telemetry data.

You already graph battery status over time, it'd be cool to have other graph options. Maybe not something that needs to be pre-rendered like battery & neighbor SNR, but at least show latest value at the top?

The two that come to mind (because I've been tinkering with them) are environment_metrics and power_metrics. Its seems that environment_metrics can potentially duplicate some (?) of the power_metrics data, but since they are sent at different times, it is actually different data. Power can have up to 3 channels of voltage/current, both of which can be negative (mis-wired or charging).

Also maybe ch util would be nice to graph as well?

Example Packets

https://meshview.armooo.net/packet/2141323731
https://meshview.armooo.net/packet/2141323689
https://meshview.armooo.net/packet/2141323679
https://meshview.armooo.net/packet/959881067
https://meshview.armooo.net/packet/2067725582
https://meshview.armooo.net/packet/1904126290

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.