inigoflores / helium-miner-log-analyzer Goto Github PK

Hello,
maybe go in contact with https://glamos.eu/walkapp/ to import the miner data into the app.
I look like a big benefit to get an overview about his miner over this combination.
Thx in advance.

Hello,
There will be a light hotspot version of this analyzer?
Thank you.

Dear Inigoflores,
on windows i started gettin a division by zeror erro. Do you have any idea why? It was working some weeks ago.

Using logs in folder processlogs\logs
Warning: Division by zero in processlogs\processlogs.php on line 168

General Witnesses Overview  
----------------------------------
Total witnesses                   =     1   (INF/hour)
Succesfully delivered             =     0     (0%)
Failed                            =     1   (100%) 
  ├── Max retry    =    0     (0%) 
  └── Crash/reboot =    1   (100%) 

Max Retry Failure Reasons 
----------------------------------
Timeout                           =     0     (0%) 
Not Found                         =     0     (0%) 
Other challenger issues           =     0     (0%) 

Challengers 
----------------------------------
Not Relayed                       =     0     (0%) 
Relayed                           =     0     (0%) 
Unknown (Probably Not Relayed)    =     1   (100%) 

console.log

Awesome little script, would it be possible to add a feature to it? How many witnesses have been sent per hour. I usually get an idea of the state of the network by analyzing this and how many hotspots are sending beacons. Usually it's a constant number but sometimes it just goes way down.

Hi,

Thanx for this script!

It works fine on a Controllino v1 miner

Log folder on Controllino v1: /var/log/miner/

looking to see if it is possible to support SensecapM1

Please implement the same for this processlogs.php as for the LoRa logs one
(keep the raw log entries in UTC, display datetimes in local tz)

Excellent stuff by the way! thanx a lot for these scripts :)

Running a grep command with the same search phrases produces different results from your php code. The php total for "failed" is the same, but grep shows 220 sent witnesses, while the php shows only 213. No witnesses were sent between runs.

/var/log/miner # date
Thu Jun 9 03:08:18 UTC 2022
/var/log/miner # /opt/miner/processlogs.php

Using logs in folder /var/log/miner/

General Witnesses Overview

Total witnesses = 213 (9.65/hour)
Succesfully delivered = 209 (98.12%)
Failed = 4 (1.88%)
/var/log/miner # date
Thu Jun 9 03:09:09 UTC 2022
/var/log/miner # grep "miner_onion_server_light:decrypt:" console.log* | grep "sending witness at RSSI"|wc -l
220
/var/log/miner # grep "@miner_poc_grpc_client_statem:send_report" console.log* | grep "failed to submit report"|wc -l
4
/var/log/miner # grep "miner_onion_server_light:decrypt:" console.log|grep "sending witness at RSSI"|sort|tail
2022-06-09 01:03:58.969 153 [info] <0.2102.0>@miner_onion_server_light:decrypt:{230,13} sending witness at RSSI: -110, Frequency: 904.7, SNR: -1.8
2022-06-09 01:09:52.240 153 [info] <0.2102.0>@miner_onion_server_light:decrypt:{230,13} sending witness at RSSI: -88, Frequency: 904.1, SNR: 10.8
2022-06-09 01:16:29.374 201 [info] <0.2096.0>@miner_onion_server_light:decrypt:{230,13} sending witness at RSSI: -103, Frequency: 904.5, SNR: 6.5
2022-06-09 01:22:46.038 201 [info] <0.2096.0>@miner_onion_server_light:decrypt:{230,13} sending witness at RSSI: -97, Frequency: 904.3, SNR: 5.0
2022-06-09 01:23:22.905 201 [info] <0.2096.0>@miner_onion_server_light:decrypt:{230,13} sending witness at RSSI: -118, Frequency: 903.9, SNR: -7.5
2022-06-09 02:17:19.792 203 [info] <0.2075.0>@miner_onion_server_light:decrypt:{230,13} sending witness at RSSI: -116, Frequency: 904.9, SNR: -7.5
2022-06-09 02:22:54.526 203 [info] <0.2075.0>@miner_onion_server_light:decrypt:{230,13} sending witness at RSSI: -115, Frequency: 903.9, SNR: -6.0
2022-06-09 02:24:42.024 203 [info] <0.2075.0>@miner_onion_server_light:decrypt:{230,13} sending witness at RSSI: -110, Frequency: 904.9, SNR: -4.0
2022-06-09 02:50:24.910 203 [info] <0.2075.0>@miner_onion_server_light:decrypt:{230,13} sending witness at RSSI: -115, Frequency: 903.9, SNR: -3.8
2022-06-09 03:00:33.104 203 [info] <0.2075.0>@miner_onion_server_light:decrypt:{230,13} sending witness at RSSI: -119, Frequency: 904.7, SNR: -11.8
/var/log/miner # date
Thu Jun 9 03:11:25 UTC 2022

hello... i can not open it i have this error maybe i went to wrong way can you tell me how to open it?
lind@Mehdis-iMac helium-miner-log-analyzer % ./processlogs.php
zsh: ./processlogs.php: bad interpreter: /usr/bin/php: no such file or directory

New feature request (not very important since you can't do much with this info): Add your current connection info to the validator
(gRPC over http/2 on outbound tcp/8080). Include: IP Address + Port and P2P address (67.211.223.5:8080, 112vvCWKx3s3JrHqxan4fAqNAoezsNXAGyFvZchZmV6mCfwmMcvA) from example below:

Also: not much info you can extract from the api on https://api.helium.io/v1/validators/112vvCWKx3s3JrHqxan4fAqNAoezsNXAGyFvZchZmV6mCfwmMcvA
https://explorer.helium.com/validators/112vvCWKx3s3JrHqxan4fAqNAoezsNXAGyFvZchZmV6mCfwmMcvA/activity

Example log contents to be processed:

2022-05-16 05:18:22.653 7 [info] <0.1822.0>@miner_poc_grpc_client_statem:init:{174,5} starting miner_poc_grpc_client_statem
2022-05-16 05:18:29.518 7 [info] <0.1822.0>@miner_poc_grpc_client_statem:setup:{219,13} *** connecting to validator with ip: "67.211.223.5", port: 8080, addr: "/p2p/112vvCWKx3s3JrHqxan4fAqNAoezsNXAGyFvZchZmV6mCfwmMcvA"
2022-05-16 05:18:29.766 7 [info] <0.1822.0>@miner_poc_grpc_client_statem:connect_validator:{514,17} successfully connected to validator via connection #{client => http2_client,host => <<"67.211.223.5">>,http_connection => <0.1854.0>,scheme => <<"http">>}

The challenge will be to detect the current validator connection. In theory, it may change over time (after initial boot up of the miner)

Hey!
Can this work with Sensecap M1 if I add the log file path in the code?
what would be the running command then in the docker?

@inigoflores

When running the tool with ./processlogs.php -l both RSSI and SNR are outputted, but not noise. This is a simple calculation (RSSI - SNR) source but a bit cumbersome to effectively do yourself in the console.

I want to see this to better understand if I would benefit from a SAW filter or not.

PS. If you don't have time to do it, but want it added, let me know and I'll do it. Was quite some time since I dabbled in php though!

Thanks!

Great product thanks!
I am using the rak v2 with a modified firmware https://github.com/HeliumDIY/helium_ansible and the logs are located in /home/pi/miner_data/log I ended up overriding $logsFolder manually, I think that it would help if you added to the list.

Thanks!

maybe you can take some additional info from it to enhance yours :)

#!/bin/bash
#!/bin/sh
#
# spot for varaiables that will be needed
miner_vm_name=$(balena ps | egrep "miner_" | awk '{print $NF}')
miner_animal_name=$(balena exec $miner_vm_name miner info name)
get_console_log=$(find /mnt/data -name "console.log")
total_witnesses=$(cat $get_console_log | egrep -w '@miner_onion_server:decrypt:' | grep -c ':')
successful_witnesses=$(cat $get_console_log | grep -c 'successfully sent witness to challenger')
failedtodial_witnesses=$(cat $get_console_log | grep -c 'failed to dial challenger')
resending_witnesses=$(cat $get_console_log | egrep -w '@miner_onion_server:send_witness:' | grep -c 're-sending')
sending_witnesses=$(cat $get_console_log | egrep -w '@miner_onion_server:send_witness:' | grep -c 'sending')
failedtosendresend_witnesses=$(cat $get_console_log | grep -c 'failed to send witness, max retry')
relaytransported_total=$(cat $get_console_log | egrep -w '@libp2p_transport_relay:connect_to:' | grep -c ':')
challenger_notfound=$(cat $get_console_log | egrep 'not_found' | grep -c 'failed to dial challenger')
challenger_timeout=$(cat $get_console_log | egrep 'timeout' | grep -c 'failed to dial challenger')
challenger_refused=$(cat $get_console_log | egrep 'econnrefused' | grep -c 'failed to dial challenger')
challenger_unreachable=$(cat $get_console_log | egrep 'ehostunreach' | grep -c 'failed to dial challenger')
challenger_nolistenaddr=$(cat $get_console_log | egrep 'no_listen_addr' | grep -c 'failed to dial challenger') 
#
#
echo "****************************************************************************"
echo "Performing actions on Node:"
echo "          VM: $miner_vm_name"
echo " Animal Name: $miner_animal_name"
echo "Log Location: $get_console_log"
echo "****************************************************************************"
echo " "
#
if [ $1 == "p2p-status" ]; then
	balena exec $miner_vm_name miner info p2p_status
elif [ $1 == "gossip-peers" ]; then
	balena exec $miner_vm_name miner peer gossip_peers
elif [ $1 == "peer-refresh" ]; then
	balena exec $miner_vm_name miner peer refresh
elif [ $1 == "relay-reset"  ]; then
	balena exec $miner_vm_name miner peer relay_reset
elif [ $1 == "deamon-restart" ]; then
	balena exec $miner_vm_name miner restart
	echo 'wait 30 seconds before running any further HMC commands to allow the miner deamon services to finish loading'
elif [ $1 == "vm-restart" ]; then
	balena exec $miner_vm_name miner reboot
	echo 'wait 1 minute before running any further HMC commands to allow the miner VM to boot up again and deamon services to finish loading'
elif [ $1 == "log-analyzer" ]; then
	echo '******************************************************************'
	echo 'Total Witnessed:                                    = '$total_witnesses
	echo '               |-- Sending:                         = '$(($sending_witnesses-$resending_witnesses))
	echo '               |-- Resending:                       = '$resending_witnesses
	echo 'Successful:                                         = '$successful_witnesses ' (' $(($successful_witnesses*100/$total_witnesses))'%)'
	echo 'Unreachable:                                        = '$failedtodial_witnesses ' (' $(($failedtodial_witnesses*100/$total_witnesses))'%)'
	echo 'Send or Re-send Failed:                             = '$failedtosendresend_witnesses ' (' $((failedtosendresend_witnesses*100/$total_witnesses))'%)'
	echo 'Other (Witness Failures):                           = '$(($total_witnesses-($successful_witnesses+$failedtodial_witnesses+$failedtosendresend_witnesses))) ' (' $(( ($total_witnesses-($successful_witnesses+$failedtodial_witnesses+$failedtosendresend_witnesses))*100/$total_witnesses))'%)'
	echo 'Challenger Issues:'
	echo '               |-- Challenger Not Found:            = '$challenger_notfound
	echo '               |-- Challenger Timed Out:            = '$challenger_timeout
	echo '               |-- Challenger Refused Connection:   = '$challenger_refused
	echo '               |-- Challenger Unreachable:          = '$challenger_unreachable
	echo '               |-- Challenger No Listening Address: = '$challenger_nolistenaddr
	echo '******************************************************************'
else
	echo "Bye"
fi

source: https://raw.githubusercontent.com/saad-akhtar/helium_miner_checker/main/hmc.sh

Hello,
I installed on sensecap and it worked.
But i can only display a few record, I mean, yesterday i could see about 120 but today only about 70.
Also i can't list yesterday's records.
Is this normal or am I using the wrong syntax?

A new miner image (2022.12.13.0 GA) was released, activating off-chain PoC witnesses and beacons for testing.
Before switching to SOL, it could be interesting to include these as an additional feature in the script already.

Some example log entries:

Beaconing
[ gateway-rs ] transmitting beacon, beacon: xxxxx ==, module: beacon

Witnessing
[ gateway-rs ] poc witness report submitted, beacon: xxxxx ==, module: beacon

I am unsure though if those are off-chain PoC witnesses or whether they may possibly be other LoRa packets.