ivlovric / hfp Goto Github PK

Latest version fails compiling due to DecodeHEP being undefined

./HFP.go:54:19: undefined: DecodeHEP
./HFP.go:119:18: undefined: DecodeHEP
./HFP.go:160:18: undefined: DecodeHEP

Hi,

I am using HFP with SSH tunnel so that when ssh tunnel is down, HFP stores the packets and sends them when tunnel is up again.

AGENT -------> HFP ---------> SSH TUNNEL -----------> DEST SERVER

I tested this with below 2 scenarios. For new connections it works fine i.e storing packets and sending when tunnel is up. However, for the active connections during which tunnel was interrupted, it throws this error and does not send the remaining packets.

2023/07/07 07:30:23 ||--> Sending HEP OUT error: write tcp4 192.168.10.14:42496->192.168.10.14:9062: write: broken pipe
2023/07/07 07:30:23 ||-->X File Send HEP from buffer to file error read tcp4 192.168.10.14:9060->192.168.10.27:57982: use of closed network connection.

Testing pattern 1:

take down ssh tunnel
make and complete the test call
bring up ssh tunnel

Result: Works fine

Testing pattern 2:

ssh tunnel up
make a call and then bring down ssh tunnel During the Call
bring up ssh tunnel again

Result: In this case the complete call flow (all packets) was not there for the call during which tunnel gets down.

[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]# ./HFP -l 0.0.0.0:9060 -r 192.168.10.14:9062 -d on
2023/07/07 07:30:05 mkdir HEP: file exists
Saved HEP file is 0 bytes long
Listening for HEP on: 0.0.0.0:9060
Proxying HEP to: 192.168.10.14:9062
IPFilter:
IPFilterAction: pass
Prometheus metrics: 8090

HFP started in proxy high performance mode

2023/07/07 07:30:05 HELLO HFP c==>V|| INITIAL Dial LOOPBACK IN success
2023/07/07 07:30:05 -->|| New connection from 127.0.0.1:55612
2023/07/07 07:30:05 ||--> Connected OUT 192.168.10.14:9062
2023/07/07 07:30:05 -->|| Got 9 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:05 ||--> Sending init HELLO HFP successful without filters to 192.168.10.14:9062

2023/07/07 07:30:19 -->|| New connection from 192.168.10.27:57982
2023/07/07 07:30:19 ||--> Connected OUT 192.168.10.14:9062
2023/07/07 07:30:19 -->|| Got 1154 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:19 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:19 -->|| Got 592 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:19 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:19 -->|| Got 238 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:19 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:20 -->|| Got 447 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:20 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:20 -->|| Got 787 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:20 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:20 -->|| Got 267 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:20 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:20 -->|| Got 605 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:20 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:21 -->|| Got 466 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:21 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:21 -->|| Got 1370 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:21 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:21 -->|| Got 898 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:21 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:21 -->|| Got 629 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:21 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:22 -->|| Got 447 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:22 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:22 -->|| Got 730 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:22 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:22 -->|| Got 1098 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:22 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:22 -->|| Got 1274 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:22 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:23 -->|| Got 515 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:23 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:23 -->|| Got 501 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:23 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:23 -->|| Got 547 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:23 ||--> Sending HEP OUT error: write tcp4 192.168.10.14:42496->192.168.10.14:9062: write: broken pipe
2023/07/07 07:30:23 ||-->X File Send HEP from buffer to file error read tcp4 192.168.10.14:9060->192.168.10.27:57982: use of closed network connection

Hi everyone!
I'm trying to deploy HFP on a Centos 7 machine to make some test, but when I run make I got the following error

go: downloading golang.org/x/sys v0.0.0-20210603081109-ebe580a85c40
# runtime/internal/sys
/usr/local/go/src/runtime/internal/sys/consts.go:13:7: StackGuardMultiplier redeclared in this block
        /usr/local/go/src/runtime/internal/sys/arch.go:27:74: previous declaration
/usr/local/go/src/runtime/internal/sys/consts.go:16:7: DefaultPhysPageSize redeclared in this block
        /usr/local/go/src/runtime/internal/sys/arch.go:36:29: previous declaration
/usr/local/go/src/runtime/internal/sys/consts.go:20:7: PCQuantum redeclared in this block
        /usr/local/go/src/runtime/internal/sys/arch.go:40:19: previous declaration
/usr/local/go/src/runtime/internal/sys/consts.go:23:7: Int64Align redeclared in this block
        /usr/local/go/src/runtime/internal/sys/arch.go:43:20: previous declaration
/usr/local/go/src/runtime/internal/sys/consts.go:30:7: MinFrameSize redeclared in this block
        /usr/local/go/src/runtime/internal/sys/arch.go:50:22: previous declaration
/usr/local/go/src/runtime/internal/sys/consts.go:34:7: StackAlign redeclared in this block
        /usr/local/go/src/runtime/internal/sys/arch.go:54:20: previous declaration
make: *** [Makefile:2: all] Error 2

I'm using go version go1.17.5 linux/amd64

Thank you

hey, does this supports TLS connection to HEP server? If so, how to configure it?

Hi @ivlovric,

Such a great tool!!

We were testing it today, if you see in the logs we disconnected receiving server and sent hep to the proxy, which worked. Upon reconnect nothing happened, we waited a full minute and then I sent new HEP through the proxy which triggered the flush.

I would have expected the flush to occur as soon as the connection is re-established.

We repeated the test with a longer wait and HEP remains in buffer until new HEP is sent through proxy before the flush.

2021/07/28 11:27:05 -->||Receiving HEP to LOG
2021/07/28 11:27:10 ||-->X Dial OUT reconnect failure - retrying <nil>
2021/07/28 11:27:15 ||-->X Dial OUT reconnect failure - retrying <nil>
2021/07/28 11:27:20 ||-->X Dial OUT reconnect failure - retrying <nil>
2021/07/28 11:27:25 ||-->X Dial OUT reconnect failure - retrying <nil>
2021/07/28 11:27:30 ||-->X Dial OUT reconnect failure - retrying <nil>
2021/07/28 11:27:35 ||-->X Dial OUT reconnect failure - retrying <nil>
2021/07/28 11:27:40 ||-->X Dial OUT reconnect failure - retrying <nil>
2021/07/28 11:27:45 ||-->X Dial OUT reconnect failure - retrying <nil>
2021/07/28 11:27:50 ||-->X Dial OUT reconnect failure - retrying <nil>
2021/07/28 11:27:55 ||-->X Dial OUT reconnect failure - retrying <nil>
2021/07/28 11:28:00 ||-->V Dial OUT reconnected <nil>
2021/07/28 11:28:05 ||-->V Dial OUT reconnected <nil>
2021/07/28 11:29:07 New connection from 127.0.0.1:55523
2021/07/28 11:29:07 ||--> Connected OUT IP.IP.IP.IP:9062 //redacted IP 
2021/07/28 11:29:07 ||-->V Send HEP from LOG OK - 37167 bytes
2021/07/28 11:29:07 Flushing HEP file
2021/07/28 11:29:07 -->||Received:

Hi,

thanks for this great project. Did i forget something? The Copy to FILE error happens - and i am unsure if its "okay" or its an issue.
The Code shows that the file should reported (is it the HFP/HEP-saved.arch) ?

Dec 06 14:25:10  HFP[81569]: 2021/12/06 14:25:10 -->X||Read IN packets error: EOF
Dec 06 14:25:10  HFP[81569]: 2021/12/06 14:25:10 -->|| New connection from 172.24.32.17:51560
Dec 06 14:25:10  HFP[81569]: 2021/12/06 14:25:10 ||--> Connected OUT 172.16.32.26:9062
Dec 06 14:45:06  HFP[81569]: 2021/12/06 14:45:06 ||--> Sending HEP OUT error: write tcp4 172.24.32.29:53632->172.16.32.26:9062: write: connection reset by peer
Dec 06 14:45:06  HFP[81569]: Copy to FILE error
Dec 06 14:45:06  HFP[81569]: read tcp4 172.24.32.29:9062->172.24.32.17:51560: use of closed network connection
Dec 06 14:50:34  HFP[81569]: 2021/12/06 14:50:34 -->|| New connection from 172.24.32.17:52816
Dec 06 14:50:34  HFP[81569]: 2021/12/06 14:50:34 ||--> Connected OUT 172.16.32.26:9062
Dec 06 18:28:58  HFP[81569]: 2021/12/06 18:28:58 -->X||Read IN packets error: EOF
Dec 06 18:28:58  HFP[81569]: 2021/12/06 18:28:58 -->|| New connection from 172.24.32.17:35432
Dec 06 18:28:58  HFP[81569]: 2021/12/06 18:28:58 ||--> Connected OUT 172.16.32.26:9062`

Systemd Service File

[Unit]
Description=HEP Fidelity Proxy - Forward or Store HEP Messages to HOMER/HEPIC
After=network.target

[Service]
WorkingDirectory=/var/lib/hfp
ExecStart=/usr/local/bin/HFP -d "off" -l ":9062" -r "my.hepic.server:9062"
ExecStop=/bin/kill ${MAINPID}
Restart=on-failure
RestartSec=10s
User=hfp
Type=simple

[Install]
WantedBy=multi-user.target

Hey, I would like to ask

• what's the buffer size? can it be configurable?
• When HEP Server is unavailable how much times the HFP retry the connection and at what interval?
• After flushing - On homer would i see the call with the captured date or when it was flushed?