ivlovric / hfp Goto Github PK
View Code? Open in Web Editor NEWHEP Fidelity Proxy
HEP Fidelity Proxy
Hi,
thanks for this great project. Did i forget something? The Copy to FILE error happens - and i am unsure if its "okay" or its an issue.
The Code shows that the file should reported (is it the HFP/HEP-saved.arch) ?
Dec 06 14:25:10 HFP[81569]: 2021/12/06 14:25:10 -->X||Read IN packets error: EOF
Dec 06 14:25:10 HFP[81569]: 2021/12/06 14:25:10 -->|| New connection from 172.24.32.17:51560
Dec 06 14:25:10 HFP[81569]: 2021/12/06 14:25:10 ||--> Connected OUT 172.16.32.26:9062
Dec 06 14:45:06 HFP[81569]: 2021/12/06 14:45:06 ||--> Sending HEP OUT error: write tcp4 172.24.32.29:53632->172.16.32.26:9062: write: connection reset by peer
Dec 06 14:45:06 HFP[81569]: Copy to FILE error
Dec 06 14:45:06 HFP[81569]: read tcp4 172.24.32.29:9062->172.24.32.17:51560: use of closed network connection
Dec 06 14:50:34 HFP[81569]: 2021/12/06 14:50:34 -->|| New connection from 172.24.32.17:52816
Dec 06 14:50:34 HFP[81569]: 2021/12/06 14:50:34 ||--> Connected OUT 172.16.32.26:9062
Dec 06 18:28:58 HFP[81569]: 2021/12/06 18:28:58 -->X||Read IN packets error: EOF
Dec 06 18:28:58 HFP[81569]: 2021/12/06 18:28:58 -->|| New connection from 172.24.32.17:35432
Dec 06 18:28:58 HFP[81569]: 2021/12/06 18:28:58 ||--> Connected OUT 172.16.32.26:9062`
Systemd Service File
[Unit]
Description=HEP Fidelity Proxy - Forward or Store HEP Messages to HOMER/HEPIC
After=network.target
[Service]
WorkingDirectory=/var/lib/hfp
ExecStart=/usr/local/bin/HFP -d "off" -l ":9062" -r "my.hepic.server:9062"
ExecStop=/bin/kill ${MAINPID}
Restart=on-failure
RestartSec=10s
User=hfp
Type=simple
[Install]
WantedBy=multi-user.target
Hi @ivlovric,
Such a great tool!!
We were testing it today, if you see in the logs we disconnected receiving server and sent hep to the proxy, which worked. Upon reconnect nothing happened, we waited a full minute and then I sent new HEP through the proxy which triggered the flush.
I would have expected the flush to occur as soon as the connection is re-established.
We repeated the test with a longer wait and HEP remains in buffer until new HEP is sent through proxy before the flush.
2021/07/28 11:27:05 -->||Receiving HEP to LOG
2021/07/28 11:27:10 ||-->X Dial OUT reconnect failure - retrying <nil>
2021/07/28 11:27:15 ||-->X Dial OUT reconnect failure - retrying <nil>
2021/07/28 11:27:20 ||-->X Dial OUT reconnect failure - retrying <nil>
2021/07/28 11:27:25 ||-->X Dial OUT reconnect failure - retrying <nil>
2021/07/28 11:27:30 ||-->X Dial OUT reconnect failure - retrying <nil>
2021/07/28 11:27:35 ||-->X Dial OUT reconnect failure - retrying <nil>
2021/07/28 11:27:40 ||-->X Dial OUT reconnect failure - retrying <nil>
2021/07/28 11:27:45 ||-->X Dial OUT reconnect failure - retrying <nil>
2021/07/28 11:27:50 ||-->X Dial OUT reconnect failure - retrying <nil>
2021/07/28 11:27:55 ||-->X Dial OUT reconnect failure - retrying <nil>
2021/07/28 11:28:00 ||-->V Dial OUT reconnected <nil>
2021/07/28 11:28:05 ||-->V Dial OUT reconnected <nil>
2021/07/28 11:29:07 New connection from 127.0.0.1:55523
2021/07/28 11:29:07 ||--> Connected OUT IP.IP.IP.IP:9062 //redacted IP
2021/07/28 11:29:07 ||-->V Send HEP from LOG OK - 37167 bytes
2021/07/28 11:29:07 Flushing HEP file
2021/07/28 11:29:07 -->||Received:
Hi everyone!
I'm trying to deploy HFP on a Centos 7 machine to make some test, but when I run make
I got the following error
go: downloading golang.org/x/sys v0.0.0-20210603081109-ebe580a85c40
# runtime/internal/sys
/usr/local/go/src/runtime/internal/sys/consts.go:13:7: StackGuardMultiplier redeclared in this block
/usr/local/go/src/runtime/internal/sys/arch.go:27:74: previous declaration
/usr/local/go/src/runtime/internal/sys/consts.go:16:7: DefaultPhysPageSize redeclared in this block
/usr/local/go/src/runtime/internal/sys/arch.go:36:29: previous declaration
/usr/local/go/src/runtime/internal/sys/consts.go:20:7: PCQuantum redeclared in this block
/usr/local/go/src/runtime/internal/sys/arch.go:40:19: previous declaration
/usr/local/go/src/runtime/internal/sys/consts.go:23:7: Int64Align redeclared in this block
/usr/local/go/src/runtime/internal/sys/arch.go:43:20: previous declaration
/usr/local/go/src/runtime/internal/sys/consts.go:30:7: MinFrameSize redeclared in this block
/usr/local/go/src/runtime/internal/sys/arch.go:50:22: previous declaration
/usr/local/go/src/runtime/internal/sys/consts.go:34:7: StackAlign redeclared in this block
/usr/local/go/src/runtime/internal/sys/arch.go:54:20: previous declaration
make: *** [Makefile:2: all] Error 2
I'm using go version go1.17.5 linux/amd64
Thank you
Hey, I would like to ask
Hi,
I am using HFP with SSH tunnel so that when ssh tunnel is down, HFP stores the packets and sends them when tunnel is up again.
AGENT -------> HFP ---------> SSH TUNNEL -----------> DEST SERVER
I tested this with below 2 scenarios. For new connections it works fine i.e storing packets and sending when tunnel is up. However, for the active connections during which tunnel was interrupted, it throws this error and does not send the remaining packets.
2023/07/07 07:30:23 ||--> Sending HEP OUT error: write tcp4 192.168.10.14:42496->192.168.10.14:9062: write: broken pipe
2023/07/07 07:30:23 ||-->X File Send HEP from buffer to file error read tcp4 192.168.10.14:9060->192.168.10.27:57982: use of closed network connection.
Testing pattern 1:
take down ssh tunnel
make and complete the test call
bring up ssh tunnel
Result: Works fine
Testing pattern 2:
ssh tunnel up
make a call and then bring down ssh tunnel During the Call
bring up ssh tunnel again
Result: In this case the complete call flow (all packets) was not there for the call during which tunnel gets down.
[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]# ./HFP -l 0.0.0.0:9060 -r 192.168.10.14:9062 -d on
2023/07/07 07:30:05 mkdir HEP: file exists
Saved HEP file is 0 bytes long
Listening for HEP on: 0.0.0.0:9060
Proxying HEP to: 192.168.10.14:9062
IPFilter:
IPFilterAction: pass
Prometheus metrics: 8090
HFP started in proxy high performance mode
2023/07/07 07:30:05 HELLO HFP c==>V|| INITIAL Dial LOOPBACK IN success
2023/07/07 07:30:05 -->|| New connection from 127.0.0.1:55612
2023/07/07 07:30:05 ||--> Connected OUT 192.168.10.14:9062
2023/07/07 07:30:05 -->|| Got 9 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:05 ||--> Sending init HELLO HFP successful without filters to 192.168.10.14:9062
2023/07/07 07:30:19 -->|| New connection from 192.168.10.27:57982
2023/07/07 07:30:19 ||--> Connected OUT 192.168.10.14:9062
2023/07/07 07:30:19 -->|| Got 1154 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:19 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:19 -->|| Got 592 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:19 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:19 -->|| Got 238 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:19 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:20 -->|| Got 447 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:20 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:20 -->|| Got 787 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:20 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:20 -->|| Got 267 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:20 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:20 -->|| Got 605 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:20 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:21 -->|| Got 466 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:21 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:21 -->|| Got 1370 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:21 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:21 -->|| Got 898 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:21 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:21 -->|| Got 629 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:21 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:22 -->|| Got 447 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:22 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:22 -->|| Got 730 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:22 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:22 -->|| Got 1098 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:22 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:22 -->|| Got 1274 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:22 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:23 -->|| Got 515 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:23 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:23 -->|| Got 501 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:23 ||--> Sending HEP OUT successful without filters to 192.168.10.14:9062
2023/07/07 07:30:23 -->|| Got 547 bytes on wire -- Total buffer size: 65535
2023/07/07 07:30:23 ||--> Sending HEP OUT error: write tcp4 192.168.10.14:42496->192.168.10.14:9062: write: broken pipe
2023/07/07 07:30:23 ||-->X File Send HEP from buffer to file error read tcp4 192.168.10.14:9060->192.168.10.27:57982: use of closed network connection
hey, does this supports TLS connection to HEP server? If so, how to configure it?
Latest version fails compiling due to DecodeHEP
being undefined
./HFP.go:54:19: undefined: DecodeHEP
./HFP.go:119:18: undefined: DecodeHEP
./HFP.go:160:18: undefined: DecodeHEP
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.