40t / go-sniffer Goto Github PK
View Code? Open in Web Editor NEW🔎Sniffing and parsing mysql,redis,http,mongodb etc protocol. 抓包截取项目中的数据库请求并解析成相应的语句。
License: MIT License
🔎Sniffing and parsing mysql,redis,http,mongodb etc protocol. 抓包截取项目中的数据库请求并解析成相应的语句。
License: MIT License
2018-11-01 16:21:39| cli -> ser |Stm id[10]: 'SELECT domain
FROM xxx
WHERE ticket
= ? LIMIT 1 ';
set @p0 = 'xxxxx';
Execute stm id[10]: using @p0;
Drop stm id[10];
2018/11/01 16:21:39 ERR : Not found stm id 19
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x7885cf]
goroutine 44 [running]:
github.com/40t/go-sniffer/plugSrc/mysql/build.(*stream).resolveClientPacket(0xc420401590, 0xc42051f200, 0x1e, 0x600, 0x0)
/Golang/src/github.com/40t/go-sniffer/plugSrc/mysql/build/entry.go:314 +0x4ef
github.com/40t/go-sniffer/plugSrc/mysql/build.(*stream).resolve(0xc420401590)
/Golang/src/github.com/40t/go-sniffer/plugSrc/mysql/build/entry.go:192 +0x98
created by github.com/40t/go-sniffer/plugSrc/mysql/build.(*Mysql).ResolveStream
/Golang/src/github.com/40t/go-sniffer/plugSrc/mysql/build/entry.go:71 +0x42e
抓mongo包,追加到文件中,二十分钟go-sniffer大概用了15G内存,生成的文件就83M左右,请问下这是什么情况?
[root❄anhk-pc:go-sniffer]☭ GOPATH=$(pwd)/vendor go get github.com/40t/go-sniffer
package plugin: unrecognized import path "plugin" (import path does not begin with hostname)
[root❄anhk-pc:go-sniffer]☭
src/github.com/google/gopacket/pcap/pcap.go:22:18: fatal error: pcap.h: No such file or directory
#include <pcap.h>
^
compilation terminated.
之前一直都在Centos下使用,没遇到过问题。这次装到自己的Macbook pro上,在安装时候报错了。
go get -v -u github.com/40t/go-sniffer
github.com/40t/go-sniffer (download)
github.com/google/gopacket (download)
github.com/google/gopacket
github.com/40t/go-sniffer/plugSrc/mongodb/build/internal/json
github.com/40t/go-sniffer/plugSrc/http/build
github.com/40t/go-sniffer/plugSrc/redis/build
github.com/40t/go-sniffer/plugSrc/mysql/build
github.com/google/gopacket/layers
github.com/40t/go-sniffer/plugSrc/mongodb/build/bson
github.com/40t/go-sniffer/plugSrc/mongodb/build
github.com/google/gopacket/pcap
github.com/google/gopacket/tcpassembly
github.com/google/gopacket/tcpassembly/tcpreader
github.com/40t/go-sniffer/core
github.com/40t/go-sniffer
# github.com/40t/go-sniffer
/usr/local/Cellar/go/1.8.3/libexec/pkg/tool/darwin_amd64/link: /usr/local/Cellar/go/1.8.3/libexec/pkg/tool/darwin_amd64/link: combining dwarf failed: Unknown load command 0x32 (50)
这个是跟go的版本有关系吗?我的版本是:
go version go1.8.3 darwin/amd64
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x7a8bf8]
goroutine 7 [running]:
github.com/40t/go-sniffer/plugSrc/mysql/build.(*stream).resolveClientPacket(0xc420056c80, 0xc42062e600, 0x1, 0x600, 0x1)
/data/home/xxx/Gosrc/src/github.com/40t/go-sniffer/plugSrc/mysql/build/entry.go:304 +0xd98
github.com/40t/go-sniffer/plugSrc/mysql/build.(*stream).resolve(0xc420056c80)
/data/home/xxx/Gosrc/src/github.com/40t/go-sniffer/plugSrc/mysql/build/entry.go:193 +0x95
created by github.com/40t/go-sniffer/plugSrc/mysql/build.(*Mysql).ResolveStream
/data/home/xxx/Gosrc/src/github.com/40t/go-sniffer/plugSrc/mysql/build/entry.go:71 +0x3ff
看代码,应该是没有判断从map取出来的是否为空指针,就直接给字段赋值,导致空指针错误了
2018-12-05 14:41:24| ser -> cli |【Ok】 Effect Row:0
2018-12-05 14:41:24| ser -> cli |【Ok】 Effect Row:0
panic: runtime error: index out of range
goroutine 34 [running]:
github.com/40t/go-sniffer/plugSrc/mysql/build.(*stream).resolveServerPacket(0xc420418000, 0xc420456200, 0x0, 0x200, 0xb)
/gomeo2o/soft/go-sniffer/src/github.com/40t/go-sniffer/plugSrc/mysql/build/entry.go:219 +0x50d
github.com/40t/go-sniffer/plugSrc/mysql/build.(*stream).resolve(0xc420418000)
/gomeo2o/soft/go-sniffer/src/github.com/40t/go-sniffer/plugSrc/mysql/build/entry.go:194 +0xc0
created by github.com/40t/go-sniffer/plugSrc/mysql/build.(*Mysql).ResolveStream
/gomeo2o/soft/go-sniffer/src/github.com/40t/go-sniffer/plugSrc/mysql/build/entry.go:71 +0x3ff
Hello, will you support SQLServer in the future?
例如MySQL没有跑在3306端口的时候可以通过:
go-sniffer en0 mysql --port=12345
通过参数--port来指定监听的端口。
在Win10的WSL环境下成功安装了go-sniffer,运行go-sniffer报以下错误:
jclazz@j-dl-5570:~$ go-sniffer eth4 mysql -p 3306
2019/08/06 10:53:48 socket for SIOCETHTOOL(ETHTOOL_GET_TS_INFO): Socket type not supported
请问我应该如何解决这个问题?
panic: runtime error: index out of range [0] with length 0
goroutine 454 [running]:
github.com/40t/go-sniffer/plugSrc/mysql/build.(*Stmt).BindArgs(0xc1884aa380, {0xc188a3320a, 0x1, 0xc18848fe88?}, {0x0, 0x0, 0x0?}, {0x0, 0x0, 0x0})
/root/go/pkg/mod/github.com/40t/[email protected]/plugSrc/mysql/build/stmt.go:79 +0xbd4
github.com/40t/go-sniffer/plugSrc/mysql/build.(*stream).resolveClientPacket(0xc00006b3b0?, {0xc188a33200, 0x14, 0x600}, 0x0?)
/root/go/pkg/mod/github.com/40t/[email protected]/plugSrc/mysql/build/entry.go:344 +0x36d
github.com/40t/go-sniffer/plugSrc/mysql/build.(*stream).resolve(0xc00006b3b0)
/root/go/pkg/mod/github.com/40t/[email protected]/plugSrc/mysql/build/entry.go:193 +0x6b
created by github.com/40t/go-sniffer/plugSrc/mysql/build.(*Mysql).ResolveStream
/root/go/pkg/mod/github.com/40t/[email protected]/plugSrc/mysql/build/entry.go:71 +0x3f0
mac系统 0.14 (18A391)
go 版本: go version go1.11.1 darwin/amd64
执行命令:
go-sniffer lo0 redis
thanks.
panic: runtime error: index out of range [3] with length 1
goroutine 14 [running]:
encoding/binary.littleEndian.Uint32(...)
/Users/voidint/.g/go/src/encoding/binary/binary.go:63
github.com/40t/go-sniffer/plugSrc/mysql/build.LengthBinary(0xc000490001, 0x31, 0x5ff, 0x4031ce0, 0x44c3230)
/Users/voidint/workspace/go/src/github.com/40t/go-sniffer/plugSrc/mysql/build/util.go:41 +0xd4
github.com/40t/go-sniffer/plugSrc/mysql/build.(*stream).resolveServerPacket(0xc000402600, 0xc000490000, 0x32, 0x600, 0x1)
/Users/voidint/workspace/go/src/github.com/40t/go-sniffer/plugSrc/mysql/build/entry.go:235 +0x306
github.com/40t/go-sniffer/plugSrc/mysql/build.(*stream).resolve(0xc000402600)
/Users/voidint/workspace/go/src/github.com/40t/go-sniffer/plugSrc/mysql/build/entry.go:195 +0xc5
created by github.com/40t/go-sniffer/plugSrc/mysql/build.(*Mysql).ResolveStream
/Users/voidint/workspace/go/src/github.com/40t/go-sniffer/plugSrc/mysql/build/entry.go:71 +0x3b6
install error in centos, I think you could provide a binary file of multi system os.
OS Version
cat /etc/issue
CentOS release 6.8 (Final)
Kernel \r on an \m
install
yum -y install libcap
yum -y install libcap-devel
go get -v -u github.com/40t/go-sniffer
github.com/40t/go-sniffer (download)
github.com/google/gopacket (download)
# runtime/debug
gc 1 @0.005s 6%: 0.029+1.2+0.069 ms clock, 0.11+0.17/1.1/1.1+0.27 ms cpu, 4->4->3 MB, 5 MB goal, 4 P
gc 2 @0.011s 6%: 0.004+1.4+0.048 ms clock, 0.019+0.057/1.3/2.6+0.19 ms cpu, 5->6->5 MB, 6 MB goal, 4 P
github.com/google/gopacket/pcap
# github.com/google/gopacket/pcap
/devops/app/go/src/github.com/google/gopacket/pcap/pcap.go:22:18: fatal error: pcap.h: No such file or directory
#include <pcap.h>
有考虑支持Postgresql吗?
环境:
系统:CentOS Linux release 7.6.1810
MySQL: 5.7.30-log
MySQL 服务端地址:192.168.66.200
端口:3306
测试连接客户端地址:192.168.66.101
监听方式:
go-sniffer ens32 mysql -p 3306
问题描述:
使用 navicat、Dbeaver 等工具连接的话是可以抓取到信息的
但使用mysql client 远程连接的话,执行的操作无法被抓取到
mysql -utestuser -p123456 -h192.168.66.200 -P 3306
mysql> show databases;
mysql> create database abc;
mysql> use abc
Database changed
mysql> create table a(id int);
Query OK, 0 rows affected (0.11 sec)
mysql> insert into a values(1);
Query OK, 1 row affected (0.02 sec)
[root@initnode ~]# go-sniffer ens32 mysql -p 3306
tcp and port 3306
# Start new stream: 192.168.66.101->192.168.66.200 46632->3306
# Start new stream: 192.168.66.200->192.168.66.101 3306->46632
输出时候 无法使用 grep ,为什么?
or add a clean plugin demo
目前有办法获取到SQL的执行时间吗
多谢楼主的这个工具,很牛逼的样子。mac本地可以使用。
我想编译一个在Linux机器上使用,但是报错。
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o go-sniffer main.go
../pkg/mod/github.com/google/[email protected]/pcap/pcap.go:30:22: undefined: pcapErrorNotActivated
../pkg/mod/github.com/google/[email protected]/pcap/pcap.go:52:17: undefined: pcapTPtr
../pkg/mod/github.com/google/[email protected]/pcap/pcap.go:64:10: undefined: pcapPkthdr
../pkg/mod/github.com/google/[email protected]/pcap/pcap.go:103:6: undefined: pcapBpfProgram
../pkg/mod/github.com/google/[email protected]/pcap/pcap.go:110:7: undefined: pcapPkthdr
../pkg/mod/github.com/google/[email protected]/pcap/pcap.go:268:33: undefined: pcapErrorActivated
../pkg/mod/github.com/google/[email protected]/pcap/pcap.go:269:33: undefined: pcapWarningPromisc
../pkg/mod/github.com/google/[email protected]/pcap/pcap.go:270:33: undefined: pcapErrorNoSuchDevice
../pkg/mod/github.com/google/[email protected]/pcap/pcap.go:271:33: undefined: pcapErrorDenied
../pkg/mod/github.com/google/[email protected]/pcap/pcap.go:748:14: undefined: pcapTPtr
../pkg/mod/github.com/google/[email protected]/pcap/pcap.go:271:33: too many errors
panic: runtime error: index out of range
goroutine 7 [running]:
github.com/40t/go-sniffer/plugSrc/mysql/build.(*stream).resolveServerPacket(0xc420056d80, 0xc4204cd000, 0x0, 0x200, 0xb)
/data/go/src/github.com/40t/go-sniffer/plugSrc/mysql/build/entry.go:219 +0x442
github.com/40t/go-sniffer/plugSrc/mysql/build.(*stream).resolve(0xc420056d80)
/data/go/src/github.com/40t/go-sniffer/plugSrc/mysql/build/entry.go:194 +0xc0
created by github.com/40t/go-sniffer/plugSrc/mysql/build.(*Mysql).ResolveStream
/data/go/src/github.com/40t/go-sniffer/plugSrc/mysql/build/entry.go:71 +0x3b8
When I use Sysbench to test the MySQL and go-sniffer, I am not able to capture all the packet.I print logs and find that the packet is in the queue.
只有下面两条,完了什么也没有了
# go-sniffer lo mysql
tcp and port 3306
# Start new stream: 127.0.0.1->127.0.0.1 38994->3306
# Start new stream: 127.0.0.1->127.0.0.1 3306->38994
两个虚拟机
[root@geone ~]# go-sniffer ens33 mysql -p 3306
tcp and port 3306
只能看到链接的信息,但是151操作137的相关操作,却看不到?为啥,
例如,捕捉到一个有1000个参数的Insert的SQL语句的时候出错:
, @p906, @p907, @p908, @p909, @p910, @p911, @p912, @p913, @p914, @p915, @p916, @p917, @p918, @p919, @p920, @p921, @p922, @p923, @p924, @p925, @p926, @p927, @p928, @p929, @p930, @p931, @p932, @p933, @p934, @p935, @p936, @p937, @p938, @p939, @p940, @p941, @p942, @p943, @p944, @p945, @p946, @p947, @p948, @p949, @p950, @p951, @p952, @p953, @p954, @p955, @p956, @p957, @p958, @p959, @p960, @p961, @p962, @p963, @p964, @p965, @p966, @p967, @p968, @p969, @p970, @p971, @p972, @p973, @p974, @p975, @p976, @p977, @p978, @p979, @p980, @p981, @p982, @p983, @p984, @p985, @p986, @p987, @p988, @p989, @p990, @p991, @p992, @p993, @p994, @p995, @p996, @p997, @p998, @p999;
Drop stm id[4];
panic: runtime error: index out of range
goroutine 1411 [running]:
encoding/binary.binary.littleEndian.Uint32(...)
/usr/local/go/src/encoding/binary/binary.go:63
github.com/40t/go-sniffer/plugSrc/mysql/build.LengthBinary(0xc4206d6c01, 0x32, 0x5ff, 0x9b47d8, 0xc420044c00)
/root/go/src/github.com/40t/go-sniffer/plugSrc/mysql/build/util.go:41 +0xc2
github.com/40t/go-sniffer/plugSrc/mysql/build.(*stream).resolveServerPacket(0xc4200571c0, 0xc4206d6c00, 0x33, 0x600, 0x1)
/root/go/src/github.com/40t/go-sniffer/plugSrc/mysql/build/entry.go:230 +0x2fc
github.com/40t/go-sniffer/plugSrc/mysql/build.(*stream).resolve(0xc4200571c0)
/root/go/src/github.com/40t/go-sniffer/plugSrc/mysql/build/entry.go:194 +0xc0
created by github.com/40t/go-sniffer/plugSrc/mysql/build.(*Mysql).ResolveStream
/root/go/src/github.com/40t/go-sniffer/plugSrc/mysql/build/entry.go:71 +0x3b8
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.