Giter Club home page Giter Club logo

myq-tools's People

Contributors

jayjanssen avatar schnitzel avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

myq-tools's Issues

1m instead of .5+m

        Cluster  Node       Outbound      Inbound       FlowC     Conflct Gcache     Appl
    time P cnf  # stat laten msgs data que msgs data que pause snt lcf bfa   ist  idx  %ef
15:14:30 P   6  2 Jing 0.7ms    0   0b   0 2152 3.5M  1m   0ns   0   0   0    90  306 132%
15:14:31 P   6  2 Jing 0.7ms    0   0b   0 2448 4.0M  1m   0ns   0   0   0   136   62 132%
15:14:32 P   6  2 Jing 0.7ms    0   0b   0 2437 3.9M  1m   0ns   0   0   0    91  309 132%
15:14:33 P   6  2 Jing 0.7ms    0   0b   0 2592 4.2M  1m   0ns   0   0   0    10   57 132%
15:14:34 P   6  2 Jing 0.7ms    0   0b   0 2462 4.0M  1m   0ns   0   0   0    84  287 132%
15:14:35 P   6  2 Jing 0.7ms    0   0b   0 2378 3.8M  1m   0ns   0   0   0    90  306 132%
15:14:36 P   6  2 Jing 0.7ms    0   0b   0 2396 3.9M  1m   0ns   0   0   0   123  404 132%
15:14:37 P   6  2 Jing 0.7ms    0   0b   0 2185 3.5M  1m   0ns   0   0   0    45  170 132%
15:14:38 P   6  2 Jing 0.7ms    0   0b   0 2209 3.6M  1m   0ns   0   0   0   119  393 131%
15:14:39 P   6  2 Jing 0.7ms    0   0b   0 2204 3.6M  1m   0ns   0   0   0   128  420 131%
15:14:40 P   6  2 Jing 0.7ms    0   0b   0 2178 3.5M  1m   0ns   0   0   0   205  651 131%
15:14:41 P   6  2 Jing 0.7ms    0   0b   0 2247 3.6M  1m   0ns   0   0   0   120  396 131%
15:14:42 P   6  2 Jing 0.7ms    0   0b   0 2553 4.1M  1m   0ns   0   0   0    31  125 131%
15:14:43 P   6  2 Jing 0.7ms    0   0b   0 2199 3.6M  1m   0ns   0   0   0   129  423 131%
15:14:44 P   6  2 Jing 0.7ms    0   0b   0 2570 4.2M  1m   0ns   0   0   0   181  578 131%
15:14:45 P   6  2 Jing 0.7ms    0   0b   0 2441 3.9M  1m   0ns   0   0   0   116  383 131%
15:14:46 P   6  2 Jing 0.7ms    0   0b   0 2595 4.2M  1m   0ns   0   0   0    35  140 131%
15:14:47 P   6  2 Jing 0.7ms    0   0b   0 2238 3.6M  1m   0ns   0   0   0    17   81 131%
15:14:48 P   6  2 Jing 0.7ms    0   0b   0 2532 4.1M  1m   0ns   0   0   0   215  679 130%
15:14:49 P   6  2 Jing 0.7ms    0   0b   0 2287 3.7M .5m   0ns   0   0   0    54  195 130%
15:14:50 P   6  2 Jing 0.7ms    0   0b   0  325 537K  1m   0ns   0   0   0    37  144 130%
15:14:51 P   6  2 Jing 0.7ms    0   0b   0 1669 2.7M .5m   0ns   0   0   0    64  226 130%
15:14:52 P   6  2 Jing 0.7ms    0   0b   0 2271 3.7M .5m   0ns   0   0   0   107  357 130%
15:14:53 P   6  2 Jing 0.7ms    0   0b   0 2691 4.3M .5m   0ns   0   0   0    10   56 130%
15:14:54 P   6  2 Jing 0.7ms    0   0b   0 1982 3.2M .5m   0ns   0   0   0   100  335 130%
15:14:55 P   6  2 Jing 0.7ms    0   0b   0 2586 4.2M .5m   0ns   0   0   0    43  162 130%

random crash

[root@ps1 sbtest]# myq_status cttf
panic: runtime error: slice bounds out of range

goroutine 15 [running]:
github.com/jayjanssen/myq-tools/myqlib.parseSamples.func1(0xc8201569be, 0x17, 0x1642, 0xc82010c00a)
/home/travis/gopath/src/github.com/jayjanssen/myq-tools/myqlib/parse_show.go:40 +0x2c4
github.com/jayjanssen/myq-tools/myqlib.parseSamples.func2(0xc8201569be, 0x1642, 0x1642, 0x2000, 0x1b0, 0x0, 0x0, 0x0, 0x0, 0x0)
/home/travis/gopath/src/github.com/jayjanssen/myq-tools/myqlib/parse_show.go:75 +0x313
github.com/jayjanssen/myq-tools/myqlib.(*Scanner).Scan(0xc820047ec8, 0xc8200527e0)
/home/travis/gopath/src/github.com/jayjanssen/myq-tools/myqlib/Scanner.go:126 +0x36c
github.com/jayjanssen/myq-tools/myqlib.parseSamples(0x7f2a454767a8, 0xc82002e0a0, 0xc820019560, 0x3b9aca00)
/home/travis/gopath/src/github.com/jayjanssen/myq-tools/myqlib/parse_show.go:91 +0x377
github.com/jayjanssen/myq-tools/myqlib.LiveLoader.harvestMySQL.func4(0xc820019560, 0x7f2a454768a8, 0xc82002e0a0, 0x3b9aca00, 0x0, 0x0)
/home/travis/gopath/src/github.com/jayjanssen/myq-tools/myqlib/loader.go:313 +0x9a
created by github.com/jayjanssen/myq-tools/myqlib.LiveLoader.harvestMySQL
/home/travis/gopath/src/github.com/jayjanssen/myq-tools/myqlib/loader.go:314 +0x5e0

goroutine 1 [runnable]:
main.main()
/home/travis/gopath/src/github.com/jayjanssen/myq-tools/myq_status.go:169 +0x165d

goroutine 5 [syscall]:
os/signal.loop()
/usr/local/go/src/os/signal/signal_unix.go:22 +0x18
created by os/signal.init.1
/usr/local/go/src/os/signal/signal_unix.go:28 +0x37

Optimize interval parsing

interval parsing in parse_show.go waits until it has a full sample and then scans back through the sample to find the uptime to decide if the sample should be dropped. Instead it should watch for the uptime as it scans forward and record it once for that sample when found.

Re-add view for galera statistics

Hello!

Do you see the possibility to re-add the view for the galera statistics to the latest version? We've been using it for monitoring and it was really handy to quickly get the cluster status.


Version 1.x

~# ./myq_status -a -uroot -a "-S /run/mysql.sock" wsrep

galera / node01 (idx: 1) / Galera 25.3.35(r545d0bf)
         Cluster  Node       Outbound      Inbound       FlowC     Conflct Gcache     Appl
    time P cnf  # stat laten msgs data que msgs data que pause snt lcf bfa   ist  idx  %ef
08:58:36 P  61  3 Sync 0.0ns    6 4.3K   0 1.1m 834M   0   0ns   0   0   0  323k    9 100%

Version: 971e478 from main branch

./myq_status -u root -S /run/mysql.sock wsrep
  view wsrep not found
  myq-tools  ()

Thanks in advance,
David

Restructure parsing to not use mysqladmin

Still plagued with 'Buffer too large'. Want to explore using the mysql cli instead in --batch mode, which should mean much smaller buffers and much more efficient parsing

Crash bug

panic: runtime error: slice bounds out of range goroutine 7 [running]: github.com/jayjanssen/myq-tools/myqlib.scanMySQLShowLines(0xc208064080, 0xc20800b320) /home/travis/gopath/src/github.com/jayjanssen/myq-tools/myqlib/parse_show.go:52 +0x746 github.com/jayjanssen/myq-tools/myqlib.func·004() /home/travis/gopath/src/github.com/jayjanssen/myq-tools/myqlib/loader.go:211 +0x8d created by github.com/jayjanssen/myq-tools/myqlib.FileLoader.harvestFile /home/travis/gopath/src/github.com/jayjanssen/myq-tools/myqlib/loader.go:212 +0x35a goroutine 1 [chan receive]: main.main() /home/travis/gopath/src/github.com/jayjanssen/myq-tools/myq_status.go:146 +0x13db goroutine 5 [syscall]: os/signal.loop() /home/travis/.gimme/versions/go1.4.linux.amd64/src/os/signal/signal_unix.go:21 +0x1f created by os/signal.init·1 /home/travis/.gimme/versions/go1.4.linux.amd64/src/os/signal/signal_unix.go:27 +0x35 goroutine 8 [runnable]: github.com/jayjanssen/myq-tools/myqlib.func·003() /home/travis/gopath/src/github.com/jayjanssen/myq-tools/myqlib/loader.go:130 +0xda created by github.com/jayjanssen/myq-tools/myqlib.GetState /home/travis/gopath/src/github.com/jayjanssen/myq-tools/myqlib/loader.go:177 +0x3aa

cannot find package "github.com/jayjanssen/myq-tools/myqlib"

I've YUM installed GO on Centos 7.2 and tried to deploy myq-tools both by downloading and extracting from ZIP, and by GIT clone. Same results :

myq_status.go:7:2: cannot find package "github.com/jayjanssen/myq-tools/myqlib" in any of:
/usr/lib/golang/src/github.com/jayjanssen/myq-tools/myqlib (from $GOROOT)
/home/golang/src/github.com/jayjanssen/myq-tools/myqlib (from $GOPATH)

The only way I've been able to get this to work is to create the whole expected path by adding in /src/github.com/jayjannsen and moving myq-tools into that location. I then have to CD into that new location to run the tool.

Platform is Centos 7.2.

Steps taken

sudo yum install git
cd /home/golang/
git clone https://github.com/jayjanssen/myq-tools.git
go run myq_status.go
cd myq-tools/
go run myq_status.go

Returns error as above.

Here is my GO env from the server I used

GOARCH="amd64"
GOBIN="/usr/lib/golang/bin"
GOCHAR="6"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/golang"
GORACE=""
GOROOT="/usr/lib/golang"
GOTOOLDIR="/usr/lib/golang/pkg/tool/linux_amd64"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0"
CXX="g++"
CGO_ENABLED="1"

To work around the issue :

mkdir /home/golang/src
mkdir /home/golang/src/github.com
mkdir /home/golang/src/github.com/jayjanssen
mv /home/golang/myq-tools /home/golang/src/github.com/jayjanssen/

Then to run:
cd /home/golang/src/github.com/jayjanssen/myq-tools/
go run myq_status.go
go run myq_status.go wsrep

This all feels a little clunky. Is this expected behavior? Did I do something wrong?

2 intervals waited before printing

[root@node3 ~]# date; myq_status -i 5s wsrep
Tue Jul 14 14:06:42 UTC 2015
mycluster / node3 (idx: 1) / Galera 3.11(ra0189ab)
         Cluster  Node       Outbound      Inbound       FlowC     Conflct Gcache     Appl
    time P cnf  # stat laten msgs data que msgs data que pause snt lcf bfa   ist  idx  %ef
14:06:52 P  11  3 Sync 1.0ms 3.1m 2.0G   0 6.4m 4.0G   0   0ns   0  2k  10 3001k   35 131%
14:06:57 P  11  3 Sync 0.9ms 1529 1.0M   0 3120 2.0M   0   0ns   0   4   0 3001k   60 131%
14:07:02 P  11  3 Sync 1.0ms 1021 673K   0 2115 1.3M   0   0ns   0   4   0 3001k  213 131%

Syntax error at loader.go:313

:/code/myq-tools$ git clone https://github.com/jayjanssen/myq-tools.git
Cloning into 'myq-tools'...
remote: Counting objects: 855, done.
remote: Total 855 (delta 0), reused 0 (delta 0), pack-reused 855
Receiving objects: 100% (855/855), 7.84 MiB | 6.36 MiB/s, done.
Resolving deltas: 100% (566/566), done.
Checking connectivity... done.
:
/code/myq-tools$ go run myq_status.go
myq_status.go:7:2: cannot find package "github.com/jayjanssen/myq-tools/myqlib" in any of:
/usr/lib/go/src/pkg/github.com/jayjanssen/myq-tools/myqlib (from $GOROOT)
($GOPATH not set)
:/code/myq-tools$ sed -i 's:github.com/jayjanssen/myq-tools/myqlib:../myq-tools/myqlib:g' myq_status.go
:
/code/myq-tools$ GOPATH=/home/user/code/ go run myq_status.go

_/home/user/code/myq-tools/myqlib

myqlib/loader.go:313: syntax error: unexpected range, expecting {
myqlib/loader.go:323: non-declaration statement outside function body
myqlib/loader.go:326: non-declaration statement outside function body
myqlib/loader.go:327: syntax error: unexpected }

Handle missing variables better:

was from wsrep view

panic: interface conversion: interface is nil, not string

goroutine 16 [running]:
runtime.panic(0x5166e0, 0xc208040fc0)
/usr/lib/golang/src/pkg/runtime/panic.c:279 +0xf5
_/vagrant/myqlib.func·010(0xc20802a2a0, 0xc20800fb80, 0x7fe4faf687a8, 0xc2080e51d0)
/vagrant/myqlib/view_defaults.go:224 +0xa7
_/vagrant/myqlib.FuncCol.Data(0x53bb50, 0x5, 0x553450, 0x1b, 0x5, 0x5746e0, 0xc20802a2a0, 0xc20800fb80)
/vagrant/myqlib/col.go:229 +0x96
_/vagrant/myqlib.(_FuncCol).Data(0xc2080244e0, 0xc20802a2a0, 0xc20800fb80)
:47 +0xa5
_/vagrant/myqlib.NormalView.Data(0xc20804e000, 0x9, 0x9, 0x5494d0, 0x17, 0x5746d8, 0xc20802a2a0, 0xc20800fb80)
/vagrant/myqlib/view.go:78 +0xec
_/vagrant/myqlib.(_NormalView).Data(0xc2080246c0, 0xc20802a2a0, 0xc20800fb80)
:115 +0xa5
main.main()
/vagrant/myq_status.go:152 +0x1207

Cleanup for can't connect errors

go run myq_status.go -a "-u test" wsrep

panic: Could not write to MySQL command any longer

goroutine 1 [running]:
github.com/jayjanssen/myq-tools/myqlib.func·005()
/go/src/github.com/jayjanssen/myq-tools/myqlib/loader.go:287 +0xce
github.com/jayjanssen/myq-tools/myqlib.LiveLoader.harvestMySQL(0x3b9aca00, 0x7fff2a7c66cd, 0x7, 0x5c3e10, 0x21, 0x7fb2ba590000, 0x0, 0x0)
/go/src/github.com/jayjanssen/myq-tools/myqlib/loader.go:291 +0x660
github.com/jayjanssen/myq-tools/myqlib.LiveLoader.getStatus(0x3b9aca00, 0x7fff2a7c66cd, 0x7, 0x522000, 0x0, 0x0)
/go/src/github.com/jayjanssen/myq-tools/myqlib/loader.go:322 +0x5c
github.com/jayjanssen/myq-tools/myqlib.(*LiveLoader).getStatus(0xc20801e560, 0xc2080340f8, 0x0, 0x0)
:90 +0xb3
github.com/jayjanssen/myq-tools/myqlib.GetState(0x7fb2ba5a2040, 0xc20801e560, 0x6a2d50, 0x0, 0x0)
/go/src/github.com/jayjanssen/myq-tools/myqlib/loader.go:126 +0x232
main.main()
/vagrant/myq_status.go:143 +0x12f6

goroutine 5 [syscall]:
os/signal.loop()
/usr/lib/golang/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
/usr/lib/golang/src/os/signal/signal_unix.go:27 +0x35

goroutine 12 [runnable]:
os/exec.func·004(0xc20801ea00)
/usr/lib/golang/src/os/exec/exec.go:327
created by os/exec.(*Cmd).Start
/usr/lib/golang/src/os/exec/exec.go:329 +0xb6d

goroutine 9 [chan receive]:
github.com/jayjanssen/myq-tools/myqlib.func·006()
/go/src/github.com/jayjanssen/myq-tools/myqlib/loader.go:297 +0xb0
created by github.com/jayjanssen/myq-tools/myqlib.LiveLoader.harvestMySQL
/go/src/github.com/jayjanssen/myq-tools/myqlib/loader.go:300 +0x74d

goroutine 11 [runnable]:
os/exec.(*Cmd).Wait(0xc208058140, 0x0, 0x0)
/usr/lib/golang/src/os/exec/exec.go:369 +0x30b
github.com/jayjanssen/myq-tools/myqlib.func·008()
/go/src/github.com/jayjanssen/myq-tools/myqlib/loader.go:311 +0x36
created by github.com/jayjanssen/myq-tools/myqlib.LiveLoader.harvestMySQL
/go/src/github.com/jayjanssen/myq-tools/myqlib/loader.go:316 +0x8ca
exit status 2

header out of order in wsrep

         Cluster  Node       Outbound      Inbound       FlowC     Conflct Gcache     Appl
    time P cnf  # stat laten msgs data que msgs data que pause snt lcf bfa   ist  idx  %ef
mycluster / node3 (idx: 2) / Galera 3.8(rf6147dd)

0.3 preferred above 3xx

0.3m -- innodb bp data, would be better as 300k in the same space

probably the precision number

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.