maliceio / malice Goto Github PK

VirusTotal Wanna Be - Now with 100% more Hipster

License: Apache License 2.0

Makefile 1.95% Shell 0.91% Ruby 0.07% Go 94.40% API Blueprint 1.06% Python 0.33% HTML 0.11% Dockerfile 1.16%

antivirus cloud cybersecurity dfir docker elasticsearch golang infosec malice malware malware-analysis malware-research virustotal

malice's Introduction

maliceio

Malice.IO website

malice's People

Contributors

Stargazers

Watchers

Forkers

gitter-badger boh pombredanne redteamcaliber antiquum ch0c01d kevinmel2000 y0d4a knowmalware awesome-security awesome-golang golang-kit hesaul ivosandoval jack51706 jhonthenripper repson pdkien furusiyya prats84 alilazy puppycodes poeblu crazykid199 ajmartel s-bourassa bigrayhicks iamstoxe securitywarrior billyprice1old secsecsec mustafakisa headmin appsdocforks visaobuon mikalv team-firebugs houcy nextsecurity olivierh59500 heikipikker wxdublin ob1xxx jonike bijaye primmus duzhanyuan ivstefano sk1llz fingerleakers ykankaya lukw00heck suriya73 benrau87 vaginessa alienzox noodled cpesitbeijing grukz 4n6strider d4mn4t10n 0x1c0x13 espandy charygao xuacker ith4cker oksbsb threatinteltest essobi cccnam5158 madisettisunil littleairmada unlimitet cikgufatah jonasmfreitas modulexcite tobey123 choubear stevenchen0x01 liebesu nakagit rajivraj n4rr34n6 porpoise-trend idkwim killvxk rodriguezcappsec charfeddinemohamedali rdsece mortezamofa gavz m00zh33 shahid1996 sajibekanti yangboyd rongqinglee utkutugrul josu22 jay52018 binlmmhc

malice's Issues

Malice binary can't talk to docker unless ran in go get src directory

ability to get a different type of hash from other intel sources to feed back into other intel plugins

For example the malice/nsrl plugin only takes md5s currently, I could ask VT about it if I had only the sha256 and then get the md5 and feed that into the nsrl plugin.

Invalid memory address

Output of go version:

go version go1.7.1 darwin/amd64

Output of docker version:

Client:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.7
 Git commit:   23cf638
 Built:        Thu Aug 18 22:32:50 UTC 2016
 OS/Arch:      darwin/amd64

Server:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   23cf638
 Built:        Thu Aug 18 17:52:38 2016
 OS/Arch:      linux/amd64

Output of docker info:

Containers: 1
 Running: 1
 Paused: 0
 Stopped: 0
Images: 1
Server Version: 1.12.1
Storage Driver: aufs
 Root Dir: /mnt/sda1/var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 7
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: host null bridge overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: seccomp
Kernel Version: 4.4.17-boot2docker
Operating System: Boot2Docker 1.12.1 (TCL 7.2); HEAD : ef7d0b4 - Thu Aug 18 21:18:06 UTC 2016
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 1.955 GiB
Name: default
ID: LIAJ:H4I2:Y56S:4YLO:U3DD:A2NW:U6JB:GBH7:TS3H:OVEH:EDOV:A37W
Docker Root Dir: /mnt/sda1/var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 18
 Goroutines: 33
 System Time: 2016-10-14T23:00:00.642758435Z
 EventsListeners: 1
Registry: https://index.docker.io/v1/
Labels:
 provider=virtualbox
Insecure Registries:
 127.0.0.0/8

I'm using Docker Toolbox.

Steps to reproduce the issue:
$ ~ λ malice --debug scan /Users/ForgottenPlayer/Downloads/pprx.exe

Describe the results you received:
I got an error stating:

DEBU[0000] Malice config loaded from /Users/ForgottenPlayer/.malice/config.toml 
DEBU[0000] Malice plugins loaded from /Users/ForgottenPlayer/.malice/plugins.toml 
DEBU[0000] Malice Version: 0.1.0-alpha, build HEAD      
DEBU[0000] Trusting 1 certs                             
DEBU[0000] Connected to docker daemon with docker-machine  ip=192.168.99.100 port=2376
DEBU[0000] Searching for container: rethink              env=development
DEBU[0000] name:  rethink   container.Name:  rethink    
DEBU[0000] MATCH:  true                                 
DEBU[0000] Container FOUND: rethink                      env=development
DEBU[0000] Attempting to connect to: rethink:28015      
DEBU[0000] Attempting to connect to: 172.17.0.2:28015   
DEBU[0002] Attempting to connect to: localhost:28015    
ERRO[0002] "runtime error: invalid memory address or nil pointer dereference"
goroutine 1 [running]:
github.com/maliceio/malice/malice/errors.CheckErrorWithMessage(0x48ba940, 0xc420016080, 0x0, 0x0, 0x0, 0x0, 0x0, 0x45bb740)
    /private/tmp/malice-20160913-13247-1qvpfbq/gopath/src/github.com/maliceio/malice/malice/errors/errors.go:24 +0x90
github.com/maliceio/malice/malice/errors.CheckError(0x48ba940, 0xc420016080, 0x4)
    /private/tmp/malice-20160913-13247-1qvpfbq/gopath/src/github.com/maliceio/malice/malice/errors/errors.go:12 +0x62
main.main()
    /private/tmp/malice-20160913-13247-1qvpfbq/gopath/src/github.com/maliceio/malice/main.go:80 +0x36e

switch all plugins to use Elasticsearch as their primary DB :'(

I love you RethinkDB I love you sooooo much. It's not you it's me I swear !!!! 😭

Malice can't create .malice dir ?

Cannot Disable Plugin

@blacktop Trying to remove the offender plugin on ticket #30 - shadow-server.

Deleting the plugin does not work .

root@malice:# malice plugin list
virustotal
shadow-server
fileinfo
yara
avg
bitdefender
clamav
comodo
fprot
f-secure
sophos
floss
root@malice:# malice plugin remove shadow-server
root@malice:# malice plugin list
virustotal
shadow-server
fileinfo
yara
avg
bitdefender
clamav
comodo
fprot
f-secure
sophos
floss
root@malice:#

need to add ability for malice to update config schema

This is going to stop people from being able to upgrade malice easily.

handle bad flags

❯❯❯ go run main.go plugin list -d -a                                                                                                            
Incorrect Usage.

Usage: malice list [OPTIONS] [arg...]
list enabled installed plugins

Options:

   --all    display all installed plugins
   --detail, -d display plugin details
ERRO[0000] "flag provided but not defined: -a"
goroutine 1 [running]:
github.com/maliceio/malice/malice/errors.CheckErrorWithMessage(0x4846100, 0xc42032dad0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x4578ac0)
    /Users/me/src/go/src/github.com/maliceio/malice/malice/errors/errors.go:24 +0x90
github.com/maliceio/malice/malice/errors.CheckError(0x4846100, 0xc42032dad0, 0x5)
    /Users/me/src/go/src/github.com/maliceio/malice/malice/errors/errors.go:12 +0x62
main.main()
    /Users/me/src/go/src/github.com/maliceio/malice/main.go:81 +0x40d

Plugins need to properly handle errors

I need to output failures into plugin markdown/JSON output.

For example: submitting via the API.

panic: runtime error: index out of range [recovered]

root@malice:/# malice scan /home/xpwd/befb88b89c2eb401900a68e9f5b78764203f2b48264fcc3f7121bf04a57fd408

File

Field	Value
Name	befb88b89c2eb401900a68e9f5b78764203f2b48264fcc3f7121bf04a57fd408
Path	/home/xpwd/befb88b89c2eb401900a68e9f5b78764203f2b48264fcc3f7121bf04a57fd408
Size	40.96 kB
MD5	669f87f2ec48dce3a76386eec94d7e3b
SHA1	6b82f126555e7644816df5d4e4614677ee0bda5c
SHA256	befb88b89c2eb401900a68e9f5b78764203f2b48264fcc3f7121bf04a57fd408
Mime	application/x-dosexec

latest: Pulling from library/busybox
8ddc19f16526: Pull complete
Digest: sha256:a59906e33509d14c036c8678d687bd4eec81ed7c4b8ce907b888c607f6a1e0e6
Status: Downloaded newer image for busybox:latest

VirusTotal

Ratio	Link	API	Scanned
85%	link	Public	2016-02-15 11:47:03

ShadowServer

Not found
panic: runtime error: index out of range [recovered]
panic: runtime error: index out of range

goroutine 1 [running]:
panic(0x819620, 0xc420016080)
/usr/lib/go-1.7/src/runtime/panic.go:500 +0x1a1
github.com/urfave/cli.HandleAction.func1(0xc420051be8)
/go/src/github.com/urfave/cli/app.go:478 +0x247
panic(0x819620, 0xc420016080)
/usr/lib/go-1.7/src/runtime/panic.go:458 +0x243
main.ParseSsdeepOutput(0x0, 0x0, 0xc420051608, 0x1)
/go/src/github.com/maliceio/malice-fileinfo/scan.go:74 +0xd7
main.main.func1(0xc42008c780, 0x0, 0x0)
/go/src/github.com/maliceio/malice-fileinfo/scan.go:192 +0x161
reflect.Value.call(0x7fe0a0, 0x8d7be0, 0x13, 0x8980fd, 0x4, 0xc420051ba8, 0x1, 0x1, 0x4ca688, 0x884ba0, ...)
/usr/lib/go-1.7/src/reflect/value.go:434 +0x5c8
reflect.Value.Call(0x7fe0a0, 0x8d7be0, 0x13, 0xc420051ba8, 0x1, 0x1, 0x8d7b28, 0x0, 0x0)
/usr/lib/go-1.7/src/reflect/value.go:302 +0xa4
github.com/urfave/cli.HandleAction(0x7fe0a0, 0x8d7be0, 0xc42008c780, 0x0, 0x0)
/go/src/github.com/urfave/cli/app.go:487 +0x1e0
github.com/urfave/cli.(*App).Run(0xc4200e0000, 0xc42000c3c0, 0x3, 0x3, 0x0, 0x0)
/go/src/github.com/urfave/cli/app.go:245 +0x59b
main.main()
/go/src/github.com/maliceio/malice-fileinfo/scan.go:227 +0x56c

F-PROT

Infected	Result	Engine	Updated
false		4.6.5.141	20161005

2016/10/05 09:12:53 exit status 1
2016/10/05 09:12:53 could not open file

Comodo

Infected	Result	Engine	Updated
true	Backdoor.Win32.Lecna.AB	1.1

F-Secure

Infected	Result	Engine	Updated
true	Backdoor.Lecna.AB	11.00 build 79	20160928

Bitdefender

Infected	Result	Engine	Updated
true	Backdoor.Lecna.AB	7.90123	20161005

Sophos

Infected	Result	Engine	Updated
true	Troj/Lecna-Q	5.27.0	20160928

ClamAV

Infected	Result	Engine	Updated
true	Win.Trojan.Backspace-1	0.99.2	20160928

AVG

Infected	Result	Engine	Updated
true	Found Win32/DH{YQMT?}	13.0.3114	20160928

Cleanup code, docker/plugin/container/ etc utils !!!

reduce code smell

Get plugins to POST JSON to malice webhook which pipes it to ELK Container

Rewrite Plugins into goroutines for speed

Rewrite to reuse containers instead of spawning new one all the time (will speed up AV)

harden plugin containers

control net/disk/cpu plus use a seccomp profile

you should have an option to output JSON from scans instead of just Markdown tables

Create API

TODO

layout the routes
minimalist framework (just gorilla mux?)
add auto SSL (self-signed/letsencrypt)
private routes for inter-communication of plugins w/ malice-engine?

Fix/finalize RethinkDB data schema

most important step now.

Not running?

I already had docker installed, so I just I did this:

but when I tried running it, **this** happened...

Here is the full error log.

This is on:
OSX 10.11.6 (15G31).

Perhaps it's my docker install?

you have to run scan with --logs flags or it errors out

Cannot install malice

Output of go version:

go version go1.6 linux/amd64

Output of docker version:

Client:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   23cf638
 Built:        Thu Aug 18 05:22:43 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   23cf638
 Built:        Thu Aug 18 05:22:43 2016
 OS/Arch:      linux/amd64

Output of docker info:

Containers: 2
 Running: 0
 Paused: 0
 Stopped: 2
Images: 1
Server Version: 1.12.1
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 5
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: null bridge host overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: apparmor
Kernel Version: 3.19.0-69-generic
Operating System: Ubuntu 14.04.5 LTS
OSType: linux
Architecture: x86_64
CPUs: 24
Total Memory: 11.73 GiB
Name: cuckoo-001
ID: NTPM:YTBN:DGZN:CYC6:PRTO:IVJV:QIH7:B2BB:UHPM:7IPM:UUIM:3MUE
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Insecure Registries:
 127.0.0.0/8

Ubuntu 14.04.5 LTS \n \l

Describe the results you received:

go get github.com/maliceio/malice
package context: unrecognized import path "context" (import path does not begin with hostname)

Installation failing .

Secure Plugins

TODO

create a seccomp profile for different plugin types
create default cgroup settings
drop most capabilities

https://docs.docker.com/engine/security/seccomp/
https://docs.docker.com/engine/admin/resource_constraints/

seccomp

HostConfig. SecurityOpt

cgroup settings

HostConfig.Resources.Memory

drop most capabilities

HostConfig.CapAdd
HostConfig.CapDrop

Standardize AV outputs so that I could aggregate them together like this from a completed job in Elasticsearch

AntiVirus

AV	Infected	Result	Engine	Updated
Avast	👿	`Win.Trojan.Backspace-1`	0.99.2	20160919
AVG	👿	`Win32:Lecna-I [Trj]`	2.1.2	20170129
Bitdefender	👿	`Backdoor.Lecna.AB`	7.90123	20160919
ClamAV	👿	`Win.Trojan.Backspace-1`	0.99.2	20160919
Comodo	👿	`Backdoor.Win32.Lecna.AB`	1.1	20160919
F-PROT	😇		4.6.5.141	20160919
F-Secure	👿	`Backdoor.Lecna.AB`	11.00 build 79	20160919
Sophos	👿	`Troj/Lecna-Q`	5.27.0	20160920

Fix AVG plugin

seems buggy and crashes on zips etc

docker in docker is not working

vagrant@vagrant-ubuntu-trusty-64:/vagrant/data/samples$ docker run --rm -v /var/run/docker.sock:/var/run/docker.sock                 -v `pwd`:/malice/samples                 -e MALICE_VT_API=$MALICE_VT_API                 malice/engine -D scan 04beed90f6a7762d84a455f8855567906de079f48ddaabe311a6a281e90bd36f
DEBU[0000] Malice config loaded from /malice/config.toml
DEBU[0000] Malice plugins loaded from /malice/plugins.toml
DEBU[0000] Malice Version: 0.1.0-alpha, build HEAD
ERRO[0000] Unable to parse endpoint:
DEBU[0000] Connected to docker daemon with docker-machine  ip= port=
DEBU[0000] Searching for container: rethink              env=development
DEBU[0000] name:  rethink   container.Name:  modest_mcclintock
DEBU[0000] MATCH:  false
DEBU[0000] name:  rethink   container.Name:  rethink
DEBU[0000] MATCH:  true
DEBU[0000] Container FOUND: rethink                      env=development
DEBU[0000] Attempting to connect to: rethink:28015
DEBU[0000] Attempting to connect to: 172.17.0.3:28015
DEBU[0000] Attempting to connect to: 172.17.0.3:28015
DEBU[0000] gorethink: Database `test` does not exist. in:
r.DBDrop("test")
DEBU[0000] gorethink: Database `malice` already exists. in:
r.DBCreate("malice")
DEBU[0000] gorethink: Table `malice.samples` already exists. in:
r.DB("malice").TableCreate("samples")
DEBU[0000] Searching for image: malice/virustotal        env=development
DEBU[0000] Image FOUND: malice/virustotal                env=development
DEBU[0000] Searching for image: malice/shadow-server     env=development
DEBU[0000] Image FOUND: malice/shadow-server             env=development
DEBU[0000] Searching for image: malice/fileinfo          env=development
DEBU[0000] Image FOUND: malice/fileinfo                  env=development
DEBU[0000] Searching for image: malice/yara              env=development
DEBU[0000] Image FOUND: malice/yara                      env=development
DEBU[0000] Searching for image: malice/avast             env=development
DEBU[0000] Image FOUND: malice/avast                     env=development
DEBU[0000] Searching for image: malice/avg               env=development
DEBU[0000] Image FOUND: malice/avg                       env=development
DEBU[0000] Searching for image: malice/bitdefender       env=development
DEBU[0000] Image FOUND: malice/bitdefender               env=development
DEBU[0000] Searching for image: malice/clamav            env=development
DEBU[0000] Image FOUND: malice/clamav                    env=development
DEBU[0000] Searching for image: malice/comodo            env=development
DEBU[0000] Image FOUND: malice/comodo                    env=development
DEBU[0000] Searching for image: malice/fprot             env=development
DEBU[0000] Image FOUND: malice/fprot                     env=development
DEBU[0000] Searching for image: malice/floss             env=development
DEBU[0000] Image FOUND: malice/floss                     env=development
DEBU[0000] All enabled plugins are installed.
#### File
| Field  | Value                                                            |
| ------ | ---------------------------------------------------------------- |
| Name   | 04beed90f6a7762d84a455f8855567906de079f48ddaabe311a6a281e90bd36f |
| Path   | 04beed90f6a7762d84a455f8855567906de079f48ddaabe311a6a281e90bd36f |
| Size   | 1.167 MB                                                         |
| MD5    | 4971104db8e7b6437a037f868e089970                                 |
| SHA1   | dffaf052c2fb8f5a7fbba0a0af41454c3a4f5cf0                         |
| SHA256 | 04beed90f6a7762d84a455f8855567906de079f48ddaabe311a6a281e90bd36f |
| Mime   | application/octet-stream                                         |
DEBU[0000] Searching for Network: malice                 env=development
DEBU[0000] Network FOUND: malice                         env=development
DEBU[0000] Searching for volume: malice                  env=development
DEBU[0000] Volume FOUND: malice                          env=development
DEBU[0000] Volume malice found.
DEBU[0000] Searching for container: copy2volume          env=development
DEBU[0000] name:  copy2volume   container.Name:  modest_mcclintock
DEBU[0000] MATCH:  false
DEBU[0000] name:  copy2volume   container.Name:  rethink
DEBU[0000] MATCH:  false
DEBU[0000] Container NOT Found: copy2volume              env=development
DEBU[0000] Searching for image: busybox                  env=development
DEBU[0000] Image FOUND: busybox                          env=development
DEBU[0000] Image `busybox` already pulled.               env=development exisits=true
DEBU[0001] First statContainerPath call.          

SampledsDir=/malice/samples 

container.Name=/copy2volume 

dstInfo={Path:/malice/04beed90f6a7762d84a455f8855567906de079f48ddaabe311a6a281e90bd36f Exists:false IsDir:false RebaseName:}

dstStat={Name: Size:0 Mode:---------- Mtime:0001-01-01 00:00:00 +0000 UTC LinkTarget:}

file.Path=04beed90f6a7762d84a455f8855567906de079f48ddaabe311a6a281e90bd36f

volSavePath=/malice/04beed90f6a7762d84a455f8855567906de079f48ddaabe311a6a281e90bd36f

ERRO[0001] "lstat /04beed90f6a7762d84a455f8855567906de079f48ddaabe311a6a281e90bd36f: no such file or directory"
goroutine 1 [running]:
github.com/maliceio/malice/malice/errors.CheckErrorWithMessage(0x7f47a31a6b38, 0xc8201aa2a0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc8201aa2a0)
    /go/src/github.com/maliceio/malice/malice/errors/errors.go:24 +0x96
github.com/maliceio/malice/malice/errors.CheckError(0x7f47a31a6b38, 0xc8201aa2a0, 0x0)
    /go/src/github.com/maliceio/malice/malice/errors/errors.go:12 +0x48
github.com/maliceio/malice/malice/maldocker.(*Docker).CopyToVolume(0xc8203228d0, 0x7ffc01e4bf1d, 0x40, 0x7ffc01e4bf1d, 0x40, 0xc82037e658, 0x8, 0xc820394b40, 0x20, 0xc8203226f0, ...)
    /go/src/github.com/maliceio/malice/malice/maldocker/volume.go:118 +0xf4f
github.com/maliceio/malice/commands.cmdScan(0x7ffc01e4bf1d, 0x40, 0x0, 0x0, 0x0)
    /go/src/github.com/maliceio/malice/commands/scan.go:63 +0x78c
github.com/maliceio/malice/commands.glob.func1(0xc82030ca00, 0x0, 0x0)
    /go/src/github.com/maliceio/malice/commands/commands.go:25 +0x151
reflect.Value.call(0x9f5c40, 0xd37fb0, 0x13, 0xbddeb8, 0x4, 0xc8204d7470, 0x1, 0x1, 0x0, 0x0, ...)
    /usr/lib/go/src/reflect/value.go:435 +0x120d
reflect.Value.Call(0x9f5c40, 0xd37fb0, 0x13, 0xc8204d7470, 0x1, 0x1, 0x0, 0x0, 0x0)
    /usr/lib/go/src/reflect/value.go:303 +0xb1
github.com/maliceio/malice/vendor/github.com/urfave/cli.HandleAction(0x9f5c40, 0xd37fb0, 0xc82030ca00, 0x0, 0x0)
    /go/src/github.com/maliceio/malice/vendor/github.com/urfave/cli/app.go:480 +0x2ee
github.com/maliceio/malice/vendor/github.com/urfave/cli.Command.Run(0xbe5ed8, 0x4, 0x0, 0x0, 0x0, 0x0, 0x0, 0xbf1900, 0xb, 0x0, ...)
    /go/src/github.com/maliceio/malice/vendor/github.com/urfave/cli/command.go:186 +0x1301
github.com/maliceio/malice/vendor/github.com/urfave/cli.(*App).Run(0xc820386000, 0xc8200660c0, 0x4, 0x4, 0x0, 0x0)
    /go/src/github.com/maliceio/malice/vendor/github.com/urfave/cli/app.go:236 +0xa9c
main.main()
    /go/src/github.com/maliceio/malice/main.go:76 +0x467


ERRO[0001] "lstat : no such file or directory"
goroutine 1 [running]:
github.com/maliceio/malice/malice/errors.CheckErrorWithMessage(0x7f47a31a6b38, 0xc8201aa300, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc8201aa300)
    /go/src/github.com/maliceio/malice/malice/errors/errors.go:24 +0x96
github.com/maliceio/malice/malice/errors.CheckError(0x7f47a31a6b38, 0xc8201aa300, 0x0)
    /go/src/github.com/maliceio/malice/malice/errors/errors.go:12 +0x48
github.com/maliceio/malice/malice/maldocker.(*Docker).CopyToVolume(0xc8203228d0, 0x7ffc01e4bf1d, 0x40, 0x7ffc01e4bf1d, 0x40, 0xc82037e658, 0x8, 0xc820394b40, 0x20, 0xc8203226f0, ...)
    /go/src/github.com/maliceio/malice/malice/maldocker/volume.go:121 +0xfe1
github.com/maliceio/malice/commands.cmdScan(0x7ffc01e4bf1d, 0x40, 0x0, 0x0, 0x0)
    /go/src/github.com/maliceio/malice/commands/scan.go:63 +0x78c
github.com/maliceio/malice/commands.glob.func1(0xc82030ca00, 0x0, 0x0)
    /go/src/github.com/maliceio/malice/commands/commands.go:25 +0x151
reflect.Value.call(0x9f5c40, 0xd37fb0, 0x13, 0xbddeb8, 0x4, 0xc8204d7470, 0x1, 0x1, 0x0, 0x0, ...)
    /usr/lib/go/src/reflect/value.go:435 +0x120d
reflect.Value.Call(0x9f5c40, 0xd37fb0, 0x13, 0xc8204d7470, 0x1, 0x1, 0x0, 0x0, 0x0)
    /usr/lib/go/src/reflect/value.go:303 +0xb1
github.com/maliceio/malice/vendor/github.com/urfave/cli.HandleAction(0x9f5c40, 0xd37fb0, 0xc82030ca00, 0x0, 0x0)
    /go/src/github.com/maliceio/malice/vendor/github.com/urfave/cli/app.go:480 +0x2ee
github.com/maliceio/malice/vendor/github.com/urfave/cli.Command.Run(0xbe5ed8, 0x4, 0x0, 0x0, 0x0, 0x0, 0x0, 0xbf1900, 0xb, 0x0, ...)
    /go/src/github.com/maliceio/malice/vendor/github.com/urfave/cli/command.go:186 +0x1301
github.com/maliceio/malice/vendor/github.com/urfave/cli.(*App).Run(0xc820386000, 0xc8200660c0, 0x4, 0x4, 0x0, 0x0)
    /go/src/github.com/maliceio/malice/vendor/github.com/urfave/cli/app.go:236 +0xa9c
main.main()
    /go/src/github.com/maliceio/malice/main.go:76 +0x467


ERRO[0001] "runtime error: invalid memory address or nil pointer dereference"
goroutine 1 [running]:
github.com/maliceio/malice/malice/errors.CheckErrorWithMessage(0x7f47a321c000, 0xc820010080, 0x0, 0x0, 0x0, 0x0, 0x0, 0x2)
    /go/src/github.com/maliceio/malice/malice/errors/errors.go:24 +0x96
github.com/maliceio/malice/malice/errors.CheckError(0x7f47a321c000, 0xc820010080, 0x4)
    /go/src/github.com/maliceio/malice/malice/errors/errors.go:12 +0x48
main.main()
    /go/src/github.com/maliceio/malice/main.go:77 +0x489

malice on linux can't find the configs

vagrant@vagrant-ubuntu-trusty-64:~$ malice -D lookup 6fe80e56ad4de610304bab1675ce84d16ab6988e
ERRO[0000] "open /home/vagrant/.malice/config.toml: no such file or directory"
goroutine 1 [running, locked to thread]:
github.com/maliceio/malice/malice/errors.CheckErrorWithMessage(0xf66780, 0xc4202a58c0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x23)
    /home/vagrant/go/src/github.com/maliceio/malice/malice/errors/errors.go:24 +0x90
github.com/maliceio/malice/malice/errors.CheckError(0xf66780, 0xc4202a58c0, 0xc42028b800)
    /home/vagrant/go/src/github.com/maliceio/malice/malice/errors/errors.go:12 +0x62
github.com/maliceio/malice/config.Load()
    /home/vagrant/go/src/github.com/maliceio/malice/config/load.go:120 +0x342
github.com/maliceio/malice/malice/logger.Init()
    /home/vagrant/go/src/github.com/maliceio/malice/malice/logger/logger.go:55 +0x26
main.init.1()
    /home/vagrant/go/src/github.com/maliceio/malice/main.go:20 +0x14
main.init()
    /home/vagrant/go/src/github.com/maliceio/malice/main.go:82 +0x65

bug in the switch to Elasticsearch 5.0.1

~/s/g/d/2.5 git:master ❯❯❯ malice lookup 6fe80e56ad4de610304bab1675ce84d16ab6988e

2016/11/23 10:49:06 elastic: Error 400 (Bad Request): Validation Failed: 1: an id must be provided if version type or value are set; [type=action_request_validation_exception]

Is expecting a ID for lookups where they should be auto generated

Add full libmagic description string

The libmagic full description string contains more information than just the mimetype. For example, it indicates multi-typed files, such as self-executing (SFX) archives.

Here is what I am thinking: 530a256

I am not a go coder, so may have done that wrong. Still figuring out how to test my changes.

Get http://localhost:9200/: dial tcp [::1]:9200: getsockopt: connection refused

Output of go version:

go version go1.7.1 linux/amd64

Output of docker version:

Client:
 Version:      1.13.0
 API version:  1.25
 Go version:   go1.7.3
 Git commit:   49bf474
 Built:        Tue Jan 17 09:44:08 2017
 OS/Arch:      linux/amd64

Server:
 Version:      1.13.0
 API version:  1.25 (minimum version 1.12)
 Go version:   go1.7.3
 Git commit:   49bf474
 Built:        Tue Jan 17 09:44:08 2017
 OS/Arch:      linux/amd64
 Experimental: false

Output of docker info:

Containers: 7
 Running: 0
 Paused: 0
 Stopped: 7
Images: 21
Server Version: 1.13.0
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 142
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins: 
 Volume: local
 Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 03e5862ec0d8d3b3f750e19fca3ee367e13c090e
runc version: 2f7393a47307a16f8cee44a37b262e8b81021e3e
init version: 949e6fa
Kernel Version: 3.16.0-4-amd64
Operating System: Debian GNU/Linux 8 (jessie)
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 5.597 GiB
Name: debian
ID: 6G3A:VAFC:U2R7:T7YQ:MXFG:JFPV:RJQC:WT74:ZUWF:EI7P:RZEB:CWR2
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No memory limit support
WARNING: No swap limit support
WARNING: No kernel memory limit support
WARNING: No oom kill disable support
WARNING: No cpu cfs quota support
WARNING: No cpu cfs period support
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, Docker For Mac, Docker Toolbox, docker-machine, etc.):
The installation worked except for avast plugin. I tried to restart the docker elk but know I get this error
Get http://localhost:9200/: dial tcp [::1]:9200: getsockopt: connection refused.
It's probably something really dumb though...

Steps to reproduce the issue:

docker stop malice-elk
malice elk start
malice -D scan ~/go/bin/malice

Describe the results you received:
DEBU[0000] Malice config loaded from: /home/algosecure/.malice/config.toml
DEBU[0000] Malice plugins loaded from: /home/algosecure/.malice/plugins.toml
DEBU[0000] Using 2 PROCS
DEBU[0000] Malice Version: 0.2.0-alpha
DEBU[0000] Running inside Docker...
DEBU[0000] Connected to docker daemon client ip=localhost port=2375
DEBU[0000] Searching for container: malice-elk env=development
DEBU[0000] name: malice-elk container.Name: loving_mccarthy
DEBU[0000] MATCH: false
DEBU[0000] name: malice-elk container.Name: cocky_lumiere
DEBU[0000] MATCH: false
DEBU[0000] name: malice-elk container.Name: objective_kalam
DEBU[0000] MATCH: false
DEBU[0000] name: malice-elk container.Name: modest_stallman
DEBU[0000] MATCH: false
DEBU[0000] name: malice-elk container.Name: malice-elk
DEBU[0000] MATCH: true
DEBU[0000] Container FOUND: malice-elk env=development
DEBU[0000] ELK is running. image=blacktop/elastic-stack:malice ip= network=default
DEBU[0000] Attempting to connect to: http://localhost:9200
2017/02/06 12:01:36 Get http://localhost:9200/: dial tcp [::1]:9200: getsockopt: connection refused

Describe the results you expected:
A normal scan with avast not working

Additional information you deem important (e.g. issue happens only occasionally):

AV should format like VT

    "AVG": {
      "detected": true,
      "version": "16.0.0.4522",
      "result": "Win32/DH{YQMT?}",
      "update": "20160214"
    }

Plugins failure

Output of go version:

go version go1.7.1 linux/amd64

Output of docker version:

Client:
 Version:      1.13.0
 API version:  1.25
 Go version:   go1.7.3
 Git commit:   49bf474
 Built:        Tue Jan 17 09:44:08 2017
 OS/Arch:      linux/amd64

Server:
 Version:      1.13.0
 API version:  1.25 (minimum version 1.12)
 Go version:   go1.7.3
 Git commit:   49bf474
 Built:        Tue Jan 17 09:44:08 2017
 OS/Arch:      linux/amd64
 Experimental: false

Output of docker info:

Containers: 38
 Running: 1
 Paused: 0
 Stopped: 37
Images: 21
Server Version: 1.13.0
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 224
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins: 
 Volume: local
 Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 03e5862ec0d8d3b3f750e19fca3ee367e13c090e
runc version: 2f7393a47307a16f8cee44a37b262e8b81021e3e
init version: 949e6fa
Kernel Version: 3.16.0-4-amd64
Operating System: Debian GNU/Linux 8 (jessie)
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 5.597 GiB
Name: debian
ID: 6G3A:VAFC:U2R7:T7YQ:MXFG:JFPV:RJQC:WT74:ZUWF:EI7P:RZEB:CWR2
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No memory limit support
WARNING: No swap limit support
WARNING: No kernel memory limit support
WARNING: No oom kill disable support
WARNING: No cpu cfs quota support
WARNING: No cpu cfs period support
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, Docker For Mac, Docker Toolbox, docker-machine, etc.):
I'm running on a real machine, quite slow but quick enough for my needs.

Steps to reproduce the issue:

Just malice scan SAMPLE after clean install

Describe the results you received:
Sorted in time, I have all those plugins that fail.
Avast, totalhash,Exiftools, NSRL, avg

File

Field	Value
Name	malware.exe
Path	Téléchargements/malware.exe
Size	5.993 kB
MD5	65bb7a968098bb6b3d62e7edf7cdae39
SHA1	012d4de9f1439348d89dae0e3a2d1ddaf33f31ac
SHA256	b9bfb323d15ad4669781cb93e3c8f01fd2ad37b60d77c43fbe57b0942fbc0598

NSRL Database

Not Found ❔

ShadowServer

Not found

2017/02/07 13:47:52 cannot open; magic mime db is already open

Comodo

Infected	Result	Engine	Updated
true	Malware	1.1	20170129
panic: runtime error: index out of range

goroutine 1 [running]:
panic(0x85cde0, 0xc420014090)
/usr/local/go/src/runtime/panic.go:500 +0x1a1
main.ParseAvastOutput(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/go/src/github.com/maliceio/malice-avast/scan.go:119 +0x575
main.AvScan(0x78, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/go/src/github.com/maliceio/malice-avast/scan.go:84 +0x2b9
main.main.func3(0xc42008a780, 0x0, 0x0)
/go/src/github.com/maliceio/malice-avast/scan.go:318 +0x17f
github.com/urfave/cli.HandleAction(0x83fe00, 0xc4200154a0, 0xc42008a780, 0x0, 0x0)
/go/src/github.com/urfave/cli/app.go:485 +0xd4
github.com/urfave/cli.(*App).Run(0xc42007cea0, 0xc42000c330, 0x3, 0x3, 0x0, 0x0)
/go/src/github.com/urfave/cli/app.go:259 +0x74f
main.main()
/go/src/github.com/maliceio/malice-avast/scan.go:357 +0x78f

F-PROT

Infected	Result	Engine	Updated
true	Toothless.873	4.6.5.141	20170129
time="2017-02-07T13:48:19Z" level=fatal msg="Please supply a valid #totalhash user/key with the flags '--user' and '--key'"
time="2017-02-07T13:48:19Z" level=fatal msg="Please supply a valid MALICE_VT_API key with the flag '--api'."

Magic

Field	Value
Mime	application/octet-stream
Description	DOS executable (COM)

SSDeep

24:kT5IyR8dK0LhNqB9sIBzHMb5Js1io1fGOwRQ1O4TulnUxm:kRRF0feBjwU1HuJoTQ

TRiD

Unknown!

Exiftool

Field	Value
error	exit status 1

Yara

No Matches

ShadowServer

Not found

time="2017-02-07T13:48:14Z" level=fatal msg="Please supply a valid SHA1 hash to query NSRL with."

Floss

Decoded Strings

No Strings

Stack Strings

No Strings

F-Secure

Infected	Result	Engine	Updated
true	PS-MPC.0873.AD.Gen	11.10 build 68	20170130
time="2017-02-07T13:48:53Z" level=fatal msg="Command /etc/init.d/avgd timed out." category=av path="/malware/b9bfb323d15ad4669781cb93e3c8f01fd2ad37b60d77c43fbe57b0942fbc0598" plugin=avg

Bitdefender

Infected	Result	Engine	Updated
true	PS-MPC.0873.AD.Gen	7.90123	20170129

Sophos

Infected	Result	Engine	Updated
true	Lady	5.31.0	20170130

ClamAV

Infected	Result	Engine	Updated
true	Win.Trojan.RedArc-5	0.99.2	20170130

Describe the results you expected:
I would like to have at least avast and/or avg working =)

Additional information you deem important (e.g. issue happens only occasionally):
Issue happens every time (Exiftools seems to work from time to time)
Thank you =)

ERRO[0021] Get http://localhost:9200/: dial tcp [::1]:9200: getsockopt: connection refused

Output of go version:

go version go1.7.4 linux/amd64

Output of docker version:

Client:
 Version:      1.12.5
 API version:  1.24
 Go version:   go1.6.4
 Git commit:   7392c3b
 Built:        Fri Dec 16 02:21:54 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.12.5
 API version:  1.24
 Go version:   go1.6.4
 Git commit:   7392c3b
 Built:        Fri Dec 16 02:21:54 2016
 OS/Arch:      linux/amd64

Output of docker info:

Containers: 2
 Running: 0
 Paused: 0
 Stopped: 2
Images: 23
Server Version: 1.12.5
Storage Driver: devicemapper
 Pool Name: docker-8:1-49557999-pool
 Pool Blocksize: 65.54 kB
 Base Device Size: 10.74 GB
 Backing Filesystem: ext4
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 7.588 GB
 Data Space Total: 107.4 GB
 Data Space Available: 99.79 GB
 Metadata Space Used: 7.213 MB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.14 GB
 Thin Pool Minimum Free Space: 10.74 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: false
 Deferred Deletion Enabled: false
 Deferred Deleted Device Count: 0
 Data loop file: /var/lib/docker/devicemapper/devicemapper/data
 WARNING: Usage of loopback devices is strongly discouraged for production use. Use `--storage-opt dm.thinpooldev` to specify a custom block storage device.
 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.137 (2016-11-30)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: null bridge host overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options:
Kernel Version: 4.9.0-kali3-amd64
Operating System: Kali GNU/Linux Rolling
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 7.783 GiB
Name: kali20171
ID: A3QN:QDEP:FKCB:OZHO:FHMA:2RTS:ZCLV:O7ZG:QPRS:TFOI:IMBT:4D7I
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Insecure Registries:
 127.0.0.0/8

Additional environment details (AWS, VirtualBox, physical, Docker For Mac, Docker Toolbox, docker-machine, etc.):
Kali Linux 2017.1 (64bit), Memory: 8GB

Steps to reproduce the issue:

apt-get update
cd /etc/apt/sources.list.d/
gedit backports.list &
add 1 line:
deb http://http.debian.net/debian wheezy-backports main
apt-get update
apt-get install apt-transport-https ca-certificates gnupg2 dirmngr
apt-key adv --keyserver hkp://ha.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
gedit /etc/apt/sources.list.d/docker.list &
add 1 line:
deb https://apt.dockerproject.org/repo debian-jessie main
apt-get update
apt-cache policy docker-engine
apt-get install docker-engine=1.12.5-0~debian-jessie
usermod -aG docker $USER
apt-get install golang
cd
gedit .bashrc &
add 2 lines at the bottom:
export GOPATH=$HOME
export PATH=$PATH:$GOPATH/bin
source .bashrc
cd /opt
wget https://github.com/maliceio/malice/releases/download/v0.2.0-alpha/malice_0.2.0-alpha_linux_amd64.zip -O /tmp/malice.zip
unzip /tmp/malice.zip -d /usr/local/bin/
malice plugin update --all
malice plugin list --all --detail
systemctl enable docker
malice elk

Describe the results you received:

malice elk

ERRO[0000] Network malice does not exist, creating now... env=development exisits=false network=malice
INFO[0000] Created Network: malice env=development name=malice
INFO[0000] Created Volume: malice env=development
INFO[0001] Elasticsearch Container Started env=development ip=localhost name="/malice-elastic" port=[9200]
INFO[0001] Waiting for Elasticsearch to come online. server="http://localhost:9200" timeout=20
ERRO[0021] connecting to elasticsearch timed out timeout=20
ERRO[0021] Get http://localhost:9200/: dial tcp [::1]:9200: getsockopt: connection refused
ERRO[0021] Get http://localhost:9200/: dial tcp [::1]:9200: getsockopt: connection refused
malice: Pulling from blacktop/kibana
627beaf3eaaf: Already exists
0c8e9a12d743: Pull complete
3fab1effe157: Pull complete
d6d275309877: Pull complete
9b7f57263aaf: Pull complete
Digest: sha256:3c069a0ec9f046d7853d53f67075c77dabdb17ac363dac72a1a11b8d20ea4e56
Status: Downloaded newer image for blacktop/kibana:malice
ERRO[0038] StartContainer error = Error response from daemon: Cannot link to a non running container: /malice-elastic AS /malice-kibana/elasticsearch
env=development
INFO[0038] Kibana Container Started env=development ip=localhost name="/malice-kibana" port=[443]

Describe the results you expected:
I think Elasticsearch container didn't start properly.

Additional information you deem important (e.g. issue happens only occasionally):
N/A

Kindest regards,
YN (nakagit)

Server Misbehaving..

docker run --net=host --rm -v /var/run/docker.sock:/var/run/docker.sock -v pwd:/malice/samples -e MALICE_VT_API=$MALICE_VT_API malice/engine scan .

2017/04/06 20:29:51 Get http://elastic:9200/: dial tcp: lookup elastic on 127.0.1.1:53: server misbehaving

Docker logs


2017-04-06 20:20:09,356 INFO stopped: nginx (exit status 0)
2017-04-06 20:22:38,014 CRIT Supervisor running as root (no user in config file)
2017-04-06 20:22:38,016 INFO supervisord started with pid 7
2017-04-06 20:22:39,018 INFO spawned: 'nginx' with pid 10
2017-04-06 20:22:39,019 INFO spawned: 'elasticsearch' with pid 11
2017-04-06 20:22:39,019 INFO spawned: 'logstash' with pid 12
2017-04-06 20:22:39,020 INFO spawned: 'kibana' with pid 13
2017-04-06 20:22:39,148 INFO exited: elasticsearch (exit status 1; not expected)
2017-04-06 20:22:40,070 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-04-06 20:22:40,070 INFO success: logstash entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-04-06 20:22:41,072 INFO spawned: 'elasticsearch' with pid 87
2017-04-06 20:22:41,512 INFO exited: elasticsearch (exit status 1; not expected)
2017-04-06 20:22:44,070 INFO spawned: 'elasticsearch' with pid 120
2017-04-06 20:22:44,070 INFO success: kibana entered RUNNING state, process has stayed up for > than 5 seconds (startsecs)
2017-04-06 20:22:44,357 INFO exited: elasticsearch (exit status 1; not expected)
2017-04-06 20:22:47,753 INFO spawned: 'elasticsearch' with pid 148
2017-04-06 20:22:47,945 INFO exited: elasticsearch (exit status 1; not expected)
2017-04-06 20:22:48,471 INFO gave up: elasticsearch entered FATAL state, too many start retries too quickly
2017-04-06 20:24:14,486 WARN received SIGTERM indicating exit request
2017-04-06 20:24:14,486 INFO waiting for nginx, logstash, kibana to die
2017-04-06 20:24:14,491 INFO stopped: kibana (exit status 143)
2017-04-06 20:24:17,632 INFO waiting for nginx, logstash to die
2017-04-06 20:24:20,634 INFO waiting for nginx, logstash to die
2017-04-06 20:24:21,251 INFO stopped: logstash (exit status 0)
2017-04-06 20:24:22,290 INFO stopped: nginx (exit status 0)

bug were on first run it will try to start blacktop/elastic-stack then exit

Figure out why wait-for-it is exiting malice

Cannot install Malice in Ubuntu 16.04.1

Output of go version:

go version go1.6.2 linux/amd64

Output of docker version:

Client:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   23cf638
 Built:        Thu Aug 18 05:33:38 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   23cf638
 Built:        Thu Aug 18 05:33:38 2016
 OS/Arch:      linux/amd64

Output of docker info:

Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 0
Server Version: 1.12.1
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 0
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: host overlay bridge null
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: apparmor seccomp
Kernel Version: 4.4.0-38-generic
Operating System: Ubuntu 16.04.1 LTS
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 3.859 GiB
Name: kalxas
ID: VTAX:V47H:7DMS:TYKW:GGM3:Y5OK:IA3F:4WSJ:TJNW:QOEE:CCMI:4WUG
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Insecure Registries:
 127.0.0.0/8

Additional environment details (AWS, VirtualBox, physical, Docker For Mac, Docker Toolbox, docker-machine, etc.):

VM inside ESXi

Steps to reproduce the issue:

Followed installation instructions for linux as mentioned here: https://github.com/maliceio/malice/blob/master/docs/installation/linux/install.md
OS is Ubuntu 16.04.1 LTS Xenial (I know instructions are for Ubuntu 14.04.3)
Added this in .bashrc and .profile:
export PATH=$PATH:/usr/local/go/bin
export GOPATH=$HOME/go
export PATH=$PATH:$GOPATH/bin
Issue occurs in this step: go get -v github.com/maliceio/malice

Describe the results you received:
$ go get -v -u github.com/maliceio/malice
github.com/maliceio/malice (download)
package context: unrecognized import path "context" (import path does not begin with hostname)
Fetching https://golang.org/x/crypto/ed25519?go-get=1
Parsing meta tags from https://golang.org/x/crypto/ed25519?go-get=1 (status code 200)
get "golang.org/x/crypto/ed25519": found meta tag main.metaImport{Prefix:"golang.org/x/crypto", VCS:"git", RepoRoot:"https://go.googlesource.com/crypto"} at https://golang.org/x/crypto/ed25519?go-get=1
get "golang.org/x/crypto/ed25519": verifying non-authoritative meta tag
Fetching https://golang.org/x/crypto?go-get=1
Parsing meta tags from https://golang.org/x/crypto?go-get=1 (status code 200)
golang.org/x/crypto (download)
Fetching https://golang.org/x/crypto/ed25519/internal/edwards25519?go-get=1
Parsing meta tags from https://golang.org/x/crypto/ed25519/internal/edwards25519?go-get=1 (status code 200)
get "golang.org/x/crypto/ed25519/internal/edwards25519": found meta tag main.metaImport{Prefix:"golang.org/x/crypto", VCS:"git", RepoRoot:"https://go.googlesource.com/crypto"} at https://golang.org/x/crypto/ed25519/internal/edwards25519?go-get=1
get "golang.org/x/crypto/ed25519/internal/edwards25519": verifying non-authoritative meta tag

Describe the results you expected:
Install Malice

Additional information you deem important (e.g. issue happens only occasionally):

you should offer to download all plugins at first run

otherwise it does during scan time

Plugin install failing Ubuntu 14.04

Output of go version:

go version go1.7.1 linux/amd64

Output of docker version:

Client:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   23cf638
 Built:        Thu Aug 18 05:22:43 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   23cf638
 Built:        Thu Aug 18 05:22:43 2016
 OS/Arch:      linux/amd64

Output of docker info:

Containers: 13
 Running: 1
 Paused: 0
 Stopped: 12
Images: 27
Server Version: 1.12.1
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 128
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: null host bridge overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: apparmor
Kernel Version: 4.4.0-31-generic
Operating System: Ubuntu 14.04.5 LTS
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 3.842 GiB
Name: malice
ID: P7KN:S3HU:SXY3:RG45:FFVN:6XFG:RG4X:FL5T:IZ3Z:Q4G3:2XTT:WR3E
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Insecure Registries:
 127.0.0.0/8

Steps to reproduce the issue:

malice plugin update --all

Describe the results you received:

Running hooks in /etc/ca-certificates/update.d....done.
.+ set -x
.+ echo Install F-PROT...
.+ tar -C /opt -zxvf /go/src/github.com/maliceio/malice-fprot/fp-Linux.x86.32-ws.tar.gz
Install F-PROT...

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now

/var/lib/docker/aufs/diff/13606656ea9e840002d158b03ab10e12b7d8475488a082cd40e81c0316914b6e/go/src/github.com/maliceio/malice-fprot/fp-Linux.x86.32-ws.tar.gz
/var/lib/docker/aufs/diff/4074f24c1deefc648e24774d894f9c9cec34c791782cf935f3e4dee2021f62b5/go/src/github.com/maliceio/malice-fprot/fp-Linux.x86.32-ws.tar.gz
/var/lib/docker/aufs/diff/604cce9e6f6d6962139c5f014ba3104abf28746ae9f04fbd1f8a0d713606ff7a/go/src/github.com/maliceio/malice-fprot/fp-Linux.x86.32-ws.tar.gz
/var/lib/docker/aufs/diff/6d25119dd51e190a8aac6858dfc2ff5d32d087acb1af8c94f55a85765a5e9257/go/src/github.com/maliceio/malice-fprot/fp-Linux.x86.32-ws.tar.gz
/var/lib/docker/aufs/diff/791cad2ef84f3ed5b142a0e20550fd0afff16e5cfad98b6a1258ab9da88ba4f7/go/src/github.com/maliceio/malice-fprot/fp-Linux.x86.32-ws.tar.gz
root@malice:/opt# file /var/lib/docker/aufs/diff/13606656ea9e840002d158b03ab10e12b7d8475488a082cd40e81c0316914b6e/go/src/github.com/maliceio/malice-fprot/fp-Linux.x86.32-ws.tar.gz
/var/lib/docker/aufs/diff/13606656ea9e840002d158b03ab10e12b7d8475488a082cd40e81c0316914b6e/go/src/github.com/maliceio/malice-fprot/fp-Linux.x86.32-ws.tar.gz: ASCII text
root@malice:/opt# file /var/lib/docker/aufs/diff/4074f24c1deefc648e24774d894f9c9cec34c791782cf935f3e4dee2021f62b5/go/src/github.com/maliceio/malice-fprot/fp-Linux.x86.32-ws.tar.gz
/var/lib/docker/aufs/diff/4074f24c1deefc648e24774d894f9c9cec34c791782cf935f3e4dee2021f62b5/go/src/github.com/maliceio/malice-fprot/fp-Linux.x86.32-ws.tar.gz: ASCII text
root@malice:/opt# file /var/lib/docker/aufs/diff/604cce9e6f6d6962139c5f014ba3104abf28746ae9f04fbd1f8a0d713606ff7a/go/src/github.com/maliceio/malice-fprot/fp-Linux.x86.32-ws.tar.gz
/var/lib/docker/aufs/diff/604cce9e6f6d6962139c5f014ba3104abf28746ae9f04fbd1f8a0d713606ff7a/go/src/github.com/maliceio/malice-fprot/fp-Linux.x86.32-ws.tar.gz: ASCII text
root@malice:/opt# file /var/lib/docker/aufs/diff/6d25119dd51e190a8aac6858dfc2ff5d32d087acb1af8c94f55a85765a5e9257/go/src/github.com/maliceio/malice-fprot/fp-Linux.x86.32-ws.tar.gz
/var/lib/docker/aufs/diff/6d25119dd51e190a8aac6858dfc2ff5d32d087acb1af8c94f55a85765a5e9257/go/src/github.com/maliceio/malice-fprot/fp-Linux.x86.32-ws.tar.gz: ASCII text
root@malice:/opt# file /var/lib/docker/aufs/diff/791cad2ef84f3ed5b142a0e20550fd0afff16e5cfad98b6a1258ab9da88ba4f7/go/src/github.com/maliceio/malice-fprot/fp-Linux.x86.32-ws.tar.gz
/var/lib/docker/aufs/diff/791cad2ef84f3ed5b142a0e20550fd0afff16e5cfad98b6a1258ab9da88ba4f7/go/src/github.com/maliceio/malice-fprot/fp-Linux.x86.32-ws.tar.gz: ASCII text

output of the files:
version https://git-lfs.github.com/spec/v1
oid sha256:aad50674ea3657894b5f9a11a7f3cdb476638c5e43c95a7a26f62ebe565083e1
size 30914110

Ability have malice 'watch' a folder for new files

and process them as they come in.

remove libmagic as a malice engine requirement

This prevents me from easily cross compiling and making statically compiled binaries 😢

you need to fix libmalice/malmachine

package maldocker

// Sample Virtualbox create independent of Machine CLI.
import (
    "encoding/json"
    "fmt"

    log "github.com/Sirupsen/logrus"
    "github.com/docker/machine/commands/mcndirs"
    "github.com/docker/machine/drivers/virtualbox"
    "github.com/docker/machine/libmachine"
    er "github.com/maliceio/malice/libmalice/errors"
)

// MakeDockerMachine creates a new docker host via docker-machine
func MakeDockerMachine(host string) {
    // log.SetDebug(true)

    client := libmachine.NewClient(mcndirs.GetBaseDir(), mcndirs.GetMachineCertDir())

    hostName := host

    // Set some options on the provider...
    driver := virtualbox.NewDriver(hostName, mcndirs.GetBaseDir())
    driver.CPU = 2
    driver.Memory = 2048

    data, err := json.Marshal(driver)
    er.CheckError(err)

    // pluginDriver, err := client.NewPluginDriver("virtualbox", data)
    // er.CheckError(err)

    h, err := client.NewHost("virtualbox", data)
    // h, err := client.NewHost(pluginDriver)
    er.CheckError(err)

    h.HostOptions.EngineOptions.StorageDriver = "overlay"

    if err := client.Create(h); err != nil {
        log.Fatal(err)
    }

    out, err := h.RunSSHCommand("df -h")
    if err != nil {
        log.Fatal(err)
    }

    fmt.Printf("Results of your disk space query:\n%s\n", out)

    fmt.Println("Powering down machine now...")
    if err := h.Stop(); err != nil {
        log.Fatal(err)
    }
}

// MachineURL returns the IP of the docker-machine
func MachineURL(name string) (url string, err error) {

    api := libmachine.NewClient(mcndirs.GetBaseDir(), mcndirs.GetMachineCertDir())

    host, err := api.Load(name)
    er.CheckError(err)
    url, err = host.URL()
    er.CheckError(err)

    return
}

// MachineIP returns the IP of the docker-machine
func MachineIP(name string) (ip string, err error) {

    api := libmachine.NewClient(mcndirs.GetBaseDir(), mcndirs.GetMachineCertDir())

    host, err := api.Load(name)
    er.CheckError(err)
    ip, err = host.Driver.GetIP()
    er.CheckError(err)

    return
}

// MachineStop stops the docker-machine
func MachineStop(name string) error {

    api := libmachine.NewClient(mcndirs.GetBaseDir(), mcndirs.GetMachineCertDir())

    host, err := api.Load(name)
    er.CheckError(err)
    err = host.Driver.Stop()

    return err
}

If user supplies `MALICE_ELASTICSEARCH` don't start blacktop/elk

❯❯❯ MALICE_ELASTICSEARCH=localhost go run main.go scan data/samples/befb88b89c2eb401900a68e9f5b78764203f2b48264fcc3f7121bf04a57fd408

ERRO[0000] ELK is NOT running, starting now...
ERRO[0000] Network malice does not exist, creating now...  env=development exisits=false network=malice
INFO[0000] Created Network: malice                       env=development name=malice
INFO[0000] Created Volume: malice                        env=development
ERRO[0001] StartContainer error = Error response from daemon: driver failed programming external connectivity on endpoint malice-elk (08faff5eb8edb70cc0a417169619304d3d7421be8e67aa31e3a0f0783a6f5d3b): Bind for 0.0.0.0:9200 failed: port is already allocated
  env=development
INFO[0001] Sleeping for 10 seconds to give blacktop/elk time to initalize.
#### File
| Field  | Value                                                                         |
| ------ | ----------------------------------------------------------------------------- |
| Name   | befb88b89c2eb401900a68e9f5b78764203f2b48264fcc3f7121bf04a57fd408              |
| Path   | data/samples/befb88b89c2eb401900a68e9f5b78764203f2b48264fcc3f7121bf04a57fd408 |
| Size   | 40.96 kB                                                                      |
| MD5    | 669f87f2ec48dce3a76386eec94d7e3b                                              |
| SHA1   | 6b82f126555e7644816df5d4e4614677ee0bda5c                                      |
| SHA256 | befb88b89c2eb401900a68e9f5b78764203f2b48264fcc3f7121bf04a57fd408              |
ERRO[0014] StartContainer error = Error response from daemon: Cannot link to a non running container: /malice-elk AS /shadow-server/elastic
  env=development
ERRO[0014] StartContainer error = Error response from daemon: Cannot link to a non running container: /malice-elk AS /virustotal/elastic
  env=development
ERRO[0017] StartContainer error = Error response from daemon: Cannot link to a non running container: /malice-elk AS /floss/elastic
  env=development
ERRO[0017] StartContainer error = Error response from daemon: Cannot link to a non running container: /malice-elk AS /avg/elastic
  env=development
ERRO[0017] StartContainer error = Error response from daemon: Cannot link to a non running container: /malice-elk AS /yara/elastic
  env=development
ERRO[0017] StartContainer error = Error response from daemon: Cannot link to a non running container: /malice-elk AS /bitdefender/elastic
  env=development
ERRO[0017] StartContainer error = Error response from daemon: Cannot link to a non running container: /malice-elk AS /clamav/elastic
  env=development
ERRO[0018] StartContainer error = Error response from daemon: Cannot link to a non running container: /malice-elk AS /fprot/elastic
  env=development
ERRO[0018] StartContainer error = Error response from daemon: Cannot link to a non running container: /malice-elk AS /fileinfo/elastic
  env=development
ERRO[0018] StartContainer error = Error response from daemon: Cannot link to a non running container: /malice-elk AS /comodo/elastic
  env=development
ERRO[0018] StartContainer error = Error response from daemon: Cannot link to a non running container: /malice-elk AS /sophos/elastic
  env=development
ERRO[0018] StartContainer error = Error response from daemon: Cannot link to a non running container: /malice-elk AS /f-secure/elastic
  env=development

Docker in Docker install - invalid memory address or nil pointer dereference

Output of go version:

go version go1.8.1 linux/amd64

Output of docker version:

Client:
 Version:      17.05.0-ce
 API version:  1.29
 Go version:   go1.7.5
 Git commit:   89658be
 Built:        Thu May  4 22:04:27 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.05.0-ce
 API version:  1.29 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   89658be
 Built:        Thu May  4 22:04:27 2017
 OS/Arch:      linux/amd64
 Experimental: false

Output of docker info:

Containers: 3
 Running: 3
 Paused: 0
 Stopped: 0
Images: 75
Server Version: 17.05.0-ce
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 165
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins: 
 Volume: local
 Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 9048e5e50717ea4497b757314bad98ea3763c145
runc version: 9c2d8d184e5da67c95d601382adf14862e4f2228
init version: 949e6fa
Kernel Version: 3.16.0-4-amd64
Operating System: Debian GNU/Linux 8 (jessie)
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 7.784GiB
Name: pcfixe
ID: Z5OE:S2LE:MED4:CDUX:SXGX:STDV:AVKM:PBKN:UQSD:G6NG:GPWS:ELM5
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

WARNING: No memory limit support
WARNING: No swap limit support
WARNING: No kernel memory limit support
WARNING: No oom kill disable support
WARNING: No cpu cfs quota support
WARNING: No cpu cfs period support

Additional environment details (AWS, VirtualBox, physical, Docker For Mac, Docker Toolbox, docker-machine, etc.):

Operating System: Debian GNU/Linux 8.8 (jessie)
Kernel: Linux 3.16.0-4-amd64
Architecture: x86-64
Memory: 8GB

Steps to reproduce the issue:

docker run --rm -v /var/run/docker.sock:/var/run/docker.sock malice/engine plugin update --all

Describe the results you received:
I've this error

[Updating Plugin] ===>  javascript
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x55bdc25c6779]

goroutine 1 [running]:
panic(0x55bdc2cab980, 0xc420012050)
	/usr/lib/go/src/runtime/panic.go:500 +0x1a5
github.com/maliceio/malice/malice/docker/client/image.Pull(0xc420384ab0, 0xc42034c980, 0x11, 0x55bdc2920fc6, 0x6)
	/go/src/github.com/maliceio/malice/malice/docker/client/image/image.go:40 +0xf9
github.com/maliceio/malice/plugins.UpdateAllPlugins(0xc420384ab0)
	/go/src/github.com/maliceio/malice/plugins/plugins.go:261 +0x306
github.com/maliceio/malice/commands.cmdUpdatePlugin(0x0, 0x0, 0x1, 0x0, 0xc4203660c0)
	/go/src/github.com/maliceio/malice/commands/plugin.go:161 +0x277
github.com/maliceio/malice/commands.glob..func8(0xc4202d3a40, 0x0, 0xc4202d3a40)
	/go/src/github.com/maliceio/malice/commands/commands.go:138 +0xc5
github.com/maliceio/malice/vendor/github.com/urfave/cli.HandleAction(0x55bdc2c82660, 0x55bdc2d65e20, 0xc4202d3a40, 0xc420366000, 0x0)
	/go/src/github.com/maliceio/malice/vendor/github.com/urfave/cli/app.go:485 +0xd6
github.com/maliceio/malice/vendor/github.com/urfave/cli.Command.Run(0x55bdc29211a0, 0x6, 0x0, 0x0, 0x0, 0x0, 0x0, 0x55bdc29258f2, 0xd, 0x0, ...)
	/go/src/github.com/maliceio/malice/vendor/github.com/urfave/cli/command.go:207 +0xb98
github.com/maliceio/malice/vendor/github.com/urfave/cli.(*App).RunAsSubcommand(0xc420091860, 0xc4202d37c0, 0x0, 0x0)
	/go/src/github.com/maliceio/malice/vendor/github.com/urfave/cli/app.go:374 +0xb1c
github.com/maliceio/malice/vendor/github.com/urfave/cli.Command.startApp(0x55bdc292104a, 0x6, 0x0, 0x0, 0x0, 0x0, 0x0, 0x55bdc2930cca, 0x1f, 0x0, ...)
	/go/src/github.com/maliceio/malice/vendor/github.com/urfave/cli/command.go:294 +0x82e
github.com/maliceio/malice/vendor/github.com/urfave/cli.Command.Run(0x55bdc292104a, 0x6, 0x0, 0x0, 0x0, 0x0, 0x0, 0x55bdc2930cca, 0x1f, 0x0, ...)
	/go/src/github.com/maliceio/malice/vendor/github.com/urfave/cli/command.go:93 +0x16a7
github.com/maliceio/malice/vendor/github.com/urfave/cli.(*App).Run(0xc4200916c0, 0xc42000c180, 0x4, 0x4, 0x0, 0x0)
	/go/src/github.com/maliceio/malice/vendor/github.com/urfave/cli/app.go:250 +0x814
main.main()
	/go/src/github.com/maliceio/malice/main.go:88 +0x54a

Click here if you want to see the full result of the command

nicolas@pcfixe: ~ # docker run --rm -v /var/run/docker.sock:/var/run/docker.sock malice/engine plugin update --all
latest: Pulling from library/busybox
Digest: sha256:32f093055929dbc23dec4d03e09dfe971f5973a9ca5cf059cbfb644c206aa83f
Status: Image is up to date for busybox:latest
5.3: Pulling from blacktop/elasticsearch
Digest: sha256:081d4717e9d570a39f33f6cbed23879d507b361a6a22b6463fc90a7a31be9eb6
Status: Image is up to date for blacktop/elasticsearch:5.3
[Updating Plugin] ===>  nsrl
sha1: Pulling from malice/nsrl
Digest: sha256:c7e50532b1861841b4a78af375abc617c07e93400bf87b303978e86f21a38edc
Status: Image is up to date for malice/nsrl:sha1
[Updating Plugin] ===>  virustotal
latest: Pulling from malice/virustotal
Digest: sha256:99d4a908677c86d9a5d576edf2b1309038f10eb221caf6fb432324f8d7d3c9fe
Status: Image is up to date for malice/virustotal:latest
[Updating Plugin] ===>  totalhash
latest: Pulling from malice/totalhash
Digest: sha256:42a49d9628919089e9093a1de54767371206abc56fe3a7f2754f89371e2cedb4
Status: Image is up to date for malice/totalhash:latest
[Updating Plugin] ===>  shadow-server
latest: Pulling from malice/shadow-server
Digest: sha256:729ee2dee5912fbba9c5df0324294855df8e11b7d0cc1da30ea45265764c1615
Status: Image is up to date for malice/shadow-server:latest
[Updating Plugin] ===>  team-cymru
latest: Pulling from malice/team-cymru
3c3d46b04bf5: Already exists 
a3ed95caeb02: Already exists 
eb1c9d68a781: Already exists 
043ca925c043: Already exists 
Digest: sha256:99c1d8b92d47cf720c1b6bfd0a9123eab8086d1b0896d8f1e465fed2ed652880
Status: Image is up to date for malice/team-cymru:latest
[Updating Plugin] ===>  fileinfo
latest: Pulling from malice/fileinfo
Digest: sha256:7bee3f79b38c97f2bcd60457d3e2daf6aeb1205c465089ea732787ac46e1103e
Status: Image is up to date for malice/fileinfo:latest
[Updating Plugin] ===>  yara
latest: Pulling from malice/yara
Digest: sha256:d9e2173cf99b23f514007a2300a1833beda1af9c52d9eac6808e04f48ca133ca
Status: Image is up to date for malice/yara:latest
[Updating Plugin] ===>  avast
latest: Pulling from malice/avast
Digest: sha256:834aa8ac01927d446345e2e0ed85437ad7e3ade40060a62583f388e0ae87b71b
Status: Image is up to date for malice/avast:latest
[Updating Plugin] ===>  avg
latest: Pulling from malice/avg
Digest: sha256:211130df8460da113c3cef33ead4b6c3a448a1e5d07d0f01948540c3f1e93d3b
Status: Image is up to date for malice/avg:latest
[Updating Plugin] ===>  bitdefender
latest: Pulling from malice/bitdefender
Digest: sha256:bf74082342d7299cfa4cf7a26873041da23f3da66b2859b42d5b95476d846e30
Status: Image is up to date for malice/bitdefender:latest
[Updating Plugin] ===>  clamav
latest: Pulling from malice/clamav
Digest: sha256:d04bcc8533b3d5ede065820592eb023137dde13218e3159e6de8cef7dcc2260f
Status: Image is up to date for malice/clamav:latest
[Updating Plugin] ===>  comodo
latest: Pulling from malice/comodo
Digest: sha256:38ab2b80022a52c5015376af39131b34e248001d7f70ba96ad1b18d4a47718ad
Status: Image is up to date for malice/comodo:latest
[Updating Plugin] ===>  fprot
latest: Pulling from malice/fprot
Digest: sha256:77b9048dea806d06914369cef04544e0238154de3032a5d474fa09c13bd2410a
Status: Image is up to date for malice/fprot:latest
[Updating Plugin] ===>  fsecure
latest: Pulling from malice/fsecure
Digest: sha256:5f8ef723b5c65b66c9b54197df4b5ebde99d375a8b4fdbc14f8c5d10e7634eac
Status: Image is up to date for malice/fsecure:latest
[Updating Plugin] ===>  sophos
latest: Pulling from malice/sophos
Digest: sha256:7f7ea8d7a2e46e80a66c9127b49fd682d603f748bad8dfe08e7f4aa1cb037f3d
Status: Image is up to date for malice/sophos:latest
[Updating Plugin] ===>  pe
latest: Pulling from malice/pe
Digest: sha256:372193ef5659e5e5255ca0a2300ecfb1e56ae8add0197e5bfe4acf7889537fec
Status: Image is up to date for malice/pe:latest
[Updating Plugin] ===>  floss
latest: Pulling from malice/floss
Digest: sha256:e4cd9a502f7735db1893e548ed04893404a2a2579912e9a7055669eb94c2c406
Status: Image is up to date for malice/floss:latest
[Updating Plugin] ===>  office
latest: Pulling from malice/office
Digest: sha256:800644b60d231dda4cc4b11671145c37c1215bb7567f064e072a7a8b25d53d5f
Status: Image is up to date for malice/office:latest
[Updating Plugin] ===>  pdf
latest: Pulling from malice/pdf
Digest: sha256:9d87327d8214efa6c5a392a1d5b6bca282c676e094d77d4e3aa17b6f46da4b92
Status: Image is up to date for malice/pdf:latest
[Updating Plugin] ===>  javascript
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x55bdc25c6779]

goroutine 1 [running]:
panic(0x55bdc2cab980, 0xc420012050)
  /usr/lib/go/src/runtime/panic.go:500 +0x1a5
github.com/maliceio/malice/malice/docker/client/image.Pull(0xc420384ab0, 0xc42034c980, 0x11, 0x55bdc2920fc6, 0x6)
  /go/src/github.com/maliceio/malice/malice/docker/client/image/image.go:40 +0xf9
github.com/maliceio/malice/plugins.UpdateAllPlugins(0xc420384ab0)
  /go/src/github.com/maliceio/malice/plugins/plugins.go:261 +0x306
github.com/maliceio/malice/commands.cmdUpdatePlugin(0x0, 0x0, 0x1, 0x0, 0xc4203660c0)
  /go/src/github.com/maliceio/malice/commands/plugin.go:161 +0x277
github.com/maliceio/malice/commands.glob..func8(0xc4202d3a40, 0x0, 0xc4202d3a40)
  /go/src/github.com/maliceio/malice/commands/commands.go:138 +0xc5
github.com/maliceio/malice/vendor/github.com/urfave/cli.HandleAction(0x55bdc2c82660, 0x55bdc2d65e20, 0xc4202d3a40, 0xc420366000, 0x0)
  /go/src/github.com/maliceio/malice/vendor/github.com/urfave/cli/app.go:485 +0xd6
github.com/maliceio/malice/vendor/github.com/urfave/cli.Command.Run(0x55bdc29211a0, 0x6, 0x0, 0x0, 0x0, 0x0, 0x0, 0x55bdc29258f2, 0xd, 0x0, ...)
  /go/src/github.com/maliceio/malice/vendor/github.com/urfave/cli/command.go:207 +0xb98
github.com/maliceio/malice/vendor/github.com/urfave/cli.(*App).RunAsSubcommand(0xc420091860, 0xc4202d37c0, 0x0, 0x0)
  /go/src/github.com/maliceio/malice/vendor/github.com/urfave/cli/app.go:374 +0xb1c
github.com/maliceio/malice/vendor/github.com/urfave/cli.Command.startApp(0x55bdc292104a, 0x6, 0x0, 0x0, 0x0, 0x0, 0x0, 0x55bdc2930cca, 0x1f, 0x0, ...)
  /go/src/github.com/maliceio/malice/vendor/github.com/urfave/cli/command.go:294 +0x82e
github.com/maliceio/malice/vendor/github.com/urfave/cli.Command.Run(0x55bdc292104a, 0x6, 0x0, 0x0, 0x0, 0x0, 0x0, 0x55bdc2930cca, 0x1f, 0x0, ...)
  /go/src/github.com/maliceio/malice/vendor/github.com/urfave/cli/command.go:93 +0x16a7
github.com/maliceio/malice/vendor/github.com/urfave/cli.(*App).Run(0xc4200916c0, 0xc42000c180, 0x4, 0x4, 0x0, 0x0)
  /go/src/github.com/maliceio/malice/vendor/github.com/urfave/cli/app.go:250 +0x814
main.main()
  /go/src/github.com/maliceio/malice/main.go:88 +0x54a

Describe the results you expected:
Install of all Plugins

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker ps -a:

CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                      NAMES
f06896879f4a        nberna/nginx        "/start.sh"              18 months ago       Up 22 minutes       0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp   nginx
7d04af9aa555        nberna/php          "/start.sh"              18 months ago       Up 22 minutes       9000/tcp                                   php
f04d8e7adabb        mariadb             "/docker-entrypoin..."   18 months ago       Up 22 minutes       3306/tcp                                   bdd

no container created.

Any idea why the install of Docker in Docker doesn't work on my computer ?

Thanks a lot,
Euca

Redesign file scanning to happen asynchronously

this is going to complicate the scan command as I am not sure how to output to the terminal? Maybe with a chan that collects finished docker std.Outs?

Fix AVG av plugin

[ERROR] colonSeparated was empty:  []
[ERROR] AVG output was:
AVG command line Anti-Virus scanner
Copyright (c) 2013 AVG Technologies CZ

Check if file already exists then display stored results

Fix Elasticsearch

TODO

fix plugin communication to ES
wait for ES to fully start (not just a dumb 10 sec wait)
monitor logs to output important info as to why it might not start (not enough RAM etc)
auto populate kibana index/viz/dashboards like how a file-beat module does it
add ~/.malice/logs/elastic.log to catch ES errors

Ability for malice to build plugin from repo if DockerHub fails

This allows for private repo plugins as well as certain AVs that are failing to build on DockerHUB 😭

Get http://localhost:9200/: EOF

Output of go version:

go version go1.8.1 darwin/amd64

Output of docker version:

Client:
 Version:      17.04.0-ce
 API version:  1.28
 Go version:   go1.7.5
 Git commit:   4845c56
 Built:        Wed Apr  5 06:06:36 2017
 OS/Arch:      darwin/amd64

Server:
 Version:      17.04.0-ce
 API version:  1.28 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   4845c56
 Built:        Tue Apr  4 00:37:25 2017
 OS/Arch:      linux/amd64
 Experimental: true

Output of docker info:

Containers: 1
 Running: 1
 Paused: 0
 Stopped: 0
Images: 1
Server Version: 17.04.0-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins: 
 Volume: local
 Network: bridge host ipvlan macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: 
containerd version: 422e31ce907fd9c3833a38d7b8fdd023e5a76e73
runc version: 9c2d8d184e5da67c95d601382adf14862e4f2228
init version: 949e6fa
Security Options:
 seccomp
  Profile: default
Kernel Version: 4.9.19-moby
Operating System: Alpine Linux v3.5
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 5.818GiB
Name: moby
ID: MINV:DBFQ:PTCY:7FAD:ATH7:USVS:X5EF:ZKQR:WRST:WEQ2:3366:PGHM
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 24
 Goroutines: 33
 System Time: 2017-04-08T18:36:35.813942524Z
 EventsListeners: 1
No Proxy: *.local, 169.254/16
Registry: https://index.docker.io/v1/
Experimental: true
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, Docker For Mac, Docker Toolbox, docker-machine, etc.):
Docker for Mac (both Stable and Edge)

Steps to reproduce the issue:

Install Docker
Run "brew install https://raw.githubusercontent.com/maliceio/malice/master/contrib/homebrew/Formula/malice.rb"
Run "malice scan eicar.com"

Describe the results you received:
NAKAnoMac:Documents naka$ malice scan eicar.com
ERRO[0000] ELK is NOT running, starting now...
ERRO[0000] Network malice does not exist, creating now... env=development exisits=false network=malice
INFO[0000] Created Network: malice env=development name=malice
INFO[0000] Created Volume: malice env=development
malice: Pulling from blacktop/elastic-stack
6daefd62341a: Pull complete
1a4b6fdf1cbc: Pull complete
f7f8f9c33278: Pull complete
fafb758ceb1e: Pull complete
fa9c50d99ebd: Pull complete
131e2f46387f: Pull complete
a93ae444a66e: Pull complete
60c40cde4484: Pull complete
951cf1a26798: Pull complete
96500fc9281c: Pull complete
ebbab98cd3d6: Pull complete
8a0a3f1821e1: Pull complete
e0b9a6e91066: Pull complete
ef6d67c6936a: Pull complete
13babcce34b4: Pull complete
ab70c63bac5d: Pull complete
43c221c5bda4: Pull complete
4e71693b9b02: Pull complete
9ea4b19c8600: Pull complete
Digest: sha256:9342541bdead2c9e12988032117395d133e8e20b72c1a1a1583ef9dd3d618fef
Status: Downloaded newer image for blacktop/elastic-stack:malice
NAKAnoMac:Documents naka$ malice scan eicar.com
2017/04/09 03:32:34 Get http://localhost:9200/: EOF

Describe the results you expected:

Additional information you deem important (e.g. issue happens only occasionally):
When I access "http://localhost:9200", the following message were shown.
ERR_EMPTY_RESPONSE

Thanks in advance,
Yukinaka

On the very first run of a malice scan it downloads blacktop/elk then stops

It should continue on. I also need to pull busybox down when I pull the plugins the first time.

vagrant@vagrant-ubuntu-trusty-64:~$ malice scan /vagrant/data/samples/befb88b89c2eb401900a68e9f5b78764203f2b48264fcc3f7121bf04a57fd408
ERRO[0000] ELK is NOT running, starting now...
ERRO[0000] Network malice does not exist, creating now...  env=development exisits=false network=malice
INFO[0000] Created Network: malice                       env=development name=malice
INFO[0000] Created Volume: malice                        env=development
latest: Pulling from blacktop/elk
6a5a5368e0c2: Pull complete
7b9457ec39de: Pull complete
d5cc639e6fca: Pull complete
2cac98b7f5b9: Pull complete
bf96dd67c9aa: Pull complete
ab05ba8362e2: Pull complete
fa7e8f9f253c: Pull complete
4fc945f0ead5: Pull complete
ed2741c9ce36: Pull complete
570ac9acb128: Pull complete
846e63a852d5: Pull complete
b4d5ca0ebd99: Pull complete
b333a6393ab0: Pull complete
4f4b78533415: Pull complete
7566a927c761: Pull complete
f1766b5d4375: Pull complete
076bc3305401: Pull complete
0c2bc09697a0: Pull complete
Digest: sha256:892016cc5f5bd7eea071c2adadacc9f2e2d3006d4119284839e89f63ebc2fbe4
Status: Downloaded newer image for blacktop/elk:latest

Finish API

come on dude... do you EVEN code?

maliceio / malice Goto Github PK

malice's Introduction

maliceio

malice's People

Contributors

Stargazers

Watchers

Forkers

malice's Issues

File

VirusTotal

ShadowServer

F-PROT

Comodo

F-Secure

Bitdefender

Sophos

ClamAV

AVG

TODO

seccomp

cgroup settings

drop most capabilities

AntiVirus

File

NSRL Database

ShadowServer

Comodo

F-PROT

Magic

SSDeep

TRiD

Exiftool

Yara

ShadowServer

Floss

Decoded Strings

Stack Strings

F-Secure

Bitdefender

Sophos

ClamAV

malice elk

Recommend Projects

Recommend Topics

Recommend Org