Error pattern

[May 13 17:45:10]  INFO [xxxxxxx:22] (331/331) Scanned zlib-devel-1.2.3-3 -> 1.2.3-7.el5 : []
[May 13 17:45:10]  INFO [xxxxxxx:22] Fetching CVE details...
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x4137087]

goroutine 3774 [running]:
panic(0x4727140, 0xc820010170)
        /Users/k1low/.anyenv/envs/goenv/versions/1.6.2/src/runtime/panic.go:481 +0x3e6
github.com/future-architect/vuls/cveapi.cvedictClient.httpGet.func1(0x0, 0x0)
        /Users/k1low/go/src/github.com/future-architect/vuls/cveapi/cve_client.go:137 +0x3f7
github.com/cenkalti/backoff.RetryNotify(0xc820ab1c20, 0x5362da8, 0xc820939380, 0x490ad20, 0x0, 0x0)
        /Users/k1low/go/src/github.com/cenkalti/backoff/retry.go:32 +0x4d
github.com/future-architect/vuls/cveapi.cvedictClient.httpGet(0x485b0b0, 0x15, 0xc820442f89, 0xd, 0xc820ab1bc0, 0x28, 0xc820342120, 0xc8203422a0)
        /Users/k1low/go/src/github.com/future-architect/vuls/cveapi/cve_client.go:144 +0x192
github.com/future-architect/vuls/cveapi.cvedictClient.FetchCveDetails.func2()
        /Users/k1low/go/src/github.com/future-architect/vuls/cveapi/cve_client.go:95 +0x303
github.com/future-architect/vuls/util.GenWorkers.func1(0xc820342420)
        /Users/k1low/go/src/github.com/future-architect/vuls/util/util.go:35 +0x60
created by github.com/future-architect/vuls/util.GenWorkers
        /Users/k1low/go/src/github.com/future-architect/vuls/util/util.go:37 +0x6d

Host: Mac OSX Yosemite
Remote host: CentOS5.5
Repeatability: Yes
Check another CentOS5.5?: No. I don't have another CentOS5.5...

Success pattern

Host: Mac OSX Yosemite
Remote host: CentOS6.6 ( Other server )

when doing:
cd vuls/setup/docker/dockerfile
docker build .

Errors:

Step 24 : RUN cp -rp /tmp/vulsrepo/src/* /usr/share/nginx/html/vulsrepo
 ---> Running in 5f31317122cc
cp: cannot stat '/tmp/vulsrepo/src/*': No such file or directory
INFO[0278] The command [/bin/sh -c cp -rp /tmp/vulsrepo/src/* /usr/share/nginx/html/vulsrepo] returned a non-zero code: 1 

When the host has docker containers is specified in config.toml, vuls detects vulnerabilities of docker host and docker containers.

I sometimes want to scan only docker containers without docker host.

Hello,

[vuls@aserver ~]$ vuls prepare -debug bserver
INFO[0000] Start Preparing (config: /home/vuls/config.toml)
[Apr 13 10:45:48] INFO [localhost] Detecting OS...
DEBU[0000] Command: set -o pipefail; ls /etc/debian_version
DEBU[0000] SSH executed. cmd: set -o pipefail; ls /etc/debian_version, status:
stdout:
ls: cannot access /etc/debian_version: No such file or directory

stderr:

[Apr 13 10:45:48] DEBUG [localhost] Not Debian like Linux. Host: bserver.domain.com:22
DEBU[0000] Command: set -o pipefail; ls /etc/fedora-release
DEBU[0000] SSH executed. cmd: set -o pipefail; ls /etc/fedora-release, status:
stdout:
ls: cannot access /etc/fedora-release: No such file or directory

stderr:

DEBU[0000] Command: set -o pipefail; ls /etc/redhat-release
DEBU[0000] SSH executed. cmd: set -o pipefail; ls /etc/redhat-release, status:
stdout:
/etc/redhat-release

stderr:

DEBU[0000] Command: set -o pipefail; cat /etc/redhat-release
DEBU[0001] SSH executed. cmd: set -o pipefail; cat /etc/redhat-release, status:
stdout:
Red Hat Enterprise Linux Server release 6.7 (Santiago)

stderr:

[Apr 13 10:45:49] INFO localhost Successfully detected. bserver: rhel 6.7
[Apr 13 10:45:49] DEBUG [localhost] []scan.osTypeInterface{
&scan.redhat{
linux: scan.linux{
ServerInfo: config.ServerInfo{
ServerName: "bserver",
User: "root",
Password: "",
Host: "bserver.domain.com",
Port: "22",
KeyPath: "/home/vuls/.ssh/id_rsa_k1",
KeyPassword: "",
SudoOpt: config.SudoOption{
ExecBySudo: true,
ExecBySudoSh: false,
},
CpeNames: []string{},
LogMsgAnsiColor: "\x1b[35m",
},
Family: "rhel",
Release: "6.7",
osPackages: scan.osPackages{
Packages: models.PackageInfoList{},
UnsecurePackages: scan.CvePacksList{},
},
log: &logrus.Entry{
Logger: &logrus.Logger{
Out: &os.File{
file: &os.file{
fd: 2,
name: "/dev/stderr",
dirinfo: (*os.dirInfo)(nil),
},
},
Hooks: {
0x01: []logrus.Hook{
&lfshook.lfsHook{
paths: {
0x04: "/var/log/vuls/bserver:22.log",
0x03: "/var/log/vuls/bserver:22.log",
0x02: "/var/log/vuls/bserver:22.log",
0x01: "/var/log/vuls/bserver:22.log",
0x00: "/var/log/vuls/bserver:22.log",
0x05: "/var/log/vuls/bserver:22.log",
},
levels: []logrus.Level{
0x05, 0x04, 0x03, 0x02, 0x01, 0x00,
},
},
},
0x00: []logrus.Hook{
&lfshook.lfsHook{...},
},
0x05: []logrus.Hook{
&lfshook.lfsHook{...},
},
0x04: []logrus.Hook{
&lfshook.lfsHook{...},
},
0x03: []logrus.Hook{
&lfshook.lfsHook{...},
},
0x02: []logrus.Hook{
&lfshook.lfsHook{...},
},
},
Formatter: &prefixed.TextFormatter{
ForceColors: false,
DisableColors: false,
DisableTimestamp: false,
ShortTimestamp: false,
TimestampFormat: "",
DisableSorting: false,
MsgAnsiColor: "\x1b[35m",
},
Level: 0x05,
mu: sync.Mutex{
state: 0,
sema: 0x00000000,
},
},
Data: {
"prefix": "bserver:22",
},
Time: (unexported time.Time),
Level: 0x00,
Message: "",
},
},
},
}
[Apr 13 10:45:49] INFO [localhost] Installing...
[Apr 13 10:45:49] INFO [bserver:22] Nothing to do
[Apr 13 10:45:49] INFO [localhost] Success

However plugin is not installed.

[root@bserver ~]# yum info yum-plugin-security
Loaded plugins: product-id, rhnplugin, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to registe
r.
This system is not registered with RHN Classic or RHN Satellite.
You can use rhn_register to register.
RHN Satellite or RHN Classic support will be disabled.
Available Packages
Name : yum-plugin-security
Arch : noarch
Version : 1.1.30
Release : 30.el6
Size : 41 k
Repo : rhel-x86_64-server-6
Summary : Yum plugin to enable security filters
URL : http://yum.baseurl.org/download/yum-utils/
License : GPLv2+
Description : This plugin adds the options --security, --cve, --bz and --advisory flags
: to yum and the list-security and info-security commands.
: The options make it possible to limit list/upgrade of packages to specific
: security relevant ones. The commands give you the security information.

When this plugin is installed manually the scan completes fine, otherwise this error appears:

[Apr 13 10:32:55] ERROR [localhost] Failed to scan. err: [email protected]:22: Failed to yum updateinfo list available --security. status: 1, stdout: Loaded plugins: product-id, rhnplugin, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
This system is not registered with RHN Classic or RHN Satellite.
You can use rhn_register to register.
RHN Satellite or RHN Classic support will be disabled.
Usage: yum [options] COMMAND

List of Commands:

check Check for problems in the rpmdb
check-update Check for available package updates
clean Remove cached data
deplist List a package's dependencies
distribution-synchronization Synchronize installed packages to the latest available versions
downgrade downgrade a package
erase Remove a package or packages from your system
groupinfo Display details about a package group
groupinstall Install the packages in a group on your system
grouplist List available package groups
groupremove Remove the packages in a group from your system
help Display a helpful usage message
history Display, or use, the transaction history
info Display details about a package or group of packages
install Install a package or packages on your system
list List a package or groups of packages
load-transaction load a saved transaction from filename
makecache Generate the metadata cache
provides Find what package provides the given value
reinstall reinstall a package
repolist Display the configured software repositories
resolvedep Determine which package provides the given dependency
search Search package details for the given string
shell Run an interactive yum shell
update Update a package or packages on your system
upgrade Update packages taking obsoletes into account
version Display a version for the machine and/or available repos.

Command line error: no such option: --security
, stderr:

I faced the following error, no idea what went wrong.

pyed@Debian7:~$ vuls prepare
INFO[0000] Begin Preparing (config: /home/pyed/config.toml) 
[Apr  9 14:50:21]  INFO [localhost] Detecting OS... 
[Apr  9 14:50:21]  INFO [localhost] Installing...
[Apr  9 14:50:21]  INFO [localhost] Success
pyed@Debian7:~$ vuls scan
INFO[0000] Begin scannig (config: /home/pyed/config.toml) 
[Apr  9 14:50:29]  INFO [localhost] Validating Config...
[Apr  9 14:50:29]  INFO [localhost] Detecting OS... 
[Apr  9 14:50:29]  INFO [localhost] Scanning vulnerabilities... 
[Apr  9 14:50:29] ERROR [localhost] Failed to scan. err: Not initialize yet..
pyed@Debian7:~$ 

Hi, my name is Alex.
I really like the work you do however, I am having trouble using VULS.
How do I check the vulnerability of the server VULS is installed in?

CVE-2016-1000110 is detected as CVE-2016-1000 in Vuls.

ChangeLog for: python-2.6.6-66.el6_8.x86_64, python-libs-2.6.6-66.el6_8.x86_64

• Tue Aug 9 21:00:00 2016 Charalampos Stratakis [email protected] - 2.6.6-66
• Fix for CVE-2016-1000110 HTTPoxy attack
  Resolves: rhbz#1359161

http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-1000110
https://access.redhat.com/security/cve/cve-2016-1000110

It's cut down by fixing 4 digits by a cord.
https://github.com/future-architect/vuls/blob/master/scan/redhat.go#L870

After 2014, CVE-ID syntax is the variable length.

https://en.wikipedia.org/wiki/Common_Vulnerabilities_and_Exposures

Changes to CVE-ID Syntax
In order to support CVE ID's beyond CVE-YEAR-9999 (aka the CVE10k problem) a change was made to the CVE syntax in 2014 and took effect on Jan 13, 2015 [5]

The new CVE-ID syntax is variable length and includes:

CVE prefix + Year + Arbitrary Digits

NOTE: The variable length arbitrary digits will begin at four (4) fixed digits and expand with arbitrary digits only when needed in a calendar year, for example, CVE-YYYY-NNNN and if needed CVE-YYYY-NNNNN, CVE-YYYY-NNNNNNN, and so on. This also means there will be no changes needed to previously assigned CVE-IDs, which all include a minimum of 4 digits.

Please consider following the GNU/POSIX standards for command line interface. More precisely at least using "--" for long options.
See GNU and IEEE

With "full" load of nodes:

$ grep "servers\." /tmp/go/src/github.com/future-architect/vuls/config.toml|wc -l
  20

$ ./vuls scan
INFO[0000] Begin scannig (config: /tmp/go/src/github.com/future-architect/vuls/config.toml)
panic: runtime error: index out of range

goroutine 1 [running]:
panic(0x4709000, 0xc8200121a0)
    /usr/local/Cellar/go/1.6/libexec/src/runtime/panic.go:464 +0x3e6
github.com/future-architect/vuls/config.TOMLLoader.Load(0xc8202aff40, 0x38, 0x0, 0x0)
    /tmp/go/src/github.com/future-architect/vuls/config/tomlloader.go:89 +0xbd5
github.com/future-architect/vuls/config.(*TOMLLoader).Load(0x4b1e8f8, 0xc8202aff40, 0x38, 0x0, 0x0)
    <autogenerated>:3 +0xaf
    ...

After deleting 14 nodes:

$ grep "servers\." /tmp/go/src/github.com/future-architect/vuls/config.toml|wc -l
   6

$ ./vuls scan
INFO[0000] Begin scannig (config: /tmp/go/src/github.com/future-architect/vuls/config.toml)
[Apr  7 19:34:59]  INFO [localhost] Validating Config...
[Apr  7 19:34:59]  INFO [localhost] Detecting OS...
[Apr  7 19:35:01]  INFO [localhost] Scanning vulnerabilities...
[Apr  7 19:35:01]  INFO [localhost] Check required packages for scanning...
[Apr  7 19:35:01]  INFO [localhost] Scanning vuluneable OS packages...

Is there expected behaviour?

Another problem I encountered is:

[Apr 14 01:39:32] ERROR [localhost] Failed: [email protected]:22: Failed to yum install -y yum-plugin-security. status: 1, stdout: zsh:set:1: no such option: pipefail

Thanks!

vuls scan後、1件も脆弱性を検出していない状態でvuls tuiを実行するとindex out of rangeが発生しました。

$ vuls scan
INFO[0000] Begin scannig (config: /home/username/config.toml)
[Apr 5 22:50:42] INFO [localhost] Validating Config...
[Apr 5 22:50:42] INFO [localhost] Detecting OS...
[Apr 5 22:50:43] INFO [localhost] Scanning vulnerabilities...
[Apr 5 22:50:43] INFO [localhost] Check required packages for scanning...
[Apr 5 22:50:43] INFO [localhost] Scanning vuluneable OS packages...
[Apr 5 22:50:48] INFO [rhel7:22] Fetching CVE details...
[Apr 5 22:50:48] INFO [rhel7:22] Done
[Apr 5 22:50:48] INFO [localhost] Scanning vulnerable software specified in CPE...
[Apr 5 22:50:48] INFO [localhost] Reporting...
rhel7 (rhel 7.2)

No unsecure packages.
[Apr 5 22:50:48] INFO [localhost] Insert to DB...

$ vuls tui
panic: runtime error: index out of range
goroutine 1 [running]:
panic(0xab0dc0, 0xc82000e100)
/usr/local/go/src/runtime/panic.go:464 +0x3e6
github.com/future-architect/vuls/report.detailLines(0x0, 0x0, 0x0, 0x0)
/home/username/bin/go/src/github.com/future-architect/vuls/report/tui.go:657 +0x2470
github.com/future-architect/vuls/report.setDetailLayout(0xc82019fe80, 0x0, 0x0)
/home/username/bin/go/src/github.com/future-architect/vuls/report/tui.go:629 +0x1ac
github.com/future-architect/vuls/report.layout(0xc82019fe80, 0x0, 0x0)
/home/username/bin/go/src/github.com/future-architect/vuls/report/tui.go:504 +0x9f
github.com/jroimartin/gocui.(_Gui).flush(0xc82019fe80, 0x0, 0x0)
/home/username/bin/go/src/github.com/jroimartin/gocui/gui.go:337 +0x16e
github.com/jroimartin/gocui.(_Gui).MainLoop(0xc82019fe80, 0x0, 0x0)
/home/username/bin/go/src/github.com/jroimartin/gocui/gui.go:265 +0xb2
github.com/future-architect/vuls/report.RunTui(0x0)
/home/username/bin/go/src/github.com/future-architect/vuls/report/tui.go:65 +0x426
github.com/future-architect/vuls/commands.(_TuiCmd).Execute(0xc82027da40, 0x7f1d02da8e20, 0xc82000e2d8, 0xc820298a80, 0x0, 0x0, 0x0, 0x418eb8)
/home/username/bin/go/src/github.com/future-architect/vuls/commands/tui.go:67 +0x65
github.com/google/subcommands.(_Commander).Execute(0xc820058070, 0x7f1d02da8e20, 0xc82000e2d8, 0x0, 0x0, 0x0, 0x7f1d02da8e20)
/home/username/bin/go/src/github.com/google/subcommands/subcommands.go:142 +0x517
github.com/google/subcommands.Execute(0x7f1d02da8e20, 0xc82000e2d8, 0x0, 0x0, 0x0, 0x10dfd60)
/home/username/bin/go/src/github.com/google/subcommands/subcommands.go:372 +0x55
main.main()
/home/username/bin/go/src/github.com/future-architect/vuls/main.go:43 +0x411

I'm running a normal vuls scan on a centos host. I'm getting the following:
[Apr 14 16:57:00] ERROR [localhost] Failed to scan. err: user@myserver:22: Failed to parse yum check-update. err: Failed to parse yum check update line: Percona-Server-shared-51--

The command that gave a problem is:
[Apr 14 16:57:00] DEBUG [sft007:22] SSH executed. cmd: set -o pipefail; echo xxxxxx | sudo -S yum check-update, status: <nil>

Please do let me know if I can give you more details.

I want to use the vuls in RancherOS.
Could you to the corresponding.

I found that there is CVE data in cnnvd.
http://www.cnnvd.org.cn/vulnerability/xmlxz

Vuls can report in Chinese using this data.
Please thumbs up if you wish.

-ssh-external doesn't seem to work for me (I changed the names in the example):

INFO[0000] cve-dictionary: /home/me/cve/cve.sqlite3
[Jun 24 14:33:13]  INFO [localhost] Validating Config...
[Jun 24 14:33:13]  INFO [localhost] Detecting Server OS...
[Jun 24 14:33:13]  INFO [localhost] (1/3) Failed AAA
[Jun 24 14:33:13]  INFO [localhost] (2/3) Failed BBB
[Jun 24 14:33:13]  INFO [localhost] (3/3) Failed CCC
[Jun 24 14:33:13] ERROR [localhost] Failed to init servers. Check the configuration. err: Failed to detect server OSes. err: [Error occurred on AAA. errs: [Unknown OS Type] Error occurred on BBB. errs: [Unknown OS Type] Error occurred on CCC. errs: [Unknown OS Type]]

Here is my config:

[servers]
[servers.AAA]
host = "AAA"

[servers.BBB]
host = "BBB"

[servers.CCC]
host = "CCC"

When some connexion are failing the following error occurs before the slack report and Vuls gives up :

time="May 10 13:47:09" level=error msg="Timeout occured while detecting" 
time="May 10 13:47:09" level=error msg="Failed to detect. servername: "a"
time="May 10 13:47:09" level=error msg="Failed to detect. servername: "b"
time="May 10 13:47:09" level=error msg="Failed to detect. servername: "c"
time="May 10 13:47:09" level=error msg="Failed to detect. servername: "d"
time="May 10 13:47:09" level=error msg="Failed to detect. servername: "e"
time="May 10 13:47:09" level=error msg="Failed to detect. servername: "f"
time="May 10 13:47:09" level=error msg="Failed to detect. servername: "g"
time="May 10 13:47:09" level=error msg="Failed to detect. servername: "h"
time="May 10 13:47:09" level=error msg="Failed to detect. servername: "i"
time="May 10 13:47:09" level=error msg="Failed to detect. servername: "j"
time="May 10 13:47:09" level=error msg="Failed to init servers. Check the configuration. err: Failed to detect the type of OS. err: Timeout" 

n.b. this is related to #37 since those errors are coming from the fact that available ciphers are not matching.

I installed go and vuls to FreeBSD 10.3/go-1.6.1 and it running now.

Does vuls supports FreeBSD vulnerability check?

• pkg audit -F tells installed packages' vulnerability informations.

Thank you to publish good tool.

Hi,

Great software package! Thank you.

When the remote user is configured to use fish shell, the check fails when executing:
set -o pipefail; rpm -qa --queryformat '%{NAME} %{VERSION} %{RELEASE}'
I believe the issue is tab characters between %{NAME} %{VERSION} %{RELEASE} headers.

It's not an issue with unknown option as debug output suggests.

Some debug output from vuls near the error:

[Apr  6 17:18:41] DEBUG [zabbix:22] SSH executed. cmd: set -o pipefail; rpm -qa --queryformat '%{NAME}  %{VERSION}  %{RELEASE}
', status: <nil>
stdout: 
set: Unknown option '-o'
Standard input: set -o pipefail; rpm -qa --queryformat '%{NAME} %{VERSION}  %{RELEASE}
                ^
hawkey  0.5.8   2.git.0.202b194.el7
<snip> rest of my packages in similar format </snip>

stderr: 

[Apr  6 17:18:41] ERROR [zabbix:22] Failed to scan installed packages

Running docker-compose up -d failed with this errors:

# github.com/future-architect/vuls/report
report/slack.go:209: cveInfo.CveDetail.CweID undefined (type "github.com/future-architect/vuls/vendor/github.com/kotakanbe/go-cve-dictionary/models".CveDetail has no field or method CweID)
report/tui.go:694: cveInfo.CveDetail.CweID undefined (type "github.com/future-architect/vuls/vendor/github.com/kotakanbe/go-cve-dictionary/models".CveDetail has no field or method CweID)
report/util.go:229: cveDetail.CweID undefined (type "github.com/future-architect/vuls/vendor/github.com/kotakanbe/go-cve-dictionary/models".CveDetail has no field or method CweID)
report/util.go:230: cveDetail.CweID undefined (type "github.com/future-architect/vuls/vendor/github.com/kotakanbe/go-cve-dictionary/models".CveDetail has no field or method CweID)
report/util.go:272: cveDetail.CweID undefined (type "github.com/future-architect/vuls/vendor/github.com/kotakanbe/go-cve-dictionary/models".CveDetail has no field or method CweID)
ERROR: Service 'vuls' failed to build: The command '/bin/sh -c go get -v -d github.com/future-architect/vuls     && cd $GOPATH/src/github.com/future-architect/vuls     && glide install     && go install' returned a non-zero code: 2

This is my dependencies version

root@asdf:~# go version
go version go1.6 linux/amd64
root@asdf:~# docker version
Client:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   23cf638
 Built:        Thu Aug 18 05:22:43 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   23cf638
 Built:        Thu Aug 18 05:22:43 2016
 OS/Arch:      linux/amd64
root@asdf:~# docker-compose --version
docker-compose version 1.8.0, build f3628c7
root@asdf:~# git version
git version 2.9.3
root@asdf:~# cat /etc/issue
Ubuntu 14.04.4 LTS \n \l

see #81

Is sudo the only option for root privilege escalation? We don't use sudo (we cannot even run it), we use other method (similar to su) and because sudo is hardcoded vuls scan fails.

If user is not 'root' in ssh connection.
'yum update --changelog' looks like stalled by waiting user input 'Is this ok [y/d/N]: ',
Because 'echo N | ' is not concatenated for command string in 'getAllChangelog' function.

scan/redhat.go

579:    command := ""
580:    if o.ServerInfo.User == "root" {
581:        command = "echo N | "
582:    }

[question]
Is vuls recommended to use 'root' user for ssh connection ?
I think 'root' user check is unnecessary.
I want to use administrator (not 'root') user for security reasons.

Hi,

I have been trying to run the tool and I think i was able to install it properly, but when trying ti finally run the scan i get the error below. I hope you can help with this

Command:
vuls scan -lang=en -config=config.toml --cve-dicitonary-dbpath=cve.sqlite3 -http-proxy=someproxy -debug 10-210-67-67

config.toml
[servers]

[servers.10-210-67-67]
host="10.210.67.67"
port="22"
user="user"
keyPath="/home/user/id_rsa"

Error:
time="2016-08-04T15:21:52-07:00" level=debug msg="SSHResult: servername: 10-210-67-67, cmd: set -o pipefail; type curl, exitstatus: 0, stdout: curl is /usr/bin/curl\r\n, stderr: , err: %!s()" prefix=10-210-67-67
time="2016-08-04T15:21:53-07:00" level=debug msg="SSHResult: servername: 10-210-67-67, cmd: set -o pipefail; curl --max-time 1 --retry 3 --noproxy 169.254.169.254 http://169.254.169.254/latest/meta-data/instance-id, exitstatus: 7, stdout: curl: (7) couldn't connect to host\r\n, stderr: , err: %!s()" prefix=10-210-67-67
----snip----
----snip----

stderr: , err: %!s()" prefix=10-210-67-67
time="2016-08-04T15:25:11-07:00" level=debug msg="SSHResult: servername: 10-210-67-67, cmd: set -o pipefail; echo ***** | sudo -S env http_proxy="someproxy" https_proxy="someproxy" HTTP_PROXY="someproxy" HTTPS_PROXY="someproxy" yum --color=never repolist, exitstatus: 1, stdout: Password:\r\n, stderr: , err: %!s()" prefix=10-210-67-67
time="2016-08-04T15:25:11-07:00" level=error msg="Failed to scan vulnerable packages" prefix=10-210-67-67

Vuls supports FreeBSD just now.
It is awesome. Very very thanks to the work!

I found 1 problem.

see following line.
DEBU[0000] SSH executed. cmd: set -o pipefail; uname, status: <nil>

This line is one of the debug log.

I think vuls assumes that target hosts shell is bash, but FreeBSD's shell is not bash.

So, on FreeBSD, you cannot use -o pipefail option.

Hi,

I have a firewall that blocks all outbound ports except port 80 (firewall can't be changed). For HTTPS there's a squid proxy available that allows HTTP tunnel using CONNECT method. Environment variables for http_proxy and https_proxy are set on the server/shell running vuls. When updating with go-cve-dictionary fetchnvd -last2y it stalls with timeout error:

[Apr 20 11:24:24]  INFO Fetching... http://static.nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2016.xml.gz
 0 / 2 [-----------------------------------------------]   0.00%
[Apr 20 11:24:24]  INFO Fetching... http://static.nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2015.xml.gz
[Apr 20 11:26:31] ERROR Failed to fetch cve data from NVD. err: HTTP error. errs: [Get https://static.nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2016.xml.gz: dial tcp 129.6.13.177:443: connection timed out], url: http://static.nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2016.xml.gz

wget tool works:

$ wget http://static.nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2015.xml.gz
--2016-04-20 11:30:04--  http://static.nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2015.xml.gz
Resolving my.org.proxy.address (my.org.proxy.address)... x.x.x.x
Connecting to my.org.proxy.address (my.org.proxy.address)|x.x.x.x|:3128... connected.
Proxy request sent, awaiting response... 301 Moved Permanently
Location: https://static.nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2015.xml.gz [following]
--2016-04-20 11:30:04--  https://static.nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2015.xml.gz
Connecting to my.org.proxy.address (my.org.proxy.address)|x.x.x.x|:3128... connected.
Proxy request sent, awaiting response... 200 OK
Length: 1505541 (1.4M) [application/x-gzip]
Saving to: ‘nvdcve-2.0-2015.xml.gz’

Not sure if it's a bug, or enhancement request to make vuls work behind proxy like wget/curl tools work on the system.

Also notice the HTTP 301 move from http://static.nvd.nist.gov address to https://static.nvd.nist.gov . Perhaps that's tripping up the vuls code somewhere. Sorry I'm not a go programmer I have no idea where to start looking.

Thank you.

...snip
[Jun  6 09:55:58] ERROR [rhel5] Failed to scan vulnerable packages
[Jun  6 09:55:58] ERROR [localhost] Failed to scan. err: [email protected]:22: Failed to yum --color=never updateinfo list available --security. status: 1, stdout: Loaded plugins: amazon-id, fastestmirror, rhui-lb, security
usage: yum [options] COMMAND
...snip

yum command option on RHEL5 is not the same as RHEL6, 7
https://access.redhat.com/solutions/10021

I found a bug on CentOS6 with Japanese Environment.

On CentOS6 Server with some obsolete rpm file environment, and run vuls scan, stops scanning with error message "パッケージを不要にしています" also mean like "Removing packages."

I think this error is in "yum check-update."

the readme suggests running the following:

$ for i in {2002..2016}; do go-cve-dictionary fetchnvd -years $i; done

but when running this I got the following error:

fetchnvd:
    fetchnvd
        [-last2y]
        [-dbpath=/path/to/cve.sqlite3]
        [-debug]
        [-debug-sql]

I ended up using "-last2y", should the readme be updated?

On debian systems vuls does sudo test -f /usr/bin/aptitude , which is rather unnecessary.

If you only yum update Kernel, become a blank line is 3 or more columns to fail parse.

vuls scan -cvss-over dose not work.

Steps.

1. go-cve-dictionary fetchnvd -years 2016
2. go-cve-dictionary fetchnvd server
3. vuls scan -(1)
4. vuls scan -cvss-over 7 -(2)

I. At the phase (1), vuls displays "All of the found CVE vulnerabilities data".
II. At the phase (2), vuls displays "All of the found CVE vulnerabilities data", too.

I think phase (2) shows over 7 of the score only, and that I want.

goroutine 534 [running]:
panic(0xae3440, 0xc8206f89b0)
    /usr/local/go/src/runtime/panic.go:481 +0x3e6
github.com/future-architect/vuls/scan.parallelSSHExec.func1(0xe7c660, 0xc820010c60, 0x7f19c96fb000, 0xc8201054a0)
    /go/src/github.com/future-architect/vuls/scan/sshutil.go:80 +0x37d
created by github.com/future-architect/vuls/scan.parallelSSHExec
    /go/src/github.com/future-architect/vuls/scan/sshutil.go:84 +0x166

I saw this on my latest scan, if it helps

Now vuls has simply 1 configuration file called "config.toml".
But just now, vuls can configureable to select go-cve-dictionary.
And so, need to organize configuration.

I think following.

• vuls.conf : default configuration for vuls environment.

cve-dictionary-dbpath="/xxx/xxx/xxx/xxx/"
#cve-dictionary-url="xxx.xxx.xxx.xxx:xxxx"
#ignore-unscored-cves
report-text
#report-mail
#report-json

• servers.toml : rename from config.toml and configrations of target environments.

[servers]

[servers.172-31-4-82]
host         = "172.31.4.82"
port        = "22"
user        = "ec2-user"
keyPath     = "/home/ec2-user/.ssh/id_rsa"
cvss-over = 4.0
ignore-unscored-cves

I found that there is CVE data in Spanish on NVD site.
https://nvd.nist.gov/download.cfm#transxml

Vuls can report in Spanish using this data.
Please thumbs up if you wish.

$ mkdir '/tmp/path having blanks/'
$ cd '/tmp/path having blanks/'
$ pwd
/tmp/path having blanks
$ go-cve-dictionary server
[Apr 14 10:49:25] ERROR SQLite3 DB path must be a *Absolute* file path. dbpath: /tmp/path having blanks/cve.sqlite3

If I run go-cve-dictionary server in a path without blanks in the name, that's fine.

BTW, could you approve my request to join slack team? I applied many days ago :(

Feature request:

In scanning an estate of servers run by different sysadmins, colorized output to yum has resulted in failed parsing.

E.g.

[Apr 20 11:25:35] ERROR [localhost] Failed to scan. err: [email protected]:22: Failed to parse yum check-update. err: Unknown format: at.x86_64 3.1.10-48.el6 repo-name-server-6-base

Perhaps some input validation can detect control characters produced by yum and an appropiate error produced for the user ? Or a mention in the documentation. Perhaps this post is enough.

Many thanks for the fixes so far.

I am using docker and try to scan my desktop computer and I am getting a panic.

`docker run -v $(pwd)/config.toml:/app/config.toml -v /home/tpagel/.ssh/id_rsa:/app/id_rsa -v /home/tpagel/.ssh/id_rsa.pub:/app/id_rsa.pub --name vuls vuls
time="May 28 12:46:39" level=info msg="Opening DB. datafile: /app/cve.sqlite3"
time="May 28 12:46:39" level=info msg="Migrating DB"
time="May 28 12:46:39" level=info msg="Starting HTTP Server..."
time="May 28 12:46:39" level=info msg="Listening on 127.0.0.1:1323"
time="2016-05-28T12:46:41Z" level=warning msg="[Deprecated] password and keypassword in config file are unsecure. Remove them immediately for a security reason. They will be removed in a future release."
time="2016-05-28T12:46:41Z" level=info msg="Start scanning (config: /app/config.toml)"
time="May 28 12:46:41" level=info msg="Validating Config..."
{"time":"2016-05-28T12:46:41Z","remote_ip":"127.0.0.1","method":"GET","uri":"/health","status":200, "latency":57,"latency_human":"57.4µs","rx_bytes":0,"tx_bytes":0}
time="May 28 12:46:41" level=info msg="Detecting Server OS... "
time="May 28 12:47:07" level=info msg="(1/1) Detected 172-17-0-1: ubuntu 16.04"
time="May 28 12:47:07" level=info msg="Detecting Container OS..."
time="May 28 12:47:07" level=info msg="Scanning vulnerabilities... "
time="May 28 12:47:07" level=info msg="Check required packages for scanning..."
time="May 28 12:47:07" level=info msg="Scanning vulnerable OS packages..."
time="May 28 12:55:33" level=info msg="(1/88) Upgradable: apport-2.20.1-0ubuntu2 -> 2.20.1-0ubuntu2.1"
panic: send on closed channel

goroutine 91 [running]:
panic(0x9e0240, 0xc8201f8280)
/usr/local/go/src/runtime/panic.go:481 +0x3e6
github.com/future-architect/vuls/scan.(_debian).fillCandidateVersion.func2.1(0xc820087080, 0xc820254cc0, 0xc820254c60, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/go/src/github.com/future-architect/vuls/scan/debian.go:333 +0x6cf
github.com/future-architect/vuls/scan.(_debian).fillCandidateVersion.func2()
/go/src/github.com/future-architect/vuls/scan/debian.go:334 +0xe8
github.com/future-architect/vuls/util.GenWorkers.func1(0xc820254d80)
/go/src/github.com/future-architect/vuls/util/util.go:35 +0x60
created by github.com/future-architect/vuls/util.GenWorkers
/go/src/github.com/future-architect/vuls/util/util.go:37 +0x6d
`

config.toml:
[servers]

[servers.172-17-0-1]
host = "172.17.0.1"
port = "22"
user = "XXX"
password = "XXX"
keyPath = "/app/id_rsa"

I have noticed that vuls uses https://godoc.org/golang.org/x/crypto/ssh. I'm not sure if this library allows it but it would be nice to see vuls use the common ssh config.
It is especially important for options such as ProxyCommand.

The way I see it, config.toml would only contain the list of hosts you'd want to scan, not how to connect to them.

Got this error on every server that I have :

WARN[0020] Failed to ssh root@xxx err: ssh: handshake failed: ssh: no common algorithm for client to server cipher; client offered: [aes128-ctr aes192-ctr aes256-ctr [email protected] arcfour256 arcfour128], server offered: [aes256-cbc], Retrying in 10.034154425s...

And i have to disable my cipher to use vuln.

OS： CentOS6
RPM-PKG： python／python-libs

* Tue Mar 31 21:00:00 2015 Matej Stuchlik <[email protected]> - 2.6.6-62
- Fix CVE-2014-7185/4650/1912 CVE-2013-1752
Resolves: rhbz#1206572

CVE-2014-7185 and CVE-2013-1752 can be detected.
4650 and 1912 are un-detecting.

➜ go vuls scan

INFO[0000] Start scanning (config: xxx/config.toml)
[Apr 14 00:15:19]  INFO [localhost] Validating Config...
[Apr 14 00:15:19]  INFO [localhost] Detecting the type of OS...
[Apr 14 00:15:20]  INFO [localhost] (1/1) Successfully detected. 10-168-176-184: centos 7.0.1406
[Apr 14 00:15:20]  INFO [localhost] Scanning vulnerabilities...
[Apr 14 00:15:20]  INFO [localhost] Check required packages for scanning...
[Apr 14 00:15:20]  INFO [localhost] Scanning vulnerable OS packages...
[Apr 14 00:15:22] ERROR [xx-xx-xx-xx:22] Failed to scan vulnerable packages
[Apr 14 00:15:22] ERROR [localhost] Failed to scan. err: [email protected]:22: Failed to parse yum check-update. err: Unknown format: Obsoleting Packages

http://changelogs.ubuntu.com/changelogs is down now.

root@49b65c694784:/# cat /etc/apt/apt.conf.d/20changelog
// Server information for apt-changelog
APT {
  Changelogs {
    Server "http://changelogs.ubuntu.com/changelogs";
  };
};

root@49b65c694784:/# apt-get changelog openssl
Err Changelog for openssl (http://changelogs.ubuntu.com/changelogs/pool/main/o/openssl/openssl_1.0.1-4ubuntu5.35/changelog)
  503  Service Temporarily Unavailable
Err Changelog for openssl (http://archive.ubuntu.com/ubuntu/pool/main/o/openssl/openssl_1.0.1-4ubuntu5.35.changelog)
  404  Not Found [IP: 91.189.88.152 80]
E: changelog for this version is not (yet) available; try https://launchpad.net/ubuntu/+source/openssl/+changelog

aptitude is also not woking now….

# aptitude changelog openssl
Err Changelog of openssl
E: Changelog download failed: 503  Service Temporarily Unavailable
E: Couldn't find a changelog for openssl

I have just tested the new -ssh-external flag but it doesn't seem to be available for the scan command.

# go get github.com/future-architect/vuls
# github.com/future-architect/vuls/report
/opt/go/src/github.com/future-architect/vuls/report/slack.go:194: nvd.CvssSeverity undefined (type "github.com/kotakanbe/go-cve-dictionary/models".Nvd has no field or method CvssSeverity)
/opt/go/src/github.com/future-architect/vuls/report/tui.go:587: d.CveDetail.Nvd.CvssSeverity undefined (type "github.com/kotakanbe/go-cve-dictionary/models".Nvd has no field or method CvssSeverity)
/opt/go/src/github.com/future-architect/vuls/report/tui.go:679: nvd.CvssSeverity undefined (type "github.com/kotakanbe/go-cve-dictionary/models".Nvd has no field or method CvssSeverity)
/opt/go/src/github.com/future-architect/vuls/report/util.go:134: d.CveDetail.Nvd.CvssSeverity undefined (type "github.com/kotakanbe/go-cve-dictionary/models".Nvd has no field or method CvssSeverity)
/opt/go/src/github.com/future-architect/vuls/report/util.go:265: nvd.CvssSeverity undefined (type "github.com/kotakanbe/go-cve-dictionary/models".Nvd has no field or method CvssSeverity)

solution?

cd ./go/src/github.com/future-architect/vuls/
git checkout -b json_writer_sort_order origin/json_writer_sort_order
go install github.com/future-architect/vuls

When configuring a server I have ssh access via my user but package installation requires sudo.

There should be an option to use sudo to the config file to allow a non-root user to prepare the OS.

An example would be

[servers]

[servers.172-31-4-82]
host        = "172.31.4.82"
port        = "22"
user        = "ec2-user"
sudo        = "true"
keyPath     = "/home/ec2-user/.ssh/id_rsa"

There may be some consideration to add a sudo_password option if passwordless sudo ( NOPASSWD ) is not configured. I don't like having passwords in config files so maybe it would just be a pre-requirement of the user installing the packages or have vuls prompt for a sudo password before sshing into a system.

When vuls runs "vuls scan", run out of remote server's memory.
This command is heavy...
bash -c set -o pipefail; yes no | yum update --changelog exim | grep CVE

vuls server
ーーーーーーーーーーーーーーーーーーーーーーーーーーーーー
OS : CentOS 7.2.1511
go : go version go1.6.2 linux/amd64
vuls : Latest version(05/07)

Remote server
ーーーーーーーーーーーーーーーーーーーーーーーーーーーーー
OS : CentOS 6.7
MEM : 1GB / Swap:4GB

Before run "vuls scan" (Remote server "free -m")
ーーーーーーーーーーーーーーーーーーーーーーーーーーーーー
total used free shared buffers cached
Mem: 995 533 462 0 2 91
-/+ buffers/cache: 438 556
Swap: 4031 0 4031

config.toml
ーーーーーーーーーーーーーーーーーーーーーーーーーーーーー

[servers]

[servers.192-168-20-3]
host = "192.168.20.3"
port = "22"
user = "root"
password = "xxxxx"

Error message
ーーーーーーーーーーーーーーーーーーーーーーーーーーーーー

[May  7 17:31:25] ERROR [localhost] Failed to scan. err: [email protected]:22: Failed to get changelog. status: 2, stdout: grep: メモリを使い果たしました
, stderr:

Love the idea, it would be great if it could be as simple as pulling it down as a docker image or build it from a dockerfile, run it in privileged mode and mount the external filesystem to run the scans against.

Fetching vulnerability data for the entire period from NVD was too heavy (It used 2.3GB of memory).
I solved this problem.

see https://github.com/kotakanbe/go-cve-dictionary#usage-fetch-nvd-data

for i in {2002..2016}; do ./go-cve-dictionary fetchnvd -years $i; done

see also vulsio/go-cve-dictionary#8

We can specify hosts in vuls scan command, like this:

$ vuls scan server1 server2

In the same way, I want to specify docker container, for example like this:

$ vuls scan server1 server2@container_name_a server2@4aa37a8b63b9