opencve / opencve Goto Github PK

Hi,

Im having problems when opencve is triying to update the database with new CVEs. The error show that are some problems inserting a new value because the query is trying to create a vendors_name_key when it exists.

Expected Behavior

Update database with new cves

Actual Behavior

[ERROR/ForkPoolWorker-4] Task HANDLE_EVENTS[d1477d2b-c975-4bc6-b92e-5f6166aa452c] raised unexpected: InvalidRequestError('This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "vendors_name_key"\nDETAIL: Key (name)=(strangerstudios) already exists.\n\n[SQL: INSERT INTO vendors (id, created_at, updated_at, name) VALUES (%(id)s, now(), now(), %(name)s)]\n[parameters: {'id': UUID('b8607d61-7f42-4e0e-b434-6c19fae8828e'), 'name': 'strangerstudios'}]\n(Background on this error at: http://sqlalche.me/e/gkpj)',)

Steps to Reproduce the Problem

I had running opencve few days and now i have this problem when im trying to update

Specifications

• OpenCVE version: opencve, version 1.0.2
• Python version: 3.6
• Platform: Ubuntu 18.04

Instead of sending mail when CVE matches subscriber list can we have a webhook to trigger some action.
For e.g, Incase a CVE matches my subscriber list I want to create a case in Case Management(TheHive).

Expected Behavior

Actual Behavior

Steps to Reproduce the Problem

I changed the server_name = 10.224.100.6:8000 in the opencv.cfg file but could not access the 10.224.100.6:8000 access page

Specifications

• OpenCVE version:
• Python version:
• Platform:

Screenshots (optional)

Expected Behavior

The command opencve update-db works properly

Actual Behavior

The command opencve update-db is roughing throwing an error.

Steps to Reproduce the Problem

C:\projects\opencve> opencve upgrade-db
Traceback (most recent call last):
  File "C:\Program Files\Python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Program Files\Python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\projects\opencve\sbox\Scripts\opencve.exe\__main__.py", line 7, in <module>
  File "c:\projects\opencve\sbox\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "c:\projects\opencve\sbox\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "c:\projects\opencve\sbox\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\projects\opencve\sbox\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\projects\opencve\sbox\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "c:\projects\opencve\sbox\lib\site-packages\click\decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "c:\projects\opencve\sbox\lib\site-packages\opencve\commands\__init__.py", line 39, in decorator
    return __ctx.invoke(f, *args, **kwargs)
  File "c:\projects\opencve\sbox\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "c:\projects\opencve\sbox\lib\site-packages\click\decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "c:\projects\opencve\sbox\lib\site-packages\flask\cli.py", line 412, in decorator
    return __ctx.invoke(f, *args, **kwargs)
  File "c:\projects\opencve\sbox\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "c:\projects\opencve\sbox\lib\site-packages\opencve\commands\upgrade_db.py", line 16, in upgrade_db
    upgrade(directory=str(migrations_path))
  File "c:\projects\opencve\sbox\lib\site-packages\flask_migrate\__init__.py", line 95, in wrapped
    f(*args, **kwargs)
  File "c:\projects\opencve\sbox\lib\site-packages\flask_migrate\__init__.py", line 280, in upgrade
    command.upgrade(config, revision, sql=sql, tag=tag)
  File "c:\projects\opencve\sbox\lib\site-packages\alembic\command.py", line 294, in upgrade
    script.run_env()
  File "c:\projects\opencve\sbox\lib\site-packages\alembic\script\base.py", line 481, in run_env
    util.load_python_file(self.dir, "env.py")
  File "c:\projects\opencve\sbox\lib\site-packages\alembic\util\pyfiles.py", line 97, in load_python_file
    module = load_module_py(module_id, path)
  File "c:\projects\opencve\sbox\lib\site-packages\alembic\util\compat.py", line 182, in load_module_py
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 790, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "C:\projects\opencve\sbox\Lib\site-packages\opencve\migrations\env.py", line 94, in <module>
    run_migrations_online()
  File "C:\projects\opencve\sbox\Lib\site-packages\opencve\migrations\env.py", line 70, in run_migrations_online
    engine = engine_from_config(
  File "c:\projects\opencve\sbox\lib\site-packages\sqlalchemy\engine\__init__.py", line 466, in engine_from_config
    return create_engine(url, **options)
  File "c:\projects\opencve\sbox\lib\site-packages\sqlalchemy\engine\__init__.py", line 423, in create_engine
    return strategy.create(*args, **kwargs)
  File "c:\projects\opencve\sbox\lib\site-packages\sqlalchemy\engine\strategies.py", line 87, in create
    dbapi = dialect_cls.dbapi(**dbapi_args)
  File "c:\projects\opencve\sbox\lib\site-packages\sqlalchemy\dialects\postgresql\psycopg2.py", line 605, in dbapi
    import psycopg2
  File "c:\projects\opencve\sbox\lib\site-packages\psycopg2\__init__.py", line 50, in <module>
    from psycopg2._psycopg import (                     # noqa
ImportError: DLL load failed while importing _psycopg: The specified module could not be found.

Specifications

• OpenCVE version: 1.0.2
• Python version: 3.9.1
• Platform: Windows

Solution

Update the requirements for psycopg2-binary to version 2.8.6.

Expected Behavior

CVE being downloaded

Actual Behavior

The CVE database got stuck

Steps to Reproduce the Problem

There are no special steps, I run the OpenCVE as indicated in the README.md. I can see the Celery logs like Scheduler: Sending due task cve-updates-15-mn (CVE_UPDATES) and INFO/SpawnPoolWorker-122] child process 5544 calling self.run(), but the CVE number is not growing.

I also used flower to investigate why the CVEs are not getting downloaded. I could see the broker online with only 7 tasks processed, but they are either not failed or succeeded.

Specifications

• OpenCVE version: 1.0.2
• Python version: 3.9
• Platform: Windows Server 2016

Screenshots (optional)

Hello, I had set my smtp details, would like to ask are there any ways to test on the email alerting feature?
Thank you in advance.

Expected Behavior

The command opencve celery worker -l INFO works correctly

Actual Behavior

The command opencve celery worker -l INFO throws an error.

Steps to Reproduce the Problem

C:\projects\opencve> opencve celery worker -l INFO
-------------- celery@hostname v4.3.0 (rhubarb)
---- **** -----
--- * ***  * -- Windows-2012ServerR2-6.3.9600-SP0 2021-02-18 16:43:52
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app:         opencve:0x8991542820
- ** ---------- .> transport:   redis://127.0.0.1:6379/0
- ** ---------- .> results:     redis://127.0.0.1:6379/1
- *** --- * --- .> concurrency: 2 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery


[tasks]
  . CVE_UPDATES
  . HANDLE_ALERTS
  . HANDLE_EVENTS
  . HANDLE_REPORTS

[2021-02-18 16:43:52,410: INFO/MainProcess] Connected to redis://127.0.0.1:6379/0
[2021-02-18 16:43:53,660: CRITICAL/MainProcess] Unrecoverable error: AttributeError("'Timer' object has no attribute 'isAlive'")
Traceback (most recent call last):
  File "c:\projects\opencve\sbox\lib\site-packages\celery\worker\worker.py", line 205, in start
    self.blueprint.start(self)
  File "c:\projects\opencve\sbox\lib\site-packages\celery\bootsteps.py", line 119, in start
    step.start(parent)
  File "c:\projects\opencve\sbox\lib\site-packages\celery\bootsteps.py", line 369, in start
    return self.obj.start()
  File "c:\projects\opencve\sbox\lib\site-packages\celery\worker\consumer\consumer.py", line 318, in start
    blueprint.start(self)
  File "c:\projects\opencve\sbox\lib\site-packages\celery\bootsteps.py", line 119, in start
    step.start(parent)
  File "c:\projects\opencve\sbox\lib\site-packages\celery\worker\consumer\heart.py", line 34, in start
    c.heart.start()
  File "c:\projects\opencve\sbox\lib\site-packages\celery\worker\heartbeat.py", line 55, in start
    self.tref = self.timer.call_repeatedly(
  File "c:\projects\opencve\sbox\lib\site-packages\celery\utils\timer2.py", line 130, in call_repeatedly
    return self._do_enter('call_repeatedly', *args, **kwargs)
  File "c:\projects\opencve\sbox\lib\site-packages\celery\utils\timer2.py", line 111, in _do_enter
    self.ensure_started()
  File "c:\projects\opencve\sbox\lib\site-packages\celery\utils\timer2.py", line 105, in ensure_started
    if not self.running and not self.isAlive():
AttributeError: 'Timer' object has no attribute 'isAlive'
[2021-02-18 16:43:54,379: INFO/SpawnPoolWorker-2] child process 4648 calling self.run()
[2021-02-18 16:43:54,379: INFO/SpawnPoolWorker-1] child process 4496 calling self.run()

Specifications

• OpenCVE version: 1.0.2
• Python version: 3.9.1
• Platform: Windows
• Redis: 2.4.5

Hello,

As the json imports all the CPE information, maybe it would be a nice option to allow users to subscribe to specific versions of product to avoid further triage on their side.

Hi,

Awesome project! since this is not a severe security issue, I am filing it here to be fixed. The issue is the fact that the value supplied in the next GET parameter doesn't seem to be validated resulting in an open redirect to external attacker-controlled domain name.

Proof of concept:

Authenticate to your account then browse to https://www.opencve.io/login?next=////example.com

You'll be redirected to example.com

According to Flask-login

Warning: You MUST validate the value of the next parameter. If you do not, your application will be vulnerable to open redirects. For an example implementation of is_safe_url.

Regards.

The page that displays the CVE provides a search field used to filter the objects based on their description. It would be practical to support the vendors and products filters in this field.

Currently the users must first select the vendor, and only after that they could see the associated CVEs.

This issue is based on this tweet : https://twitter.com/0daydorpher/status/1345423538711355393

tried few times ，but still can't parse cpe list

#####################
Importing CPE list...
#####################
[*] Downloading https://nvd.nist.gov/feeds/xml/cpe/dictionary/official-cpe-dictionary_v2.3.xml.gz... (done in 13.315s).
[*] Parsing XML elements...Killed

There are any possibility to get the reports filtered only by the published date of the CVE? It would be very interesting to get informed about new CVEs to address.

Hello,

while navigating and testing, i see a small error in cwe link to mitre: in https://www.opencve.io/cwe

Actual:
http://cwe.mitre.org/data/definitions/**CWE-**998.html
Correct:
http://cwe.mitre.org/data/definitions/998.html

Well, i know, that's a not a big issue...

Best regards

When I use the command：“opencve upgrade-db”

I get the following error

INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.

After successful setup, vendor list does not display on the app. (i followed the instructions to install the service)

Hi There - What level of error reporting is available and where can these logs be found ?

I previously had an issue where the list of vendors was empty and this was caused by a lack of memory allocated to the vm im running opencve in, however this wasn't immediately obvious. I now have an issue where at least one CVE is not present in opencve (but is present in cve/mitre & nvd). So i'm concluding it is an issue where that CVE (CVE-2020-25654) was skipped on import due to an issue, be it memory or otherwise.

This means I could be searching for CVE or vulns by criticality for a vendor and they are not returned in opencve; leading to the determination that a vendor doesn't have a certain vul but it is because or opencve hasn't imported it, I would not know.

Can you please expand the error reporting to show where record have not been imported from nvd and for what reason ? Thanks All.

Hi All
After working through the install steps, finally hitting my head against this...

$ opencve webserver
[2021-01-06 14:45:24 +0000] [5395] [INFO] Starting gunicorn 20.0.4
[2021-01-06 14:45:24 +0000] [5395] [ERROR] Connection in use: ('127.0.0.1', 8000)
[2021-01-06 14:45:24 +0000] [5395] [ERROR] Retrying in 1 second.
[2021-01-06 14:45:25 +0000] [5395] [ERROR] Connection in use: ('127.0.0.1', 8000)

opencve.cfg has the server_name set to the static IP address of the server & have tried rebooting and also changing the defined addresses fro the redis comms from127.0.0.1 to the static IP and back again - still the same issue.

When i run the server in another server, with the command $ opencve webserver -b 0.0.0.0:8000, i found that the webserver use the implicit inclusion of static files, and all the urls are like this: <script src="http://0.0.0.0:8000/static/js/jquery.min.js" type="text/javascript"></script>

(below, a screenshot of the problem)

Is your feature request related to a problem?
Add tagging to CVE vulnerbilities and can filter based on tags

Do you have a solution in mind, or a suggestion to improve OpenCVE?
just like issues on github, we can add tags to issue and filter by tags

Additional comment
nope

hello
thanks you for this project
can this tool support notification push to slack channel

Is your feature request related to a problem?
It's not very obvious to guess the vulnerability types of a CVE because only the CWE IDs are displayed.

Do you have a solution in mind, or a suggestion to improve OpenCVE?
Display the vulnerable type next to the CWE ID (n CWEs means n SQL queries, but a CVE does not have lots of associated CWEs, so it's ok).

Additional comment
Based on this tweet : https://twitter.com/chen_erlich/status/1351862087165816832

Hi guys,
please i want to help me.
When i run the command (# opencve upgrade-db. ) here is the error message displayed ( Connection refused Is the server running on host "localhost" (127.0.0.1) and accepting TCP/IP connections on port 5432?)
i allow all ports in firewall.
Thank you

Expected Behavior

The command opencve webserver launches the webserver

Actual Behavior

The command opencve webserver throws an error

Steps to Reproduce the Problem

C:\projects\opencve> opencve webserver
Traceback (most recent call last):
  File "C:\Program Files\Python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Program Files\Python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\projects\opencve\sbox\Scripts\gunicorn.exe\__main__.py", line 4, in <module>
  File "c:\projects\opencve\sbox\lib\site-packages\gunicorn\app\wsgiapp.py", line 9, in <module>
    from gunicorn.app.base import Application
  File "c:\projects\opencve\sbox\lib\site-packages\gunicorn\app\base.py", line 11, in <module>
    from gunicorn import util
  File "c:\projects\opencve\sbox\lib\site-packages\gunicorn\util.py", line 9, in <module>
    import fcntl
ModuleNotFoundError: No module named 'fcntl'

Specifications

• OpenCVE version: 1.0.2
• Python version: 3.9.1
• Platform: Windows

Potential solutions

Instead of using gunicorn an option for Windows platforms could be waitress. Adding waitress==1.4.4 in the requirements and then checking for the platform in the webserver command will do the job. The change will be something like:

def webserver(args):
    """Run the webserver."""
    if sys.platform == 'win32':
        args = ["waitress-serve"] + list(args)
    else:
        args = ["gunicorn"] + list(args)
    args.append("opencve.app:app")
    os.execvp(args[0], args)

Hi all,

is it possible to use letsencrypt with opencve?
No contraindication or problematic?

Thanks

Antonin

Hello,
I want to customize the opencve to collect the vulnerabilities aligned (/or only dedicated) to my assets (my Information system scope).
Thank you

Hello,

I managed to install everything on my own no worries, but I think it would be more convenient for many users if you specifiy on the README.md file that you will need to have postgresql and redis installed to make it work.

Expected Behavior

The header static assets get loaded either way the app is running behind a proxy.

Actual Behavior

The application fulfills the headers with the whole URL.

Steps to Reproduce the Problem

Configure Nginx on top of the opencve application.

Specifications

• OpenCVE version: 1.0.2
• Python version: Python 3.9
• Platform: Windows

Screenshots (optional)

Hi All - What is the update schedule for the Celery workers (I can assume its every 24hrs, but want to check) ?
Also, what time does it run and can this be triggered manually ?

Expected Behavior

opencve upgrade-db successfully

Actual Behavior

opencve upgrade-db error

Steps to Reproduce the Problem

[root@sze0-sec-test-1006 data]# opencve upgrade-db
INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
INFO [alembic.runtime.migration] Running upgrade -> 8bdc527d8d49, Initial migration
Traceback (most recent call last):
File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
cursor.execute(statement, parameters)
psycopg2.errors.UndefinedObject: type "jsonb" does not exist
LINE 7: json JSONB,
^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/bin/opencve", line 11, in
sys.exit(cli())
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 829, in call
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/opencve/commands/init.py", line 39, in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/flask/cli.py", line 412, in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/opencve/commands/upgrade_db.py", line 16, in upgrade_db
upgrade(directory=str(migrations_path))
File "/usr/local/lib/python3.6/site-packages/flask_migrate/init.py", line 95, in wrapped
f(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/flask_migrate/init.py", line 280, in upgrade
command.upgrade(config, revision, sql=sql, tag=tag)
File "/usr/local/lib/python3.6/site-packages/alembic/command.py", line 294, in upgrade
script.run_env()
File "/usr/local/lib/python3.6/site-packages/alembic/script/base.py", line 490, in run_env
util.load_python_file(self.dir, "env.py")
File "/usr/local/lib/python3.6/site-packages/alembic/util/pyfiles.py", line 97, in load_python_file
module = load_module_py(module_id, path)
File "/usr/local/lib/python3.6/site-packages/alembic/util/compat.py", line 182, in load_module_py
spec.loader.exec_module(module)
File "", line 678, in exec_module
File "", line 219, in _call_with_frames_removed
File "/usr/local/lib/python3.6/site-packages/opencve/migrations/env.py", line 94, in
run_migrations_online()
File "/usr/local/lib/python3.6/site-packages/opencve/migrations/env.py", line 86, in run_migrations_online
context.run_migrations()
File "", line 8, in run_migrations
File "/usr/local/lib/python3.6/site-packages/alembic/runtime/environment.py", line 813, in run_migrations
self.get_context().run_migrations(**kw)
File "/usr/local/lib/python3.6/site-packages/alembic/runtime/migration.py", line 560, in run_migrations
step.migration_fn(**kw)
File "/usr/local/lib/python3.6/site-packages/opencve/migrations/versions/8bdc527d8d49_initial_migration.py", line 32, in upgrade
sa.PrimaryKeyConstraint("id"),
File "", line 8, in create_table
File "", line 3, in create_table
File "/usr/local/lib/python3.6/site-packages/alembic/operations/ops.py", line 1109, in create_table
return operations.invoke(op)
File "/usr/local/lib/python3.6/site-packages/alembic/operations/base.py", line 354, in invoke
return fn(self, operation)
File "/usr/local/lib/python3.6/site-packages/alembic/operations/toimpl.py", line 101, in create_table
operations.impl.create_table(table)
File "/usr/local/lib/python3.6/site-packages/alembic/ddl/impl.py", line 277, in create_table
self._exec(schema.CreateTable(table))
File "/usr/local/lib/python3.6/site-packages/alembic/ddl/impl.py", line 146, in _exec
return conn.execute(construct, multiparams)
File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
return meth(self, multiparams, params)
File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
return connection._execute_ddl(self, multiparams, params)
File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1050, in _execute_ddl
compiled,
File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
e, statement, parameters, cursor, context
File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedObject) type "jsonb" does not exist
LINE 7: json JSONB,
^

[SQL:
CREATE TABLE cves (
id UUID NOT NULL,
created_at TIMESTAMP WITH TIME ZONE NOT NULL,
updated_at TIMESTAMP WITH TIME ZONE NOT NULL,
cve_id VARCHAR NOT NULL,
json JSONB,
vendors JSONB,
cwes JSONB,
summary VARCHAR NOT NULL,
cvss2 FLOAT,
cvss3 FLOAT,
PRIMARY KEY (id)
)

]
(Background on this error at: http://sqlalche.me/e/f405)

Specifications

• OpenCVE version:
• Python version:
• Platform:

Screenshots (optional)

Hello,

ive found a file called "report_message.txt" in the /templates/emails/ folder, is it possible to tell opencve that it should use the "Text Only" E-Mail Template.
Or is it a entry in the "default.cfg"?

Thanks

Hi guys,

Hope you are all well !

Thanks for this awesome repository, I ll install it today in my toolkit of devsecops :-)

I was wondering if you can add to the todo list the following features:

• Slack Integration for sending the latest CVEs found for vendors enabled subscriptions
• Rest API or RSS Feed for vendors enabled subscriptions results
• Dockerized version of opencve (Do you want me to do that ? Are PR welcomed ?)

Thanks for any insights or inputs on these.

Cheers,
Luc Michalski

[root@cve opencve]# /usr/local/python3/bin/opencve celery worker -l INFO
Traceback (most recent call last):
File "/usr/local/python3/bin/opencve", line 8, in
sys.exit(cli())
File "/usr/local/python3/lib/python3.7/site-packages/click/core.py", line 829, in call
return self.main(*args, **kwargs)
File "/usr/local/python3/lib/python3.7/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/usr/local/python3/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/python3/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/python3/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/python3/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/usr/local/python3/lib/python3.7/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/python3/lib/python3.7/site-packages/opencve/commands/init.py", line 39, in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/usr/local/python3/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/usr/local/python3/lib/python3.7/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/python3/lib/python3.7/site-packages/flask/cli.py", line 412, in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/usr/local/python3/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/usr/local/python3/lib/python3.7/site-packages/opencve/commands/celery.py", line 21, in worker
os.execvp(args[0], args)
File "/usr/local/python3/lib/python3.7/os.py", line 574, in execvp
_execvpe(file, args)
File "/usr/local/python3/lib/python3.7/os.py", line 616, in _execvpe
raise last_exc
File "/usr/local/python3/lib/python3.7/os.py", line 607, in _execvpe
exec_func(fullname, *argrest)
FileNotFoundError: [Errno 2] No such file or directory

[root@cve opencve]# /usr/local/python3/bin/opencve webserver
Traceback (most recent call last):
File "/usr/local/python3/bin/opencve", line 8, in
sys.exit(cli())
File "/usr/local/python3/lib/python3.7/site-packages/click/core.py", line 829, in call
return self.main(*args, **kwargs)
File "/usr/local/python3/lib/python3.7/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/usr/local/python3/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/python3/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/python3/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/usr/local/python3/lib/python3.7/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/python3/lib/python3.7/site-packages/opencve/commands/init.py", line 39, in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/usr/local/python3/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/usr/local/python3/lib/python3.7/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/python3/lib/python3.7/site-packages/flask/cli.py", line 412, in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/usr/local/python3/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/usr/local/python3/lib/python3.7/site-packages/opencve/commands/webserver.py", line 17, in webserver
os.execvp(args[0], args)
File "/usr/local/python3/lib/python3.7/os.py", line 574, in execvp
_execvpe(file, args)
File "/usr/local/python3/lib/python3.7/os.py", line 616, in _execvpe
raise last_exc
File "/usr/local/python3/lib/python3.7/os.py", line 607, in _execvpe
exec_func(fullname, *argrest)
FileNotFoundError: [Errno 2] No such file or directory

os version:CentOS Linux release 7.7.1908 (Core)
python version:Python 3.7.9

Hi all, I have enabled all notifications in users' profiles but they don't receive emails when a CVE is created/updated. Also, users are subscribed to vendors and products, but Reports tab still shows the message "No report yet. Did you subscribe to vendors or products ?" . How can I solve it?

Apart from these issues, I want to know how can I edit the register email template, I can't find it.

Cheers,
Chessi

Consider supporting XCCDF and other SCAP content

XCCDF/OVAL are the scanners that are part of the standard that defines CVE.

SCAP: https://en.m.wikipedia.org/wiki/Security_Content_Automation_Protocol

XCCDF: https://en.m.wikipedia.org/wiki/Extensible_Configuration_Checklist_Description_Format

Hi all,
I have configured OpenCVE to download CVEs' database every 60 minutes. OpenCVE is installed in a server (inside a virtual env) and Postgres database is in another server (inside a Docker container). When I run beat, worker and webserver everything works just fine, but after one hour or sometimes a little bit more, when worker is checking for CVEs it gives an exception:
psycopg2.DatabaseError: could not receive data from server: Connection timed out

After that, worker tries again (in an hour) to execute the check, and same exception is rised. After a lot of tries, it gives this exception:
sqlalchemy.exc.InvalidRequestError: Can't reconnect until invalid transaction is rolled back

While checking traffic with Wireshark, I found out that the packet containing this query is retransmited when exception is raised:
SELECT metas.id AS metas_id, metas.created_at AS metas_created_at, metas.updated_at AS metas_updated_at, metas.name AS metas_name, metas.value AS metas_value FROM metas WHERE metas.name = 'nvd_last_sha256' LIMIT 1

Screenshots at the end of this post show logs from worker and Postgres.

If I install OpenCVE (inside virtual env) and Postgres (inside Docker) in the same server or in servers inside the same VLAN, this problem doesn't happen.

Expected Behavior

Exception should be catched and worker should keep working normally. The app shouldn't stop.

Actual Behavior

Exception is raised infinitely.

Steps to Reproduce the Problem

• Server A and Server B are in different VLANs
• Install and configure OpenCVE inside a virtual environment in server A
• Install PostgreSQL on a Docker Container in server B and configure OpenCVE to use this database
• Allow traffic between both servers (a Firewall rule)
• Configure OpenCVE's scheduler (beat) to run every 60 minutes ["schedule": timedelta(minutes=60)]
• Run beat, worker and (optionally) webserver
• Just let it run for hours until the exception occurs, it's sporadic. Sometimes it takes just one hour, sometimes more than 4 hours.

Specifications

• OpenCVE version: 1.0.2
• Python version: 3.6.9
• Platform: Ubuntu 18.04.5 LTS (Server A) - Ubuntu 18.04.4 LTS (Server B)
• PostgreSQL version: 13.1

Screenshots (optional)

Worker's log

Postgres' log

Expected Behavior

OpenCVE to be working withput DB issues

Actual Behavior

Running OpenCVE webserver command raises an error.

Steps to Reproduce the Problem

$ opencve webserver
Traceback (most recent call last):
  File "C:\projects\opencve\sbox\Lib\site-packages\opencve\cli.py", line 5, in <module>
    from opencve import __version__
  File "C:\projects\opencve\sbox\lib\site-packages\opencve\__init__.py", line 66, in <module>
    import_submodules(__name__, ("models", "controllers", "checks", "tasks"))
  File "C:\projects\opencve\sbox\lib\site-packages\opencve\__init__.py", line 51, in import_submodules
    for loader, name, is_pkg in pkgutil.walk_packages(package.__path__):
  File "C:\Program Files\Python39\lib\pkgutil.py", line 93, in walk_packages
    __import__(info.name)
  File "C:\projects\opencve\sbox\Lib\site-packages\opencve\models\__init__.py", line 33, in <module>
    users_vendors = db.Table(
  File "C:\projects\opencve\sbox\lib\site-packages\flask_sqlalchemy\__init__.py", line 58, in _make_table
    return sqlalchemy.Table(*args, **kwargs)
  File "<string>", line 2, in __new__
  File "C:\projects\opencve\sbox\lib\site-packages\sqlalchemy\util\deprecations.py", line 130, in warned
    return fn(*args, **kwargs)
  File "C:\projects\opencve\sbox\lib\site-packages\sqlalchemy\sql\schema.py", line 473, in __new__
    raise exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Table 'users_vendors' is already defined for this MetaData instance.  Specify 'extend_existing=True' to redefine options and columns on an existing Table object.

Specifications

• OpenCVE version: 1.0.2
• Python version: 3.9
• Platform: Windows

Do you have a solution in mind, or a suggestion to improve OpenCVE?
Can you add support for exploit db? https://www.exploit-db.com/

https://github.com/offensive-security/exploitdb

Additional comment
I can try to help I knew how new exploits are added.

Hi All - Wanted to point out a rather obvious issue you have, you are incorrectly displaying the CVSSv3 scoring brackets.
Firstly you only have three ratings (Low/Med/High) when you need five: None/Low/Med/High/Critical

The scorings you are using to split these are also out, you have High defined as 7.5-10 when it should be 7-8.9 (With Critical being 9-10) and the scoring for None needs to be 0-01. This may help with a number of other issues you could be having with filtering.

It appears you may be using the scoring fields from CVSSv2 for the CVSSv3 data you are importing. Just a heads up.

Dear developers.
Could you please add the feature to sort the CVE (by CVE number, CVE date, Score).
Also full text searching and case independent searching (for example when I try to search for "esxi" - it finds only one result, but "ESXi" returns 110 results) in Vulnerabilities (CVE) section.

with best regards
Vasyl

Little error inside template maybe

Expected Behavior

Actual Behavior

Steps to Reproduce the Problem

Specifications

• OpenCVE version:
• Python version:
• Platform:

Screenshots (optional)

Some cve cannot be searched out, and there are some gaps with the official website of cve

Example CVE: CVE-2020-25654

See bottom of screenshot. 😄

Expected Behavior

I would like send notification to be sure smtp configuration works and I will receive CVE alerts.

Actual Behavior

I correctly configured smtp in ssmtp.conf and ssmtp command works.
My smtp dont need Login/password, but if I remove "smtp_username" and "smtp_password" in opencve.cfg
I have this message error :
configparser.NoOptionError: No option 'smtp_username' in section: 'mail'
How to make check from opencve email notification feature ? And how to resolve this option error in my case ?

Steps to Reproduce the Problem

Try with Open SMTP without login password and without TLS.

Specifications

• OpenCVE version:
• Python version:
• Platform:

Screenshots (optional)

Dear all，
When I run the command：opencve celery worker -l INFO
The program will report an error：

   [tasks]
 . CVE_UPDATES
 . HANDLE_ALERTS
 . HANDLE_EVENTS
 . HANDLE_REPORTS

[2021-02-19 02:11:07,539: INFO/MainProcess] Connected to redis://127.0.0.1:6379/0
[2021-02-19 02:11:07,546: INFO/MainProcess] mingle: searching for neighbors
[2021-02-19 02:11:08,565: INFO/MainProcess] mingle: all alone
[2021-02-19 02:11:08,587: INFO/MainProcess] celery@kali ready.
[2021-02-19 02:23:50,383: INFO/MainProcess] Received task: CVE_UPDATES[43c9f1ee-fdcd-4bec-ac75-14355943fa47]  
[2021-02-19 02:23:50,415: INFO/MainProcess] Received task: HANDLE_EVENTS[889f5812-5cb5-454d-8555-def043db95a5]  
[2021-02-19 02:23:50,415: INFO/ForkPoolWorker-1] Task CVE_UPDATES[43c9f1ee-fdcd-4bec-ac75-14355943fa47] succeeded in 0.028891843001474626s: <AsyncResult: 7d980a92-c3dc-47b4-81df-a599a8324475>
[2021-02-19 02:23:50,418: INFO/ForkPoolWorker-2] HANDLE_EVENTS[889f5812-5cb5-454d-8555-def043db95a5]: Checking for new events...
[2021-02-19 02:23:50,419: INFO/ForkPoolWorker-2] HANDLE_EVENTS[889f5812-5cb5-454d-8555-def043db95a5]: Downloading https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.meta...
[2021-02-19 02:23:51,991: ERROR/ForkPoolWorker-2] Task HANDLE_EVENTS[889f5812-5cb5-454d-8555-def043db95a5] raised unexpected: AttributeError("'NoneType' object has no attribute 'value'")                                          
Traceback (most recent call last):                                                                                
 File "/home/kali/.local/lib/python3.8/site-packages/celery/app/trace.py", line 385, in trace_task               
   R = retval = fun(*args, **kwargs)                                                                             
 File "/home/kali/.local/lib/python3.8/site-packages/celery/app/trace.py", line 648, in __protected_call__       
   return self.run(*args, **kwargs)                                                                              
 File "/home/kali/.local/lib/python3.8/site-packages/opencve/tasks/events.py", line 94, in handle_events         
   current_sum, new_sum = has_changed()                                                                          
 File "/home/kali/.local/lib/python3.8/site-packages/opencve/tasks/events.py", line 33, in has_changed           
   if nvd_sha256 != last_nvd256.value:                                                                           
AttributeError: 'NoneType' object has no attribute 'value'                     

It seems that this error will result in the failure to update the new CVE to the database

Hi,

I try to install opencve but when i launch the import data i got an error with psycopg2.

Specifications

• OpenCVE version: 1.0.2
• Python version: 3.7.9
• Platform: FreeBSD 12.1-RELEASE
• Postgresql version : 11

Error :

###################### Importing CVE for 2002 ###################### [] Downloading https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2002.json.gz... (done in 4.231s). [] Parsing JSON elements... (done in 0.871s). [] Creating model objects... (done in 3.724s). [] Inserting CVE...Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1224, in _execute_context cursor, statement, parameters, context File "/usr/local/lib/python3.7/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 725, in do_executemany cursor.executemany(statement, parameters) psycopg2.errors.UntranslatableCharacter: unsupported Unicode escape sequence LINE 1: ...13T04:00:00+00:00'::timestamptz, 'CVE-1999-0497', '{"cve": {... ^ DETAIL: Unicode escape values cannot be used for code point values above 007F when the server encoding is not UTF8. CONTEXT: JSON data, line 1: ...nymous FTP is enabled."}, {"lang": "en", "value":...

The complete error : https://paste.swordarmor.fr/cd0q

Thanks for your help!

Hi There - What level of error reporting is available and where can these logs be found ?

I previously had an issue where the list of vendors was empty and this was caused by a lack of memory allocated to the vm im running opencve in, however this wasn't immediately obvious. I now have an issue where at least one CVE is not present in opencve (but is present in cve/mitre & nvd). So i'm concluding it is an issue where that CVE (CVE-2020-25654) was skipped on import due to an issue, be it memory or otherwise.

This means I could be searching for CVE or vulns by criticality for a vendor and they are not returned in opencve; leading to the determination that a vendor doesn't have a certain vul but it is because or opencve hasn't imported it, I would not know.

Can you please expand the error reporting to show where record have not been imported from nvd and for what reason ? Thanks All.

Hi! When creating an user from administration panel I found two problems:
When password field is completed, the plaintext is saved in the database without hashing. If user tries to access with that user:password combination, server will give 500 error.

This is the form in administration:

This is the table users in the database:

If you complete password field with a hash (I copied one from another user that had been created with register form), login will be successful (because a hash is being saved into the database).

The second problem is that when I want to edit an user's information, I can't save it because it throws an error (saying that user and email already exists):

I have to edit user's name and email so I can save it.

Then I have to edit again that user's profile just to change the information an get back the original values (the correct username and email).

Regards
Daniela

Expected Behavior

In opencve.cfg, the option to set SMTP password should be smtp_password.

Actual Behavior

Actually, it's stmp_password.

[mail]
; Choices are 'smtp' or 'sendmail'
email_adapter = smtp

; The 'From' field of the sent emails
email_from = [email protected]

; Configuration to set up SMTP mails.
smtp_server = tata.smtp.intranet
smtp_port = 465
smtp_username = toto
stmp_password = tata

If you start opencve with smtp_password option instead of stmp_password option, it basically crashes saying "stmp_password option not found".

Steps to Reproduce the Problem

• install opencve with pip
• cat opencve.cfg

Specifications

• OpenCVE version: 1.0.2
• Python version: 3.6.9
• Platform: Debian 10

Thanks.

Opencve on premise install in a VM. When viewing the dashboard, CVE's are listed along with vendor tags but clicking the vendor tag does nothing.

When accessing the Vendor tab on sidebar, no vendors are listed and none are displayed when queried from the search function. The Vendors section of the Admin panel is empty.

Running opencve import-data again returns:
[*] Import already done

Expected Behavior

The command opencve init works properly.

Actual Behavior

Using the pip version the command opencve init produces an error.

Steps to Reproduce the Problem

pip install opencve

opencve init
(sbox) PS C:\projects\opencve> opencve init
Traceback (most recent call last):
  File "C:\Program Files\Python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Program Files\Python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\projects\opencve\sbox\Scripts\opencve.exe\__main__.py", line 4, in <module>
  File "c:\projects\opencve\sbox\lib\site-packages\opencve\__init__.py", line 10, in <module>
    from opencve.extensions import db
  File "c:\projects\opencve\sbox\lib\site-packages\opencve\extensions.py", line 5, in <module>
    from flask_migrate import Migrate
  File "c:\projects\opencve\sbox\lib\site-packages\flask_migrate\__init__.py", line 11, in <module>
    from alembic import __version__ as __alembic_version__
  File "c:\projects\opencve\sbox\lib\site-packages\alembic\__init__.py", line 3, in <module>
    from . import context  # noqa
  File "c:\projects\opencve\sbox\lib\site-packages\alembic\context.py", line 1, in <module>
    from .runtime.environment import EnvironmentContext
  File "c:\projects\opencve\sbox\lib\site-packages\alembic\runtime\environment.py", line 1, in <module>
    from .migration import MigrationContext
  File "c:\projects\opencve\sbox\lib\site-packages\alembic\runtime\migration.py", line 5, in <module>
    from sqlalchemy import Column
  File "c:\projects\opencve\sbox\lib\site-packages\sqlalchemy\__init__.py", line 8, in <module>
    from . import util as _util  # noqa
  File "c:\projects\opencve\sbox\lib\site-packages\sqlalchemy\util\__init__.py", line 14, in <module>
    from ._collections import coerce_generator_arg  # noqa
  File "c:\projects\opencve\sbox\lib\site-packages\sqlalchemy\util\_collections.py", line 16, in <module>
    from .compat import binary_types
  File "c:\projects\opencve\sbox\lib\site-packages\sqlalchemy\util\compat.py", line 331, in <module>
    time_func = time.clock
AttributeError: module 'time' has no attribute **'clock'**

Specifications

• OpenCVE version: 1.0.2
• Python version: python 3.9.1
• Platform: Windows

Other

The issue is related to the SQLAlchemy and Flask-SQLAlchemy. They both are using the module time and using the functions clock which is deprecated in Python 3.8.

Potential solutions

• Update the SQLAlchemy and Flask-SQLAlchemy to a newer version.
• Provide a patch

I particularly patched both libraries like so:

Edit: /Lib/site-packages/sqlalchemy/util/compat.py
Replace around line 330:

if win32 or jython:
        time_func = time.clock
else:
    time_func = time.time

By:

if win32 or jython:
    if "clock" in dir(time):
        time_func = time.clock
    else:
        time_func = time.time
else:
    time_func = time.time

Edit: /Lib/site-packages/flask_sqlalchemy/init.py
Replace around line 38:

if sys.platform == 'win32':
        _timer = time.clock
else:
    _timer = time.time

By:

if sys.platform == 'win32':
    if "clock" in dir(time):
        _timer = time.clock
    else:
        _timer = time.time
else:
    _timer = time.time