quarkslab / irma Goto Github PK

IRMA is an asynchronous & customizable analysis system for suspicious files.

License: Apache License 2.0

Python 28.66% PowerShell 0.56% Shell 2.22% Batchfile 0.01% Mako 0.02% CSS 3.11% HTML 0.96% JavaScript 63.78% DIGITAL Command Language 0.01% Gherkin 0.09% Less 0.50% Jinja 0.10%

irma's People

Contributors

Stargazers

Watchers

irma's Issues

module Kaspersky 17.0 avp

line 186 in file probe/modules/antivirus/base.py
if paths.lower() in filename or
os.path.relpath(paths.lower()) in filename:
....
Kaspersky output :
2016-12-13 15:02:38 c:\users\irmaprobe\appdata\local\temp\tmpi0plfn detected EICAR-Test-File

filename ---> "c:\users\irmaprobe\appdata\local\temp\tmpi0plfn"
but paths equals "c:\users\irmapr~1\appdata\local\temp\tmpi0plfn"

Unable to install: deb-src pubkey failed

Hi, I'm running into this issue when installing IRMA using default settings and the command:
python irma-ansible.py environments/allinone_prod.yml playbooks/provisioning.yml

Output:

(...)
[+] launching ansible-playbook

PLAY [all] ****************************************************************************************************************
2021-02-18T11:04:20.898772 (delta: 0.005543)         elapsed: 0.005543 ******** 
=============================================================================== 

TASK [Check ansible version] **********************************************************************************************
2021-02-18T11:04:20.922660 (delta: 0.023862)         elapsed: 0.029431 ******** 
ok: [brain.irma -> localhost] => {
    "changed": false,
    "msg": "All assertions passed"
}

PLAY [Common tasks to execute on all Linux servers] ***********************************************************************
2021-02-18T11:04:20.959915 (delta: 0.037161)         elapsed: 0.066686 ******** 
=============================================================================== 

TASK [quarkslab.apt : Check Apt last update] ******************************************************************************
2021-02-18T11:04:20.973442 (delta: 0.013467)         elapsed: 0.080213 ******** 
The authenticity of host '172.16.1.30 (172.16.1.30)' can't be established.
ECDSA key fingerprint is SHA256:0LPq1Ft47huodOEkDDZ0u/ddoq7fJNPVg27NEreV95Y.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
ok: [brain.irma]

TASK [quarkslab.apt : Update Apt if needed] *******************************************************************************
2021-02-18T11:04:30.736515 (delta: 9.763047)         elapsed: 9.843286 ******** 
skipping: [brain.irma]

TASK [quarkslab.apt : Debian Security] ************************************************************************************
2021-02-18T11:04:30.767799 (delta: 0.031224)         elapsed: 9.87457 ********* 
changed: [brain.irma] => (item=['deb', 'http://security.debian.org/', 'stretch/updates', "['main']"])
failed: [brain.irma] (item=['deb-src', 'http://security.debian.org/', 'stretch/updates', "['main']"]) => {"changed": false, "item": ["deb-src", "http://security.debian.org/", "stretch/updates", "['main']"], "module_stderr": "load pubkey \"/home/fc/.vagrant.d/insecure_private_key\": invalid format\r\nConnection to 172.16.1.30 closed.\r\n", "module_stdout": "Traceback (most recent call last):\r\n  File \"/home/vagrant/.ansible/tmp/ansible-tmp-1613646281.874775-155547501374282/AnsiballZ_apt_repository.py\", line 113, in <module>\r\n    _ansiballz_main()\r\n  File \"/home/vagrant/.ansible/tmp/ansible-tmp-1613646281.874775-155547501374282/AnsiballZ_apt_repository.py\", line 105, in _ansiballz_main\r\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\r\n  File \"/home/vagrant/.ansible/tmp/ansible-tmp-1613646281.874775-155547501374282/AnsiballZ_apt_repository.py\", line 48, in invoke_module\r\n    imp.load_module('__main__', mod, module, MOD_DESC)\r\n  File \"/tmp/ansible_apt_repository_payload_BjFBs8/__main__.py\", line 549, in <module>\r\n  File \"/tmp/ansible_apt_repository_payload_BjFBs8/__main__.py\", line 541, in main\r\n  File \"/usr/lib/python2.7/dist-packages/apt/cache.py\", line 483, in update\r\n    raise FetchFailedException(e)\r\napt.cache.FetchFailedException: E:Failed to fetch http://security.debian.org/dists/stretch/updates/InRelease  Unable to find expected entry '['main']/source/Sources' in Release file (Wrong sources.list entry or malformed file), E:Some index files failed to download. They have been ignored, or old ones used instead.\r\n", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

PLAY RECAP ****************************************************************************************************************
brain.irma                 : ok=2    changed=0    unreachable=0    failed=1   

2021-02-18T11:04:44.739383 (delta: 13.971556)         elapsed: 23.846154 ****** 
===============================================================================

Any ideas?

Thanks.

Support endpoint similar to Virus Total Api to launch a scan

It would be good if opensource IRMA tool supports an Endpoint which is similar to Virtus total Api endpoints. So that people can use their existing automated scritps to work with IRMA also.

support for following 3 url's: https://www.virustotal.com/en/documentation/public-api/

To start a scan:
https://www.virustotal.com/vtapi/v2/file/scan (Which should upload a file + do a scan)

To do rescan
Which is force scan in our case

https://www.virustotal.com/vtapi/v2/file/rescan

To get reports:
https://www.virustotal.com/vtapi/v2/file/report

We can add new url's and try to call same functions in our code base which supports VT api endpoints.

improper address binding

Greetings,

I am a security researcher, who is looking for security smells in Ansible scripts. I noticed instances of binding to 0.0.0.0. Binding an address to 0.0.0.0 indicates allowing connections from all IP addresses. I would like to draw attention to these instances. Binding to 0.0.0.0 may lead to denial of service attacks. Practitioners have reported how binding to 0.0.0.0 facilitated security issues for MySQL (https://serversforhackers.com/c/mysql-network-security), Memcached (https://news.ycombinator.com/item?id=16493480), and Kibana (https://www.elastic.co/guide/en/kibana/5.0/breaking-changes-5.0.html).

Any feedback is appreciated.

Source: https://github.com/quarkslab/irma/blob/master/ansible/playbooks/group_vars/brain.yml

Brain.Irma not reachable after installation.

Hello, I am trying to deploying IRMA to integrate it with Cuckoo Sandbox. After performing the installation as per the guide in the documentation, I run the command :

python2.7 irma-ansible.py environments/allinone_prod.yml playbooks/playbook.yml

and following error is returned.

brain.irma : ok=0 changed=0 unreachable=1 failed=0
localhost : ok=3 changed=0 unreachable=0 failed=0

I have followed the standard documentation guide. Basically I need the URL to IRMA installation in order to integrate it with cuckoo. On searching, I found out that the default URL for frontend is http://172.16.1.30 but I believe it belongs to brain.irma box which I do not have installed. Can yo please tell me what I need to do in order to get a valid IRMA URL? Thanks

column irma_probeResult.doc does not exist

Manual install of version 1.5.3. File upload returns api error with this in the frontend_api.log file:

ProgrammingError: (psycopg2.ProgrammingError) column irma_probeResult.doc does not exist

LINE 1: ...rma_probeResult".name AS "irma_probeResult_name", "irma_prob...
                                                             ^
 [SQL: 'SELECT "irma_probeResult".id AS "irma_probeResult_id", "irma_probeResult".type AS "irma_probeRes
ult_type", "irma_probeResult".name AS "irma_probeResult_name", "irma_probeResult".doc AS "irma_probeResu
lt_doc", "irma_probeResult".status AS "irma_probeResult_status", "irma_probeResult".id_file AS "irma_pro
beResult_id_file", anon_1."irma_fileWeb_id" AS "anon_1_irma_fileWeb_id" \nFROM (SELECT "irma_fileWeb".id
 AS "irma_fileWeb_id" \nFROM "irma_fileWeb" \nWHERE %(param_1)s = "irma_fileWeb".id_scan) AS anon_1 JOIN
 "irma_probeResult_fileWeb" AS "irma_probeResult_fileWeb_1" ON anon_1."irma_fileWeb_id" = "irma_probeRes
ult_fileWeb_1".id_fw JOIN "irma_probeResult" ON "irma_probeResult".id = "irma_probeResult_fileWeb_1".id_
pr ORDER BY anon_1."irma_fileWeb_id"'] [parameters: {'param_1': 5}]
Exception <class 'sqlalchemy.exc.ProgrammingError'>:(psycopg2.ProgrammingError) column irma_probeResult.
doc does not exist

Command alembic current on frontend node produces:

INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
56608def7269 (head)

Is a migration file missing? In the frontend/extras/migrations/versions/ directory I have:
2cc69d5c53eb_db_revision_creation.py
56608def7269_version_1_5_2.py
430a70c8aa21_version_1_2_1.py
eb7141efd75a_version_1_3_0.py

IRMA deployment

I want to install IRMA in our company (offline deployment), it is possible or not.

Adding a probe

When you add a probe in Windows7 and Ubuntu there is one error:
Traceback (most recent call last):
File "./venv/bin/celery", line 11, in
sys.exit(main())
File "/opt/irma/irma-probe/current/venv/local/lib/python2.7/site-packages/celery/main.py", line 30, in main
main()
File "/opt/irma/irma-probe/current/venv/local/lib/python2.7/site-packages/celery/bin/celery.py", line 81, in main
cmd.execute_from_commandline(argv)
File "/opt/irma/irma-probe/current/venv/local/lib/python2.7/site-packages/celery/bin/celery.py", line 793, in execute_from_commandline
super(CeleryCommand, self).execute_from_commandline(argv)))
File "/opt/irma/irma-probe/current/venv/local/lib/python2.7/site-packages/celery/bin/base.py", line 309, in execute_from_commandline
argv = self.setup_app_from_commandline(argv)
File "/opt/irma/irma-probe/current/venv/local/lib/python2.7/site-packages/celery/bin/base.py", line 469, in setup_app_from_commandline
self.app = self.find_app(app)
File "/opt/irma/irma-probe/current/venv/local/lib/python2.7/site-packages/celery/bin/base.py", line 489, in find_app
return find_app(app, symbol_by_name=self.symbol_by_name)
File "/opt/irma/irma-probe/current/venv/local/lib/python2.7/site-packages/celery/app/utils.py", line 235, in find_app
sym = symbol_by_name(app, imp=imp)
File "/opt/irma/irma-probe/current/venv/local/lib/python2.7/site-packages/celery/bin/base.py", line 492, in symbol_by_name
return symbol_by_name(name, imp=imp)
File "/opt/irma/irma-probe/current/venv/local/lib/python2.7/site-packages/kombu/utils/init.py", line 96, in symbol_by_name
module = imp(module_name, package=package, **kwargs)
File "/opt/irma/irma-probe/current/venv/local/lib/python2.7/site-packages/celery/utils/imports.py", line 101, in import_from_cwd
return imp(module, package=package)
File "/usr/lib/python2.7/importlib/init.py", line 37, in import_module
import(name)
File "/opt/irma/irma-probe/current/probe/tasks.py", line 103, in
p.plugin_mimetype_regexp)
File "/opt/irma/irma-probe/current/probe/controllers/braintasks.py", line 32, in register_probe
args=[name, display_name, category, mimetype_regexp])
File "/opt/irma/irma-probe/current/probe/helpers/celerytasks.py", line 31, in async_call
raise IrmaTaskError("Celery error - {0}".format(taskname))
lib.irma.common.exceptions.IrmaTaskError: Celery error - register_probe

what am I doing wrong.

Unable to connect to http://172.16.1.30/

I installed irma using these commands:
$ git clone https://github.com/quarkslab/irma
$ cd /irma/ansible
$ vagrant up
Installation is completed successfully but cant open web interface,kindly help.

Automatically report scan results to email address.

If an antivirus detects a scanned file as malicious request the user to add their details to the scan (e.g. work ID or email) and automatically report the results page to a specific email address (e.g. soc).

This would be very useful for security operations centre and would allow analysts to react to security incidents in real time.

report object in JSON

Hi,

would it make sense for you guys to have report object unified to string across all probes and in case additional data is available then save it in another object (for example report_data) ?
Background behind that is when I roll over all probe_results then I get to a point where normally results value is always a string. However PE results gives a JSON object back. Therefore it gets quite complicated to save the result to Elasticsearch as there is no unified mapping to be created so it will always be a workaround :/
Case where I bumped into it was : cuckoosandbox/cuckoo#1217

Cheers,
razu

Where is irma-probe-app-1.1.0.tar.gz located?

Hi,

I am using the finished virtual machine (1.3.2) that is available on the website and I am trying to configure a seperate probe. I have followed the installation instructions to the letter and cloned the repository, but when it comes to this line:

pip install irma-probe-app-1.1.0.tar.gz --install-option="--install-base=/opt/irma/irma-probe"

This files does not exsist anywhere inside the irma-probe repo? Am I missing something? Where is this file supposed to be located?

Appreciate any help in this matter.

Query on installation process

Hi, i am interested with this framework to be installed for educational purposes but I am not experienced in Linux. I am currently installing on a Ubuntu 20.04 LTS in VirtualBox, as in allinone_dev.

My installation process came to a halt when I attempted to run the following command:

python3 irma-ansible.py environments/allinone_dev.yml playbooks/playbook.yml

install_log.txt

Attached with an install log where the error indicates, and I would like to know how to resolve it.

Thank you.

Symbolic CPU

ETA symbolic CPU in IRMA ?

Thanks guys !

Kaspersky Endpoint Security 10

Kaspersky Endpoint Security 10 outputs the result "clean" in the Web, although the Probe detection is. Where am I wrong?

Unable to run IRMA without Vagrant

I tried to install IRMA in production mode to pre-configured Debian10 hosts which were deployed using Debian OpenStack cloud images.

Ansible playbooks were successful after some modifications to the code, but application is not working properly. I checked system logs and found that brain and frontend services are not started. Thus frontend is serving the UI but there aren't any probes registred. It might be because I've made some serious code changes and that's why it's not working. Maybe you can guide me, what might be causing the errors and how can I fix it?

Brain journalctl error:

Feb 13 16:53:19 irma-brain systemd[1]: Started irma.result_app.service.
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]: Traceback (most recent call last):
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:     "__main__", mod_spec)
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:     exec(code, run_globals)
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "/opt/irma/irma-brain/releases/20210213163311/brain/results_tasks.py", line 81, in <module>
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:     results_app.worker_main(options)
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "/opt/irma/irma-brain/current/venv/lib/python3.7/site-packages/celery/app/base.py", line 206, in worker_main
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:     app=self).execute_from_commandline(argv)
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "/opt/irma/irma-brain/current/venv/lib/python3.7/site-packages/celery/bin/base.py", line 311, in execute_from_commandline
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:     return self.handle_argv(self.prog_name, argv[1:])
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "/opt/irma/irma-brain/current/venv/lib/python3.7/site-packages/celery/bin/base.py", line 377, in handle_argv
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:     return self(*args, **options)
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "/opt/irma/irma-brain/current/venv/lib/python3.7/site-packages/celery/bin/base.py", line 274, in __call__
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:     ret = self.run(*args, **kwargs)
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "/opt/irma/irma-brain/current/venv/lib/python3.7/site-packages/celery/bin/worker.py", line 194, in run
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:     pool_cls = (concurrency.get_implementation(pool_cls) or
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "/opt/irma/irma-brain/current/venv/lib/python3.7/site-packages/celery/concurrency/__init__.py", line 29, in get_implementation
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:     return symbol_by_name(cls, ALIASES)
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "/opt/irma/irma-brain/current/venv/lib/python3.7/site-packages/kombu/utils/__init__.py", line 96, in symbol_by_name
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:     module = imp(module_name, package=package, **kwargs)
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:     return _bootstrap._gcd_import(name[level:], package, level)
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "<frozen importlib._bootstrap>", line 983, in _find_and_load
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "<frozen importlib._bootstrap_external>", line 728, in exec_module
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "/opt/irma/irma-brain/current/venv/lib/python3.7/site-packages/celery/concurrency/prefork.py", line 20, in <module>
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:     from celery.concurrency.base import BasePool
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "/opt/irma/irma-brain/current/venv/lib/python3.7/site-packages/celery/concurrency/base.py", line 21, in <module>
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:     from celery.utils import timer2
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:   File "/opt/irma/irma-brain/current/venv/lib/python3.7/site-packages/celery/utils/timer2.py", line 19
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:     from kombu.async.timer import Entry, Timer as Schedule, to_timestamp, logger
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]:                    ^
Feb 13 16:53:20 irma-brain [irma.brain.results][32278]: SyntaxError: invalid syntax
Feb 13 16:53:20 irma-brain systemd[1]: irma.result_app.service: Main process exited, code=exited, status=1/FAILURE
Feb 13 16:53:20 irma-brain systemd[1]: irma.result_app.service: Failed with result 'exit-code'.

Frontend journalctl logs:

Feb 13 16:32:36 irma-frontend systemd[1]: Started irma.frontend_app.service.
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]: Traceback (most recent call last):
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:     "__main__", mod_spec)
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:     exec(code, run_globals)
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "/opt/irma/irma-frontend/releases/20210213163208/api/tasks/frontend_app.py", line 175, in <module>
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:     frontend_app.worker_main(options)
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "/opt/irma/irma-frontend/current/venv/lib/python3.7/site-packages/celery/app/base.py", line 206, in worker_main
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:     app=self).execute_from_commandline(argv)
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "/opt/irma/irma-frontend/current/venv/lib/python3.7/site-packages/celery/bin/base.py", line 311, in execute_from_commandline
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:     return self.handle_argv(self.prog_name, argv[1:])
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "/opt/irma/irma-frontend/current/venv/lib/python3.7/site-packages/celery/bin/base.py", line 377, in handle_argv
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:     return self(*args, **options)
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "/opt/irma/irma-frontend/current/venv/lib/python3.7/site-packages/celery/bin/base.py", line 274, in __call__
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:     ret = self.run(*args, **kwargs)
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "/opt/irma/irma-frontend/current/venv/lib/python3.7/site-packages/celery/bin/worker.py", line 194, in run
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:     pool_cls = (concurrency.get_implementation(pool_cls) or
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "/opt/irma/irma-frontend/current/venv/lib/python3.7/site-packages/celery/concurrency/__init__.py", line 29, in get_implementation
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:     return symbol_by_name(cls, ALIASES)
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "/opt/irma/irma-frontend/current/venv/lib/python3.7/site-packages/kombu/utils/__init__.py", line 96, in symbol_by_name
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:     module = imp(module_name, package=package, **kwargs)
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:     return _bootstrap._gcd_import(name[level:], package, level)
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "<frozen importlib._bootstrap>", line 983, in _find_and_load
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "<frozen importlib._bootstrap_external>", line 728, in exec_module
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "/opt/irma/irma-frontend/current/venv/lib/python3.7/site-packages/celery/concurrency/prefork.py", line 20, in <module>
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:     from celery.concurrency.base import BasePool
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "/opt/irma/irma-frontend/current/venv/lib/python3.7/site-packages/celery/concurrency/base.py", line 21, in <module>
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:     from celery.utils import timer2
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:   File "/opt/irma/irma-frontend/current/venv/lib/python3.7/site-packages/celery/utils/timer2.py", line 19
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:     from kombu.async.timer import Entry, Timer as Schedule, to_timestamp, logger
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]:                    ^
Feb 13 16:32:37 irma-frontend [irma.frontend][1848]: SyntaxError: invalid syntax
Feb 13 16:32:37 irma-frontend systemd[1]: irma.frontend_app.service: Main process exited, code=exited, status=1/FAILURE
Feb 13 16:32:37 irma-frontend systemd[1]: irma.frontend_app.service: Failed with result 'exit-code'.

Marco

Refreshing probe list return an empty list

Hello guys,

I am tryharding to make IRMA work more than one day.

Indeed, it is working only less than 24h in a row, then the probe list is randomly emptied because brain and probes do not communicate anymore.

All my supervisor tasks are in debug mode, but I have no logs explaining why the connection between probes and brain is timed out.

Here is an example of brain supervisor.log showed.
brain log.txt

If you have an idea of where I can find the origin of the issue... Thank you.

Review merge of all documentation

Public IP

Hey there, I am interested in hosting this on a VPS in order to be accessible from everywhere from a public IP. Is this possible to and if so, how?

error during ansible setup

Hello,

I got an error while setting up ansible (prod):
python irma-ansible.py environments/allinone_prod.yml playbooks/playbook.yml

The errors are:
`
TASK [ANXS.postgresql : PostgreSQL | Make sure the dependencies are installed] ************
2019-05-22T11:31:33.799812 (delta: 1.529815) elapsed: 54.781102 *******
failed: [brain.irma] (item=[u'python-psycopg2', u'python-pycurl', u'locales']) => {"changed": false, "item": ["python-psycopg2", "python-pycurl", "locales"], "msg": "Failed to update apt cache."}

PLAY RECAP ********************************************************************************
brain.irma : ok=41 changed=6 unreachable=0 failed=1
localhost : ok=1 changed=0 unreachable=0 failed=0
`

Also, python-psycopg2, python-pycurl and locales are well installed.

Any idea?

Add asciinema demo of cli version

and irma-ansible provisioning ?

Instance of hardcoded secret

Greetings,

I am a security researcher, who is looking for security smells in Ansible scripts.
I found instances where usernames and passwords are specified within a Ansible script.
According to the Common Weakness Enumeration organization this is a security weakness
(CWE-798: Hard-coded credentials https://cwe.mitre.org/data/definitions/798.html).

I am trying to find out if you agree with the findings and the reasons the usernames and passwords were introduced. Any feedback is appreciated.

Any feedback is appreciated.

Source: https://github.com/quarkslab/irma/blob/master/ansible/playbooks/group_vars/all.yml

Error on launching playbook provisioning.yml

Hello,

I have been trying to install IRMA on Ubuntu 20.04.6. I am getting below error upon executing command python irma-ansible.py environments/allinone_prod.yml playbooks/provisioning.yml

/home/wonder/.local/lib/python2.7/site-packages/ansible/parsing/vault/__init__.py:41: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release.
  from cryptography.exceptions import InvalidSignature
irma-ansible.py:38: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  self.config = yaml.load(f.read())
[+] Ansible vars written to /tmp/tmpxXfVWJ/vars.yml
[+] Ansible inventory written to /tmp/tmpxXfVWJ/inventory
[+] launching ansible-galaxy
- extracting ANXS.postgresql to /home/wonder/irma/ansible/roles/ANXS.postgresql
- ANXS.postgresql (cec55c974e6020044ef908e2acca9377d470b473) was installed successfully
- extracting franklinkim.sudo to /home/wonder/irma/ansible/roles/franklinkim.sudo
- franklinkim.sudo (1.9.0) was installed successfully
- extracting franklinkim.ufw to /home/wonder/irma/ansible/roles/franklinkim.ufw
- franklinkim.ufw (1.7.1) was installed successfully
- extracting jdauphant.nginx to /home/wonder/irma/ansible/roles/jdauphant.nginx
- jdauphant.nginx (v2.19) was installed successfully
- extracting Mayeu.RabbitMQ to /home/wonder/irma/ansible/roles/Mayeu.RabbitMQ
- Mayeu.RabbitMQ (47742f6d7c09edcd4ffb5b5c23ffff75599ed4e6) was installed successfully
- extracting mivok0.users to /home/wonder/irma/ansible/roles/mivok0.users
- mivok0.users (v1.2.5) was installed successfully
- extracting willshersystems.sshd to /home/wonder/irma/ansible/roles/willshersystems.sshd
- willshersystems.sshd (v0.7.2) was installed successfully
[+] launching ansible-playbook

PLAY [all] *********************************************************************
2023-07-04T13:06:29.128201 (delta: 0.022225)         elapsed: 0.022225 ******** 
=============================================================================== 

TASK [Gathering Facts] *********************************************************
2023-07-04T13:06:29.160314 (delta: 0.03207)         elapsed: 0.054338 ********* 
The authenticity of host '172.16.1.30 (172.16.1.30)' can't be established.
ECDSA key fingerprint is SHA256:0LPq1Ft47huodOEkDDZ0u/ddoq7fJNPVg27NEreV95Y.
Are you sure you want to continue connecting (yes/no/[fingerprint])? ok: [localhost]
yes
ok: [brain.irma]

TASK [Check ansible version] ***************************************************
2023-07-04T13:06:37.381036 (delta: 8.220652)         elapsed: 8.27506 ********* 
ok: [brain.irma -> localhost] => {
    "changed": false, 
    "msg": "All assertions passed"
}

PLAY [Common tasks to execute on all Linux servers] ****************************
2023-07-04T13:06:37.434029 (delta: 0.052917)         elapsed: 8.328053 ******** 
=============================================================================== 

TASK [quarkslab.apt : Check Apt last update] ***********************************
2023-07-04T13:06:37.469934 (delta: 0.035861)         elapsed: 8.363958 ******** 
ok: [brain.irma]

TASK [quarkslab.apt : Update Apt if needed] ************************************
2023-07-04T13:06:39.651884 (delta: 2.181906)         elapsed: 10.545908 ******* 
skipping: [brain.irma]

TASK [quarkslab.apt : Debian Security] *****************************************
2023-07-04T13:06:39.700841 (delta: 0.048885)         elapsed: 10.594865 ******* 
failed: [brain.irma] (item=[u'deb', u'http://security.debian.org/', u'stretch/updates', u'main']) => {"changed": false, "item": ["deb", "http://security.debian.org/", "stretch/updates", "main"], "msg": "Failed to auto-install python-apt. Error was: 'W: The repository 'http://deb.debian.org/debian stretch Release' does not have a Release file.\nW: The repository 'http://security.debian.org/debian-security stretch/updates Release' does not have a Release file.\nE: Failed to fetch http://deb.debian.org/debian/dists/stretch/main/source/Sources  404  Not Found [IP: 199.232.22.132 80]\nE: Failed to fetch http://security.debian.org/debian-security/dists/stretch/updates/main/source/Sources  404  Not Found [IP: 199.232.22.132 80]\nE: Some index files failed to download. They have been ignored, or old ones used instead.'"}
failed: [brain.irma] (item=[u'deb-src', u'http://security.debian.org/', u'stretch/updates', u'main']) => {"changed": false, "item": ["deb-src", "http://security.debian.org/", "stretch/updates", "main"], "msg": "Failed to auto-install python-apt. Error was: 'W: The repository 'http://security.debian.org/debian-security stretch/updates Release' does not have a Release file.\nW: The repository 'http://deb.debian.org/debian stretch Release' does not have a Release file.\nE: Failed to fetch http://security.debian.org/debian-security/dists/stretch/updates/main/source/Sources  404  Not Found [IP: 199.232.22.132 80]\nE: Failed to fetch http://deb.debian.org/debian/dists/stretch/main/source/Sources  404  Not Found [IP: 199.232.22.132 80]\nE: Some index files failed to download. They have been ignored, or old ones used instead.'"}

PLAY RECAP *********************************************************************
brain.irma                 : ok=3    changed=0    unreachable=0    failed=1   
localhost                  : ok=1    changed=0    unreachable=0    failed=0   

2023-07-04T13:06:51.089691 (delta: 11.388769)         elapsed: 21.983715 ******

I have referred to various issues present in the repository but could not find any solution to fix this issue. Please help me to find out the solution

Setting up IRMA

Hello,

i've tried setting up IRMA the easiest way,

everything goes right until i want to see the frontend interface of irma i saw in a previous issue that i need to setup irma-frontend and i'd love to but it seems that the command :

python irma-ansible.py environments/allinone_prod.yml playbooks/playbook.yml

it cannot find irma-ansible.py and the shortcut seems broken,
is the command i put above the one that will setup irma frontend ?
could i have some insight on what to do to fix this ?

Thank you

Don't know how to install irma and set it up

add waffle.io badges

Scripts are not working

Hi,

From the Brian configuration
$ cd /opt/irma/irma-brain/current/
$ ./venv/bin/python -m scripts.create_user

scripts are not working and couldn't find the scripts anywhere, can you help ?

IRMA in maintenance mode

Hi, would like to ask for help again.

My installation was a success after referring to my post #74. I am able to see the web interface on 172.16.1.30 for the first time.

However, the next time I reboot the machine and also manually turn on ansible_brainirma in VM, I am getting a message whereas it shows "Irma is currently in maintenance mode".

Help is greatly appreciated..

Search is slow in web interface

I found the search feature in the web user interface to be rather slow after adding a custom probe with very verbose output. I believe this is because of all the fields in the FileWebSchema that are getting pulled in. I was able to speed up searching considerably by limiting FileWebSchema to result_id, name, and file_infos, since all the other fields are not even displayed (or used?) on the search page.

Quarkslab box not found

Bringing machine 'default' up with 'virtualbox' provider...
==> default: Box 'quarkslab/debian-9.0.0-amd64' could not be found. Attempting to find and install...
default: Box Provider: virtualbox
default: Box Version: 20180312
The box 'quarkslab/debian-9.0.0-amd64' could not be found or
could not be accessed in the remote catalog. If this is a private
box on HashiCorp's Atlas, please verify you're logged in via
vagrant login. Also, please double-check the name. The expanded
URL and error message are shown below:

URL: ["https://atlas.hashicorp.com/quarkslab/debian-9.0.0-amd64"]
Error: The requested URL returned error: 404 Not Found

[Feature] Moving Web-UI outside of irma-frontend repository

Hi,
I'm planning to move the web directory from quarkslab/irma-frontend repository outisde of it. In fact, the irma-frontend main role is just to provide an API that the Web-ui can use. So, depending on how the server is configured, it'll be possible to install the web interface outside of the server (required CORS activation).
Moreover, anothe repository, quarkslab/web-ui-dist will be created and generated automatically from any merge on quarkslab/web-ui repository.

Keep in touch!
Regards.

Irma frontend not working

Hi, I trying deploy irma via Vagrant ( export VM_ENV=allinone_dev) deploying but nginx not installing, frontend not working.
Log deploying irma via Vagrant:

Please help me, what am I doing wrong

Log deploing irma via Vagrant:

root@cuckoo2:/opt/irma/ansible# vagrant up
172.16.1.30
Bringing machine 'brain.irma' up with 'virtualbox' provider...
==> brain.irma: Importing base box 'quarkslab/debian-9.0.0-amd64'...
==> brain.irma: Matching MAC address for NAT networking...
==> brain.irma: Checking if box 'quarkslab/debian-9.0.0-amd64' is up to date...
==> brain.irma: Setting the name of the VM: ansible_brainirma_1536582797123_91188
==> brain.irma: Clearing any previously set network interfaces...
==> brain.irma: Preparing network interfaces based on configuration...
brain.irma: Adapter 1: nat
brain.irma: Adapter 2: hostonly
==> brain.irma: Forwarding ports...
brain.irma: 80 (guest) => 8080 (host) (adapter 1)
brain.irma: 22 (guest) => 2222 (host) (adapter 1)
==> brain.irma: Running 'pre-boot' VM customizations...
==> brain.irma: Booting VM...
==> brain.irma: Waiting for machine to boot. This may take a few minutes...
brain.irma: SSH address: 127.0.0.1:2222
brain.irma: SSH username: vagrant
brain.irma: SSH auth method: private key
==> brain.irma: Machine booted and ready!
==> brain.irma: Checking for guest additions in VM...
==> brain.irma: Setting hostname...
==> brain.irma: Configuring and enabling network interfaces...
==> brain.irma: Installing rsync to the VM...
==> brain.irma: Rsyncing folder: /opt/irma/common/ => /opt/irma/irma-common/releases/sync
==> brain.irma: - Exclude: [".vagrant/", ".git/", "venv/"]
==> brain.irma: Rsyncing folder: /opt/irma/frontend/ => /opt/irma/irma-frontend/releases/sync
==> brain.irma: - Exclude: [".vagrant/", ".git/", "venv/", "web/dist", "web/node_modules", "app/components"]
==> brain.irma: Rsyncing folder: /opt/irma/brain/ => /opt/irma/irma-brain/releases/sync
==> brain.irma: - Exclude: [".vagrant/", ".git/", "venv/", "db/"]
==> brain.irma: Rsyncing folder: /opt/irma/probe/ => /opt/irma/irma-probe/releases/sync
==> brain.irma: - Exclude: [".vagrant/", ".git/", "venv/"]

authentication failure

Activating analyzer

sudo su irma
cd /opt/irma/irma -probe/current/
venv/bin/pip install -r \ modules/external/virustotal/requirements.txt

At step 3 error occurred and it says permission denied when step 3 is executed.
how to access the root and need the root password
I have imported the Irma 1.5.3 .ova file.
It need the password for sudo su irma ?

Update service.py

Python 3.5 does not support multiprocessing.forking and ConfigParser modules.

Original script was updated for Python 3+
https://github.com/antoinemartin/django-windows-tools/blob/master/django_windows_tools/service.py

Can you update your script, please.
https://github.com/quarkslab/irma/blob/master/probe/extras/winsrv/service.py

missing _ in irma/probe/modules/custom/skeleton/plugin.py

line 36: there's a _ missing at the end of _plugin_display_name

"/opt/sophos-av/bin/savscan --version" "[Errno 2] No such file or directory"

Hi quarkslab,

the VM_ENV=dev vagrant provision (TASK [quarkslab.sophos : Sophos | Check installed version]) fails with above error message but it is installed and at the correct place.

System time 10:17:51 AM, System date 15 December 2017

Product version : 5.38.0
Engine version : 3.69.2
Virus data version : 5.41
User interface version : 2.03.069
Platform : Linux/AMD64
Released : 27 June 2017
Total viruses (with IDEs) : 13603638

Next error is:
TASK [quarkslab.sophos : Sophos | Download installation archive]
fatal: [avs-linux.irma]: FAILED! => {"changed": false, "dest": "/tmp/sophos/sav-linux.tgz", "failed": true, "msg": "Request failed", "response": "HTTP Error 404: Not Found", "state": "absent", "status_code": 404, "url": "http://downloads.sophos.com/XXXX"}

I've also placed the renamed original file at above destination.

root@ubuntu:/opt/irma/ansible# ls -l /tmp/sophos/sav-linux.tgz
-rw-r--r-- 1 root root 539920492 Dec 15 10:01 /tmp/sophos/sav-linux.tgz

Could you please help me? :-)

Thanks a lot and have a nice weekend.
Marcus

Report

Hi all,
is possible to add a report module to export the results to xml for example.

Thanks.

ssh2.exceptions.SFTPProtocolError

Hello quarkslab,

thanks for the tool, but while I use the OVA I came on some strange message that keeps me from using it. - Despite the query for help resolfing the issue, what is IRMA trying to use the ssh for?

Cheers and thanks a lot
Marcus

[2019-03-02 13:56:30,733: ERROR/Worker-2] brain.scan_tasks.scan_flush[3d0da3d9-8077-490e-93a0-49ed1b1b05c6]:
Traceback (most recent call last):
File "/opt/irma/irma-brain/current/venv/lib/python3.5/site-packages/irma/common/ftp/ftp.py", line 157, in delete
self._rm(full_dstpath)
File "/opt/irma/irma-brain/current/venv/lib/python3.5/site-packages/irma/common/ftp/sftpv2.py", line 101, in _rm
self._client.unlink(remote)
File "ssh2/sftp.pyx", line 296, in ssh2.sftp.SFTP.unlink
File "ssh2/utils.pyx", line 157, in ssh2.utils.handle_error_codes
ssh2.exceptions.SFTPProtocolError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/irma/irma-brain/releases/20180719115813/brain/scan_tasks.py", line 192, in scan_flush
scan_ctrl.flush(scan, session)
File "/opt/irma/irma-brain/current/venv/lib/python3.5/site-packages/fasteners/process_lock.py", line 252, in wrapper
return f(*args, **kwargs)
File "/opt/irma/irma-brain/releases/20180719115813/brain/controllers/scanctrl.py", line 57, in flush
ftp_ctrl.flush(ftpuser, scan.files)
File "/opt/irma/irma-brain/releases/20180719115813/brain/controllers/ftpctrl.py", line 30, in flush
ftp.delete(".", filename)
File "/opt/irma/irma-brain/current/venv/lib/python3.5/site-packages/irma/common/ftp/ftp.py", line 159, in delete
raise self._Exception("{0}".format(e))
irma.common.base.exceptions.IrmaSFTPv2Error

Analyze big file with IRMA

Hi guys,
I got a problem when i scan big file (between 200MO and 1 GO) with IRMA.
The log file tells me its an error 413, a Request Entity Too Large error.
So i deactivate the functionality in frontend and frontend-https with the option :

client_max_body_size 0.

But i still get an error 413. What do i miss ?

quarkslab / irma Goto Github PK

irma's People

Contributors

Stargazers

Watchers

Forkers

irma's Issues

Recommend Projects

Recommend Topics

Recommend Org