shirosaidev / diskover-web Goto Github PK

Web file manager, disk space usage, storage search engine and file system analytics for diskover

License: Apache License 2.0

analytics disk-space disk-space-analyzer disk-usage elasticsearch file-analysis file-indexing file-management file-manager file-tagging filemanager filesystem meta-data metadata php rest-api storage storage-analysis storage-analytics visualization

diskover-web's People

Contributors

Stargazers

Watchers

diskover-web's Issues

REST API | tagging multiple files results in error from ES

Hi guys when i wanna tag multiples files, diskover return a weird error

here is my REST code

d = {'tag': 'keep', 'files': ['/volt/minuscule2/SHOTS/1390/1390_0050/GuerillaScene/VERSIONS/1390_0050_GuerillaScene_v002.gproject', '/volt/minuscule2/SHOTS/1390/1390_0050/GuerillaScene/VERSIONS/1390_0050_GuerillaScene_v012.gproject']}
r = requests.put(
            "http://diskover.theyard.lan/api.php/diskover-2020-10-29-volt/tagfile",
            data=json.dumps(d),
            timeout=5,
        )

it return this error

{'error': 'Message: '
          'Elasticsearch\\Common\\Exceptions\\UnexpectedValueException: "id" '
          'is not a valid parameter. Allowed parameters are "_source", '
          '"_source_exclude", "_source_include", "allow_no_indices", '
          '"analyze_wildcard", "analyzer", "batched_reduce_size", "client", '
          '"custom", "default_operator", "df", "docvalue_fields", '
          '"expand_wildcards", "explain", "fielddata_fields", "filter_path", '
          '"filter_path", "from", "human", "ignore_unavailable", '
          '"indices_boost", "lenient", "lowercase_expanded_terms", '
          '"preference", "q", "query_cache", "request_cache", "routing", '
          '"scroll", "search_type", "size", "slice", "sort", "source", '
          '"stats", "stored_fields", "suggest_field", "suggest_mode", '
          '"suggest_size", "suggest_text", "terminate_after", "timeout", '
          '"typed_keys", "version" in '
          '/app/diskover-web/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Endpoints/AbstractEndpoint.php:237\n'
          'Stack trace:\n'
          '#0 '
          '/app/diskover-web/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Endpoints/AbstractEndpoint.php(74): '
          'Elasticsearch\\Endpoints\\AbstractEndpoint->checkUserParams(Array)\n'
          '#1 '
          '/app/diskover-web/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Client.php(952): '
          'Elasticsearch\\Endpoints\\AbstractEndpoint->setParams(Array)\n'
          '#2 /app/diskover-web/public/api.php(277): '
          'Elasticsearch\\Client->search(Array)\n'
          '#3 /app/diskover-web/public/api.php(38): put(Array, Array)\n'
          '#4 {main}'}

i'm using python3.7

ERR_TOO_MANY_REDIRECTS

Getting too many redirects error when trying to access diskover-web from chrome

crawl is successful, index gets created, verified with kibana
using elasticsearch 5.6

not sure how to fix this

[root@app1 public]# php -S app1:8088
PHP 5.6.33 Development Server started at Thu Mar 15 17:16:23 2018
Listening on http://app1:8088
Document root is /opt/diskover-web/public
Press Ctrl-C to quit.
[Thu Mar 15 17:16:23 2018] 172.16.20.4:60344 [302]: /
[Thu Mar 15 17:16:23 2018] 172.16.20.4:60346 [302]: /selectindices.php
[Thu Mar 15 17:16:23 2018] 172.16.20.4:60348 [302]: /selectindices.php
[Thu Mar 15 17:16:23 2018] 172.16.20.4:60350 [302]: /selectindices.php
[Thu Mar 15 17:16:23 2018] 172.16.20.4:60352 [302]: /selectindices.php
[Thu Mar 15 17:16:23 2018] 172.16.20.4:60354 [302]: /selectindices.php
[Thu Mar 15 17:16:23 2018] 172.16.20.4:60356 [302]: /selectindices.php
[Thu Mar 15 17:16:23 2018] 172.16.20.4:60358 [302]: /selectindices.php
[Thu Mar 15 17:16:23 2018] 172.16.20.4:60360 [302]: /selectindices.php
[Thu Mar 15 17:16:23 2018] 172.16.20.4:60362 [302]: /selectindices.php
[Thu Mar 15 17:16:23 2018] 172.16.20.4:60364 [302]: /selectindices.php
[Thu Mar 15 17:16:23 2018] 172.16.20.4:60366 [302]: /selectindices.php
[Thu Mar 15 17:16:23 2018] 172.16.20.4:60368 [302]: /selectindices.php
[Thu Mar 15 17:16:24 2018] 172.16.20.4:60370 [302]: /selectindices.php
[Thu Mar 15 17:16:24 2018] 172.16.20.4:60372 [302]: /selectindices.php
[Thu Mar 15 17:16:24 2018] 172.16.20.4:60374 [302]: /selectindices.php
[Thu Mar 15 17:16:24 2018] 172.16.20.4:60376 [302]: /selectindices.php
[Thu Mar 15 17:16:24 2018] 172.16.20.4:60378 [302]: /selectindices.php
[Thu Mar 15 17:16:24 2018] 172.16.20.4:60380 [302]: /selectindices.php
[Thu Mar 15 17:16:24 2018] 172.16.20.4:60382 [302]: /selectindices.php
[Thu Mar 15 17:16:24 2018] 172.16.20.4:60384 [302]: /selectindices.php

Paths with urlescaped characters in their filenames break the tree-view

I have a directory with a bunch of files and folders where the names of the files/folders have urlescaped punctuation in their actual names.

This appears to completely confuse at least the tree-view. In my case, there are folders which have urlescaped punctuation, and a folder named the exact same without the urlescaping (the root cause of this was a bug in a web spider on my end, but there shouldn't be a filename that can break the web-UI).

I also have some directories with newlines in the folder name. This is technically valid (at least on linux). I'm working to fix it, but it also appears to also confuse the web-up (the newlines appear to be just stripped). This issue might be just the HTML doing text-reflow.

Basically, I think there's a missed escape somewhere in the file processing.

Problem when opening the tool

I changed the Host on Constants.php with my server ip then I start the application with PHP -S ip:8000 and I'm getting the following error:

[Wed Sep  4 14:13:30 2019] PHP Fatal error:  Uncaught Elasticsearch\Common\Exceptions\NoNodesAvailableException: No alive nodes found in your cluster in /home/tito/Downloads/diskover-web/vendor/elasticsearch/elasticsearch/src/Elasticsearch/ConnectionPool/StaticNoPingConnectionPool.php:51
Stack trace:
#0 /home/tito/Downloads/diskover-web/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Transport.php(72): Elasticsearch\ConnectionPool\StaticNoPingConnectionPool->nextConnection()
#1 /home/tito/Downloads/diskover-web/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Transport.php(90): Elasticsearch\Transport->getConnection()
#2 /home/tito/Downloads/diskover-web/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php(256): Elasticsearch\Transport->performRequest('HEAD', '/', Array, NULL, Array)
#3 /home/tito/Downloads/diskover-web/vendor/react/promise/src/FulfilledPromise.php(25): Elasticsearch\Connections\Connection->Elasticsearch\Connections\{closure}(Array)
#4 /home/tito/Downloads/diskover-web/vendor/guzzlehttp/ringphp/src/Fu in /home/tito/Downloads/diskover-web/vendor/elasticsearch/elasticsearch/src/Elasticsearch/ConnectionPool/StaticNoPingConnectionPool.php on line 51
[Wed Sep  4 14:13:30 2019] 189.73.67.165:48042 [500]: / - Uncaught Elasticsearch\Common\Exceptions\NoNodesAvailableException: No alive nodes found in your cluster in /home/tito/Downloads/diskover-web/vendor/elasticsearch/elasticsearch/src/Elasticsearch/ConnectionPool/StaticNoPingConnectionPool.php:51
Stack trace:
#0 /home/tito/Downloads/diskover-web/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Transport.php(72): Elasticsearch\ConnectionPool\StaticNoPingConnectionPool->nextConnection()
#1 /home/tito/Downloads/diskover-web/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Transport.php(90): Elasticsearch\Transport->getConnection()
#2 /home/tito/Downloads/diskover-web/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php(256): Elasticsearch\Transport->performRequest('HEAD', '/', Array, NULL, Array)
#3 /home/tito/Downloads/diskover-web/vendor/react/promise/src/FulfilledPromise.php(25): Elasticsearch\Connections\Connection->Elasticsearch\Connections\{closure}(Array)
#4 /home/tito/Downloads/diskover-web/vendor/guzzlehttp/ringphp/src/Fu in /home/tito/Downloads/diskover-web/vendor/elasticsearch/elasticsearch/src/Elasticsearch/ConnectionPool/StaticNoPingConnectionPool.php on line 51

Anyone can help me?

Output before header function is called

I found an issue while using diskover-web. After I select an index, and click in the submit button, the pages is not redirect and an warning message is presented:
Warning: Cannot modify header information - headers already sent by
It happens that this issue was being cause because there was output before the header function is called:
https://stackoverflow.com/questions/8028957/how-to-fix-headers-already-sent-error-in-php

There output being generated in the selectindices.php file before this line.
https://github.com/shirosaidev/diskover-web/blob/master/public/selectindices.php#L70

Overlay glitches

Noticed some glitches with the information overlay when viewing the heatmap and when trying to work with the drop-down menu for reindexing directories:

Heatmap: https://chrisyocum.com/Images/ShareX/2019-07-15_21-25-09.gif

Reindexing: https://chrisyocum.com/Images/ShareX/2019-07-15_21-26-28.gif

file last_access based view in filetree.php

Do we have any option to replace mtime with last-access time.

api: list indices

is there a way to list all indices via the API?

Request: support multiple tags per file, directory

I would like to be able to set multiple tags on a file, or a directory. When I try add a second tag to a file, the original one gets displaced. I used the diskover webpage to add the tags by modifying the tag of a search.

REST API: How is it supposed to work?

Using curl with the given examples, I only manage to get:

{
    "version": "diskover REST API v1",
    "message": "endpoint not found"
}

Secondly, how is the API supposed to work when I tuned on authentication?

Thanks,

Time Change Analytics Broken

Spoke with shirosai on the LinuxServer.io discord server and he asked me to open this as he was also able to reproduce the issue.

The "Time Change" analytics section is totally broken. When I restart the container, I am able to at least load the "Time Change" page but the graphs are all empty. When I change the filter (to mtime=1 day), I get a 500 error when I reload "Time Change".

run in virtualhost

Hi, when I run diskover-web with php -S :8000 the dashboard works fine, but when I run it in a virtualhost, it gives all kind of errors, like:
00, 'query' => [ 'match' => [ 'state' => 'running' ] ] ]; $queryResponse = $client->search($searchParams); $totalcrawls = $queryRespo...

Here's my virtualhost:

<VirtualHost *:8000>
  DocumentRoot /usr/share/diskover-web/public
  ErrorLog ${APACHE_LOG_DIR}/error-diskover-web.log
  CustomLog ${APACHE_LOG_DIR}/access-diskover-web.log combined
  DirectoryIndex index.php

  alias /src "/usr/share/diskover-web/src"
  alias /vendor "/usr/share/diskover-web/vendor"

  <Directory "/usr/share/diskover-web/public/">
        Options +Indexes +FollowSymLinks +MultiViews
        AllowOverride None
        Order allow,deny
        allow from all
  </Directory>
  <Directory "/usr/share/diskover-web/src/">
        Options +Indexes +FollowSymLinks +MultiViews
        AllowOverride None
        Order allow,deny
        allow from all
  </Directory>
  <Directory "/usr/share/diskover-web/vendor/">
        Options +Indexes +FollowSymLinks +MultiViews
        AllowOverride None
        Order allow,deny
        allow from all
  </Directory>
</VirtualHost>

What can I do to make diskover-web work in apache2?

Cheers,
Rene

"No diskover indices found in Elasticsearch" when testing via Docker

After running $ docker-compose build && docker-compose up per the readme, and navigating to localhost:8080, the app has a message that says No diskover indices found in Elasticsearch. Please run a crawl and come back. and needs me to specify Index and Index 2 before I can proceed. Elasticsearch and Redis appear to have been deployed successfully via Docker; do I need to POST to Elasticsearch directly at localhost:9200 to get it to do something, or should I be editing config inside the Docker container? It's not clear from the documentation.

streams & ringphp is abandoned

guzzlehttp/streams and guzzlehttp/ringphp is abandoned
you should avoid using it. No replacement was suggested from composer.

How to recreate --

`root@storage01-diskover:/usr/local/diskover # git clone https://github.com/shirosaidev/diskover-web.git
Cloning into 'diskover-web'...
remote: Enumerating objects: 183, done.
remote: Counting objects: 100% (183/183), done.
remote: Compressing objects: 100% (118/118), done.
remote: Total 3153 (delta 117), reused 102 (delta 54), pack-reused 2970
Receiving objects: 100% (3153/3153), 47.54 MiB | 18.02 MiB/s, done.
Resolving deltas: 100% (2302/2302), done.
root@storage01-diskover:/usr/local/diskover # cd diskover-web/
root@storage01-diskover:/usr/local/diskover/diskover-web # composer install
Do not run Composer as root/super user! See https://getcomposer.org/root for details
Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 5 installs, 0 updates, 0 removals

Installing react/promise (v2.7.1): Downloading (100%)
Installing guzzlehttp/streams (3.0.0): Downloading (100%)
Installing guzzlehttp/ringphp (1.1.1): Downloading (100%)
Installing psr/log (1.1.3): Downloading (100%)
Installing elasticsearch/elasticsearch (v5.5.0): Downloading (100%)
elasticsearch/elasticsearch suggests installing monolog/monolog (Allows for client-level logging and tracing)
Package guzzlehttp/streams is abandoned, you should avoid using it. No replacement was suggested.
Package guzzlehttp/ringphp is abandoned, you should avoid using it. No replacement was suggested.
Writing lock file
Generating autoload files
root@storage01-diskover:/usr/local/diskover/diskover-web # `

Packages installed
php72 php72-curl php72-extensions php72-composer php72-gd php72-json

Age of the files should be calculated with respect to the date they were indexed

In the analytics option the age of the files is shown with respect to the current date, I think it should be calculated with respect to the date they were indexed.
Thanks. Great App.

Hardlinks are counted more than once

(Not really sure wether or not this is for the diskover or web part)

When counting directory sizes hardlinks are counted twice or more.

It also affects the percentage indexed counter.

CSV output is mangled

Version: 1.5.0-rc16
Docker Container: https://github.com/scr512/docker-diskover-web/blob/master/Dockerfile
When exporting a query to CSV, delimiting breaks... both in Windows land with Excel and with Linux tools (cut)

Example:

$ head -15 diskover_export_file\ \(1\).csv | cut -d, -f 14
filesize
1855808923
1872143765
1619683671
1864039435
1857076267
1850683799
1857484136
8056344320
2018-09-15T09:30:40.856155
2018-09-15T09:30:40.865604
2018-09-15T09:30:40.878265
2018-09-15T09:30:40.893790
2018-09-15T09:30:40.097447
2018-09-15T09:30:40.107179

I clearly expect to see filesize, not a timedate stamp

Here is the first 15 lines without parsing:

dupe_md5,extension,owner,tag,last_change,indexing_date,group,inode,last_modified,last_access,hardlinks,filename,worker_name,filesize,filehash,path_parent,tag_custom
,fsdb,user02,,2018-02-21T14:13:34,2018-09-16T19:07:42.457574,partner,3261587137,2018-02-21T14:13:34,2018-02-21T14:13:34,1,transient0.tran.fsdb,hpc-admin0.11061,1855808923,2d1e78abdd97f1ba0c7423b26f9ddcf7,/my_company/scratch_space/engineering_division/partner/partner_technology/engineering_team/user/user02/sim/sim.raw,
,fsdb,user02,,2018-02-22T10:38:00,2018-09-16T19:07:42.316971,partner,3300252844,2018-02-22T10:38:00,2018-02-22T10:38:00,1,transient0.tran.fsdb,hpc-admin0.11061,1872143765,29e2a870aced3e73e42dab29d866e2d8,/my_company/scratch_space/engineering_division/partner/partner_technology/engineering_team/user/user02/wl_header_update/VDDPE_OPTIMIZATION/sim/sim.raw,
,fsdb,user02,,2018-02-22T18:07:08,2018-09-16T19:07:42.343690,partner,3317985758,2018-02-22T18:07:07,2018-02-22T18:07:07,1,transient0.tran.fsdb,hpc-admin0.11061,1619683671,280b295ec628826abc241554b2e1fdba,/my_company/scratch_space/engineering_division/partner/partner_technology/engineering_team/user/user02/wl_header_update/WDX4I_AST/CONLY/sim/sim.raw,
,fsdb,user02,,2018-02-22T18:41:35,2018-09-16T19:07:42.379715,partner,3322573322,2018-02-22T18:41:35,2018-02-22T18:41:35,1,transient0.tran.fsdb,hpc-admin0.11061,1864039435,5de9ed413df86ae1dc8e5f106c7aa81d,/my_company/scratch_space/engineering_division/partner/partner_technology/engineering_team/user/user02/wl_header_update/WDX4I_AST/Header_wd/sim/sim.raw,
,fsdb,user02,,2018-02-22T14:50:33,2018-09-16T19:07:42.420287,partner,3312210146,2018-02-22T14:50:33,2018-02-22T14:50:33,1,transient0.tran.fsdb,hpc-admin0.11061,1857076267,3cb2c2cc2298d491d05ce96c7959ffe5,/my_company/scratch_space/engineering_division/partner/partner_technology/engineering_team/user/user02/wl_header_update/WDX4I_AST/Reduce_footer/sim/sim.raw,
,fsdb,user02,,2018-02-22T14:41:14,2018-09-16T19:07:42.397875,partner,3297754556,2018-02-22T14:41:14,2018-02-22T14:41:14,1,transient0.tran.fsdb,hpc-admin0.11061,1850683799,4009b6cac7896686744a039c72cf432a,/my_company/scratch_space/engineering_division/partner/partner_technology/engineering_team/user/user02/wl_header_update/WDX4I_AST/sim/sim.raw,
,fsdb,user02,,2018-02-22T07:39:42,2018-09-16T19:07:42.438273,partner,3292589808,2018-02-22T07:39:42,2018-02-22T07:39:42,1,transient0.tran.fsdb,hpc-admin0.11061,1857484136,47839a89213ccdbb601e24e38e6b5505,/my_company/scratch_space/engineering_division/partner/partner_technology/engineering_team/user/user02/wl_header_update/sim/sim.raw,
,fsdb,user01,,2018-03-06T21:04:22,2018-09-15T09:35:22.213673,partner,3623915697,2018-02-28T23:15:28,2018-02-28T23:15:28,1,c0.fsdb,hpc-admin0.11061,8056344320,d0707d6d0fc8c281c07f78fe653952b9,/my_company/scratch_space/engineering_division/partner/partner_technology/engineering_team/user/user01/r7/custom_tool/simulation/timing/tt_0p75v_0p75v_85cUHS_FCI/i128X84m2fb1fs2w/full/setup_CLK_r_EMA_r/control,
hpc-admin0.11051,fsdb,2269263876,partner,5508efa3bc5e8e57c44db141cb3f3293,,2018-02-14T05:23:14,2018-02-15T21:28:00,user01,,,1,c0.fsdb,2018-09-15T09:30:40.856155,/my_company/scratch_space/engineering_division/partner/partner_technology/engineering_team/user/user01/partner_partner_technologymy_company_sw_tool/custom_tool/simulation/timing/tt_0p75v_0p75v_85cUHS_FCI/i128X16m2fb1fs2w/full/delay_CLK_r_Q_r/control,2805513528,2018-02-14T05:23:14
hpc-admin0.11051,fsdb,2972766162,partner,e8216289179972638df34f3c7e72fbc5,,2018-02-16T06:49:14,2018-02-16T06:49:14,user01,,,1,c0_new4.fsdb,2018-09-15T09:30:40.865604,/my_company/scratch_space/engineering_division/partner/partner_technology/engineering_team/user/user01/partner_partner_technologymy_company_sw_tool/custom_tool/simulation/timing/tt_0p75v_0p75v_85cUHS_FCI/i128X16m2fb1fs2w/full/delay_CLK_r_Q_r/control/ema,2947038418,2018-02-16T06:49:14
hpc-admin0.11051,fsdb,3162196612,partner,769c05ab6c33f016d5275f13dd063783,,2018-02-16T06:51:37,2018-02-16T06:51:37,user01,,,1,c0_new4.fsdb,2018-09-15T09:30:40.878265,/my_company/scratch_space/engineering_division/partner/partner_technology/engineering_team/user/user01/partner_partner_technologymy_company_sw_tool/custom_tool/simulation/timing/tt_0p75v_0p75v_85cUHS_FCI/i128X16m2fb1fs2w/full/delay_CLK_r_Q_r/control/ema/new,2947824310,2018-02-16T06:51:37
hpc-admin0.11051,fsdb,2405317631,partner,612210b3e671b9bf9459641eb291d453,,2018-02-14T05:30:44,2018-02-15T21:28:00,user01,,,1,c0.fsdb,2018-09-15T09:30:40.893790,/my_company/scratch_space/engineering_division/partner/partner_technology/engineering_team/user/user01/partner_partner_technologymy_company_sw_tool/custom_tool/simulation/timing/tt_0p75v_0p75v_85cUHS_FCI/i128X16m2fb1fs2w/full/delay_CLK_r_Q_r/control/new,2809553166,2018-02-14T05:30:44
hpc-admin0.11051,fsdb,6352033331,partner,420d734f5dc1d5fb43948f2db672b319,,2018-02-15T21:25:27,2018-02-15T21:27:59,user01,,,1,c0_new.fsdb,2018-09-15T09:30:40.097447,/my_company/scratch_space/engineering_division/partner/partner_technology/engineering_team/user/user01/partner_partner_technologymy_company_sw_tool/custom_tool/simulation/timing/tt_0p75v_0p75v_85cUHS_FCI/i128X60m2fb1fs2w/full/delay_CLK_r_Q_r/control,2908678752,2018-02-15T21:25:27
hpc-admin0.11051,fsdb,6762929436,partner,50b46f2c2fbf702b1f8a8c10c6f55c07,,2018-02-15T21:34:20,2018-02-15T21:34:20,user01,,,1,c0_new.fsdb,2018-09-15T09:30:40.107179,/my_company/scratch_space/engineering_division/partner/partner_technology/engineering_team/user/user01/partner_partner_technologymy_company_sw_tool/custom_tool/simulation/timing/tt_0p75v_0p75v_85cUHS_FCI/i128X60m2fb1fs2w/full/delay_CLK_r_Q_r/control/new,2918391401,2018-02-15T21:34:20

From just glancing at the data, it appears to be an issue with empty fields. For example, I'm not using custom tags nor dedupe.

Blank values should be

,, or , ,

Works fine with JSON:

$ cat diskover_export_file.json | python -mjson.tool | grep '"filesize":' | head -15
        "filesize": 1855808923, 
        "filesize": 1872143765, 
        "filesize": 1619683671, 
        "filesize": 1864039435, 
        "filesize": 1857076267, 
        "filesize": 1850683799, 
        "filesize": 1857484136, 
        "filesize": 8056344320, 
        "filesize": 2269263876, 
        "filesize": 2972766162, 
        "filesize": 3162196612, 
        "filesize": 2405317631, 
        "filesize": 6352033331, 
        "filesize": 6762929436, 
        "filesize": 3202411778,

Running On Apache

Can i get some general guidance on how best to setup diskover-web? As recommended im trying to avoid using the php test method (Which works so i know that elasticsearch is running). I have tried both Apache and nginx. I get a login page asking for credentials but get the following error in the Apache logs after login:

PHP Fatal error: Uncaught Elasticsearch\Common\Exceptions\NoNodesAvailableException: No alive nodes found in your cluster in /var/www/vendor/elasticsearch/elasticsearch/src/Elasticsearch/ConnectionPool/StaticNoPingConnectionPool.php:51\nStack trace:\n#0 /var/www/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Transport.php(72): Elasticsearch\ConnectionPool\StaticNoPingConnectionPool->nextConnection()\n#1 /var/www/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Transport.php(90): Elasticsearch\Transport->getConnection()\n#2 /var/www/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php(240): Elasticsearch\Transport->performRequest('HEAD', '', Array, NULL, Array)\n#3 /var/www/vendor/react/promise/src/FulfilledPromise.php(25): Elasticsearch\Connections\Connection->Elasticsearch\Connections\{closure}(Array)\n#4 /var/www/vendor/guzzlehttp/ringphp/src/Future/CompletedFutureValue.php(55): React\Promise\FulfilledPromise->then(Object(Closure), NULL, NULL)\n#5 /var/www/vendor/guzzlehttp/ri in /var/www/vendor/elasticsearch/elasticsearch/src/Elasticsearch/ConnectionPool/StaticNoPingConnectionPool.php on line 51

Create a docker image for this project or a docker-compose config

Cool project!

A docker image would allow people to spin this up quickly.
A docker-compose config would also allow separate containers for each role, eg nginx vs elasticsearch.
You could use the official Elastic image for the Elasticsearch too.

Connecting diskover-web to AWS ElasticSearch

When trying to connect diskover-web to an AWS elasticsaerch cluster I am getting the error messages below.

In 'Constants.php' I've set the ES_HOST to my VPC endpoint, and can successfully get a curl request back from the endpoint from the instance (to rule out security group issues). ES_PORT has been set to 443.

I also have AWS = TRUE

Am I missing something simple?

PHP Fatal error:  Uncaught Elasticsearch\Common\Exceptions\NoNodesAvailableException: No alive nodes found in your cluster in /home/ubuntu/diskover-web/vendor/elasticsearch/elasticsearch/src/Elasticsearch/ConnectionPool/StaticNoPingConnectionPool.php:51
Stack trace:
#0 /home/ubuntu/diskover-web/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Transport.php(72): Elasticsearch\ConnectionPool\StaticNoPingConnectionPool->nextConnection()
#1 /home/ubuntu/diskover-web/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Transport.php(90): Elasticsearch\Transport->getConnection()
#2 /home/ubuntu/diskover-web/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php(240): Elasticsearch\Transport->performRequest('HEAD', '/', Array, NULL, Array)
#3 /home/ubuntu/diskover-web/vendor/react/promise/src/FulfilledPromise.php(25): Elasticsearch\Connections\Connection->Elasticsearch\Connections\{closure}(Array)
#4 /home/ubuntu/diskover-web/vendor/guzzlehttp/ringphp/src/Future/CompletedFutureValue.php(55): React\Promise in /home/ubuntu/diskover-web/vendor/elasticsearch/elasticsearch/src/Elasticsearch/ConnectionPool/StaticNoPingConnectionPool.php on line 51

The diskover bot/crawler can communicate with ElasticSearch, it is only the web portion having trouble.

diskover-web shows all indicies

when I query the following link
http://url/index.php?index=&index2=
diskover-web will start summing up all inidicies in the database
it also choses a random path and so on

shirosaidev / diskover-web Goto Github PK

diskover-web's People

Contributors

Stargazers

Watchers

Forkers

diskover-web's Issues

Recommend Projects

Recommend Topics

Recommend Org