Giter Club home page Giter Club logo

Comments (18)

Torah avatar Torah commented on May 30, 2024 3

经过人因工程学和作者联系上了。这个问题很可能可以通过如下方式解决(我这里是这样解决的，也可以查看log去查找问题的解决办法https://www.cnblogs.com/yehui/p/9087845.html)：
sudo nano /etc/sysctl.conf //修改参数
在最后添加vm.max_map_count=262144
sudo reboot
cat /proc/sys/vm/max_map_count再看一眼
重新build

from house-renting.

Paulswith avatar Paulswith commented on May 30, 2024 1

经过人因工程学和作者联系上了。这个问题很可能可以通过如下方式解决(我这里是这样解决的，也可以查看log去查找问题的解决办法https://www.cnblogs.com/yehui/p/9087845.html)：%EF%BC%9A)
sudo nano /etc/sysctl.conf //修改参数
在最后添加vm.max_map_count=262144
sudo reboot
cat /proc/sys/vm/max_map_count再看一眼
重新build

这个sudo sysctl -p 就可以生效， 记得还原ELASTICSEARCH_URL=http://elastic:9200， 谢谢

from house-renting.

kezhenxu94 avatar kezhenxu94 commented on May 30, 2024

@BadGrem 你是使用docker-compose一次性启动的es和kibana的，还是在自己机器运行es？

from house-renting.

954-Ivory avatar 954-Ivory commented on May 30, 2024

我用的是docker-compose
ubuntu 18.04
docker-compose version 1.17.1, build unknown
docker-py version: 2.5.1
CPython version: 2.7.15rc1
OpenSSL version: OpenSSL 1.1.0g 2 Nov 2017

from house-renting.

kezhenxu94 avatar kezhenxu94 commented on May 30, 2024

@BadGrem 你可以尝试将

    kibana:
        image: docker.elastic.co/kibana/kibana-oss:6.2.4
        container_name: kibana
        networks:
            - localhost
        ports:
            - 5601:5601
        environment:
            - ELASTICSEARCH_URL=http://elastic:9200
        depends_on:
            - elastic

其中的 http://elastic:9200 换成 http://127.0.0.1:9200

from house-renting.

954-Ivory avatar 954-Ivory commented on May 30, 2024

Unable to connect to Elasticsearch at http://127.0.0.1:9200.

from house-renting.

kezhenxu94 avatar kezhenxu94 commented on May 30, 2024

@BadGrem 使用docker logs elastic 看看es是否启动成功

from house-renting.

954-Ivory avatar 954-Ivory commented on May 30, 2024 
$ docker logs elastic
Error: No such container: elastic

$ docker logs elasticsearch
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
[2018-07-30T17:59:31,254][INFO ][o.e.n.Node               ] [] initializing ...
[2018-07-30T17:59:31,350][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: Failed to create node environment
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:125) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.2.4.jar:6.2.4]        at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.2.4.jar:6.2.4]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:85) ~[elasticsearch-6.2.4.jar:6.2.4]
Caused by: java.lang.IllegalStateException: Failed to create node environment
        at org.elasticsearch.node.Node.<init>(Node.java:267) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.node.Node.<init>(Node.java:246) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:213) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:213) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:323) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-6.2.4.jar:6.2.4]
        ... 6 more
Caused by: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/nodes
        at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) ~[?:?]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:?]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:?]
        at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:384) ~[?:?]
        at java.nio.file.Files.createDirectory(Files.java:674) ~[?:1.8.0_161]
        at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781) ~[?:1.8.0_161]
        at java.nio.file.Files.createDirectories(Files.java:767) ~[?:1.8.0_161]
        at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:204) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.node.Node.<init>(Node.java:264) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.node.Node.<init>(Node.java:246) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:213) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:213) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:323) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-6.2.4.jar:6.2.4]
        ... 6 more

我又尝试了在Windows下利用docker-toolbox尝试配置，仍然是报一样的错误：
Unable to connect to Elasticsearch at http://elastic:9200.

from house-renting.

954-Ivory avatar 954-Ivory commented on May 30, 2024 
$ docker-compose up --build -d
Creating network "houserentingmaster_localhost" with the default driver
Building crawler
Step 1/6 : FROM python:3
 ---> 638817465c7d
Step 2/6 : MAINTAINER kezhenxu94 <[email protected]>
 ---> Using cache
 ---> 6db9c8ea5370
Step 3/6 : COPY . /house-renting/crawler
 ---> Using cache
 ---> d2f157adee46
Step 4/6 : VOLUME /etc/scrapyd/ /var/lib/scrapyd/
 ---> Using cache
 ---> 4b9f106328a9
Step 5/6 : WORKDIR /house-renting/crawler
 ---> Using cache
 ---> 22d293e7c8d8
Step 6/6 : RUN pip install -i https://pypi.tuna.tsinghua.edu.cn/simple -r requirements.txt
 ---> Using cache
 ---> 3d551172bac2
Successfully built 3d551172bac2
Successfully tagged house-renting/crawler:latest
Building scrapyd
Step 1/6 : FROM python:3
 ---> 638817465c7d
Step 2/6 : MAINTAINER kezhenxu94 <[email protected]>
 ---> Using cache
 ---> 6db9c8ea5370
Step 3/6 : VOLUME /etc/scrapyd/ /var/lib/scrapyd/
 ---> Using cache
 ---> 06d040cbf51e
Step 4/6 : COPY ./scrapyd.conf /etc/scrapyd/
 ---> Using cache
 ---> c41ae1de321b
Step 5/6 : RUN pip install -i https://pypi.tuna.tsinghua.edu.cn/simple scrapyd
 ---> Using cache
 ---> e982fa7f2eb5
Step 6/6 : ENTRYPOINT ["scrapyd"]
 ---> Using cache
 ---> 81132e097275
Successfully built 81132e097275
Successfully tagged house-renting/scrapyd:latest
Creating redis         ... done
Creating elasticsearch ... done
Creating crawler       ... done
Creating kibana        ... done
Creating scrapyd       ... done
Creating 58            ... done
Creating douban        ... done
Creating lianjia       ... done

看起来好像没啥问题...

qq 20180731030221

from house-renting.

954-Ivory avatar 954-Ivory commented on May 30, 2024

我尝试将image换成下面这两个

image: elasticsearch
image: kibana

在 index pattern 中出现了问题 Unable to fetch mapping. Do you have indices matching the pattern?

所以我又将docker-compose.yml换成84374c5版本，但是58同城的信息貌似没有被爬取到。

from house-renting.

kezhenxu94 avatar kezhenxu94 commented on May 30, 2024

@BadGrem Unable to fetch mapping. Do you have indices matching the pattern?
是因为没有爬到数据入 es，如果没有爬到，可能是反爬机制升级，我再看看，你有空也可以帮忙看看🤝

from house-renting.

hao-lee avatar hao-lee commented on May 30, 2024

你这是权限问题啊老哥，你把源码目录下的 data/elastic/ 改成权限 777 就好了。我也遇到了这个问题，参考这个问题下的评论才解决。

from house-renting.

smelike avatar smelike commented on May 30, 2024

你这是权限问题啊老哥，你把源码目录下的 data/elastic/ 改成权限 777 就好了。我也遇到了这个问题，参考这个问题下的评论才解决。

改了 777，也不行

from house-renting.

hao-lee avatar hao-lee commented on May 30, 2024

@smelike 重新 build 呢

from house-renting.

Torah avatar Torah commented on May 30, 2024

@smelike 重新 build 呢

也是不行。作者应该给一下微信号，那个微信和支付宝大赏转了帐，但是号码显示不全，联系不到作者。这个系统的价值在于elasticsearch，年初我看到过类似的项目，本来想改一下，但是因为是java，不是特别擅长。现在这个作为字段查找不错，但是这个elastic问题不解决就没法正常挂起es，这是这个系统的核心。

from house-renting.

MangoMinLi avatar MangoMinLi commented on May 30, 2024

把源码目录下的 data/elastic/ 改成权限 777，重新build，在重新启动，可以解决这个问题。谢谢@hao-lee

from house-renting.

hao-lee avatar hao-lee commented on May 30, 2024

@MangoMinLi 😏
作者已经把这个 issue 置顶了 😄

from house-renting.

jiangchao1987 avatar jiangchao1987 commented on May 30, 2024

可以用single-node模式，参考elastic/elasticsearch#45815

elastic:
  environment:
    - discovery.type=single-node

from house-renting.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.