clickhouse is fail to start about clickhouse-backup HOT 20 CLOSED

issue is not related to clickhouse-backup
your clickhouse-server is not running

look to something like that journalctl -xue --no-pager clickhouse-server.service

from clickhouse-backup.

also look to
/var/log/clickhouse-server/clickhouse-server.err.log

from clickhouse-backup.

(version 21.10.2.15 (official build))
2024.05.02 10:48:01.266771 [ 2462 ] {} Application: Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: 172.31.42.152:2181
Poco::Exception. Code: 1000, e.code() = 0, Timeout: connect timed out: 172.31.42.152:2181 (version 21.10.2.15 (official build)), 172.31.42.152:2181
Poco::Exception. Code: 1000, e.code() = 0, Timeout: connect timed out: 172.31.42.152:2181 (version 21.10.2.15 (official build)), 172.31.42.152:2181
Poco::Exception. Code: 1000, e.code() = 0, Timeout: connect timed out: 172.31.42.152:2181 (version 21.10.2.15 (official build)), 172.31.42.152:2181
(Connection loss): Cannot attach table default.test_user_local from metadata file /var/lib/clickhouse/store/889/889aad3d-55fa-499d-889a-ad3d55fa299d/test_user_local.sql from query ATTACH TABLE default.test_user_local UUID 'b35cdd81-6f1d-4407-b35c-dd816f1da407' (tenant_id String, username String, user_id String, email String, joining_date Date) ENGINE = ReplicatedMergeTree('/clickhouse/all_replicated/xxxxx_test/tables/test_user_local/shard1', 'replica') PARTITION BY toYYYYMM(joining_date) ORDER BY username SETTINGS index_granularity = 8192: while loading database default from path /var/lib/clickhouse/metadata/default

from clickhouse-backup.

-- The start-up result is done.
May 02 10:49:40 ip-172-31-54-11.us-west-2.compute.internal clickhouse-server[3055]: Processing configuration file '/etc/clickhouse-server/config.xml'.
May 02 10:49:40 ip-172-31-54-11.us-west-2.compute.internal clickhouse-server[3055]: Merging configuration file '/etc/clickhouse-server/config.d/cluster.xml'.
May 02 10:49:40 ip-172-31-54-11.us-west-2.compute.internal clickhouse-server[3055]: Merging configuration file '/etc/clickhouse-server/config.d/config.xml'.
May 02 10:49:40 ip-172-31-54-11.us-west-2.compute.internal clickhouse-server[3055]: Merging configuration file '/etc/clickhouse-server/config.d/macros.xml'.
May 02 10:49:40 ip-172-31-54-11.us-west-2.compute.internal clickhouse-server[3055]: Merging configuration file '/etc/clickhouse-server/config.d/zookeeper.xml'.
May 02 10:49:40 ip-172-31-54-11.us-west-2.compute.internal clickhouse-server[3055]: Logging trace to /var/log/clickhouse-server/clickhouse-server.log
May 02 10:49:40 ip-172-31-54-11.us-west-2.compute.internal clickhouse-server[3055]: Logging errors to /var/log/clickhouse-server/clickhouse-server.err.log
May 02 10:49:41 ip-172-31-54-11.us-west-2.compute.internal clickhouse-server[3055]: Processing configuration file '/etc/clickhouse-server/config.xml'.
May 02 10:49:41 ip-172-31-54-11.us-west-2.compute.internal clickhouse-server[3055]: Merging configuration file '/etc/clickhouse-server/config.d/cluster.xml'.
May 02 10:49:41 ip-172-31-54-11.us-west-2.compute.internal clickhouse-server[3055]: Merging configuration file '/etc/clickhouse-server/config.d/config.xml'.
May 02 10:49:41 ip-172-31-54-11.us-west-2.compute.internal clickhouse-server[3055]: Merging configuration file '/etc/clickhouse-server/config.d/macros.xml'.
May 02 10:49:41 ip-172-31-54-11.us-west-2.compute.internal clickhouse-server[3055]: Merging configuration file '/etc/clickhouse-server/config.d/zookeeper.xml'.
May 02 10:49:41 ip-172-31-54-11.us-west-2.compute.internal clickhouse-server[3055]: Saved preprocessed configuration to '/var/lib/clickhouse/preprocessed_configs/config.xml'.
May 02 10:49:41 ip-172-31-54-11.us-west-2.compute.internal clickhouse-server[3055]: Processing configuration file '/etc/clickhouse-server/users.xml'.
May 02 10:49:41 ip-172-31-54-11.us-west-2.compute.internal clickhouse-server[3055]: Saved preprocessed configuration to '/var/lib/clickhouse/preprocessed_configs/users.xml'.
May 02 10:49:45 ip-172-31-54-11.us-west-2.compute.internal systemd[1]: clickhouse-server.service: main process exited, code=exited, status=70/n/a
May 02 10:49:45 ip-172-31-54-11.us-west-2.compute.internal systemd[1]: Unit clickhouse-server.service entered failed state.
May 02 10:49:45 ip-172-31-54-11.us-west-2.compute.internal systemd[1]: clickhouse-server.service failed.

from clickhouse-backup.

run your zookeeper / keeper on 172.31.42.152:2181

from clickhouse-backup.

clickhouse-server cant' attach ReplicatedMergeTree tables cause can't connect to zookeeper
look section in your configs

from clickhouse-backup.

one I increased only disk - but why change these things I am not under stand.

from clickhouse-backup.

172.31.42.152:2181
is this IP of your server?
is this the same server with clickhouse-server?
if yes
what show
systemctl status zookeeper.service ?

from clickhouse-backup.

is this the same server with clickhouse-server

above IP server is different
and this clickhouse server is different
here onlynodejs user only. its not user also I think.

from clickhouse-backup.

● zookeeper.service - Zookeeper Service
Loaded: loaded (/usr/lib/systemd/system/zookeeper.service; enabled; vendor preset: disabled)
Active: active (running) since Thu 2024-05-02 09:13:56 UTC; 2h 11min ago
Main PID: 3089 (java)
CGroup: /system.slice/zookeeper.service
└─3089 java -Dzookeeper.log.dir=/opt/zookeeper/apache-zookeeper-3.7.0-bin/bin/../logs -Dzookeeper.log.file=zookeeper-zookeeper-server-ip-172-31-54-11.us-west-2.compute.interna...

May 02 09:13:56 ip-172-31-54-11.us-west-2.compute.internal systemd[1]: Started Zookeeper Service.
May 02 09:13:57 ip-172-31-54-11.us-west-2.compute.internal zookeeper[2810]: /usr/bin/java
May 02 09:13:57 ip-172-31-54-11.us-west-2.compute.internal zookeeper[2810]: ZooKeeper JMX enabled by default
May 02 09:13:57 ip-172-31-54-11.us-west-2.compute.internal zookeeper[2810]: Using config: /opt/zookeeper/apache-zookeeper-3.7.0-bin/bin/../conf/zoo.cfg

from clickhouse-backup.

2024.05.02 11:36:08.795947 [ 13169 ] {} Application: starting up
2024.05.02 11:36:08.795968 [ 13169 ] {} Application: OS name: Linux, version: 3.10.0-957.10.1.el7.x86_64, architecture: x86_64
2024.05.02 11:36:08.994423 [ 13169 ] {} Application: Calculated checksum of the binary: 902F2F159F07FEEC02834AEF44CBFB70, integrity check passed.
2024.05.02 11:36:08.994692 [ 13169 ] {} Application: Will do mlock to prevent executable memory from being paged out. It may take a few seconds.
2024.05.02 11:36:09.010801 [ 13169 ] {} Application: The memory map of clickhouse executable has been mlock'ed, total 211.02 MiB
2024.05.02 11:36:09.011213 [ 13169 ] {} Application: DB::Exception: Effective user of the process (root) does not match the owner of the data (clickhouse). Run under 'sudo -u clickhouse'.
2024.05.02 11:36:09.011241 [ 13169 ] {} Application: shutting down
2024.05.02 11:36:09.011251 [ 13169 ] {} Application: Uninitializing subsystem: Logging Subsystem
2024.05.02 11:36:09.011347 [ 13170 ] {} BaseDaemon: Received signal -2
2024.05.02 11:36:09.011399 [ 13170 ] {} BaseDaemon: Stop SignalListener thread

from clickhouse-backup.

2024.05.02 11:36:09.011213 [ 13169 ] {} Application: DB::Exception: Effective user of the process (root) does not match the owner of the data (clickhouse). Run under 'sudo -u clickhouse'.

this is part of clickhouse-server logs
how did you exactly try to run it?

use
sudo systemctl start clickhouse-server.service

from clickhouse-backup.

sudo systemctl start clickhouse-server.service

from clickhouse-backup.

telnet 172.31.42.152 2181

from clickhouse-backup.

Loaded: loaded (/usr/lib/systemd/system/zookeeper.service; enabled; vendor preset: disabled)
Active: active (running) since Thu 2024-05-02 09:13:56 UTC; 2h 11min ago

is this result of systemctl status from 172.31.42.152? or from the server where run clickhouse-server?

from clickhouse-backup.

telnet 172.31.42.152 2181

from click house server

from clickhouse-backup.

can you login to ssh 172.31.42.152

Do you know what is 172.31.42.152 address?

check your clickhouse zookeeper settings

grep -C 10 zookeeper -r /etc/clickhouse-server/

is these settings still actual?

from clickhouse-backup.

grep -C 10 zookeeper -r /etc/clickhouse-server/

in both servers

from clickhouse-backup.

check your AWS console, and ask your devops to check network policies which restricted access to your zookeeper server

after change EC2 instance type you could change your internal IP address

could you please stop write here, i'm not free AWS support, your troubles is unrelated to clickhouse-backup, completelly

from clickhouse-backup.

ge EC2 instance type you could change your internal IP address

private ips right no need to change.

ok

from clickhouse-backup.