Comments (12)
@sonu27 in the mean time, I have been working around this issue by setting the NEO4J_dbms_memory_heap_initial__size
and NEO4J_dbms_memory_heap_max__size
environment variables in my docker run
command:
docker run -d \
--env NEO4J_dbms_allow__format__migration=true \
--env NEO4J_dbms_memory_heap_initial__size=1G \
--env NEO4J_dbms_memory_heap_max__size=1G \
--env NEO4J_dbms_security_auth__enabled=false \
--name neo4j \
--publish 7474:7474 \
--publish 7687:7687 \
--restart=always \
--volume /ebs/xvdi/data:/data \
--volume /ebs/xvdi/logs:/logs \
neo4j:3
from docker-neo4j.
Updated instructions can be viewed here
https://neo4j.com/docs/operations-manual/current/installation/docker/#docker-conf-volume
from docker-neo4j.
@jicksta I'm sorry that you've had trouble with this behaviour. I can see that it's surprising if you are deriving your own image from the standard one and copying in config files -- we hadn't considered that case when designing the configuration mechanism.
The intended way of providing your own complete config files to the image is via a /conf
volume. If you were to add config files to /conf
in your derived image, you would find that they don't get modified and you won't need to define the environment variables.
Neo4j itself does not have any way of injecting configuration via environment variables, so the only way that the Docker image can provide this is by modifying the config files. We hope to enhance Neo4j so that it can take config from the environment natively, but we won't be able to do that for 3.1.
The environment variables that the image responds to are documented here. You'll see that the only ones we provide default overrides for are the memory-related settings. This is because Neo4j's default behaviour is tuned for a server deployment where Neo4j is the only resource-hungry software running on the server; so by default Neo4j will consume the majority of the available memory. We consider that to be inappropriate for container deployment, so we've limited the default memory usage.
On the specific question of initial vs maximum heap size, our standard recommendation for Neo4j is to configure them to be the same. This gives much more predictable memory consumption, avoiding problems where servers run out of memory under load due to individual queries. We decided to bake this recommendation into the Docker image to simplify things for operators.
from docker-neo4j.
The intended way of providing your own complete config files to the image is via a /conf volume. If you were to add config files to /conf in your derived image, you would find that they don't get modified and you won't need to define the environment variables.
I'll give this a try. I should add tho that AFAIK it's pretty common to COPY
config files into a derivative image. Volumes can bring their own hassles, so I'm not sure why having a volume for just /conf
would be helpful to our particular use case.
Also thanks for the link the to documentation about the env variables.
On the specific question of initial vs maximum heap size, our standard recommendation for Neo4j is to configure them to be the same. This gives much more predictable memory consumption, avoiding problems where servers run out of memory under load due to individual queries. We decided to bake this recommendation into the Docker image to simplify things for operators.
Consider that most folks using dockerized Neo4j in production would probably have Neo4j wired up into a local Docker-Compose (or similar) system running on a local development environment. We want to set the initial memory size to be something smaller so we can have a Neo4j instance running in the local devmode cluster that doesn't suck up precious laptop memory resources unless necessary.
from docker-neo4j.
@jicksta The /conf
volume is intended for direct users of the standard Neo4j image, not for people extending the image. I only mention it because it gives you a simpler workaround for your problem.
from docker-neo4j.
@jicksta I take your point about Compose etc. However we think that simplifying the surface and removing the possibility for nasty surprises in production is more important in this case.
from docker-neo4j.
@benbc I am still having trouble with this, despite using a /conf
volume as directed. This is how I'm running Neo4j:
docker run -d \
--publish 7474:7474 \
--publish 7687:7687 \
--restart=always \
--volume /ebs/xvdi/conf:/conf \
--volume /ebs/xvdi/data:/data \
--volume /ebs/xvdi/logs:/logs \
neo4j
In /ebs/xvdi/conf/neo4j.conf
I have uncommented and modified these lines:
dbms.memory.heap.initial_size=1G
dbms.memory.heap.max_size=1G
But Neo4j is only running with 512M, judging by the output of docker top
:
$ sudo docker top eager_swartz
UID PID PPID C STIME TTY TIME CMD
root 1731 1714 1 Oct24 ? 00:00:26 /usr/lib/jvm/java-1.8-openjdk/jre/bin/java -cp /var/lib/neo4j/plugins:/var/lib/neo4j/conf:/var/lib/neo4j/lib/*:/var/lib/neo4j/plugins/* -server -Xms512M -Xmx512M -XX:+UseG1GC -XX:-OmitStackTraceInFastThrow -XX:+AlwaysPreTouch -XX:+UnlockExperimentalVMOptions -XX:+TrustFinalNonStaticFields -XX:+DisableExplicitGC -Djdk.tls.ephemeralDHKeySize=2048 -Dunsupported.dbms.udc.source=tarball -Dfile.encoding=UTF-8 org.neo4j.server.CommunityEntryPoint --home-dir=/var/lib/neo4j --config-dir=/var/lib/neo4j/conf
This is with the official 3.2.5 CE image.
from docker-neo4j.
I did some more digging in my running container.
It looks like docker-entrypoint.sh
makes a copy of /conf/neo4j.conf
and overwrites my un-commented lines.
docker-neo4j/src/3.2/docker-entrypoint.sh
Lines 93 to 106 in a846084
This directly contradicts the documentation, which states:
Any configuration files in the /conf volume will override files provided by the image. This includes values that may have been set in response to environment variables passed to the container by Docker.
Please advise.
from docker-neo4j.
@spacecowboy @srbaker π (sorry, I'm not sure who the right people are to ping)
from docker-neo4j.
Sorry you're having trouble with this @adamrothman. We'll take a look and get back to you.
from docker-neo4j.
Any update on this?
from docker-neo4j.
Hi @adamrothman Apologies that it took a while to reply to you. We think the documentation is at fault here. It should instead say
Any configuration files in the
/conf
volume will override files provided by the image. Environment variables passed to the container by Docker will still override the values in configuration files in/conf
volume
With the updated documentation, your work around will be the correct way to achieve what you want.
from docker-neo4j.
Related Issues (20)
- Missing docker-entrypoint.sh in latest image? HOT 2
- Error using or configuring APOC in GithubAction HOT 1
- Why am I encountering an issue with the GDS library on a Neo4j Docker container on Windows, despite it working on Linux? HOT 5
- Version 4.4.19 fails when installing APOC HOT 3
- Missing proper health check
- chown: changing ownership of '/var/lib/neo4j/conf/neo4j.conf': Read-only file system HOT 3
- Volume inaccessible, although uid and gid match those of the user parameter HOT 2
- my_ip:7474 >> This site canβt be reached HOT 3
- Pods unready/CrashLoopBackOff with message "sed: cannot rename /var/lib/neo4j/conf/sed8pylkX: Operation not permitted" HOT 2
- neo4j-admin:5.10.0 not released on Docker Hub HOT 2
- Unable to mount local config folder with apoc.conf HOT 2
- ERROR: No compatible "graph-data-science" plugin found for Neo4j 5.13.0. HOT 8
- neo4j-admin:5.17.0 not released on Docker Hub HOT 1
- Docker bind only 192.168.58.110 address and not all interfaces HOT 1
- Plugins for "genai","n10s","graphql","graph-algorithms" missing after docker install neo4j:5.18 HOT 3
- Critical/High Severity issues reported by Snyk for neo4j:5.18.1
- conf folder not persisted
- openshift support with "random UID"
- neo4j-admin:5.20.0 not released on Docker Hub HOT 1
- Error when dumping database neo4j-admin Docker HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from docker-neo4j.