michalmiklas / atlas-docker Goto Github PK

Hi,

Thanks for trying to get this running on Docker, it has helped heaps previously :) I was wondering if you have seen the below issue and know how to resolve it? It seems whenever I use a docker compose script I get the following error:

The container shows starting atlas then errors in main? Any help would be greatly appreciated.

PS, I have tested with v2.0.0, v1.1.0, v1.0.0 and they all seem to fail in a compose script (varying errors). The compose script has the following services:

• atlas
• zookeeper
• nifi
• kafka
• spark
• python
• nginx

services:
    atlas:
        image: mmiklas/atlas-docker:v2.0.0
        ports:
            - 21000:21000

atlas_1            | configured for local solr.
atlas_1            | Cassandra embedded configured.
atlas_1            | zookeeper started.
atlas_1            | solr started.
atlas_1            | setting up solr collections...
atlas_1            | starting atlas on host localhost
atlas_1            | starting atlas on port 21000
atlas_1            | .....................................
atlas_1            | Apache Atlas Server started!!!
atlas_1            | 
atlas_1            |    at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:520)
atlas_1            |    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
atlas_1            |    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
atlas_1            |    at org.eclipse.jetty.server.Server.start(Server.java:422)
atlas_1            |    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105)
atlas_1            |    at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
atlas_1            |    at org.eclipse.jetty.server.Server.doStart(Server.java:389)
atlas_1            |    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
atlas_1            |    at org.apache.atlas.web.service.EmbeddedServer.start(EmbeddedServer.java:98)
atlas_1            |    at org.apache.atlas.Atlas.main(Atlas.java:133)

Hi @michalmiklas thanks for the great docker image, started up and saw UI very quickly. I am attempting to get this atlas docker image (running locally) to bring in the metadata of topics from a kafka cluster (running locally). Not having much luck with the "import-kafka.sh"

bash-4.4# ./import-kafka.sh -t "topicname"

./import-kafka.sh
/apache-atlas-1.0.0
Could not find a valid KAFKA configuration

Any suggestions? Thanks in advance.

Hi @michalmiklas,

Thanks for sharing this atlas-docker. Great way to start exploring the apache project. I did encounter some hick-ups when running the docker so I wanted to share this.

When running the docker, you're calling atlas_start.py. However, going to localhost:21000 doesn't give access to atlas. Manually stopping and starting atlas given the python files does seem to solve the problem for me.

To play around with atlas, I've launched the quick_start.py script. Half-way the script I get a connection refused error.

bash-4.4# python quick_start.py
log4j:WARN No such property [maxFileSize] in org.apache.log4j.PatternLayout.
log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.PatternLayout.
log4j:WARN No such property [maxFileSize] in org.apache.log4j.PatternLayout.
log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.PatternLayout.
log4j:WARN No such property [maxFileSize] in org.apache.log4j.PatternLayout.
log4j:WARN No such property [maxFileSize] in org.apache.log4j.PatternLayout.
log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.PatternLayout.
Enter username for atlas :- admin
Enter password for atlas :-

Creating sample types:
Created type [DB]
Created type [Table]
Created type [StorageDesc]
Created type [Column]
Created type [LoadProcess]
Created type [View]
Created type [JdbcAccess]
Created type [ETL]
Created type [Metric]
Created type [PII]
Created type [Fact]
Created type [Dimension]
Created type [Log Data]
Created type [Table_DB]
Created type [View_DB]
Created type [View_Tables]
Created type [Table_Columns]
Created type [Table_StorageDesc]

Creating sample entities:
Created entity of type [DB], guid: a45aebef-9ae9-4c83-b443-9d0e5d1b2c2c
Created entity of type [DB], guid: a57f189b-b377-451f-b2cf-1367ea03a384
Created entity of type [DB], guid: d69ddce4-f511-470f-849a-1811e2e8cf6b
Created entity of type [Table], guid: ab292f01-890a-482e-bf4b-c0346ed3698d
Exception in thread "main" com.sun.jersey.api.client.ClientHandlerException: java.net.ConnectException: Connection refused (Connection refused)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155)
at com.sun.jersey.api.client.filter.HTTPBasicAuthFilter.handle(HTTPBasicAuthFilter.java:105)
at com.sun.jersey.api.client.Client.handle(Client.java:652)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:634)
at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:382)
at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:353)
at org.apache.atlas.AtlasBaseClient.callAPI(AtlasBaseClient.java:229)
at org.apache.atlas.AtlasClientV2.createEntity(AtlasClientV2.java:320)
at org.apache.atlas.examples.QuickStartV2.createInstance(QuickStartV2.java:387)
at org.apache.atlas.examples.QuickStartV2.createTable(QuickStartV2.java:495)
at org.apache.atlas.examples.QuickStartV2.createEntities(QuickStartV2.java:314)
at org.apache.atlas.examples.QuickStartV2.runQuickstart(QuickStartV2.java:166)
at org.apache.atlas.examples.QuickStartV2.main(QuickStartV2.java:147)
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:840)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1587)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:253)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:153)
... 14 more
No sample data added to Apache Atlas Server.

It makes the atlas server crash as I need to restart it. Re-running the quick-start script doesn't work as Atlas doesn't allow to create same objects twice or the script doesn't first clean up the environment before re-inserting the sample data.

A similar problem occurs when running quick-start_v1.

Exception in thread "main" org.apache.atlas.exception.AtlasBaseException: Failed to run QuickStart: customer_dim table entity creation failed
at org.apache.atlas.examples.QuickStart.table(QuickStart.java:380)
at org.apache.atlas.examples.QuickStart.createEntities(QuickStart.java:266)
at org.apache.atlas.examples.QuickStart.runQuickstart(QuickStart.java:93)
at org.apache.atlas.examples.QuickStart.main(QuickStart.java:74)
Caused by: com.sun.jersey.api.client.ClientHandlerException: java.net.ConnectException: Connection refused (Connection refused)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155)
at com.sun.jersey.api.client.filter.HTTPBasicAuthFilter.handle(HTTPBasicAuthFilter.java:105)
at com.sun.jersey.api.client.Client.handle(Client.java:652)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:634)
at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:382)
at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:353)
at org.apache.atlas.AtlasBaseClient.callAPI(AtlasBaseClient.java:229)
at org.apache.atlas.AtlasClient.callAPIWithBody(AtlasClient.java:906)
at org.apache.atlas.AtlasClient.createEntity(AtlasClient.java:406)
at org.apache.atlas.AtlasClient.createEntity(AtlasClient.java:424)
at org.apache.atlas.examples.QuickStart.createInstance(QuickStart.java:311)
at org.apache.atlas.examples.QuickStart.table(QuickStart.java:378)
... 3 more
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:840)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1587)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:253)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:153)
... 16 more
No data was added to the Apache Atlas Server.

Again it crashes the server and atlas_start.py needs to be executed.
Any idea how to resolve these issues?

• I noted you configured ATLAS_SERVER_HEAP as recommended: https://atlas.apache.org/InstallationSteps.html
• I noted they refer to set HBase to true, while your script sets it to false..

ENV MANAGE_LOCAL_HBASE=false
ENV MANAGE_LOCAL_SOLR=true

Any suggestions? Thanks for sharing!

Hi, @michalmiklas, I tried the latest v1.1.0 docker image. The atla-server was successfully started, but I couldn't open the web ui on localhost:21000.
Any replies will be appreciated!