/applications/postgres-docker isn't shared properly about redarc HOT 14 CLOSED

That error is from running this:

$ docker run \
  --name pgsql-dev \
  --network redarc \
  -e POSTGRES_PASSWORD=test1234 \
  -d \
  -v ${PWD}/postgres-docker:/var/lib/postgresql/data \
  -p 5432:5432 postgres 

The postgres container is trying to mount /applications/postgres-docker which isn't exported but you also shouldn't be running this in the /applications folder. Try setting the volume in /Users, /Volumes, /tmp or /private. Those paths seem to be exported by default according to https://stackoverflow.com/questions/45122459/mounts-denied-the-paths-are-not-shared-from-os-x-and-are-not-known-to-docke. Seems to be a mac related issue

eg:

$ docker run \
  --name pgsql-dev \
  --network redarc \
  -e POSTGRES_PASSWORD=test1234 \
  -d \
  -v /tmp/postgres-docker:/var/lib/postgresql/data \
  -p 5432:5432 postgres 

from redarc.

@Yakabuff Thanks for your response!

With the command above, I was able to run docker!
However, while I build 'redarc' using docker build . -t redarc , I got error this

"ERROR: failed to solve: failed to read dockerfile: open /var/lib/docker/tmp/buildkit-mount58663604/Dockerfile: no such file or directory"

I tried to access that address ( /var/lib/docker ..), but I couldn't make it.
Thanks in advance :)

Best,
Sangpil

from redarc.

I think you're running docker build . -t redarc in the wrong directory. It's looking for Dockerfile in /var/lib/docker/tmp/buildkit-mount58663604/Dockerfile which does not sound correct

Can you make sure you running docker build . -t redarc in the redarc project folder? (Folder that was created when you cloned the repo with the Dockerfile)

from redarc.

Hmm, I think I still don't get it how to install redarc in the redarc project folder.
I've tried to move to redarc project folder, but I couldn't find the way to move that folder nor the location of the folder.

Is there any way to find the redarc project folder?

from redarc.

@philz0918 Can you try this?

cd

git clone https://github.com/Yakabuff/redarc.git

cd redarc

docker build . -t redarc

from redarc.

@Yakabuff Thanks for your help!
I think now it works fine:)

I have one more question, I think I need to create comment_file/submission_file to get comment/submission.
I'm guessing these files have JSON format
For example,
{ "subreddit": "conservative" "body":... }

I think we can use before/after to set the time range, however, I don't think it can run with load_comments/submission.py .

Would you please provide an example for submission_file.txt?

Once again, thank you so much.

from redarc.

@philz0918
https://github.com/Yakabuff/redarc#download-pushshift-dumps

https://the-eye.eu/redarcs

from redarc.

@Yakabuff Sorry for bothering you.

I was wondering how I run my SQL file onto the db here.

Thanks.

from redarc.

Use the postgres client: psql -h localhost -U postgres -a -f file.sql

from redarc.

Not sure why you would need to do that though

from redarc.

@Yakabuff Thank you.

I just wanted to get data within a certain date range, and I thought I need to create comment_file.txt to run

python3 load_sub.py submission_file.txt

because there is no submission_file.txt in the script folder, and I thought submission_file includes some SQL to get the data I want.

from redarc.

ok just to make things clear:

You need to download the pushshift JSON data dumps from one of the sources here: https://github.com/Yakabuff/redarc#download-pushshift-dumps

Then you need to run python3 scripts/load_sub.py <your_pushshift_datadump_file> or python3 scripts/load_comments.py <your_pushshift_datadump_file>

When you are finished inserting the data, run python3 scripts/index.py

Note: the data dumps may be compressed in zstd. unzstd <file> to decompress it. The datadumps may also have inconsistencies/errors in them which you will either have to fix it or tweak the script. Please make a Github issue if you come across these inconsistencies

@philz0918

from redarc.

@Yakabuff

Oh, I've misunderstood some processes!
I appreciate your help, now it works well now! Thanks again.

from redarc.

No problem :)

from redarc.