Database Setup about cti-website-frontend HOT 23 CLOSED

Example API data from deploy on Heroku:

https://test-civictechindexadmin.herokuapp.com/api/organizations/Hack%20for%20LA%20Brigade/

from cti-website-frontend.

Requirements Document

from cti-website-frontend.

@cnk As discussed during the OPS meeting, I will like you to let me know the way forward to proceed with the creation of Django BackEnd.

from cti-website-frontend.

@cnk is going to work on a Django cookie cutter for several orgs to use.

from cti-website-frontend.

@ExperimentsInHonesty google doc link where I have put the results to rudimentary analysis.

The repository on which you can find the code to the analysis.

from cti-website-frontend.

Remove the space in open X where X is a city or location to accommodate/match the ones that have no space or have a space.

from cti-website-frontend.

@cnk is going to work on a Django cookie cutter for several orgs to use.

[Query] Should I provide any inputs to the Django Cookie Cutter?

from cti-website-frontend.

Remove the space in open X where X is a city or location to accommodate/match the ones that have no space or have a space.

I converted the two names in all lower cases and also removed all the whitespaces in both the strings. I have asked Bo to verify once.
Updated Result in the Repository:

length of boData =  264
length of cfaData =  216
Matched entries =  119

from cti-website-frontend.

Bo did a verification for above results. I will look into the results.
Google Doc with organizations that appear in both versions marked in yellow, and along with some comments.
Bo claims that maybe 27 entries are same still.

from cti-website-frontend.

Removed the special characters from XLSX data.
Matched entries = 131

from cti-website-frontend.

@ddkbky is going to do relook at the data and identify if there are any more uncaught matches.

from cti-website-frontend.

I will provide an update about how to set up a Postgres App on Heroku on Sunday's meeting [June 14, 2020] by doing a test implementation.
@ExperimentsInHonesty @arunprakash87 @cnk If possible could you let me know the backend that we have agreed upon so that I can narrow down my research and test implementation for setting up a Postgres App on Heroku.

from cti-website-frontend.

@Abhishek-AC @cnk @ExperimentsInHonesty
A comparison of the organizations in Bo's Excel spreedsheet and CFA's json document. Also have identified some organizations may not exist any longer:
https://drive.google.com/file/d/1Kv6vT5tEnZJxL3CkkE2Xx6osVppE-NCK/view?usp=sharing

from cti-website-frontend.

@Abhishek-AC - still working on items from this comment re the back end: #32 (comment)
@Abhishek-AC - will look at bo's spreadsheet above.

@cnk will deploy container for postgress and backend. It will not contain react for the time being.

from cti-website-frontend.

@ddkbky I have added a column to handle the matched entries in both the data sources.
Sheets Link

Matched Entries JSON file repo link

Requesting you to verify.

Matched entries = 137

from cti-website-frontend.

@Abhishek-AC I have checked these files. They look all right. I think the differences between the two versions of database have all been identified.

from cti-website-frontend.

I will provide an update about how to set up a Postgres App on Heroku on Sunday's meeting [June 14, 2020] by doing a test implementation.
@ExperimentsInHonesty @arunprakash87 @cnk If possible could you let me know the backend that we have agreed upon so that I can narrow down my research and test implementation for setting up a Postgres App on Heroku.

There are quite a few resources available for deploying a Django app with the Postgres database. Listing a few below that I perused.

1. Deploying Django to Heroku: Connecting Heroku Postgres
2. Deploying Django App on Heroku with Postgres as Backend
3. Configuring Django Apps for Heroku

Except for Django and psycopg2 python library, the following needs to be installed to the python environment:

• gunicorn
• whitenoise
• dj-database-url

from cti-website-frontend.

@cnk
I cloned and forked the repo: https://github.com/cnk/civictechindexadmin
I was able to run locally using docker.
Primarily followed the following: https://cookiecutter-django.readthedocs.io/en/latest/deployment-on-heroku.html
However, I wasn't able to deploy on Heroku.
Error message:

remote: -----> $ python manage.py collectstatic --noinput
remote:        Traceback (most recent call last):
remote:          File "manage.py", line 31, in <module>
remote:            execute_from_command_line(sys.argv)
remote:          File "/app/.heroku/python/lib/python3.8/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
remote:            utility.execute()
remote:          File "/app/.heroku/python/lib/python3.8/site-packages/django/core/management/__init__.py", line 345, in execute
remote:            settings.INSTALLED_APPS
remote:          File "/app/.heroku/python/lib/python3.8/site-packages/django/conf/__init__.py", line 76, in __getattr__
remote:            self._setup(name)
remote:          File "/app/.heroku/python/lib/python3.8/site-packages/django/conf/__init__.py", line 63, in _setup
remote:            self._wrapped = Settings(settings_module)
remote:          File "/app/.heroku/python/lib/python3.8/site-packages/django/conf/__init__.py", line 142, in __init__
remote:            mod = importlib.import_module(self.SETTINGS_MODULE)
remote:          File "/app/.heroku/python/lib/python3.8/importlib/__init__.py", line 127, in import_module
remote:            return _bootstrap._gcd_import(name[level:], package, level)
remote:          File "<frozen importlib._bootstrap>", line 1014, in _gcd_impor
remote:          File "<frozen importlib._bootstrap>", line 991, in _find_and_load
remote:          File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
remote:          File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
remote:          File "<frozen importlib._bootstrap_external>", line 783, in exec_module
remote:          File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
remote:          File "/tmp/build_ad8fb3a22d84a3a8a9555ace7cc27f7f/config/settings/production.py", line 3, in <module>
remote:            django_heroku.settings(locals())
remote:          File "/app/.heroku/python/lib/python3.8/site-packages/django_heroku/core.py", line 89, in settings
remote:            config['STATIC_ROOT'] = os.path.join(config['BASE_DIR'], 'staticfiles')
remote:        KeyError: 'BASE_DIR'
remote: 
remote:  !     Error while running '$ python manage.py collectstatic --noinput'
remote:        See traceback above for details.
remote: 
remote:        You may need to update application code to resolve this error.
remote:        Or, you can disable collectstatic for this application:
remote: 
remote:           $ heroku config:set DISABLE_COLLECTSTATIC=1
remote: 
remote:        https://devcenter.heroku.com/articles/django-assets
remote:  !     Push rejected, failed to compile Python app.
remote: 
remote:  !     Push failed
remote: Verifying deploy...
remote: 
remote: !	Push rejected to arcane-thicket-54698.
remote: 
To https://git.heroku.com/arcane-thicket-54698.git
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'https://git.heroku.com/arcane-thicket-54698.git'
(civictechindexadmin) AbhishekChhabra civictechindexadmin $ 

I have tried to debug.
Tried the following:

• Inside /civictechindexadmin/config/settings/base.py
  made the changes as done here
• Locally run manage.py collectstatic
  docker-compose run --rm django python manage.py collectstatic
• Again tried to execute
  git push heroku master

But got no luck.
However, was able to deploy using the following repository https://github.com/heroku/python-getting-started

Resolution

• I will proceed now with creating a node backend
• Parallelly, if I get a solution for deploying Django cookiecutter then I will update this ticket.

from cti-website-frontend.

@cnk
I was able to deploy the Django BackEnd with Postgres and I think it was working as required.

Test Deployment Heroku

Username: test
Password: samplepass

Repository Link

I have used your codebase. I have removed the env file and scaled-down the settings file as it was creating some issues (according to me) accessing the config folder.

Please let me know if this approach can be taken ahead of and gradually we might want to add separate settings file for the various environments.

from cti-website-frontend.

@Abhishek-AC The repository link goes to the project on Heroku. I tried lookin at your fork on GitHub but don't see new commits there. I suspect you have a new branch that is deployed on Heroku but I am not sure how to access it in order to pull directly from the deployed code.

Since we are continuing with Django, I'll work on making a data loading script.

from cti-website-frontend.

Apologies. Here is the repository link: https://github.com/Abhishek-AC/test-civictechindexadmin
I created a new repository to separate the codebase and understand the code flow by proceeding in a step-by-step manner.

from cti-website-frontend.

@Abhishek-AC The repository link goes to the project on Heroku. I tried lookin at your fork on GitHub but don't see new commits there. I suspect you have a new branch that is deployed on Heroku but I am not sure how to access it in order to pull directly from the deployed code.

Since we are continuing with Django, I'll work on making a data loading script.

@cnk Were we able to create a data loading script? I wanted to start exposing and documenting the endpoints.
Please let me know if there's anything I can contribute to for loading the data.

from cti-website-frontend.

I have an import script but need to refactor so it doesn't depend on having a local data file. Use 'requests' to pull data from https://docs.google.com/spreadsheets/d/1xiLeyMwZc-n6eB1R_XdCJ00YrarfnR_w/edit#gid=1137923926

from cti-website-frontend.