miserlou / django-zappa Goto Github PK
View Code? Open in Web Editor NEWTHIS PROJECT IS DEPRECATED, GO HERE INSTEAD:
Home Page: https://github.com/Miserlou/Zappa
License: MIT License
THIS PROJECT IS DEPRECATED, GO HERE INSTEAD:
Home Page: https://github.com/Miserlou/Zappa
License: MIT License
Just created an annoying zip called '..-demo-1457630734.zip'. Let's avoid doing that.
Hello, first of all sorry if this is in the wrong place.
There is something that got my attetion, about how you need to use python-mysql-connector which is pure python implemented (no C dependencies).
This may not be good if you need high performance on database queries (even more considering a 1 minute timeout as max execution time), and this driver is quite slow compared to the ones with C extensions.
There's mysqlclient connector which does not depend on MySQL Binaries to be installed (but uses C code, and it is one of the fastests connectors) which of course can not be included easily due to the C dependencies. However, I have once deployed python code with C code already compiled on a EC2 machine (there's on AWS Lambda docs the exact ec2 image they use) which worked fine on Lambda, at the cost of having a bigger zip. Have you guys ever considered this as an option if you are looking to include C code?
The deploy and update command is not including the package created by the django-admin startproject command. Only the packages that are in the installed apps. This results in a <project_name>.urls import error. No module found zipzip.urls
If you look at any travis build you can clearly see that tests should be failing but there are no asserts to test the invoke commands and so they're being called improperly and nothing is detecting the fail.
Avoid packaging up static files, etc., and whatever else should be excluded. This should arrive shortly in the core lib, just need to add d-z support.
when i tried to use django zappa for configure the django admin panel. session get logged out for every two subsequent clicks. I was unable to reach till adding a uses from django zappa based admin panel. I thought zappa cokkie getting expired.
Hi chaps,
apologies for raising this if it's out of scope.
I've read news about Azure Functions being introduced, and apparently there is less of a cold start issue and Functions are guaranteed to stay hot for at least 5 minutes.
Here is a link to Azure Functions:
https://azure.microsoft.com/en-us/services/functions/
Started in Zappa core, ability to
python manage.py tail beta
And get a tail of the logs going on there.
I'm worried that this line might not be unicode safe: https://github.com/Miserlou/django-zappa/blob/master/django_zappa/handler.py#L105
if '~' in settings_file:
settings_file = settings_file.replace('~', os.path.expanduser('~'))
can be replaced with:
settings_file = os.path.expanduser(settings_file)
I tried deploying, and keep getting this error when deploying Sphere.
~/.aws$ cat credentials
[default]
aws_access_key_id = [censored]
aws_secret_access_key = [censored]
The only file I really needed to change was spheres/production_settings.py
Followed the video, and seems like I'm just getting a 500 Internal Server Error?
https://l0723k4q1j.execute-api.us-east-1.amazonaws.com/production
I created a project for building leaderboards using redis and it was causing import errors with djang-zappa..
So I have created a django project for trying out django-zappa.
It works fine on my local system but when I hit the api provided by django-zappa, I get following error:
Unable to import module 'handler': No module named django.core.handlers.wsgi
Also the endpoints created by django-zappa in API Gateway look like
Am I supposed to pass some parameters to django-zappa? If yes, how?
Do I need to explicitly create API endpoints in Lambda as I didn't see any in the Lambda console??
It was pointed out during the Q & A that the Django Middleware could be replaced with a WSGI Middleware that would live in Zappa core. This has a number of obvious benefits, such as more shared code, less client-specific code, and less client-specific integration.
This should probably come after #11, but it'd be great to be able to profile a request so that we can start shaving some time off each response. I bet we can get it below 100ms.
When I try to deploy I get this error after the zip file is uploaded and after Creating API Gateway routes.. is printed to the console:
botocore.exceptions.ClientError: An error occurred (NotFoundException) when calling the PutIntegrationResponse operation: No integration defined for method
Any ideas what I might be doing wrong?
Generally need a better understanding and documentation about static/media/admin file handling with D-Z. Should come soon.
Raises a warning if the Middleware isn't in the settings, even if it would be loaded safely by calling from "from .base import *", etc
Create a command that can be run that will clean up the AWS account after the deploy command (ie restore it to how it was).
Since the IAM role is global, we probably don't want to delete that (by default).
Add a management command to call zappas rollback function
The implicit status code handling for redirects will moste likely misbehave if any other than a 200 response code provides a location.
A good example would be a 201 CREATED, should include a location header.
I guess it makes more sense to filter for 3xx status codes rather than NOT 200.
Hey all -
First off, great job on all of this! I've been loving Zappa, and can't wait to migrate all of my applications over to using it.
But - I currently have an issue that I would love some clarity on.
I've integrated Boto and S3 static and media hosting, as described here:
https://www.caktusgroup.com/blog/2014/11/10/Using-Amazon-S3-to-store-your-Django-sites-static-and-media-files/
It works quite well, but there is some weirdness when deployed with Zappa.
Specifically, when running locally, all files are uploaded as expected to the S3 servers (I'm just testing through the /admin console).
When running on a Zappa deploy, some uploaded files are mangled (they end up being twice as large as they should be). Generally, I am trying to upload audio files ... and the result is an invalid file.
Text files work fine though ... which leads me to believe it has something to do with escape characters getting added? Since the audio files are non-ASCII in nature.
So ... to sum it up:
Running at localHost:8000, and using /admin ... I can upload FileFields to S3:
Running remotely as a Zappa deploy, and using /admin ... I can upload FileFields to S3:
Also, running the same code on an EC2 instance works just fine ... so it must be something with the fact that it is running serverless?
Not sure ... but hoping someone here might have an idea :)
Hello,
I have tried multiple times to run the deploy from various environments, and each time I run into this set of errors: https://gist.github.com/anonymous/0b078b15465bb3ab104e3ed9f631acfb
I am running Django 1.9.5 with Python 3.4.4
I'm sorry I'm extremely lazy to create a pull request, but looking at handler.py while trying to help out with an issue at the django forums I noticed some really minor optimizations that you can do (they are really minor and won't change any timing significantly, but it is always good to do things in the best way as possible, every milisecond counts!)
Debug and logger calls: Are you using AWS logging? If I'm not wrong that goes to cloud watch which can be pretty slow, I would really avoid any logging when debug is false and do not test any timings with code that writes to those logs.
Line 61, def start(a,b)... Defining a function over and over is quite expensive, and nested functions are a bit more expensive on python also. I would have that one defined at module level only once.
Line 87: if response.status_code in [400, 401, 403, 404, 500]: You are defining this list over and over, and also the IN operator on a list is always O(n) worst case, I would define this list as a global variable and even better make it a set or dictionary which will be O(1) most of the time (although for such a small list, the difference is really small, on a 10000000 loop it is about 25% faster on python2.7)
Overall, try to use either format or % operator for string concatenation, it is way more efficient and readable than +
I couldn't find how does the common_log work, but you are calling it every time even with no debug, which might be giving you some unnecessary slowdowns.
Way too much repeated code there, they should all inherit from a ZappaCommand.
Any relative imports in settings files do not work as it is being copied to a different location.
For developers that work on a team where everyone doesn't have production deployment privileges, loading the settings file from S3 would be helpful. If the user's IAM credentials don't give them access to the S3 file they can't access the settings file. We can already limit the ability to create or update Lambda function code and API Gateway resources via IAM policies.
ZAPPA_SETTINGS = {
'production': {
's3_bucket': 'production-bucket',
'settings_file': 's3://your_bucket:your_folder/production_settings.py',
},
'staging': {
's3_bucket': 'staging-bucket',
'settings_file': '~/Projects/MyApp/settings/staging_settings.py',
},
}
Must be the result of the new boto3 stuff?
It is unclear to me how the PYTHONPATH and general environment are setup. I can successfully deploy an application to Lambda, however the deployed Django is unable to find the urls file.
I have, for example, project myproject, with core application myproject (the standard Django setup) which contains urls.py. Thus, the settings files have:
ROOT_URLCONF = 'myproject.urls'
But (in debug mode) I am getting: No module named myproject.urls
The settings I am using work find when running locally. My suspicion is that the PYTHONPATH is not being set so that myproject is available on the path, but this is just a guess really. I noted that the test settings file has BASE_DIR set to an empty string, which strikes me as unusual, but I have no idea if that is related to this. Setting BASE_DIR empty in my own project breaks the update process for me with a botocore function not found error.
https://swe19z3zr7.execute-api.us-east-1.amazonaws.com/fadmin/ returns {"message": "Internal server error"}.
The first request to favicon.ico returns a 403 error followed by code 500 for the document.
Hey guys!
First of all, thanks for this app, it amazing =D
Im trying to use it with postgres RDS but, the Sphere app does not run. I get always request timed out.
There's any kind of different driver for use postgres with lambda?
Im using this as default engine -> django.db.backends.postgresql_psycopg2
Is that right ?
Thanks in advance! =D
Ran "pip install zappa" in a new virtualenv.
I got:
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DUSE__THREAD -I/usr/include/ffi -I/usr/include/libffi -I/usr/include/python2.7 -c c/_cffi_backend.c -o build/temp.linux-x86_64-2.7/c/_cffi_backend.o
c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
and fixed it by:
"sudo apt-get install libffi-dev"
It looks like the repo needs a singular static requirements.txt
file - I'm quite used to having at least 3, one for common stuff, one for dev and one for prod (and the latter 2 include the former).
I had a quick skim of the code and it looks like it's looking for the lines of requirements.txt
in setup.py
-- looks like there's a package (requirements-parser
) that can slot in here, making the requirements an arg somewhere:
In [1]: import requirements
In [2]: fp = open('requirements/dev.txt')
In [3]: list(requirements.parse(fp))
Out[3]:
[<Requirement: "Pillow==2.8.1">,
<Requirement: "boto==2.38.0">,
<Requirement: "hashids==1.1.0">,`
...
This ended up being way more detailed than I anticipated.
profile.txt
This question gets asked a lot and should be included in the docs
There appears to be a problem with the certification authority of the SSL certificate for https://zappa.gun.io/.
When I set my ZAPPA_SETTINGS to be something like below
ZAPPA_SETTINGS = {
'dev': {
'aws_region': 'ap-northeast-1',
's3_bucket': 'MY_BUCKET_NAME,
'settings_file': 'PATH_TO_MY_SETTINGS/settings.py'
}
}
the standrad "python manage.py deploy dev" command doesn't work.
However, If I remove the "aws_region" parameter, things magically work. I believe this behaviour is easily reproduciable and might be non-us-region specific.
ie myapp.settings.remote.production
and then inspect.getsource
I setup a domain for my AWS Gateway and a default stage so that going to https://sub.domain.com brings up the homepage for the Spheres app. When I try to go to the admin section the app redirects to https://dev/admin/login/?next=/dev/admin/
I have the domain setting setup in the zappa settings:
'domain': 'sub.domain.com',
How do I get the django app to use my custom domain that I've setup and get rid of the /dev/ section of the url?
It's annoying to upload a zip, only to realize that
- You are over the limit of APIs
- There is already an API with this name
There should be a pre_deploy function to prevent those annoyances.
512 is probably too high for a default, I bet we can drop that by half at least.
I'm running into an issue while using python social auth. The problem is when I pass it a token (ex: facebook token). The lambda makes a graph call it never returns. The request always ends up timing out.
Would be convenient if had ability to specify which routes API gateway resources/methods were created.
I've had a look myself and I might tackle it soonish. Hopefully.
Thanks for all the hard work on this amazing project!
The deployment is successful, however when I tried to access the url, I got {"message":"Forbidden"}. Don't know what happened...
Hello,
Sorry to come with another issue, luckily I am just working on a small test project so switching from python 3.4 to python 2.7.6 was easy enough since I am not doing much that is impacted by switching versions (just testing mostly) though now when I run the deploy, it bundles the files up nicely, creates the lambda app, then create the API Gateway part which seems to complete without issue, but now when Deploying the API Gateway it errors with this: https://gist.github.com/tcross911/369d204d983919a1bbb236831206ef56
Would be very cool to showcase some Zappa-powered instances of
and would also help us discover the bugs/quirks necessary for deploying those.
..any others?
Unfortunately, this is not exposed to boto yet, but maybe we can hack it together before the official SDK is upgraded?
So I deployed my app in Lambda, with the URL like this https://16xf96vt2l.execute-api.eu-central-1.amazonaws.com/production/
.
How do I remove /production/
on it?
Hello,
Is there anyone working on postgresql support on RDS ? If not I can try to make it work and submit a pull request.
Thanks
Amine
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.