falling-fruit / falling-fruit-api Goto Github PK

View Code? Open in Web Editor NEW

5.0 5.0 1.0 630 KB

API (v0.3) for Falling Fruit

Home Page: https://fallingfruit.org/api/0.3

License: GNU General Public License v3.0

JavaScript 100.00%

api express foraging nodejs

falling-fruit-api's Introduction

Falling Fruit Legacy

This is a Rails 3 web application and API for Falling Fruit, built for use with a PostgreSQL + PostGIS database.

Who is responsible?

Falling Fruit co-founders Caleb Phillips and Ethan Welty. More info at fallingfruit.org/about.

How can I help?

If you want to help with development, feel free to fork the project. If you have something to submit upstream, send a pull request from your fork. Cool? Cool!

Status

The website is live at fallingfruit.org. However, maintaining both a website and a mobile app that do not share any code proved too time consuming, so we are slowly phasing out this project in favor of a mobile-friendly web app (falling-fruit-web). However, all versions of the mobile app still rely on the Rails API.

Development

Install PostgreSQL, Ruby, and dependencies

PostgreSQL (15) & PostGIS (3.3), for example with Homebrew:

brew install postgresql@15 postgis

ImageMagick:

brew install imagemagick

Ruby (2.3.4), for example with rbenv:

rbenv install 2.3.4
rbenv shell 2.3.4

Bundler (1.17.3) with RubyGems:

gem install bundler -v 1.17.3

Project gems with Bundler:

bundle install

Initialize the configuration files:

cp config/database.yml.dist config/database.yml
cp config/s3.yml.dist config/s3.yml
cp config/initializers/credentials.rb.dist config/initializers/credentials.rb
cp config/initializers/secret_token.rb.dist config/initializers/secret_token.rb
cp .phraseapp.yml.dist .phraseapp.yml

Add a desired development database name and your database username, password, and port to config/database.yml. Add Amazon S3 and Google API credentials to config/s3.yml and config/initializers/credentials.rb. If working with translations, add Phrase credentials to .phraseapp.yml.

Start the app

Create and structure the database, then seed it with db/seeds.rb:

rake db:create
rake db:migrate
rake db:seed

Install and start the Falling Fruit API.

Finally, start the web server and navigate to localhost:3000:

thin start

Translation

For translators

Website translations are managed on the Phrase project Falling Fruit (web). To contribute, email us ([email protected]) and we'll add you as a translator.

For developers

Install the Phrase CLI:

brew install phrase-cli
cp .phraseapp.yml.dist .phraseapp.yml

Add your Phrase access token to .phraseapp.yml.

Add a new translation

In the Falling Fruit (web) project, select the default locale (English/en), and add a new translation key. If the same word or phrase appears often, add it as glossary.<key name>.

Then, update your translation files (in config/locales/*.yml):

phrase pull

Use the translation key in your template.

<!-- Instead of adding text to the markup: -->
<span>Map</span>

<!-- Evoke the translation key value with translate() -->
<span><%= translate("glossary.map") %></span>

falling-fruit-api's People

Contributors

Stargazers

Watchers

Forkers

epigmeniocruz

falling-fruit-api's Issues

Photo upload sometimes fails due to unknown error

I am tearing my hair out trying to debug this problem. Photo upload (POST /api/0.3/photos) fails with HTTP 500 response in certain conditions:

From a browser with the website running either locally or on https://beta.fallingfruit.org, or programmatically (e.g. Postman). In the latter case, sending repeat requests in quick succession often results in either HTTP 500 or HTTP 503.
API running at https://fallingfruit.org/api/0.3. I cannot reproduce the problem with the API running locally.
And only sometimes. It can fail, then work immediately on the second try, or keep failing. Try your luck at https://beta.fallingfruit.org/locations/3383
A large photo is not required. I've been testing with a 22 KB file.
No difference whether authenticated or not.

I cannot find where the error originates. The request does not seem to reach the API, nor even the lighttpd web server (at least, I find nothing in either website access logs nor server error logs when request fails). Could it originate from Apache? Or even Cloudflare? Not the latter according to the response body (see below), which is devoid of cloudflare (see https://developers.cloudflare.com/support/troubleshooting/cloudflare-errors/troubleshooting-cloudflare-5xx-errors/#error-500-internal-server-error).

Could it be high traffic? Lighttpd error log has many such lines:

2023-05-21 21:02:24: (gw_backend.c.315) gw-server re-enabled: tcp:127.0.0.1:3301 127.0.0.1 3301  
2023-05-21 21:05:28: (gw_backend.c.236) establishing connection failed: Connection refused socket: tcp:127.0.0.1:3301 
2023-05-21 21:05:28: (gw_backend.c.939) all handlers for {request_url} on  are down.
2023-05-21 21:05:30: (gw_backend.c.315) gw-server re-enabled: tcp:127.0.0.1:3301 127.0.0.1 3301
...

Request

POST https://fallingfruit.org/api/0.3/photos?api_key={api_key} with form-data body with file: {photo_path}

:authority: beta.fallingfruit.org
:method: POST
:path: /api/0.3/photos?api_key={api_key}
:scheme: https
accept: application/json, text/plain, */*
accept-encoding: gzip, deflate, br
accept-language: en-US,en;q=0.9,fr-FR;q=0.8,fr;q=0.7,de;q=0.6
content-length: 22032
content-type: multipart/form-data; boundary=----WebKitFormBoundarytgWDaB8BBekMo9JT
cookie: __utma=71168135.1257686696.1678958397.1678958397.1678958397.1; __utmc=71168135; __utmz=71168135.1678958397.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); _session_id=218d88dd6060fd9a6dd45fe12607497e; welcome_modal_shown=yes
origin: https://beta.fallingfruit.org
referer: https://beta.fallingfruit.org/locations/3383/@40.04059999999998,-105.26319,16z
sec-ch-ua: "Google Chrome";v="111", "Not(A:Brand";v="8", "Chromium";v="111"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "macOS"
sec-fetch-dest: empty
sec-fetch-mode: cors
sec-fetch-site: same-origin
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36

Response

<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
         "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title>500 Internal Server Error</title>
</head>
<body>
<h1>500 Internal Server Error</h1>
</body>
</html>

alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400
cf-cache-status: DYNAMIC
cf-ray: 7cafc2862cf81d90-FRA
content-type: text/html
date: Sun, 21 May 2023 21:02:21 GMT
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=L1M%2BXhIy7II6I28gxj9XsCnn8xgjBzv9Guk0YAMmLWgd7PPrj6yWrQ23TrWid9dY479qEilr2dYV90yWhHaBfOdcZQjnd4R9r5zdyHZRXrM90M7P4Is5jBOPATXR8gqnpAVqOzQHBmA%3D"}],"group":"cf-nel","max_age":604800}
server: cloudflare

Translate user emails

Accept a language query parameter and use this to change the language of the email sent to the user.

Simple "ripe now" filter

A "ripe now" filter is the most oft-demanded feature by far. Although extremely complex to implement, we could start with the simplest possible version: filtering based on location.season_start and location.season_end, and possibly observation.fruiting_status (in conjunction with observation.observed_on).

A benefit of rolling this out early would be to give people added incentive to add this information to locations.

Require observation date if fruiting status is provided

Review observed_on should be required if fruiting_status is not null, since otherwise the latter is meaningless.

Filter changes by user foraging range

Add the ability to filter changes made within a user's foraging range.

Questions:

Should this include locations previously in the range that were moved out of the range?
Should this be public or restricted to that user?

Add public user profile

If a user has a non-null name (essentially a username), that user is identified on locations and reviews that they created. Their name could be a link to a public user profile, which would include at minimum:

name
bio

Should the profile also include these information?

range
user activity

Filter by access (e.g. private, public)

Requested by a user:

Wondering if there's anyway to filter by whether a tree is on private or public property? I know that may not be possible for many trees, but since it's a question you ask when people submit trees, or that may be available in city datasets, thought I'd ask if there's a way to do that.

Will require adding a parameter to the API locations endpoint.

Improve performance of locations and clusters endpoints

The locations and clusters endpoints represent the majority of Falling Fruit traffic. A few challenges stand in the way of optimizing these further:

Each location can have many types. This makes filtering locations by type slower than if a location had only one type.
In cities with dense tree inventories, there can be upwards of 100,000 locations to filter, even at zoom level 12-13 where clusters currently switch to locations. This results in slow location queries over these areas (seconds rather than milliseconds).
Since there are thousands of types, we cannot index or cache locations for each type filter combination.

Simple things that could be done immediately include:

Remove RANDOM ordering of returned locations. This is slow and results in different results to identical queries. To ensure that returned locations are spatially randomly distributed, use a generated and index random id column.
Limit decimal places of returned location coordinates to 5–6.

But for real gains, deeper re-engineering will be needed. Ideas include:

Restructure the database and/or the SQL query to speedup n-nearest-location searches when used concurrently with a type filter.
Use cached vector tiles of locations / clusters for the most common filter combinations: muni (on/off) / type filter presets (all/forager/freegan).
Cache arbitrary vector tiles on request to adapt to user preference dynamically.
Perform filtering of locations on the client side.
Either drop clusters (require users to zoom in to filter locations) or perform type filtering of clusters on the client side. This would require type counts for each grid cell. The center of mass of each type would also need to be included for cluster coordinates to be adjustable based on the filter.
Switch to Mapbox.

Require a minimum password length

Require passwords to be at least 6 characters long, as required by the current mobile app.

Cannot create a new location

POST locations results in the following error:

{"error":"null value in column \"created_at\" violates not-null constraint"}

Add endpoints to `routes` table (location lists)

Users can organize locations into ordered collections, named "routes", and display these on a map with Google Map directions overlaid. This feature is buried in the interface and very rarely used.

This feature should be rebranded and the emphasis on routing removed, but otherwise the existing data can be reused.

GET user/locations/lists - Get user's location lists
POST locations/lists – Create a list
PUT locations/lists/{id} – Update a list
DELETE locations/lists/{id} – Delete a list
POST locations/{id}/lists/{id} – Add location to list
DELETE locations/{id}/lists/{id} – Remove location from list
GET locations/{id} – For authenticated users with lists, should return membership in lists.

Add support for existing type categories

The four supported type categories reflect four different user communities: forager, freegan, grafter, honeybee. Support for these are needed by falling-fruit/falling-fruit-web#246 to handle historic URLs. However, they could also be seen as built-in type lists (see #9). The simplest solution would be to return categories of each type from GET types.

However, for performance (#2), it might make sense to pre-compute these type filters as indices on the locations table. This would avoid the need for filtering locations by a long list of type ids dynamically. This would instead require a new query parameter in GET locations (e.g. categories).

Add closed access level

There are situations when we need to specify that a location is off-limits. The two common cases are:

Fruit trees on private land for which the landowners want to prohibit harvesting. The location is tagged but stays on the map so as not to be re-added by a user.
Potential dumpster locations (e.g. grocery store) that cannot be harvested (e.g. dumpster is locked or a compactor). The location is tagged but stays on the map so that dumpster divers can easily see which locations have already been scouted.

Is there a word or phrase in English that can be used in both cases?

Closed
Not permitted / possible

lat, lng (and address from which coordinates were derived)
range_radius and range_radius_unit

Or simply a polygon directly:

range

This can be simplified to simply accepting and returning a range (as a GeoJSON polygon).

Add endpoints for type lists

Similar to #8 for location types, except that this would be a brand new feature.

Questions

Does order make sense for types, as it does for locations?
Does adding a type to a list also include all children types?

Endpoints

GET user/types/lists - Get user's type lists
POST types/lists – Create a list
PUT types/lists/{id} – Update a list
DELETE types/lists/{id} – Delete a list
POST types/{id}/lists/{id} – Add type to list
DELETE types/{id}/lists/{id} – Remove type from list
GET types/{id} – For authenticated users with lists, return membership in lists.

Allows an observation date that is in the future

Although this is physically impossible, an observation date that lies in the future is currently permitted.