wanderer-moe / api Goto Github PK

Will (probably) relocate the current authentication on the site over to the API once all issues with setting cross origin cookies in lucia are fixed and their behaviour works as intended across all browsers - significantly easier in the long run to split the frontend/backend instead of a janky setup & can also allow for better ratelimiting by account and not per ip.

Could possibly allow for migrating db to CF D1 instead of Planetscale so everything is strictly CF (if D1 is out of alpha by then) but none of this is of high priority

something like rekognition should work for images (i love aws)
ref: https://docs.aws.amazon.com/rekognition/latest/APIReference/API_DetectModerationLabels.html

anything that somehow avoids the filter can just be reported lol

users should be able to invite others to collections as "collaborators" - where they can just remove or add assets

this can be done with a table collection_collaborators
contents of the table should be collection_id, user_id, date_added, accepted_invite [accepted, pending]

see https://github.com/axiomhq/axiom-js/tree/main/packages/js

related to #51 - instead of routes being /v2/asset/upload-asset.ts, we can restructure like so:

/v2/asset/upload/route.ts (logic)
/v2/asset/upload/openapi.ts (openapi definition)
/v2/asset/upload/schema.ts (zod schema)

for routes which require an id or name, we can have /v2/user/get/[name]/route.ts, etc

allow for users to link multiple providers to one account, e.g credentials & discord etc

cf image resizing is expensive (pro: $9/50k) and hundreds of thousands of images are viewed/downloaded a day.

creating 128px clones of every image image-1.png => image-1-128px.png will significantly expedite asset display as all files are strongly cached in the R2 bucket as their content will likely never change

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• fix(deps): update dependency @typescript-eslint/eslint-plugin to v7.7.1
• chore(deps): update dependency @cloudflare/workers-types to v4.20240423.0
• chore(deps): update dependency wrangler to v3.52.0
• chore(deps): update actions/checkout action to v4
• chore(deps): update actions/setup-node action to v4
• chore(deps): update cloudflare/wrangler-action action to v3
• Click on this checkbox to rebase all open PRs at once

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

• chore(deps): update dependency eslint to v9
• fix(deps): update dependency itty-router to v5

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

see: https://usend.email/
or maybe create a seperate worker for emails? idk yet

all middleware is inconsistently invoking, middlewares control where we expect the following headers: Access-Control-Allow-Origin, Content-Length, Content-Type, access-control-allow-credentials, x-ratelimit-limit, x-ratelimit-policy, x-ratelimit-remaining, x-ratelimit-reset to be set.

however the API seems to inconsistently set Content-Length, Content-Type only.

i have no clue what is causing this at all,,,,,? can't wait to fix this

• modify user attributes
• saving [oc gen responses, assets]
• make logout actually fucking work
• upload pfp/banner endpoint to r2 bucket inside /avatars and /banners fire emoji

• official/fanmade
• game version number
• #53

adding onto #35 - using cf durable objects seems like the perfect approach for this as they're consistent [i love cloudflare]

/{game}/{asset} has been created as a new location within every games rows, while also still appending the requests value inside {asset} locations? Seems to be present within all backups from the past 16 hours.

I'm unsure on what's caused/causing this - doesn't affect the API's ability to sort assets by request popularity.

having the default credentials authentication is fine but giving user options to login thru different methods can make their life somewhat easier

if signing up through discord - attempt to set their username to their current discord username, if not available, has special characters (old system before pomelo) or what can be considered filtered text etc just randomly generate something (i.e merge name if clean && discrim) then is this is the case it can get passed back to onboarding redir

self explanatory

self explanatory

typings aren't correctly passed through routes because we aren't initializing hono correctly within routes [me issue] - and also why the ????!? im extending bindings as env with unknown???????????????????????????

generates openapi documentation with type-safe params using zod for all routes; introduces type sharing between the frontend and backend, thus ensuring end-to-end type-safety for all routes && streamlining local development.

related to #43.

• use experimental backend (+ metrics)
• create table called requests, which will store this information, instead of having a table for each game and rows containing said data

this can probably be done with 2 new tables and new atlas directory in R2

• atlas [id, r2 path, other metadata.. etc]
• atlasToAsset [atlas.id <=> asset.id]

if asset category involves spines, we can check for linked atlas files then return it on /asset/[id]

could also create an atlas tag? [and/or atlasExists inside asset]

create totalAssetDownloadCount inside of games table, set to the amount of assets downloaded for each game, /games/all should sort by this col :3

robust database transactions where appropriate guarantees the success of all queries within said transaction or nothing happens at all

drizzle provides APIs to handle transactions: https://orm.drizzle.team/docs/transactions, where all db queries can be wrapped like so:

await drizzle.transaction(async (trans) => {
    await trans.insert...
    await trans.delete....
})

response messages should be detailed, instead of something generic like error: "500: something went wrong" because for one that's just not very descriptive and we want to keep the API as lightweight as possible with no $$$s spent in unnecessary logging when stuff like wrangler tail literally exists

also, in the case of no entries returned for a specific users attributes, i.e saved oc generator responses, 200 should be returned with an empty array. 404 should ONLY be returned if user/specific whatever doesn't exist..

++ responses are very inconsistent rn

Append lastUpdated to:

• each game in /games,
• /games/:gameId & all subfolders listed
• after asset in /games/:gameId:asset

Used to display Last updated: DD MM YYYY e.g Last updated: 3rd June 2023 for each asset category & each game, also useful for possible (game / category) sorting options in the future.

v3 is expected to release Q1 2024; considering the amount of breaking changes from v2 => v3 where oslo plays a big part alongside lucia, it makes more sense to migrate now to prevent significant issues later

Somewhat related to #7

https://developers.cloudflare.com/r2/pricing/#class-a-operations - Current Class A operations count is around 1,000,000/week, where it should be a lot lower than that.

It's unnecessary to list all contents in the bucket repeatedly, all code used to list files in sub-folders should be rewritten as soon as possible, pairing this with caching requests will help lower this number down drastically

certain queries can sometimes benefit from switching to the crud api
especially for fine-grained data manipulations, in some cases better typing and the similarity w/ writing raw sql

tbc after #20

:3

self explanatory

authenticated users should be able to request assets/games/asset categories directly onto the site, with an upvoting system - similar to https://productlane.com/roadmap

entries should be approved by staff where all users can then vote on them, contributors & staff can mark as completed

this can be done with 2 tables
asset_request (request id, user id, title, created at, description)
asset_request_upvotes (request id, user id)

query param which is passed should search for authUserName, gameName, assetCategoryName & tagName, If empty, return empty arr for each ""category"", indexing is a must

considering oc generators has like 8 objects, we can just index r2 bucket as it's stored /oc-generators/[game]
queries don't have to be exact, e.g "genshin" should return genshin-impact as entry in array game && oc-generator, complexity can be added in the future

can just create 2 separate tables for this, with the following cols:

• likedCollectionId & likedByUserId
• likedAssetId & likedByUserId

for collections: check publicity & disable this feature if private, same with asset status is not approved 🤔

• Move all routes into routes.mjs
• Rename Variables (and/or Functions) for consistent naming conventions
• Use https://github.com/cloudflare/wrangler-action to Deploy to wrangler on main commit using GH Actions

please help me

w/ accounts, users will be able to set/recieve toggle-able notifications for the following:

++ "watch" for new game && assetcategory upload [e.g watch uploads for genshin impact character sheets] (has to be toggled by user)
++ site updates & planned maintenance (enabled by default if user gives notifs permissions)

also checks to prevent game && assetcategory watch notif spam if user(s) are mass uploading

no more changing db, this is final - drizzle schema needs to be updated to sqlite
also need to re-implement asset data [which will need to be linked to a user] plus ALL endpoints
games and assetCategories will also be separate tables now, as well as tags per asset to support many [wanderer-moe/site#244]

Move all game and oc-generator routes outside of routes.mjs into a /routes subfolder, for better clarity & readability.

• Move /games, /game/:gameId & /game/:gameId/:asset into /routes/games
• Move /oc-generators & /oc-generator/:gameId into /routes/oc-generator

• refine searching r2 bucket
• #17

i dont think im actually passing the discord token to the v2 worker

Implement popularity to each game & asset with a numerical value, the higher being the more popular.

This can be calculated from anonymized analytics from the site & CDN, therefore allowing for sorting by "user engagement".

This will be calculated using D1.

• Implement storing & adding game & asset data
• Collect game & game_asset data
• Implement popularity sort, getting values for game asset & then appending popularity (1 = highest, etc)

Related to wanderer-moe/site#203 - makes sense to be implemented onto the API first.
Requires #5 to be completed beforehand

this will take in r2 & the drizzle (orm) instance, so instead of having to update multiple queries when they're reused, we can do stuff like:

user managing:
userManager.getUserById(user) userManager.getFollowers(user) userManager.followUser(user1, user2) userManager.updateUserAttributes(user, newAttributes) userManager.clearSesions(user)

asset managing:
assetManager.getAssetsFromQuery(tags[]?, assetCategories[]?, game[]?, uploadedBy[]?, name?)
assetManager.uploadAsset(assetData, tags[]?)

etc

this is a more effective approach for connecting various files to assets in general, like fan-made .psd files - opposed to tables for each extension.

putting high priority (tbd asap) on this due to the nature of the change.

had permission from one of the creators to do use their API for this (on the terms we use a custom UA for identification in-case of high request counts)

they have 2 very useful (& documented) endpoints: https://en.pronouns.page/api/pronouns & https://en.pronouns.page/api/pronouns/[pronoun] (i.e he/him) which can be strictly cached, contains useful data like pronunciations & morphemes & examples; i guess this can also be used for validation. fuck yeah i love inclusiveness********* 🏳️‍⚧️💯💯

ngl i nearly forgot about this