Store and read data from a database instead of a JSON file about verto HOT 12 CLOSED

@dnafication Thanks for all the pre-work. I am thinking of running this on Vercel, since it's already there. Vercel has some integrations with db-providers, see https://vercel.com/docs/integrations. Examples are supabase, mongodb atlas, etc...

from verto.

Okay. I'll explore what options are available on vercel for this. Meanwhile if anyone has suggestions on the API design or any other aspects, please share them.

from verto.

Hey, been a while since I last perused this project but loving the direction it's going in!

fwiw, have we considered having the scheduled jobs interfacing with the api rather than the db directly?

off the top of my head, the main benefit with this approach is that the database is purely just implementation detail--it means we can make schema changes transparent and independent from consumers, as long as the api contract doesn't change, we're good :D

though, I would say that this is more of a "future-proofing" suggestion more than anything (since atm there isn't a requirement to design for multiple clients writing to the database which would necessitate for clearer data ownership, for example)

from verto.

@dnafication We can use NextJS as the backend instead of a separate nodeJS server, and as @lucavallin is thinking of running the app on Vercel, then NextJS API routes would be better. API routes will not even be needed if NextJS 13 App Router is used for SSR, though we will need to migrate from the Pages directory to the App directory, but that won't be much of a problem as lots of rewrite is needed anyway.

ngl, I kind of like the idea of using Next 13's server components lol.

as long as we know that the client will never need to write to the database, or if it does and we can ensure that the client and scheduled jobs won't be writing over the same data and stepping over each others toes--then this is a pretty solid option given that it eliminates the need to have a separate api layer entirely (both these entities can read/write to the db directly)

from verto.

@lucavallin Thanks for keeping this alive. 😄

Before we dive into the implementation details, let's take a moment to identify and outline the existing limitations. This will serve as a valuable foundation for shaping our new approach. Kicking things off with a few observations:

• Presently, our data is tightly coupled with the site and is stored in a JSON file, as mentioned earlier by Luca. This data is used during the page generation process.

• The appeal of static sites lies in their speed and efficiency. I believe it's essential to retain these advantages while also keeping our content up to date on the client side, especially when new data becomes available.

• Looking ahead, we should consider introducing features that allow users to sign up and potentially save issues that pique their interest. Furthermore, it might be worth exploring the development of a recommendation engine that leverages user preferences and interests.

Please feel free to add your thoughts and any additional limitations you've identified.

from verto.

So the new site will still be a static site with client-side code sprinkled in, or a server-side rendered? Or a combination of both?

Also, should we move the project to the new NextJS App Router for these changes, cause SSR is better in NextJS 13+ in many ways?

from verto.

I am leaning towards the Static Site but SSR sounds good too, both will require API anyways. It boils down to cost factor I guess.

from verto.

I'm interested in working on this issue. Has a decision been made regarding the design? I'd be happy to contribute to the discussion or begin working on a solution. @dnafication

from verto.

Hey @lukebrichey, we haven't made much progress on the API design yet. We need to decide where and how to host it. There are many options, but I'm considering using Amplication (https://docs.amplication.com/), which can help us generate a large chunk of backend code and includes built-in authentication. We could also explore other options, though. @lucavallin, do you have any suggestions for where to host the API?

from verto.

Amplication can generate production-ready NestJS code, or we could start from scratch. We also need to decide on a database. I'm leaning towards PostgreSQL for SQL or DynamoDB for NoSQL. I don't see many open source projects using AWS, but it's easy to set up an AWS account, although it does have some other considerations.

from verto.

I'm looking for cloud computing platforms that offer free credits for open source projects like ours. DigitalOcean has some criteria for supporting open source projects, and we'll be eligible for their Testing tier based on our star count 🌟.
Ref:

1. https://www.digitalocean.com/open-source/credits-for-projects
2. https://www.heroku.com/open-source-credit-program
3. https://free-for.dev/#/?id=free-fordev

Do you guys know any other options to host a Node.js server and a PostgreSQL db for free or the based on the budget (which @lucavallin is the right person to decide)?

from verto.

Do you guys know any other options to host a Node.js server and a PostgreSQL db for free or based on the budget (which @lucavallin is the right person to decide)?

@dnafication We can use NextJS as the backend instead of a separate nodeJS server, and as @lucavallin is thinking of running the app on Vercel, then NextJS API routes would be better. API routes will not even be needed if NextJS 13 App Router is used for SSR, though we will need to migrate from the Pages directory to the App directory, but that won't be much of a problem as lots of rewrite is needed anyway.

from verto.