dev-mastery / comments-api Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
License: MIT License
Hi, first of all thank you very much fore sharing the code and producing such a great video. This is very valuable and I can imaging it helps a lot of developers including me.
I try to think a step further and it seams like I can not get my head around the following use-case. Basically how would I handle a list which will be accessed through the root. Bear with me while I try to come up with an example that might be related to comments-api
to explain it a little bit better.
How would I handle something like tagging a comment for example with cheers, likes, applause or emojis. Or tag it with labels as question, helpful etc.
Basically when I create a comment it contains the comment text like before. But I might have an empty list of tagables.
I can then assign a certain tag (whatever it might be) to a comment and also count them up if there are multiple of the same.
Would there be something like makeTagList and makeTag? Or would it be enough to just have the makeTag function injected into makeComment? Or how would I handle the nesting.
I hope may example makes sense and kind of transports what I am trying to ask.
It would be great, if you find the time to look at this and share your thoughts.
Looking forward to it.
Hi Bill!
Thanks a lot for your code - it was really useful for me.
I have the next question - how correct is it to use validation in entities? Is it still within business rules?
Maybe we should use validation in other layers of the application?
Hello there Bill.
First I would like to thank you for this awesome video and repo you came with, it opened my mind in ways I can't even express and I'm learning a lot with it. Now to the question.
I'm building a more complex app using a very similar approach but using typescript.
Simplifying the whole thing, I have an entity User
which declares a dependency on a UserRepository
that will be used to check if the given email is unique across the database on creation (which is an enterprise business rule, therefore needs to be here). This UserRepository
interface has a findByEmail
method that returns a simple user data structure (name, email and password).
So far so good, but I have one use case ShowUser
which will be used to display the user information with some extra data. This use case should use the same UserRepository
(defined in the entity), but the return value of findByEmail
now should return a slightly different user data structure, because now I want it to include the user boards, which you can think of as a join of 2 tables.
Now comes the whole problem, with a lot of solutions which seem bad to me.
Solution 1: Easily fix the problem by going to the User
entity and altering the user data structure to include a boards
field. But now I modified the data structure with an application business rule in a place that should only be concerned with enterprise business rules.
Solution 2: Create another method in the interface called maybe findByEmailWithBoards
, which sounds horrible just by the name (what if I need more fields?) and also has the same problem as solution 1.
Solution 3: Make the repository simpler, returning a simple user data structure and creating a new BoardsRepository
to create a "manual join" of both data structures. This sounds a bit better but I completely lose the speed and convenience of joining multiple tables (or even having embedded documents in mongodb) that databases offer, leading me to have to make multiple queries (which can grow a lot) instead of 1, which will probably have a huge impact on performance in certain cases.
Solution 4: Having one UserRepository
interface in the User
entity and another for the use case, which then the implementation would implement both, satisfying both needs. But that sounds very weird for me, especially on how would I name them to differentiate between them.
There are probably more solutions that I came about but all of them seem to have a big drawback like these, so I'm not sure if this is gonna have to be a trade off or maybe I'm missing something.
This approach is a bit different than yours since you don't use the repository/database inside your entities, but there's definitely gonna be a point where you'll have to if you want to have more complex business rules.
So what to do when your application business rules conflict with your enterprise ones like here?
Thanks again and sorry for the long question! I hope you have the time to discuss this because I'm kinda out of ideas for now and I'm really hopping to get to something solid.
Hi Bill
How can I debug/step through the code in VS Code?
Regards
Hello,
First of all thanks for a wonderful tutorial!!
I am working on an Express-based API that uses MySQL for data persistence. I have to implement a confirm order functionality
I know how to run Mysql specific transactions and all the things, but with a lot of coupling and poor source code architecture. (If I do it that way, then DB won't be like plugin anymore)
I am not able to understand how to do this while maintaining the architecture and at what layer to implement this transaction thing, use-case/data-access, or what?
Thanks in advance
PS: also suggest to me, if it is an automation logic or business logic?
About the is-questionable.js
file, I wonder if you are not really taking dependency injection to far.
While I agree that we should only have dependencies from the outer circles to the inner ones and not the other way around, I believe that this rule could be relaxed for well known functions that probably won't change or break unexpectedly.
For example, pipe
is a well-known function composition utility. While it has several implementations (and you also rolled your own), their behavior is the same (or almost). querystring
is a native Node.js module, so you probably wouldn't want to roll out your own API to handle query strings.
Would it be so bad if we had something like:
import pipe from '@dev-master/pipe';
import querystring from 'querystring';
export default function makeIsQuestionable ({
issueHttpRequest,
}) {
// ....
}
Also, in the index.js
file in the same folder, you are just assigning issueHttpRequest
to axios
, querystring
to Node's native querystring
module, etc., which means your is-questionable
input ports are actually the APIs from axios
and querystring
, so the dependency direction here is actually the opposite it should be.
axios
might be a case apart because you can say:
Oh, I do this because of testing. It's easier to not make actual HTTP request doing so.
Well, you could actually achieve that using something like axios-mock-adapter
.
And if you ever drop axios
in favor of request
, you'd be required to write an adapter from the request
API to the axios
API.
Furthermore, when talking about querystring
(which is pretty much a module holding pure functions) and pipe
(a pure function), I don't see any actual gain in injecting it the way you're doing here.
So, in the end, simply writing:
import pipe from '@dev-master/pipe';
import querystring from 'querystring';
import request from 'axios`;
export default async function isQuestionable //...
Would not be less compliant to the Clean Architecture definition as your current version already is.
I understand this repository was created for didactic reasons and tries to follow the architecture definition as far as JavaScript allows it, but this is a really cool discussion I've been willing to have for a while and couldn't find someone to do that, so forgive my daring 😅.
I received the following question from a Dev Mastery newsletter subscriber who graciously allowed me to share it here:
Got some questions which might be related to it that trouble me and I can find a way through them..
- Lets say that there is a precondition to your application that it must have valid mongoDB connection before it can listen on port. ( start up).
- What happen if we want to preload data(from ext source) that our application must work with before we can start listening to requests. where do you do all that ?
All ur use cases looked like requirements that are business logic, but this is more of an architecture design.
who is responsible to satisfy these conditions, and how can we do it in a clean way
Hi, thank you for the great work + tutorial.
I think that DM_API_ROOT=/api
should be added to the sampledotenv
file as it would otherwise not work currently.
Hi, thank you for a great video and for sharing the code. I have a question regarding the end to end tests that jest runs:
Does the below line depend on the response from a call to a third party service? I am just trying to learn and understand the code.
comments-api/__test__/e2e.spec.js
Line 36 in 0d119f7
What I mean is, I can see makeFakeComment sets published to true on the comment, but it comes back as false in the response. I think I have answered my own question: I do not have Askimet or Content Moderator set up correctly?
Thanks again.
Hello @arcdev1 and thank you so much for the sample code and the amazing videos.
However I would like to discuss about the dependency injection topic.
As you can see from a sample project I made I usually enforce the dependency injection in order to have a great flexibility in testing as follows:
Connect to the database => pass the connection to the repository => pass the repository to the service => pass the service to the controller => pass di controller to the router => pass the router to the server
I'm also trying to figure it out if:
the clean pattern used in this project (comments-api) gives me the same testing flexibility that I explained above
how would you change my project in order to respect the clean pattern
I think that in order to better understand the pattern and the testing and future changes flexibility it would be great to add another entity which interacts with the main one and see in action integration testing with mocks as well
Does this mapping make sense(?):
Your post-comment controller is using the add-comment use-case.
The add-comment use-case needs the db and is using the comment entity.
You can unit test from the inside (entity) to the outside (use-case and so on) but (e.g) if I want to run a test from the very outside changing the implementation of one of the libs used by the entity my feeling is that I wont be able to do that
Does it makes sense as soon as I've more entities to have a central (e.g where I create the server) point where I:
Sorry for the long post, I hope that all my questions and examples let you understand my difficulties in approaching the clean architecture.
Maybe I'm also intending testing in the wrong way so I'm happy to improve myself and learn how to properly do it :)
Hi, the sampledotenv doesn't contain a DM_ENV,
Before the build the index.js code looks like this:
if (process.env.DM_ENV === 'dev') {
After build, the code looks like this:
if ((process && process.env && process.env.DM_ENV || "dev") === 'dev') {
which I think should equate to "dev" === 'dev' but somehow doesn't.
so index.js never starts listening:
app.listen(3000, () => {
until I add DM_ENV=dev to my .env file.
Which is kind of fundamental to you all running this code so are you all just adding DM_ENV and then not mentioning it because it's so obvious, or is something else going on that I just cannot see?
Hi @arcdev1,
This is not an issue with your code, only the tools, so feel free to close this issue.
After watching your video on Clean Architecture related to this repo I decided to try the pattern out.
There is, in my opinion, a large disadvantage to the dependency injection.
Visual Studio Code, the text editor I am using, looses IntelliSense on the module dependencies.
I tried changing the dependency injection from:
function ({ dep1, dep2, dep3 }) {}
to:
function (dep1, dep2, dep3) {}
hoping that it would improve the situation. It did not.
I'm using pure JavaScript, not TypeScript. I imagine I could fix the IntelliSense by adding jsDocs with type details to the functions however that seems rather redundant.
I know IntelliSense is not required however from a productivity point of view it will slow down development. For example If you don't remember the exact name of the function you are calling on a dependence you will need to look it up.
Suggestions welcome.
Thanks for the great insight this subject has given me.
Is is possible to use your structure As a code for a full project or is this code style only good for micro services and if I am to add design patterns for a more complex project based on this architecture is this a good or bad idea ?
Hey, first of all thanks a lot for sharing this code & your episode, it really had a huge impact on arch. style in my projects. But I have ran into an issue that I see is also present in your repository, and that is: let's say you have 2 concurrent updates on a single comment entity - that's not a problem if you have only text fields, but it might become one when you have i.e. an array inside a document - one operation will overwrite the other one (if there were 2 pushes, only one will be performed even though both should be merged somehow) - my question is, how do I deal with that, and most importantly - should I even care?
Hi Bill,
First, thanks a lot for sharing this code and the YouTube explanation, (we are still waiting for more : ).
It changes the way i think about architecture.
1: How would you implement Abstract model and abstract collection ?
Say you want to add cache to all your db queries of collection,
2: Say you have around 300 entities in your app (real case), all have 1 or more fields with same logic
(e.g strings field which need to be parsed, int field which you want to validate, date fields which you want to relate them to specific timezone)
How would you make code which can be adapt to all of them
would that be a DB concern or model concern ? or would you import at the top or extend with prototype the each model?
3: Did you ever use this architecture at this scale ?
Hello Bill,
I just wanted to try and mess around with the app similar to how you did it in mastery Monday by doing a POST with a body of {
"author":"Wayne",
"text":"Party time",
"postId":"cjt65art5350vy000hm1rp3s9"
}
However I only get 404 errors. Do I need to do something additional to get the routes working?
Thanks for your excellent example of applying Clean Architecture to a Node/JavaScript app. I have been working on refactoring an existing app to Clean Architecture and I have arrived at a solution quite similar to yours. As an aside, I was surprised at how the code trends to a very functional programming feel. Anyway, I have hit a bit of a wall trying to integrate authentication and authorization into Clean Architecture.
Suppose, as in many real world cases, your app has administrative use cases that are restricted to users with admin privileges. Also suppose, as Uncle Bob describes repeatedly, that your app has a web API that a web app GUI consume, as well as console API (which I might actually implement) that might authenticate based on the OS user or maybe even get an authentication token and store it in the user's home directory. Where would you place the logic that authenticates the user invoking the use case? When using Express to build a web API, I'm sure you're aware the common solution is to use middleware that extracts an authentication token from HTTP headers and uses the token to look up the associated session and user in some session store and user database. With respect to this, here some questions that have caused me to stumble.
interface CommentsPrivilegeService {
currentUserCanEdit(comment: Comment): Promise<void>
currentUserCanDelete(comment: Comment): Promise<void>
}
and injecting an implementation of that service into the use case. That's where I ran into trouble, though, because during an HTTP request that, there is no way to track the user from the request headers through to the use case without actually passing the user or the user ID explicitly to the use case, then passing the user to the CommentsPrivilegeService
methods.
So, I see three options.
async handleEditComment(httpRequest: CommentsAppHttpRequest): CommentsAppHttpResponse {
// this was injected into the controller
const userId: UserId = userIdFromSessionTokenInHttpRequest(httpRequest)
const privileges: CommentsPrivilegeService = makePrivilegeServiceForCurrentUser(userId)
const editComment = makeEditComment(privileges)
const editCommentArgs = argsFromRequest(httpRequest)
const result = await editComment(editCommentArgs)
return CommentsAppHttpResponse.from(result)
}
I am sure there are best practices for authentication and authorization in clean architecture and domain driven design, but I am having trouble coming up with anything concrete in the context of Node/JavaScript researching on the web. I have seen examples in Java and .Net that use AOP frameworks to authenticate and authorize outside the use cases, while the use cases assume the calling user is authorized, but I like the idea of writing the use cases to explicitly call out the authorizations they require.
I am very interested to know your thoughts.
So you are using factories to implement dependency injection, taking advantage of closures. Is this a known pattern, does it have a name? I can only assume that is used in FP style languages? Is it used in Java?
Trying to use it with an inner class instead of inner function.
function makeComment({Id, md5, sanitize, makeSource}) {
return class Comment {
constructor(
author,
createdOn = Date.now(),
id = Id.makeId(),
source,
modifiedOn = Date.now(),
postId,
published = false,
replyToId,
text
) {
....
}
}
}
The fact that it is using dependencies from outer scope makes me feel like breaking encapsulation (but we know that a javascript class is a function constructor). Maybe because of the habit of passing all needed dependencies to the constructor.
Next code from use case add-comment has race condition error in design.
If two the same requests will be received at the same time,
then if (exists)
will be false for each of them
and two comments with same hash will be inserted
...
const exists = await commentsDb.findByHash({ hash: comment.getHash() })
if (exists) {
return exists
}
const moderated = await handleModeration({ comment })
const commentSource = moderated.getSource()
return commentsDb.insert({
...
This example app has no business rules.
Crud methods can be mapped to controller without need of use cases.
If I need to add real business logic this error will be critical e.g:
How can I avoid this error in clean architecture?
I see that the factory builders for entities like comment
accept Id
as an argument. Is the implementation of Id
in the Id folder considered part of the "framework" layer in clean architecture because it's relying on an external dependency (cuid) directly?
How should I be thinking about this?
Hi man, first of all: thank you for your contribution for the community !
I was looking at this piece of code, and I don't understand why you import the DB and inside this code you use logic technology-specific... what's the point ?
I mean... If I need to change the data base I need to change makeDb origin, and ALSO this technology specific logic.
Am I right?
Thanks !
Thank you @arcdev1 for the great article and repo on clean architecture.
I have a few questions related to use case creation in clean architecture.
I am working on an application which moves mongodb records to elasticsearch and I am trying to develop it using clean architecture approach.
At high level, I could think of only one use case for this application which is :- moveDatafromDbToEs.
But it will involve several steps like :-
1)Fetch data from db in a batch
2)prepare/Format the records to be pushed to elasticsearch
3)Bulk Push data to elasticsearch
4)Filter failed and successfully pushed records.
5)Create a final response with how many records were pushed successfully and how many records failed.
Usecase creation approach:-
Method 1:- Create one big use case with all the above steps as small functions within it.
Method 2:- Break the big use case into small use case For eg:- a use case that handles elasticsearch push related operations 1) Format data, 2) push data to elasticsearch, 3) filter which records successfully pushed and which failed ) and inject this usecase as a dependency in main use case (moveDatafromDbToEs). This approach is similar to handlemoderation usecase of your application.
Question 1 :- If I follow the method 1 Won’t it make the use case look very big and break srp.
Question 2:- If I go with method 2, is it ok to have one usecase dependent on other usecases
Question 3 (in general ):- Is it ok to have a usecase having more than 3-4 dependencies.
Awaiting your response. Thanks
Hi, first of all thanks a lot for your video explaining clean architecture and the example service. It really was of great help. I have a question on my mind, how to support api error responses with attached error codes and subcodes without allowing the usecase to know which http status and error codes | subcodes returned through the api. The first option that came to my mind is to create custom errors for the usecases to throw and in the controller layer I would map those errors to http status codes and service error codes. The second option is to store the mapping within the custom error.
If the first option is viable should the mapping be based on error messages or error types.
Thank you.
class AddUser{ constructor(db){ this.db = db; } async addUser(){ // logic } }
like this?
Hi,
Firstly, thanks for the tutorial; I’ve spent the weekend refactoring my node js monolith and it is so much cleaner, and unit testing has never been so easy!
To my question: what is the difference between using classes vs factory functions in this architecture? Is there any difference or is it just semantics? Performance implications? I’m using TypeScript, do you think classes are a better fit in this case?
It would be nice if you can make typescript version of this, thank you!
comments-api/src/use-cases/index.js
Line 7 in 7c93f3d
Shouldn't the import be
import isQuestionable from '../is-questionable'
?
Thank you for your awesome videos and examples btw !
Hi Bill, I am confused about the build and make methods and was wondering if you could provide some clarity on them.
When looking at comment.js it seems to me that buildMakeComment() is used to inject dependencies and makeComment() is used to construct the comment entity. However in other files you use the word make as a prefix for functions that take in dependencies. So I don't think that is their purpose.
I am really interested in Clean Architecture and your video was amazing, your response will help get me closer to implementing something similar in my applications.
Many thanks in advance,
Andy
First, I'd like to give you credit, this repo and the related video are amazing and have helped me grasp a lot of Clean Architecture concepts I've been having issues with. One thing I'm still having an issue with is making the switch from my reliance on using Mongoose models as the Business Entities to placing Mongoose on the outter circle.
I've seen your comments on the youtube video, suggesting we use Mongoose as just another framework to inject into the Comments-db. But, would that free us from writing the code that's in comments-db.js? That file seems like it covers a lot of what Mongoose does (e.g. findbyid.. findall..). Which, for my applications that tend to not be monolithic MEAN stack apps, it seems a bit excessive to do for each Entity.
Could you suggest whether you think that, for a monolithic API, could we save on boilerplate code and leverage Mongoose or should we just give up on mongoose and build a robust '-db.js' dbo file for each entity?
thanks!
Hi @arcdev1, first of all thank you so much for this repo and the lesson about clean architecture on Youtube. I learnt a lot from it and now I'm using both as a guideline for my first own node app.
In data-access/index.js
there's something I had to do slightly different because it wasn't working the way you did it, which is here:
In my case, the variable declared in .env
(URI in my case) is undefined because (as I found out) the whole data-access module gets executed even before src/index.js
does, which contains dotenv.config(),. I solved this by moving URI and the instantiation of MongoClient inside the makeDb
function, which gets executed only once the server is started.
I wonder what happened in my case would be expected and if there is any drawback with my solution?
Cheers.
Hello @arcdev1 and thank you so much for the sample code and the amazing videos.
I am facing signup issue while creating this reference url "https://contentmoderator.cognitive.microsoft.com" which you given.
I got this below error always, even I have good internet connection.
Can you suggest me, how to resolve on this same. Thanks in advance sir
Hey Bill, thanks for the awesome video! Just a question...
Why are you injecting an use-case in a controller, which is at a lower level compared to an use-case? Wouldn't be a source-code dependency in this case more "right"?
I was wondering if data-access methods should be returning entities. In this case, we could call makeComment
on each result to have a list of entities rather than a list a plain javascript objects.
In javascript we can't see the difference, but in a typed language (like Typescript) we would.
I know you don't have a formal Repository file to abstract away your data access concerns, but how do you handle errors with MongoDB in your add-comment
Use Case and Repo?
Errors having to do with validation or duplicate emails, for example, are, to me, a data layer concern (at least, the specific error message from the DB Driver should be transformed to something more useful to the domain layer).
Thanks.
Hi, how do we go about implementing logging? Do we just import a logger function anywhere in our use cases or pass it down from the frameworks & drivers layer?
Why do you inject the add-comment
in the controller post-comment
. Should inner layers not be imported into outer layers usually? I can not wrap my head around this. On all other layers you imported the inner ones and injected the outer layers.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.