pauljamescleary / scala-pet-store Goto Github PK

View Code? Open in Web Editor NEW

1.1K 36.0 206.0 1.33 MB

An implementation of the java pet store using FP techniques in scala

License: Apache License 2.0

Scala 80.08% Shell 1.97% Python 17.96%

scala tagless-final reference circe http4s rest doobie typelevel rest-api cats

scala-pet-store's Introduction

Scala Pet Store

An implementation of the java pet store using FP techniques in scala.

Thank you!

Special thank you to Zak Patterson who also maintains this project; as well as the many contributors who continue to improve the pet store

Status

I have most of the endpoints in place. There are few big pieces remaining in case y'all want to lend a hand:

Create a ScalaJS React front end
Build tests using scala check in an idiomatic sane way
Create a microsite documenting the patterns in the pet store. Kind of a mini-book

Want to help out?

Join the chat at Scala Pet Store gitter.im

If you have general feedback on how things could be better, feel free to post an issue / gist or open a PR! I am looking for ideas like:

If you are a FP or TypeLevel contributor, let me know if something is not idiomatic as per FP or one of the TypeLevel libs I am using. I want this app to be an example of how things should be done as much as possible.
If you are an OO dev new-ish to Scala, let me know if something is confusing. I am trying to keep this project approachable. Feel free to email me or open an issue.

Why you doing this?

The goal for this project is to demonstrate how to build an application using FP techniques in Scala. When starting out in Scala coming from a Java background, it was extremely difficult to piece together all of the little bits in order to make a cohesive whole application.

How are you building it?

As the goal of the project is to help Java / Spring folks understand how to build an application in Scala. I hope to use good practice / conventions using FP and Scala, while maintaining approachability for OO peeps.

I will reach out to Java developers along the way, to see if techniques that I use are too confusing, or have a low enough barrier to entry to pick up quickly.

What is your stack?

I am going to work with the TypeLevel stack initially and see how far I can go with it. I believe that framing the concepts in code in an easy to understand way should be possible with Typelevel.

Http4s as the web server. I could have gone with finch, twitter server, or akka-http here as well, but I have been interested in learning http4s
Circe for json serialization
Doobie for database access
Cats for FP awesomeness
ScalaCheck for property based testing
Circe Config for app config
Tagless Final for the core domain.

What is going on here!?

This project has developed over-time and has embraced some traditional OO concepts, in addition to modern FP concepts and libraries. Let's talk about the foundational design patterns that emerge in the pet store.

Domain Driven Design (DDD)

Domain driven design is all about developing a ubiquitous language, which is a language that you can use to discuss your software with business folks (who presumably do not know programming). The key concept is that language surfaces in your code. Gotta thing called a "Pet", I should see a Pet in my code. I strongly recommend the book Domain Driven Design by Eric Evans.

The book discusses a lot of patterns, some of those we see in play in the pet store. DDD is all about making your code expressive, making sure that how you talk about your software materializes in your code. One of the best ways to do this is to keep you domain pure. That is, allow the business concepts and entities to be real things, and keep all the other cruft out. For example, while it is valuable to know that a Transaction in banking relies on a Debit as well as a Credit; these concepts should surface in your domain someplace. However, HTTP, JDBC, SQL are not essential to your domain, so you want to decouple those as much as possible.

Onion (or Hexagonal) Architecture

In concert with DDD, the Onion Architecture and Hexagonal Architecture from Cockburn give us patterns on how to separate our domain from the ugliness of implementation.

We fit DDD an Onion together via the following mechanisms:

The domain package The domain package constitutes the things inside our domain. It is deliberately free of the ugliness of JDBC, JSON, HTTP, and the rest. We use Services as coarse-grained interfaces to our domain. These typically represent real-world use cases. We see a lot of CRUD in the pet store, but use cases can be things like withdrawl, or register in other domains. Often times, you see a 1-to-1 mapping of Services to Endpoints or HTTP API calls your application surfaces.

Inside of the domain, we see a few concepts:

Service - the coarse grained use cases that work with other domain concepts to realize your use-cases
Repository - ways to get data into and out of persistent storage. Important: Repositories do not have any business logic in them, they should not know about the context in which they are used, and should not leak details of their implementations into the world.
models - things like Pet, Order, and User are all domain objects. We keep these lean (i.e. free of behavior). All of the behavior comes via Validations and Services

Note that Repository is kind of like an interface in Java. It is a trait that is to be implemented elsewhere.

The infrastructure package The infrastructure package is where the ugliness lives. It has HTTP things, JDBC things, and the like.

endpoint - contains the HTTP endpoints that we surface via http4s. You will also typically see JSON things in here via circe
repository - contains the JDBC code, implementations of our Repositories. We have 2 implementations, an in-memory version as well as a doobie version.

The config package The config package could be considered infrastructure, as it has nothing to do with the domain. We use Circe Config to load configuration objects when the application starts up. circe config Provides a neat mapping of config file to case classes for us, so we really do not have to do any code.

What about dependency injection?

The pet store does currently use classes for certain things (some would argue this isn't very FP). There are lots of ways to do dependency injection, including function arguments, implicits, and monad transformers. Using class constructors is rather OO like, but I believe this is simpler for people with OO backgrounds to digest.

There is no spring, guice, or other dependency injection / inversion of control (IoC) framework at use here. The author of the pet store is strongly opinionated against these kinds of libraries.

Fitting it all together

The idea with FP in general is to keep your domain pure, and to push the ugliness to the edges (which we achieve in part via DDD and Hexagonal Architecture). The way the application is bootstrapped is via the Server class. It's job is to make sure that all the parts are configured and available so that our application can actually start up. The Server will

Load the configuration using pure config. If the user has not properly configured the app, it will not start
Connect to the database. Here, we also run flyway migrations to make sure that the database is in good order. If the database cannot be connected to, the app will not start
Create our Repositories and Services. This wires together our domain. We do not use any kind of dependency injection framework, rather we pass instances where needed using constructors
Bind to our port and expose our services. If the port is unavailable, the app will not start

What is with this F thing?

You see in most of the core domain that we use F[_] in a lot of places. This is called a higher kinded type, and simply represents a type that holds (or works with) another type. For example, List and Option are examples of types that hold other types, like List[Int] or Option[String].

We use F[_] to mean "some effect type". We can leave this abstract, and bind to it "at the end of the world" in the Server when we bootstrap the application. This demonstrates the idea of late binding, leave your code abstract and only bind to it when absolutely necessary.

When you see a signature like def update(pet: Pet)(implicit M: Monad[F]):, we are saying that the F[_] thing must have a Monad type class available at the call site.

In this application, we use cats effect IO as our effect type, and use cats for Monads and other FP type classes and data types. We could just as easily use scalazIO and scalaz in an alternative implementation without changing the code dramatically.

Getting Started

Be aware that this project targets Java 11.

Start up sbt:

sbt --java-home {your.java.11.location}

Once sbt has loaded, you can start up the application

> ~reStart

This uses revolver, which is a great way to develop and test the application. Doing things this way the application will be automatically rebuilt when you make code changes

To stop the app in sbt, hit the Enter key and then type:

> reStop

Testing

Building out a test suite using Python. The reason is that typically we want to run tests against a live environment when we deploy our code in order to make sure that everything is running properly in the target environment. It is reassuring to know that your code works across clients.

In order to run the functional tests, your machine will need to have Python 3 and pip, and virtualenv.

To install pip on a Mac, run sudo easy_install pip
Then install virutalenv sudo pip install virtualenv

To test out the app, first start it up following the directions above and doing reStart

Then, in a separate terminal, run the test suite:

> cd functional_test
> ./run.py live_tests -v

scala-pet-store's People

Contributors

Stargazers

Watchers

Forkers

ml10 hderms christopherdavenport tpolecat pierangeloc fanf agsmith som-snytt gclaramunt tharindudg debasishg michael-smith mitruly ramsinha comcastsamples britneywright ldrygala vicchugu diebauer yaroslavhq loicdescotte tbrown1979 jammerwock flavioprosperi jeperez ipoemi pjurczenko ultish pvillega huberttatar blmeadows berntan zakpatterson deepakmohanakrishnan07 bobstead nicflores gitter-badger soul-d fpaschos abelbryo branislavlazic yqian-chwy nebtrx nimaeskandary zmccoy serioussoftware luca-filipponi thofis vladkopanev cnguy sergaben pelluru robertlemmens wibisono nolledge ssanj ssun3 fthomas tirthankarmukherjee phill101 littlenag noraglinden guizmaii sshark scala-steward franklinchou 271 vaedama gitviscodeorg fagossa mmenestret kailuowang 2efper sethtisue faberpro gingerkirsch cipriannec diyankrasimirov ggiscan irumiha bunyod hiroki1117 tpartyka jfeng61 mathiassp abhijitsm ehufer caiofralmeida artvovc nadberezny gollen833 ayonsanyal suprf1y erlangxk chlin501 qburst-shyam dmytroorlov jimmi-chuks m99coder caobinzi

scala-pet-store's Issues

`UserEndpointsSpec` specs are not been executed

Firstable, excellent job with this sort of toy/teaching project. I love it! 🙌

Having said that, I can go straight to bussiness😁

I noticed none of the UserEndpointsSpec specs are been executed. If you take a closer look here, the for comprehension corresponds to an IO[Assertion] computation which never gets executed. That actually doesn't happen in the rest of the endpoints specs, which execute the computation by calling unsafeRunSync.

I'm trying to push a PR fixing it but I'm getting two weird http4s-circe decoding errors. I'm not sure why those are happening because standalone circe decodes both examples perfectly

Here are the errors:

The first one, if I use the arbitrary instance generation for User case class

InvalidMessageBodyFailure was thrown during property evaluation.
[info]     Message: Invalid message body: Could not decode JSON: {
[info]     "userName" : "",
[info]     "firstName" : "",
[info]     "lastName" : "",
[info]     "email" : "",
[info]     "hash" : "",
[info]     "phone" : "",
[info]     "id" : 1
[info]   }
[info]     Occurred when passed generated values (
[info]       arg0 = User(,,,,,,Some(1))
[info]     )

The second one, If I use the user value provided in the code

org.http4s.InvalidMessageBodyFailure: Invalid message body: Could not decode JSON: {
[info]   "userName" : "username",
[info]   "firstName" : "firstname",
[info]   "lastName" : "lastname",
[info]   "email" : "email",
[info]   "hash" : "password",
[info]   "phone" : "phone",
[info]   "id" : null
[info] }
[info]   at org.http4s.circe.CirceInstances.$anonfun$jsonOf$2(CirceInstances.scala:55)
[info]   at scala.util.Either.fold(Either.scala:189)
[info]   at org.http4s.circe.CirceInstances.$anonfun$jsonOf$1(CirceInstances.scala:56)
[info]   at cats.data.EitherT.$anonfun$flatMap$1(EitherT.scala:87)
[info]   at cats.effect.internals.IORunLoop$.liftedTree3$1(IORunLoop.scala:207)
[info]   at cats.effect.internals.IORunLoop$.step(IORunLoop.scala:207)
[info]   at cats.effect.IO.unsafeRunTimed(IO.scala:304)
[info]   at cats.effect.IO.unsafeRunSync(IO.scala:239)
[info]   at io.github.pauljamescleary.petstore.infrastructure.endpoint.UserEndpointsSpec.$anonfun$new$1(UserEndpointsSpec.scala:37)
[info]   at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
[info]   ...
[info]   Cause: io.circe.DecodingFailure$$anon$2: Attempt to decode value on failed cursor: DownField(password)

I think someone else ran into the first issue before. He must have tried to fix it using a fixed value for the User case class but forgot to execute the computation and therefore the spec turned successful when it was actually broken. All this is just speculation on my end.

I'll take a closer look at http4s-circe code later to check if I can spot the issue. Meanwhile, if someone has a clue on this I'll be glad to listen/reading about it in order to fix the issue.

Add Delete Order Endpoint

https://github.com/ariesprayoga/swagger-java-sample-app/blob/master/src/main/java/com/wordnik/swagger/sample/resource/PetStoreResource.java

Add User Login Endpoint

https://github.com/ariesprayoga/swagger-java-sample-app/blob/master/src/main/java/com/wordnik/swagger/sample/resource/UserResource.java

Fix up REPLACE INTO / put in Repositories

We used put and REPLACE INTO in our repositories. Unfortunately, these make it difficult to type check our queries.

We should more formally do insert v. update as opposed to just put

Keep the existing validation checks in the algebra as they demonstrate how to do validations (even though they would theoretically be redundant).

Update the database schema to put unique constraints.

Be sure to handle the situation where the insert and update fail due to constraints...insert would fail if the record already exists, update would fail if the record does not exist.

Hopefully this solves the doobie type checking.

Seperate out the REST interface from the core functionality.

This is to serperate out the core functionality from it's REST interface. This would give a chance to other REST frameworks to show how to integrate with the project.

This would make a new repo called: scala-pet-store-http4s

Add Get User by Name Endpoint

https://github.com/ariesprayoga/swagger-java-sample-app/blob/master/src/main/java/com/wordnik/swagger/sample/resource/UserResource.java

Enforce invariants in the models

Our entities / domain model have virtually no validations.

Add invariant checks so that no domain model can be created.

For example, for a user, the phone number must be a valid form, the email must be a valid form, first name and last name must only contain valid characters.

Can possibly add other invariants for Pets and Orders. The invariants to add can be arbitrary.

Also, looking for ideas on how to tie these into Circe serialization.

My current thinking is to have custom decoders that call out to the smart constructors, and somehow translate the ValidatedNels to an appropriate response.

Here is an example...

case class User(...)
object User {
  def apply(userName: String, firstName: String ... ): ValidatedNel[String, User] = ...

Add Database migration support

Add support for managing database schemas.

I use flyway, but am not terribly opinionated on the actual mechanism.

Implement Tracing and Stats

Hi @pauljamescleary Do you think it would be good idea to do instrumentation using OpenCensus tracing technique. I like https://github.com/census-ecosystem/opencensus-scala has support for http4s and doobie which are used in this project.

We can start with opencensus' io.opencensus.exporter.trace.logging.LoggingTraceExporter in the beginning, and think about the tracing backend later (I think it's pluggable in opencensus).

Implement Logging

Hi, do you think it would be a good idea to add logging since its such a common use case in every project?

Enforce uniqueness constraints using doobie

The way we check if something exists, or is unique, is done via Validations.

The correct way to do this in a SQL db application is to enforce those invariants in the data base using unique constraints.

Consider using http://tpolecat.github.io/doobie/docs/09-Error-Handling.html as a guide for handling these kinds of exceptions.

Use OptionT when appropriate

We use a lot of F[Option[A]] in the system. We should use OptionT[F, A] when appropriate as it gives us a lot more goodies.

Add github pages deployment to travis

On successful build from travis, the most recent generated docs should be deployed to the github pages site.

So far this has been done manually: https://github.com/pauljamescleary/scala-pet-store/deployments

Adding docs deployment to the travis configuration

Necessary steps:

⬜️ Add secure github api token to repository so it can be used in travis build
⬜️ Add tut build step to travis build, fail the build if tut fails.
⬜️ Deploy docs to github pages if successful, fail the build if deployment fails.

G8 template

Why not, amirite?

Update to cats 1.0.1

I tried this locally, and had no issues. I was simply holding off until we get official releases of our dependencies (doobie, circe, cats-effect, http4s, etc) that also use cats this way we can do it all in a single PR.

Add Update User Endpoint

https://github.com/ariesprayoga/swagger-java-sample-app/blob/master/src/main/java/com/wordnik/swagger/sample/resource/UserResource.java

Ordering and Filtering

Hello,

Im adding this issue more as a question mark. I myself use some rest service based on this project and one thing i find difficult to think about in a clean, non redundant way is filtering and ordering. Are there any plans on adding these features someday? Id love to see how to go about that in a nice way. Right now i just have doobie queries matching the different kind of filterings / ordering i need with the matching endpoints. This somehow seems like a very bloated way to do it. I envision maybe something like an Ordering middleware thats able to map orderby=(field name) and orderType=(asc/desc). This seems hard to do however because of the different types and domains involved, but im not experienced enough to judge on that ;).

What are your thoughts?

Error Handling

Hi,
thank you very much for this project. I had to look a while before finding a 'real life' example for a functional webservice and I am glad I have found one :)
Currently I am asking myself where to put a more finegraded error handling. For example:

A repository algebra defines a function like this:

def get(orderId: Long): F[Option[Order]]

In this scenario the Option type just represents an order being present or not. But what about other errors that might occure? How would I handle network errors, constraint violations, parse errors and all of that stuff? Would you switch to an Either type? Is this part of the service layer and somehow has to be represented by the context F[_]? I am not sure where to handle those errors if, for example, I would like to return a BadGateway error instead of a generic InternalServerError.

Add scala-steward to project

Scala Steward automatically submits PRs for upgrading dependencies.

Example PR by scala steward:
typelevel/cats#2660

Many typelevel projects use it:
https://github.com/fthomas/scala-steward/blob/master/repos.md

Relax scalac settings for REPL

With fatal-warnings, we can't use the repl...

scala> import cats._                                                                                    
<console>:11: warning: Unused import
       import cats._
                   ^
error: No warnings can be incurred under -Xfatal-warnings.

scala> import cats.implicits._                                                                          
<console>:11: warning: Unused import
       import cats.implicits._
                             ^
error: No warnings can be incurred under -Xfatal-warnings.

Having problem with sbt compile

PS C:\Projects\scala\petstore> sbt compile
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0
[info] Loading settings for project global-plugins from metals.sbt ...
[info] Loading global plugins from C:\Users\dmitry.shyryayev.sbt\1.0\plugins
[info] Updating ProjectRef(uri("file:/C:/Users/dmitry.shyryayev/.sbt/1.0/plugins/"), "global-plugins")...
Waiting for lock on C:\Users\dmitry.shyryayev.ivy2.sbt.ivy.lock to be available...
[info] Done updating.
[error] java.lang.ClassNotFoundException: $992f12684041ef4e0e20$
[error] at scala.reflect.internal.util.AbstractFileClassLoader.findClass(AbstractFileClassLoader.scala:64)
[error] at java.lang.ClassLoader.loadClass(Unknown Source)
[error] at java.lang.ClassLoader.loadClass(Unknown Source)
[error] at java.lang.Class.forName0(Native Method)
[error] at java.lang.Class.forName(Unknown Source)
[error] at sbt.compiler.Eval$.getModule(Eval.scala:584)
[error] at sbt.compiler.Eval$.getValue(Eval.scala:576)
[error] at sbt.compiler.Eval.$anonfun$eval$1(Eval.scala:129)
[error] at sbt.internal.EvaluateConfigurations$.$anonfun$evaluateDslEntry$1(EvaluateConfigurations.scala:249)
[error] at sbt.internal.EvaluateConfigurations$.$anonfun$evaluateSbtFile$6(EvaluateConfigurations.scala:172)
[error] at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:233)
[error] at scala.collection.immutable.List.foreach(List.scala:388)
[error] at scala.collection.TraversableLike.map(TraversableLike.scala:233)
[error] at scala.collection.TraversableLike.map$(TraversableLike.scala:226)
[error] at scala.collection.immutable.List.map(List.scala:294)
[error] at sbt.internal.EvaluateConfigurations$.$anonfun$evaluateSbtFile$4(EvaluateConfigurations.scala:172)
[error] at sbt.internal.Load$.loadSettingsFile$1(Load.scala:1137)
[error] at sbt.internal.Load$.$anonfun$discoverProjects$2(Load.scala:1144)
[error] at scala.collection.MapLike.getOrElse(MapLike.scala:127)
[error] at scala.collection.MapLike.getOrElse$(MapLike.scala:125)
[error] at scala.collection.AbstractMap.getOrElse(Map.scala:59)
[error] at sbt.internal.Load$.memoLoadSettingsFile$1(Load.scala:1143)
[error] at sbt.internal.Load$.$anonfun$discoverProjects$4(Load.scala:1151)
[error] at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:233)
[error] at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:58)
[error] at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:51)
[error] at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
[error] at scala.collection.TraversableLike.map(TraversableLike.scala:233)
[error] at scala.collection.TraversableLike.map$(TraversableLike.scala:226)
[error] at scala.collection.AbstractTraversable.map(Traversable.scala:104)
[error] at sbt.internal.Load$.loadFiles$1(Load.scala:1151)
[error] at sbt.internal.Load$.discoverProjects(Load.scala:1165)
[error] at sbt.internal.Load$.discover$1(Load.scala:862)
[error] at sbt.internal.Load$.loadTransitive(Load.scala:937)
[error] at sbt.internal.Load$.loadProjects$1(Load.scala:726)
[error] at sbt.internal.Load$.$anonfun$loadUnit$11(Load.scala:729)
[error] at sbt.internal.Load$.timed(Load.scala:1395)
[error] at sbt.internal.Load$.$anonfun$loadUnit$1(Load.scala:729)
[error] at sbt.internal.Load$.timed(Load.scala:1395)
[error] at sbt.internal.Load$.loadUnit(Load.scala:688)
[error] at sbt.internal.Load$.$anonfun$builtinLoader$4(Load.scala:484)
[error] at sbt.internal.BuildLoader$.$anonfun$componentLoader$5(BuildLoader.scala:176)
[error] at sbt.internal.BuildLoader.apply(BuildLoader.scala:241)
[error] at sbt.internal.Load$.loadURI$1(Load.scala:546)
[error] at sbt.internal.Load$.loadAll(Load.scala:562)
[error] at sbt.internal.Load$.loadURI(Load.scala:492)
[error] at sbt.internal.Load$.load(Load.scala:471)
[error] at sbt.internal.Load$.$anonfun$apply$1(Load.scala:251)
[error] at sbt.internal.Load$.timed(Load.scala:1395)
[error] at sbt.internal.Load$.apply(Load.scala:251)
[error] at sbt.internal.Load$.buildPluginDefinition(Load.scala:1312)
[error] at sbt.internal.Load$.buildPlugins(Load.scala:1242)
[error] at sbt.internal.Load$.plugins(Load.scala:1225)
[error] at sbt.internal.Load$.$anonfun$loadUnit$2(Load.scala:694)
[error] at sbt.internal.Load$.timed(Load.scala:1395)
[error] at sbt.internal.Load$.$anonfun$loadUnit$1(Load.scala:694)
[error] at sbt.internal.Load$.timed(Load.scala:1395)
[error] at sbt.internal.Load$.loadUnit(Load.scala:688)
[error] at sbt.internal.Load$.$anonfun$builtinLoader$4(Load.scala:484)
[error] at sbt.internal.BuildLoader$.$anonfun$componentLoader$5(BuildLoader.scala:176)
[error] at sbt.internal.BuildLoader.apply(BuildLoader.scala:241)
[error] at sbt.internal.Load$.loadURI$1(Load.scala:546)
[error] at sbt.internal.Load$.loadAll(Load.scala:562)
[error] at sbt.internal.Load$.loadURI(Load.scala:492)
[error] at sbt.internal.Load$.load(Load.scala:471)
[error] at sbt.internal.Load$.$anonfun$apply$1(Load.scala:251)
[error] at sbt.internal.Load$.timed(Load.scala:1395)
[error] at sbt.internal.Load$.apply(Load.scala:251)
[error] at sbt.internal.Load$.defaultLoad(Load.scala:69)
[error] at sbt.BuiltinCommands$.liftedTree1$1(Main.scala:829)
[error] at sbt.BuiltinCommands$.doLoadProject(Main.scala:829)
[error] at sbt.BuiltinCommands$.$anonfun$loadProjectImpl$2(Main.scala:800)
[error] at sbt.Command$.$anonfun$applyEffect$4(Command.scala:142)
[error] at sbt.Command$.$anonfun$applyEffect$2(Command.scala:137)
[error] at sbt.Command$.process(Command.scala:181)
[error] at sbt.MainLoop$.processCommand(MainLoop.scala:151)
[error] at sbt.MainLoop$.$anonfun$next$2(MainLoop.scala:139)
[error] at sbt.State$$anon$1.runCmd$1(State.scala:246)
[error] at sbt.State$$anon$1.process(State.scala:250)
[error] at sbt.MainLoop$.$anonfun$next$1(MainLoop.scala:139)
[error] at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:16)
[error] at sbt.MainLoop$.next(MainLoop.scala:139)
[error] at sbt.MainLoop$.run(MainLoop.scala:132)
[error] at sbt.MainLoop$.$anonfun$runWithNewLog$1(MainLoop.scala:110)
[error] at sbt.io.Using.apply(Using.scala:22)
[error] at sbt.MainLoop$.runWithNewLog(MainLoop.scala:104)
[error] at sbt.MainLoop$.runAndClearLast(MainLoop.scala:59)
[error] at sbt.MainLoop$.runLoggedLoop(MainLoop.scala:44)
[error] at sbt.MainLoop$.runLogged(MainLoop.scala:35)
[error] at sbt.StandardMain$.runManaged(Main.scala:138)
[error] at sbt.xMain.run(Main.scala:89)
[error] at xsbt.boot.Launch$$anonfun$run$1.apply(Launch.scala:109)
[error] at xsbt.boot.Launch$.withContextLoader(Launch.scala:128)
[error] at xsbt.boot.Launch$.run(Launch.scala:109)
[error] at xsbt.boot.Launch$$anonfun$apply$1.apply(Launch.scala:35)
[error] at xsbt.boot.Launch$.launch(Launch.scala:117)
[error] at xsbt.boot.Launch$.apply(Launch.scala:18)
[error] at xsbt.boot.Boot$.runImpl(Boot.scala:56)
[error] at xsbt.boot.Boot$.main(Boot.scala:18)
[error] at xsbt.boot.Boot.main(Boot.scala)
Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? [error] java.lang.ClassNotFoundException: $992f12684041ef4e0e20$
[error] Use 'last' for the full log.

Update Travis to run func tests on build

In travis, we need to background the pet store process, and then run the func tests.

Unclear on the best way to background start then kill the process in travis.

Add Delete User By Name Endpoint

https://github.com/ariesprayoga/swagger-java-sample-app/blob/master/src/main/java/com/wordnik/swagger/sample/resource/UserResource.java

Dockerize sbt build

The sbt-native-packager is already added to this project.

Break up the repository into multiple traits

A lot of the logic in the repositories looks replicated throughout all of the repositories. This may be better to break this up into reusable traits.

I.e.:

OrderService extends ... with Creatable[t] with Deleteable[T] with Updateable[T]

Add User Logout Endpoint

https://github.com/ariesprayoga/swagger-java-sample-app/blob/master/src/main/java/com/wordnik/swagger/sample/resource/UserResource.java

n

`sbt run` not working

When using sbt run to start the project, it appears that pureconfig.loadConfig cannot find the reference.conf file. It complains that the petstore key is missing even though it's there.

pureconfig.error.ConfigReaderException: Cannot convert configuration to a io.github.pauljamescleary.petstore.config.PetStoreConfig. Failures are:
  at the root:
    - Key not found: 'petstore'.

Add Create User Endpoint

https://github.com/ariesprayoga/swagger-java-sample-app/blob/master/src/main/java/com/wordnik/swagger/sample/resource/UserResource.java

Tut up some docs!

Waiting until I have flushed out the remaining features. Namely:

Finish the endpoints (i.e. stop being lazy)
Work on the domain
Would be nice to have a scalajs front-end

Start a tut microsite and doc this jawn up!

Use Refined for core domain?

I would like to enforce invariants in the core domain.

A few issues that always troubled me:

Do we have a separate "layer" for JSON vs. our core domain model?
How do we enforce invariants in our core domain model while remaining compatible with json and database serialization?

It maybe possible to do this with Refined + Circe Refined.

Travis is not connected

At: https://travis-ci.org/pauljamescleary/scala-pet-store
Getting the message:

We couldn't find the repository
pauljamescleary/scala-pet-store

Add Update Pet Endpoint

https://github.com/ariesprayoga/swagger-java-sample-app/blob/master/src/main/java/com/wordnik/swagger/sample/resource/PetResource.java

The Update Pet Endpoint should return a 404 Not Found if the pet being update does not already exist.

Switch to circe-config

For consistency, we should use circe-config instead of pureconfig.

kudo's

Just wanna leave some kudo's. "Real world", easy to read examples are somewhat scarce in the FP community. Allot of theoretical information is out there, but projects like these are whats missing imo. Seeing the knots tied together here shows me the real value of these FP constructs.

Thanks.

Add authentication middleware

Implement authentication middleware for logged in endpoints.

It appears as though TSec is the way to go here...

https://jmcardon.github.io/tsec/docs/http4s/auth-jwt.html

Need a few things here:

Need some kind of login endpoint. That endpoint will take the user and password and
generate the token
Need authentication middleware using TSec
Need to update any HttpServices that should be authenticated. Certainly placing orders

Add Find By Tags Endpoint

To achieve compatibility with https://github.com/ariesprayoga/swagger-java-sample-app/blob/master/src/main/java/com/wordnik/swagger/sample/resource/PetResource.java

Package structure

Hi,
thanks for this project, it's really helpful to get to know typelevel libraries. I get the clue behind splitting domain and infrastructure package.
While the domain package has a feature based structure, the infrastructure stays to component/technology based structure. But why?
Thinking further to add more domains like order, customer, shop and so on. Is it useful to place these domains into packages like doobie or inmemory. In case of several data sinks (like Elasticsearch, or a HTTP-REST service) it would be necessary to place implementations into packages like elasticsearch or http/rest

Would it be more significant to apply a feature based structure to the infrastructure package?

Greets

add Scala 2.13 crossbuild

I can easily imagine you have dependencies that aren't published for 2.13.0-M5 yet, but adding the cross build and seeing where it gets stuck would be a way to find out what's missing and then we can go nag those maintainers

Next Steps for the Pet Store

Background
The pet store is rather far along, but there a lot of ideas for next steps to make it better. These include things like:

Review all the bits collaboratively and make it more idiomatic / clearer
Create documentation to walk through concepts in the pet store
Implement some FS2 bits, for example reading messages off a message queue

So what now?
Would be great to have additional collaboration on the next steps, generate a backlog of issues, and get others to contribute.

Using this issue as a starter thread for ideas and will go from there.

What's been discussed?
The pet store could use an accompanying guide/tutorial that talks about the overall organization of the codebase, with the ability to inspect specific implementations (e.g. how we did this with doobie).

Could use a round of reviews for cleanup, perhaps making things more principled / idiomatic wrt the specific libraries being used.

Add Get Order By Id Endpoint

https://github.com/ariesprayoga/swagger-java-sample-app/blob/master/src/main/java/com/wordnik/swagger/sample/resource/PetStoreResource.java

Create Front End Application

The front end application should communicate securely (i.e. secure cookies / jwt / etc) to the backend api.

This will require some kind of authorization middleware to be setup.

The front end application should be a ScalaJS app and mimic the existing pet store app.

Use `requests.request(params=xxx)` for python client

Currently, we manually append the params to the url / query string using urljoin. Instead, we can send the query params in as a dictionary

Doc Request: Write up what patterns were used and why

This is a documentation request that there should be more documentation about how and why the project is structured as it is. I.e. Why is the PetService broken up into an algebra and what tagless is.

question: Why Resource in Server.scala?

Hi, I'm using your app as a reference for my pet project.

I get the idea of returning Server as a Resource, I understand, why we have config file as Resource.

But is it a good idea to have all the dependencies created in the same for expression? Multiple expressions had to be lifted to Resource and they are not resources essentially, there's no cleanup logic for them.

Also, correct me please if I'm wrong, it seems, that if we were to acquire any resources in that same for (for example, opening files), we'd hold resources open for as long, as our server is running

for {
  configFile <- Resource.fromCloseable ...
  ...
  server <- BlaseZerverBuilder ... .resource
} yield server // configFile will be cleaned up only after the server is cleaned up

So to my untrained eye it doesn't look like the best practice. I'd highly appreciate your explanations and suggestions on how to organize that piece of code in a better way if possible.

Improve enums

Currently, the enums we use are unsavory.

Let's use Enumeratum (and circe Enumeratum). Should be a seemless add library dependencies, imports where appropriate, and refactor the few enums that we have.

Remove unnecessary `unsafeRunSync`

First of all, apologies for not sending a PR directly.

In any case, there is an unsafeRunSync in Server to go from F[Stream[F, A]] to Stream[F, A]: this is unnecessary, prefer Stream.force(FofStream) instead.

Even better, avoid having F[Stream[F, A]] in the first place, and instead lift the actions returning F into Stream using Stream.eval, and have the whole for be in Stream. The reason for this is to allow creation of resources with Stream.bracket, which takes a finaliser and guarantees resource safety, and it's a very useful and common pattern when dealing with resources that e.g. need to be closed or cleaned up.

Update endpoints should be _PUT /<entity>/<entity key>_

This applies to both the updatePet endoint as well as updateUser.

Application gets blocked on db connection acquisition if more than 10 concurrent clients connect

I'm not sure if it's about this application or this is a property of all http4s / doobie / hikari applications. It seems that something gets stuck when there are more clients connected than available connections in the connection pool.

How to reproduce:

do sbt dist (to remove the sbt environment as a factor)
unpack and run the app
run siege http://localhost:8080/users

Use top, htop or whatever to monitor system activity. You will notice nothing is happening. After 30 seconds exceptions start popping up:

[scala-execution-context-global-33] ERROR o.h.s.service-errors - Error servicing request: GET /users from 127.0.0.1 
java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 30000ms.
	at com.zaxxer.hikari.pool.HikariPool.createTimeoutException(HikariPool.java:676)
	at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:190)
	at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:155)
	at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:128)
	at doobie.util.transactor$Transactor$fromDataSource$FromDataSourceUnapplied.$anonfun$apply$5(transactor.scala:260)
	at cats.effect.internals.IORunLoop$.cats$effect$internals$IORunLoop$$loop(IORunLoop.scala:87)
	at cats.effect.internals.IORunLoop$.startCancelable(IORunLoop.scala:41)
	at cats.effect.internals.IOBracket$BracketStart.run(IOBracket.scala:86)
	at cats.effect.internals.Trampoline.cats$effect$internals$Trampoline$$immediateLoop(Trampoline.scala:70)
	at cats.effect.internals.Trampoline.startLoop(Trampoline.scala:36)
	at cats.effect.internals.TrampolineEC$JVMTrampoline.super$startLoop(TrampolineEC.scala:93)
	at cats.effect.internals.TrampolineEC$JVMTrampoline.$anonfun$startLoop$1(TrampolineEC.scala:93)
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
	at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:85)
	at cats.effect.internals.TrampolineEC$JVMTrampoline.startLoop(TrampolineEC.scala:93)
	at cats.effect.internals.Trampoline.execute(Trampoline.scala:43)
	at cats.effect.internals.TrampolineEC.execute(TrampolineEC.scala:44)
	at cats.effect.internals.IOBracket$BracketStart.apply(IOBracket.scala:72)
	at cats.effect.internals.IOBracket$BracketStart.apply(IOBracket.scala:52)
	at cats.effect.internals.IORunLoop$.cats$effect$internals$IORunLoop$$loop(IORunLoop.scala:136)
	at cats.effect.internals.IORunLoop$RestartCallback.signal(IORunLoop.scala:351)
	at cats.effect.internals.IORunLoop$RestartCallback.apply(IORunLoop.scala:372)
	at cats.effect.internals.IORunLoop$RestartCallback.apply(IORunLoop.scala:312)
	at cats.effect.internals.IOShift$Tick.run(IOShift.scala:36)
	at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)

Stop the siege tool and observe the report:

** SIEGE 4.0.4
** Preparing 25 concurrent users for battle.
The server is now under siege...^C
Lifting the server siege...
Transactions:		           0 hits
Availability:		        0.00 %
Elapsed time:		       71.92 secs
Data transferred:	        0.00 MB
Response time:		        0.00 secs
Transaction rate:	        0.00 trans/sec
Throughput:		        0.00 MB/sec
Concurrency:		        8.22
Successful transactions:           0
Failed transactions:	          34
Longest transaction:	       61.85
Shortest transaction:	        0.00

Now if you restart the app and run siege again, but limit the number of concurrent "users" to 10:

$ siege -c10 http://localhost:8080/users
** SIEGE 4.0.4
** Preparing 10 concurrent users for battle.
The server is now under siege...^C
Lifting the server siege...
Transactions:		      134101 hits
Availability:		      100.00 %
Elapsed time:		       11.06 secs
Data transferred:	        0.26 MB
Response time:		        0.00 secs
Transaction rate:	    12124.86 trans/sec
Throughput:		        0.02 MB/sec
Concurrency:		        9.26
Successful transactions:      134101
Failed transactions:	           0
Longest transaction:	        0.03
Shortest transaction:	        0.00

Everything works OK. Why 10? This application does not configure the Hikari connection pool so defaults are used.

I am entirely unable to debug this but I thought I'd share my findings.

Create Integration Test Foundation

Create integration tests that stage data in H2 where necessary and hit the API endpoints.

We would like to avoid specific unit tests except for our core business rules. Most tests should be possible by staging data in the database and hitting the APIs directly.

Once the foundation is setup, we can create issues for testing the endpoints.

Also, should add test coverage here as well, possibly adding some minimum test coverage check.

pauljamescleary / scala-pet-store Goto Github PK

scala-pet-store's Introduction

Scala Pet Store

Thank you!

Status

Want to help out?

Why you doing this?

How are you building it?

What is your stack?

What is going on here!?

Domain Driven Design (DDD)

Onion (or Hexagonal) Architecture

What about dependency injection?

Fitting it all together

What is with this F thing?

Getting Started

Testing

scala-pet-store's People

Contributors

Stargazers

Watchers

Forkers

scala-pet-store's Issues

Necessary steps:

Recommend Projects

Recommend Topics

Recommend Org