Giter Club home page Giter Club logo

naptime's Introduction

Naptime

Making REST APIs so easy, you can build them in your sleep!

Build Status

Project Status: Alpha. Although Coursera has been using Naptime in production since 2013, and is responsible for nearly 100% of our traffic, expect a number of API and binary incompatibilities until we reach beta.

Coursera leverages interactive clients to provide a high quality learning experience. In practice, this means Javascript clients for the web, and native apps for iOS and Android. In order to avoid duplicated work, we re-use the same APIs across web and mobile clients. This has forced us to build generic, re-usable APIs. Naptime is the result of these efforts. Naptime helps server-side developers build canonical, RESTful, re-usable APIs.

Why Naptime?

We adopted Play! and Scala as part of our migration away from PHP & Python. We initially built APIs using the stock Play! APIs. Play!'s APIs are powerful and general, but we found we could trade off some of that power for a big gain in productivity and REST standardization. We believed that an opinionated, optimized framework could DRY out our code, and increase developer productivity. After a number of false starts, we built Naptime. Today, over 95% of new APIs at Coursera are built using Naptime. Developers like using Naptime because it helps them get their job done more quickly.

Naptime Principles

We've attempted to follow a few principles to help guide development. They are roughly:

  • Developer Productivity: Naptime is optimized for developer productivity above all else.
  • Canonical/Standardized: Naptime codifies a set of conventions to reduce ambiguity in generic REST APIs. It's much faster for both API and client developers to work on a myriad of products and features across our platform with these standardized APIs.
  • Type Safety: In our experience, due to the nature of our product, we've found that leveraging the compiler to catch bugs early is better for developers, in addition to taking advantage of IDEs (e.g. autocomplete) and other tooling.
  • Performance: If developers have to hack around performance problems, that ends up making more work not less. Naptime performance should be good, without compromising developer productivity.
  • Easy Learning: We strongly avoid DSLs and symbol operators. Additionally, despite leveraging advanced Scala capabilities (e.g. macros, path dependent types, the type-class pattern, etc.) to power the library, authoring a REST API should not require knowledge of quasiquotes or other advanced language features.

Using Naptime

To learn more about how to use naptime, check out our getting started guide!

naptime's People

Contributors

amory-coursera avatar brandontram-coursera avatar cliu587 avatar davidswinegar avatar dguo-coursera avatar dplaha-coursera avatar eeasley-coursera avatar fyi-coursera avatar gargrishabh75 avatar josh-newman avatar kyewei avatar mbarackman-coursera avatar mkovacs-coursera avatar ppaskaris-coursera avatar ptbarthelemy avatar saeta avatar tanonev avatar vishalkuo avatar vkuo-coursera avatar yifan-coursera avatar yunhaolucky avatar zhaojunz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

naptime's Issues

StringKey("!~").asOpt[IntArray] throws exception instead of returning None

StringKey.asOpt throws an exception instead of returning none if StringKeyCodec on input. This breaks the contract that this method should return None upon encountering an error.

import org.coursera.common.stringkey.StringKey
import org.coursera.naptime.courier.CourierFormats
import org.coursera.courier.data.IntArray

implicit val stringKeyFormat = CourierFormats.arrayTemplateStringKeyFormat[IntArray]
val key = StringKey("!~").asOpt[IntArray]

Expected:
None

Actual:

java.io.IOException: `,' expected but ! found line: 1 column: 1
  at org.coursera.naptime.courier.StringKeyCodec$StringListParser.handleParseErrors(StringKeyCodec.scala:382)
  at org.coursera.naptime.courier.StringKeyCodec$StringListParser.parse(StringKeyCodec.scala:353)
  at org.coursera.naptime.courier.StringKeyCodec$Parser.parseList(StringKeyCodec.scala:182)
  at org.coursera.naptime.courier.StringKeyCodec.readList(StringKeyCodec.scala:144)
  at org.coursera.naptime.courier.StringKeyCodec.bytesToList(StringKeyCodec.scala:140)
  at org.coursera.naptime.courier.CourierFormats$$anon$4.reads(CourierFormats.scala:342)
  at org.coursera.common.stringkey.StringKey.asOpt(StringKey.scala:32)
  ... 43 elided

`documentation` method does not print version on some resources where it should

We have a resource like

@Singleton
class EnterpriseVoucherResource @Inject() (
    manager: EnterpriseVoucherManager,
    siteAuths: SiteAuths,
    superuserAuths: SuperuserAuths)
    (implicit ec: ExecutionContext)
  extends CourierCollectionResource[EnterpriseVoucherId, EnterpriseVoucher] {

  override def resourceName = "enterpriseVouchers"
  override def resourceVersion = 1
}

When printing out the documentation for this resource, I'd expect the paths to look like enterpriseVouchers.v1/$id. Instead, they look like enterpriseVouchers/$id.

It looks like it must be hitting this branch:
https://github.com/coursera/naptime/blob/master/naptime/src/main/scala/org/coursera/naptime/router2/NaptimePlayRouter.scala#L164

Original conversation at: https://coursera.slack.com/archives/dragon/p1484855939000114

500's should return JSON responses

500's are currently returned as the default Play Internal Server Error page -- unfortunately, this causes downstream response parsing to fail.

Ideally, 500's should return a standard error JSON response with the proper error code.

We should support dynamic arguments for reverse relations.

in regards to #147, based on some of the changes to the resources themselves, it looks like arguments passed to the reverse relation are hardcoded; however, we're definitely going to want to be able to manipulate those reverse relations like any other resource / finder.

under the hood, we could potentially take a reference to the designated finder and infer the arguments from there.

here's an example of what i'm talking about:

query {
  courses {
    memberships(someParameter: true, someOtherParameter: false) {
      id 
    }
  }
}

where memberships is designated as a reverse relation on the courses resource.

/cc @bryan-coursera

Using StringKey.asOpt and CourierFormats.recordTemplateFormats with records containing typerefs can result in bad lazy vals

Suppose we have a custom class TestPositiveInt that represents positive integers and has a coercer to Int.

TestPositiveInt.scala:

package org.coursera.naptime.courier

import com.linkedin.data.template.Custom
import com.linkedin.data.template.DirectCoercer

case class TestPositiveInt(value: Int) extends AnyVal

object TestPositiveInt {
  object Coercer extends DirectCoercer[TestPositiveInt] {
    override def coerceInput(obj: TestPositiveInt): AnyRef = {
      Int.box(obj.value)
    }

    override def coerceOutput(obj: Any): TestPositiveInt = {
      obj match {
        case value: java.lang.Integer =>
          if (value > 0) {
            TestPositiveInt(value)
          } else {
            throw new IllegalArgumentException(s"$value is not positive")
          }
        case _: Any =>
          throw new IllegalArgumentException()
      }
    }

    def registerCoercer(): Unit = {
      Custom.registerCoercer(new TestPositiveIntCoercer, classOf[TestPositiveInt])
    }
  }

  Coercer.registerCoercer()
}

TestPositiveInt.courier:

namespace org.coursera.naptime.courier

@scala.class = "org.coursera.naptime.courier.TestPositiveInt"
@scala.coercerClass = "org.coursera.naptime.courier.TestPositiveInt.Coercer"

typeref TestPositiveInt = int

TestPositiveIntKeyTest.courier:

namespace org.coursera.naptime.courier

import org.coursera.naptime.courier.TestPositiveInt

record TestPositiveIntKey {
  value: TestPositiveInt
}

Now we try to parse a negative integer as this key type:

import org.coursera.common.stringkey.StringKey
import org.coursera.naptime.courier.CourierFormats
import org.coursera.naptime.courier.TestPositiveIntKey

implicit val stringKeyFormat = CourierFormats.arrayTemplateStringKeyFormat[TestPositiveIntKey]
val key = StringKey("-1").asOpt[TestPositiveIntKey]

Expected:
key is None

Actual:
key is instantiated of type Some[TestPositiveIntKey]. It has a lazy val called value. Upon accessing value (such as by printing key to the console), TestPositiveInt.Coercer.coerceOutput is run and causes an exception to be thrown.

The same error would have occurred if the validation was performed in the constructor of TestPositiveInt instead of the coercer.

Deprecate Gets in favor of MultiGets

As discussed with @saeta and @josh-newman -- gets are basically just a MultiGet of one id, so it doesn't make sense to write both Gets and MultiGets. We should add support to Naptime to convert a get to a multiGet of one element, and deprecate gets going forward. Developers will then be able to only write a multiget, and get support for both gets and multiget.

Always use provided executor context

There are a couple places where we use our own execution context. Instead, we should use a provided execution context.

naptime/src/main/scala/org/coursera/naptime/actions/RestAction.scala
62:  import play.api.libs.concurrent.Execution.Implicits.defaultContext

naptime/src/main/scala/org/coursera/naptime/actions/RestActionBuilder.scala
98:      import play.api.libs.concurrent.Execution.Implicits._

naptime-testing/src/main/scala/org/coursera/naptime/actions/RestActionTester.scala
44:      import play.api.libs.concurrent.Execution.Implicits.defaultContext

CourierFormats return insufficiently informative error messages

For example,

record Example {
  foo: string
  bar: int
}

implicit val f  = CourierFormats.recordTemplateFormats[Example]
Json.parse("""{"foo": 1, "bar": 1}""").validate[Example]

outputs

JsError(List((,List(JsonValidationError(List(Unsupported JSON type: class play.api.libs.json.JsNumber),WrappedArray())))))

This does not contain the path where the problem happened. In complicated models, this makes it very hard to figure out the reason for the error.

Union resolution results in an extra level of nesting

in a query like this:

query ($bundleName: String!) {
  RapidashAppsV1Resource {
    bundleName(bundleName: $bundleName, limit: 1) {
      elements {
        deploys: deployIds(limit: 10) {
          elements {
            id
            buildId {
              buildDetails {
                ... on org_coursera_rapidash_ProductionBuildDetailsMember {
          -->  org_coursera_rapidash_ProductionBuildDetails {
                    commitId {
                      summary
                      message
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}

the nested field org_coursera_rapidash_ProductionBuildDetails is unnecessary since we've already subselected the particular union type via the inline fragment. at this point, we should have resolved the RapidashBuildDetails. ideally this should look like:

buildDetails {
  ... on org_coursera_rapidash_ProductionBuildDetailsMember {
    commitId {
      summary
      message
    }
  }
}

or (stretch goal) renaming the union as well.

buildDetails {
  ... on productionBuildDetails {
    commitId {
      summary
      message
    }
  }
}

/cc @bryan-coursera

404's thrown within Naptime should return JSON

A 404 that is triggered within Naptime seems to return content-type text/html and the "Action not found" Play page.

Instead, these should return JSON with the proper error code.

Reproducible by hitting any GET request with a malformed ID

Query parameter parsing of Courier records ignores custom StringKeyFormat

If you use Courier records as query parameters, Naptime uses InlineStringCodec / StringKeyCodec to parse that parameter instead of whatever StringKeyFormat you specify.

We call into CourierQueryParsers here which calls into those codecs here.

If the Courier record query parameter contains any typedef fields that have a custom StringKeyFormat defined, that StringKeyFormat is ignored. Suppose, if I have a Courier model containing a UUID field and a base64 encoded field. The base64 encoded field uses a custom StringKeyFormat. If the record is in the ID position for a GET, the base64 decoding happens correctly. If the record is in the query param position in a finder, the decoding does not happen. Also, the decoding works properly if a Scala case class is used instead of a Courier record.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.