u-wave / http-api Goto Github PK

Not sure why this is happening suddenly, but the YT duration parser is throwing on valid durations. Probably should look into YT duration parsers on npm.

Currently using pbkdf2 with only 2048 iterations (recommended >100k). that's not very good. Switch to scrypt or bcrypt instead. Will break existing logins on the test server.

We called this thing api-v1, but I expect when this is stable enough for an actual v1.x release we won't be finished. We really can't do major bumps in the Web api bc all clients would have to update. We also can't really create a new repository for every api major version, since most likely the breaking changes will not be that big, or will only be in one part (eg only in the websocket server).

Instead, I think we should rename this to u-wave-http-api or something similar. All major versions will live in this repository side by side, and they'll all remain available on side by side routes. I have to look more into how other people do it, but right now I'm thinking that different Web api versions can call into the same controller files and use different controller methods only where the versions differ.

(brief bc on mobile)

churny as all heck, but:

https://nodejs.org/api/esm.html

Need to figure out a good time to do this, probably going to finish up the existing PRs and try not to make new ones in the mean time

Version 8.2.6 of babel-eslint was just published.

This version is covered by your current version range and after updating it in your project the build failed.

babel-eslint is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

rate limiting errors totally clog up our free bugsnag account, they aren't bugs so they really shouldn't be logged at all.

There have been updates to the babel7 monorepoundefined

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

This monorepo update includes releases of one or more dependencies which all belong to the babel7 group definition.

babel7 is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

(e; Maybe not to the wrong item, I'm not entirely sure. they don't apply to the correct item at least 😄 )

Example:

Request URL:http://u-wave.dev:8080/v1/playlists/563f390cf059363574f4d4dd/media/5663878c55c1ad895690fd5c?token=…

PUT data:
{"artist":"Marnie Stern ","title":"Ruler","start":0,"end":220}

Note the item id: 5663878c55c1ad895690fd5c

Response:

{"_id":"562babb5e5e6a8b72991703e","media":"562babb5e5e6a8b72991703d","artist":"Marnie Stern ","title":" Ruler ","__v":0,"end":220,"start":0}

Note the different item _id.

The original item remains unchanged.

Currently all known state is always returned from /now. Ideally however it would only return the bits that clients need.

It should accept a comma-separated string:

fields=user,playlists
fields=booth,users,waitlist

When someone starts DJing, their playlist "cycles", i.e. the first item is moved to the very end. A socket event (playlistCycle, perhaps) should be dispatched for this too, so clients also know to cycle their playlist.

Advance stuff happens here, for now:
https://github.com/goto-bus-stop/u-wave-api-v1/blob/8ad01591dfe77948312bbdbad3a72ba3066667d7/src/sockets.js#L222-L244

This error message is bizarrely unhelpful.

Unfortunately, MongoDB's error message is also bizarrely unhelpful:

MongoError: E11000 duplicate key error index: wlk_uwave.users.$slug_1 dup key: { : "reanna" }

So we should probably catch "user create" errors and check for:

• .code 11000 or 11001
• if the error message string contains "slug" or "username", throw a duplicate username error
• if the error message string contains "email", throw a duplicate email error

…and make sure those duplicate errors aren't hidden.

Currently api-v1 contains all of the things, and it's very tightly coupled with core. There should be a meaningful separation between the two so other plugins/interfaces/ can be added by server hosts easily, while keeping everything in sync.

Ideally we'd want some multiprocess support so we can have like millions of people online and do 0-downtime server upgrades (I mean, I assume that's how it works! not an expert. But we need to be able to do that, anyway.) in the long term.

Idea:
https://docs.google.com/drawings/d/1xq_F4mUoF0gsE_q5LW26fA2pElgTBf2zOKKkQaKq6AU/view

To prevent joining the waitlist without a song to play

Version 0.63.3 of rollup was just published.

This version is covered by your current version range and after updating it in your project the build failed.

rollup is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

https://www.npmjs.com/package/joi

Currently you can still send messages to the socket when you're logged out, if you connected with a valid session. That shouldn't be possible.

A simple-ish way to fix that would be to force-disconnect a logged-out user.

Another simple-ish way would be to replace the AuthedConnection with a GuestConnection in src/sockets.js.

Maybe on /playlists/search?query=$QUERY initially.

(So yeah, if #14 turns out good, it'd be AMAZING to have playlists, and the room history as search sources (like woa, abstractions).)

Currently API requests by users who aren't in the "online users" list, but do have a valid auth token, just go through. This can be a problem when a user does not connect to the socket, but does eg. join the waitlist, because now everybody else gets a "waitlist join" notification for a user who doesn't even appear to be online.

One way to fix this is to just block HTTP API requests when a connection is not alive. One better way would be to only block API requests that mutate state, so you can still view most things. Perhaps return something like, "You have to be online to do X"

Currently when you favourite a song, the existing playlist item is added to your own playlist:

https://github.com/goto-bus-stop/u-wave-api-v1/blob/ca8b5b2da6d04eea7c100fa27285884b4795d0c7/src/controllers/booth.js#L92-L101

That sort of works, but it means that the playlist item is shared. There are a few problems with this:

• One user editing the playlist item, will update the item in every playlist, even other people's playlists.
• One user deleting the playlist item removes the item model and removes it from that user's playlists, but it doesn't remove it from other users' playlists. So, people who grabbed a song that's later deleted, will end up with a nonexistent item in their playlist, which isn't nice for clients to deal with.

Solution: create a clone of the playlist item when favouriting a song, and add the clone to the relevant playlist.

…with things that need to be done on the web API end of things before it can be considered "publically usable". (i.e., by people who are not We ♥ Kpop)

Also, tradeoffs that were made for the sake of Getting It Working (Mostly)™, and what is useful to refactor.

Currently YouTube and SoundCloud queries are hardcoded into the search controller. Both should be abstracted into their own Source objects, with a common API, so we can much more easily add more source types later.

Initially they'll only need a search(query) method to get ~25 search results and return them as üWave-style objects ({ sourceType, sourceID, artist, title, duration }). That can then be used by the search controller, matched up with the media items that are already known, etc. (much like how it's done already)

They're a bit unusual on the user endpoints, needing POST requests to mute someone, DELETE requests to unmute, and with no way to list current mutes.

Instead we should have /mutes endpoints that you can PUT to to mute someone, DELETE to unmute, and GET to list.

Voting happens here:

https://github.com/goto-bus-stop/u-wave-api-v1/blob/master/src/sockets.js#L199

Needs to check that the booth is actually occupied.

To prevent servers from dying.

A good library could be https://github.com/tj/node-ratelimiter, which would work for both socket and HTTP request limiting. It can use our existing Redis connection from u-wave-core.

Tangential: Could also be used for limiting new socket connections, instead of just socket messages, to deal with massive user spam like Musiqpad saw on its first days.

Currently artist names and titles are handled pretty poorly. We should probably steal something like web-scrobbler's YouTube artist/title extraction:

https://github.com/david-sabata/web-scrobbler/blob/master/connectors/v2/youtube.js#L64-L109
https://github.com/david-sabata/web-scrobbler/blob/master/core/content/connector.js#L407-L422

That deals with all kinds of weird separators, and all the OFFICIAL VIDEO fluff as well, so we should have pretty good results without manual intervention 💪

Version 2.4.0 of eslint-plugin-import just got published.

This version is covered by your current version range and after updating it in your project the build failed.

As eslint-plugin-import is “only” a devDependency of this project it might not break production or downstream projects, but “only” your build or test tools – preventing new deploys or publishes.

I recommend you give this issue a high priority. I’m sure you can resolve this 💪

Your Greenkeeper Bot 🌴

There doesn't seem to be an easy way to move playlist items to the very end currently.

Would be useful to look into what other APIs that deal with lazy-loaded ordered lists do.

And once their turn ends, they're added back to the waitlist even if they're not online.

The simpler solution would be to remove the DJ from the booth when they leave.

ATM errors are handled entirely inside api-v1. It would be neat to have a hook when an error occurs so we can intercept it and in case of welovekpop.club report it via bugsnag.

I'm thinking of having an option for custom error handler middleware, since error reporting libraries usually know how to make nice stack traces from express middleware and maybe not from an 'error' event (not sure). for bugsnag that would be something like:

app.use(Bugsnag.requestHandler)
createWebApi(uw, {
  // options
  errorHandler: Bugsnag.errorHandler
})

Eg. for joining. If the joining user is the current DJ, the waitlist will happily accept them in.

Most playlist endpoints call toPlaylistResponse before returning a playlist to the client, but the createPlaylist controller does not. This is an issue because now the returned size of a newly created playlist is undefined instead of 0.

Solution: call toPlaylistResponse on the created playlist.

When an error occurs, currently the api replies with a single English string. That's not particularly helpful for clients: some errors might be recoverable, and some might be because of an internal error whereas others are because of incorrect user input. Sometimes multiple errors occur, and they are currently either concatenated or ignored.

Ideally we'd return an object with an errors property if errors occur, and list error objects there with useful metadata.

http://jsonapi.org/format/#errors would be a good format 😉

These votes should be ignored.

Something for @xBytez maybe? :)

Moving users to the end of the waitlist appears to move them to the start instead.

Any position that's at the end, or after the end of the waitlist should move the user to the end.

We probably want to add tests during #3—probably will be some refactoring going on, and that's a great opportunity to break things up into easily testable chunks.

Should do something like this:

https://en.wikipedia.org/wiki/Network_Time_Protocol

but like…over HTTP.

(This should also be included in the /now endpoint.)

Version 13.5.0 of joi was just published.

This version is covered by your current version range and after updating it in your project the build failed.

joi is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Your Greenkeeper Bot 🌴

The createPlaylistItem controller currently always uses the artist, title and start/end times from the global media object (the one derived from the source media). This overrides any custom tags that users may have entered. For example, when your playlist contains a video of a full album, and you've set it to only play 3 minutes of a song halfway, it'll revert back to playing the full album when you drag it to another playlist.

Instead, it should copy the custom artist, title, and start/end times, and only fall back to the global defaults if no custom properties are available.

There's a booth/skip endpoint but it only supports skips by moderators. However sometimes a user may want to skip their own turn as well.

Proposal: A POST request to /booth/skip without a body should skip the current user (if they're DJing.)

https://github.com/goto-bus-stop/u-wave-api-v1/blob/master/src/routes/authenticate.js#L18

This would broadcast a

{
  "command": "guests",
  "data": 7
}

message whenever the amount of connected guests changes.

https://github.com/goto-bus-stop/u-wave-api-v1/blob/master/src/routes/authenticate.js#L18

To prevent timeouts.

We need to standardise on a generic API response format, and also on the shape of the different types of objects that are in play.

I've looked into JSON-API a bit, which is really nice because it's standardised--but I'm not sure that it's exactly fit to our use case. Of course, it doesn't have to be for us to use it :)

Some important things to consider:

• Consistency. Currently responses contain either a JSON object (the data), or an error message as a string (with a non-200 response code), or a JSON object with metadata and the actual data on a property (paginated results do this, for example).
• Error reporting. This is something that JSON-API is really good at, it prescribes an error format with (meaningful) error codes instead of plaintext messages, and it can deal with multiple errors (useful for eg. a form that has multiple invalid inputs)
• Duplication. Requesting the room history can contain a lot of duplicate user objects, and requesting a user's history can contain a lot of duplicate media objects. This makes responses large, but it might not matter much with gzip because duplication is easy to compress.

JSON-API:

• Consistent ✓
• Error reporting ✓
• Duplication: good at it, but it's a bit strange? I'm not sure how it'd work in a real app, so may have to look more into that.
• Also contains a "links" field for API discovery, which is kind of a cool thing. You put links to other related API endpoints there so developers can copy-paste through them and so automated tools can index them/generate docs(!).
• Has client-side libraries in some languages to deserialise into class instances easily, for example.

That's here:

https://github.com/goto-bus-stop/u-wave-api-v1/blob/master/src/controllers/now.js