Review system improvements about itch.io HOT 8 CLOSED

I'm aware that 5 star ratings are fundamentally flawed because people either pick 1 or 5 but there were a couple considerations that went into this. My approach to ratings is probably going to be a little different than other sites. Instead of ratings as a stand alone thing I want them to be highly associated with the user that made them. For example leaving a 5 star rating versus 4 star rating is a method of categorization. That way either the user or someone viewing the user can easily discern what is good and what is really good when looking at the games.

Just showing an average "these are the best ranked games out of all users" will be nice but I think things could be done better. The problem with this method is that you're not going to satisfy people with different tastes because it's just an average rating. For the time being I don't intend to use the average rating for anything.

Ideally you visit a user's page that you're interested in and you see which games they gave a 5, which games they gave a 4, etc. I think this is a much more powerful way of finding new games that are interesting and it lets less well known content circulate easier. Content discovery through users instead of through itch.io's browse pages is a good direction and this is part of getting there.

Anyway, even with all these plans I can't really predict what will happen. As long as I'm collecting the rating data I can see how things are trending and make a decision. If it turns out everyone rates 1 and 5, and no one cares about categorizing the games then I can always convert the 5 star system to like/dislike, but I can't go the other way around.

Also reviews are coming.

from itch.io.

I see. Yeah I guess that's the best approach (trialling it for a bit to see how it works out). Looking forward to seeing how it all pans out.

from itch.io.

Did you check whether the player has ever downloaded the game. I doubt that you can check the time the user played the game like on steam.
Maybe at least for html games!?
Some stats how long a user has spend for a webgame would also be cool!

from itch.io.

Can you please implement something like this if you ever use the ratings for discovery/ranking?

http://www.evanmiller.org/how-not-to-sort-by-average-rating.html

from itch.io.

For the time being I don't intend to use the average rating for anything.

https://itch.io/games/top-rated/

from itch.io.

@aaronfranke the text you quoted is 4 years old. Are you suggesting you would prefer not to see a way to sort games by their rating? (Btw that page doesn't use pure average, but a normalized average that takes into account confidence level through number of ratings, and volume of ratings)

from itch.io.

No, I am not. I do prefer the ability to sort games by rating.

I only quoted the text and posted the link to show information relating to this issue, in hopes of resolving it easier. Specifically, information which is no longer valid.

from itch.io.

@aaronfranke Ah thanks.

I'm going to close this issue. We are going to be revamping the rating system again, moving a way from 5 stars, but not a recommend/not recommend system.

from itch.io.