Giter Club home page Giter Club logo

cheesegull's Introduction

CheeseGull Build Status

CheeseGull creates an unofficial "slave" database of the official osu! beatmap database, trying to keep every beatmap up to date as much as possible, as well as a cache middleman for osz files.

The main purpose for this is, as you can see from the owner of the repository, for running a sort of "beatmap mirror" for Ripple's osu! direct. We originally used an actual osu! beatmap mirror which had all of the beatmaps on osu! downloaded, but it ended up taking wayyyy too much space, and the cheapest server we could find that had at least 2 TB of HDD had an upload speed of 30mbit - as you can guess, this meant that for the Ripple users who didn't have a third world connection the download speed was pretty poor.

CheeseGull tries to hold a replica of the osu! database as updated as possible. Though of course, not having any way to see the latest updated beatmaps, or have a system for subscribing to updates to beatmap, or anything else which could help us identify what has been updated recently makes it very hard to do keep an updated copy at all times (Takeaway: the osu! API is completely shit). In order to do this, CheeseGull updates WIP, Pending or Qualified beatmaps when at least 30 minutes have passed since the time they were checked, whereas for all other beatmaps (including Graveyard, Ranked, Approved, etc) at least 4 days must have passed. This is not a problem for ranked/approved (it's highly unlikely for a ranked beatmap to ever change state, and Graveyard beatmaps are rarely resurrected, so there's that).

Beatmap downloads are also provided by going at /d/<id>. In case the beatmap is not stored in the local cache, the beatmap will be downloaded on-the-fly (this assumes the machine's internet connection is fast enough to download a beatmap before a HTTP timeout happens). In case the beatmap already is in the cache, then well, as you can imagine, it is served straight from there. Oh, yes, multiple people downloading a not cached beatmap at the same time is a case we handle. Or should be able to handle, at least.

Getting Started

You can find binaries of the latest release here.

If you want to compile from source, if you have Go installed it should only be a go get github.com/osuripple/cheesegull away.

The only requirements at the moment are a MySQL server and an osu! account. Check out cheesegull --help to see how you can set them up for cheesegull to work properly.

Contributing

No strict contribution guide at the moment. Just fork, clone, then

git checkout -b your-new-feature
code . # make changes
git add .
git commit -m "Added thing"
git push origin your-new-feature

Go to the GitHub website, and create a pull request.

Sphinx set-up

If you want to test search using Sphinx, you will need to set it up. Here is the sphinx.conf used in production, you probably only need to change lines 23-35

To index the data, you'd then need to run sudo indexer --all. If you want to run the indexer without having to shutdown sphinx, run sudo indexer --all --rotate. In production, this is run as a cronjob every 5 minutes.

(No, we're not using ElasticSearch. Search is meant to be fast and not take too much memory. Any Java solution can thus be tossed away since it does not suit these basic two requirements.)

License

Seeing as this project is not meant exclusively for usage by Ripple, the license, unlike most other Ripple projects, is MIT.

cheesegull's People

Contributors

mempler avatar thehowl avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.