Giter Club home page Giter Club logo

emojitracker's Introduction

Emojitracker 💫

Emojitracker.com tracks realtime emoji usage on Twitter.

capture

Emojitracker is a complex project with a service-oriented architecture, and thus has now been split up into multiple repositories. This repository mainly just serves as table of contents of sorts. Major components of the project are listed below.

Note Since 2021, Emojitracker was acquired by Emojipedia, and is now maintained by them. You can find all of the repositories in the Emojitracker GitHub organization.

Architecture

Bad and somewhat outdated diagram of how the main parts fit together in production: emojitracker-infrastructure

Core Applications

The core applications of Emojitracker. Note that while these are open source for educational purposes, they are currently all rights reserved. Please contact me directly if you want to use them for something.

The core applications are now mostly located in the Emojitracker GitHub Organization, including some old deprecated stuff not mentioned above.

Libraries and Tools

Most of the generalizable and useful pieces of Emojitracker have been carved out into maintained open-source libraries. These libraries are all freely licensed (see individual repositories for details).

Emoji Encoding

  • emoji_data.rb Utility library for handling the Emoji vs Unicode nightmare (Ruby).
  • emoji-data-js Utility library for handling the Emoji vs Unicode nightmare (NodeJS port).
  • exmoji Utility library for handling the Emoji vs Unicode nightmare (Elixir/Erlang port).

Emoji Assets

  • emojistatic Generates static Emoji assets for a public CDN.
  • cssquirt Embeds images (or directories of images) directly into CSS via the Data URI scheme.

Streaming

  • sseserver High-performance Server-Sent Events endpoint for Go.
  • sse-bench Benchmarks and load tests Server-Sent Events endpoints.

The Emojitracker APIs

Emojitracker also provides a REST API and Streaming API.

Other Information

The narrative version of how version 1.0 of this project was built is in the Medium post "How I Built Emojitracker", and the followup "How I Kept Building Emojitracker". Note however, both are now quite out of date, and do not reflect a substantial amount of change over the past few years.

emojitracker's People

Contributors

bitdeli-chef avatar jeffreytierney avatar mroth avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

emojitracker's Issues

implement sinatra-assetpack

by moving to an asset pipeline we can reduce reliance on external CDN hosted libraries (although they are possibly still a good idea for popular libraries), and make sure responses are minified and concatted etc.

this would make it much easier to break up our JS/css into some more manageable files for our own sanity.

a number of local libraries were inherited in #20 so this is becoming a bigger issue.

performance profile client-side

I suspect the potential CPU consuming areas we are looking at are:

  • Rendering of CSS transitions
  • Overall page re-rendering based on content changes?
  • Javascript operations related to updating the page via jQuery
  • Javascript operations relating to consuming/processing the SSE stream
  • Browser cycles from consuming the HTTP stream feed before it even hits JS?

If we can start by figuring out what area causes most of the CPU usage, we can look at how to optimize it.

text rendering issue on iOS

top of text getting cut off in detail pages in iOS Safari.. now idea why it should be different from desktop safari, but apparently it is!

probably related to bootstrap somehow.
screen shot 2013-12-29 at 4 19 03 pm

Why some emojis are flashing green/red ?

Hello,

As you can see on this screenshot, some emojis are flashing green/red. I don't understand the logic behind the choice of the emojis that are being flashed, and there's no explanation in your doc. Could you enlight me ?
Screenshot_20210224_102852

Doesn't work when opened too many tabs

I so like your service, that I want to open it on more than one tab.
When I do it, I can open just only six of it. Then next attempts will fall into infinite loading.
How can I open more entitles than now?

Implement client-side error logging

Need to add something like Airbrake or whatnot, as #32 just evidenced, it is apparently possible for us to have breaking issues in production without noticing for almost a week. We need a better smoke detector here, and it will have the positive benefit of finding potential browser issues.

Is there a good free solution?

(Client-side tests would be good too but probably wouldn't have helped in that case, since if we're using a mock object it won't catch stuff stemming from upstream issues coming from the data layer.)

About deprecated streamer

Hi,
Do you deprecate Ruby and NodeJS versions because issue performance ?

Your feedback on why depreciation should be very interesting.

performance profile server-side feeder

Currently uses about ~12% CPU on a 3.4GHz quad-core Core i7 when consuming full stream (so half a full core?). Manageable but worth looking into whether there are some easy gains in efficiency -- as at 150-200 events per second this seems a little high?

trim byte count of scoreStream

this gets pushed constantly, so anywhere a single character can be removed will help. possible candidates include separating space and any extra carriage returns.

Trending (e.g. 24-hour) view

It'd be nice to be able to see which emojis had been used the most in the past e.g. 24 hours. For example, at the time of writing, ♻️ is the #3 emoji, but it's rapidly catching up to ❤️ (which is #2) due to du3a.org and zad-muslim.com. It may even be outpacing the #1 😂; it's hard to tell from the current view.

On a related note, a breakdown by tweet language would also be handy; I'm pretty sure ♻️ is only #1 for Arabic.

Mojifall: an alternative visualization of Emojitracker's data

First of all, thanks for your work on Emojitracker. I want to let you know about a project of mine that uses the data collected by Emojitracker and that could be fun for you to know about, at least.

So, what I have worked on is an alternative visualization of the emoji score chart. As comparing large numbers is a bit hard for people, and because bar graphs are boring, this viz aims to represent the frequency distribution of emojis as a waterfall, or mojifall if you may. There is already a demo available at:

demo.akselipalen.com/mojifall-openmoji-emojitracker/

For a bit of background, the project started as an approach to display the variety of great work done by OpenMoji organization. Soon the project found an interest to reflect the real-world use of emojis and we chose Emojitracker to be a suitable data source. For details about the development this far, see OpenMoji Issue #30.

Currently Mojifall reads a dump of emojitracker's data I downloaded some days ago via emojitracker-rest-api. In other words, it does not consume the API directly. However, I am interested to develop Mojifall further to visualize the current use of emojis, for example over the last 24 hours (see related issue #37). My motive here, in addition to being an open-source enthusiast and OpenMoji patron, is to find good real-world use cases for Sprinkler.js, an open-source stream-visualization lib I have been baking. Usage of emojis might be one of those.

Any thoughts? Also, if you feel Emojitracker could benefit from Mojifall in a way or another, I am happy to help.

adopt JS templating

Realistically, we only have one thing it makes sense to template-ify, which is the .styled_tweet class, but this will help make the code prettier and will be a good learning experience for myself.

Plan is to use Handlebars.js for this.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.