Giter Club home page Giter Club logo

tweetback-canonical's Introduction

tweetback Twitter Archive

Take ownership of your Twitter data. First talked about at Jamstack Conf 2019 and in this blog post.

Demos

Features

  • Built with Eleventy
  • Each tweet has its own independent URL (with backwards/forwards threading!)
  • Uses @tweetback/canonical to resolve other Twitter archives URLs (internal links stay in the archive and don’t link out to Twitter).
  • t.co links are bypassed and original hyperlinks URLs are used.
  • Links to users, tweets, non-truncated URLs.
  • Nicer link formatting for links-to-tweets: @username/:id.
  • Support some markdown: I sometimes use backtick markdown notation for code in my tweet text. This translates to <code> properly.
  • Analytics:
    • See your most popular tweets
    • Who you retweet the most
    • Who you reply to the most
    • Frequently used swear words
    • Top emoji
    • Top hashtags

Usage

  • Clone/download this repository
  • In your terminal, cd to the folder of the project
  • Install Node.js
  • Run npm install

Populate the database from your Twitter Archive zip

  1. Copy ./data/tweets.js from your Twitter Archive zip file into the ./database directory of this project.
    • Rename window.YTD.tweet.part0 in tweets.js to module.exports
  2. If you want to exclude Twitter Circles tweets (these are included in the archive, why 😭): copy ./data/twitter-circle-tweet.js from your Twitter Archive zip file into the ./database directory of this project.
    • Rename window.YTD.tweet.part0 in twitter-circle-tweet.js to module.exports
  3. Run npm run import or npm run import-without-circles

Build the web site

  1. Edit the _data/metadata.js file to add metadata information.
  2. Run npm run build (will just create the proper files) or npm start (will run a server to look at them in your browser).
    • Optional: If you want the web site to live in a subdirectory (e.g. /twitter/), use Eleventy’s Path Prefix feature via the command line --pathprefix=twitter or via a return object in your configuration file. Careful: this is an option to Eleventy and not npm, so it needs to live after a -- separator (for instance, npm run build -- --pathprefix=twitter).

⚠️ Warning: the first build may take quite a long time (depending on the size of your archive), as remote media is fetched/downloaded into your project locally. Repeat builds will be much faster.

Fetch additional tweets from the API (optional)

If you want to fetch additional tweets from the API and put them into your sqlite database:

  1. You will need a twitter developer token an a TWITTER_BEARER_TOKEN environment variable (from the Twitter API v2). Read more about App-only Bearer Tokens.
  2. Run npm run fetch-new-data

Add your production URL to @tweetback/canonical (optional)

https://github.com/tweetback/tweetback-canonical has a mapping.js file that stores the existing twitter username => canonical URL mappings. These will be transformed automatically to point to other archives in all tweetback instances.

Publish your archive (optional)

tweetback-canonical's People

Contributors

aarongustafson avatar accudio avatar aldreth avatar bandit avatar bmann avatar chrisburnell avatar christopher-hayes avatar cjdunn avatar colinaut avatar cooljeanius avatar dryan avatar edm00se avatar ginader avatar hteumeuleu avatar j-f1 avatar jacklorusso avatar jakejarvis avatar jcolag avatar jefflembeck avatar jeffsikes avatar julianoe avatar kamblack avatar ksbarnt avatar mvsde avatar nhoizey avatar reedyn avatar reinhart1010 avatar semantic-release-bot avatar xdesro avatar zachleat avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

tweetback-canonical's Issues

Force trailing slash instead of preserving?

Right now this repo preserves trailing slashes or no trailing slashes given the source URL, for example

https://twitter.com/zachleat/status/123

// returns
https://www.zachleat.com/twitter/123
https://twitter.com/zachleat/status/123/

// returns
https://www.zachleat.com/twitter/123/

I wonder if this default should be changed, given that I think URLs from Twitter (exclusively?) leave off the trailing slash.

This causes a problem when redirects are not in place. Just as an example I’ve noticed that @matthewp’s URLs don’t work without trailing slash: https://matthewphillips.info/tweets/1592558080667312130 doesn’t work and is what is currently given back from this package.

A trailing slash needs to be added for it to work: https://matthewphillips.info/tweets/1592558080667312130/

Should we always force a trailing slash? It’s certainly the safest thing for tweetback archive output (but most hosts fix these redirects automatically, see https://www.zachleat.com/web/trailing-slash/)

One other idea here would be to force the convention in https://github.com/tweetback/tweetback-canonical/blob/main/mapping.js. If the source has a trailing slash, force it. If not, leave it off.

Alphabetize `mapping.js`

Just a suggestion; it might make it easier for new contributors to figure out where to put their own entry if existing ones were alphabetized. If there's some other sort of ordering, then maybe include that in a comment instead?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.