Giter Club home page Giter Club logo

fetchtwitterfollow's Introduction

Fetch Twitter Follow

Export a user's twitter follows or following to a CSV, including data on each user's description, links, etc.

Setup Steps

  1. Create an application under development on twitter https://developer.twitter.com/en/portal/projects-and-apps
  2. Give it a name and fill in some fluff on the purpose such as "testing or exploring the api"
  3. Note: As of Nov 15th 2022, projects now require elevated API access. Check here to sign your dev app up for it. It seems to be instantly granted currently.
  4. Open auth.ini to copy some stuff in
  5. Copy the api key, api key secret and bearer token
  6. Click in bottom right "App settings"
  7. In the Keys and Tokens page on the app settings, click Generate under "Access Token and Secret"
  8. Copy the access token and access token secret
  9. Fill in all the fields in auth.ini with the copied values

Running the program

  1. Open terminal where the .exe is, and make sure the other files are present (auth.ini, USER_FIELDS)

  2. CLI Options

    • -u
      • REQUIRED
      • The username of the account you want to fetch data from (like your username, no @ symbol)
    • -m
      • SEMI-REQUIRED
      • DEFAULT: following
      • Mode select. "following" for getting a list of accounts the user follow. "followers" for user's followers. No quotes
    • -auth
      • OPTIONAL BUT FILE IS REQUIRED
      • DEFAULT: ./auth.ini
      • The directory of the auth.ini file
    • --urlexpand
      • OPTIONAL BUT RECOMMENDED
      • Usage: "--urlexpand" at the end of command
      • If added, the "website url" field on each user account will be expanded from the https://t.co/ link to the full link. WARNING: If included, it will make this tool MUCH MUCH MUCH slower, since it needs to do a GET request on each URL individually. NOTE: It only does it for the "url" website field. Any in the tweet "status" or description won't be expanded.
  3. Run Examples:

    • ./FetchTwitterFollow.exe -m followers -u _WolfwithSword --urlexpand
    • ./FetchTwitterFollow.exe -m following -u _WolfwithSword -auth ./custom-auth.ini
    • ./FetchTwitterFollow.exe -m following -u _WolfwithSword
    • ./FetchTwitterFollow.exe -u _WolfwithSword
  4. Output

    • It will output a CSV based on the mode, username, date, and columns found in USER_FIELDS (see below for this)

    • Format: [username]-[mode: following or followers]-[date: YYYYMMDD].csv

  5. Notes

    • Rate limiting is a thing. This program will pause/sleep on rate limits for about 5 minutes, but will continually do so.
  6. USER_FIELDS

    • These are the fields that exist on a User object and will be columns in the CSV

    • Below are the available options. In the default file, a bunch are removed for being somewhat useless.

    • I also put descriptions next to some which aren't intuitive

  • USER_FIELDS OPTIONS

    •    contributors_enabled
         created_at 
         default_profile 
         default_profile_image 
         description 
         email 
         favourites_count        : # of likes
         followers_count          
         following 
         friends_count             : # of following
         geo_enabled 
         id 
         id_str 
         lang 
         listed_count 
         location 
         name 
         notifications 
         profile_background_color 
         profile_background_image_url 
         profile_background_image_url_https 
         profile_background_tile 
         profile_banner_url 
         profile_image_url 
         profile_image_url_https 
         profile_link_color 
         profile_sidebar_border_color 
         profile_sidebar_fill_color 
         profile_text_color 
         profile_use_background_image 
         protected 
         screen_name 
         status                   : Their status aka most recent tweet. This will be in JSON
         statuses_count           : # of tweets
         time_zone 
         url                      : The "website" field
         utc_offset 
         verified                 : Haha, yeah, this API is from way before the plague of "blue verified" existed.
         withheld_in_countries 
         withheld_scope 
      

fetchtwitterfollow's People

Contributors

wolfwithsword avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

fetchtwitterfollow's Issues

API now requires elevated access

Hi, thanks for this project!

For future reference, since Nov 15th, the developer account know requires elevated access, so one has to request them through this page and fill the form. It has been instantaneous access for me, no human validation behind

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.