Giter Club home page Giter Club logo

4cat's Introduction

4CAT: Capture and Analysis Toolkit

DOI: 10.5117/CCR2022.2.007.HAGE DOI: 10.5281/zenodo.4742622 License: MPL 2.0 Requires Python 3.8 Docker image status

4CAT has a website at 4cat.nl.

A screenshot of 4CAT, displaying its 'Create Dataset' interfaceA screenshot of 4CAT, displaying a network visualisation of a dataset

4CAT is a research tool that can be used to analyse and process data from online social platforms. Its goal is to make the capture and analysis of data from these platforms accessible to people through a web interface, without requiring any programming or web scraping skills. Our target audience is researchers, students and journalists interested using Digital Methods in their work.

In 4CAT, you create a dataset from a given platform according to a given set of parameters; the result of this (usually a CSV or JSON file containing matching items) can then be downloaded or analysed further with a suite of analytical 'processors', which range from simple frequency charts to more advanced analyses such as the generation and visualisation of word embedding models.

4CAT has a (growing) number of supported data sources corresponding to popular platforms that are part of the tool, but you can also add additional data sources using 4CAT's Python API. The following data sources are currently supported actively and can be used to collect data with 4CAT:

  • 4chan and 8kun
  • BitChute
  • Reddit
  • Telegram
  • Tumblr
  • Twitter API v2 (Academic and regular tracks)

The following platforms are supported through other tools, with which you can collect data to import data into 4CAT for analysis:

  • Instagram, TikTok, 9gag, Imgur, LinkedIn, Parler (via Zeeschuimer)
  • Facebook and Instagram (via CrowdTangle exports)

A number of other platforms have built-in support that is untested, or requires e.g. special API access. You can view the data sources in our wiki or review the data sources' code in the GitHub repository. It is also possible to import your own CSV files into 4CAT for analysis.

Installation

You can install 4CAT locally or on a server via Docker or manually. For easiest installation, we reccomend copying our docker-compose.yml file, .env file, and running this terminal command in the folder where those files have been saved:

docker-compose up -d

In depth instructions on both Docker installation and manual installation can be found in our wiki.

A video walkthrough installing 4CAT via Docker can be found on YouTube here.

Currently scraping of 4chan, 8chan, and 8kun require additional steps; please see the wiki.

Please check our issues and create one if you experience any problems (pull requests are also very welcome).

Modules

4CAT is a modular tool and easy to extend. The following two folders in the repository are of interest for this:

  • datasources: Data source definitions. This is a set of configuration options, database definitions and python scripts to process this data with. If you want to set up your own data sources, refer to the wiki.
  • processors: A collection of data processing scripts that can plug into 4CAT to manipulate or process datasets created with 4CAT. There is an API you can use to make your own processors.

Credits & License

4CAT was created at OILab and the Digital Methods Initiative at the University of Amsterdam. The tool was inspired by DMI-TCAT, a tool with comparable functionality that can be used to scrape and analyse Twitter data.

4CAT development is supported by the Dutch PDI-SSH foundation through the CAT4SMR project.

4CAT is licensed under the Mozilla Public License, 2.0. Refer to the LICENSE file for more information.

4cat's People

Contributors

dale-wahl avatar danprisk avatar dependabot[bot] avatar guidoajansen avatar kuchosauronad0 avatar muneerahp avatar pgr-me avatar sal-uva avatar stijn-uva avatar stijnstijn avatar xmacex avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.