mpeds / mpeds-coder Goto Github PK

MPEDS Annotation Interface

License: MIT License

Python 54.01% JavaScript 20.66% CSS 2.01% HTML 23.32%

protest-events movement sociology social-movements event-data

mpeds-coder's Introduction

MPEDS: Machine-learning Protest Event Data System

MPEDS is a tool for facilitating the creation of protest event data. MPEDS uses recent innovations from machine learning and natural language processing to generate protest event data with little to no human intervention. The system aims to have the effect of increasing the speed and reducing the labor costs associated with identifying and coding collective action events in news sources, thus increasing the timeliness of protest data and reducing biases due to excessive reliance on too few news sources.

Pre-requisites

You will need to install Git LFS to properly download the large classifier and vectorizer files from this repository.

Publications

Hanna, Alex. 2017. MPEDS: Automating the Generation of Protest Event Data. SocArXiv. DOI: 10.31235/osf.io/xuqmv.
Oliver, Pamela, Chaeyoon Lim, Morgan Matthews and Alex Hanna. 2022. "Black Protests in the United States, 1994-2010." Sociological Science 9(May):275-312. DOI: 10.15195/v9.a12.
Oliver, Pamela, Alex Hanna, Chaeyoon Lim. 2022. “Constructing Relational and Verifiable Protest Event Data: Four Challenges and Some Solutions” Forthcoming in Mobilization. Preprint available https://osf.io/preprints/socarxiv/d89g7/

DOI

The DOI for this repository has been created with Zenodo.

You can cite this software as:

Hanna, Alex. 2017. MPEDS: Machine-Learning Protest Event Data System (v1.0). Zenodo. https://doi.org/10.5281/zenodo.886459

Acknowledgments

Development of this software has been supported by a National Science Foundation Graduate Research Fellowship and National Science Foundation grant SES-1423784. Thanks to Emanuel Ubert and Katie Fallon for working with this system since its inception, and to many undergraduate annotators who have put a lot of time working with and refining this system.

mpeds-coder's People

Contributors

Stargazers

Watchers

Forkers

ronggui joshzyj davidskalinder alexhanna

mpeds-coder's Issues

Merge branches

There's enough divergence between the campus anti-racism branch and master that we need to work on a more generalized solution that pulls the branch back in. The campus anti-racism branch has a set of 'yes/no' questions which it pulls from a CSV file. We need to merge this functionality back into the master.

Article assignment by ID doesn't work in admin interface

Offending line is here:

articles = assign_lib.getArticlesbyID(ids)

in mpeds_coder.py

Upgrade from Bootstrap 3

Currently the project is running on Bootstrap 3.3.7. Versions 4+ of Bootstrap allow for doing auto-layout using columns (e.g. https://getbootstrap.com/docs/4.6/layout/grid/#auto-layout-columns) which may be helpful for the adjudication interface. It'll also allow us to source the datetime-picker from a CDN (see #77).

Clobber queues

Add an option to clear out queues in admin interface.

Generalize data model by merging queues and code tables

coder_event_creator
coder_first_pass
coder_second_pass

and

article_queue
event_creator_queue
second_pass_queue

are all the same table, effectively. merge them in the data model and add a field for which "pass" it is.

Adapt JavaScript to work outside of Firefox

A major issue with this interface is that it doesn't work outside of Firefox. The place where this issue fails the most is in addSelectedText function in the static/js/shared.js. This function enables the text select functionality.

Create ability to do "bursty" assignment in admin interface

"bursty" is not a good name for it but the idea is from something Pam wrote a while ago... about how annotators could use some context and so may benefit from stretches of contiguous articles rather than random assignment.

Add third column for event cards

Re-integrate a third column in the event coder view which will feature event cards which will display the description of the event. This will look similar to the old pass 2 view. @davidskalinder may also be working on this.

yes-no.yml -> yes-no.yaml

Need to standardize suffix for YAML files.

Source JS and CSS from CDNs

This has been done in the adjudication-master branch, but we still need to upgrade the bootstrap-timepicker to a CDN. This may mean we need to upgrade to a newer version of bootstrap.

The most recent version of the datetime picker is at https://getdatepicker.com/

Create branches to which to push unfinished BPP features

As discussed in today's meeting, there are five or six branches for unfinished features in the BPP repo that I won't get to before I leave, so the best thing will be to create some new branches in this repo and PR the unfinished ones to those new branches; after that @alexhanna can manage them as she wants.

First step: see if I can create them thar new branches.

Moving duration estimates over to Basic Info tab

Need to do this in template/event-creator-block.html and the JavaScript.