dimsum16 / dimsum-data Goto Github PK
View Code? Open in Web Editor NEWData for the DiMSUM shared task at SEMEVAL 2016
Home Page: http://dimsum16.github.io/
Data for the DiMSUM shared task at SEMEVAL 2016
Home Page: http://dimsum16.github.io/
$ fgrep 'AUX' dimsum16.train | fgrep 'v.stative' | wc -l
342
$ fgrep 'AUX' dimsum16.train | fgrep -v 'v.stative' | wc -l
2352
$ fgrep 'v.stative' dimsum16.train | fgrep -v 'AUX' | wc -l
3089
E.g., the following is a copula, so should be VERB
rather than AUX
:
1 Is is AUX O v.stative lowlands-4
2 your your DET O lowlands-4
3 cat cat NOUN O n.animal lowlands-4
4 safe safe ADJ O lowlands-4
5 from from ADP O lowlands-4
6 disease disease NOUN O n.state lowlands-4
Whereas the following is an auxiliary, so there should be no supersense:
1 Were were AUX O v.stative lowlands-32
2 ordered ordered VERB O v.communication lowlands-32
3 to to ADP O lowlands-32
4 chant chant VERB O v.communication lowlands-32
5 praise praise NOUN O n.communication lowlands-32
What were annotators told for the CPH supersense-labeled tweets? I need to give my annotator a policy for various tweet-centric phenomena.
Differences I am noticing between Lowlands and Ritter datasets:
Lowlands consists mostly of promotional material/shares of other content. Most tweets are short, headline-like, and end with a URL. Ritter data is more conversational.
URLs, usernames, numbers, etc. are obscured in Lowlands data, but not Ritter data.
In Lowlands data, numbers at the beginning of a noun phrase are joined to form an MWE.
In Lowlands data, usernames are supersense-tagged as PERSON
and URLs as COMMUNICATION
. They are not supersense-tagged in Ritter. I would argue that it's probably not the best use of annotator time to assign labels that can be applied deterministically, nor is it interesting to give credit for such labels in our evaluation.
Some Ritter tokens are inexplicably missing supersense labels. E.g. in ritter-train.tsv:
follow NN O
back NN O
With the CMU Twitter dataset (Tweebank), some kinds of MWEs are already annotated. It would be nice to either (i) choose supersenses subject to the annotated MWE segmentation, or (ii) choose the segmentation and supersenses such that the annotated MWEs, at minimum, are present (additional MWEs may be predicted).
These occur between sentences lowlands-73
and lowlands-76
(are 74 and 75 supposed to be there?), as well as before the start of the Reviews data. I removed them manually in 07db7a2 but just wanted to check that these aren't due to a preprocessing bug.
The lexical head field is 0
in the Reviews data and blank in the rest of the data.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.