Light

dayyass / qaner Goto Github PK

View Code? Open in Web Editor NEW

62.0 62.0 6.0 2.35 MB

Unofficial implementation of QaNER: Prompting Question Answering Models for Few-shot Named Entity Recognition.

Home Page: https://arxiv.org/abs/2203.01543

License: MIT License

Python 100.00%

data-science machine-learning named-entity-recognition natural-language-processing ner nlp python python3 question-answering

qaner's Introduction

Hi, my name is Dani 👋 and I ❤️ AI and Open-Source

Field of interests: AGI, NLP, RL, Graphs, Distributed Systems

My telegram channel: Cat's Shredinger

Skills 🛠️

Languages: Python, SQL
DS/ML/DL: SkLearn, PyTorch, Transformers
Big Data: Hadoop, Hive, Spark
DevOps: Linux, Git, Docker

Work experience 👔

Job Position	Company	Field	Work Period
Research Scientist	SberDevices	AGI, LLM, RLHF, GigaChat	2023-04 — now
NLP Team Lead	SberDevices	Search, Information Retrieval	2022-10 — 2023-04
NLP Tech Lead	Sber AI Lab	NLP, MLOps, Mentoring	2021-05 — 2022-10
Senior NLP Engineer	Tinkoff AI Lab	Virtual Assistant "Oleg"	2021-02 — 2021-04
Middle NLP Engineer	MTS AI Lab	NER with Pseudo-Labeling	2020-05 — 2021-02
Junior Data Scientist	Sberbank	ML with Tabular Data, CV	2018-07 — 2020-05

Education 🎓

Master’s Degree @ Lomonosov Moscow State University (2019 - 2023)
Bachelor's Degree @ Plekhanov Russian University of Economics (2015 - 2019)

Projects 🐾

QaNER - unofficial implementation of QaNER paper (NER via Extractive Question Answering)
RLLib - Reinforcement Learning library
MUSE as Service - REST API for sentence embedding using Multilingual Universal Sentence Encoder
PyTorch NER - pipeline for training NER models using PyTorch
Text Classification Baseline - pipeline for building text classification TF-IDF + LogReg baselines
Graph-Based Clustering - clustering using graph connected components and spanning trees

Public talks 🗣

From Model to Service: Flask + Gunicorn + Docker @ Sberloga
QaNER - NER via Exractive QA @ Sberloga
Git Hooks Is All You Need @ Sberloga
Web-Service for Sentence Embeddings @ Sberloga
How to start a career in DS @ REU Data Science Club

Certifications 📜

Practical Reinforcement Learning (with honors) @ Coursera
Introduction to Deep Learning (with honors) @ Coursera
Bayesian Methods for Machine Learning (with honors) @ Coursera
Hadoop. System for processing large amounts of data @ Stepik

Hackathon participation 💻

deNews @ ETHOnline 2022
Alzheimer's MRI Analysis @ Synthetic Health Data Hackathon 2020

Achievements 🏆

Published 10 packages to PyPI with 25k+ downloads
Contributor to PyTorch, Scikit-Learn, SciPy
Open Data Science Best Contributor 2020

GitHub Stats ⭐

More information in my LinkedIn 🚀

qaner's People

Contributors

Stargazers

Watchers

Forkers

sakutepov akshit17 paihengxu jiachenfan1208 moziofmoon zhanghaok

qaner's Issues

add readme.md dayyass/qaner-conll-bert-base-uncased

https://huggingface.co/dayyass/qaner-conll-bert-base-uncased

add pytorch-autotrainer

https://github.com/dayyass/pytorch-autotrainer

fine-tune QaNER model with deepset/roberta-base-squad2

https://huggingface.co/deepset/roberta-base-squad2

replace conll-2003-like dataset format with spacy-like

standardize metrics calculation

F1 score may be obly concern about the entity

Thank you for your project. I noticed that the confusion matrix is used to calculate the metrics, and when calculating the final macro F1 score, the "O" type is not excluded. I think this is incorrect. Since there are far fewer entities compared to "O", and the focus is mainly on extracting entities, it is generally necessary to exclude predictions for entities. I made the following changes in metric.py:
` for span_true, span_pred in zip(spans_true_batch, spans_pred_batch_top_1):
span_pred = span_pred[0] # type: ignore

    i = entity_mapper[span_true.label]
    j = entity_mapper[span_pred.label]  # type: ignore
    
    confusion_matrix_true_denominator[i] += 1
    confusion_matrix_pred_denominator[j] += 1
    if span_true == span_pred:
        ner_confusion_matrix[i, j] += 1
    if i!=0:
        all_true+=1
        if span_true == span_pred:
            all_pred_correct+=1
        if j!=0:
            all_pred+=1

p1=all_pred_correct/all_pred if all_pred_correct!=0 else 0
r1=all_pred_correct/all_true if all_pred_correct!=0 else 0
f1=2*p1*r1/(p1+r1) if p1+r1!=0 else 0
metrics["ner_p"] = p1
metrics["ner_r"] = r1
metrics["ner_f1"]= f1`

ner_f1 indicates "micro_f1" about entity type

add pip install

release v0.1.1

compare results with standard BERT-NER fine-tuning

https://huggingface.co/docs/transformers/v4.19.4/en/tasks/token_classification

add CI/CD

add original paper experiments

About the parameter 'n_best_size'

Hello, I have some questions regarding the parameter 'n_best_size'. In the program, 'n_best_size' is set to a fixed value, but in reality, each sentence may contain a different number of entities. Therefore, I would like to inquire about your opinion on this matter. I am looking forward to your response.

compare results with HF conll2003 dataset

https://huggingface.co/datasets/conll2003

add badges

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.