Giter Club home page Giter Club logo

dataset_automated_medical_transcription's Introduction

DOI

Motivation

We generated this dataset to train a machine learning model for automatically generating psychiatric case notes from doctor-patient conversations. Since, we did have access to real doctor-patient conversations, we used transcripts from two different sources to generate audio recordings of enacted conversations between a doctor and a patient. We employed eight students who worked in pairs to generate these recordings. Six of the transcripts that we used to produce this recordings were hand-written by Cheryl Bristow and rest of the transcripts were adapted from Alexander Street which were generated from real doctor-patient conversations. Our study requires recording the doctor and the patient(s) in seperate channels which is the primary reason behind generating our own audio recordings of the conversations.

We used Google Cloud Speech-To-Text API to transcribe the enacted recordings. These newly generated transcripts are auto-generated entirely using AI powered automatic speech recognition whereas the source transcripts are either hand-written or fine-tuned by human transcribers (transcripts from Alexander Street).

We provided the generated transcripts back to the students and asked them to write case notes. The students worked independently using a software that we developed earlier for this purpose. The students had past experience of writing case notes and we let the students write case notes as they practiced without any training or instructions from us.

Authors

  • Kazi, Nazmul
  • Kuntz, Matt
  • Kanewala, Upulee
  • Kahanda, Indika

Contributors

  • Bristow, Cheryl
  • Arzubi, Eric

Description of the dataset

Case note categories:

Index Category Abbr.
0 Client Details CD
1 Chief Complaint CC
2 History of Present Illness HPI
3 Past Psychiatric History PPH
4 History of Substance Use HSU
5 Social History SH
6 Family History FH
7 Review of Systems RS

Directories:

Directory Description
transcripts/source Source transcripts that are used to generate the audio recordings.
recordings Audio recordings of the enacted doctor-patient conversations.
transcripts/transcribed Transcripts generated from the audio recordings using Google Cloud Speech-To-Text API.
casenotes Casenotes written by the students, i.e. annotators.

Transcript file structure:

[
	{
		"speaker"  : 1,
		"dialogue" : ["sentence 1", "sentence 2", ...]
	},
	...
]

Definitions

Term Definition
speaker Speaker of the current dialogue turn.
dialogue Sentence(s) spoken by the speaker in current dialogue turn.

Case note file structure:

[
	{
		"categoryId" : "0",
		"sourceId"   : "0",
		"formalText" : "formal text"
	},
	...
]

Definitions

Term Definition
categoryId Index of the case note category, e.g. 5 = Social History, to which this sentence is used. This property is zero-indexed.
sourceId Index of the source sentence in the transcript. This property is zero-indexed.
formalText Modified version of the sentence as it is used in the case note.

NOTE: If a sentnce is used in multiple casenote categories, a record will appear for each use. "sourceId":"n" refers to the sentence whose index is n in the whole transcript whereas multiple sentences can belong to the same dialogue turn. In the following transcript, "sourceId":"3" refers to sentence_d:

[
	{
		"speaker"  : 1,
		"dialogue" : ["sentence_a", "sentence_b"]
	},
	{
		"speaker"  : 2,
		"dialogue" : ["sentence_c", "sentence_d",  "sentence_e"]
	}
]

dataset.pickle

This is a pickle file (protocol version 4) containing all the transcribed transcripts and the casenotes for easy and quick access of the data using python.

Funding

This project is funded by CATalyst Gap fund, Fall 2019.

Transcripts adapted from Alexander Street

D0420-S2-T01 D0420-S3-T02 D0420-S3-T03 D0420-S4-T01 D0420-S4-T02 D0421-S1-T01 D0421-S1-T02 D0421-S1-T03 D0421-S1-T04 D0421-S1-T05 D0421-S2-T01 D0421-S2-T02 D0421-S3-T01 D0421-S3-T02 D0421-S3-T03 D0421-S3-T04 D0421-S3-T05 D0422-S1-T01 D0422-S1-T02 D0422-S1-T03 D0422-S1-T04 D0422-S2-T01 D0422-S2-T02 D0422-S3-T01 D0422-S3-T02 D0422-S3-T03 D0422-S3-T04 D0422-S3-T05 D0422-S3-T06 D0422-S4-T01 D0422-S4-T02 D0422-S4-T03 D0422-S4-T04 D0422-S4-T05 D0423-S1-T01 D0423-S1-T02 D0423-S1-T03 D0423-S2-T01 D0423-S2-T02 D0423-S2-T03 D0424-S1-T01 D0424-S1-T02 D0424-S1-T03 D0424-S2-T01 D0424-S2-T02 D0424-S2-T03 D0424-S2-T04 D0424-S2-T05 D0424-S2-T06 D0424-S3-T01 D0424-S3-T02 D0424-S3-T03 D0424-S3-T04 D0425-S1-T01 D0425-S1-T02 D0425-S1-T03 D0425-S2-T01 D0425-S2-T02 D0425-S2-T03 D0425-S2-T04 D0425-S3-T01 D0425-S3-T02 D0425-S3-T03 D0425-S3-T04 D0425-S3-T05

Disclaimer

This dataset is provided "As Is" without warranty of any kind. In no event shall the authors or copyright holders be liable for any claim, damages or other liability. The source transcripts may not be enacted word-to-word as they appear in the transcript. Similarly, we used automatic speech recognition to transcribe the recordings and the transcribed transcripts may not match exactly as the words appear in the audio recordings.

DOI

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

dataset_automated_medical_transcription's People

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.