serenayarda / sign-language Goto Github PK

UNIMPEDED COMMUNICATION WITH DEEP LEARNING

In daily life, people communicate with each other while living their lives. Hearing or speech impaired individuals with communication impairments use sign language, which is not universal and varies from country to country, in order to establish this interaction. Sign language; It is a silent and visual language that uses hand gestures, facial expressions and body language as a whole. Sign language is a communication method in which words, phrases and thoughts are expressed with hand, face and body movements. In this study, in order to facilitate the communication of hearing / speech impaired individuals with non-handicapped individuals. First of all, the data was created with image processing, then the training and testing of the data set was carried out by creating a CNN network, and then the text was printed on the screen and voiced. Thus, it is aimed to facilitate the communication of individuals with hearing impairment with individuals without hearing impairment. In this proposed system, the data set of double finger gestures was labeled, trained and tested with the Convolutional Neural Network (CNN) model. While training the data set, it is aimed that the accuracy rate will be high considering the optimization methods, activation function and hyper parameters. With this system, hand movements showing sign language will be captured by the camera and analyzed with algorithms used in image asset analysis. Captured letters were converted into text using the Python programming language. At the end of this project, it was ensured that the letters of the sign language alphabet and other defined characters were detected in real time in the region shown in the selected area and in an environment where the necessary conditions were met, and after a simultaneous word or a sentence was written, it was translated into sound. The designed system is used in communication with the hearing impaired and in TID training, providing benefits and convenience to relations.

The data set used is shared as a drive file. And here is the link (https://drive.google.com/drive/u/0/folders/1DyIQ6JJ-RoC8UJEOx1-B2FWivEwQxUD-)