Drishti (Nepali: दृष्टि | English: Vision) is an open-source cross-platform mobile application project at Incubate Nepal that incorporates Machine Learning and Artificial Intelligence to help visually impaired people recognize different currency bills and perform daily cash transactions more effectively.
We plan to expand Drishti to other applications like Short Text and Document Reading in the future.
- Table of Contents
- General Information
- Project Deliverables
- Installation
- App Demo
- Screenshots
- Technologies
- Techstack
- Features
- Status
- Inspiration
- Contributing
- Contact Us
- License
The mobile application is used by the visually impaired community to recognize different currency bills and perform their daily cash transactions more easily. This user-friendly app uses the phone camera, accessibility features, and gestures to provide the desired results.
- Mobile Application
- Computer Vision Model
- Website
The app is now available on Google Play Store as Drishti Nepal.
demo.mp4
- Flutter
- Python
- Tensorflow
- Frontend: Flutter
- Data science:
- Experimentation: PyTorch
- Training: Tensorflow
- Deployment: Tflite (Tensorflow Lite)
- Website: HTML, CSS, JavaScript
List of implemented features:
- Audio feedback in Native Language (Nepali)
- Ad-free
- Available to use Offline
- History Feature
- Keep Track of Scanned Notes
- Sum up the transactions
- Classify the Scanned Notes ("Today", "This Week", "This Month")
TODOs for future development:
- Add haptic feedback for Cash Recognition
- Implement Short Text and Document Reading (Nepali)
- Cash Recognition Feature
- Short Text and Document Reading
Drishti app is particularly developed for the blind and low vision community of Nepal. To understand the need of our end users, the team studied and empathized with the blind and low-vision individuals from Blind Youth Association Nepal rather than digging into the financial problems faced by them.
After interviewing more than 50 visually impaired individuals, we found that they are already using some cash recognizing apps but are not satisfied with them. Some interviewees also shed light on their daily difficulties while traveling. They have suggested to include some additional features of location tracking and user friendly GPS in our future iterations. Our basic goal is to make a robust application which could benefit them.
| As a/an | I want to | So that I can | Status of progress |
|---|---|---|---|
| User | Scan the currency notes at a bank | Confirm the total sum of money I received | Done |
| User | Identify the bills in the dark using my flashlight | Transact money during the evening too | Done |
| Developer | Get the feedback from the users | Make necessary upgrades in the application | Done |
| Product Owner | Know the progress made in the application during each iteration | Prioritize the future goals for the project | Done |
| User | Recognize the bills without having to ask others | Be independent in my financial affairs | Done |
| User | Have a simple UI in my application | Easily navigate into the app | Done |
| User | Have the buttons labeled in the app | Properly recognize the buttons using my phone’s screen reader | Done |
| Software engineer | Easily get the source code of the app | Modify the app and improve its accuracy | Done |
| User | See the history of my scanned notes in a day | I can keep a record of my expenditure list conveniently | Done |
These are the ways in which you can contribute to this project:
- Submit bugs and feature requests
- Review source code changes
- Contribute directly to improve Drishti (adding new features and fixing bugs). The steps to do so include:
- Fork this repository
- Clone the forked repository to your machine
- Create a branch with your GitHub username
- Commit your changes, push the changes to GitHub
- Open a pull request here
- Email: [email protected]
Drishti is MIT licensed, as found in the LICENSE file.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent