chaya's People
chaya's Issues
Reconsider highlighting of OCR-recognized words
This was implemented for #15 and is working. But I'm having second thoughts about this feature:
Pro
- Makes it easier to see which words got recognized by the OCR, and which got missed
Cons
- The size of the .chaya files gets much larger (TODO: quantify), because of having to store bounding box for each word
- In-browser memory usage increases (TODO: quantify), for the same reason. This may make editing sluggish (or even cause tab to crash) for large PDFs or on devices with less memory.
- Code complexity increases (Not just the code to highlight these areas, but also the interaction between pdf.js output being at the page level v/s this information being at the line level: eg when loading from file, cannot simply add one chunk at a time, but need to accumulate chunks until a page is done)
- It only shows which words were recognized in the initial OCR response, which is something that may not remain meaningful after manual editing
- It's not clear how useful it is: how valuable is "not recognized", when anything else could just as well be "incorrectly recognized"? (OCR systems probably differ in whether they will always output something, or will just leave out uncertain words)
- A deficiency of the current implementation: The words unrecognized by OCR actually get dimmed (when in fact we should probably highlight them more)
- Subjective: Looks ugly?
Alternatives
Some alternative ways to show which words were recognized:
Show lines with tight fit, i.e. only display the part of each line that corresponds to the recognized words(This is dangerous as it may leave out precisely the words we want to catch)- Indent each line of the editable text by an indent corresponding to the first recognized word in the image. This is more likely to place recognized text directly under the corresponding image (at least at roughly similar zoom sizes)
- That thing that scribeocr.com does, of superimposing the text (rendered in some font with matching font metrics) onto the image.
Manually incorporate prosemirror-example-setup
Two-column mode / OCR on manually specified regions
Will likely need this
Auto-detect chunks
Using line-widths, example. Idea by Andrew Ollett.
Edit: Even a heuristic of "group together adjacent lines if they have (roughly) the same xmin and xmax" may work.
Highlight (un)recognized words
Suggested by Suhas: when words get skipped by the OCR, would be useful to know.
Maybe we can draw rectangles around recognized words, or increase their relative brightness/opacity (or whatever), so that unrecognized words get noticed
Get started
- Accompanying data file that can be saved
- Export to print?
- Single file that holds the PDF bounding boxes inside it.
Initial Prosemirror setup
Looks like I need to install the package etc:
corepack enable
yarn init -2
yarn add prosemirror-state
yarn add pdfjs-dist
and build it:
# Build JS.
js-dev:
npx esbuild main.ts --outfile=main.js --bundle --watch --format=esm
# Build JS for production.
js-prod:
npx esbuild main.ts --outfile=main.js --bundle --minify --format=esm
Ungroup button
Like the button to group lines into chunks, there should be a button to ungroup a chunk into single-line chunks.
PM save/load
Before getting further into the code, would be good to implement the feature to:
- save the generated PM doc into a
.sc
file, and - load a previously saved PM doc, instead of doing OCR to generate pages from scratch.
Part of this is UI work, and part is the actual serialization / deserialization.
Button to delete chunk
Had mentioned it earlier in #17 but tracking it separately as that issue got closed.
Mark for non-breaking hyphen
Maybe we can add a mark (ProseMirror term for things like "bold" and "italic") that can be used for end-of-line hyphens — in reading view or when chunks wrap, they should be hidden.
Break apart lines
Sometimes a line
can span multiple lines, and right now there's no way to insert a line break within a line.
![image](https://private-user-images.githubusercontent.com/767530/331653905-0c68e72b-cf7c-4f29-8596-e38f0d55e36e.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjE0NzEzNzAsIm5iZiI6MTcyMTQ3MTA3MCwicGF0aCI6Ii83Njc1MzAvMzMxNjUzOTA1LTBjNjhlNzJiLWNmN2MtNGYyOS04NTk2LWUzOGYwZDU1ZTM2ZS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNzIwJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDcyMFQxMDI0MzBaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT02ZjUyMzk1OWQzNzA4ZTBkOWYwNTg3NjNlNmIzZjg2MzFmNDQ0MThjMjJmZDc1ZGMyODkwZTVmMzBlMjE5YTUwJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.Z6YCJ1D1foQ4MxLXigTxQUbCzT_D3IJ_WWpBZ4JP1SE)
We could either:
- Change schema to have a
line
be a<p>
, so that it contain line breaks internally. - Allow re-drawing lines and re-doing OCR (#8).
Chunk types (paragraph, verse, footnote, …)
Continuing discussion from #7 (comment) — we want at least these types:
- heading (displayed as
h1
,h2
etc) - paragraph (lines will be displayed
inline-block
or whatever, with wrapping) (unless in line-by-line mode) - verse (lines will be centered)
and a "footnote label", which is blank by default. (The point is that a single footnote may have multiple chunks, like paragraphs and verses.)
Release versions
Before making this public:
- A page
/versions/
that links to severalindex.html
files (chaya-202401.html
etc)- Manually created
- Say "This is version [N]. Older versions are here" and link to that page.
- At least, link to the page.
- Code to check that schema versions match (otherwise redirect?)
- Some process for building version files (maybe just before changing schema).
- Manual instructions in Makefile
- Some message about "use at your own risk, no guarantee of future compatibility, just save this file locally and use it".
Move save button into PM menu
Also maybe the text/image zoom sliders?
Custom image size and text (font) size
It should be possible to change the sizes.
(And also of course where the text is displayed: side-by-side, or below the image.)
Image view options
Add to the Prosemirror menu with three options:
- No images (reading mode)
- Chunk by chunk (default)
- Line by line (for re-editing)
More prominent warning when creating a new file
A user (my mother) did a few pages, saved them, then (on a later day) opened the PDF file and ended up re-doing OCR, not knowing that she should have loaded from the saved file. I think we can be more prominent about "saved" v/s "new", then have the OCR options under New.
Joining lines into regions (chunks)
Right now, all the text from each page of OCR is processed into a single blob of text. We should instead use the line-level information and make smaller regions.
For now / MVP, it would be a good start to just get paragraphs into the output, rather than a single paragraph with all the text:
Lines 308 to 309 in d60d4d2
Lines 344 to 346 in d60d4d2
Prevent clicking on "create new" till PDF file has been loaded
OCR alternatives
Option to use Google OCR (bring your own key) instead of Tesseract.js
Warn on closing tab when there are unsaved changes
I think the Ambuda one had it (even after my changes).
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.