dbmdz / mirador-textoverlay Goto Github PK

Neither application/alto+xml nor application/xml+alto (which is currently evaluated and mentioned in the README) are official media types, but there is a (somewhat older) discussion in altoxml/schema#40 that suggests application/alto+xml as format option inclusion of ALTO via seeAlso. Also, application/alto+xml follows the pattern of other application/xml media types.

If you agree, I could provide a PR offering application/alto+xml as an additional option.

Currently performance is absolutely atrocious on slower devices, and even on beefier machines, the frame time tanks once we have larger objects (like newspaper spreads). Some preliminary profiling revealed that most of the time is not actually spent in code, but in the browser's layout engine. On my machine, panning and zooming on a large newspaper spread renders at a stable 60fps, but once I enable the text overlay, it goes to ~7fps, i.e. 10x as slow, which is clearly not acceptable.

There's a post from a Khan Academy developer from 2014 where they talk about improving SVG animation performance by applying CSS transformations to a div surrounding it instead of to the svg directly, maybe this is something we could consider.

I have a small performance optimization for the case where the text overlay is not visible in the pipeline, this should improve rendering performance significantly for the most common use case.

Hi there,

We would like to use this plugin to display transcriptions for newspapers, preferably by creating IIIF 3 manifests. However, the sample provided does not display any annotations in the left sidebar panel when clicking the "Annotations" button (it just says "Showing 0 annotations"): https://iiifv3--mirador-textoverlay.netlify.app/?iiif-content=https://iiif.io/api/cookbook/recipe/0068-newspaper/newspaper_issue_2-manifest.json

Is this because of how the manifest is set up, and thus changes to the manifest itself are needed in order to display the annotations in the sidebar, or does Mirador not yet have full support for displaying annotations in the sidebar for IIIF 3 manifests?

👍 Great job on this plugin!!

We are publishing seeAlso for each canvas with Alto xml available.

Example: https://purl.stanford.edu/fg165hz3589/iiif/manifest

{
          "@id": "https://purl.stanford.edu/fg165hz3589/iiif/canvas/fg165hz3589_1",
          "@type": "sc:Canvas",
          "label": "Page 1",
          "height": 3279,
          "width": 2530,
          "rendering": [
            {
              "@id": "https://stacks.stanford.edu/file/fg165hz3589/fg165hz3589_1.jp2",
              "label": "Original source file (1.49 MB)",
              "format": "image/jp2"
            }
          ],
          "seeAlso": [
            {
              "@id": "https://stacks.stanford.edu/file/fg165hz3589/fg165hz3589_1.xml",
              "label": "OCR text",
              "format": "application/xml",
              "profile": "http://www.loc.gov/standards/alto/ns-v2#"
            }
          ],
          "images": [
            {
              "@id": "https://purl.stanford.edu/fg165hz3589/iiif/annotation/fg165hz3589_1",
              "@type": "oa:Annotation",
              "motivation": "sc:painting",
              "resource": {
                "@id": "https://stacks.stanford.edu/image/iiif/fg165hz3589%2Ffg165hz3589_1/full/full/0/default.jpg",
                "@type": "dctypes:Image",
                "format": "image/jpeg",
                "height": 3279,
                "width": 2530,
                "service": {
                  "@context": "http://iiif.io/api/image/2/context.json",
                  "@id": "https://stacks.stanford.edu/image/iiif/fg165hz3589%2Ffg165hz3589_1",
                  "profile": "http://iiif.io/api/image/2/level2.json"
                }
              },
              "on": "https://purl.stanford.edu/fg165hz3589/iiif/canvas/fg165hz3589_1"
            }
          ]
        },

Based on the requirements, I would expect this to work. Is there something that I am overlooking? https://github.com/dbmdz/mirador-textoverlay#requirements-for-supported-iiif-manifests

Thanks!

We currently render word-level annotations (even though not stated explicitely in the README), but the way they are currently rendered (top-level <text> elements per word) results in broken text selection, especially when the canvas has multi-column text. A good example for this is https://texashistory.unt.edu/ark:/67531/metapth1262460/manifest/:

The clip was taken from a recent Firefox version, but the behavior on Chrome is identical.

I'm looking for guidance on how to effectively use the dbmdz/mirador-textoverlay plugin with Mirador 3. I've attempted to install it via the iiifv3 branch, but I've encountered an issue where the folder in the node_modules directory appears to be empty.

Steps Taken

I attempted to install the dbmdz/mirador-textoverlay plugin using the iiifv3 branch with the following command:

npm install dbmdz/mirador-textoverlay#iiifv3

I build my own Mirador with textoverlay-plugin 0.3.8:

import Mirador from 'mirador/dist/es/src/index';
import downloadDialogPlugin from 'mirador-downloaddialog/es';
import imageCropperPlugin from 'mirador-imagecropper/es';
import keyboardShortcutsPlugin from 'mirador-keyboardshortcuts/es';
//import miradorDownloadPlugin from 'mirador-dl-plugin/es';
import { miradorImageToolsPlugin } from 'mirador-image-tools';
import miradorSharePlugin from 'mirador-share-plugin';
import textOverlayPlugin from 'mirador-textoverlay/es';

window.initMirador = function initMirador(config) {
  Mirador.viewer(config, [
    ...downloadDialogPlugin,
    ...imageCropperPlugin,
    ...keyboardShortcutsPlugin,
    //...miradorDownloadPlugin,
    ...miradorImageToolsPlugin,
    ...miradorSharePlugin,
    ...textOverlayPlugin
  ]);
}

I embedded it into html with this settings:

<div id="my-mirador"	style="position: relative !important; min-height: 500px;"></div>
				<script src="[/js/mirador/main.js](view-source:http://localhost:8080/js/mirador/main.js)"></script>
				<script type="text/javascript">
					const config = {
						id : 'my-mirador',
						textOverlay: true,
						windows : [ {
							imageToolsEnabled : true,
							imageToolsOpen : false,
							manifestId : 'https://alexana.org/ResCarta-Web/iiif/api/presentation/3/49ce0abe-dfce-4e8d-b982-dfc768f760ea%252FALX00000%252F00000001%252F00000012/manifest',
							// "canvasId": "",
							// "canvasIndex": 1,
							textOverlay : {
								enabled : true
							},
							"thumbnailNavigationPosition" : 'far-bottom'
						} ],
						theme : {
							palette : {
								primary : {
									main : '#1967d2',
								},
							},
						},
						window : {
							allowClose : false,
							allowFullscreen : true,
							allowMaximize : false,
							textOverlay : {
								enabled : true,
								selectable : true,
								visible : false
							}
						},
						workspace : {
							type : "not-mosaic-or-elastic"
						},
						workspaceControlPanel : {
							enabled : false
						},
					};
					initMirador(config);
				</script>

But tool button is not shown on any canvas. Setup should be fine, because it works with a manifest with hOCR.

Is it a problem with my ALTO-Format?

My manifest is:

https://alexana.org/ResCarta-Web/iiif/api/presentation/3/49ce0abe-dfce-4e8d-b982-dfc768f760ea%252FALX00000%252F00000001%252F00000012/manifest

when marking text without activated "show text", the blue highlighting is too in-transparent to read underlying image-text.
is it possible to make it more transparent?
maybe use "::selection" in CSS

Currently we don't include any heuristics for detecting text that is rotated, such as in this example from a Wellcome volume:

Due to this, we stretch the text content to fill the available width, which results in this garbled output (which also takes a long time to render for some reason):

For this case it probable suffices to include a simple heuristic to detect 90-degree rotated text. Since the OCR does not include any indication of the direction of the rotation, it should probably fall back to counter-clockwise due to the reading direction.

Obviously this should only apply to texts in western LTR scripts, Asian scripts that are written top-down would obviously be rendered badly with this approach.

I don't think we can "fix" this without help from the actual OCR markup. Even in the western script case, a line might be rotated 90 or 270 degrees, there's no way of telling from this markup, since it includes neither baseline nor rotation information:

<TextBlock ID="P9_TB00006" HEIGHT="316" WIDTH="57" HPOS="649" VPOS="651" STYLEREFS="TXT_43 PAR_CENTER">
  <TextLine ID="P9_TL00007" HEIGHT="316" WIDTH="31" HPOS="649" VPOS="651">
    <String ID="P9_ST00034" CONTENT="Vestry.—33" HEIGHT="167" WIDTH="31" HPOS="649" VPOS="800" WC="0.93" CC="0700000000"/>
    <SP ID="P9_SP00028" WIDTH="0" HPOS="674" VPOS="763"/>
    <String ID="P9_ST00035" CONTENT="Meetings." HEIGHT="133" WIDTH="30" HPOS="650" VPOS="651" WC="1" CC="000000000"/>
  </TextLine>
  <TextLine ID="P9_TL00008" HEIGHT="287" WIDTH="25" HPOS="681" VPOS="666">
    <String ID="P9_ST00036" CONTENT="No." HEIGHT="49" WIDTH="23" HPOS="681" VPOS="904" WC="1" CC="000"/>
    <SP ID="P9_SP00029" WIDTH="0" HPOS="706" VPOS="874"/>
    <String ID="P9_ST00037" CONTENT="of" HEIGHT="27" WIDTH="15" HPOS="690" VPOS="861" WC="1" CC="00"/>
    <SP ID="P9_SP00030" WIDTH="0" HPOS="706" VPOS="828"/>
    <String ID="P9_ST00038" CONTENT="Attendances." HEIGHT="184" WIDTH="22" HPOS="682" VPOS="666" WC="1" CC="000000000000"/>
  </TextLine>
</TextBlock>

Currently we're hiding annotations drawn by Mirador itself when the overlay is set to visible. This is because our overlay is positioned after the canvas overlay drawing the annotations, thus occluding it when we render the line boxes and text.

This is not easy to solve, since a hard requirement for us is that the text must be both selectable and visible. If we just positioned our overlay before the canvas overlay, our text would be visible, but no longer selectable.

A solution could be access the annotation state and render them ourselves into our SVG. However, this would incur a lot of future work, since we'd essentially have to replicate any and all modifications to the annotation rendering (which is done on a canvas in M3 itself) to our SVG-rendering.

A "good enough" solution would be to restrict ourselves to just rendering search annotations. These are just rectangles and could easily be added to the SVG.

I've tried following annotationPage with no luck

{
    "id": "https://api.dev.etu.wiki/c/3/9a2febe6-8569-4ab3-b4e1-19e0ad53cbd8/ap/c",
    "type": "AnnotationPage",
    "items": [
        {
            "id": "https://api.dev.etu.wiki/c/3/9a2febe6-8569-4ab3-b4e1-19e0ad53cbd8/ap/c/a/043a7bb7-d77b-44bd-9517-71bf6f551a1a",
            "type": "Annotation",
            "motivation": "supplementing",
            "body": {
                "type": "TextualBody",
                "value": "authorA",
                "format": "text/plain",
                "language": "zh-Hants"
            },
            "target": "https://api.dev.etu.wiki/c/3/9a2febe6-8569-4ab3-b4e1-19e0ad53cbd8#xywh=958,5101,493,493"
        },
        {
            "id": "https://api.dev.etu.wiki/c/3/9a2febe6-8569-4ab3-b4e1-19e0ad53cbd8/ap/c/a/24453f8f-9c8b-4c82-8ae9-ffa8a779f8a6",
            "type": "Annotation",
            "motivation": "supplementing",
            "body": {
                "type": "TextualBody",
                "value": "authorB",
                "format": "text/plain",
                "language": "zh-Hants"
            },
            "target": "https://api.dev.etu.wiki/c/3/9a2febe6-8569-4ab3-b4e1-19e0ad53cbd8#xywh=968,4544,490,462"
        }
    ],
    "@context": "http://iiif.io/api/p/3/context.json"
}

Hi there :)

I have been playing with the plugin in the last day, and something failed, probably because of a bad ALTO but I have stopped javascript for too long to be able to pinpoint more than a specific issue, which is IIIF-Commons/manifesto#121

Basically, there seems to be an issue with Fragment, which could be caused downstream of this plugin or downstream of manifesto. I wanted you to know this :)

Currently the user is only notified about the presence of text for a canvas once the text for that canvas has been completely fetched and parsed. With slow connections/slow servers, this can sometimes take a decent amount of time. It would be good if, during this time, the tool bubble could be collapsed and show a spinning circle to indicate that the fetching is in progress.

Ideally this change would include an open/close animation for the tool menu so that when the text has finished fetching and the options (if any of them are enabled) become available, the transition is not too jarring for the user.

Hello
I'm trying to implement the 'mirador-textoverlay' plugin
to integrate ALTO files In the Mirador, to view a transcription of the image

It seems to me that there is a problem with Hebrew support
Or there is another definition that needs to be defined.

I check on the demo site
https://mirador-textoverlay.netlify.app/
with this manifest
https://vf.genizah.org/testManifest/FL45509099/manifest.json
That has an ALTO file (the reference to it is in the manifest)
https://vf.genizah.org/testManifest/FL45509099/FL45509099_Alto.xml

The problem is that it looks really bad,
I attached a screenshot.

I would be happy for any help!

IIIF manifest is attached.
text-overlay.zip
(https://dev.opendata.uni-halle.de//json/iiif/123456789/21145/21e5d46a-1eac-4ec1-bc32-57e7f79bbb9c/manifest)

There are the following canvases:

• Canvas with label "[Seite 3]" with no fulltext
• Canvas with label "[Seite 4]" with no fulltext
• Canvas with label "[Seite 5]" with fulltext
• ...

But the fulltext of [Seite 5] is shown on [Seite 4].

The fulltext for "[Seite 6]" is correct shown.

dev.opendata.uni-halle.de have a restricted IP whitelist. @jbaiter I can add your IP for access to the manifest and the images...

The following page shows a site not found message:

https://mirador-textoverlay.netlify.com

This is listed on the repo information and the README.md

Hi,

it would be awesome to have support for svg transformations. E.g. when using the Mirador Image Tools, that e.g. support flipping or rotating images:

Currently the background/foreground colors are hardcoded to black and white. It would be neat if users could select their own colors to better match the overlay to the page image.
This should be rather simple to do with an <input type="color">, it even allows picking a color from the screen in Firefox.

ALTO supports a @BASELINE attribute that can define a polyline on which the text rests. hOCR also includes support for this information. These values could be used for a more accurate estimation of the font size and position used for rendering the SVG.

Unfortunately I don't have access to any samples of OCR data with this information at the moment.

The test fixtures now include an hOCR file (generated by Tesseract 4) that has baseline information. Since both hOCR and ALTO define baselines as polynomials, an hOCR-based implementation should work with ALTO with (hopefully) minimal modifications.

In Firefox, horizontal stretching of the SVG text does not seem to be working, as noticed by @mbennett-uoe on the IIIF Slack. The reason for this is that we use spacingAndGlyphs for the lengthAdjust parameter, which seems to be bugged in Firefox (although it's supposed to be supported: https://developer.mozilla.org/en-US/docs/Web/SVG/Attribute/lengthAdjust).