Giter Club home page Giter Club logo

pdf-extractor's Introduction

pdf-extractor

Pdf-extractor is a wrapper around pdf.js to generate images, svgs, html files, text files and json files from a pdf on node.js.

  • Image: A DOM Canvas is used to render and export the graphical layer of the pdf. Canvas exports *.png as a default but can be extended to export to other file types like *.jpg.
  • SVG: Pdf objects are converted to svg using the SVGGraphics parser of pdf.js.
  • HTML: Pdf text is converted to HTML. This can be used as a (transparent) layer over the image to enable text selection.
  • Text: Pdf text is extracted to a text file for different usages (e.g. indexing the text).

PDF.js on Node.js

This library is in it's most basic form a node.js wrapper for pdf.js. It has default renderers to generate a default output, but is easily extended to incorporate custom logic or to generate different output. It uses a node.js DOM and the node domstub from pdf.js do make pdf parsing available on node.js without a browser.

Box View / Crocodoc

This project is inspired by the Box View / Crocodoc way of converting documents (with this tool pdfs) to web-assets: images and html. The generated files match the files of Box View. This makes this library an option to transition from the Box View API to an open-source solution.

Examples

This library can be used as-is to generate assets from a pdf. The only requirements are a pdf as input and a writable directory as output. The extractor can also be used for rendering in different ways. The renderers can be extended or new ones can be injected into the extractor to render a pdf in new ways.

Extractor

How to use the default extractor to render png, html and text files for pdf pages:

const PdfExtractor = require('pdf-extractor').PdfExtractor;

let outputDir = '/path/to/output',

pdfExtractor = new PdfExtractor(outputDir, {
    viewportScale: (width, height) => {
        //dynamic zoom based on rendering a page to a fixed page size 
        if (width > height) {
            //landscape: 1100px wide
            return 1100 / width;
        }
        //portrait: 800px wide
        return 800 / width;
    },
    pageRange: [1,5],
});

pdfExtractor.parse('/path/to/dummy.pdf').then(function () {
	console.log('# End of Document');
}).catch(function (err) {
	console.error('Error: ' + err);
});

This results in these generated files:

info.json
page-1.png
page-2.png
page-3.png
page-4.png
page-5.png
stylesheet.css
text-1.html
text-1.txt
text-2.html
text-2.txt
text-3.html
text-3.txt
text-4.html
text-4.txt
text-5.html
text-5.txt

Using a custom renderer

It is relatively easy to extend or create new renderers/writers. Below an example to use the Canvas capability for *.jpg file generation.

const PdfExtractor = require('pdf-extractor').PdfExtractor;
const CanvasRenderer = require('pdf-extractor').CanvasRenderer;
const SvgRenderer = require('pdf-extractor').SvgRenderer;
const FileWriter = require('pdf-extractor').FileWriter;

class JPGWriter extends FileWriter
{
	getFilePathForPage(page) {
		return super.getPagePath(page.pageNumber, 'png');
	}

	writeCanvasPage(page, viewport, canvas) {
		return this.writeStreamToFile(canvas.jpgStream(), this.getFilePathForPage(page))
	}
}

class JPGCanvasRenderer extends CanvasRenderer
{
	getWriters(writerOptions) {
		let writers = super.getWriters(writerOptions);
		writers.push(new JPGWriter(this.outputDir, writerOptions));
		return writers;
	}
}

let outputDir = '/path/to/output',

pdfExtractor = new PdfExtractor(outputDir, {
	renderers: [
		new JPGCanvasRenderer(outputDir, rendererOptions),
		new SvgRenderer(outputDir, rendererOptions)
	]
});

pdfExtractor.parse('/path/to/dummy.pdf').then(function () {
	console.log('# End of Document');
}).catch(function (err) {
	console.error('Error: ' + err);
});

This adds jpg images to the generated files.

pdf-extractor's People

Contributors

annervisser avatar github-actions[bot] avatar harmenm avatar hikariii avatar lightpriest avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pdf-extractor's Issues

NPM package writes me an error and not installed

I tried install package from npm and get error:

C:\Users\USERNAME\Documents\MYPROJECT> npm i pdf-extractor
npm WARN deprecated [email protected]: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm ERR! code 1
npm ERR! path C:\Users\USERNAME\Documents\MYPROJECT\node_modules\canvas
npm ERR! command failed
npm ERR! command C:\Windows\system32\cmd.exe /d /s /c node-pre-gyp install --fallback-to-build --update-binary
npm ERR! Warning: Missing input files:
npm ERR! C:\GTK\bin\libpango-1.0-0.dll
npm ERR! C:\GTK\bin\libfreetype-6.dll
npm ERR! C:\GTK\bin\libfontconfig-1.dll
npm ERR! C:\GTK\bin\libglib-2.0-0.dll
npm ERR! C:\GTK\bin\libexpat-1.dll
npm ERR! C:\GTK\bin\libpangocairo-1.0-0.dll
npm ERR! C:\GTK\bin\zlib1.dll
npm ERR! C:\GTK\bin\libgmodule-2.0-0.dll
npm ERR! C:\GTK\bin\libgobject-2.0-0.dll
npm ERR! C:\GTK\bin\libpangowin32-1.0-0.dll
npm ERR! C:\GTK\bin\libpng14-14.dll
npm ERR! C:\GTK\bin\libintl-8.dll
npm ERR! C:\GTK\bin\libpangoft2-1.0-0.dll
npm ERR! C:\GTK\bin\libcairo-2.dll
npm ERR! C:\GTK\bin\libgthread-2.0-0.dll
npm ERR! Failed to execute 'C:\Program Files\nodejs\node.exe C:\Users\USERNAME\AppData\Roaming\nvm\v18.0.0\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js build --fallback-to-build --update-binary --module=C:\Users\USERNAME\Documents\MYPROJECT\node_modules\canvas\build\Release\canvas.node --module_name=canvas --module_path=C:\Users\USERNAME\Documents\MYPROJECT\node_modules\canvas\build\Release --napi_version=8 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v108' (7)
npm ERR! node-pre-gyp info it worked if it ends with ok
npm ERR! node-pre-gyp info using [email protected]
npm ERR! node-pre-gyp info using [email protected] | win32 | x64
npm ERR! node-pre-gyp http GET https://github.com/Automattic/node-canvas/releases/download/v2.9.3/canvas-v2.9.3-node-v108-win32-unknown-x64.tar.gz
npm ERR! node-pre-gyp ERR! install response status 404 Not Found on https://github.com/Automattic/node-canvas/releases/download/v2.9.3/canvas-v2.9.3-node-v108-win32-unknown-x64.tar.gz
npm ERR! node-pre-gyp WARN Pre-built binaries not installable for [email protected] and [email protected] (node-v108 ABI, unknown) (falling back to source compile with node-gyp)
npm ERR! node-pre-gyp WARN Hit error response status 404 Not Found on https://github.com/Automattic/node-canvas/releases/download/v2.9.3/canvas-v2.9.3-node-v108-win32-unknown-x64.tar.gz
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using [email protected]
npm ERR! gyp info using [email protected] | win32 | x64
npm ERR! gyp info ok
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using [email protected]
npm ERR! gyp info using [email protected] | win32 | x64
npm ERR! gyp info find Python using Python version 3.9.1 found at "C:\Users\USERNAME\AppData\Local\Programs\Python\Python39\python.exe"
npm ERR! gyp info find VS using VS2017 (15.9.28307.1321) found at:
npm ERR! gyp info find VS "C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools"
npm ERR! gyp info find VS run with --verbose for detailed information
npm ERR! gyp info spawn C:\Users\USERNAME\AppData\Local\Programs\Python\Python39\python.exe
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args   'C:\\Users\\USERNAME\\AppData\\Roaming\\nvm\\v18.0.0\\node_modules\\npm\\node_modules\\node-gyp\\gyp\\gyp_main.py',
npm ERR! gyp info spawn args   'binding.gyp',
npm ERR! gyp info spawn args   '-f',
npm ERR! gyp info spawn args   'msvs',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   'C:\\Users\\USERNAME\\Documents\\MYPROJECT\\node_modules\\canvas\\build\\config.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   'C:\\Users\\USERNAME\\AppData\\Roaming\\nvm\\v18.0.0\\node_modules\\npm\\node_modules\\node-gyp\\addon.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   'C:\\Users\\USERNAME\\AppData\\Local\\node-gyp\\Cache\\18.0.0\\include\\node\\common.gypi',
npm ERR! gyp info spawn args   '-Dlibrary=shared_library',
npm ERR! gyp info spawn args   '-Dvisibility=default',
npm ERR! gyp info spawn args   '-Dnode_root_dir=C:\\Users\\USERNAME\\AppData\\Local\\node-gyp\\Cache\\18.0.0',
npm ERR! gyp info spawn args   '-Dnode_gyp_dir=C:\\Users\\USERNAME\\AppData\\Roaming\\nvm\\v18.0.0\\node_modules\\npm\\node_modules\\node-gyp',
npm ERR! gyp info spawn args   '-Dnode_lib_file=C:\\\\Users\\\\USERNAME\\\\AppData\\\\Local\\\\node-gyp\\\\Cache\\\\18.0.0\\\\<(target_arch)\\\\node.lib',
npm ERR! gyp info spawn args   '-Dmodule_root_dir=C:\\Users\\USERNAME\\Documents\\MYPROJECT\\node_modules\\canvas',
npm ERR! gyp info spawn args   '-Dnode_engine=v8',
npm ERR! gyp info spawn args   '--depth=.',
npm ERR! gyp info spawn args   '--no-parallel',
npm ERR! gyp info spawn args   '--generator-output',
npm ERR! gyp info spawn args   'C:\\Users\\USERNAME\\Documents\\MYPROJECT\\node_modules\\canvas\\build',        
npm ERR! gyp info spawn args   '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! gyp info ok
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using [email protected]
npm ERR! gyp info using [email protected] | win32 | x64
npm ERR! gyp info spawn C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\MSBuild\15.0\Bin\MSBuild.exe
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args   'build/binding.sln',
npm ERR! gyp info spawn args   '/clp:Verbosity=minimal',
npm ERR! gyp info spawn args   '/nologo',
npm ERR! gyp info spawn args   '/p:Configuration=Release;Platform=x64'
npm ERR! gyp info spawn args ]
npm ERR! gyp ERR! UNCAUGHT EXCEPTION
npm ERR! gyp ERR! stack Error: spawn C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\MSBuild\15.0\Bin\MSBuild.exe ENOENT
npm ERR! gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:283:19)
npm ERR! gyp ERR! stack     at onErrorNT (node:internal/child_process:476:16)
npm ERR! gyp ERR! stack     at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
npm ERR! gyp ERR! System Windows_NT 10.0.18363
...

P.S. I use Node v18.0.0 and Windows 10

How to actually display the PDF assets ?

The question is in the title... ;-).

I have tested a few documents with the default Box viewer.js but the PDF do not render although this library is presented as a replacement for the legacy BoxView.

TypeError: Cannot read property 'prototype' of null

Having this issue when trying to run test samples or any pdf. Any thoughts?

/Users/xxxxxx/projects/pdf-extractor/lib/pdfjs-utils/domfacade.js:117
Object.defineProperty(Canvas.prototype, "getContext", {
                             ^

TypeError: Cannot read property 'prototype' of null
    at Object.<anonymous> (/Users/xxxxxxx/projects/pdf-extractor/lib/pdfjs-utils/domfacade.js:117:30)
    at Module._compile (module.js:660:30)
    at Object.Module._extensions..js (module.js:671:10)
    at Module.load (module.js:573:32)
    at tryModuleLoad (module.js:513:12)
    at Function.Module._load (module.js:505:3)
    at Module.require (module.js:604:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/Users/xxxxxx/projects/pdf-extractor/lib/PdfExtractor.js:4:19)
    at Module._compile (module.js:660:30)

stylesheet.css is empty

I have tested the library with two PDF documents and the resulting stylesheet.css is empty. It thus can't be displayed by crocodoc viewer (js) (or the legacy Boxview viewer.js.

Also, when i replace the stylesheet.css (from an old legacy document just to see if id partially worked, ii get the following errors).

crocodoc.viewer.min.js:3 Uncaught TypeError: Cannot read property 'setAttribute' of undefined
    at h (crocodoc.viewer.min.js:3)
    at Object.i (crocodoc.viewer.min.js:3)
    at j (jquery-1.11.2.min.js:2)
    at Object.add [as done] (jquery-1.11.2.min.js:2)
    at Object.load (crocodoc.viewer.min.js:3)
    at Object.load (crocodoc.viewer.min.js:3)
    at Object.loadPage (crocodoc.viewer.min.js:3)
    at d (crocodoc.viewer.min.js:3)


crocodoc.viewer.min.js:3 Uncaught TypeError: Failed to execute 'importNode' on 'Document': parameter 1 is not of type 'Node'.
    at Object.d (crocodoc.viewer.min.js:3)
    at j (jquery-1.11.2.min.js:2)
    at Object.add [as done] (jquery-1.11.2.min.js:2)
    at Object.load (crocodoc.viewer.min.js:3)
    at Object.load (crocodoc.viewer.min.js:3)
    at Object.loadPage (crocodoc.viewer.min.js:3)
    at d (crocodoc.viewer.min.js:3)

Issue rendering pdf images (imageObject)

Fatal error:

Unhandled rejection TypeError: Image or Canvas expected
    at CanvasGraphics_paintJpegXObject [as paintJpegXObject] (/usr/src/app/node_modules/pdfjs-dist/build/pdf.js:14502:11)
    at CanvasGraphics_executeOperatorList [as executeOperatorList] (/usr/src/app/node_modules/pdfjs-dist/build/pdf.js:13624:22)
    at InternalRenderTask__next [as _next] (/usr/src/app/node_modules/pdfjs-dist/build/pdf.js:4682:39)
From previous event:
    at InternalRenderTask__scheduleNext [as _scheduleNext] (/usr/src/app/node_modules/pdfjs-dist/build/pdf.js:4675:36)
    at InternalRenderTask__continue [as _continue] (/usr/src/app/node_modules/pdfjs-dist/build/pdf.js:4668:14)
    at InternalRenderTask_operatorListChanged [as operatorListChanged] (/usr/src/app/node_modules/pdfjs-dist/build/pdf.js:4658:12)
    at PDFPageProxy_renderPageChunk [as _renderPageChunk] (/usr/src/app/node_modules/pdfjs-dist/build/pdf.js:3761:36)
    at WorkerTransport.transportRender (/usr/src/app/node_modules/pdfjs-dist/build/pdf.js:4250:14)
    at LoopbackPort.MessageHandler._onComObjOnMessage (/usr/src/app/node_modules/pdfjs-dist/build/pdf.js:1036:19)
    at LoopbackPort.<anonymous> (/usr/src/app/node_modules/pdfjs-dist/build/pdf.js:3835:20)
    at Array.forEach (<anonymous>)
    at /usr/src/app/node_modules/pdfjs-dist/build/pdf.js:3834:27
From previous event:
    at LoopbackPort.postMessage (/usr/src/app/node_modules/pdfjs-dist/build/pdf.js:3833:22)
    at MessageHandler.postMessage (/usr/src/app/node_modules/pdfjs-dist/build/pdf.worker.js:1328:19)
    at MessageHandler.send (/usr/src/app/node_modules/pdfjs-dist/build/pdf.worker.js:1059:10)
    at OperatorList.flush (/usr/src/app/node_modules/pdfjs-dist/build/pdf.worker.js:5813:27)
    at /usr/src/app/node_modules/pdfjs-dist/build/pdf.worker.js:28040:22
From previous event:
    at Page.getOperatorList (/usr/src/app/node_modules/pdfjs-dist/build/pdf.worker.js:28034:65)
    at /usr/src/app/node_modules/pdfjs-dist/build/pdf.worker.js:22415:14
From previous event:
    at Object.wphSetupRenderPage (/usr/src/app/node_modules/pdfjs-dist/build/pdf.worker.js:22410:37)
    at LoopbackPort.MessageHandler._onComObjOnMessage (/usr/src/app/node_modules/pdfjs-dist/build/pdf.worker.js:1036:19)
    at LoopbackPort.<anonymous> (/usr/src/app/node_modules/pdfjs-dist/build/pdf.js:3835:20)
    at Array.forEach (<anonymous>)
    at /usr/src/app/node_modules/pdfjs-dist/build/pdf.js:3834:27
From previous event:
    at LoopbackPort.postMessage (/usr/src/app/node_modules/pdfjs-dist/build/pdf.js:3833:22)
    at MessageHandler.postMessage (/usr/src/app/node_modules/pdfjs-dist/build/pdf.js:1330:19)
    at MessageHandler.send (/usr/src/app/node_modules/pdfjs-dist/build/pdf.js:1059:10)
    at PDFPageProxy_render [as render] (/usr/src/app/node_modules/pdfjs-dist/build/pdf.js:3582:39)
    at CanvasRenderer.renderPage (/usr/src/app/node_modules/pdf-extractor/lib/renderer/CanvasRenderer.js:87:28)
    at lastPromise.then (/usr/src/app/node_modules/pdf-extractor/lib/PdfExtractor.js:75:21)
From previous event:
    at PdfExtractor.renderPageData (/usr/src/app/node_modules/pdf-extractor/lib/PdfExtractor.js:74:30)
    at doc.getPage.then (/usr/src/app/node_modules/pdf-extractor/lib/PdfExtractor.js:55:59)
From previous event:
    at /usr/src/app/node_modules/pdf-extractor/lib/PdfExtractor.js:55:39
From previous event:
    at PdfExtractor.parseDocument (/usr/src/app/node_modules/pdf-extractor/lib/PdfExtractor.js:62:30)
    at PDFJSLib.getDocument.then (/usr/src/app/node_modules/pdf-extractor/lib/PdfExtractor.js:44:25)
    at runCallback (timers.js:789:20)
    at tryOnImmediate (timers.js:751:5)
    at processImmediate [as _immediateCallback] (timers.js:722:5)
From previous event:
    at PDFDocumentLoadingTask_then [as then] (/usr/src/app/node_modules/pdfjs-dist/build/pdf.js:3393:32)
    at PdfExtractor.parseFromFileBuffer (/usr/src/app/node_modules/pdf-extractor/lib/PdfExtractor.js:44:6)
    at PdfExtractor.parse (/usr/src/app/node_modules/pdf-extractor/lib/PdfExtractor.js:31:15)
    at <anonymous>

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.