textify's Introduction

Textify

An HTML to plaintext conversion library for .NET Standard 2.0 written in C#.

Features

Supports HTML headings, paragraphs, containers, lists and tables (basic support)
Takes an HTML string as an input or an INode from AngleSharp
Outputs a readable text representation of the web page
Targets .NET Standard 2.0
Full test coverage

Installation

Install-Package Textify

dotnet add package Textify

Usage

HtmlToTextConverter converter = new HtmlToTextConverter();
string output = converter.Convert(html);

By default, the whole page will be converted.

If you're interested in converting only a part of it, parse the page by yourself with AngleSharp and pass the INode you're interested in. You don't need to install AngleSharp because Textify already depends on it.

HtmlParser parser = new HtmlParser();
IHtmlDocument doc = parser.ParseDocument(html);
IElement element = doc.QuerySelector("#main");

HtmlToTextConverter converter = new HtmlToTextConverter();
string output = converter.Convert(element);

Example

Input:

<div id="page">
    <header>
        <a href="http://example.com" class="site-logo">
        	<img src="logo.png" alt="Logo" />
        </a>
        <h1>
            Site title
        </h1>
    </header>
    <main>
    	<article>
        	<h2>Article title</h2>
            
            <p>
                <strong>Lorem ipsum</strong> dolor sit amet, consectetur adipiscing elit,
                sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
            </p>

            <p>Ut enim ad minim veniam, quis nostrud exercitation ullamco
            laboris nisi ut aliquip ex ea commodo consequat.</p>
            
            Here is a list of things anyway:

            <ul>
                <li>One</li>
                <li>Two</li>
                <li>Three</li>
            </ul>

            But maybe a table is nicer:<br><br>
            
            <table>
                <thead>
                	<th>Key</th>
                    <th>Value</th>
                </thead>
                <tr>
                	<td>One</td>
                    <td>Value</td>
                </tr>
            </table>
        </article>
    </main>
</div>

Output:

[IMG: Logo] [1]

++++++++++
Site title
++++++++++

-------------
Article title
-------------

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.

Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.

Here is a list of things anyway:

* One
* Two
* Three

But maybe a table is nicer:

| Key | Value |

| One | Value |

[1] http://example.com

License

MIT license.

Thanks to Jay Taylor for the inspiration with his html2text Go module.

textify's People

Contributors

Stargazers

Watchers

textify's Issues

Improve traversal

After #3 was merged, some issues were introduced:

empty bullet points are rendered in lists
empty headings are rendered
headings "symbols" (+++ or ---) now have a fixed length instead of depending on the heading length

A possible solution is to restore the previous nested traversal with multiple levels of HtmlTraversal instances, and then passing around a LinksTracker object to gather information about links that should be rendered only at the end of the top-level traversal instance.

Recommend Projects