keepsafe / ks-email-parser Goto Github PK

View Code? Open in Web Editor NEW

37.0 37.0 3.0 357 KB

A command line tool to render HTML and text emails of markdown content.

License: Apache License 2.0

Makefile 1.77% Python 95.86% CSS 0.45% HTML 1.93%

ks-email-parser's People

Contributors

Stargazers

Watchers

Forkers

flyeven sathyaeprocure marykravets

ks-email-parser's Issues

Text rendering

Desired output of Markdown to text

Links

[Link name](link_url) --> Link name (link_url)

Lists

Lists (-) are not rendered in text correct.

GUI destroys formatting and removes placeholder

File: welcome_verify.xml

Original XML:

Original visual:

When I press "edit" in the GUI, it turns into this:

notice that the {{url}} placeholder disappeared

Add globale placeholders

Right now all placeholders in one email have to be defined in the emails .xml file.

We should have a global.xml that contains text placeholders that can be used in every email. e.g. The unsubscribe text at the bottom of promotional emails.

Placeholders that are defined in the global.xml file can be used in any template with the prefix global_.

Meaning

<string name="unsubscribe"><![CDATA[Unsubscribe from KeepSafe updates, click [here]({{unsubscribe_link}})]]></string>

Can be used in the HTML template as

<tr>
    <td style="font: 12px/18px Helvetica, Arial, sans-serif; color:#AAA; padding:6px 0 10px;">{{global_unsubscribe}}</td>
</tr>

placeholder config and test should accommodate for this.

default templates

if a template can't be parsed default to a given language. the default language should be english. right now it fails and the file is missing.

example:

welcome_to_keepsafe.xml can't be parsed -> replace it with an english template

markdown parser returns <code> tag instead of HTML

Problem
Markdown gets not rendered to HTML properly.

when parsing the follow Markdown:

Congratulations!   
    You’ve unlocked a KeepSafe Premium feature and have begun a free trial.

    We just wanted to reach out to make sure that you know about all of our special features. These include:

    * **Fake PIN** - A second PIN that will lead you into a separate KeepSafe. Really handy when someone forces you to open KeepSafe.
    * **Secret Door** - Makes KeepSafe look like any other virus scanner. You need to hold on the logo to get to the PIN.
    * **Break-In Alerts** - Takes a picture of anyone trying to open your KeepSafe with the wrong PIN. Break-in Alerts will also lock your KeepSafe for a set time after multiple wrong attempts.
    * **PIN timeout** - Allows you to leave KeepSafe for 30 seconds without needing to re-enter you PIN.
    * **Private Cloud sync**  - Your KeepSafe Private Cloud now holds up to a 1000 photos.
    * **Album Lock** - Lock individual albums with an additional password.
    * **Custom Album Covers** - Choose your favorite picture within an album and set it as a custom album cover.

    We hope you try and enjoy all our Premium features.

    The KeepSafe Team

it gets rendered to

    <p>Congratulations! <br />
    You’ve unlocked a KeepSafe Premium feature and have begun a free trial.</p>
<pre><code>We just wanted to reach out to make sure that you know about all of our special features. These include:

* **Fake PIN** - A second PIN that will lead you into a separate KeepSafe. Really handy when someone forces you to open KeepSafe.
* **Secret Door** - Makes KeepSafe look like any other virus scanner. You need to hold on the logo to get to the PIN.
* **Break-In Alerts** - Takes a picture of anyone trying to open your KeepSafe with the wrong PIN. Break-in Alerts will also lock your KeepSafe for a set time after multiple wrong attempts.
* **PIN timeout** - Allows you to leave KeepSafe for 30 seconds without needing to re-enter you PIN.
* **Private Cloud sync**  - Your KeepSafe Private Cloud now holds up to a 1000 photos.
* **Album Lock** - Lock individual albums with an additional password.
* **Custom Album Covers** - Choose your favorite picture within an album and set it as a custom album cover.

We hope you try and enjoy all our Premium features.

The KeepSafe Team
</code></pre>

That renders to

Add {{local}} param to email parser

In some emails we use locals in the email e.g.

###[{{code}}](https://accounts.getkeepsafe.com/redirect/acode/{{code}}/{{bundle}}?locale=en)

Translators tend to translate them but not reliable e.g.

###[{{code}}]( https://accounts.getkeepsafe.com/redirect/acode/{{code}}/{{bundle}}?locale=ar)

I would like to introduce a local param that can be put in place instead of en in the original email and that gets replaced when rendering in all the local specific locals.

The param might be to be named different then {{local}} to distinguished it from other client side params passed in.

Task

Please come up with a solution and add documentation to how to use this.

GUI image path param not working

When starting the GUI, make the -I command take a relative path to the value

e.g.

ks-email-parser gui -I images/

Should load all the images out of that folder. Currently it loads images from the remote server.

aiohttp 2.0.7 is currently available on pypi. You guys could make use of it in some cases if any of the changes does not break current code. If it does it could be patched to work for the current version.

Update Makefile

image names in GUI get

When in the GUI and loading an email, the GUI keeps the [[name]] around the image name. When saving the file, it adds new brackets, so that on the next load you have [[[[name]]]] and so on.

Strip the [[]] when loading from disk and add them back when saving.

Add base image directory

Currently we only support an absolute path for images.

we should have a configuration file that can contain a base location for images. e.g. http://www.getkeepsafe.com/images/
When I then define a image tag with ![image](image.jpg) it should resolve on rendering to the full URL.

For images in the templates. they should have a leading string {{base_url}} that gets replaced. So that images in the HTML template would look like
<img src="{{base_url}}/image.jpg">

Faster parsing

Improve parsing speed by running it in parallel

support for right to left languages

languages that are right to left need to be supported. Not quite sure what the best way is.

add ks-email-parser to pip

We want to have ks-email-parser to be available in PIP so users can install it without checking out the source.

support URL shortener

Markdown links get rendered in their text representation as Linkname (link). When using full URLs this will break the layout of the text.

URL shortener config

ks-email-parser should support URL shortener for text rendering.

The shortener should be configurable through the config file url-shortener or command line param --url-shortener.
The value for shortener us a URL that gets called with a GET request and a placeholder {{url}} for the URL that gets shorten. e.g.

url-shortener: http://www.getkeepsafe.com/shortener/{{url}}

The shortened URL is return in the body of the request. This will be compatible with our own URL shortener and bit.ly.

Using url shortener

When rendering the text representation of emails, all URLs should be shortened when a URL shortener is defined in the config or through command line.

Option to only compile one email

For faster development it would be helpful to be able to compile a subset of emails. This can be with a direct path like including white lables. E.g.

ks-email-parser -e src/en/email_name.xml
ks-email-parser -e src/en/*

console output work with | (pipes) in shell

Console output of ks-email-parser can not be pipes into other programs or redirected into a file with >

I can't save my work.

Better error messages

We need better error messages on the console when something is not working. Something that we can do something about without reading code

e.g. this is shit

Parsing emails...
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.4/bin/ks-email-parser", line 9, in <module>
    load_entry_point('email-localization==0.0.1', 'console_scripts', 'ks-email-parser')()
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/email_localization-0.0.1-py3.4.egg/email_parser.py", line 294, in main
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/email_localization-0.0.1-py3.4.egg/email_parser.py", line 253, in parse_emails
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/email_localization-0.0.1-py3.4.egg/email_parser.py", line 239, in save_email_content_as_html
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/email_localization-0.0.1-py3.4.egg/email_parser.py", line 146, in to_html
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/email_localization-0.0.1-py3.4.egg/email_parser.py", line 132, in content_to_html
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/email_localization-0.0.1-py3.4.egg/email_parser.py", line 141, in _inline_css
TypeError: 'NoneType' object is not iterable

Better error messages

when some XML is wrongly formatted we get this error

Traceback (most recent call last):
  File "/Users/philipp/coding/ks-email-parser/venv/bin/ks-email-parser", line 9, in <module>
    load_entry_point('ks-email-parser==0.2.2', 'console_scripts', 'ks-email-parser')()
  File "/Users/philipp/coding/ks-email-parser/venv/lib/python3.4/site-packages/ks_email_parser-0.2.2-py3.4.egg/email_parser/__init__.py", line 64, in main
  File "/Users/philipp/coding/ks-email-parser/venv/lib/python3.4/site-packages/ks_email_parser-0.2.2-py3.4.egg/email_parser/__init__.py", line 26, in parse_emails
  File "/Users/philipp/coding/ks-email-parser/venv/lib/python3.4/site-packages/ks_email_parser-0.2.2-py3.4.egg/email_parser/renderer.py", line 157, in render
  File "/Users/philipp/coding/ks-email-parser/venv/lib/python3.4/site-packages/ks_email_parser-0.2.2-py3.4.egg/email_parser/renderer.py", line 115, in render
  File "/Users/philipp/coding/ks-email-parser/venv/lib/python3.4/site-packages/ks_email_parser-0.2.2-py3.4.egg/email_parser/renderer.py", line 115, in <dictcomp>
  File "/Users/philipp/coding/ks-email-parser/venv/lib/python3.4/site-packages/ks_email_parser-0.2.2-py3.4.egg/email_parser/renderer.py", line 97, in _render_placeholder
  File "/Users/philipp/coding/ks-email-parser/venv/lib/python3.4/site-packages/ks_email_parser-0.2.2-py3.4.egg/email_parser/renderer.py", line 74, in _inline_css
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/xml/etree/ElementTree.py", line 1325, in XML
    parser.feed(text)
xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 18, column 13

Something that indicates what XML file and maybe line would be good.

nose conflict

(venv) Philipps-MacMini:emails philipp$ ks-email-parser config placeholders
Traceback (most recent call last):
  File "/Users/philipp/coding/ks-email-parser/venv/lib/python3.4/site-packages/pkg_resources.py", line 2697, in <module>
    working_set.require(__requires__)
  File "/Users/philipp/coding/ks-email-parser/venv/lib/python3.4/site-packages/pkg_resources.py", line 669, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/Users/philipp/coding/ks-email-parser/venv/lib/python3.4/site-packages/pkg_resources.py", line 576, in resolve
    raise VersionConflict(dist,req) # XXX put more info here
pkg_resources.VersionConflict: (nose 1.3.7 (/Users/philipp/coding/ks-email-parser/venv/lib/python3.4/site-packages), Requirement.parse('nose==1.3.4'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/philipp/coding/ks-email-parser/venv/bin/ks-email-parser", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/Users/philipp/coding/ks-email-parser/venv/lib/python3.4/site-packages/pkg_resources.py", line 2701, in <module>
    parse_requirements(__requires__), Environment()
  File "/Users/philipp/coding/ks-email-parser/venv/lib/python3.4/site-packages/pkg_resources.py", line 572, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: nose==1.3.4

Add config options to --help menu

When calling

ks-email-parser config --help

Add the available config options to the help menu. e.g. placeholders

include segment name and file name for exception

When throwing an exception. include the file name and the segment ID so it's easier to find where the error is.

e.g.

Traceback (most recent call last):
  File "/Users/philipp/coding/ks-email-parser/venv/bin/ks-email-parser", line 9, in <module>
    load_entry_point('ks-email-parser==0.2.6', 'console_scripts', 'ks-email-parser')()
  File "/Users/philipp/coding/ks-email-parser/venv/lib/python3.4/site-packages/ks_email_parser-0.2.6-py3.4.egg/email_parser/__init__.py", line 62, in main
  File "/Users/philipp/coding/ks-email-parser/venv/lib/python3.4/site-packages/ks_email_parser-0.2.6-py3.4.egg/email_parser/__init__.py", line 31, in parse_emails
  File "/Users/philipp/coding/ks-email-parser/venv/lib/python3.4/site-packages/ks_email_parser-0.2.6-py3.4.egg/email_parser/renderer.py", line 177, in render
  File "/Users/philipp/coding/ks-email-parser/venv/lib/python3.4/site-packages/ks_email_parser-0.2.6-py3.4.egg/email_parser/renderer.py", line 102, in render
  File "/Users/philipp/coding/ks-email-parser/venv/lib/python3.4/site-packages/ks_email_parser-0.2.6-py3.4.egg/email_parser/renderer.py", line 102, in <dictcomp>
  File "/Users/philipp/coding/ks-email-parser/venv/lib/python3.4/site-packages/ks_email_parser-0.2.6-py3.4.egg/email_parser/renderer.py", line 83, in _render_placeholder
  File "/Users/philipp/coding/ks-email-parser/venv/lib/python3.4/site-packages/ks_email_parser-0.2.6-py3.4.egg/email_parser/renderer.py", line 56, in _inline_css
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/xml/etree/ElementTree.py", line 1325, in XML
    parser.feed(text)
xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 6, column 24

Formating validator report

there should be a option that runs the content validator and creates a report for all emails in one HTML file that show wrong formatting and missing markdown items in an overview.

GUI side by side layout

ks-email-parser gui should make it easier to create emails in a visual way.

To improve this we want to be able to show final result of the email next to the edit mode.

On the left side the email will be displayed with all it's placeholders as text boxes that contain markdown. on the right side the email will be displayed in it's rendered from.

The rendered email will be updated when:

courser unfocused a text field
user presses Preview
every 10 seconds if content has changed

warn on extra placeholders

if there is a key in the xml which is not later used in the template display warning

for ex. there is a string with key content in the xml but there is no {{content}} in html template

<string name="the_name"> always wrapped in <p>

When defining a piece of content with

<string name="footer_content"><![CDATA[content]]></string>

the result is always

<p>content</p>

For the case of defining URLs with {{}} Params that then get included in <img like img src="{{image_url}}" /> does not work. as the image your will be wrapped in <p></p>

We need a different xml attribute or attribute name to define values that should not be converted with Markdown.

add link checker

if strings are links (via regEx) and don't contain a a placeholder like {{locale}} then we should besides checking that the link is valid, also check that the translation of that string is the same link

add save param to ks-email-parser gui

The gui command should have an addition --save parameter that takes a bash file that gets executed when the user hits save in the gui.

This will allow to do custom tasks like committing changes to git on save.