Ability to change text encoding post-parse about drywetmidi HOT 7 CLOSED

I've added DecodeTextCallback property to ReadingSettings.

from drywetmidi.

Just a btw i'm one of the clone hero developers that we were talking on discord a while back, for which I also messaged this basic idea in that group message @melanchall

from drywetmidi.

Hi Matthew,

Do you need to have ability to change encoding somewhere in the future after file is parsed or you need to try different encodings during file reading?

For first case I suggest just get bytes from string and convert it to string using different encodings. Then you can set Text property of a MIDI event.

For second case I suggest to introduce some callback for text parsing to ReadingSettings/WritingSettings.

Please describe your task in more details so we can come to best solution.

Max

from drywetmidi.

@melanchall The issue with doing that is that is that decoding text from bytes into a string is not a lossless process. For example if you have a latin1 encoding decoded as UTF-8 or ASCII unsupported characters will be replaced with alternative codepoints in those encoding ranges.

from drywetmidi.

I don't really specifically need access to the text parsing stage of decoding as mentioned in option 2, that wouldn't really work too well in how the application is setup. Having access to change encoding after the file is parsed would be most useful like i mentioned originally.

Basically what I'm doing is parsing lyrics from midi files, and some of these lyrics are in various different encodings. Mainly UTF-8 and ISO-8859-1. Currently i'm checking if "U+FFFD" is produced and re trying parsing that phrase as a different encoding through a tweak in ReadContent but i'd like to avoid this hack if i can.

from drywetmidi.

I bow to option 2. My arguments:

• MIDI file hasn't such concept as "encoding". Text events just hold text string, there is no encoding here.
• Text events are mutable. What if I set new string to Text property? What encoding this string in? Without this information it's impossible to convert string to another encoding.

When I'm designing API I should always think about different cases and most versatile solution. Introducing something like "encoding" in MIDI file creates more troubles than profit.

As I mentioned in option 2 ReadingSettings will provide callback for text parsing which can be used to any processing you want including charset detection (exact API is a subject of discussion).

Also conversion between different encodings is lossless in terms of bytes. At now you can just use this code without modifying the library code:

var originalEncoding = readingSettings.TextEncoding ?? Encoding.ASCII;
var bytes = originalEncoding.GetBytes(textEvent.Text);

// convert bytes to any encoding and set Text property of event

Not good but will work. So it seems implementing text parsing callback is a best option.

You wrote:

wouldn't really work too well in how the application is setup

Can you explain why it wouldn't really work too well?

from drywetmidi.

@mdsitton So is it good to have text parsing callback?

from drywetmidi.