tfidf(s): remove normalization, improve docstring about texthero HOT 4 CLOSED

Working on this with @mk2510 🐐

from texthero.

Should the new tfidf function already return such a Representation Series?

Working on it, we currently implemented it so that it calculates the Representation Series correctly (using some of your code from #43 ) and then use a function representation_series_to_flat_series to make the output the same as tfidf currently produces. Thus, when #43 will be fully implemented later on, in tfidf only one line has to be changed to get the Representation Series output. But we could also already return the Representation Series output now.

from texthero.

Working on it, we currently implemented it so that it calculates the Representation Series correctly (using some of your code from #43 ) and then use a function representation_series_to_flat_series to make the output the same as tfidf currently produces. Thus, when #43 will be fully implemented later on, in tfidf only one line has to be changed to get the Representation Series output. But we could also already return the Representation Series output now.

representation_series_to_flat_series: that's sounds great! (this function will be super useful anyway for users to move from the two representation). By the way, we might want to figure out two different XSeries names to differentiate between the two representations. (Also, for instance, pca will still return a Pandas Series where each cell is a list, right?)

Should the new tfidf function already return such a Representation Series?

What's your opinion? We might also want to do both tasks 1 and 2 in #84 before releasing a new version. This is fine as well. In this case, we can already return the "new" Pandas Representation but this means that many tests will fail ...

... have a look at the unit tests to check for the "new" Pandas Representation under #43 (it took me a long while to implement that, it might spare you a bit of time too!)

from texthero.

We just opened a PR at #97 .

By the way, we might want to figure out two different XSeries names to differentiate between the two representations

Yes, that is very important to discuss, and probably should be handled later on in #85 2.

What's your opinion? We might also want to do both tasks 1 and 2 in #84 before releasing a new version.

I think it makes sense to still return a "old" output at the moment, so that's what we did currently. It probably makes sense to overhaul all the representation functions at the same time in #85 2. As the new tfidf already calculates with the representation series and just gives the old output, it will be easy to change.

from texthero.