Better Language Detection based on word list about jetwick HOT 5 CLOSED

use google translate (to english) when indexing.
does it give you the recognized lang?

from jetwick.

ah, yes. while indexing I could detect the language + even translate it. But I am really sure that google will kick me out if I'm hitting their service >1mio per day :-)

from jetwick.

True. You could translate in batches though. (100 tweets = 1 'paragraph')?
I am really sure or "I am not really sure"? ;}
Or let's just user more than 200 words ;]

from jetwick.

True. You could translate in batches though.

but then I'll have to be sure that all the batched tweets have the same language ...

what do you mean with:
"I am really sure or "I am not really sure"? ;} Or let's just user more than 200 words ;]"

from jetwick.

This is fixed now with an english wordlist of 2.6k words then translated to various languages. btw I added french and portuguese

I'll need to push this to github though ...

from jetwick.