Giter Club home page Giter Club logo

easygettext's Introduction

easygettext

Build Status codecov.io

Radically simple gettext tokens extraction tool for:

files.

Also ships with a PO-to-JSON converter.

Installation

You can install the easygettext package by running

npm install --save-dev easygettext

or

yarn add --dev easygettext

Usage & Examples

HTML token extraction

Simply invoke the tool on the templates you want to extract a POT dictionary template from. The optional '--output' argument enables you to directly output to a file.

gettext-extract --output dictionary.pot foo.html bar.pug component.vue sourcefile.js

CLI usage:

gettext-extract [--attribute EXTRA-ATTRIBUTE] [--filterPrefix FILTER-PREFIX] [--output OUTFILE] [--parser auto|acorn|babel] <FILES>

It recognizes the following token flavours (currently; feel free to extend it!)

<div translate>Hello World</div>

<div translate translate-context="According to...">Hello World</div>

<div translate translate-comment="My comment...">Hello World</div>

<div translate translate-plural="Hello worlds">Hello World</div>

<div placeholder="{{ 'Hello World' | translate }}"></div>

<div placeholder="{{ scopeVariable || ('Hello World' | translate) }}"></div>

<get-text>Hello World</get-text>

<i18n>Hello World</i18n>

<translate>Hello World</translate>

<!--  The following becomes 'Broken strings are joined'  --> 
<span ng-bind="{{ 'Broken '
 + 'strings ' +
 'are ' + 
 'joined' |translate}}"></span>

 <span ng-bind="'Bed n\'' + ' breakfast' |translate"></span>

 <!-- JavaScript expressions are parsed and compiled -->
<span ng-bind="true ? 'Always' : 'Never' |i18n "></span>

<!--  By supplying the  --filterPrefix '::' parameter  -->  
<span>{{:: 'Something …' |translate}}</span>

<!--  The default delimiters '{{' and '}}' must be changed to empty strings to handle these examples  -->
<span ng-bind=":: 'Something …' |translate"></span>

<div placeholder="'Hello World' | translate"></div>

You can combine any context, comment and plural together. Also, you can use 'i18n' instead of 'translate' as master token.

You can also provide your own master tokens:

gettext-extract --attribute v-translate --output dictionary.pot foo.html bar.jade

gettext-extract --attribute v-translate --attribute v-i18n --output dictionary.pot foo.html bar.jade

gettext-extract --startDelimiter '[#' --endDelimiter '#]' --output dictionary.pot foo.html bar.jade

gettext-extract can also remove optional HTML whitespaces inside tags to translate (see PR 68 for more information):

gettext-extract --removeHTMLWhitespaces --output dictionary.pot foo.html

Supports parsing with acorn and babel

If you want to use optional-chaining, nullish-coalesce or any babel plugin, you might want to set the parameter --parser babel.

gettext-extract --parser babel --output dictionary.pot foo.html

It can be set to: --parser auto|acorn|babel

More info at PR 72

Javascript/ES7 token extraction

The usage stays the same but with a Javascript file !

gettext-extract somefile.js
const myVar = $gettext("My fantastic msgid")

const myConcatVar = $gettext(
  "My"
  + "fantastic"
  + "msgid"
)

const myTempVar = $gettext(
  `My
  fantastic
  msgid`
)

const myContextualizedVar = $pgettext("some context", "Some other string")

const myPluralVar = $ngettext(...)

We recognize the $gettext, $pgettext and $ngettext tokens as the ones from which we can extract text from.

Those tokens are frozen for now, but feel free to make a pull request and add support for variable ones :)

We currently can't extract template strings with variables though.

Extract from Vue components

You can also extract the strings marked as translatable inside the <script> and <template> sections of Vue.js components:

gettext-extract MyComponent.vue

With a component that looks like this:

    <template>
        <h1>{{ greeting_message }}</h1>
        <p>{{ number_of_people_here }}</p>

        <h2 v-translate> Some text to be translated
    </template>

    <script>
        export default {
            name: "greetings",
            computed: {
                greeting_message() {
                    return this.$gettext("Hello there!")
                },
                number_of_people_here(nb_folks) {
                    return this.$ngettext("There is ${ n } person here.", "There are ${ n } persons here.", nb_folks)
                }
            }
        }
    </script>

The Javascript & HTML (or Pug) extraction within a Vue component works with the same rules as stated upper in this document.

Extracting from multiple files

gettext-extract needs the exact file paths to work. If you want to extract gettext from all files in a folder, you can use the UNIX find command. Here is an example as a npm script:

{
  //...
  "scripts": {
    // This is for VueJS files, please adapt for HTML or Jade/Pug templates
    "extract-gettext-cli": "gettext-extract --attribute v-translate --output dictionary.pot $(find scripts/src/components -type f -name '*.vue')"
  }
}

gettext-compile

Outputs or writes to an output file, the sanitized JSON version of a PO file.

gettext-compile --output translations.json fr.po en.po de.po

AngularJS

If you use easygettext to extract files from an AngularJS code base, you might find the following tips helpful.

To cover the cases (1)

<input placeholder="{{:: 'Insert name …' |translate }}">
<input placeholder="{{ 'Insert name …' |translate }}">

and (2)

<a href="#" ng-bind=":: 'Link text' |translate"></a>
<a href="#" ng-bind="'Link text' |translate"></a>
<a href="#">{{::'Link text' |translate}}</a>
<a href="#">{{'Link text' |translate}}</a>

you should run the extraction tool twice. Once with the command-line arguments

--startDelimiter '{{' --endDelimiter '}}' --filterPrefix '::'

and once with the command-line arguments

--output ${html_b} --startDelimiter '' --endDelimiter '' --filterPrefix '::'

The following Bash script shows how msgcat might help

#!/usr/bin/env bash

input_files="$(find ./src/ -iname \*\.html)"
workdir=$(mktemp -d "${TMPDIR:-/tmp/}$(basename $0).XXXXXXXXXXXX") || exit 1

html_a=${workdir}/messages-html-interpolate.pot
html_b=${workdir}/messages-html.pot

./dist/extract-cli.js --output ${html_a} --startDelimiter '{{' --endDelimiter '}}' --filterPrefix '::' ${input_files}
./dist/extract-cli.js --output ${html_b} --startDelimiter '' --endDelimiter '' --filterPrefix '::' ${input_files}

# Extract gettext “messages” from JavaScript files here, into ${es_a} …
es_a=${workdir}/ecmascript.pot
# [...] > ${es_a}

# Merge the different catalog templates with `msgcat`:  
merged_pot=${workdir}/merged.pot
msgcat ${html_a} ${html_b} ${es_a} > ${merged_pot}

# Cleanup, in case `msgcat` gave merge-conflicts in catalog header.
header=${workdir}/header.pot
sed -e '/^$/q' < ${html_a} > ${header}

body=${workdir}/body.pot
sed '1,/^$/d' < ${merged_pot} > ${body}

cat ${header} ${body} > ${output_file}

# Remove temporary directory with working files.
rm -r ${workdir}

Please note that the script needs to be modified to match your needs and environment.

Testing

Run the tests using jest:

npm test

Testing the CLI

Run:

./src/extract-cli.js --attribute v-translate --attribute v-i18n ~/output.html

Motivation

angular-gettext is a very neat tool, that comes with Grunt tooling to extract translation tokens from your Pug/Jade/HTML templates and JavaScript code.

Unfortunately, this has two drawbacks:

  • It isn't a simple command-line interface, and forces the usage of Grunt;
  • It is angular-specific.

This library comes up with two simple CLI tools to extract and compile your HTML tokens.

Why This Library?

Our frontend toolchain, systematic doesn't rely on Grunt/Gulp/Broccoli/... and uses a combination of simple Makefiles and CLI tools to do the job.

The toolchain being framework-agnostic, we don't want to depend on Angular to extract our HTML translation tokens. On another note, we use the standard xgettext tool to extract our JavaScript translation tokens.

Nevertheless, the way angular-gettext does it (with tokens, directly in HTML) is elegant, is used by many other libraries and will also be the way to do it in Angular2.

Also, by utilizing either acorn or babel, this tool will parse and compile typical JavaScript expressions used in translate-filter expressions. For example, exposed to a (AngularJS-based) fragment like

<span ng-bind="isNight ? 'Moon' + 'shine' : 'Day' + 'light' |translate"></span>
<span ng-bind="isC ? 'C' + (isD ? 'D' : 'd') : 'c' + (isE ? 'E' : 'e') |i18n "></span>

will produce the following strings

Moonshine
Daylight
CD
Cd
cE
ce

Which will be correctly looked up and translated during runtime, at least by angular-gettext.

Known Issues

TypeScript support is currently limited in that line numbers are not tracked and won't show in generated .po files. This can lead to issues with more complex translations and should be kept in mind.

Credits

Thanks a million to @rubenv for the initial ideas and implementations in angular-gettext-tools, which inspired me a lot.

Thanks to ES6 and Babel for being awesome.

Licensing

MIT

easygettext's People

Contributors

briquet avatar brunobg avatar bulv1ne avatar cedricco avatar dependabot[bot] avatar gartmeier avatar georgebarbarosie avatar gorkat avatar hyzual avatar janlazo avatar junish avatar knogobert avatar lburg avatar lesuisse avatar lzurbriggen avatar martigan avatar mtr avatar muromec avatar ouradze avatar p4t5h3 avatar rkurbatov avatar rsebille avatar scrum avatar sebastien-nicolas avatar shin-- avatar stephane avatar tcitworld avatar vperron avatar youchenlee avatar ziemenz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

easygettext's Issues

gettext-extract - Unexpected Token (

First of all, thank you for the work!

What is the problem?

As the title says, I have a problem with getting easygettext to work in CentOs7.

The command I ran is:
./node_modules/.bin/gettext-extract --attributes v-i18n ./src/main.js

The errors I get are as follow:

// Base path is masked as ~ for confidentiality

~/node_modules/easygettext/src/extract.js:283
);
^

SyntaxError: Unexpected token )
    at createScript (vm.js:56:10)
    at Object.runInThisContext (vm.js:97:10)
    at Module._compile (module.js:549:28)
    at Object.Module._extensions..js (module.js:586:10)
    at Module.load (module.js:494:32)
    at tryModuleLoad (module.js:453:12)
    at Function.Module._load (module.js:445:3)
    at Module.require (module.js:504:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> 
(~/node_modules/easygettext/src/extract-cli.js:9:17)

Expected Behavior

No error, and text extracted successfully

Environment

CentOs7
easygettext 2.17.0
GNU gettext 0.19.8.1

Additional Info

I tried the same command on another instance using Windows 10, and it works without a problem.

The windows environment is as follows:
Windows 10
easygettext 2.17.0
GNU gettext 0.21

Incorrect processing of <br> tag in the .vue-files

Somewhere between v2.3.0 and v2.4.0 a new behavior of processing tags within translatable messages has been introduced. In v2.3.0 strings like

<label for="radio3" v-translate>Some string <br>splitted in two lines</label>
<label for="radio4" v-translate>Another string with XHTML tag <br/>...continuation</label>

both gave <br> in .pot/.po-files:

 msgid "Some string <br>splitted in two lines"
 msgid "Another string with XHTML tag <br>...continuation"

So far so good.

In v.2.4.0 <br/> is translated to <br/> in the msgids. Unfortunately strings with <br> are translated to incorrect msgids like this:

msgid "Some string <br>splitted in two lines</br>"

The problem with <br/> is that it's supported by vue-gettext. Its processing engine splits such strings by inner HTML tags and then joins parts of the message with opening tags to be compatible with previous versions of easygettext. Thus for a string Another string with XHTML tag <br/>...continuation it still constructs Another string with XHTML tag <br>...continuation and expectedly fails to find it in the dictionary, making impossible to use easygettext-v2.4.0 with vue-gettext.

Escape quotes bug

Example from docs with same quote interpolation

div(placeholder="{{ 'I\'m best of the best' | translate }}")

get me error

Trace: SyntaxError: Unexpected token, when trying to parse `{{ 'I'm best of the best' | translate }}`

I would like to know why I am getting errorsfor this case.
Thanks

PS: I think solution, but problem from above is a bug.

My solution:

  1. Excape self-made
placeholder="{{ &quot;I'm best of the best&quot; | translate }}"
  1. Auto escape by pug
placeholder=`{{ "I\'m best of the best" | translate }}`

BUT
My solution not working as needed if we have 2 types of quotes

placeholder=`{{ "I\'m best of the \"best\"" | translate }}`

That extract

msgid "I'm best of the &quot;best&quot;"

instead of

msgid "I'm best of the \"best\""

How to fix this issues?
Thanks!

Support nativescript-vue/multiple templates

Nativescript vue allow files with multiple templates to target native and web, like this one:

<template web>
  <div class="container">
      <translate>Translate this</translate>
  </div>
</template>
<template native>
  <Page class="page">
      <Label :text="'Translate that' | translate"></Label>
  </Page>
</template>

<script>
export default {
};
</script>

<style>
</style>

gettext-extract does not handle this properly, apparently only dealing with the second section. It'd be nice to support multiple template sections. Is this viable?

Support es7 decorators

Hi, acorn doesn't support ES7 decorators by default.

We're using vue-class-component and it's failing with Unexpected character '@'

Vue file

<template>
<div>
  Test1: <span v-translate>Test2</span>
</div>
</template>

<script>
import { Vue, Component } from "vue-property-decorator";
import Header from "@/header.vue";

@Component({
  components: {
    Header
  }
})

export default class App extends Vue {
  details = {}
}
</script>

Command

node_modules/.bin/gettext-extract --attribute v-translate --quiet --output /tmp/template.pot src/App.vue
[easygettext] extracting: 'src/App.vue
[easygettext] could not read: 'src/App.vue
Trace: { SyntaxError: Unexpected character '@' (10:0)
    at Parser.pp$4.raise (node_modules/acorn/dist/acorn.js:2756:13)
    at Parser.pp$8.getTokenFromCode (node_modules/acorn/dist/acorn.js:4905:8)
    at Parser.<anonymous> (node_modules/acorn-class-fields/inject.js:41:21)
    at Parser.getTokenFromCode (node_modules/acorn-private-methods/inject.js:30:21)
    at Parser.pp$8.readToken (node_modules/acorn/dist/acorn.js:4627:15)
    at Parser.pp$8.nextToken (node_modules/acorn/dist/acorn.js:4618:15)
    at Parser.pp$8.next (node_modules/acorn/dist/acorn.js:4575:8)
    at Parser.pp.eat (node_modules/acorn/dist/acorn.js:577:10)
    at Parser.pp.semicolon (node_modules/acorn/dist/acorn.js:624:13)
    at Parser.pp$1.parseImport (node_modules/acorn/dist/acorn.js:1482:8)
    at Parser.pp$1.parseStatement (node_modules/acorn/dist/acorn.js:801:47)
    at Parser.parseStatement (node_modules/acorn-dynamic-import/lib/inject.js:47:31)
    at Parser.parseStatement (node_modules/acorn-import-meta/inject.js:39:50)
    at Parser.pp$1.parseTopLevel (node_modules/acorn/dist/acorn.js:706:23)
    at Parser.parse (node_modules/acorn/dist/acorn.js:551:15)
    at Object.parse (node_modules/acorn/dist/acorn.js:5287:37)
  pos: 972,
  loc: Position { line: 10, column: 0 },
  raisedAt: 972 }
    at node_modules/easygettext/src/extract-cli.js:71:13
    at Array.forEach (<anonymous>)
    at Object.<anonymous> (node_modules/easygettext/src/extract-cli.js:57:7)
    at Module._compile (module.js:652:30)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)
    at tryModuleLoad (module.js:505:12)
    at Function.Module._load (module.js:497:3)
    at Function.Module.runMain (module.js:693:10)
    at startup (bootstrap_node.js:191:16)
    at bootstrap_node.js:612

I have tried to add the ES7 plugin to acorn at easygettext/src/extract.js:6 but it didn't make any difference or maybe I'm not doing it right?

Better support for HTML tags inside translated strings

(This is a followup to Polyconseil/vue-gettext#93.)

As an example, let's look at the following simple Vue template:

<template>
  <translate>
     XXX
     <div translate-name="NNN">foo</div>
     YYY
  </translate>
</template>

Currently this is extracted as:

msgid ""
"XXX\n"
"   <div translate-name=\"NNN\">foo</div>\n"
"   YYY"

I would propose that instead of requiring HTML to be embedded inside translation, which opens your to XSS attacks, and which makes it possible only to use standard HTML but not things like Vue components inside translations, we would parse such string in a smarter way to allow Vue library like vue-gettext to render those HTML segments through Vue.

I would suggest that the above is parsed as:

msgid "XXX %{NNN} YYY"

Because it is HTML, we can collapse whitespace. Moreover, we provide a placeholder to interpolate the internal HTML in. So in a way, only top-level text nodes would be used and everything else would be delegated to placeholder and library to fill. translate-name can be used to control the name of the placeholder (otherwise by default we could use some sequence number).

We should also support parsing the case where internal <div> is translated as well. In that case foo should become another msgId.

Add plain JavaScript support

I have a Vue component mixin in form of a plain *.js file which includes translatable strings. By default it is ignored because it does not have the allowed file extension defined in src/extract-cli.js.

To automatically execute string extraction I wrote a custom webpack plugin. That webpack plugin used to execute the command line script. To circumvent the limitation based on file extension I copied and adapted much of extract-cli.js and it works fine like this (partially renamed identifiers):

if (ext === 'js') {
    const extractedStringsFromScript = javascriptExtraction.extractStringsFromJavascript(file, data);
    extractor.processStrings(extractedStringsFromScript);
} else {
    extractor.parse(file, extraction.preprocessTemplate(data, ext));
    extractor.parseVueJavascript(file, extraction.preprocessScriptTags(data, ext));
}

I just noticed that gettext annotations like comments and context are ignored in JavaScript and also seem not to be looked for anyway. That might be worth a different issue, if the feature request of plain JavaScript support is not declined.

New NPM Release?

Dear @vperron,

I merged PR #29 into master before Christmas last year, and wonder if you would be so kind as to publish a new NPM release?

How to compile from pug files that uses pug locals?

[easygettext] could not read: './dev/pages/base-variables.pug' using acorn as parser
Trace: TypeError: ./dev/pages/base-variables.pug:13
    11| - let langInUrl = false
    12| - let appLinkIconSuffix = ''
  > 13| -
    14|         let languages = Object.keys(isoLanguages).filter(code => {
    15|                 return _.includes(supportedLanguages, code);
    16|         }).map(code => {

Cannot read property 'includes' of undefined
    at eval (eval at wrap (/Users/7iomka/Desktop/Projects/Core_new/core_new/node_modules/pug/node_modules/pug-runtime/wrap.js:6:10), <anonymous>:752:11)

My locals and pug build looks like this

const _ = require('lodash');
const locals = { _, ... }
...

 // Compile the source code of file with compilation option from pug-config
      const compiledFunction = pug.compileFile(source, options);
      // Render a set of data
      const html = compiledFunction({
        ...locals,
        langToRender: lang,
        variablesToRender: locals.includeVariables(lang),
      });

Incorrect processing of <br> tags when exporting

Hello! First of all, thanks for your work.

What is the problem?

When you run the export command for a .vue file, translation strings containing the <br> tag are not generated correctly in .po files: </br> is added at the end of any such line. Therefore, the translation does not work because the keys are not the same.

Source string in template:
image

String in .po file after export:
image

Expected Behavior

An extra closing </br> tag should not be generated.

Environment

Windows 10
Vue 3.0.0
@vue/cli 4.5.10
easygettext 2.16.1

2.14.0 stopped extracting strings from some Vue components

I'm using this project together with vue-gettext. I have Vue components in my project written in two different ways.

In .vue files like this:

<template>
  <h1 v-translate>Test String 1</h1>
</template>

<script>
export default {
  name: 'TestComponent1'
}
</script>

And in .js files written like this:

import Vue from 'vue'

const TestComponent2 = Vue.component('TestComponent2', {
  template: `
    <h1 v-translate>Test String 2</h1>
  `
})

export default TestComponent2

I am trying to update from 2.13.0 to 2.14.0, but that causes the strings from Vue components written using the second style to be ignored.

I set up a repository that shows this. Take a look here: https://github.com/angelikatyborska/easygettext-2-14-0-bug/commits/master

First I set up everything using easygettext 2.13.0 (angelikatyborska/easygettext-2-14-0-bug@fd64bde) and running gettext-extract (see my 'gettext-extract' script in package.json for details) extracts strings from both components like I want it to. Then, I updated to easygettext 2.14.0 (angelikatyborska/easygettext-2-14-0-bug@f5c4e73) and running gettext-extract removed some of the strings.

extract of pure js files is not working ...

The given example is simply not working - https://www.npmjs.com/package/easygettext#javascriptes7-token-extraction

➭ ./node_modules/.bin/gettext-extract somefile.js 
[easygettext] extracting: 'somefile.js
msgid ""
msgstr ""
"Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Generated-By: easygettext\n"
"Project-Id-Version: \n"

➭ more somefile.js 
const myVar = $gettext("My fantastic msgid")

const myConcatVar = $gettext(
  "My"
  + "fantastic"
  + "msgid"
)

const myTempVar = $gettext(
  `My
  fantastic
  msgid`
)

const myContextualizedVar = $pgettext("some context", "Some other string")

gettext-extract is not extracting text from vue files.

Here's the relevent codes.

First, a Component:

<template>
  <div class="hello">
    <h1>{{msg}}</h1>
    <span v-translate>Hello!</span>
    <translate>Hello!</translate>
  </div>
</template>

<script>
 export default {
   name: 'HelloWorld',
   data () {
     return {
       msg: 'Welcome to Your Vue.js App'
     }
   }
 }
</script>

Here's the Makefile modified per my project requirement:

# On OSX the PATH variable isn't exported unless "SHELL" is also set, see: http://stackoverflow.com/a/25506676
SHELL = /bin/ash
NODE_BINDIR = /app/node_modules/.bin
export PATH := $(NODE_BINDIR):$(PATH)

# Where to write the files generated by this makefile.
OUTPUT_DIR = /app/server/assets/translation
SOURCE_DIR = /app/server/assets/js

# Available locales for the app.
LOCALES = en_US bn_BD

# Name of the generated .po files for each available locale.
LOCALE_FILES ?= $(patsubst %,$(OUTPUT_DIR)/locale/%/LC_MESSAGES/app.po,$(LOCALES))

GETTEXT_HTML_SOURCES = $(shell find $(SOURCE_DIR) -name '*.vue' -o -name '*.html' 2> /dev/null)
GETTEXT_JS_SOURCES = $(shell find $(SOURCE_DIR) -name '*.vue' -o -name '*.js')

# Makefile Targets
.PHONY: clean makemessages translations

clean:
	rm -f /tmp/template.pot $(OUTPUT_DIR)/translations.json

makemessages: /tmp/template.pot

translations: $(OUTPUT_DIR)/translations.json

# Create a main .pot template, then generate .po files for each available language.
# Thanx to Systematic: https://github.com/Polyconseil/systematic/blob/866d5a/mk/main.mk#L167-L183
/tmp/template.pot: $(GETTEXT_HTML_SOURCES)
# `dir` is a Makefile built-in expansion function which extracts the directory-part of `$@`.
# `$@` is a Makefile automatic variable: the file name of the target of the rule.
# => `mkdir -p /tmp/`
	mkdir -p $(dir $@)
	which gettext-extract
# Extract gettext strings from templates files and create a POT dictionary template.
	gettext-extract --attribute v-translate --quiet --output $@ $(GETTEXT_HTML_SOURCES)
# Extract gettext strings from JavaScript files.
	xgettext --language=JavaScript --keyword=npgettext:1c,2,3 \
		--from-code=utf-8 --join-existing --no-wrap \
	 	--package-name=logolagbe \
	 	--package-version=1.0.0 \
	 	--output $@ $(GETTEXT_JS_SOURCES)
# Generate .po files for each available language.
	@for lang in $(LOCALES); do \
		export PO_FILE=$(OUTPUT_DIR)/locale/$$lang/LC_MESSAGES/app.po; \
		echo "msgmerge --update $$PO_FILE $@"; \
		mkdir -p $$(dirname $$PO_FILE); touch $$PO_FILE \
		[ -f $$PO_FILE ] && msgmerge --lang=$$lang --update $$PO_FILE $@ || msginit --no-translator --locale=$$lang --input=$@ --output-file=$$PO_FILE; \
		msgattrib --no-wrap --no-obsolete -o $$PO_FILE $$PO_FILE; \
	done;

$(OUTPUT_DIR)/translations.json: clean /tmp/template.pot
	mkdir -p $(OUTPUT_DIR)
	gettext-compile --output $@ $(LOCALE_FILES)

And here's a .gettext.json because it was complaining that there is no configuration:

{
  "js": {
    "parsers": [
      {
        "expression": "gettext",
        "arguments": {
          "text": 0
        }
      },
      {
        "expression": "ngettext",
        "arguments": {
          "text": 0,
          "textPlural": 1
        }
      },
      {
        "expression": "pgettext",
        "arguments": {
          "context": 0,
          "text": 1
        }
      }
    ],
    "glob": {
      "pattern": "src/**/*.ts",
      "options": {
        "ignore": "src/**/*.spec.ts"
      }
    }
  },
  "html": {
    "parsers": [
      {
        "element": "[translate]",
        "attributes": {
          "textPlural": "translate-plural",
          "context": "translate-context"
        }
      },
      {
        "attribute": "translate-text",
        "attributes": {
          "textPlural": "translate-plural",
          "context": "translate-context"
        }
      }
    ],
    "glob": {
      "pattern": "src/**/*.html"
    }
  },
  "headers": {
    "Language": ""
  },
  "output": "translations/template.pot"
}

But it is not compiling. Showing this error:

/app/server # make makemessages
mkdir -p /tmp/
which gettext-extract
/app/node_modules/.bin/gettext-extract
gettext-extract --attribute v-translate --quiet --output /tmp/template.pot /app/server/assets/js/App.vue /app/server/assets/js/components/FooterComponent.vue /app/server/assets/js/components/HomeAnimation.vue /app/server/assets/js/components/LoginForm.vue /app/server/assets/js/components/test.html /app/server/assets/js/components/HelloWorld.vue /app/server/assets/js/components/Bla.vue /app/server/assets/js/components/HeaderComponent.vue

  0 messages extracted
  --------------------------
  0 total usages
  0 files (0 with messages)
  0 message contexts

xgettext --language=JavaScript --keyword=npgettext:1c,2,3 \
        --from-code=utf-8 --join-existing --no-wrap \
        --package-name=logolagbe \
        --package-version=1.0.0 \
        --output /tmp/template.pot /app/server/assets/js/store/actions.type.js /app/server/assets/js/store/auth.module.js /app/server/assets/js/store/mutations.type.js /app/server/assets/js/store/index.js /app/server/assets/js/App.vue /app/server/assets/js/router/index.js /app/server/assets/js/common/token.service.js /app/server/assets/js/common/config.js /app/server/assets/js/common/api.service.js /app/server/assets/js/components/FooterComponent.vue /app/server/assets/js/components/HomeAnimation.vue /app/server/assets/js/components/LoginForm.vue /app/server/assets/js/components/HelloWorld.vue /app/server/assets/js/components/Bla.vue /app/server/assets/js/components/HeaderComponent.vue /app/server/assets/js/index.js
xgettext: error while opening "/tmp/template.pot" for reading: No such file or directory
make: *** [Makefile:38: /tmp/template.pot] Error 1

Feature request: use peer/optional dependencies to reduce node_modules

I am using this package for translations in Javascript and Vue. I would like to avoid installing pug listed in https://github.com/Polyconseil/easygettext/blob/master/package.json. Likewise, if I am extracting translations from pug files only, then I would like to avoid installing vue packages.

I am mentioning pug here because I want to avoid installing its transitive dependencies, specifically https://github.com/fsevents/fsevents. fsevents is hard to upgrade because of other higher-level packages that depend on it but removing pug is one less linkage to deprecated versions of fsevents.

How to use gettext-extract?

Hello.
Could someone please explain, how to use tool "gettext-extract"? If I simply call "gettext-extract" with params in command line, it shows error, because does not know "gettext-extract" command. Should I install it globally or installing "npm install --dev easygettext" is enough?

Thank you.

gettext commands work but end with DeprecationWarning

./node_modules/easygettext/dist/extract-cli.js --output *.vue

This ends with a warning:

[easygettext] extracting: 'foo.vue
(node:41795) DeprecationWarning: Calling an asynchronous function without callback is deprecated.

compile-cli.js also ends with such warning.

Nothing serious, but wanted to mention it.

HTML broken since 2.10.0

Parsing a HTML file throws the following error

$ npx gettext-extract po.html
[easygettext] extracting: 'po.html
[easygettext] could not read: 'po.html
Trace: SyntaxError: Unexpected token (1:0)
    at Object.pp$4.raise (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn/dist/acorn.js:2825:15)
    at Object.pp.unexpected (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn/dist/acorn.js:689:10)
    at Object.pp$3.parseExprAtom (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn/dist/acorn.js:2270:12)
    at Object.parseExprAtom (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn-dynamic-import/lib/index.js:75:117)
    at Object.parseExprAtom (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn-import-meta/index.js:17:75)
    at Object.pp$3.parseExprSubscripts (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn/dist/acorn.js:2089:21)
    at Object.pp$3.parseMaybeUnary (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn/dist/acorn.js:2066:19)
    at Object.parseMaybeUnary (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn-private-class-elements/index.js:141:29)
    at Object.pp$3.parseExprOps (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn/dist/acorn.js:2010:21)
    at Object.pp$3.parseMaybeConditional (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn/dist/acorn.js:1993:21)
    at Object.pp$3.parseMaybeAssign (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn/dist/acorn.js:1968:21)
    at Object.pp$3.parseExpression (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn/dist/acorn.js:1933:21)
    at Object.pp$1.parseStatement (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn/dist/acorn.js:877:47)
    at Object.parseStatement (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn-dynamic-import/lib/index.js:63:118)
    at Object.parseStatement (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn-import-meta/index.js:39:22)
    at Object.pp$1.parseTopLevel (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn/dist/acorn.js:746:23) {
  pos: 0,
  loc: Position { line: 1, column: 0 },
  raisedAt: 1
}
    at /Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/easygettext/src/extract-cli.js:74:13
    at Array.forEach (<anonymous>)
    at Object.<anonymous> (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/easygettext/src/extract-cli.js:60:7)
    at Module._compile (internal/modules/cjs/loader.js:1147:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1167:10)
    at Module.load (internal/modules/cjs/loader.js:996:32)
    at Function.Module._load (internal/modules/cjs/loader.js:896:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
    at /usr/local/lib/node_modules/npm/node_modules/libnpx/index.js:268:14

If I install version 2.10.0 everything works as expected:

$ npx gettext-extract po.html                        
[easygettext] extracting: 'po.html
msgid ""
msgstr ""
"Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Generated-By: easygettext\n"
"Project-Id-Version: \n"

#: po.html:31
msgid "Broken strings are joined"
msgstr ""

#: po.html:14
msgctxt "According to..."
msgid "Hello World"
msgstr ""

#. My comment...
#: po.html:12
#: po.html:16
#: po.html:18
#: po.html:20
#: po.html:24
#: po.html:26
#: po.html:28
msgid "Hello World"
msgid_plural "Hello worlds"
msgstr[0] ""
msgstr[1] ""

#: po.html:42
msgid "Something …"
msgstr ""

I believe it is caused by the typescript support added in 2.11.0. It always tries to parse JavaScript, if the file is not TypeScript:

Screenshot 2020-06-09 at 17 05 47

po.html

<div translate>Hello World</div>

<div translate translate-context="According to...">Hello World</div>

<div translate translate-comment="My comment...">Hello World</div>

<div translate translate-plural="Hello worlds">Hello World</div>

<div placeholder="{{ 'Hello World' | translate }}"></div>

<div placeholder="{{ scopeVariable || ('Hello World' | translate) }}"></div>

<get-text>Hello World</get-text>

<i18n>Hello World</i18n>

<translate>Hello World</translate>

<!--  The following becomes 'Broken strings are joined'  -->
<span ng-bind="{{ 'Broken '
 + 'strings ' +
 'are ' +
 'joined' |translate}}"></span>

<span ng-bind="'Bed n\'' + ' breakfast' |translate"></span>

<!-- JavaScript expressions are parsed and compiled -->
<span ng-bind="true ? 'Always' : 'Never' |i18n "></span>

<!--  By supplying the  --filterPrefix '::' parameter  -->
<span>{{:: 'Something …' |translate}}</span>

<!--  The default delimiters '{{' and '}}' must be changed to empty strings to handle these examples  -->
<span ng-bind=":: 'Something …' |translate"></span>

<div placeholder="'Hello World' | translate"></div>

Webpack require and translate tag

Sorry to disturb, but npm still has easygettext v1.0.2 which fails on webpack's require in jade and doesn't recognize simple translate tag (PR for that exists but can't be merged).

I'd also liked to know how to extract files recursively. ./app/*.jade extracts only ./app/index.jade and ./app/**/*.jade extracts exactly all first-level files like ./app/aaa/bbb.jade, neither index nor more nested files.

Question: extract `:label="$gettext('MY_STRING')"` possible?

I have following vue component:

<template>
      <special-input
        :label="$gettext('mylabel')"
      />
</template>
<script>
  ...
</script>

Is it possible to extract these strings with easygettext as well?
At the moment I'm using gettext-vue (which is using xgettext-template under the hood).

Not sure if this is or should be scope of this project. But now that it's handling .vue files pretty well already I would like to use it for all cases.

A work-around would be this:

<template>
      <special-input
        :label="mylabel"
      />
</template>
<script>
  ...
  computed: {
    mylabel() {
      return $gettext('MY_LABEL')
    },
  },
  ...
</script>

How to use with Pug for server-side localization?

I am using gettext-extract to extract tokens in Pug templates. However, it seems Pug do not have a default way of doing l10n when it renders templates at the server-side. I resort to use a simple i18n function with Pug's interpolation, so the text that needs to be translated looks like #{i18n && i18n("Please translate this to your language.")}. The trouble is, now I cannot tweak gettext-extract to recognize these tokens.

Can someone with experiences in using this with Pug templates share how they did it?

Extracting from html file is broken

Extracting html files works with [email protected], but not with 2.8.0 or later. It seems like it is fed into acorn which can only deal with js, not html:

$ npx gettext-extract --output /tmp/dictionary.pot /var/tmp/test.html
[easygettext] extracting: '/var/tmp/test.html
[easygettext] could not read: '/var/tmp/test.html
Trace: { SyntaxError: Unexpected token (1:0)
    at Object.pp$4.raise (/home/vbraun/node_modules/acorn/dist/acorn.js:2825:15)
    at Object.pp.unexpected (/home/vbraun/node_modules/acorn/dist/acorn.js:689:10)
    at Object.pp$3.parseExprAtom (/home/vbraun/node_modules/acorn/dist/acorn.js:2270:12)
    at Object.parseExprAtom (/home/vbraun/node_modules/acorn-stage3/node_modules/acorn-dynamic-import/lib/index.js:75:117)
    at Object.parseExprAtom (/home/vbraun/node_modules/acorn-import-meta/index.js:17:75)
    at Object.pp$3.parseExprSubscripts (/home/vbraun/node_modules/acorn/dist/acorn.js:2089:21)
    at Object.pp$3.parseMaybeUnary (/home/vbraun/node_modules/acorn/dist/acorn.js:2066:19)
    at Object.parseMaybeUnary (/home/vbraun/node_modules/acorn-private-class-elements/index.js:112:29)
    at Object.pp$3.parseExprOps (/home/vbraun/node_modules/acorn/dist/acorn.js:2010:21)
    at Object.pp$3.parseMaybeConditional (/home/vbraun/node_modules/acorn/dist/acorn.js:1993:21)
    at Object.pp$3.parseMaybeAssign (/home/vbraun/node_modules/acorn/dist/acorn.js:1968:21)
    at Object.pp$3.parseExpression (/home/vbraun/node_modules/acorn/dist/acorn.js:1933:21)
    at Object.pp$1.parseStatement (/home/vbraun/node_modules/acorn/dist/acorn.js:877:47)
    at Object.parseStatement (/home/vbraun/node_modules/acorn-stage3/node_modules/acorn-dynamic-import/lib/index.js:63:118)
    at Object.parseStatement (/home/vbraun/node_modules/acorn-import-meta/index.js:38:22)
    at Object.pp$1.parseTopLevel (/home/vbraun/node_modules/acorn/dist/acorn.js:746:23) pos: 0, loc: Position { line: 1, column: 0 }, raisedAt: 1 }
    at /home/vbraun/node_modules/easygettext/src/extract-cli.js:85:13
    at Array.forEach (<anonymous>)
    at Object.<anonymous> (/home/vbraun/node_modules/easygettext/src/extract-cli.js:57:7)
    at Module._compile (internal/modules/cjs/loader.js:778:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)
    at Function.Module.runMain (internal/modules/cjs/loader.js:831:12)
    at findNodeScript.then.existing (/usr/lib/node_modules/npm/node_modules/libnpx/index.js:268:14)

My use case is defining vue components in separate files, so the vue file is only <template src="./filename.html"></template>. To extract I'm calling gettext-extract on the html file.

Attributes param also matches tags

Currently the attribute parameter cannot distinguish between tags and attributes. I have tags and and they are both extracted when I provide param --attribute v-translate

recursive path support

for example we got files

- foo
   |-- x.html
   `-- bar/
         |-- y.html

Can we simply use foo/**/*.html

Not extracting strings from $gettext calls in SequenceExpression assignments

When $gettext/$ngettext calls are part of array/object literal declarations the nodes are ignored. It used to trigger an exception 'TypeError: expression.arguments is not iterable' before issue #37 was fixed.

The example I have (will be part of test-fixtures in an upcoming pull request):

export default {
  name: 'greetings-sequence',
  computed: {
    messages_object() {
      return {
        a_string: this.$gettext('Hello there!'),
        an_array: [this.$gettext('Hello there!'), this.$gettext('Hello there!')]
      }
    }
  }
}

How to set Chinese as the native language in gettext

Well I didn't find any configuration to set the non-English language as native language, so after the po file generated, gettext take English as the default, so the msgstr in en.po is filled, but my native file zh-cn.po is empty.

I didn't find gettext repo on github, so open an issue here.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.