Giter Club home page Giter Club logo

fluent-templates's Introduction

Fluent Templates: A High level Fluent API.

Build & Test crates.io Help Wanted Lines Of Code Documentation

fluent-templates lets you to easily integrate Fluent localisation into your Rust application or library. It does this by providing a high level "loader" API that loads fluent strings based on simple language negotiation, and the FluentLoader struct which is a Loader agnostic container type that comes with optional trait implementations for popular templating engines such as handlebars or tera that allow you to be able to use your localisations in your templates with no boilerplate.

Loaders

Currently this crate provides two different kinds of loaders that cover two main use cases.

  • static_loader! — A procedural macro that loads your fluent resources at compile-time into your binary and creates a new StaticLoader static variable that allows you to access the localisations. static_loader! is most useful when you want to localise your application and want to ship your fluent resources with your binary.

  • ArcLoader — A struct that loads your fluent resources at run-time using Arc as its backing storage. ArcLoader is most useful for when you want to be able to change and/or update localisations at run-time, or if you're writing a developer tool that wants to provide fluent localisation in your own application such as a static site generator.

static_loader!

The easiest way to use fluent-templates is to use the static_loader! procedural macro that will create a new StaticLoader static variable.

Basic Example

fluent_templates::static_loader! {
    // Declare our `StaticLoader` named `LOCALES`.
    static LOCALES = {
        // The directory of localisations and fluent resources.
        locales: "./tests/locales",
        // The language to falback on if something is not present.
        fallback_language: "en-US",
        // Optional: A fluent resource that is shared with every locale.
        core_locales: "./tests/locales/core.ftl",
    };
}

Customise Example

You can also modify each FluentBundle on initialisation to be able to change configuration or add resources from Rust.

use fluent_bundle::FluentResource;
use fluent_templates::static_loader;
use once_cell::sync::Lazy;

static_loader! {
    // Declare our `StaticLoader` named `LOCALES`.
    static LOCALES = {
        // The directory of localisations and fluent resources.
        locales: "./tests/locales",
        // The language to falback on if something is not present.
        fallback_language: "en-US",
        // Optional: A fluent resource that is shared with every locale.
        core_locales: "./tests/locales/core.ftl",
        // Optional: A function that is run over each fluent bundle.
        customise: |bundle| {
            // Since this will be called for each locale bundle and
            // `FluentResource`s need to be either `&'static` or behind an
            // `Arc` it's recommended you use lazily initialised
            // static variables.
            static CRATE_VERSION_FTL: Lazy<FluentResource> = Lazy::new(|| {
                let ftl_string = String::from(
                    concat!("-crate-version = {}", env!("CARGO_PKG_VERSION"))
                );

                FluentResource::try_new(ftl_string).unwrap()
            });

            bundle.add_resource(&CRATE_VERSION_FTL);
        }
    };
}

Locales Directory

fluent-templates will collect all subdirectories that match a valid Unicode Language Identifier and bundle all fluent files found in those directories and map those resources to the respective identifier. fluent-templates will recurse through each language directory as needed and will respect any .gitignore or .ignore files present.

Example Layout

locales
├── core.ftl
├── en-US
│   └── main.ftl
├── fr
│   └── main.ftl
├── zh-CN
│   └── main.ftl
└── zh-TW
    └── main.ftl

Looking up fluent resources

You can use the Loader trait to lookup a given fluent resource, and provide any additional arguments as needed with lookup_with_args.

Example

 # In `locales/en-US/main.ftl`
 hello-world = Hello World!
 greeting = Hello { $name }!

 # In `locales/fr/main.ftl`
 hello-world = Bonjour le monde!
 greeting = Bonjour { $name }!

 # In `locales/de/main.ftl`
 hello-world = Hallo Welt!
 greeting = Hallo { $name }!
use std::collections::HashMap;

use unic_langid::{LanguageIdentifier, langid};
use fluent_templates::{Loader, static_loader};

const US_ENGLISH: LanguageIdentifier = langid!("en-US");
const FRENCH: LanguageIdentifier = langid!("fr");
const GERMAN: LanguageIdentifier = langid!("de");

static_loader! {
    static LOCALES = {
        locales: "./tests/locales",
        fallback_language: "en-US",
        // Removes unicode isolating marks around arguments, you typically
        // should only set to false when testing.
        customise: |bundle| bundle.set_use_isolating(false),
    };
}

fn main() {
    assert_eq!("Hello World!", LOCALES.lookup(&US_ENGLISH, "hello-world"));
    assert_eq!("Bonjour le monde!", LOCALES.lookup(&FRENCH, "hello-world"));
    assert_eq!("Hallo Welt!", LOCALES.lookup(&GERMAN, "hello-world"));

    let args = {
        let mut map = HashMap::new();
        map.insert(String::from("name"), "Alice".into());
        map
    };

    assert_eq!("Hello Alice!", LOCALES.lookup_with_args(&US_ENGLISH, "greeting", &args));
    assert_eq!("Bonjour Alice!", LOCALES.lookup_with_args(&FRENCH, "greeting", &args));
    assert_eq!("Hallo Alice!", LOCALES.lookup_with_args(&GERMAN, "greeting", &args));
}

Tera

With the tera feature you can use FluentLoader as a Tera function. It accepts a key parameter pointing to a fluent resource and lang for what language to get that key for. Optionally you can pass extra arguments to the function as arguments to the resource. fluent-templates will automatically convert argument keys from Tera's snake_case to the fluent's preferred kebab-case arguments.

fluent-templates = { version = "*", features = ["tera"] }
use fluent_templates::{FluentLoader, static_loader};

static_loader! {
    static LOCALES = {
        locales: "./tests/locales",
        fallback_language: "en-US",
        // Removes unicode isolating marks around arguments, you typically
        // should only set to false when testing.
        customise: |bundle| bundle.set_use_isolating(false),
    };
}

fn main() {
    let mut tera = tera::Tera::default();
    let ctx = tera::Context::default();
    tera.register_function("fluent", FluentLoader::new(&*LOCALES));
    assert_eq!(
        "Hello World!",
        tera.render_str(r#"{{ fluent(key="hello-world", lang="en-US") }}"#, &ctx).unwrap()
    );
    assert_eq!(
        "Hello Alice!",
        tera.render_str(r#"{{ fluent(key="greeting", lang="en-US", name="Alice") }}"#, &ctx).unwrap()
    );
}

Handlebars

In handlebars, fluent-templates will read the lang field in your handlebars::Context while rendering.

fluent-templates = { version = "*", features = ["handlebars"] }
use fluent_templates::{FluentLoader, static_loader};

static_loader! {
    static LOCALES = {
        locales: "./tests/locales",
        fallback_language: "en-US",
        // Removes unicode isolating marks around arguments, you typically
        // should only set to false when testing.
        customise: |bundle| bundle.set_use_isolating(false),
    };
}

fn main() {
    let mut handlebars = handlebars::Handlebars::new();
    handlebars.register_helper("fluent", Box::new(FluentLoader::new(&*LOCALES)));
    let data = serde_json::json!({"lang": "zh-CN"});
    assert_eq!("Hello World!", handlebars.render_template(r#"{{fluent "hello-world"}}"#, &data).unwrap());
    assert_eq!("Hello Alice!", handlebars.render_template(r#"{{fluent "greeting" name="Alice"}}"#, &data).unwrap());
}

Handlebars helper syntax.

The main helper provided is the {{fluent}} helper. If you have the following Fluent file:

foo-bar = "foo bar"
placeholder = this has a placeholder { $variable }
placeholder2 = this has { $variable1 } { $variable2 }

You can include the strings in your template with

<!-- will render "foo bar" -->
{{fluent "foo-bar"}}
<!-- will render "this has a placeholder baz" -->
{{fluent "placeholder" variable="baz"}}

You may also use the {{fluentparam}} helper to specify variables, especially if you need them to be multiline.

{{#fluent "placeholder2"}}
    {{#fluentparam "variable1"}}
        first line
        second line
    {{/fluentparam}}
    {{#fluentparam "variable2"}}
        first line
        second line
    {{/fluentparam}}
{{/fluent}}

FAQ

Why is there extra characters around the values of arguments?

These are called "Unicode Isolating Marks" that used to allow the text to be bidirectional. You can disable this with FluentBundle::set_isolating_marks being set to false.

static_loader! {
    static LOCALES = {
        locales: "./tests/locales",
        fallback_language: "en-US",
        // Removes unicode isolating marks around arguments.
        customise: |bundle| bundle.set_use_isolating(false),
    };
}

fluent-templates's People

Contributors

aaron1011 avatar alerque avatar benjaminwinger avatar burrbull avatar campeis avatar debug-richard avatar dorezyuk avatar ecton avatar github-actions[bot] avatar goweiwen avatar jamolnng avatar kgv avatar manishearth avatar mondeja avatar mrtact avatar pietroalbini avatar robjtede avatar technic avatar torokati44 avatar valkum avatar xampprocky avatar xdarksome avatar zbraniecki avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

fluent-templates's Issues

Example does not work

Hi something went wrong. rustc 1.52.1

use std::collections::HashMap;

use unic_langid::{LanguageIdentifier, langid};
use fluent_templates::{Loader, static_loader};

const US_ENGLISH: LanguageIdentifier = langid!("en-US");
const FRENCH: LanguageIdentifier = langid!("fr");
const GERMAN: LanguageIdentifier = langid!("de");

static_loader! {
    static LOCALES = {
        locales: "./tests/locales",
        fallback_language: "en-US",
        // Removes unicode isolating marks around arguments, you typically
        // should only set to false when testing.
        customise: |bundle| bundle.set_use_isolating(false),
    };
}

fn main() {
    assert_eq!("Hello World!", LOCALES.lookup(&US_ENGLISH, "hello-world"));
    assert_eq!("Bonjour le monde!", LOCALES.lookup(&FRENCH, "hello-world"));
    assert_eq!("Hallo Welt!", LOCALES.lookup(&GERMAN, "hello-world"));

    let args = {
        let mut map = HashMap::new();
        map.insert(String::from("name"), "Alice".into());
        map
    };

    assert_eq!("Hello Alice!", LOCALES.lookup_with_args(&US_ENGLISH, "greeting", &args));
    assert_eq!("Bonjour Alice!", LOCALES.lookup_with_args(&FRENCH, "greeting", &args));
    assert_eq!("Hallo Alice!", LOCALES.lookup_with_args(&GERMAN, "greeting", &args));
}



thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Fluent { source: FluentError([ParserError { pos: (0, 1), slice: Some((0, 64)), kind: ExpectedCharRange { range: "a-zA-Z" } }]) }', src/main.rs:10:1
stack backtrace:
   0: rust_begin_unwind
             at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/panicking.rs:493:5
   1: core::panicking::panic_fmt
             at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/core/src/panicking.rs:92:14
   2: core::option::expect_none_failed
             at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/core/src/option.rs:1329:5
   3: core::result::Result<T,E>::unwrap
             at /home/f/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/result.rs:1037:23
   4: Test::LOCALES::{{closure}}::RESOURCES::{{closure}}
             at ./src/main.rs:10:1
   5: core::ops::function::FnOnce::call_once
             at /home/f/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
   6: core::ops::function::FnOnce::call_once
             at /home/f/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
   7: once_cell::sync::Lazy<T,F>::force::{{closure}}
             at /home/f/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.7.2/src/lib.rs:1023:28
   8: once_cell::sync::OnceCell<T>::get_or_init::{{closure}}
             at /home/f/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.7.2/src/lib.rs:845:57
   9: once_cell::imp::OnceCell<T>::initialize::{{closure}}
             at /home/f/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.7.2/src/imp_std.rs:95:19
  10: once_cell::imp::initialize_inner
             at /home/f/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.7.2/src/imp_std.rs:171:31
  11: once_cell::imp::OnceCell<T>::initialize
             at /home/f/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.7.2/src/imp_std.rs:93:9
  12: once_cell::sync::OnceCell<T>::get_or_try_init
             at /home/f/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.7.2/src/lib.rs:885:13
  13: once_cell::sync::OnceCell<T>::get_or_init
             at /home/f/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.7.2/src/lib.rs:845:19
  14: once_cell::sync::Lazy<T,F>::force
             at /home/f/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.7.2/src/lib.rs:1022:13
  15: <once_cell::sync::Lazy<T,F> as core::ops::deref::Deref>::deref
             at /home/f/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.7.2/src/lib.rs:1032:13
  16: Test::LOCALES::{{closure}}::BUNDLES::{{closure}}
             at ./src/main.rs:10:1
  17: core::ops::function::FnOnce::call_once
             at /home/f/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
  18: core::ops::function::FnOnce::call_once
             at /home/f/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
  19: once_cell::sync::Lazy<T,F>::force::{{closure}}
             at /home/f/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.7.2/src/lib.rs:1023:28
  20: once_cell::sync::OnceCell<T>::get_or_init::{{closure}}
             at /home/f/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.7.2/src/lib.rs:845:57
  21: once_cell::imp::OnceCell<T>::initialize::{{closure}}
             at /home/f/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.7.2/src/imp_std.rs:95:19
  22: once_cell::imp::initialize_inner
             at /home/f/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.7.2/src/imp_std.rs:171:31
  23: once_cell::imp::OnceCell<T>::initialize
             at /home/f/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.7.2/src/imp_std.rs:93:9
  24: once_cell::sync::OnceCell<T>::get_or_try_init
             at /home/f/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.7.2/src/lib.rs:885:13
  25: once_cell::sync::OnceCell<T>::get_or_init
             at /home/f/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.7.2/src/lib.rs:845:19
  26: once_cell::sync::Lazy<T,F>::force
             at /home/f/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.7.2/src/lib.rs:1022:13
  27: <once_cell::sync::Lazy<T,F> as core::ops::deref::Deref>::deref
             at /home/f/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.7.2/src/lib.rs:1032:13
  28: Test::LOCALES::{{closure}}
             at ./src/main.rs:10:1
  29: core::ops::function::FnOnce::call_once
             at /home/f/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
  30: core::ops::function::FnOnce::call_once
             at /home/f/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
  31: once_cell::sync::Lazy<T,F>::force::{{closure}}
             at /home/f/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.7.2/src/lib.rs:1023:28
  32: once_cell::sync::OnceCell<T>::get_or_init::{{closure}}
             at /home/f/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.7.2/src/lib.rs:845:57
  33: once_cell::imp::OnceCell<T>::initialize::{{closure}}
             at /home/f/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.7.2/src/imp_std.rs:95:19
  34: once_cell::imp::initialize_inner
             at /home/f/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.7.2/src/imp_std.rs:171:31
  35: once_cell::imp::OnceCell<T>::initialize
             at /home/f/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.7.2/src/imp_std.rs:93:9
  36: once_cell::sync::OnceCell<T>::get_or_try_init
             at /home/f/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.7.2/src/lib.rs:885:13
  37: once_cell::sync::OnceCell<T>::get_or_init
             at /home/f/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.7.2/src/lib.rs:845:19
  38: once_cell::sync::Lazy<T,F>::force
             at /home/f/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.7.2/src/lib.rs:1022:13
  39: <once_cell::sync::Lazy<T,F> as core::ops::deref::Deref>::deref
             at /home/f/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.7.2/src/lib.rs:1032:13
  40: Test::main
             at ./src/main.rs:21:32
  41: core::ops::function::FnOnce::call_once
             at /home/f/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

message translated with LOCALES.lookup_complete() contains unexpected extra characters around place-ables

Hello,

i'm trying to write simple wrapper for my app to allow translation to a language changeable during runtime.

When I try to get the translation, the args are filled into the ftl string, and i get the translated result, but it contains extra "wrapper" characters around every place-able.

FTL file contents:

# $tokens (String) - Number of tokens on input.
# $values (String) - Number of values on input.
valid-count-mismatch =
    Number of tokens is not equal to number of values on input!
    Tokens: { $tokens }
    Values: { $values }

Code being executed:

type ArgsMap<'m> = HashMap<String, FluentValue<'m>>;

fn translate(text_id: &str, args: Option<&ArgsMap>) -> String {
    let lang_code = current_lang().to_lang_code();
    if let Some(li) = LANG_IDS.get(&lang_code) {
        println!("LanguageIdentifier:\n\t{:?}", li);
        println!("text_id:\n\t{:?}", text_id);
        println!("args:\n\t{:?}", args);
        let translated = &*LOCALES.lookup_complete(&li, text_id, args);
        println!("translated:\n\t{}", translated);
        println!("translated debug:\n\t{:?}", translated);
        return translated.to_string();
    }
    text_id.to_string()
}

Console output (please note extra characters around two place-ables on last line of printout):

LanguageIdentifier:
        LanguageIdentifier { language: Language(Some("en")), script: None, region: Some(Region("US")), variants: None }
text_id:
        "valid-count-mismatch"
args:
        Some({"values": String("1"), "tokens": String("3")})
translated:
        Number of tokens is not equal to number of values on input!
Tokens: ⁨3⁩
Values: ⁨1⁩
translated debug:
        "Number of tokens is not equal to number of values on input!\nTokens: \u{2068}3\u{2069}\nValues: \u{2068}1\u{2069}"

Windows console shows chars as sort graphical noise (console line translated:).
Github here does not show these above... It causes problem on windows, as GUI dialogs etc.
It is visible in debug dump on next line (translated debug:) - \u{2068} - \u{2069}.

Is this place-able wrapper intentionally done by the LOOKUP methods? If so, can it be disabled?
If not, what could be causing this behavior?...

I do NOT need any type specific extra localization/internationalization features, so limiting myself to string is not a problem...
I'm not sure whether other argument types (thank String shown above) have same issue. I tried to work with generic argument map originally, that could allow numbers etc. (via Into), but failed to pass them around as parameters of methods due to being Rust beginner...

All files (FTL, rust .rs, ...) are UTF-8 encoded, LF line endings.

Thank you for tips/guidelines...

In a production environment, panic! may not be a good choice.

In a production environment, panic! may not be a good choice. Can we directly return the error message as a string, i.e., change the code from

panic!( "Failed to format a message for locale {} and id {}.\nErrors\n{:?}", lang, text_id, errors  ) 

to

Some(format!("Failed to format a message for locale {} and id {}.\nErrors\n{:?}",   lang, text_id, errors))

Handlebars example from the docs appears not to compile

I have created a demo repo which contains the complete implementation of the handlebars demo from the docs.

When I try to build this repo, I get a pretty gnarly type conversion error:

error[E0277]: expected a `std::ops::Fn<(&handlebars::render::Helper<'reg, 'rc>, &'reg handlebars::registry::Registry<'reg>, &'rc handlebars::context::Context, &mut handlebars::render::RenderContext<'reg, 'rc>, &mut dyn handlebars::output::Output)>` closure, found `fluent_templates::loader::FluentLoader<&fluent_templates::loader::static_loader::StaticLoader>`
  --> src/main.rs:15:42
   |
15 |     handlebars.register_helper("fluent", Box::new(FluentLoader::new(&*LOCALES)));
   |                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected an `Fn<(&handlebars::render::Helper<'reg, 'rc>, &'reg handlebars::registry::Registry<'reg>, &'rc handlebars::context::Context, &mut handlebars::render::RenderContext<'reg, 'rc>, &mut dyn handlebars::output::Output)>` closure, found `fluent_templates::loader::FluentLoader<&fluent_templates::loader::static_loader::StaticLoader>`
   |
   = help: the trait `for<'r, 'reg, 'rc, 's, 't0> std::ops::Fn<(&'r handlebars::render::Helper<'reg, 'rc>, &'reg handlebars::registry::Registry<'reg>, &'rc handlebars::context::Context, &'s mut handlebars::render::RenderContext<'reg, 'rc>, &'t0 mut (dyn handlebars::output::Output + 't0))>` is not implemented for `fluent_templates::loader::FluentLoader<&fluent_templates::loader::static_loader::StaticLoader>`
   = note: required because of the requirements on the impl of `handlebars::helpers::HelperDef` for `fluent_templates::loader::FluentLoader<&fluent_templates::loader::static_loader::StaticLoader>`
   = note: required for the cast to the object type `dyn handlebars::helpers::HelperDef + std::marker::Send + std::marker::Sync`

I would have submitted a PR for this, but frankly it's a bit beyond my understanding at the present time.

Note that I tried to build this both with 1.44 and nightly, so I don't think it's a recent feature that the lib is relying on.

Static Loader doesn't notice new files when rebuilding

When I use include_str or similar macros, if I change the referenced file and rebuild, cargo notices the change and rebuilds the source file.

Unfortunately, in my initial testing, I can't seem to get the static_loader! to reload new content without changing some other source file. Is there a way to get the static_loader! macro to notify Cargo that it's depending on those files and to rebuild if they change?

The way I'm testing is I have this snippet:

fluent_templates::static_loader! {
    // Declare our `StaticLoader` named `LOCALES`.
    pub static LOCALES = {
        // The directory of localisations and fluent resources.
        locales: "./shared/src/strings",
        // The language to fallback on if something is not present.
        fallback_language: "en-US",
    };
}

And I load it and render a string out from the referenced LOCALES, and if I run the program once, change the string, cargo run again I don't see the new value. If I cargo clean I do.

Thank you for the great library; it is very easy to use.

`arc-swap` yanked; `fluent_templates 0.5.0` fails to build

Hi, it looks like fluent_templates relies on arc-swap = "^0.4.7", but many recent arc-swap versions have been yanked: https://docs.rs/arc-swap/0.4.7/arc_swap/index.html. This is causing build issues such as:

error: failed to select a version for the requirement `arc-swap = "^0.4.7"`
  candidate versions found which didn't match: 1.1.0, 0.4.1, 0.4.0, ...
  location searched: crates.io index
required by package `fluent-templates v0.5.0`
    ... which is depended on by ...

I think the proper solution is to change the arc-swap dependency to either 0.4.1 or 1.1.0

ArcLoader won't work without a top level directory

I spent a while trying to figure out why I was only getting panics trying to follow examples from the docs. I finally narrowed it down to this bit of code:

let arc = ArcLoader::builder("tests/locales", unic_langid::langid!("en-US"))

The path from this test tests/locales from the test and examples works, but using "./" or any other iteration I can think of to use $CWD to look for resources ends in a panic.

Fallback chain for sibling specificity in languages

So basically going for the proposed solution from #1 this results in the fallback key to be present in the base zh language.

I am not sure if we should change that to have a resolution order like:
zh-CN -> zh-CN, zh-TW, zh, default_fallback

The question then would be: What to do with a specificity higher than 2?
de-DE-1996 -> de-DE-1996, [de-CH-VARIANT], de-DE, de-CH, de, default_fallback?
Not sure how the best approach should be.
If we consider the current state as a linked list, other language regions like CH would then be siblings in a tree.
The question is how many branches of the main path between the requested one and the root should be visited.

fallback <- de <- de-DE <- de-DE-1996
               Î         ^- de-DE-VARIANT2          
               \ de-CH <- de-CH-VARIANT

In which order should they be visited?

And what about multiple requested languages? (possibly with weights), can we ignore them for now? I guess currently one can argue that this is only localization resolution and language negotiation should be done ahead.

Originally posted by @valkum in #35 (comment)

How to determine available translations

The list of available translation is determined from the locales directory (for StaticLoader).
Is there a way to access this list from the program?
Basically when user asks for some language with Accept-Language http header I need to know whether I have that language and provide some fallback otherwise. There is a fallback option in static_loader!, and also fallbacks vec in the struct. I don't understand how is it supposed to work?

Compile time i18n macro

Discussed in #52

Originally posted by patefacio September 5, 2023
For compile-time support is it possible to get a compile error when compiling if the lookup fails? I tried cargo-i18n with i18n-embed and in that setup a mispelling will trigger a compile error. My issue with that setup was I could not figure out how to make it work in wasm. This project works just fine in wasm but the forced breakage at compile time would be great.

assert_eq!("Hello World!", LOCALES.lookup(&US_ENGLISH, "hello-world-ooops"));

Thank you for question! I think this should be possible if implemented as procedural macro. I don't have much time to work on this at the moment, but I'd be happy to review a PR for it. We already have a macro crate, so what would work would be to add a new macro similar to the existing one, but that uses the ArcLoader to load localisations into the macro itself, and then provides a panic message when it's not found using a separate lookup macro.

how to type the returned value for `LOCALES.locales()`?

I need to get all language identifier for the static loader in a function, how can type the return object please?

use unic_langid::{LanguageIdentifier, langid};
use fluent_templates::{Loader, static_loader};

const US_ENGLISH: LanguageIdentifier = langid!("en-US");
const SIMPLE_CHINESE: LanguageIdentifier = langid!("zh-CN");

static_loader! {
    static LOCALES = {
        locales: "./translations",
        fallback_language: "en-US",
        // Optional: A fluent resource that is shared with every locale.
        core_locales: "./translations/core.ftl",
    };
}

pub fn trans(locale: &LanguageIdentifier, message: &str) -> String {
    LOCALES.lookup(locale, message)
}

//Error:  The lifetime bound for this object type cannot be deduced from context; please supply an explicit bound
pub fn locales() -> Box<dyn Iterator<Item = &LanguageIdentifier> + '_> {
    LOCALES.locales()
}

Fallback chain

Hi! Congrats on the release and thank you for the project, it's exciting!

Reading through the docs I noticed that you use fallback_language as the "language to fallback on".

I would recommend sticking a bit closer to the Fluent high-level API model of lazy iterators with fallback chains.
In other words, if a message is missing in user requested locale, the next locale to try shouldn't be hardcoded, but rather follow user requested fallback chain.

That allows for user to specify ["es-CL', "es", "fr", "it", "en"] chain fluidly, rather than ending up with ["es-CL", "en"].

This is important to devalue en as "catch all" which is a very western-centric POV, but it also allows for microlocales - like es-CL may be just used to provide 10% of strings needed to override over generic es.

Feature request: configurable path to lang (in context)

It would be useful if the 'lang' template context path is configurable when registering a fluent helper. If I pass a common context object to every template I use fluent with, I could for example promise the lang data can be found at {{ my_context_object.language }} instead of {{ lang }}:
handlebars.register_helper("fluent", Box::new(FluentLoader::new(&*LOCALES, "my_context_object.language")));

I know this makes code in this lib a bit more complex, but it definitely will make my project using this library cleaner.

Why would someone want this? Well, I already inserted an object containing lots of common data into the context (including language). I just want it to read that object, instead of having to clone the lang data to another field in the context.

It is one line of code less, for every route in my project (nice). But even nicer... currently if I forget to pass this variable to the 'context!' it will just use the fallback language, and I don't even get a compiler warning! If I forget to pass the common object, it's a lot easier to spot :D

Thanks for reading!

lookup attribute

Can't find a way to get attribute message.
Is this in the plans?

Example Code for tera does not compile

-> tera.register_function("fluent", FluentLoader::new(&*LOCALES));
Gives away a long error message

-------------------------------------------_
`

error[E0277]: expected a std::ops::Fn<(&std::collections::HashMap<std::string::String, serde_json::value::Value>,)> closure, found fluent_templates::loader::FluentLoader<&fluent_templates::loader::static_loader::StaticLoader>
--> src/main.rs:72:38
|
72 | tera.register_function("fluent", FluentLoader::new(&*LOCALES));
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected an Fn<(&std::collections::HashMap<std::string::String, serde_json::value::Value>,)> closure, found fluent_templates::loader::FluentLoader<&fluent_templates::loader::static_loader::StaticLoader>
|
= help: the trait for<'r> std::ops::Fn<(&'r std::collections::HashMap<std::string::String, serde_json::value::Value>,)> is not implemented for fluent_templates::loader::FluentLoader<&fluent_templates::loader::static_loader::StaticLoader>
= note: required because of the requirements on the impl of tera::builtins::functions::Function for fluent_templates::loader::FluentLoader<&fluent_templates::loader::static_loader::StaticLoader>

error[E0277]: expected a std::ops::FnOnce<(&std::collections::HashMap<std::string::String, serde_json::value::Value>,)> closure, found fluent_templates::loader::FluentLoader<&fluent_templates::loader::static_loader::StaticLoader>
--> src/main.rs:72:38
|
72 | tera.register_function("fluent", FluentLoader::new(&*LOCALES));
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected an FnOnce<(&std::collections::HashMap<std::string::String, serde_json::value::Value>,)> closure, found fluent_templates::loader::FluentLoader<&fluent_templates::loader::static_loader::StaticLoader>
|
= help: the trait std::ops::FnOnce<(&std::collections::HashMap<std::string::String, serde_json::value::Value>,)> is not implemented for fluent_templates::loader::FluentLoader<&fluent_templates::loader::static_loader::StaticLoader>
= note: required because of the requirements on the impl of tera::builtins::functions::Function for fluent_templates::loader::FluentLoader<&fluent_templates::loader::static_loader::StaticLoader>

`

ArcLoader requires locale directory to exist to even use fallbacks

Using the ArcLoader is really picky and I'm having a hard time setting it up for use (adding Fluent functions to a tera CLI). The shared resources feature works, as do fallback locales. The trouble is neither work if a directory in the locales path does not exist for every possible locale variant that might be used. Even if the message data would otherwise fall back to another locale or the shared resources, an empty directory needs to exist to stuff the bundles.

c.f. #41

0.8 breaking changes are a little bit obscure

I am using fluent-templates in several projects with 400+ keys overall and tried to upgrade to 0.8.0.
The issues are related to the #32 changes which add a Result return to all lookup functions.

I think that there should definitely be a function to query the existence of a key, but the change works against the practical usage.
Or in other words, I have not needed such a return value once.
This is also easily explained, why do I need a default handler for a fallback handler?
I don't want to hardcode an additional default value for every case.

So I wrote a wrapper that works but I realized that it doesn't work for the tera/rocket handler because the static_loader can't be referenced easily. After two hours I gave up because I didn't want to write complicated code that does something that already worked perfectly.

With the new behavior the application panices every time a key can't be found.
This is great if you write a new template and use placeholders.

I would like to change this again but the question is should the old behavior be restored and the result functions added under a new name or vice versa?

Passing lang argument to each call in tera template is not practical

Hi,

I need to put same language argument in all function invocation in the Tera template. This looks like unnecessary boilerplate, because the language argument is always the same. Ideally the language should be a global variable of FluentLoader (or its "successor"). Then the language can be set before call to render function of tera template.

Unfortunately I don't think that it is possible to access context of tera template from the tera function, so we need to have another mutable state on the side.

Have Loader::lookup_complete return Option<String>

Having Loader::lookup_complete and subsequently lookup_with_args and lookup will let the user choose what to do in case the fluent template is not found with the given language identifier or the fallback identifier

Allow arguments to be &'static str

I suggest сhange signature lookup_complete(... args: Option<&HashMap<String, FluentValue>>) -> String
to something like lookup_complete<T: AsRef<str> + 'static>(... args: Option<&HashMap<T, FluentValue>>) -> String.
This will remove unnecessary allocations from user code.

playground

Better error handling semanitcs

I will say #33 leaves a little ugly way of handling any errors given off by FluentLoader::call as this is my current solution

let fl = FluentLoader::new(loader).with_default_lang(langid!("en-US"));
tera.register_function(
  "fluent",
  move |args: &HashMap<String, serde_json::Value>| match fl.call(&args) {
    Ok(r) => Ok(r),
    Err(e) => match args.get("key").and_then(serde_json::Value::as_str) {
      Some(key) => Ok(key.into()),
      None => Err(e),
    },
  },
);

a solution to this might be to have something like

pub fn call_wrapper(...) -> Result<Json, loader::tera::Error> {
}

pub fn call(...) -> Result<Json, tera::Error> {
  call_wrapper(...)
}

which would change

let fl = FluentLoader::new(loader).with_default_lang(langid!("en-US"));
tera.register_function(
  "fluent",
  move |args: &HashMap<String, serde_json::Value>| match fl.call_wrapper(&args) {
    Ok(s) => Ok(s),
    Err(e) {
      match e {
        NoLangArgument => ...,
        LangArgumentInvalid => ...,
        NoFluentArgument => ...,
        JsonToFluentFail => ...,
        /* new */
        UnknownKey(key) => ...,
      }
    }
  },
);

thoughts?

Some confusion for me

I am a beginner in Rust and I want to know how to use a specified FTL file.
For example, en-US has main.ftl and other.ftl, but I only want to read the content of other.ftl.

Compile error on mac regarding sharing Cell between threads

Hi, if I pull the latest version of the app (fluent-templates = "0.5.12"), I get a compile error:

/Users/wys/.cargo/bin/cargo test --color=always --no-run --package dialog --lib i18n::test
   Compiling fluent v0.12.0
   Compiling fluent-template-macros v0.5.9
error[E0277]: `std::cell::Cell<bool>` cannot be shared between threads safely
   --> /Users/wys/.cargo/registry/src/github.com-1ecc6299db9ec823/fluent-template-macros-0.5.9/src/lib.rs:125:9
    |
125 | /         Box::new(|result| {
126 | |             let tx = tx.clone();
127 | |             if let Ok(entry) = result {
128 | |                 if entry
...   |
138 | |             ignore::WalkState::Continue
139 | |         })
    | |__________^ `std::cell::Cell<bool>` cannot be shared between threads safely
    |
    = help: within `flume::Sender<std::string::String>`, the trait `std::marker::Sync` is not implemented for `std::cell::Cell<bool>`
    = note: required because it appears within the type `flume::Sender<std::string::String>`
    = note: required because of the requirements on the impl of `std::marker::Send` for `&flume::Sender<std::string::String>`
    = note: required because it appears within the type `[closure@/Users/wys/.cargo/registry/src/github.com-1ecc6299db9ec823/fluent-template-macros-0.5.9/src/lib.rs:125:18: 139:10 tx:&flume::Sender<std::string::String>]`
    = note: required for the cast to the object type `dyn std::ops::FnMut(std::result::Result<ignore::walk::DirEntry, ignore::Error>) -> ignore::walk::WalkState + std::marker::Send`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0277`.
error: could not compile `fluent-template-macros`.

To learn more, run the command again with --verbose.

Process finished with exit code 101

This seems to be caused by the most recent commit: f47d95e

OS: Mac 10.15.6
rustc: 1.45.0

Pass through bundle errors? Don't panic!

Hi XAMPPRocky,

I am using fluent-templates with the ArcLoader and Handlebars and it is working really well so far - thank you!

I noticed that when I call build() on the ArcLoaderBuilder that if I make an error it will panic due to the calls to expect() in build(). For my project that means I have to use std::panic::set_hook() to trap those errors which is not ideal.

Because ArcLoaderBuilder.build already returns a Result with the Error trait I think it would make more sense to just pass those errors back to the caller untouched - I am a beginner with Rust so not totally sure but I think it is more idiomatic to return them (?).

Happy to take a look at it further and submit a PR if you think this approach is worthwhile.

Loader cannot be made into an object

Hi,

I am trying to update this dependency from 0.5.16 to 0.8 and I reference the Loader trait in a struct field:

/// Lookup a language string in the underlying loader.
pub struct FluentHelper {
    loader: Box<dyn Loader + Send + Sync>,
    /// Escape messages, default is `true`.
    pub escape: bool,
}

Now I am unable to use the trait boxed due to this compiler error:

error[E0038]: the trait `Loader` cannot be made into an object
  --> src/lib.rs:95:17
   |
95 |     loader: Box<dyn Loader + Send + Sync>,
   |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `Loader` cannot be made into an object
   |
note: for a trait to be "object safe" it needs to allow building a vtable to allow the call to be resolvable dynamically; for more information visit <https://doc.rust-lang.org/reference/items/traits.html#object-safety>
  --> /Users/muji/.cargo/registry/src/github.com-1ecc6299db9ec823/fluent-templates-0.8.0/src/loader.rs:35:8
   |
35 |     fn lookup_with_args<T: AsRef<str>>(
   |        ^^^^^^^^^^^^^^^^ the trait cannot be made into an object because method `lookup_with_args` has generic type parameters
...
45 |     fn lookup_complete<T: AsRef<str>>(
   |        ^^^^^^^^^^^^^^^ the trait cannot be made into an object because method `lookup_complete` has generic type parameters

I think it may be ok for Loader to just accept &str, I wonder what you think?

Seems like impl for tera::Function is not brought into the scope

I try to use the loader with

rocket = { version = "0.5.0-rc.1" }
rocket_dyn_templates = { version = "0.1.0-rc.2", features = ["tera"] }
static_loader! {
  static LOCALES = {
    locales: "./locales",
    fallback_language: "en",
  };
}
...
    .attach(Template::custom(|engines| {
      engines.tera.register_function("fluent", FluentLoader::new(&*LOCALES));
    }))

and get:

error[E0277]: expected a `Fn<(&HashMap<std::string::String, rocket_dyn_templates::tera::Value>,)>` closure, found `FluentLoader<&StaticLoader>`
   --> src/main.rs:43:48
    |
43  |       engines.tera.register_function("fluent", FluentLoader::new(&*LOCALES));
    |                    -----------------           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected an `Fn<(&HashMap<std::string::String, rocket_dyn_templates::tera::Value>,)>` closure, found `FluentLoader<&StaticLoader>`
    |                    |
    |                    required by a bound introduced by this call
    |
    = help: the trait `for<'r> Fn<(&'r HashMap<std::string::String, rocket_dyn_templates::tera::Value>,)>` is not implemented for `FluentLoader<&StaticLoader>`
    = note: required because of the requirements on the impl of `rocket_dyn_templates::tera::Function` for `FluentLoader<&StaticLoader>`
note: required by a bound in `Tera::register_function`
   --> /home/bodqhrohro/.cargo/registry/src/github.com-1ecc6299db9ec823/tera-1.17.0/src/tera.rs:563:33
    |
563 |     pub fn register_function<F: Function + 'static>(&mut self, name: &str, function: F) {
    |                                 ^^^^^^^^ required by this bound in `Tera::register_function`

tera.rs does not have anything public, is it even imported? Or maybe there is something wrong with the signature?

Static Loader causes doc tests to fail in library crates in a workspace

It appears that static_loader! is generating doc tests, and those doc tests fail.

  • Create new project
  • Add fluent-templates dependency
  • Add this to main.rs:
fluent_templates::static_loader! {
    // Declare our `StaticLoader` named `LOCALES`.
    static LOCALES = {
        // The directory of localisations and fluent resources.
        locales: "./locales",
        // The language to falback on if something is not present.
        fallback_language: "en-US",
    };
}
  • Create a folder named locales
  • Run cargo test

This error is printed:

   Compiling fluent-templates-test v0.1.0 (C:\Users\jon\repos\fluent-templates-test)
error: proc macro panicked
 --> src\main.rs:1:1
  |
1 | / fluent_templates::static_loader! {
2 | |     // Declare our `StaticLoader` named `LOCALES`.
3 | |     static LOCALES = {
4 | |         // The directory of localisations and fluent resources.
... |
8 | |     };
9 | | }
  | |_^
  |
  = help: message: called `Result::unwrap()` on an `Err` value: Os { code: 3, kind: NotFound, message: "The system cannot find the path specified." }

error: aborting due to previous error

error: could not compile `fluent-templates-test`.

Add support for yarte

Yarte claims to be the fastest template engine so it would be nice to add support for it too

Does not work with handlesbar 4.0, only handlesbar 3.0

They must have change the trait of their helpers between 4.0 and 3.0. Here is the error :

help: the trait `for<'r, 'reg, 'rc, 's, 't0> Fn<(&'r Helper<'reg, 'rc>, &'reg Handlebars<'reg>, &'rc handlebars::Context, &'s mut RenderContext<'reg, 'rc>, &'t0 mut (dyn handlebars::Output + 't0))>` is not implemented for `FluentLoader<&StaticLoader>`
   = note: required because of the requirements on the impl of `HelperDef` for `FluentLoader<&StaticLoader>`

Compilation hanging on 1.59 stable

I have confirmed on two separate windows machines that the fluent-template-macros: v0.6.0 crate does not appear to finish compilation.

Digging into it further.

Question: Is there a way to pass through a variable to be resolved by Handlebars?

It seems from looking at the Project Fluent syntax guide, placeables should be what I want. I would expect that the fluent helper would allow placeables in the message to get resolved from the data passed to the Handlebars template renderer. However, based on the example from the docs (and my extensive experience in failing to make it work 😁) it looks like under the hood, the helperdef is likely calling lookup_with_args, passing the args that are baked into the helper call, and all variable resolution happens at the Fluent level.

Is this functionality that exists in the library today, and I'm just not grokking how to make it work, or would this be something I have to add?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.